Spaces:
Running
Running
File size: 1,392 Bytes
1cad725 46cfdd4 dff6689 46cfdd4 dff6689 46cfdd4 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 |
---
title: README
emoji: 😻
colorFrom: purple
colorTo: yellow
sdk: static
pinned: false
---
Run Open source LLMs and create OpenAI-compatible API services for the Llama2 series of LLMs locally With LlamaEdge!
## Give it a try
Run a single command in your command line terminal.
```
bash <(curl -sSfL 'https://raw.githubusercontent.com/LlamaEdge/LlamaEdge/main/run-llm.sh') --interactive
```
Follow the on-screen instructions to install the WasmEdge Runtime and download your favorite open-source LLM. Then, choose whether you want to chat with the model via the CLI or via a web UI.
[See it in action](https://youtu.be/Hqu-PBqkzDk) | [GitHub](https://github.com/LlamaEdge/LlamaEdge) | [Docs](https://www.secondstate.io/articles/run-llm-sh/)
## Why?
[LlamaEdge](https://github.com/second-state/LlamaEdge), powered by Rust and WasmEdge, provides a strong alternative to Python in AI inference.
* Lightweight. The total runtime size is 30MB.
* Fast. Full native speed on GPUs.
* **Portable. Single cross-platform binary on different CPUs, GPUs, and OSes.**
* Secure. Sandboxed and isolated execution on untrusted devices.
* Container-ready. Supported in Docker, containerd, Podman, and Kubernetes.
## Learn more
Please visit the [LlamaEdge](https://github.com/second-state/LlamaEdge) project to learn more.
|