Spaces:
Running
Running
Vaibhav Srivastav
commited on
Add steps to install from source for llama.cpp (#1396)
Browse files* Add steps to install from source for llama.cpp
* Formatting.
README.md
CHANGED
@@ -37,9 +37,22 @@ You can quickly start a locally running chat-ui & LLM text-generation server tha
|
|
37 |
|
38 |
**Step 1 (Start llama.cpp server):**
|
39 |
|
|
|
|
|
40 |
```bash
|
41 |
# install llama.cpp
|
42 |
brew install llama.cpp
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
43 |
# start llama.cpp server (using hf.co/microsoft/Phi-3-mini-4k-instruct-gguf as an example)
|
44 |
llama-server --hf-repo microsoft/Phi-3-mini-4k-instruct-gguf --hf-file Phi-3-mini-4k-instruct-q4.gguf -c 4096
|
45 |
```
|
|
|
37 |
|
38 |
**Step 1 (Start llama.cpp server):**
|
39 |
|
40 |
+
Install llama.cpp w/ brew (for Mac):
|
41 |
+
|
42 |
```bash
|
43 |
# install llama.cpp
|
44 |
brew install llama.cpp
|
45 |
+
```
|
46 |
+
|
47 |
+
or [build directly from the source](https://github.com/ggerganov/llama.cpp/blob/master/docs/build.md) for your target device:
|
48 |
+
|
49 |
+
```
|
50 |
+
git clone https://github.com/ggerganov/llama.cpp && cd llama.cpp && make
|
51 |
+
```
|
52 |
+
|
53 |
+
Next, start the server with the [LLM of your choice](https://huggingface.co/models?library=gguf):
|
54 |
+
|
55 |
+
```bash
|
56 |
# start llama.cpp server (using hf.co/microsoft/Phi-3-mini-4k-instruct-gguf as an example)
|
57 |
llama-server --hf-repo microsoft/Phi-3-mini-4k-instruct-gguf --hf-file Phi-3-mini-4k-instruct-q4.gguf -c 4096
|
58 |
```
|