Vaibhav Srivastav commited on
Commit
ac1e280
·
unverified ·
1 Parent(s): 06feee8

Add steps to install from source for llama.cpp (#1396)

Browse files

* Add steps to install from source for llama.cpp

* Formatting.

Files changed (1) hide show
  1. README.md +13 -0
README.md CHANGED
@@ -37,9 +37,22 @@ You can quickly start a locally running chat-ui & LLM text-generation server tha
37
 
38
  **Step 1 (Start llama.cpp server):**
39
 
 
 
40
  ```bash
41
  # install llama.cpp
42
  brew install llama.cpp
 
 
 
 
 
 
 
 
 
 
 
43
  # start llama.cpp server (using hf.co/microsoft/Phi-3-mini-4k-instruct-gguf as an example)
44
  llama-server --hf-repo microsoft/Phi-3-mini-4k-instruct-gguf --hf-file Phi-3-mini-4k-instruct-q4.gguf -c 4096
45
  ```
 
37
 
38
  **Step 1 (Start llama.cpp server):**
39
 
40
+ Install llama.cpp w/ brew (for Mac):
41
+
42
  ```bash
43
  # install llama.cpp
44
  brew install llama.cpp
45
+ ```
46
+
47
+ or [build directly from the source](https://github.com/ggerganov/llama.cpp/blob/master/docs/build.md) for your target device:
48
+
49
+ ```
50
+ git clone https://github.com/ggerganov/llama.cpp && cd llama.cpp && make
51
+ ```
52
+
53
+ Next, start the server with the [LLM of your choice](https://huggingface.co/models?library=gguf):
54
+
55
+ ```bash
56
  # start llama.cpp server (using hf.co/microsoft/Phi-3-mini-4k-instruct-gguf as an example)
57
  llama-server --hf-repo microsoft/Phi-3-mini-4k-instruct-gguf --hf-file Phi-3-mini-4k-instruct-q4.gguf -c 4096
58
  ```