Spaces:

yusufs
/

vllm-inference

Paused

yusufs commited on Jan 28

Commit

c0cde8e

1 Parent(s): 8c5a84b

feat(runner.sh): only enable prefix caching and disable log request

Files changed (1) hide show

runner.sh CHANGED Viewed

@@ -53,5 +53,5 @@ python -u /app/openai_compatible_api_server.py \
     --dtype float16 \
     --enforce-eager \
     --gpu-memory-utilization 0.9 \
-    --enable-chunked-prefill \
-    --enable-prefix-caching

     --dtype float16 \
     --enforce-eager \
     --gpu-memory-utilization 0.9 \
+    --enable-prefix-caching \
+    --disable-log-requests