marc
/
BrainDock
Template
0
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

compose-vLLM.yaml aktualisiert 

gpu-memory-utilization
This commit is contained in:
Marc Blatter 2026-07-10 17:59:23 +00:00
parent 9254499bc6
commit 4edd0f7a2e
1 changed files with 1 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
compose-vLLM.yaml
View File

@ -28,7 +28,7 @@ services:
--model Qwen/Qwen2.5-14B-Instruct-AWQ --model Qwen/Qwen2.5-14B-Instruct-AWQ
--host 0.0.0.0 --host 0.0.0.0
--port 8000 --port 8000
--gpu-memory-utilization 0.90 --gpu-memory-utilization 0.65
--max-model-len 16384 --max-model-len 16384
--enable-auto-tool-choice --enable-auto-tool-choice
--tool-call-parser hermes --tool-call-parser hermes