marc
/
BrainDock
Template
0
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

compose-vLLM.yaml aktualisiert

This commit is contained in:
Marc Blatter 2026-07-10 18:18:27 +00:00
parent b37585ec18
commit 6fee84f4a0
1 changed files with 1 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
compose-vLLM.yaml
View File

@ -29,7 +29,7 @@ services:
--model Qwen/Qwen2.5-14B-Instruct-AWQ --model Qwen/Qwen2.5-14B-Instruct-AWQ
--host 0.0.0.0 --host 0.0.0.0
--port 8000 --port 8000
--gpu-memory-utilization 0.65 --gpu-memory-utilization 0.9
--max-model-len 16384 --max-model-len 16384
--enable-auto-tool-choice --enable-auto-tool-choice
--tool-call-parser hermes --tool-call-parser hermes