marc
/
BrainDock
Template
0
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

compose-vLLM.yaml aktualisiert

This commit is contained in:
Marc Blatter 2026-07-10 18:35:06 +00:00
parent 21984763cf
commit 038f30bd6c
1 changed files with 2 additions and 2 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

4
compose-vLLM.yaml
View File

@ -29,7 +29,7 @@ services:
--model Qwen/Qwen2.5-7B-Instruct-AWQ --model Qwen/Qwen2.5-7B-Instruct-AWQ
--host 0.0.0.0 --host 0.0.0.0
--port 8000 --port 8000
--gpu-memory-utilization 0.75 --gpu-memory-utilization 0.85
--max-model-len 16384 --max-model-len 65536
--enable-auto-tool-choice --enable-auto-tool-choice
--tool-call-parser hermes --tool-call-parser hermes