mistralai

/

Devstral-2-123B-Instruct-2512

like 318

Follow

Mistral AI_ 16.8k

Output totally mess

using 4 H200 141GB
both docker and vllm cli

CUDA_VISIBLE_DEVICES=4,5,6,7 vllm serve /mnt/extend/models/llm/mistralai/Devstral-2-123B-Instruct-2512 --tool-call-parser mistral --enable-auto-tool-choice --tensor-parallel-size 2 --served-model-name devstral-2-123b-instruct

docker run -d --name vllm-devstral   --gpus '"device=4,5,6,7"'   -p 8000:8000   -v /mnt/extend/models/llm/mistralai/Devstral-2-123B-Instruct-2512:/model   mistralllm/vllm_devstral:latest   --model /model   --tool-call-parser mistral   --enable-auto-tool-choice   --tensor-parallel-size 4   --served-model-name devstral-2-123b-instruct