how to run 4bit model on vLLM?

by selmee - opened Feb 26

Feb 26

I'd appreciate it if you could provide a guideline to run Qwen3.5-27B 4bit model using vLLM

Feb 26

able to run using llama.cpp, impressive result. thanks. looking forward to vLLM support

selmee changed discussion status to closed Feb 26

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment