unsloth
/

orpheus-3b-0.1-ft-bnb-4bit

text-generation

text-generation-inference

4-bit precision

Model card Files Files and versions

vLLM compatibility and concurrency

#1

by chatboo - opened May 15, 2025

Can this be run with vLLM / Orpheus speech and concurrent requests? Cheers.

Unsloth AI org May 16, 2025

Unsure unfortunately. Uou might need to ask the vLLM team or test it 🙏

•

edited May 16, 2025

Thanks I did a GPTQ export any way

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment