on RTX 3090

by faheemraza1 - opened 13 days ago

hi, can this model be served through vLLM on a RTX 3090?

yes, working fine with vllm v0.19.1

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment