Can vllm or sglang run the weights ?

by nwzjk - opened 13 days ago

I'm working with 4090

baa.ai org 13 days ago

The advantage of MLX is that it supports mixed precision, so you would need to look for an engine that supports that on your platform.

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment