Request: oQ4 (MLX) quantization for Apple Silicon?

#1
by 0xtimi2233 - opened

Hi,

Great model! Is anyone planning to convert this model to the oQ4 (MLX) format using omlx?
(Ref: https://github.com/jundot/omlx/blob/main/docs/oQ_Quantization.md)

This format would be perfect for running it locally on Mac.

Thanks!

Sign up or log in to comment