Non GGUF quantizations of Voxtral
#1
by quasoft2 - opened
There are almost no quantizations of Voxtral models. Is it possible to convert any of your GGUF quants for this model to format supported by transformers?
For example, I really like the Q6_K_L quantization, but have no idea how to make a similar one or convert it for use with transformers.