Do you plan on making quantized GGUF for normistral?
#3
by rsolva - opened
We would really like to play with normistral locally via ollama. Having quantized GGUF models would make this model much more accessible for Ola and Kari Nordmann :)
We just uploaded Q3_K_M, Q4_K_M, Q5_K_M, Q6_K_M, and Q8_0 quantized GGUF versions of the model. Hope this makes it more accessible!
davda54 changed discussion status to closed