need Q4_K_S
hey, first im really appreciate this! but due to my limited vram, so could you provide a q4_k_s version (non-gguf) please? thanks!
would be better and awesome if you also provide for other model variants (2b, 4b, 27b)!
sorry i mean in 4-bit quantization, this would be great!
Hi, thank you very much for your support and kind words! Other model variants (2B, 4B, 27B) have already been released on the model homepage, and they are available with different quantization options. Currently, GGUF format is supported, which already includes Q4_K_S quantization.
Are you referring to a 4-bit quantized version in MLX format? Thanks!
hey thanks for the reply. yeah thats right, that is what im requesting! maybe with bnb/awq format too (for CUDA)?
i was trying to finetune again using this model (unsloth) with my own dataset (luau coding task) where it also gathered from claude opus 4.6 thinking model.
hey i seen you adding it, thanks for adding the quantized variants!