Size, mxfp4
#2
by dagb - opened
ggerganovs MXFP4 gguf is 63.3 GB. So, "GGUF'ified", but otherwise with the original weights if I understand it correctly.
Is it possible to derestrict and maintain the MXFP4 weight format?
ggerganovs MXFP4 gguf is 63.3 GB. So, "GGUF'ified", but otherwise with the original weights if I understand it correctly.
Is it possible to derestrict and maintain the MXFP4 weight format?
Maybe. This is a quantization of https://huggingface.co/ArliAI/gpt-oss-120b-Derestricted, which was only released with bf16 weights.
One possibility might be to use a lora adapter, which could be applied to the MXFP4 model, but unfortunately, @ArliAI did not released the lora adapter (not sure if one was used at all, could have been a full finetune)