Check your tensors
#2
by nightmedia - opened
My quants from April 8 work
quant arc arc/e boolq hswag obkqa piqa wino
mxfp8 0.518,0.709,0.755,0.657,0.418,0.759,0.626
Quant created yesterday, Apr 13:
mxfp8 0.508,0.707,0.756,0.658,0.424,0.760,0.624
I re-quanted yesterday and now they're missing the tensor:
language_model.model.per_layer_model_projection.scales
Check your model.safetensors.index.json if you have this layer.
Avoid mxfp4/mxfp8 quants made recently, until the matter is resolved.