Can we get a 9B-FP8 version next
#5
by kq - opened
Tested Qwen3.5-27B-FP8. It’s flawless—zero loss vs. FP16. Can we get a 9B-FP8 version next?
Hey @littlebird13 can we please get an official FP8 quant? These small models greatly benefit from a good quant.
since fp8 does not require calib data, everyone can make one.
Any update on that?
@Apatsi-dox I found one quant that looks promissing, try: https://huggingface.co/lovedheart/Qwen3.5-9B-FP8, I'll try this one in the upcoming days
Any updated on this?
https://huggingface.co/lovedheart/Qwen3.5-9B-FP8 -> this doesn't seem to be very efficient.