Can you produce a quantized 2.4bpw model of this model?
#1
by xldistance - opened
xldistance changed discussion title from Can you produce a quantized 2.25bpw model of this model? to Can you produce a quantized 2.4bpw model of this model?
Sure thing, was on vacation before so I wasn't able to, but I'm starting the process now; will keep you posted
@xldistance uploading 2.4bpw here: https://huggingface.co/async0x42/Rombos-LLM-V2.5-Qwen-72b-exl2_2.4bpw
async0x42 changed discussion status to closed
@xldistance uploading 2.4bpw here: https://huggingface.co/async0x42/Rombos-LLM-V2.5-Qwen-72b-exl2_2.4bpw
Thank you so much.