Smaller quant
#1
by Sliderpro93 - opened
Hi! Can you please make a smaller quant to fit into 64gb ram? Something like ~Q3_K_XL?
Uploading something that's hopefully around the size you're after. Should be there in an hour or two.
I assume there should also be the standard size mradermacher quants appearing in a day or two as well.
Edit: Quant came out at 2.5gb. Will take a look in the morning and figure it out.
mradermacher quants came out, yay
btw any plans for qwen35-122b-new Nemotron 120b? Asking for a friend :D
Need to see how support pans out for those models first. Qwen3.5 122b I'll probably give a go at some point if support matures.