TQ1_0 Quant?
#2
by QuietImpostor - opened
I'm not quite sure how this would work on the Qwen3 architecture, but if possible it could mean people with 24GBs VRAM and 64GBs RAM could run it locally! Admittedly with quite heavy accuracy degradation and very slowly. I'd seriously love to see it if it is at all possible.
I'm not quite sure how this would work on the Qwen3 architecture, but if possible it could mean people with 24GBs VRAM and 64GBs RAM could run it locally! Admittedly with quite heavy accuracy degradation and very slowly. I'd seriously love to see it if it is at all possible.
For now maybe not but we'll see! :)
I see, well thanks for responding anyways!
QuietImpostor changed discussion status to closed