no quants working
#12
by audioedge - opened
tried a few quants, just get gibberish, anyone have a working quant?
check out this setup from Sudo su:
"i pointed hermes agent at nvidia's nemotron cascade 2 30B-A3B on a single RTX 3090 24GB. IQ4_XS quant by bartowski, 187 tok/s, 625K context. had it discover its own hardware, create an identity file, then build a full GPU marketplace UI from a single prompt."