Can you create an IQ2_M quantization?
#1
by xldistance - opened
This model is too large, my graphics card only has 48GB of VRAM
Indiscriminate IQ2 quantization (e.g Unsloth) quality in general is extremely poor on the 122B which is why we chose to offer the single highest quality lowest possible PRISM dynamic quant. Reach out on kofi we’ll work on the 35b model for smaller hardware