No BF16 weights? if so please add IQ4_XS quant
#2
by mpasila - opened
Like the title says. If you just added BF16 weights a certain someone could just do the quants for you and also give me IQ4_XS since that is the most optimal size for a 8GB GPU.
Yeah, good call... I’ll look into uploading the BF16 weights so it can be quanted properly. Makes sense if people want to run it at IQ4_XS on smaller cards. I’ll leave the quanting to the experts. Appreciate the heads-up.