Int8 or int4 version possible?
#11
by timonvanhasselt - opened
Hi, thanks for your work! The model is pretty big, are there maybe plans for quantized versions, int8 or int4? Thanks in advance!
Hi, thanks for your work! The model is pretty big, are there maybe plans for quantized versions, int8 or int4? Thanks in advance!