Supported Quantizations
#1
by Naphula - opened
I tested all the quantizations I can make on my PC. Only TQ seems broken, the rest are fully supported.
Regular Quants
- f16
- Q8_K_XL
- Q8_0
- Q6_K
- Q5_1
- Q5_0
- Q5_K_M
- Q5_K_S
- Q4_1
- Q4_0
- Q4_K_M
- Q4_K_S
- Q3_K_L
- Q3_K_M
- Q3_K_S
- Q2_K
IMatrix Quants
- IQ6_K
- IQ4_NL
- IQ4_XS
- IQ3_M
- IQ3_S
- IQ3_XS
- IQ3_XXS
- Q2_K_S
- IQ2_M
- IQ2_S
- IQ2_XS
- IQ2_XXS
- IQ1_M
- IQ1_S
Ternary Quants
- TQ2_0
- TQ1_0