ENOSYS/Qwen3-Embedding-0.6B-250-v1-GGUF

Experimental global target bits‑per‑weight quantization of unsloth/Qwen3-Embedding-0.6B

• Using non-standard (forked) LLaMA C++ branch for quantization.
• Using a CLI tool to build KLD evaluation and imatrix calibration datasets for GGUF models, sourced from eaddario/imatrix-calibration.
• Using dataset sources: tools, text_en, text_ru.
• Using dataset chunks: 250.
• Tensors quantinization F16 instead of BF16, Nvidia Pascal architecture friendly like P100.
• Small set of patches added.

Many thanks to Ed Addario for an impressive job.

Quantization comparison

BPW	PPL correlation	PPL mean ratio	ΔPPL	Mean KLD	Maximum KLD	99.9% KLD	Mean Δp	RMS Δp
5.00	97.51%	1.229589 ± 0.004913	123.695152 ± 3.455547	0.254131 ± 0.001247	13.281865	3.295894	-0.945 ± 0.030 %	7.861 ± 0.073 %
5.25	97.93%	1.198584 ± 0.004387	106.990729 ± 3.089464	0.201051 ± 0.001020	16.191372	2.680833	-0.726 ± 0.027 %	7.070 ± 0.070 %
5.30	98.05%	1.199072 ± 0.004269	107.253986 ± 3.067627	0.181907 ± 0.000926	11.642469	2.371547	-0.680 ± 0.026 %	6.811 ± 0.067 %
5.50	98.42%	1.143577 ± 0.003665	77.354913 ± 2.488164	0.132384 ± 0.000693	9.056372	1.854777	-0.483 ± 0.022 %	5.860 ± 0.061 %
5.75	98.67%	1.107662 ± 0.003264	58.005137 ± 2.117964	0.097616 ± 0.000599	11.806184	1.765460	-0.368 ± 0.019 %	5.023 ± 0.062 %
5.80	98.72%	1.117510 ± 0.003238	63.310457 ± 2.168267	0.092513 ± 0.000578	10.540298	1.608747	-0.386 ± 0.019 %	4.906 ± 0.059 %
6.00	98.71%	1.125959 ± 0.003273	67.862792 ± 2.234984	0.092148 ± 0.000585	12.128985	1.559407	-0.435 ± 0.019 %	4.931 ± 0.061 %
6.25	98.89%	1.082024 ± 0.002912	44.191950 ± 1.831029	0.067634 ± 0.000431	7.849868	1.006397	-0.264 ± 0.016 %	4.210 ± 0.055 %
6.30	98.94%	1.089430 ± 0.002886	48.181944 ± 1.865568	0.062879 ± 0.000377	7.467944	0.950639	-0.262 ± 0.016 %	4.112 ± 0.055 %
6.50	99.07%	1.110195 ± 0.002773	59.369600 ± 1.962020	0.046909 ± 0.000273	5.759321	0.702734	-0.293 ± 0.013 %	3.510 ± 0.044 %
6.75	99.22%	1.071155 ± 0.002479	38.336145 ± 1.614146	0.027021 ± 0.000217	6.292867	0.423712	-0.069 ± 0.011 %	2.756 ± 0.051 %
6.80	99.23%	1.079566 ± 0.002483	42.867759 ± 1.667153	0.026098 ± 0.000205	6.737493	0.397282	-0.098 ± 0.010 %	2.714 ± 0.051 %
7.00	99.24%	1.083987 ± 0.002490	45.249386 ± 1.706124	0.023685 ± 0.000190	6.608492	0.377540	-0.084 ± 0.010 %	2.559 ± 0.049 %
7.25	99.27%	1.088304 ± 0.002447	47.575202 ± 1.717173	0.019738 ± 0.000130	2.865925	0.332578	-0.090 ± 0.009 %	2.290 ± 0.037 %
7.30	99.28%	1.084746 ± 0.002430	45.658594 ± 1.686292	0.019120 ± 0.000148	4.343514	0.295245	-0.092 ± 0.009 %	2.291 ± 0.048 %
7.50	99.29%	1.085544 ± 0.002411	46.088344 ± 1.685529	0.017551 ± 0.000142	4.905227	0.298486	-0.078 ± 0.008 %	2.215 ± 0.050 %
7.75	99.32%	1.091225 ± 0.002385	49.149079 ± 1.714694	0.014173 ± 0.000128	4.346512	0.237379	-0.101 ± 0.008 %	2.013 ± 0.052 %
7.80	99.29%	1.083209 ± 0.002410	44.830380 ± 1.669455	0.016992 ± 0.000133	3.100509	0.285571	-0.077 ± 0.008 %	2.162 ± 0.041 %
8.00	99.31%	1.080260 ± 0.002373	43.241571 ± 1.636024	0.015367 ± 0.000130	4.035001	0.269038	-0.060 ± 0.008 %	2.071 ± 0.049 %
8.25	99.33%	1.088309 ± 0.002352	47.578112 ± 1.683232	0.012024 ± 0.000105	4.370751	0.192501	-0.089 ± 0.007 %	1.860 ± 0.044 %
8.30	99.34%	1.081929 ± 0.002328	44.140973 ± 1.628892	0.011498 ± 0.000084	1.861967	0.182563	-0.075 ± 0.007 %	1.784 ± 0.037 %
8.50	99.36%	1.078487 ± 0.002286	42.286422 ± 1.587902	0.009095 ± 0.000081	2.475720	0.147575	-0.064 ± 0.006 %	1.635 ± 0.048 %
8.75	99.37%	1.078905 ± 0.002274	42.511791 ± 1.587059	0.007852 ± 0.000073	2.163608	0.118868	-0.054 ± 0.006 %	1.535 ± 0.049 %
8.80	99.37%	1.078538 ± 0.002273	42.314029 ± 1.583733	0.007742 ± 0.000079	2.609957	0.127498	-0.053 ± 0.006 %	1.533 ± 0.053 %
9.00	99.37%	1.077841 ± 0.002265	41.938514 ± 1.575722	0.007380 ± 0.000073	2.547039	0.125235	-0.050 ± 0.006 %	1.499 ± 0.053 %
9.25	99.37%	1.075351 ± 0.002256	40.596804 ± 1.555539	0.006905 ± 0.000066	2.041534	0.103068	-0.042 ± 0.006 %	1.448 ± 0.044 %
9.30	99.37%	1.071370 ± 0.002244	38.451639 ± 1.524417	0.006834 ± 0.000062	1.959199	0.105753	-0.031 ± 0.006 %	1.444 ± 0.040 %
9.50	99.38%	1.073834 ± 0.002244	39.779313 ± 1.540846	0.006470 ± 0.000062	1.991340	0.098428	-0.041 ± 0.005 %	1.400 ± 0.041 %
9.75	99.38%	1.074517 ± 0.002240	40.147460 ± 1.543907	0.006206 ± 0.000068	2.638701	0.101805	-0.038 ± 0.005 %	1.388 ± 0.053 %
9.80	99.38%	1.076663 ± 0.002244	41.303655 ± 1.559774	0.006146 ± 0.000069	2.405736	0.094807	-0.044 ± 0.005 %	1.401 ± 0.057 %
10.00	99.38%	1.076016 ± 0.002240	40.955259 ± 1.553864	0.005796 ± 0.000055	2.059185	0.095681	-0.049 ± 0.005 %	1.313 ± 0.039 %
10.25	99.39%	1.074654 ± 0.002230	40.220957 ± 1.540922	0.005526 ± 0.000061	2.114943	0.099634	-0.043 ± 0.005 %	1.315 ± 0.050 %
10.30	99.39%	1.072069 ± 0.002222	38.828737 ± 1.519978	0.005418 ± 0.000059	2.062696	0.094137	-0.038 ± 0.005 %	1.288 ± 0.047 %
10.50	99.39%	1.070241 ± 0.002212	37.843815 ± 1.503987	0.005140 ± 0.000048	1.621503	0.087822	-0.032 ± 0.005 %	1.218 ± 0.028 %
10.75	99.39%	1.066075 ± 0.002195	35.599290 ± 1.470596	0.004415 ± 0.000060	2.666409	0.074945	-0.011 ± 0.004 %	1.152 ± 0.049 %
10.80	99.40%	1.066699 ± 0.002195	35.935359 ± 1.473675	0.004566 ± 0.000051	2.669465	0.074244	-0.025 ± 0.004 %	1.109 ± 0.017 %
11.00	99.40%	1.071004 ± 0.002207	38.254894 ± 1.509298	0.004029 ± 0.000035	1.280535	0.060520	-0.019 ± 0.004 %	1.112 ± 0.045 %
11.25	99.40%	1.071390 ± 0.002201	38.462580 ± 1.508764	0.003687 ± 0.000024	0.614354	0.054238	-0.025 ± 0.004 %	0.995 ± 0.011 %
11.30	99.40%	1.071772 ± 0.002202	38.668624 ± 1.511744	0.003665 ± 0.000024	0.613789	0.055209	-0.023 ± 0.004 %	0.984 ± 0.011 %
11.50	99.40%	1.072891 ± 0.002200	39.271504 ± 1.518324	0.003498 ± 0.000022	0.426572	0.052573	-0.031 ± 0.004 %	0.977 ± 0.013 %
11.75	99.41%	1.067707 ± 0.002184	36.478332 ± 1.478998	0.002967 ± 0.000023	0.803354	0.045490	-0.001 ± 0.003 %	0.895 ± 0.013 %
11.80	99.41%	1.065799 ± 0.002180	35.450545 ± 1.465811	0.002931 ± 0.000021	0.697018	0.045669	0.006 ± 0.003 %	0.899 ± 0.018 %