Experimental global target bits‑per‑weight quantization of ai-sage/GigaChat3.1-10B-A1.8B-bf16
- Using non-standard (forked) LLaMA C++ branch for quantization.
- Using a CLI tool to build KLD evaluation and imatrix calibration datasets for GGUF models, sourced from eaddario/imatrix-calibration.
- Using dataset sources: tools, math, code, text_en, text_ru.
- Using dataset chunks: 750.
- Tensors quantinization F16 instead of BF16, Nvidia Pascal architecture friendly like P100.
- Small set of patches added.
Many thanks to Ed Addario for an impressive job.
Quantization comparison
| BPW/TGS | SIZE (MiB) | PPL correlation | PPL mean ratio | ΔPPL | Mean KLD | Median KLD | Maximum KLD | 99.9% KLD | Mean Δp | RMS Δp |
|---|---|---|---|---|---|---|---|---|---|---|
| 2.750 | 3504 | 90.09% | 1.468708 ± 0.005521 | 7.175448 ± 0.098913 | 0.633578 ± 0.002363 | 0.313926 | 30.660967 | 10.392284 | -3.925 ± 0.046 % | 21.062 ± 0.068 % |
| 2.875 | 3663 | 90.85% | 1.367633 ± 0.004848 | 5.628103 ± 0.082593 | 0.562811 ± 0.002170 | 0.277341 | 25.643534 | 10.194194 | -3.832 ± 0.044 % | 20.141 ± 0.067 % |
| 3.000 | 3822 | 91.16% | 1.358840 ± 0.004750 | 5.493483 ± 0.081473 | 0.538277 ± 0.002095 | 0.263967 | 27.631903 | 9.956902 | -3.381 ± 0.043 % | 19.653 ± 0.067 % |
| 3.125 | 3981 | 93.15% | 1.238643 ± 0.003773 | 3.653381 ± 0.062362 | 0.421670 ± 0.001800 | 0.200371 | 28.351372 | 9.239901 | -2.627 ± 0.038 % | 17.619 ± 0.064 % |
| 3.250 | 4140 | 93.72% | 1.236406 ± 0.003615 | 3.619143 ± 0.060636 | 0.385529 ± 0.001704 | 0.183709 | 28.903301 | 8.885584 | -3.066 ± 0.037 % | 17.138 ± 0.064 % |
| 3.375 | 4299 | 94.49% | 1.182225 ± 0.003213 | 2.789691 ± 0.052334 | 0.335596 ± 0.001537 | 0.161642 | 26.339941 | 8.339195 | -2.754 ± 0.035 % | 16.065 ± 0.062 % |
| 3.500 | 4458 | 95.70% | 1.098661 ± 0.002612 | 1.510403 ± 0.040558 | 0.242724 ± 0.001334 | 0.095245 | 30.348700 | 7.182995 | -2.212 ± 0.031 % | 14.103 ± 0.063 % |
| 3.625 | 4617 | 95.60% | 1.077755 ± 0.002564 | 1.190348 ± 0.038773 | 0.254184 ± 0.001299 | 0.107248 | 34.746563 | 6.869932 | -1.966 ± 0.031 % | 14.120 ± 0.061 % |
| 3.750 | 4776 | 96.46% | 1.059019 ± 0.002278 | 0.903527 ± 0.034863 | 0.199596 ± 0.001136 | 0.079606 | 28.798738 | 6.495603 | -1.650 ± 0.028 % | 12.762 ± 0.060 % |
| 3.875 | 4935 | 96.56% | 1.014364 ± 0.002130 | 0.219904 ± 0.032325 | 0.193066 ± 0.001097 | 0.080995 | 28.850201 | 6.062064 | -1.619 ± 0.027 % | 12.511 ± 0.058 % |
| 4.000 | 5094 | 96.14% | 1.040658 ± 0.002317 | 0.622426 ± 0.034936 | 0.212281 ± 0.001107 | 0.095067 | 34.268818 | 5.975020 | -0.839 ± 0.028 % | 12.849 ± 0.057 % |
| 4.125 | 5253 | 97.13% | 0.989743 ± 0.001899 | -0.157030 ± 0.029319 | 0.158050 ± 0.000985 | 0.066715 | 33.086075 | 5.331661 | -1.389 ± 0.025 % | 11.412 ± 0.056 % |
| 4.250 | 5412 | 97.56% | 0.991427 ± 0.001760 | -0.131246 ± 0.027099 | 0.128950 ± 0.000883 | 0.053147 | 33.578720 | 4.832679 | -0.858 ± 0.023 % | 10.258 ± 0.054 % |
| 4.375 | 5571 | 98.42% | 1.014575 ± 0.001460 | 0.223132 ± 0.022368 | 0.080255 ± 0.000676 | 0.027194 | 25.935036 | 3.807253 | -0.339 ± 0.018 % | 8.266 ± 0.053 % |
| 4.500 | 5730 | 98.53% | 1.020259 ± 0.001424 | 0.310144 ± 0.021933 | 0.074827 ± 0.000709 | 0.024782 | 28.591846 | 3.737915 | -0.211 ± 0.017 % | 7.880 ± 0.052 % |
| 4.625 | 5890 | 98.43% | 1.012727 ± 0.001456 | 0.194840 ± 0.022305 | 0.077205 ± 0.000675 | 0.025733 | 25.961536 | 3.821413 | -0.249 ± 0.018 % | 8.150 ± 0.053 % |
| 4.750 | 6049 | 98.69% | 1.005256 ± 0.001318 | 0.080470 ± 0.020171 | 0.065456 ± 0.000611 | 0.021828 | 23.609543 | 3.366829 | -0.227 ± 0.017 % | 7.487 ± 0.052 % |
| 4.875 | 6208 | 98.68% | 1.007570 ± 0.001329 | 0.115888 ± 0.020355 | 0.064722 ± 0.000622 | 0.022027 | 23.776411 | 3.370074 | -0.192 ± 0.016 % | 7.400 ± 0.051 % |
| 5.000 | 6367 | 98.76% | 1.003428 ± 0.001279 | 0.052475 ± 0.019572 | 0.060466 ± 0.000579 | 0.020747 | 25.811150 | 3.064398 | -0.248 ± 0.016 % | 7.209 ± 0.051 % |
| 5.125 | 6526 | 98.77% | 1.006995 ± 0.001282 | 0.107087 ± 0.019632 | 0.059681 ± 0.000599 | 0.020448 | 32.101528 | 2.992807 | -0.140 ± 0.016 % | 7.118 ± 0.050 % |
| 5.250 | 6685 | 98.83% | 1.002481 ± 0.001243 | 0.037985 ± 0.019023 | 0.056570 ± 0.000609 | 0.019174 | 33.078915 | 2.996349 | -0.198 ± 0.015 % | 6.909 ± 0.050 % |
| 5.375 | 6844 | 98.89% | 1.006384 ± 0.001216 | 0.097726 ± 0.018624 | 0.053188 ± 0.000581 | 0.017634 | 29.220249 | 2.952291 | -0.157 ± 0.015 % | 6.679 ± 0.050 % |
| 5.500 | 7003 | 98.98% | 1.011065 ± 0.001177 | 0.169390 ± 0.018105 | 0.048359 ± 0.000525 | 0.015868 | 24.784534 | 2.487171 | 0.002 ± 0.014 % | 6.533 ± 0.049 % |
| 5.625 | 7162 | 99.18% | 0.993691 ± 0.001026 | -0.096579 ± 0.015738 | 0.035009 ± 0.000502 | 0.010065 | 24.782936 | 2.058697 | -0.135 ± 0.012 % | 5.578 ± 0.050 % |
| 5.750 | 7321 | 99.21% | 1.001571 ± 0.001022 | 0.024053 ± 0.015656 | 0.032926 ± 0.000478 | 0.009236 | 28.890903 | 1.993644 | -0.053 ± 0.012 % | 5.388 ± 0.049 % |
| 5.875 | 7480 | 99.28% | 1.000869 ± 0.000974 | 0.013302 ± 0.014916 | 0.029095 ± 0.000409 | 0.008163 | 22.879532 | 1.781184 | -0.064 ± 0.011 % | 5.062 ± 0.047 % |
| 6.000 | 7639 | 99.32% | 1.008715 ± 0.000956 | 0.133418 ± 0.014736 | 0.027239 ± 0.000437 | 0.007511 | 22.681236 | 1.728597 | -0.083 ± 0.011 % | 4.871 ± 0.048 % |
| 6.125 | 7798 | 99.34% | 1.002073 ± 0.000939 | 0.031741 ± 0.014379 | 0.025381 ± 0.000372 | 0.007354 | 22.027390 | 1.516017 | -0.093 ± 0.010 % | 4.741 ± 0.046 % |
| 6.250 | 7957 | 99.37% | 1.020782 ± 0.000945 | 0.318146 ± 0.014972 | 0.024460 ± 0.000432 | 0.006627 | 25.393843 | 1.640636 | -0.052 ± 0.010 % | 4.564 ± 0.047 % |
| 6.375 | 8116 | 99.39% | 1.012039 ± 0.000919 | 0.184303 ± 0.014282 | 0.023308 ± 0.000416 | 0.006228 | 23.982016 | 1.526792 | -0.043 ± 0.010 % | 4.502 ± 0.047 % |
| 6.500 | 8275 | 99.40% | 1.020302 ± 0.000918 | 0.310798 ± 0.014536 | 0.022949 ± 0.000411 | 0.006061 | 25.441256 | 1.560629 | -0.073 ± 0.010 % | 4.459 ± 0.047 % |
| 6.625 | 8434 | 99.50% | 1.015903 ± 0.000836 | 0.243456 ± 0.013148 | 0.017084 ± 0.000349 | 0.004404 | 21.899401 | 1.088672 | -0.047 ± 0.009 % | 3.880 ± 0.046 % |
| 6.750 | 8593 | 99.54% | 1.025129 ± 0.000819 | 0.384694 ± 0.013381 | 0.014613 ± 0.000324 | 0.003346 | 22.134905 | 1.044071 | 0.025 ± 0.008 % | 3.586 ± 0.047 % |
| 6.875 | 8752 | 99.53% | 1.021068 ± 0.000819 | 0.322524 ± 0.013152 | 0.014827 ± 0.000345 | 0.003374 | 31.316954 | 1.047663 | 0.000 ± 0.008 % | 3.596 ± 0.046 % |
| 7.000 | 8911 | 99.55% | 1.024709 ± 0.000805 | 0.378267 ± 0.013154 | 0.014163 ± 0.000307 | 0.003245 | 21.271984 | 1.047468 | 0.011 ± 0.008 % | 3.549 ± 0.047 % |
| 7.125 | 9070 | 99.55% | 1.022398 ± 0.000807 | 0.342897 ± 0.013045 | 0.014525 ± 0.000327 | 0.003295 | 21.503429 | 1.024429 | 0.014 ± 0.008 % | 3.593 ± 0.048 % |
| 7.250 | 9229 | 99.56% | 1.028127 ± 0.000808 | 0.430596 ± 0.013412 | 0.013705 ± 0.000289 | 0.003147 | 19.222500 | 1.020773 | 0.032 ± 0.008 % | 3.505 ± 0.047 % |
| 7.375 | 9388 | 99.57% | 1.027008 ± 0.000792 | 0.413460 ± 0.013127 | 0.013062 ± 0.000278 | 0.002880 | 18.601868 | 0.991505 | 0.021 ± 0.008 % | 3.478 ± 0.049 % |
| 7.500 | 9547 | 99.58% | 1.022934 ± 0.000782 | 0.351091 ± 0.012730 | 0.012368 ± 0.000291 | 0.002729 | 18.961008 | 0.900389 | 0.011 ± 0.007 % | 3.332 ± 0.047 % |
| 7.625 | 9706 | 99.57% | 1.027221 ± 0.000793 | 0.416723 ± 0.013155 | 0.012728 ± 0.000334 | 0.002821 | 26.743662 | 0.889915 | 0.039 ± 0.007 % | 3.399 ± 0.048 % |
| 7.750 | 9865 | 99.58% | 1.025006 ± 0.000784 | 0.382820 ± 0.012889 | 0.012181 ± 0.000286 | 0.002689 | 17.761831 | 0.925288 | 0.023 ± 0.007 % | 3.367 ± 0.048 % |
| 7.875 | 10024 | 99.58% | 1.026968 ± 0.000785 | 0.412852 ± 0.013041 | 0.012134 ± 0.000297 | 0.002717 | 20.470541 | 0.858458 | 0.043 ± 0.007 % | 3.291 ± 0.045 % |
| 8.000 | 10183 | 99.59% | 1.024610 ± 0.000777 | 0.376758 ± 0.012760 | 0.011861 ± 0.000302 | 0.002600 | 23.916935 | 0.892956 | 0.008 ± 0.007 % | 3.292 ± 0.048 % |
| 8.125 | 10342 | 99.60% | 1.025372 ± 0.000763 | 0.388413 ± 0.012623 | 0.011646 ± 0.000280 | 0.002636 | 19.365534 | 0.855019 | 0.028 ± 0.007 % | 3.250 ± 0.045 % |
| 8.250 | 10501 | 99.60% | 1.024733 ± 0.000765 | 0.378644 ± 0.012606 | 0.011539 ± 0.000290 | 0.002505 | 23.350641 | 0.838322 | 0.004 ± 0.007 % | 3.250 ± 0.047 % |
| 8.375 | 10661 | 99.60% | 1.022929 ± 0.000761 | 0.351025 ± 0.012442 | 0.011353 ± 0.000303 | 0.002429 | 24.381865 | 0.840560 | 0.023 ± 0.007 % | 3.225 ± 0.048 % |
| 8.500 | 10819 | 99.61% | 1.021716 ± 0.000752 | 0.332456 ± 0.012246 | 0.010923 ± 0.000278 | 0.002375 | 25.404078 | 0.804361 | 0.025 ± 0.007 % | 3.175 ± 0.049 % |
| 8.625 | 10978 | 99.61% | 1.023591 ± 0.000750 | 0.361155 ± 0.012322 | 0.010546 ± 0.000277 | 0.002151 | 18.915178 | 0.821481 | 0.022 ± 0.007 % | 3.139 ± 0.050 % |
| 8.750 | 11137 | 99.66% | 1.016991 ± 0.000696 | 0.260111 ± 0.011190 | 0.008489 ± 0.000264 | 0.001579 | 22.072056 | 0.705830 | 0.024 ± 0.006 % | 2.832 ± 0.052 % |
| 8.875 | 11296 | 99.66% | 1.016703 ± 0.000691 | 0.255707 ± 0.011100 | 0.008476 ± 0.000277 | 0.001562 | 20.473799 | 0.683693 | 0.014 ± 0.006 % | 2.759 ± 0.050 % |
| 9.000 | 11456 | 99.66% | 1.016588 ± 0.000695 | 0.253941 ± 0.011152 | 0.008375 ± 0.000274 | 0.001422 | 20.314676 | 0.684510 | 0.018 ± 0.006 % | 2.763 ± 0.050 % |
| 9.125 | 11614 | 99.66% | 1.017290 ± 0.000692 | 0.264687 ± 0.011147 | 0.008240 ± 0.000274 | 0.001384 | 20.799490 | 0.658757 | 0.029 ± 0.006 % | 2.735 ± 0.050 % |
| 9.250 | 11774 | 99.66% | 1.018057 ± 0.000697 | 0.276428 ± 0.011269 | 0.008108 ± 0.000228 | 0.001403 | 16.491100 | 0.711742 | 0.035 ± 0.006 % | 2.749 ± 0.048 % |
| 9.375 | 11932 | 99.67% | 1.015225 ± 0.000685 | 0.233083 ± 0.010937 | 0.007767 ± 0.000270 | 0.001289 | 19.463987 | 0.674894 | 0.012 ± 0.006 % | 2.675 ± 0.048 % |
| 9.500 | 12091 | 99.67% | 1.018677 ± 0.000692 | 0.285927 ± 0.011223 | 0.008071 ± 0.000272 | 0.001354 | 22.177876 | 0.690820 | 0.034 ± 0.006 % | 2.780 ± 0.051 % |
| 9.625 | 12250 | 99.67% | 1.014632 ± 0.000683 | 0.224007 ± 0.010875 | 0.007610 ± 0.000280 | 0.001276 | 24.419128 | 0.619948 | 0.015 ± 0.006 % | 2.639 ± 0.048 % |
| 9.750 | 12408 | 99.67% | 1.014877 ± 0.000684 | 0.227755 ± 0.010913 | 0.007592 ± 0.000262 | 0.001231 | 19.517189 | 0.601438 | 0.014 ± 0.006 % | 2.645 ± 0.050 % |
| 9.875 | 12568 | 99.67% | 1.013450 ± 0.000679 | 0.205913 ± 0.010769 | 0.007440 ± 0.000277 | 0.001202 | 20.140268 | 0.636559 | 0.022 ± 0.006 % | 2.612 ± 0.050 % |
| 10.000 | 12727 | 99.67% | 1.014093 ± 0.000683 | 0.215754 ± 0.010861 | 0.007558 ± 0.000274 | 0.001231 | 19.714268 | 0.632921 | 0.014 ± 0.006 % | 2.641 ± 0.050 % |
| 10.125 | 12886 | 99.67% | 1.013744 ± 0.000678 | 0.210406 ± 0.010762 | 0.007376 ± 0.000246 | 0.001186 | 19.442703 | 0.630977 | 0.010 ± 0.006 % | 2.646 ± 0.053 % |
| 10.250 | 13046 | 99.67% | 1.013908 ± 0.000680 | 0.212922 ± 0.010808 | 0.007539 ± 0.000281 | 0.001231 | 23.797590 | 0.607142 | 0.013 ± 0.006 % | 2.639 ± 0.050 % |
| 10.375 | 13204 | 99.68% | 1.014028 ± 0.000673 | 0.214749 ± 0.010714 | 0.007474 ± 0.000272 | 0.001193 | 20.718899 | 0.612245 | 0.013 ± 0.006 % | 2.612 ± 0.050 % |
| 10.500 | 13364 | 99.67% | 1.013952 ± 0.000676 | 0.213597 ± 0.010750 | 0.007299 ± 0.000261 | 0.001145 | 19.410480 | 0.618763 | 0.013 ± 0.006 % | 2.578 ± 0.049 % |
| 10.625 | 13523 | 99.67% | 1.013976 ± 0.000677 | 0.213963 ± 0.010763 | 0.007451 ± 0.000261 | 0.001189 | 20.569273 | 0.622427 | 0.019 ± 0.006 % | 2.592 ± 0.049 % |
| 10.750 | 13680 | 99.67% | 1.014306 ± 0.000677 | 0.219003 ± 0.010782 | 0.007248 ± 0.000247 | 0.001148 | 19.228298 | 0.613274 | 0.011 ± 0.006 % | 2.586 ± 0.050 % |
| 10.875 | 13840 | 99.67% | 1.015255 ± 0.000679 | 0.233540 ± 0.010856 | 0.007507 ± 0.000281 | 0.001193 | 20.199493 | 0.593870 | 0.017 ± 0.006 % | 2.613 ± 0.050 % |
| 11.000 | 13999 | 99.67% | 1.014683 ± 0.000678 | 0.224779 ± 0.010816 | 0.007216 ± 0.000247 | 0.001145 | 17.941696 | 0.647005 | 0.016 ± 0.006 % | 2.611 ± 0.051 % |
| 11.125 | 14159 | 99.69% | 1.013409 ± 0.000664 | 0.205278 ± 0.010543 | 0.007003 ± 0.000258 | 0.001083 | 18.874243 | 0.592008 | 0.017 ± 0.006 % | 2.540 ± 0.048 % |
| 11.250 | 14318 | 99.69% | 1.012913 ± 0.000664 | 0.197686 ± 0.010520 | 0.006675 ± 0.000245 | 0.001053 | 19.168644 | 0.563273 | 0.015 ± 0.006 % | 2.505 ± 0.050 % |
| 11.375 | 14477 | 99.68% | 1.013141 ± 0.000668 | 0.201168 ± 0.010595 | 0.006782 ± 0.000237 | 0.001087 | 18.673468 | 0.586665 | 0.015 ± 0.005 % | 2.490 ± 0.048 % |
| 11.500 | 14636 | 99.69% | 1.012523 ± 0.000660 | 0.191714 ± 0.010454 | 0.006529 ± 0.000236 | 0.001042 | 18.647163 | 0.548347 | 0.019 ± 0.005 % | 2.449 ± 0.049 % |
| 11.625 | 14792 | 99.69% | 1.013233 ± 0.000657 | 0.202580 ± 0.010438 | 0.006673 ± 0.000254 | 0.001027 | 18.959124 | 0.565285 | 0.015 ± 0.005 % | 2.463 ± 0.051 % |
| 11.750 | 14953 | 99.68% | 1.013257 ± 0.000669 | 0.202953 ± 0.010613 | 0.007152 ± 0.000285 | 0.001047 | 18.918600 | 0.630295 | 0.018 ± 0.006 % | 2.532 ± 0.051 % |
| 11.875 | 15113 | 99.69% | 1.013355 ± 0.000661 | 0.204458 ± 0.010493 | 0.006514 ± 0.000231 | 0.001013 | 19.042858 | 0.575434 | 0.011 ± 0.005 % | 2.461 ± 0.048 % |
| 12.000 | 15271 | 99.68% | 1.013285 ± 0.000665 | 0.203384 ± 0.010551 | 0.006936 ± 0.000268 | 0.001065 | 21.028812 | 0.592060 | 0.013 ± 0.006 % | 2.541 ± 0.051 % |
| 12.125 | 15431 | 99.69% | 1.013224 ± 0.000660 | 0.202447 ± 0.010488 | 0.006615 ± 0.000235 | 0.001027 | 19.068192 | 0.602703 | 0.024 ± 0.006 % | 2.530 ± 0.051 % |
| 12.250 | 15588 | 99.69% | 1.013507 ± 0.000659 | 0.206774 ± 0.010476 | 0.006742 ± 0.000270 | 0.001005 | 19.197704 | 0.560796 | 0.016 ± 0.005 % | 2.460 ± 0.049 % |
| 12.375 | 15749 | 99.68% | 1.012585 ± 0.000666 | 0.192658 ± 0.010549 | 0.006849 ± 0.000263 | 0.001032 | 19.046614 | 0.624302 | 0.023 ± 0.006 % | 2.511 ± 0.050 % |
| 12.500 | 15909 | 99.69% | 1.013416 ± 0.000656 | 0.205388 ± 0.010426 | 0.006976 ± 0.000341 | 0.000996 | 30.070782 | 0.553184 | 0.015 ± 0.005 % | 2.453 ± 0.051 % |
| 12.625 | 16068 | 99.69% | 1.013417 ± 0.000664 | 0.205405 ± 0.010551 | 0.006847 ± 0.000255 | 0.001034 | 18.872683 | 0.592905 | 0.021 ± 0.006 % | 2.511 ± 0.051 % |
| 12.750 | 16225 | 99.69% | 1.013988 ± 0.000661 | 0.214149 ± 0.010524 | 0.006546 ± 0.000255 | 0.000997 | 19.983679 | 0.570902 | 0.020 ± 0.005 % | 2.420 ± 0.047 % |
| 12.875 | 16384 | 99.69% | 1.012895 ± 0.000656 | 0.197416 ± 0.010405 | 0.006727 ± 0.000276 | 0.000977 | 20.882362 | 0.548220 | 0.020 ± 0.005 % | 2.481 ± 0.052 % |
| 13.000 | 16544 | 99.68% | 1.012696 ± 0.000668 | 0.194360 ± 0.010572 | 0.006706 ± 0.000273 | 0.000997 | 21.368870 | 0.577613 | 0.015 ± 0.005 % | 2.482 ± 0.051 % |
| 13.125 | 16696 | 99.69% | 1.012827 ± 0.000655 | 0.196367 ± 0.010396 | 0.006580 ± 0.000276 | 0.000961 | 19.636349 | 0.539285 | 0.022 ± 0.005 % | 2.418 ± 0.049 % |
| 13.250 | 16862 | 99.70% | 1.013051 ± 0.000652 | 0.199793 ± 0.010356 | 0.006516 ± 0.000245 | 0.001011 | 18.378595 | 0.584926 | 0.018 ± 0.005 % | 2.416 ± 0.049 % |
| 13.375 | 17022 | 99.70% | 1.012315 ± 0.000650 | 0.188526 ± 0.010298 | 0.006461 ± 0.000271 | 0.000967 | 19.541945 | 0.532116 | 0.014 ± 0.005 % | 2.427 ± 0.050 % |
| 13.500 | 17180 | 99.69% | 1.012152 ± 0.000661 | 0.186035 ± 0.010446 | 0.006609 ± 0.000256 | 0.001001 | 19.819275 | 0.579343 | 0.012 ± 0.005 % | 2.414 ± 0.050 % |
| 13.625 | 17340 | 99.69% | 1.012344 ± 0.000663 | 0.188980 ± 0.010478 | 0.006545 ± 0.000266 | 0.000967 | 21.722115 | 0.581532 | 0.014 ± 0.005 % | 2.406 ± 0.049 % |
| 13.750 | 17492 | 99.69% | 1.012725 ± 0.000656 | 0.194806 ± 0.010399 | 0.006602 ± 0.000311 | 0.000947 | 31.362030 | 0.558576 | 0.016 ± 0.005 % | 2.458 ± 0.052 % |
| 13.875 | 17657 | 99.69% | 1.012659 ± 0.000654 | 0.193798 ± 0.010372 | 0.006449 ± 0.000240 | 0.000980 | 18.928963 | 0.584710 | 0.017 ± 0.005 % | 2.418 ± 0.049 % |
| 14.000 | 17813 | 99.70% | 1.012146 ± 0.000651 | 0.185948 ± 0.010307 | 0.006304 ± 0.000235 | 0.000948 | 18.758667 | 0.537241 | 0.018 ± 0.005 % | 2.407 ± 0.050 % |
| 14.125 | 17975 | 99.68% | 1.013601 ± 0.000667 | 0.208224 ± 0.010593 | 0.006542 ± 0.000246 | 0.000976 | 21.416483 | 0.633320 | 0.009 ± 0.005 % | 2.474 ± 0.053 % |
| 14.250 | 18134 | 99.69% | 1.012774 ± 0.000655 | 0.195555 ± 0.010383 | 0.006236 ± 0.000247 | 0.000934 | 18.788988 | 0.538596 | 0.012 ± 0.005 % | 2.376 ± 0.048 % |
| 14.375 | 18288 | 99.70% | 1.012470 ± 0.000649 | 0.190906 ± 0.010282 | 0.006284 ± 0.000296 | 0.000906 | 31.802967 | 0.547245 | 0.022 ± 0.005 % | 2.411 ± 0.052 % |
| 14.500 | 18453 | 99.70% | 1.012954 ± 0.000653 | 0.198314 ± 0.010367 | 0.006192 ± 0.000230 | 0.000945 | 18.992279 | 0.574698 | 0.018 ± 0.005 % | 2.371 ± 0.047 % |
| 14.625 | 18609 | 99.70% | 1.013029 ± 0.000652 | 0.199464 ± 0.010350 | 0.006264 ± 0.000297 | 0.000903 | 29.076559 | 0.532940 | 0.017 ± 0.005 % | 2.400 ± 0.052 % |
| 14.750 | 18771 | 99.70% | 1.012923 ± 0.000650 | 0.197831 ± 0.010328 | 0.006385 ± 0.000257 | 0.000951 | 19.178585 | 0.540540 | 0.023 ± 0.005 % | 2.390 ± 0.050 % |
| 14.875 | 18929 | 99.69% | 1.012335 ± 0.000657 | 0.188844 ± 0.010402 | 0.006149 ± 0.000233 | 0.000920 | 17.733660 | 0.523330 | 0.012 ± 0.005 % | 2.344 ± 0.047 % |
| 15.000 | 19084 | 99.70% | 1.012698 ± 0.000649 | 0.194390 ± 0.010300 | 0.006208 ± 0.000292 | 0.000891 | 29.146189 | 0.530963 | 0.019 ± 0.005 % | 2.389 ± 0.051 % |
| 15.125 | 19247 | 99.69% | 1.012992 ± 0.000655 | 0.198896 ± 0.010407 | 0.006141 ± 0.000245 | 0.000910 | 19.324158 | 0.547946 | 0.019 ± 0.005 % | 2.343 ± 0.048 % |
| 15.250 | 19406 | 99.70% | 1.013381 ± 0.000646 | 0.204855 ± 0.010286 | 0.006350 ± 0.000257 | 0.000887 | 17.933092 | 0.592217 | 0.023 ± 0.005 % | 2.422 ± 0.051 % |
| 15.375 | 19564 | 99.70% | 1.012699 ± 0.000650 | 0.194414 ± 0.010315 | 0.006196 ± 0.000285 | 0.000861 | 24.918396 | 0.519731 | 0.024 ± 0.005 % | 2.337 ± 0.051 % |
| 15.500 | 19719 | 99.70% | 1.012664 ± 0.000652 | 0.193868 ± 0.010342 | 0.005820 ± 0.000237 | 0.000838 | 18.379749 | 0.492615 | 0.028 ± 0.005 % | 2.299 ± 0.050 % |
| 15.625 | 19884 | 99.70% | 1.013846 ± 0.000651 | 0.211964 ± 0.010386 | 0.006172 ± 0.000278 | 0.000867 | 26.033194 | 0.505156 | 0.023 ± 0.005 % | 2.343 ± 0.050 % |
| 15.750 | 20039 | 99.70% | 1.013008 ± 0.000651 | 0.199144 ± 0.010352 | 0.005814 ± 0.000238 | 0.000836 | 18.426023 | 0.492568 | 0.023 ± 0.005 % | 2.297 ± 0.050 % |
| 15.875 | 20202 | 99.70% | 1.013150 ± 0.000650 | 0.201306 ± 0.010331 | 0.005794 ± 0.000236 | 0.000824 | 18.269939 | 0.495334 | 0.023 ± 0.005 % | 2.287 ± 0.050 % |
| 16.000 | 20354 | 99.71% | 1.011306 ± 0.000639 | 0.173086 ± 0.010084 | 0.005484 ± 0.000244 | 0.000750 | 18.494905 | 0.477412 | 0.018 ± 0.005 % | 2.187 ± 0.049 % |
- Downloads last month
- 36,690
Hardware compatibility
Log In to add your hardware
We're not able to determine the quantization variants.
Model tree for ENOSYS/GigaChat3.1-10B-A1.8B-750-v1-GGUF
Base model
ai-sage/GigaChat3-10B-A1.8B-base Finetuned
ai-sage/GigaChat3.1-10B-A1.8B-bf16