Experimental global target bits‑per‑weight quantization of ai-sage/GigaChat3.1-10B-A1.8B-bf16

  • Using non-standard (forked) LLaMA C++ branch for quantization.
  • Using a CLI tool to build KLD evaluation and imatrix calibration datasets for GGUF models, sourced from eaddario/imatrix-calibration.
  • Using dataset sources: tools, math, code, text_en, text_ru.
  • Using dataset chunks: 750.
  • Tensors quantinization F16 instead of BF16, Nvidia Pascal architecture friendly like P100.
  • Small set of patches added.

Many thanks to Ed Addario for an impressive job.

Quantization comparison

BPW/TGS SIZE (MiB) PPL correlation PPL mean ratio ΔPPL Mean KLD Median KLD Maximum KLD 99.9% KLD Mean Δp RMS Δp
2.750 3504 90.09% 1.468708 ± 0.005521 7.175448 ± 0.098913 0.633578 ± 0.002363 0.313926 30.660967 10.392284 -3.925 ± 0.046 % 21.062 ± 0.068 %
2.875 3663 90.85% 1.367633 ± 0.004848 5.628103 ± 0.082593 0.562811 ± 0.002170 0.277341 25.643534 10.194194 -3.832 ± 0.044 % 20.141 ± 0.067 %
3.000 3822 91.16% 1.358840 ± 0.004750 5.493483 ± 0.081473 0.538277 ± 0.002095 0.263967 27.631903 9.956902 -3.381 ± 0.043 % 19.653 ± 0.067 %
3.125 3981 93.15% 1.238643 ± 0.003773 3.653381 ± 0.062362 0.421670 ± 0.001800 0.200371 28.351372 9.239901 -2.627 ± 0.038 % 17.619 ± 0.064 %
3.250 4140 93.72% 1.236406 ± 0.003615 3.619143 ± 0.060636 0.385529 ± 0.001704 0.183709 28.903301 8.885584 -3.066 ± 0.037 % 17.138 ± 0.064 %
3.375 4299 94.49% 1.182225 ± 0.003213 2.789691 ± 0.052334 0.335596 ± 0.001537 0.161642 26.339941 8.339195 -2.754 ± 0.035 % 16.065 ± 0.062 %
3.500 4458 95.70% 1.098661 ± 0.002612 1.510403 ± 0.040558 0.242724 ± 0.001334 0.095245 30.348700 7.182995 -2.212 ± 0.031 % 14.103 ± 0.063 %
3.625 4617 95.60% 1.077755 ± 0.002564 1.190348 ± 0.038773 0.254184 ± 0.001299 0.107248 34.746563 6.869932 -1.966 ± 0.031 % 14.120 ± 0.061 %
3.750 4776 96.46% 1.059019 ± 0.002278 0.903527 ± 0.034863 0.199596 ± 0.001136 0.079606 28.798738 6.495603 -1.650 ± 0.028 % 12.762 ± 0.060 %
3.875 4935 96.56% 1.014364 ± 0.002130 0.219904 ± 0.032325 0.193066 ± 0.001097 0.080995 28.850201 6.062064 -1.619 ± 0.027 % 12.511 ± 0.058 %
4.000 5094 96.14% 1.040658 ± 0.002317 0.622426 ± 0.034936 0.212281 ± 0.001107 0.095067 34.268818 5.975020 -0.839 ± 0.028 % 12.849 ± 0.057 %
4.125 5253 97.13% 0.989743 ± 0.001899 -0.157030 ± 0.029319 0.158050 ± 0.000985 0.066715 33.086075 5.331661 -1.389 ± 0.025 % 11.412 ± 0.056 %
4.250 5412 97.56% 0.991427 ± 0.001760 -0.131246 ± 0.027099 0.128950 ± 0.000883 0.053147 33.578720 4.832679 -0.858 ± 0.023 % 10.258 ± 0.054 %
4.375 5571 98.42% 1.014575 ± 0.001460 0.223132 ± 0.022368 0.080255 ± 0.000676 0.027194 25.935036 3.807253 -0.339 ± 0.018 % 8.266 ± 0.053 %
4.500 5730 98.53% 1.020259 ± 0.001424 0.310144 ± 0.021933 0.074827 ± 0.000709 0.024782 28.591846 3.737915 -0.211 ± 0.017 % 7.880 ± 0.052 %
4.625 5890 98.43% 1.012727 ± 0.001456 0.194840 ± 0.022305 0.077205 ± 0.000675 0.025733 25.961536 3.821413 -0.249 ± 0.018 % 8.150 ± 0.053 %
4.750 6049 98.69% 1.005256 ± 0.001318 0.080470 ± 0.020171 0.065456 ± 0.000611 0.021828 23.609543 3.366829 -0.227 ± 0.017 % 7.487 ± 0.052 %
4.875 6208 98.68% 1.007570 ± 0.001329 0.115888 ± 0.020355 0.064722 ± 0.000622 0.022027 23.776411 3.370074 -0.192 ± 0.016 % 7.400 ± 0.051 %
5.000 6367 98.76% 1.003428 ± 0.001279 0.052475 ± 0.019572 0.060466 ± 0.000579 0.020747 25.811150 3.064398 -0.248 ± 0.016 % 7.209 ± 0.051 %
5.125 6526 98.77% 1.006995 ± 0.001282 0.107087 ± 0.019632 0.059681 ± 0.000599 0.020448 32.101528 2.992807 -0.140 ± 0.016 % 7.118 ± 0.050 %
5.250 6685 98.83% 1.002481 ± 0.001243 0.037985 ± 0.019023 0.056570 ± 0.000609 0.019174 33.078915 2.996349 -0.198 ± 0.015 % 6.909 ± 0.050 %
5.375 6844 98.89% 1.006384 ± 0.001216 0.097726 ± 0.018624 0.053188 ± 0.000581 0.017634 29.220249 2.952291 -0.157 ± 0.015 % 6.679 ± 0.050 %
5.500 7003 98.98% 1.011065 ± 0.001177 0.169390 ± 0.018105 0.048359 ± 0.000525 0.015868 24.784534 2.487171 0.002 ± 0.014 % 6.533 ± 0.049 %
5.625 7162 99.18% 0.993691 ± 0.001026 -0.096579 ± 0.015738 0.035009 ± 0.000502 0.010065 24.782936 2.058697 -0.135 ± 0.012 % 5.578 ± 0.050 %
5.750 7321 99.21% 1.001571 ± 0.001022 0.024053 ± 0.015656 0.032926 ± 0.000478 0.009236 28.890903 1.993644 -0.053 ± 0.012 % 5.388 ± 0.049 %
5.875 7480 99.28% 1.000869 ± 0.000974 0.013302 ± 0.014916 0.029095 ± 0.000409 0.008163 22.879532 1.781184 -0.064 ± 0.011 % 5.062 ± 0.047 %
6.000 7639 99.32% 1.008715 ± 0.000956 0.133418 ± 0.014736 0.027239 ± 0.000437 0.007511 22.681236 1.728597 -0.083 ± 0.011 % 4.871 ± 0.048 %
6.125 7798 99.34% 1.002073 ± 0.000939 0.031741 ± 0.014379 0.025381 ± 0.000372 0.007354 22.027390 1.516017 -0.093 ± 0.010 % 4.741 ± 0.046 %
6.250 7957 99.37% 1.020782 ± 0.000945 0.318146 ± 0.014972 0.024460 ± 0.000432 0.006627 25.393843 1.640636 -0.052 ± 0.010 % 4.564 ± 0.047 %
6.375 8116 99.39% 1.012039 ± 0.000919 0.184303 ± 0.014282 0.023308 ± 0.000416 0.006228 23.982016 1.526792 -0.043 ± 0.010 % 4.502 ± 0.047 %
6.500 8275 99.40% 1.020302 ± 0.000918 0.310798 ± 0.014536 0.022949 ± 0.000411 0.006061 25.441256 1.560629 -0.073 ± 0.010 % 4.459 ± 0.047 %
6.625 8434 99.50% 1.015903 ± 0.000836 0.243456 ± 0.013148 0.017084 ± 0.000349 0.004404 21.899401 1.088672 -0.047 ± 0.009 % 3.880 ± 0.046 %
6.750 8593 99.54% 1.025129 ± 0.000819 0.384694 ± 0.013381 0.014613 ± 0.000324 0.003346 22.134905 1.044071 0.025 ± 0.008 % 3.586 ± 0.047 %
6.875 8752 99.53% 1.021068 ± 0.000819 0.322524 ± 0.013152 0.014827 ± 0.000345 0.003374 31.316954 1.047663 0.000 ± 0.008 % 3.596 ± 0.046 %
7.000 8911 99.55% 1.024709 ± 0.000805 0.378267 ± 0.013154 0.014163 ± 0.000307 0.003245 21.271984 1.047468 0.011 ± 0.008 % 3.549 ± 0.047 %
7.125 9070 99.55% 1.022398 ± 0.000807 0.342897 ± 0.013045 0.014525 ± 0.000327 0.003295 21.503429 1.024429 0.014 ± 0.008 % 3.593 ± 0.048 %
7.250 9229 99.56% 1.028127 ± 0.000808 0.430596 ± 0.013412 0.013705 ± 0.000289 0.003147 19.222500 1.020773 0.032 ± 0.008 % 3.505 ± 0.047 %
7.375 9388 99.57% 1.027008 ± 0.000792 0.413460 ± 0.013127 0.013062 ± 0.000278 0.002880 18.601868 0.991505 0.021 ± 0.008 % 3.478 ± 0.049 %
7.500 9547 99.58% 1.022934 ± 0.000782 0.351091 ± 0.012730 0.012368 ± 0.000291 0.002729 18.961008 0.900389 0.011 ± 0.007 % 3.332 ± 0.047 %
7.625 9706 99.57% 1.027221 ± 0.000793 0.416723 ± 0.013155 0.012728 ± 0.000334 0.002821 26.743662 0.889915 0.039 ± 0.007 % 3.399 ± 0.048 %
7.750 9865 99.58% 1.025006 ± 0.000784 0.382820 ± 0.012889 0.012181 ± 0.000286 0.002689 17.761831 0.925288 0.023 ± 0.007 % 3.367 ± 0.048 %
7.875 10024 99.58% 1.026968 ± 0.000785 0.412852 ± 0.013041 0.012134 ± 0.000297 0.002717 20.470541 0.858458 0.043 ± 0.007 % 3.291 ± 0.045 %
8.000 10183 99.59% 1.024610 ± 0.000777 0.376758 ± 0.012760 0.011861 ± 0.000302 0.002600 23.916935 0.892956 0.008 ± 0.007 % 3.292 ± 0.048 %
8.125 10342 99.60% 1.025372 ± 0.000763 0.388413 ± 0.012623 0.011646 ± 0.000280 0.002636 19.365534 0.855019 0.028 ± 0.007 % 3.250 ± 0.045 %
8.250 10501 99.60% 1.024733 ± 0.000765 0.378644 ± 0.012606 0.011539 ± 0.000290 0.002505 23.350641 0.838322 0.004 ± 0.007 % 3.250 ± 0.047 %
8.375 10661 99.60% 1.022929 ± 0.000761 0.351025 ± 0.012442 0.011353 ± 0.000303 0.002429 24.381865 0.840560 0.023 ± 0.007 % 3.225 ± 0.048 %
8.500 10819 99.61% 1.021716 ± 0.000752 0.332456 ± 0.012246 0.010923 ± 0.000278 0.002375 25.404078 0.804361 0.025 ± 0.007 % 3.175 ± 0.049 %
8.625 10978 99.61% 1.023591 ± 0.000750 0.361155 ± 0.012322 0.010546 ± 0.000277 0.002151 18.915178 0.821481 0.022 ± 0.007 % 3.139 ± 0.050 %
8.750 11137 99.66% 1.016991 ± 0.000696 0.260111 ± 0.011190 0.008489 ± 0.000264 0.001579 22.072056 0.705830 0.024 ± 0.006 % 2.832 ± 0.052 %
8.875 11296 99.66% 1.016703 ± 0.000691 0.255707 ± 0.011100 0.008476 ± 0.000277 0.001562 20.473799 0.683693 0.014 ± 0.006 % 2.759 ± 0.050 %
9.000 11456 99.66% 1.016588 ± 0.000695 0.253941 ± 0.011152 0.008375 ± 0.000274 0.001422 20.314676 0.684510 0.018 ± 0.006 % 2.763 ± 0.050 %
9.125 11614 99.66% 1.017290 ± 0.000692 0.264687 ± 0.011147 0.008240 ± 0.000274 0.001384 20.799490 0.658757 0.029 ± 0.006 % 2.735 ± 0.050 %
9.250 11774 99.66% 1.018057 ± 0.000697 0.276428 ± 0.011269 0.008108 ± 0.000228 0.001403 16.491100 0.711742 0.035 ± 0.006 % 2.749 ± 0.048 %
9.375 11932 99.67% 1.015225 ± 0.000685 0.233083 ± 0.010937 0.007767 ± 0.000270 0.001289 19.463987 0.674894 0.012 ± 0.006 % 2.675 ± 0.048 %
9.500 12091 99.67% 1.018677 ± 0.000692 0.285927 ± 0.011223 0.008071 ± 0.000272 0.001354 22.177876 0.690820 0.034 ± 0.006 % 2.780 ± 0.051 %
9.625 12250 99.67% 1.014632 ± 0.000683 0.224007 ± 0.010875 0.007610 ± 0.000280 0.001276 24.419128 0.619948 0.015 ± 0.006 % 2.639 ± 0.048 %
9.750 12408 99.67% 1.014877 ± 0.000684 0.227755 ± 0.010913 0.007592 ± 0.000262 0.001231 19.517189 0.601438 0.014 ± 0.006 % 2.645 ± 0.050 %
9.875 12568 99.67% 1.013450 ± 0.000679 0.205913 ± 0.010769 0.007440 ± 0.000277 0.001202 20.140268 0.636559 0.022 ± 0.006 % 2.612 ± 0.050 %
10.000 12727 99.67% 1.014093 ± 0.000683 0.215754 ± 0.010861 0.007558 ± 0.000274 0.001231 19.714268 0.632921 0.014 ± 0.006 % 2.641 ± 0.050 %
10.125 12886 99.67% 1.013744 ± 0.000678 0.210406 ± 0.010762 0.007376 ± 0.000246 0.001186 19.442703 0.630977 0.010 ± 0.006 % 2.646 ± 0.053 %
10.250 13046 99.67% 1.013908 ± 0.000680 0.212922 ± 0.010808 0.007539 ± 0.000281 0.001231 23.797590 0.607142 0.013 ± 0.006 % 2.639 ± 0.050 %
10.375 13204 99.68% 1.014028 ± 0.000673 0.214749 ± 0.010714 0.007474 ± 0.000272 0.001193 20.718899 0.612245 0.013 ± 0.006 % 2.612 ± 0.050 %
10.500 13364 99.67% 1.013952 ± 0.000676 0.213597 ± 0.010750 0.007299 ± 0.000261 0.001145 19.410480 0.618763 0.013 ± 0.006 % 2.578 ± 0.049 %
10.625 13523 99.67% 1.013976 ± 0.000677 0.213963 ± 0.010763 0.007451 ± 0.000261 0.001189 20.569273 0.622427 0.019 ± 0.006 % 2.592 ± 0.049 %
10.750 13680 99.67% 1.014306 ± 0.000677 0.219003 ± 0.010782 0.007248 ± 0.000247 0.001148 19.228298 0.613274 0.011 ± 0.006 % 2.586 ± 0.050 %
10.875 13840 99.67% 1.015255 ± 0.000679 0.233540 ± 0.010856 0.007507 ± 0.000281 0.001193 20.199493 0.593870 0.017 ± 0.006 % 2.613 ± 0.050 %
11.000 13999 99.67% 1.014683 ± 0.000678 0.224779 ± 0.010816 0.007216 ± 0.000247 0.001145 17.941696 0.647005 0.016 ± 0.006 % 2.611 ± 0.051 %
11.125 14159 99.69% 1.013409 ± 0.000664 0.205278 ± 0.010543 0.007003 ± 0.000258 0.001083 18.874243 0.592008 0.017 ± 0.006 % 2.540 ± 0.048 %
11.250 14318 99.69% 1.012913 ± 0.000664 0.197686 ± 0.010520 0.006675 ± 0.000245 0.001053 19.168644 0.563273 0.015 ± 0.006 % 2.505 ± 0.050 %
11.375 14477 99.68% 1.013141 ± 0.000668 0.201168 ± 0.010595 0.006782 ± 0.000237 0.001087 18.673468 0.586665 0.015 ± 0.005 % 2.490 ± 0.048 %
11.500 14636 99.69% 1.012523 ± 0.000660 0.191714 ± 0.010454 0.006529 ± 0.000236 0.001042 18.647163 0.548347 0.019 ± 0.005 % 2.449 ± 0.049 %
11.625 14792 99.69% 1.013233 ± 0.000657 0.202580 ± 0.010438 0.006673 ± 0.000254 0.001027 18.959124 0.565285 0.015 ± 0.005 % 2.463 ± 0.051 %
11.750 14953 99.68% 1.013257 ± 0.000669 0.202953 ± 0.010613 0.007152 ± 0.000285 0.001047 18.918600 0.630295 0.018 ± 0.006 % 2.532 ± 0.051 %
11.875 15113 99.69% 1.013355 ± 0.000661 0.204458 ± 0.010493 0.006514 ± 0.000231 0.001013 19.042858 0.575434 0.011 ± 0.005 % 2.461 ± 0.048 %
12.000 15271 99.68% 1.013285 ± 0.000665 0.203384 ± 0.010551 0.006936 ± 0.000268 0.001065 21.028812 0.592060 0.013 ± 0.006 % 2.541 ± 0.051 %
12.125 15431 99.69% 1.013224 ± 0.000660 0.202447 ± 0.010488 0.006615 ± 0.000235 0.001027 19.068192 0.602703 0.024 ± 0.006 % 2.530 ± 0.051 %
12.250 15588 99.69% 1.013507 ± 0.000659 0.206774 ± 0.010476 0.006742 ± 0.000270 0.001005 19.197704 0.560796 0.016 ± 0.005 % 2.460 ± 0.049 %
12.375 15749 99.68% 1.012585 ± 0.000666 0.192658 ± 0.010549 0.006849 ± 0.000263 0.001032 19.046614 0.624302 0.023 ± 0.006 % 2.511 ± 0.050 %
12.500 15909 99.69% 1.013416 ± 0.000656 0.205388 ± 0.010426 0.006976 ± 0.000341 0.000996 30.070782 0.553184 0.015 ± 0.005 % 2.453 ± 0.051 %
12.625 16068 99.69% 1.013417 ± 0.000664 0.205405 ± 0.010551 0.006847 ± 0.000255 0.001034 18.872683 0.592905 0.021 ± 0.006 % 2.511 ± 0.051 %
12.750 16225 99.69% 1.013988 ± 0.000661 0.214149 ± 0.010524 0.006546 ± 0.000255 0.000997 19.983679 0.570902 0.020 ± 0.005 % 2.420 ± 0.047 %
12.875 16384 99.69% 1.012895 ± 0.000656 0.197416 ± 0.010405 0.006727 ± 0.000276 0.000977 20.882362 0.548220 0.020 ± 0.005 % 2.481 ± 0.052 %
13.000 16544 99.68% 1.012696 ± 0.000668 0.194360 ± 0.010572 0.006706 ± 0.000273 0.000997 21.368870 0.577613 0.015 ± 0.005 % 2.482 ± 0.051 %
13.125 16696 99.69% 1.012827 ± 0.000655 0.196367 ± 0.010396 0.006580 ± 0.000276 0.000961 19.636349 0.539285 0.022 ± 0.005 % 2.418 ± 0.049 %
13.250 16862 99.70% 1.013051 ± 0.000652 0.199793 ± 0.010356 0.006516 ± 0.000245 0.001011 18.378595 0.584926 0.018 ± 0.005 % 2.416 ± 0.049 %
13.375 17022 99.70% 1.012315 ± 0.000650 0.188526 ± 0.010298 0.006461 ± 0.000271 0.000967 19.541945 0.532116 0.014 ± 0.005 % 2.427 ± 0.050 %
13.500 17180 99.69% 1.012152 ± 0.000661 0.186035 ± 0.010446 0.006609 ± 0.000256 0.001001 19.819275 0.579343 0.012 ± 0.005 % 2.414 ± 0.050 %
13.625 17340 99.69% 1.012344 ± 0.000663 0.188980 ± 0.010478 0.006545 ± 0.000266 0.000967 21.722115 0.581532 0.014 ± 0.005 % 2.406 ± 0.049 %
13.750 17492 99.69% 1.012725 ± 0.000656 0.194806 ± 0.010399 0.006602 ± 0.000311 0.000947 31.362030 0.558576 0.016 ± 0.005 % 2.458 ± 0.052 %
13.875 17657 99.69% 1.012659 ± 0.000654 0.193798 ± 0.010372 0.006449 ± 0.000240 0.000980 18.928963 0.584710 0.017 ± 0.005 % 2.418 ± 0.049 %
14.000 17813 99.70% 1.012146 ± 0.000651 0.185948 ± 0.010307 0.006304 ± 0.000235 0.000948 18.758667 0.537241 0.018 ± 0.005 % 2.407 ± 0.050 %
14.125 17975 99.68% 1.013601 ± 0.000667 0.208224 ± 0.010593 0.006542 ± 0.000246 0.000976 21.416483 0.633320 0.009 ± 0.005 % 2.474 ± 0.053 %
14.250 18134 99.69% 1.012774 ± 0.000655 0.195555 ± 0.010383 0.006236 ± 0.000247 0.000934 18.788988 0.538596 0.012 ± 0.005 % 2.376 ± 0.048 %
14.375 18288 99.70% 1.012470 ± 0.000649 0.190906 ± 0.010282 0.006284 ± 0.000296 0.000906 31.802967 0.547245 0.022 ± 0.005 % 2.411 ± 0.052 %
14.500 18453 99.70% 1.012954 ± 0.000653 0.198314 ± 0.010367 0.006192 ± 0.000230 0.000945 18.992279 0.574698 0.018 ± 0.005 % 2.371 ± 0.047 %
14.625 18609 99.70% 1.013029 ± 0.000652 0.199464 ± 0.010350 0.006264 ± 0.000297 0.000903 29.076559 0.532940 0.017 ± 0.005 % 2.400 ± 0.052 %
14.750 18771 99.70% 1.012923 ± 0.000650 0.197831 ± 0.010328 0.006385 ± 0.000257 0.000951 19.178585 0.540540 0.023 ± 0.005 % 2.390 ± 0.050 %
14.875 18929 99.69% 1.012335 ± 0.000657 0.188844 ± 0.010402 0.006149 ± 0.000233 0.000920 17.733660 0.523330 0.012 ± 0.005 % 2.344 ± 0.047 %
15.000 19084 99.70% 1.012698 ± 0.000649 0.194390 ± 0.010300 0.006208 ± 0.000292 0.000891 29.146189 0.530963 0.019 ± 0.005 % 2.389 ± 0.051 %
15.125 19247 99.69% 1.012992 ± 0.000655 0.198896 ± 0.010407 0.006141 ± 0.000245 0.000910 19.324158 0.547946 0.019 ± 0.005 % 2.343 ± 0.048 %
15.250 19406 99.70% 1.013381 ± 0.000646 0.204855 ± 0.010286 0.006350 ± 0.000257 0.000887 17.933092 0.592217 0.023 ± 0.005 % 2.422 ± 0.051 %
15.375 19564 99.70% 1.012699 ± 0.000650 0.194414 ± 0.010315 0.006196 ± 0.000285 0.000861 24.918396 0.519731 0.024 ± 0.005 % 2.337 ± 0.051 %
15.500 19719 99.70% 1.012664 ± 0.000652 0.193868 ± 0.010342 0.005820 ± 0.000237 0.000838 18.379749 0.492615 0.028 ± 0.005 % 2.299 ± 0.050 %
15.625 19884 99.70% 1.013846 ± 0.000651 0.211964 ± 0.010386 0.006172 ± 0.000278 0.000867 26.033194 0.505156 0.023 ± 0.005 % 2.343 ± 0.050 %
15.750 20039 99.70% 1.013008 ± 0.000651 0.199144 ± 0.010352 0.005814 ± 0.000238 0.000836 18.426023 0.492568 0.023 ± 0.005 % 2.297 ± 0.050 %
15.875 20202 99.70% 1.013150 ± 0.000650 0.201306 ± 0.010331 0.005794 ± 0.000236 0.000824 18.269939 0.495334 0.023 ± 0.005 % 2.287 ± 0.050 %
16.000 20354 99.71% 1.011306 ± 0.000639 0.173086 ± 0.010084 0.005484 ± 0.000244 0.000750 18.494905 0.477412 0.018 ± 0.005 % 2.187 ± 0.049 %
Downloads last month
36,690
GGUF
Model size
11B params
Architecture
deepseek2
Hardware compatibility
Log In to add your hardware

We're not able to determine the quantization variants.

Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for ENOSYS/GigaChat3.1-10B-A1.8B-750-v1-GGUF

Quantized
(4)
this model

Dataset used to train ENOSYS/GigaChat3.1-10B-A1.8B-750-v1-GGUF