Add files using upload-large-folder tool

README.md

@@ -0,0 +1,91 @@
+---
+license: other
+license_name: deepseek
+license_link: https://github.com/deepseek-ai/DeepSeek-V3/blob/main/LICENSE-MODEL
+base_model: deepseek-ai/DeepSeek-V4-Flash
+tags:
+- quantized
+- gptq
+- int2
+- moe
+- deepseek
+- deepseek-v4-flash
+pipeline_tag: text-generation
+---
+
+# DeepSeek-V4-Flash INT2-G64
+
+INT2 group-64 quantization of [DeepSeek-V4-Flash](https://huggingface.co/deepseek-ai/DeepSeek-V4-Flash)'s 256 routed experts. The full 284B-parameter MoE fits in 96 GB of VRAM and runs on a single GPU.
+
+**Inference code, kernels, and the full quantization pipeline live at [github.com/Infatoshi/dsv4-int2](https://github.com/Infatoshi/dsv4-int2).** This repository contains weights only — they will not load with vanilla `transformers` or `vllm`.
+
+## Numbers
+
+| | |
+|---|---|
+| Checkpoint size | **75 GB** (vs 132 GB MXFP4, 543 GB BF16) |
+| Routed-expert format | INT2 g64, FP16 scale + INT4 zero |
+| Layers | 43 expert MoE layers (one per `layer_NN.safetensors`) |
+| MMLU 0-shot, 14,042 questions, V4 chat template | **72.46%** |
+| Decode throughput, RTX PRO 6000 Blackwell | 17 tok/s eager (reference path; not perf-tuned) |
+
+The official BF16 V4-Flash-Base 5-shot MMLU is 88.7%; the gap is partly setup (0-shot vs 5-shot) and partly real quantization cost.
+
+## Format
+
+Each `layer_NN.safetensors` holds the routed experts for one MoE layer. For each of the three projections (`w1` gate, `w3` up, `w2` down):
+
+- `w_packed`: `[E=256, K_out, K_in/16]` `uint32` — 16 INT2 values per `uint32`
+- `w_scale`: `[E, K_out, K_in/G]` `float16` — per-group of `G=64` input channels
+- `w_zero_packed`: `[E, K_out, K_in/(2G)]` `int8` — INT4 zero-points, two-per-byte
+
+Non-expert weights (MLA, embeddings, norms, shared expert, indexer, compressor, head) are NOT in this checkpoint — pull them from the upstream [DeepSeek-V4-Flash](https://huggingface.co/deepseek-ai/DeepSeek-V4-Flash) MXFP4 release. The hybrid loader in the GitHub repo does this automatically.
+
+`quant_stats.json` records per-layer GPTQ reconstruction error and routing-coverage stats (RTN-fallback count, visit min/max/median per expert).
+
+## Method
+
+Standard GPTQ with INT2 g64, run per-expert. Calibration uses Mistral-7B-v0.1 layer-16 hidden states as the proxy distribution — chosen for portability rather than parity with V4. Two implications worth knowing before quoting these numbers:
+
+- Across 41 layers, 211 of 256 routed experts received zero calibration tokens (V4's HC-sinkhorn routing is highly domain-specific and Mistral natural-text activations don't reach all experts). Under-covered experts fall back to per-channel RTN.
+- V4 self-calibration would close this; it is not run here. See `quant/v4_self_calib.py` in the GitHub repo for a starting point.
+
+## Loading
+
+This is research code; there is no `from_pretrained` path. To run inference:
+
+```bash
+git clone https://github.com/Infatoshi/dsv4-int2
+cd dsv4-int2
+uv venv && uv sync
+
+# point the loader at this checkpoint + the upstream V4-Flash release
+export DSV4_REF=/path/to/DeepSeek-V4-Flash             # MXFP4 release (tokenizer + non-expert weights)
+export DSV4_INT2=/path/to/this/checkpoint              # this directory
+
+PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True \
+  uv run python eval/v4_int2/repl.py
+```
+
+## Limitations
+
+- **Quantization-only.** This is a quant + reference inference path, not a perf-tuned serving stack. Decode hits ~26% of HBM peak.
+- **Custom kernel required.** Cannot be loaded with stock transformers or vLLM. Triton kernels in the GitHub repo handle dequantization on-the-fly.
+- **Calibration coverage gap.** 211/256 experts per layer get zero calibration visits under our setup. Rare-domain quality may be worse than the headline MMLU suggests.
+- **Single-GPU only.** Loader assumes `world_size=1`. No tensor parallelism.
+- **Hardware tested:** RTX PRO 6000 Blackwell SM_120 (96 GB). Other architectures should work via Triton autotune but have not been measured.
+
+## License
+
+Source code on GitHub is MIT. These weights are derivatives of DeepSeek-V4-Flash and inherit the [DeepSeek Model License](https://github.com/deepseek-ai/DeepSeek-V3/blob/main/LICENSE-MODEL).
+
+## Citation
+
+```bibtex
+@misc{dsv4int2,
+  title  = {dsv4-int2: INT2 quantization of DeepSeek-V4-Flash for single-GPU inference},
+  author = {Arledge, Elliot},
+  year   = {2026},
+  url    = {https://github.com/Infatoshi/dsv4-int2}
+}
+```

layer_00.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:65efc8d5129ef6ae89dc74121cd00188aa1664b988dee6bc0d3ff9f8cdbffe7e
+size 1862526984

layer_01.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9267c67376da701e28475a87648c77b4bacd3fda74255d2a5ea46dbb49e813eb
+size 1862526984

layer_02.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fc5fd16ad3b6bc8b79ddf899144067c8937808545b77e3b4b1715e3236dd1c59
+size 1862526984

layer_03.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fd78f5672a751e206167daafba1ecbb5bd8e840730728e7b8ac985fa76a1881c
+size 1862526984

layer_04.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f317860d0cb9ee67cffeab4249a4068f895a5f5a4560808244d83efc2c39eedc
+size 1862526984

layer_05.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:64d7106f4a2a9f6fec4e649615fef52a42ef1aae431c011c681960de699985c0
+size 1862526984

layer_06.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:910f534cda33349d58c38ffbe4f79d164e45307c34498a00aa5b572774de99c8
+size 1862526984

layer_07.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8125827cc86436bd2b99620a84baaa3e3520644316586e7df7df203e0d4a6dea
+size 1862526984

layer_08.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:93c345bafbfd60ae55cc1c9006a0b3944040a17490c11ddbf7d29b23e77e3b20
+size 1862526984

layer_09.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6883dedc618912fe33afd6e95cf1936e09a53fae9fe491aadd8db8faadd55b4b
+size 1862526984

layer_10.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a8b00b22b0851043cdf7dfece331725586551cc8efabbb5c333e20c2dba0ae47
+size 1862529288

layer_11.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f8b081b04ff241c66f8be2d14a501bdf916e7b1fb6ac8cb12ed545870602cfac
+size 1862529288

layer_12.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f84144949f4dfa73df10df6b2dc1d109c1c7b7f7557e2b05b8b32ec6da4dee5e
+size 1862529288

layer_13.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8b631cdcc85f960dfbdd5734c220502b0be09cd645add621047888a91d6f8979
+size 1862529288

layer_14.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e5dc0ac9805d325d75ac33415862d3a3936986dc0512fdfa662e8efbdfed67e0
+size 1862529288

layer_15.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0af0cb88f440d526cc482217c19d965e9e254796c9a2c21de2172d0612a6956c
+size 1862529288

layer_16.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:75b611c5527851c5d955cf7295fec5e54d180f29b3f4a790a95618b8fe0edc30
+size 1862529288

layer_17.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d87e402a5532cdf18eac78c455fd0f093abe332524eec133124d822e20132a06
+size 1862529288

layer_18.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c8440b33b5561e8b0af982d9ba5a308003296641eaeae62e2b2a490e230bc98c
+size 1862529288

layer_19.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2e985cce134eb7917031e1de5f549f6ba2807ea1537ed1ee74d13d5a65373559
+size 1862529288

layer_20.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:aa39c0b722c3b649c147c9697fd60b5eab0aa9a89335db3bff0836670827eb06
+size 1862529288

layer_21.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:afbd21e08c6774ed7ee8e70680eb9353c1911c331f3e4e51bd5531d746c69ee1
+size 1862529288

layer_22.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:abe5873072c47810680843420293aee5042d510c1374ccba117c2ded69d6fe0e
+size 1862529288

layer_23.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:339dcd50418dbdcfbfcb96ae6e3df9e6cff8e31f10e57776c7d0dc9c57952c48
+size 1862529288

layer_24.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:dd050087812249fe67728900363ca996567f38e5b839efe2e4c4360d032b3760
+size 1862529288

layer_25.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b0263720eeb5412cc1dd08b6cbad9dce4cf6c1411f54e8594499d976b3a224e3
+size 1862529288

layer_26.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5207982d43c48e27ce526ae49c2fd77683c7897f60a87699dce0e9ae7960706f
+size 1862529288

layer_27.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:432712b3aed0a3d7579b97d9b06a979ce498b7a66c7086a8b8a85c806c21539f
+size 1862529288

layer_28.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f7a3378c5f1ab0abd955c5839a818195014000f6b71d3e90d02efe5ca4971172
+size 1862529288

layer_29.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b2a833193d2976d66b3cfc05d8214b6ed87abd77553c21a15c0bf9bb0ba927eb
+size 1862529288

layer_30.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:16eccf1962a09cf88358a2c62fdba802b50c4774983a15182220e370dc4c69ee
+size 1862529288

layer_31.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fbaf165518c85fa1e1084b56db2ff155db5f3d8197ba14819b9dafc2b35c3238
+size 1862529288

layer_32.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5d767d7931f03f7340b51528b64e860632a7b1133a86e77328b3e57caf717b28
+size 1862529288

layer_33.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:627ce332da629a064160dc63187bdccf0f8a9c11fd4685e98415a12d76158210
+size 1862529288

layer_34.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:65de880b52ac4dee7f578fe8aa8e24972c27d69a6218e3c4171e1e634dd505df
+size 1862529288

layer_35.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e6f493eedc1751a84c59a55f8197b929b90b61c4f88f9fde27bfe2fadfaddb2c
+size 1862529288

layer_36.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:cc440e6d277c7e5a25a1c9125a57a9ec9c1e4b5bf5f7689d6536541caff344e4
+size 1862529288

layer_37.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:266640e7a0a846772620c3f886d7e4ca1dd4bab5a308ad2700b4b1517d670d5f
+size 1862529288

layer_38.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5bf4b675b0acdbdc2156315119fd3d6f859492e55fbd4a3783e2c286529cdcaa
+size 1862529288

layer_39.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:883076068bf168099eeae703e0adf59836e59a3f36dc66ed1dbf289ff6292216
+size 1862529288

layer_40.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ced81e608d8c109620d97ca6c6532042202faef1289bec97923be33fb3211e6c
+size 1862529288

layer_41.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6af4fb89fb72e1c72f584b53ec38132c3fff73c6a62b379c27bc19c6e09ba220
+size 1862529288

layer_42.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1a06a67cf899d3a954a562c86d05c4323ea3534d43f0e58de66f05240428dcc7
+size 1862529288

quant_stats.json

@@ -0,0 +1,561 @@
+[
+  {
+    "layer": 0,
+    "n_experts": 256,
+    "n_rtn_fallback": 20,
+    "visit_min": 0,
+    "visit_max": 104701,
+    "visit_median": 1554,
+    "rec_err_mean": {
+      "w1": 0.012415080978826154,
+      "w2": 0.01343309399089776,
+      "w3": 0.01253566544255591
+    }
+  },
+  {
+    "layer": 1,
+    "n_experts": 256,
+    "n_rtn_fallback": 13,
+    "visit_min": 0,
+    "visit_max": 90933,
+    "visit_median": 2058,
+    "rec_err_mean": {
+      "w1": 0.012326016072620405,
+      "w2": 0.013364352460484952,
+      "w3": 0.012376106271403842
+    }
+  },
+  {
+    "layer": 2,
+    "n_experts": 256,
+    "n_rtn_fallback": 14,
+    "visit_min": 0,
+    "visit_max": 89672,
+    "visit_median": 1993,
+    "rec_err_mean": {
+      "w1": 0.012438499546988169,
+      "w2": 0.013484358845744282,
+      "w3": 0.012577164387039375
+    }
+  },
+  {
+    "layer": 3,
+    "n_experts": 256,
+    "n_rtn_fallback": 28,
+    "visit_min": 0,
+    "visit_max": 87656,
+    "visit_median": 1593,
+    "rec_err_mean": {
+      "w1": 0.012284352793358266,
+      "w2": 0.013022524533880642,
+      "w3": 0.01228655110753607
+    }
+  },
+  {
+    "layer": 4,
+    "n_experts": 256,
+    "n_rtn_fallback": 22,
+    "visit_min": 0,
+    "visit_max": 91759,
+    "visit_median": 2509,
+    "rec_err_mean": {
+      "w1": 0.012322144266363466,
+      "w2": 0.013148894046025816,
+      "w3": 0.012335499126493232
+    }
+  },
+  {
+    "layer": 5,
+    "n_experts": 256,
+    "n_rtn_fallback": 22,
+    "visit_min": 0,
+    "visit_max": 53067,
+    "visit_median": 2692,
+    "rec_err_mean": {
+      "w1": 0.012205896578961983,
+      "w2": 0.013076323386485456,
+      "w3": 0.012235811740538338
+    }
+  },
+  {
+    "layer": 6,
+    "n_experts": 256,
+    "n_rtn_fallback": 13,
+    "visit_min": 0,
+    "visit_max": 105315,
+    "visit_median": 2462,
+    "rec_err_mean": {
+      "w1": 0.012188850159873255,
+      "w2": 0.013173459687095601,
+      "w3": 0.012253230812348193
+    }
+  },
+  {
+    "layer": 7,
+    "n_experts": 256,
+    "n_rtn_fallback": 10,
+    "visit_min": 0,
+    "visit_max": 89430,
+    "visit_median": 2105,
+    "rec_err_mean": {
+      "w1": 0.0121999701623281,
+      "w2": 0.013163029976567486,
+      "w3": 0.012289142563531641
+    }
+  },
+  {
+    "layer": 8,
+    "n_experts": 256,
+    "n_rtn_fallback": 21,
+    "visit_min": 0,
+    "visit_max": 69468,
+    "visit_median": 2146,
+    "rec_err_mean": {
+      "w1": 0.012145253331254935,
+      "w2": 0.013199328950577183,
+      "w3": 0.012249131043063244
+    }
+  },
+  {
+    "layer": 9,
+    "n_experts": 256,
+    "n_rtn_fallback": 20,
+    "visit_min": 0,
+    "visit_max": 74772,
+    "visit_median": 1885,
+    "rec_err_mean": {
+      "w1": 0.012222615572682116,
+      "w2": 0.013086030532576842,
+      "w3": 0.012252453707333189
+    }
+  },
+  {
+    "layer": 10,
+    "n_experts": 256,
+    "n_rtn_fallback": 28,
+    "visit_min": 0,
+    "visit_max": 78211,
+    "visit_median": 2122,
+    "rec_err_mean": {
+      "w1": 0.012135391662013717,
+      "w2": 0.01296397465193877,
+      "w3": 0.012168045926955529
+    }
+  },
+  {
+    "layer": 11,
+    "n_experts": 256,
+    "n_rtn_fallback": 24,
+    "visit_min": 0,
+    "visit_max": 59807,
+    "visit_median": 2105,
+    "rec_err_mean": {
+      "w1": 0.012127576275815954,
+      "w2": 0.012990429178898921,
+      "w3": 0.012161536022176733
+    }
+  },
+  {
+    "layer": 12,
+    "n_experts": 256,
+    "n_rtn_fallback": 23,
+    "visit_min": 0,
+    "visit_max": 120073,
+    "visit_median": 1352,
+    "rec_err_mean": {
+      "w1": 0.012006504137389129,
+      "w2": 0.012896691245259717,
+      "w3": 0.012060902728990186
+    }
+  },
+  {
+    "layer": 13,
+    "n_experts": 256,
+    "n_rtn_fallback": 33,
+    "visit_min": 0,
+    "visit_max": 86203,
+    "visit_median": 1580,
+    "rec_err_mean": {
+      "w1": 0.011981866897258442,
+      "w2": 0.012806476159312297,
+      "w3": 0.012059378317644587
+    }
+  },
+  {
+    "layer": 14,
+    "n_experts": 256,
+    "n_rtn_fallback": 18,
+    "visit_min": 0,
+    "visit_max": 84296,
+    "visit_median": 2398,
+    "rec_err_mean": {
+      "w1": 0.012034156163281295,
+      "w2": 0.01298547148326179,
+      "w3": 0.012165923355496489
+    }
+  },
+  {
+    "layer": 15,
+    "n_experts": 256,
+    "n_rtn_fallback": 17,
+    "visit_min": 0,
+    "visit_max": 90098,
+    "visit_median": 1650,
+    "rec_err_mean": {
+      "w1": 0.012026469823467778,
+      "w2": 0.012949477848451352,
+      "w3": 0.012168781540822238
+    }
+  },
+  {
+    "layer": 16,
+    "n_experts": 256,
+    "n_rtn_fallback": 16,
+    "visit_min": 0,
+    "visit_max": 138818,
+    "visit_median": 2080,
+    "rec_err_mean": {
+      "w1": 0.0119970367049973,
+      "w2": 0.013099638956191484,
+      "w3": 0.012215566694067093
+    }
+  },
+  {
+    "layer": 17,
+    "n_experts": 256,
+    "n_rtn_fallback": 22,
+    "visit_min": 0,
+    "visit_max": 81805,
+    "visit_median": 1463,
+    "rec_err_mean": {
+      "w1": 0.011958844468608731,
+      "w2": 0.013121607535140356,
+      "w3": 0.01224544375872938
+    }
+  },
+  {
+    "layer": 18,
+    "n_experts": 256,
+    "n_rtn_fallback": 17,
+    "visit_min": 0,
+    "visit_max": 118432,
+    "visit_median": 1752,
+    "rec_err_mean": {
+      "w1": 0.011972683361818781,
+      "w2": 0.013337772059458075,
+      "w3": 0.01234140323867905
+    }
+  },
+  {
+    "layer": 19,
+    "n_experts": 256,
+    "n_rtn_fallback": 16,
+    "visit_min": 0,
+    "visit_max": 62625,
+    "visit_median": 2217,
+    "rec_err_mean": {
+      "w1": 0.01252702346755541,
+      "w2": 0.01334010675054742,
+      "w3": 0.012576570428791456
+    }
+  },
+  {
+    "layer": 20,
+    "n_experts": 256,
+    "n_rtn_fallback": 29,
+    "visit_min": 0,
+    "visit_max": 98240,
+    "visit_median": 1558,
+    "rec_err_mean": {
+      "w1": 0.012045820782077499,
+      "w2": 0.013375726299273083,
+      "w3": 0.01248381885670824
+    }
+  },
+  {
+    "layer": 21,
+    "n_experts": 256,
+    "n_rtn_fallback": 23,
+    "visit_min": 0,
+    "visit_max": 95130,
+    "visit_median": 1399,
+    "rec_err_mean": {
+      "w1": 0.012071445104083978,
+      "w2": 0.013229186693934025,
+      "w3": 0.012358925163425738
+    }
+  },
+  {
+    "layer": 22,
+    "n_experts": 256,
+    "n_rtn_fallback": 22,
+    "visit_min": 0,
+    "visit_max": 112438,
+    "visit_median": 1997,
+    "rec_err_mean": {
+      "w1": 0.012246504975337302,
+      "w2": 0.013522858513169922,
+      "w3": 0.01259598697288311
+    }
+  },
+  {
+    "layer": 23,
+    "n_experts": 256,
+    "n_rtn_fallback": 18,
+    "visit_min": 0,
+    "visit_max": 92691,
+    "visit_median": 2472,
+    "rec_err_mean": {
+      "w1": 0.012362111072434345,
+      "w2": 0.013373729965678649,
+      "w3": 0.012536489943158813
+    }
+  },
+  {
+    "layer": 24,
+    "n_experts": 256,
+    "n_rtn_fallback": 14,
+    "visit_min": 0,
+    "visit_max": 80941,
+    "visit_median": 1817,
+    "rec_err_mean": {
+      "w1": 0.012180447134596761,
+      "w2": 0.013405652913206723,
+      "w3": 0.012389732190058567
+    }
+  },
+  {
+    "layer": 25,
+    "n_experts": 256,
+    "n_rtn_fallback": 9,
+    "visit_min": 0,
+    "visit_max": 58896,
+    "visit_median": 2313,
+    "rec_err_mean": {
+      "w1": 0.01233544500428252,
+      "w2": 0.013383177894866094,
+      "w3": 0.01257531678857049
+    }
+  },
+  {
+    "layer": 26,
+    "n_experts": 256,
+    "n_rtn_fallback": 10,
+    "visit_min": 8,
+    "visit_max": 100000,
+    "visit_median": 2916,
+    "rec_err_mean": {
+      "w1": 0.012328778535447782,
+      "w2": 0.013479596622346435,
+      "w3": 0.012599166751897428
+    }
+  },
+  {
+    "layer": 27,
+    "n_experts": 256,
+    "n_rtn_fallback": 10,
+    "visit_min": 0,
+    "visit_max": 69160,
+    "visit_median": 1757,
+    "rec_err_mean": {
+      "w1": 0.012426368215528782,
+      "w2": 0.013210742919909535,
+      "w3": 0.012652500234253239
+    }
+  },
+  {
+    "layer": 28,
+    "n_experts": 256,
+    "n_rtn_fallback": 8,
+    "visit_min": 8,
+    "visit_max": 102322,
+    "visit_median": 2247,
+    "rec_err_mean": {
+      "w1": 0.012327002572419588,
+      "w2": 0.013446205142827239,
+      "w3": 0.012607248230779078
+    }
+  },
+  {
+    "layer": 29,
+    "n_experts": 256,
+    "n_rtn_fallback": 16,
+    "visit_min": 0,
+    "visit_max": 105865,
+    "visit_median": 1893,
+    "rec_err_mean": {
+      "w1": 0.012322824510192731,
+      "w2": 0.01330637131104595,
+      "w3": 0.012568867361551384
+    }
+  },
+  {
+    "layer": 30,
+    "n_experts": 256,
+    "n_rtn_fallback": 12,
+    "visit_min": 0,
+    "visit_max": 91393,
+    "visit_median": 2442,
+    "rec_err_mean": {
+      "w1": 0.012352064775768667,
+      "w2": 0.013372166551562259,
+      "w3": 0.012607292326720199
+    }
+  },
+  {
+    "layer": 31,
+    "n_experts": 256,
+    "n_rtn_fallback": 11,
+    "visit_min": 0,
+    "visit_max": 129256,
+    "visit_median": 2164,
+    "rec_err_mean": {
+      "w1": 0.012379363139189081,
+      "w2": 0.01339083846687572,
+      "w3": 0.012637160616577603
+    }
+  },
+  {
+    "layer": 32,
+    "n_experts": 256,
+    "n_rtn_fallback": 14,
+    "visit_min": 0,
+    "visit_max": 117053,
+    "visit_median": 1704,
+    "rec_err_mean": {
+      "w1": 0.012300585967750521,
+      "w2": 0.013404446373897372,
+      "w3": 0.01260461766651133
+    }
+  },
+  {
+    "layer": 33,
+    "n_experts": 256,
+    "n_rtn_fallback": 14,
+    "visit_min": 0,
+    "visit_max": 86566,
+    "visit_median": 1884,
+    "rec_err_mean": {
+      "w1": 0.012293105512071634,
+      "w2": 0.013338803430087864,
+      "w3": 0.012589504429342924
+    }
+  },
+  {
+    "layer": 34,
+    "n_experts": 256,
+    "n_rtn_fallback": 27,
+    "visit_min": 0,
+    "visit_max": 104265,
+    "visit_median": 1972,
+    "rec_err_mean": {
+      "w1": 0.01226013481937116,
+      "w2": 0.013365076509217033,
+      "w3": 0.012629955643205903
+    }
+  },
+  {
+    "layer": 35,
+    "n_experts": 256,
+    "n_rtn_fallback": 19,
+    "visit_min": 0,
+    "visit_max": 81482,
+    "visit_median": 1441,
+    "rec_err_mean": {
+      "w1": 0.012239336298080161,
+      "w2": 0.013319830293767154,
+      "w3": 0.012585276490426622
+    }
+  },
+  {
+    "layer": 36,
+    "n_experts": 256,
+    "n_rtn_fallback": 31,
+    "visit_min": 0,
+    "visit_max": 76229,
+    "visit_median": 1601,
+    "rec_err_mean": {
+      "w1": 0.012273650947463466,
+      "w2": 0.01333218400759506,
+      "w3": 0.012649128948396537
+    }
+  },
+  {
+    "layer": 37,
+    "n_experts": 256,
+    "n_rtn_fallback": 29,
+    "visit_min": 0,
+    "visit_max": 140116,
+    "visit_median": 2095,
+    "rec_err_mean": {
+      "w1": 0.012312455397477606,
+      "w2": 0.01347610967059154,
+      "w3": 0.012669756655668607
+    }
+  },
+  {
+    "layer": 38,
+    "n_experts": 256,
+    "n_rtn_fallback": 29,
+    "visit_min": 0,
+    "visit_max": 85568,
+    "visit_median": 1999,
+    "rec_err_mean": {
+      "w1": 0.012275624583708122,
+      "w2": 0.013469703590089921,
+      "w3": 0.012665822468989063
+    }
+  },
+  {
+    "layer": 39,
+    "n_experts": 256,
+    "n_rtn_fallback": 35,
+    "visit_min": 0,
+    "visit_max": 116615,
+    "visit_median": 1707,
+    "rec_err_mean": {
+      "w1": 0.012040700054058107,
+      "w2": 0.01377619295817567,
+      "w3": 0.01264991252537584
+    }
+  },
+  {
+    "layer": 40,
+    "n_experts": 256,
+    "n_rtn_fallback": 40,
+    "visit_min": 0,
+    "visit_max": 112269,
+    "visit_median": 1404,
+    "rec_err_mean": {
+      "w1": 0.012377002352877753,
+      "w2": 0.013884072603104869,
+      "w3": 0.012798941337678116
+    }
+  },
+  {
+    "layer": 41,
+    "n_experts": 256,
+    "n_rtn_fallback": 35,
+    "visit_min": 0,
+    "visit_max": 164712,
+    "visit_median": 2210,
+    "rec_err_mean": {
+      "w1": 0.01262030862926622,
+      "w2": 0.014570762788935099,
+      "w3": 0.012976833964785328
+    }
+  },
+  {
+    "layer": 42,
+    "n_experts": 256,
+    "n_rtn_fallback": 24,
+    "visit_min": 0,
+    "visit_max": 82726,
+    "visit_median": 1562,
+    "rec_err_mean": {
+      "w1": 0.01345529514219379,
+      "w2": 0.015836902584851487,
+      "w3": 0.013570939121564152
+    }
+  }
+]