Add files using upload-large-folder tool

README.md

@@ -0,0 +1,109 @@
+---
+license: cc-by-nc-4.0
+tags:
+  - larql
+  - vindex
+  - mechanistic-interpretability
+  - feature-extraction
+model_name: Llama 3.1-8B
+base_model: meta-llama/Llama-3.1-8B
+---
+
+# Llama 3.1-8B — LarQL Vindex
+
+**Source model**: [meta-llama/Llama-3.1-8B](https://huggingface.co/meta-llama/Llama-3.1-8B)
+**Vindex short ID**: `c39fad08`
+**Layers**: 32  **Hidden size**: 4096  **Features per layer**: 128
+
+## What This Is
+
+A **LarQL vindex** (vector index) — a compact binary representation of the feature geometry of `meta-llama/Llama-3.1-8B`. It contains the top-128 SVD directions of every MLP gate_proj and down_proj matrix in the network, plus token embeddings, layer norms, and vocabulary projection metadata.
+
+## What This Is NOT
+
+This is **not** a model you can run for inference. It has no weights sufficient to generate text. It is a mechanistic interpretability artifact: a feature database for probing, editing, and comparing what `meta-llama/Llama-3.1-8B` has learned.
+
+## Universal Constants (Phase 2 Measurements)
+
+Measured via forward-pass hooks on a 256-token factual probe text.
+
+| Constant | Symbol | Value | Interpretation |
+|----------|--------|-------|----------------|
+| FFN Sparsity | C1 | 0.387 | Fraction of near-zero SwiGLU activations |
+| Top-8 Prob Mass | C2 | 0.491 | Probability mass on top-8 output tokens |
+| Gate Coherence | C3 | 0.808 | Mean cosine sim of adjacent gate_proj directions |
+| Layer Temperature | C4 | 0.012 | Mean per-neuron SwiGLU activation variance |
+| Circuit Stages | C5 | 2 | CKA transition count + 1 |
+
+**Notes**: Base (non-instruct) model — C2=0.491 reflects flat continuation distribution, not constrained prediction. C4=0.012 is significantly below the Gemma/Ministral range (0.036–0.042), tentatively a Llama family signature.
+
+## Gate 3 Status (DELETE Patch Test)
+
+PENDING — forward-pass ΔW achieves only 1.3% Paris suppression; MLP compensation trap confirmed. Full multi-layer LarQL service required.
+
+Gate 3 tests whether a rank-1 ΔW patch to `gate_proj.weight` at the top Paris→capital feature layer suppresses P(Paris) by ≥70% with ≤30% Berlin collateral damage.
+
+## Files
+
+| File | Description |
+|------|-------------|
+| `gate_vectors.bin` | Top-128 SVD directions of gate_proj per layer \[L×F×H, f16\] |
+| `down_features.bin` | Top-128 SVD directions of down_proj per layer \[L×F×H, f16\] |
+| `embeddings.bin` | Token embedding matrix \[V×H, f16\] |
+| `norms.bin` | Layer norm weight vectors |
+| `down_meta.bin` | Per-feature top-k vocabulary projections |
+| `index.json` | Vindex metadata (layers, hidden_size, num_feats, etc.) |
+| `manifest.json` | Build provenance (source SHA, extraction timestamp) |
+| `SHA256SUMS` | File integrity checksums |
+
+## How to Use
+
+```python
+import numpy as np, json
+
+vindex_dir = "path/to/downloaded/vindex"
+
+with open(f"{vindex_dir}/index.json") as f:
+    idx = json.load(f)
+
+L, F, H = idx["num_layers"], idx["num_feats"], idx["hidden_size"]
+V = idx["vocab_size"]
+
+# Load gate feature directions [L, F, H]
+gate = np.frombuffer(
+    open(f"{vindex_dir}/gate_vectors.bin", "rb").read(),
+    dtype=np.float16
+).reshape(L, F, H).astype(np.float32)
+
+# Load embeddings [V, H]
+emb = np.frombuffer(
+    open(f"{vindex_dir}/embeddings.bin", "rb").read(),
+    dtype=np.float16
+).reshape(V, H).astype(np.float32)
+
+# Score a token against all features (cosine similarity)
+emb_n = emb / (np.linalg.norm(emb, axis=1, keepdims=True) + 1e-8)
+gate_n = gate / (np.linalg.norm(gate, axis=2, keepdims=True) + 1e-8)
+
+token_id = 12379  # e.g., " Paris"
+scores = gate_n @ emb_n[token_id]  # [L, F]
+l_max, f_max = np.unravel_index(scores.argmax(), scores.shape)
+print(f"Top feature: layer={l_max}, feature={f_max}, score={scores[l_max, f_max]:.4f}")
+```
+
+## License
+
+CC-BY-NC 4.0 — same terms as the source model. Research use only.
+
+## Citation
+
+If you use this vindex in published work, please cite:
+
+```
+@misc{divinci2026larql,
+  title  = {LarQL Vindex: Llama 3.1-8B},
+  author = {Divinci AI},
+  year   = {2026},
+  url    = {https://huggingface.co/Divinci-AI/llama-3.1-8b-vindex}
+}
+```

SHA256SUMS

@@ -0,0 +1,10 @@
+a813b9cd1938558ba3d0fe59b0c76d993d67e61430864b5c90f428914073fc01  README.md
+8981aed9ef5ae30baea31a9a55e95a0de5ce2a59e7e2b148f78e44c5a52f3941  down_features.bin
+42281f61eb844669dd6b6ab9544a07698c86d2910a70e3ec0b6cc35f278535d7  down_meta.bin
+f1f60979ed67d6dea5906bcd5939cf043a595a7bd5dc82572d7741591afc2d08  embeddings.bin
+62deb5e6420dfcc7d49828017189bc3beac1e920cc3d4460420f120cb7a06780  gate3_results.json
+098f1acd53c23f7dc37768823ffba66dbae560f2e724c754ea54caa88c2742d5  gate_vectors.bin
+8d092ec2c2cc5a63b8273be1725774baf7f14223f9a47bd8d68c647484056659  index.json
+9139d50f3fd3b07159496f95f4193f8029d4a254b4f31d25588100f1c22bee4e  manifest.json
+7da1dd5262fe1805f355ec5454e4d8dc650adfbb96a8fe026b3229913eeb56dc  norms.bin
+fcb9cba3c23f8612eb7237aa6d3e331a87b252adcb178310bd8fa5bb23a129a8  router_weights.bin

down_features.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8981aed9ef5ae30baea31a9a55e95a0de5ce2a59e7e2b148f78e44c5a52f3941
+size 33554432

down_meta.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:42281f61eb844669dd6b6ab9544a07698c86d2910a70e3ec0b6cc35f278535d7
+size 360592

embeddings.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f1f60979ed67d6dea5906bcd5939cf043a595a7bd5dc82572d7741591afc2d08
+size 1050673152

gate3_results.json

@@ -0,0 +1,15 @@
+{
+  "model": "llama31",
+  "vindex_dir": "/home/ubuntu/vindex/llama31-8b/vindex",
+  "paris_layer": 9,
+  "paris_feature": 79,
+  "paris_score_pre": 0.0679,
+  "paris_score_post": 0.0546,
+  "paris_drop_pct": 19.6,
+  "berlin_score_pre": 0.0671,
+  "berlin_score_post": 0.0671,
+  "berlin_drop_pct": 0.0,
+  "collateral_ratio": 0.0,
+  "verdict": "FAIL",
+  "reason": "Paris suppression too weak (19.6% < 70% required)"
+}

gate_vectors.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:098f1acd53c23f7dc37768823ffba66dbae560f2e724c754ea54caa88c2742d5
+size 33554432

index.json

@@ -0,0 +1,311 @@
+{
+  "version": 1,
+  "model": "meta-llama/Llama-3.1-8B",
+  "family": "llama",
+  "source": {
+    "huggingface_repo": "/home/ubuntu/hf/hub/models--meta-llama--Llama-3.1-8B/snapshots/d04e592bb4f6aa9cfee91e2e20afa771667e1d4b",
+    "huggingface_revision": null,
+    "safetensors_sha256": null,
+    "extracted_at": "2026-04-21T09:31:31.631308Z",
+    "larql_version": "0.2.0-python"
+  },
+  "num_layers": 32,
+  "hidden_size": 4096,
+  "intermediate_size": 14336,
+  "vocab_size": 128256,
+  "embed_scale": 1.0,
+  "extract_level": "features",
+  "dtype": "f16",
+  "has_model_weights": false,
+  "down_top_k": 10,
+  "layer_bands": {
+    "syntax": [
+      0,
+      8
+    ],
+    "knowledge": [
+      8,
+      24
+    ],
+    "output": [
+      24,
+      31
+    ]
+  },
+  "layers": [
+    {
+      "layer": 0,
+      "num_features": 128,
+      "offset": 0,
+      "length": 1048576,
+      "num_experts": null,
+      "num_features_per_expert": null
+    },
+    {
+      "layer": 1,
+      "num_features": 128,
+      "offset": 1048576,
+      "length": 1048576,
+      "num_experts": null,
+      "num_features_per_expert": null
+    },
+    {
+      "layer": 2,
+      "num_features": 128,
+      "offset": 2097152,
+      "length": 1048576,
+      "num_experts": null,
+      "num_features_per_expert": null
+    },
+    {
+      "layer": 3,
+      "num_features": 128,
+      "offset": 3145728,
+      "length": 1048576,
+      "num_experts": null,
+      "num_features_per_expert": null
+    },
+    {
+      "layer": 4,
+      "num_features": 128,
+      "offset": 4194304,
+      "length": 1048576,
+      "num_experts": null,
+      "num_features_per_expert": null
+    },
+    {
+      "layer": 5,
+      "num_features": 128,
+      "offset": 5242880,
+      "length": 1048576,
+      "num_experts": null,
+      "num_features_per_expert": null
+    },
+    {
+      "layer": 6,
+      "num_features": 128,
+      "offset": 6291456,
+      "length": 1048576,
+      "num_experts": null,
+      "num_features_per_expert": null
+    },
+    {
+      "layer": 7,
+      "num_features": 128,
+      "offset": 7340032,
+      "length": 1048576,
+      "num_experts": null,
+      "num_features_per_expert": null
+    },
+    {
+      "layer": 8,
+      "num_features": 128,
+      "offset": 8388608,
+      "length": 1048576,
+      "num_experts": null,
+      "num_features_per_expert": null
+    },
+    {
+      "layer": 9,
+      "num_features": 128,
+      "offset": 9437184,
+      "length": 1048576,
+      "num_experts": null,
+      "num_features_per_expert": null
+    },
+    {
+      "layer": 10,
+      "num_features": 128,
+      "offset": 10485760,
+      "length": 1048576,
+      "num_experts": null,
+      "num_features_per_expert": null
+    },
+    {
+      "layer": 11,
+      "num_features": 128,
+      "offset": 11534336,
+      "length": 1048576,
+      "num_experts": null,
+      "num_features_per_expert": null
+    },
+    {
+      "layer": 12,
+      "num_features": 128,
+      "offset": 12582912,
+      "length": 1048576,
+      "num_experts": null,
+      "num_features_per_expert": null
+    },
+    {
+      "layer": 13,
+      "num_features": 128,
+      "offset": 13631488,
+      "length": 1048576,
+      "num_experts": null,
+      "num_features_per_expert": null
+    },
+    {
+      "layer": 14,
+      "num_features": 128,
+      "offset": 14680064,
+      "length": 1048576,
+      "num_experts": null,
+      "num_features_per_expert": null
+    },
+    {
+      "layer": 15,
+      "num_features": 128,
+      "offset": 15728640,
+      "length": 1048576,
+      "num_experts": null,
+      "num_features_per_expert": null
+    },
+    {
+      "layer": 16,
+      "num_features": 128,
+      "offset": 16777216,
+      "length": 1048576,
+      "num_experts": null,
+      "num_features_per_expert": null
+    },
+    {
+      "layer": 17,
+      "num_features": 128,
+      "offset": 17825792,
+      "length": 1048576,
+      "num_experts": null,
+      "num_features_per_expert": null
+    },
+    {
+      "layer": 18,
+      "num_features": 128,
+      "offset": 18874368,
+      "length": 1048576,
+      "num_experts": null,
+      "num_features_per_expert": null
+    },
+    {
+      "layer": 19,
+      "num_features": 128,
+      "offset": 19922944,
+      "length": 1048576,
+      "num_experts": null,
+      "num_features_per_expert": null
+    },
+    {
+      "layer": 20,
+      "num_features": 128,
+      "offset": 20971520,
+      "length": 1048576,
+      "num_experts": null,
+      "num_features_per_expert": null
+    },
+    {
+      "layer": 21,
+      "num_features": 128,
+      "offset": 22020096,
+      "length": 1048576,
+      "num_experts": null,
+      "num_features_per_expert": null
+    },
+    {
+      "layer": 22,
+      "num_features": 128,
+      "offset": 23068672,
+      "length": 1048576,
+      "num_experts": null,
+      "num_features_per_expert": null
+    },
+    {
+      "layer": 23,
+      "num_features": 128,
+      "offset": 24117248,
+      "length": 1048576,
+      "num_experts": null,
+      "num_features_per_expert": null
+    },
+    {
+      "layer": 24,
+      "num_features": 128,
+      "offset": 25165824,
+      "length": 1048576,
+      "num_experts": null,
+      "num_features_per_expert": null
+    },
+    {
+      "layer": 25,
+      "num_features": 128,
+      "offset": 26214400,
+      "length": 1048576,
+      "num_experts": null,
+      "num_features_per_expert": null
+    },
+    {
+      "layer": 26,
+      "num_features": 128,
+      "offset": 27262976,
+      "length": 1048576,
+      "num_experts": null,
+      "num_features_per_expert": null
+    },
+    {
+      "layer": 27,
+      "num_features": 128,
+      "offset": 28311552,
+      "length": 1048576,
+      "num_experts": null,
+      "num_features_per_expert": null
+    },
+    {
+      "layer": 28,
+      "num_features": 128,
+      "offset": 29360128,
+      "length": 1048576,
+      "num_experts": null,
+      "num_features_per_expert": null
+    },
+    {
+      "layer": 29,
+      "num_features": 128,
+      "offset": 30408704,
+      "length": 1048576,
+      "num_experts": null,
+      "num_features_per_expert": null
+    },
+    {
+      "layer": 30,
+      "num_features": 128,
+      "offset": 31457280,
+      "length": 1048576,
+      "num_experts": null,
+      "num_features_per_expert": null
+    },
+    {
+      "layer": 31,
+      "num_features": 128,
+      "offset": 32505856,
+      "length": 1048576,
+      "num_experts": null,
+      "num_features_per_expert": null
+    }
+  ],
+  "model_config": {
+    "model_type": "llama",
+    "hidden_size": 4096,
+    "moe": {
+      "num_experts": 1,
+      "top_k": 1,
+      "moe_intermediate_size": 14336,
+      "aggregated_features": 128,
+      "aggregation": "router_weighted_svd"
+    }
+  },
+  "checksums": {
+    "gate_vectors.bin": "098f1acd53c23f7dc37768823ffba66dbae560f2e724c754ea54caa88c2742d5",
+    "embeddings.bin": "f1f60979ed67d6dea5906bcd5939cf043a595a7bd5dc82572d7741591afc2d08",
+    "norms.bin": "7da1dd5262fe1805f355ec5454e4d8dc650adfbb96a8fe026b3229913eeb56dc",
+    "down_features.bin": "8981aed9ef5ae30baea31a9a55e95a0de5ce2a59e7e2b148f78e44c5a52f3941",
+    "down_meta.bin": "42281f61eb844669dd6b6ab9544a07698c86d2910a70e3ec0b6cc35f278535d7"
+  }
+}

manifest.json

@@ -0,0 +1,16 @@
+{
+  "vindexSha256": "c39fad08e04da51ae78079a3f0c921f984bff335e535101d01f3843b587d45d6",
+  "shortId": "c39fad08",
+  "baseModel": "meta-llama/Llama-3.1-8B",
+  "extractLevel": "features",
+  "f16": true,
+  "totalBytes": 1118937296,
+  "files": {
+    "gate_vectors.bin": "098f1acd53c23f7dc37768823ffba66dbae560f2e724c754ea54caa88c2742d5",
+    "embeddings.bin": "f1f60979ed67d6dea5906bcd5939cf043a595a7bd5dc82572d7741591afc2d08",
+    "norms.bin": "7da1dd5262fe1805f355ec5454e4d8dc650adfbb96a8fe026b3229913eeb56dc",
+    "down_features.bin": "8981aed9ef5ae30baea31a9a55e95a0de5ce2a59e7e2b148f78e44c5a52f3941",
+    "down_meta.bin": "42281f61eb844669dd6b6ab9544a07698c86d2910a70e3ec0b6cc35f278535d7",
+    "router_weights.bin": "fcb9cba3c23f8612eb7237aa6d3e331a87b252adcb178310bd8fa5bb23a129a8"
+  }
+}

norms.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7da1dd5262fe1805f355ec5454e4d8dc650adfbb96a8fe026b3229913eeb56dc
+size 532480

router_weights.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fcb9cba3c23f8612eb7237aa6d3e331a87b252adcb178310bd8fa5bb23a129a8
+size 262208