fix: rename 4b-e2b → e2b throughout; add cross-arch results to research findings

README.md

@@ -0,0 +1,149 @@
+---
+license: cc-by-nc-4.0
+library_name: larql
+tags:
+  - vindex
+  - larql
+  - gemma4
+  - gguf
+  - mechanistic-interpretability
+  - knowledge-editing
+  - constellation-edits
+base_model: google/gemma-4-e2b-it
+---
+
+# Gemma 4 e2b — LarQL Vindex v0.2
+
+First-ever published [LarQL](https://github.com/chrishayuk/larql) vindex for Google's Gemma 4.
+
+A **vindex** is a transformer's weights decompiled into a queryable feature database — entity associations, circuit structure, and knowledge-editing surfaces exposed as APIs. No GPU required for most operations.
+
+## What this is / What this is not
+
+| ✅ What this IS | ❌ What this IS NOT |
+|----------------|-------------------|
+| A feature-space index for Gemma4-e2b-it | A language model |
+| Exposes entity associations via `/v1/walk` | `/v1/infer` does NOT produce factual completions |
+| Enables rank-1 knowledge edits (DELETE/INSERT) | Not a replacement for the base Gemma4 weights |
+| Circuit analysis (broadcast→domain→entity→prediction) | Not a general inference engine |
+
+**Critical note on `/v1/infer`:** This endpoint returns a feature-modulated projection of the host model's activations — not a coherent text-generation distribution. Output is incoherent subword tokens by design (the vindex is a feature graph, not a full transformer forward pass). For factual completion, use `google/gemma-4-e2b-it` directly. Use `/v1/walk` and `/v1/patch` for the validated operations this vindex is designed for.
+
+**Validated surfaces:** `/v1/walk` (entity-association retrieval), `/v1/describe` (feature neighborhood), `/v1/patch` DELETE/INSERT (rank-1 weight editing, Gate 3 confirmed).
+
+## Quick start
+
+```bash
+# Install LarQL (requires our fork with Gemma 4 support until upstreamed)
+git clone https://github.com/Divinci-AI/larql.git
+cd larql && cargo build --release
+
+# Set environment variables
+export LARQL_SERVICE_URL=<your_larql_cloud_run_url>
+export INTERNAL_LARQL_S2S_TOKEN=<your_s2s_token>
+
+# Query entity associations
+curl "$LARQL_SERVICE_URL/v1/walk?prompt=Paris&layers=14-27&top=10" \
+  -H "Authorization: Bearer $INTERNAL_LARQL_S2S_TOKEN"
+
+# Gate 3 repro: DELETE the Paris→capital feature then verify suppression
+curl -X POST "$LARQL_SERVICE_URL/v1/patches/apply" \
+  -H "Authorization: Bearer $INTERNAL_LARQL_S2S_TOKEN" \
+  -H "Content-Type: application/json" \
+  -d '{"name":"delete-paris-capital","patch":{"version":1,"base_model":"gemma4-e2b","created_at":"2026-04-20T00:00:00Z","operations":[{"op":"delete","entity":"Paris","relation":"capital","target":"서울","weight":1.0,"layer":27,"feature":11179}]}}'
+
+# Before: feature 11179 (gate_score=18.1) present in walk
+# After:  feature 11179 absent from walk (complete suppression confirmed)
+```
+
+## Contents
+
+| File | Size | Description |
+|------|------|-------------|
+| `gate_vectors.bin` | 1.0 GB | FFN gate matrices, per-layer variable (f16) |
+| `down_features.bin` | ~1.0 GB | Down-projection transposed [features × hidden], enables walk-mode feature retrieval |
+| `embeddings.bin` | 768 MB | Token embeddings, 262,144 × 1,536 (f16) |
+| `down_meta.bin` | 29 MB | Feature labels via vocab projection |
+| `feature_clusters.jsonl` | 4 MB | K-means clusters over gate features |
+| `relation_clusters.json` | 15 MB | Wikidata relation matching |
+| `norms.bin` | 423 KB | Per-layer normalization weights |
+| `tokenizer.json` | 11 MB | Substitute tokenizer (Qwen 2.5 — real Gemma 4 tokenizer was gated during extraction) |
+| `index.json` | 5 KB | Metadata: 35 layers, hidden=1536, variable FFN (6144 → 12288) |
+| `manifest.json` | 1.1 KB | Vindex version manifest |
+
+Total: ~2.8 GB (without full weight files)
+
+> **Note on `down_features.bin`:** Generated from `down_weights.bin` via a Python transposition step that handles Gemma 4's variable intermediate sizes per layer (L0-14: 6144, L15-34: 12288). The Rust `build_down_features` binary segfaults on variable intermediate sizes; our fix is the Python Cloud Build step in `build-larql-service.sh`. Required for walk-mode feature retrieval.
+
+## Gate 3 Validation (DELETE patch confirmed)
+
+Gate 3 test: DELETE patch on Paris → 서울 (Seoul/capital) feature at layer 27, feature 11179.
+
+| Metric | Before DELETE | After DELETE |
+|--------|--------------|-------------|
+| Feature 11179 gate_score | 18.10 | ABSENT |
+| Paris capital rank | #2 overall | Absent from top-25 |
+| Walk hits | Feature 11179 present (score 18.1) | Feature 11179 completely absent |
+
+**Walk vs dense diverge** after fix: confirms `down_features.bin` is loaded and active.
+
+```
+Before: feature=11179 score=18.10 target='서울'   ← rank #1
+After:  feature=7327  score=9.40  target='PMA'    ← 서울 COMPLETELY ABSENT
+```
+
+Gate 3 result: **PASS ✓**
+
+## Architecture details
+
+- **Architecture**: Gemma 4 dense (e2b variant)
+- **Layers**: 35 (L0-14: FFN=6144, L15-34: FFN=12288 — per-layer variable)
+- **Hidden size**: 1536
+- **Head dim**: 256
+- **Attention**: 8 Q heads, 1 KV head (GQA 8:1)
+- **Quantization source**: Q4_K GGUF
+
+## Research findings
+
+This vindex enabled the following findings (see `notebooks/PAPER_universal_constants.md` in [Divinci-AI/server](https://github.com/Divinci-AI/server)):
+
+**Five universal constants across transformer architectures:**
+1. ~12% dominant FFN sparsity (scale-invariant)
+2. Top-8 output concentration (~99.7% at each position)
+3. ~0.97 gate coherence across all layers
+4. ~0.042 layer temperature (log-activation variance)
+5. Broadcast → Domain → Entity → Prediction circuit (4-stage)
+
+**Predictive formula:** `active_experts ≈ 1/dominant_sparsity` predicts Gemma 4's top-8 MoE routing within 4% error from structural analysis alone.
+
+**Constellation Edits (knowledge editing):** Rank-1 DELETE at the TRACE-identified crown layer (L25 for geography facts) achieves FQ=1.00 in 80ms with full reversibility. Gradient ascent fails due to softmax saturation (gradient=0 at P=1.0 float32). Cross-architecture validation: Mistral-7B FQ=1.00/MU=0.88 (structural rank-1), Qwen2.5-1.5B FQ=1.00 (ROME-style k*). See `notebooks/PAPER_CONSTELLATION_EDITS_DRAFT.md`.
+
+## Important notes
+
+1. **Substitute tokenizer**: Feature labels show Qwen 2.5 tokens (151,643-vocab), not Gemma 4 tokens. Gate vectors are correct Gemma 4 weights; only the label mapping is approximate.
+
+2. **Built with patched LarQL**: 7 bug fixes required for Gemma 4 (column-major loading, Q4_K block size, variable FFN size support, etc.). See https://github.com/Divinci-AI/larql and upstream PR https://github.com/chrishayuk/larql/pull/24.
+
+3. **License**: CC-BY-NC 4.0. Academic and research use. Contact [mike@divinci.ai](mailto:mike@divinci.ai) for commercial licensing.
+
+## Citation
+
+```bibtex
+@misc{mooring2026universalconstants,
+  title={Universal Constants of Transformer Intelligence},
+  author={Mooring, Mike},
+  year={2026},
+  note={Preprint. arXiv forthcoming.}
+}
+
+@misc{mooring2026constellation,
+  title={Constellation Edits: Training-Free Knowledge Injection and Auditable Unlearning via Multi-Layer Feature Patches},
+  author={Mooring, Mike},
+  year={2026},
+  note={Preprint. arXiv forthcoming.}
+}
+```
+
+## Acknowledgments
+
+Chris Hayuk for creating LarQL. Google DeepMind for Gemma 4. Cloudflare for frontier model hosting.