move fles into pemf folder

pemf/README_PEMF.md

@@ -0,0 +1,40 @@
+# Pre‑Emergence Mechanics Framework (PEMF) — ARC‑AGI
+
+Short summary  
+The Pre‑Emergence Mechanics Framework (PEMF) frames ARC tasks as a boundary‑constrained field problem solved by minimizing irreducible residue (o) under writability gates. PEMF implements four core primitives — **Scalar Potential (+)**, **Gradient Ordering (V)**, **Residue (o)**, and **Boundary Charge (p_q)** — and composes atomic transforms (tile, shifted tile, fill_enclosed, rotate, reflect, etc.) in a beam search to drain residue and produce stable outputs.
+
+Why this matters  
+PEMF shows how ARC tasks can be solved mechanically (o‑minimization + gates) rather than by symbolic heuristics. The approach maps CTS/ITT primitives to executable operators (potential fields, gradients, Dirichlet masks, complex projections) and yields a reproducible solver recipe.
+
+Key concepts (one line each)
+- **Scalar Potential (+):** represent grid as numeric potential field (initialize_potential).  
+- **Gradient Ordering (V):** discrete gradients direct admissible edits.  
+- **Residue (o):** L1 misalignment after quantize+tile; objective to minimize.  
+- **Boundary Charge (p_q):** Dirichlet boundary mask that enforces writability gates.  
+- **Layer‑1 diagnostics:** complex projection (FFT imag component) to find latent edit zones when real signal is weak.
+
+Files and examples
+- **Skill artifacts:** `SKILLS/pre_emergence_mechanics_framework/` — howto, runnable example `references/examples/verify_pemf.py`, and README for the skill.  
+- **Postprocess logs:** `experiments/postprocess_logs.py` — coerce gate booleans and attach candidate snapshots for offline inspection.  
+- **Headless entry:** `scripts/entrypoint.py` — run experiments from CLI; `--use_wandb` flag is optional and defaults to off.
+
+Quick verification (headless)
+1. Run the PEMF example to verify primitives and a tiny compositional loop:
+```bash
+python SKILLS/pre_emergence_mechanics_framework/references/examples/verify_pemf.py
+```
+2. Run a single experiment (example):
+```bash
+python scripts/entrypoint.py --task example1 --out_dir experiments
+```
+3. Postprocess logs to attach candidate snapshot and coerce gates:
+```bash
+python experiments/postprocess_logs.py
+```
+
+Acceptance checks
+- `verify_pemf.py` prints a residue trace and reports at least one admissible edit zone from the complex projection.  
+- `experiments/*_phi_best.npy` and `experiments/*_logs.fixed.json` exist after a run and contain candidate snapshot and boolean gates for inspection.
+
+References and provenance
+This README summarizes the executable PEMF recipe derived from the ARC‑AGI exposition (PEMF / CTS / ITT). See `SKILLS/pre_emergence_mechanics_framework/references/` for runnable examples and a step‑by‑step how‑to.

pemf/arc_results/RESULTS.md

@@ -0,0 +1,34 @@
+# PEMF Solver — ARC-AGI Training Set Evaluation
+
+## Results (v4 — ITT + Predicate + DSL)
+
+| Metric | v1 | v2 | v3 | **v4** |
+|---|---|---|---|---|
+| **Tasks solved** | 31 (7.8%) | 40 (10.0%) | 47 (11.8%) | **70 (17.5%)** |
+| via ITT | — | — | 16 | **16** |
+| via Predicate | — | — | — | **25** |
+| via DSL | 31 | 40 | 31 | **29** |
+| Total time | 17s | 51s | 36s | **38s** |
+| Regressions | — | 0 | 0 | **0** |
+
+## Predicate Engine Breakdown (25 new solves)
+
+| Rule Type | Tasks | Description |
+|---|---|---|
+| neighborhood_rule | 20 | CA-style: (center_color, neighbor_signature) → output_color |
+| global_enclosed_fill | 2 | Fill all bg regions not reachable from border |
+| object predicate×action | 2 | E.g. "remove smallest object" |
+| per_object_enclosed_fill | 1 | Fill each object's interior with its color |
+
+## Architecture: 3-Pass Pipeline
+
+```
+Task → ITT Physics → Predicate Enumeration → DSL Beam Search
+         (16 tasks)      (25 tasks)            (29 tasks)
+```
+
+1. **ITT** (PhiField + σ-analysis + Fan Signatures → rule learning)
+2. **Predicate** (enclosed fill → neighborhood rules → object predicate×action)
+3. **DSL** (33 transforms + dual-strategy beam + greedy stacker)
+
+Each pass only runs if the previous one fails. Zero regression risk.

pemf/arc_results/already_solved.json

@@ -0,0 +1 @@
+["007bbfb7", "00d62c1b", "0d3d703e", "1190e5a7", "1cf80156", "1e0a9b12", "1f85a75f", "2013d3e2", "22168020", "22eb0ac0", "239be575", "23b5c85d", "28bf18c6", "2dee498d", "3618c87e", "3906de3d", "3aa6fb7a", "3af2c5a8", "3c9b0459", "42a50994", "4347f46a", "50cb2852", "6150a2bd", "62c24649", "67385a82", "67a3c6ac", "67e8384a", "68b16354", "6d0aefbc", "6f8cd79b", "6fa7a44f", "746b3537", "74dd1130", "7b7f7511", "7e0986d6", "7f4411dc", "868de0fa", "8be77c9e", "8d5021e8", "91714a58", "9172f3a0", "9565186b", "9dfd6313", "a416b8f3", "a5313dff", "a699fb00", "aabf363d", "aedd82e4", "b1948b0a", "b6afb2da", "ba97ae07", "bb43febb", "bda2d7a6", "be94b721", "c0f76784", "c59eb873", "c8f0f002", "c9e6f938", "d10ecb37", "d23f8c26", "d511f180", "d631b094", "d90796e8", "d9fac9be", "de1cd16c", "ded97339", "e26a3af2", "eb5a1d5d", "ed36ccf7", "f76d97a5"]

pemf/experiments/example1_20260428T172250Z_logs.json

@@ -0,0 +1 @@
+[[{"atomic": "<Transform tile_to_target>", "score": 98.0, "residue": 98.0, "energy": 2352.0, "gates": {"A_boundary": true, "B_localization": "True", "C_quantization": "True", "passed": "True"}, "accepted": true, "shape": [9, 9]}, {"atomic": "<Transform FillEnclosedHarmonic>", "score": 98.0, "residue": 98.0, "energy": 2352.0, "gates": {"A_boundary": true, "B_localization": "True", "C_quantization": "True", "passed": "True"}, "accepted": true, "shape": [9, 9]}, {"atomic": "<Transform Rotate_90>", "score": 98.0, "residue": 98.0, "energy": 2352.0, "gates": {"A_boundary": true, "B_localization": "True", "C_quantization": "True", "passed": "True"}, "accepted": true, "shape": [9, 9]}, {"atomic": "<Transform Reflect_h>", "score": 98.0, "residue": 98.0, "energy": 2352.0, "gates": {"A_boundary": true, "B_localization": "True", "C_quantization": "True", "passed": "True"}, "accepted": true, "shape": [9, 9]}], [{"atomic": "<Transform tile_to_target>", "score": 98.0, "residue": 98.0, "energy": 2352.0, "gates": {"A_boundary": true, "B_localization": "True", "C_quantization": "True", "passed": "True"}, "accepted": true, "shape": [9, 9]}, {"atomic": "<Transform FillEnclosedHarmonic>", "score": 98.0, "residue": 98.0, "energy": 2352.0, "gates": {"A_boundary": true, "B_localization": "True", "C_quantization": "True", "passed": "True"}, "accepted": true, "shape": [9, 9]}, {"atomic": "<Transform Rotate_90>", "score": 98.0, "residue": 98.0, "energy": 2352.0, "gates": {"A_boundary": true, "B_localization": "True", "C_quantization": "True", "passed": "True"}, "accepted": true, "shape": [9, 9]}, {"atomic": "<Transform Reflect_h>", "score": 98.0, "residue": 98.0, "energy": 2352.0, "gates": {"A_boundary": true, "B_localization": "True", "C_quantization": "True", "passed": "True"}, "accepted": true, "shape": [9, 9]}, {"atomic": "<Transform tile_to_target>", "score": 98.0, "residue": 98.0, "energy": 2352.0, "gates": {"A_boundary": true, "B_localization": "True", "C_quantization": "True", "passed": "True"}, "accepted": true, "shape": [9, 9]}, {"atomic": "<Transform FillEnclosedHarmonic>", "score": 98.0, "residue": 98.0, "energy": 2352.0, "gates": {"A_boundary": true, "B_localization": "True", "C_quantization": "True", "passed": "True"}, "accepted": true, "shape": [9, 9]}, {"atomic": "<Transform Rotate_90>", "score": 98.0, "residue": 98.0, "energy": 2352.0, "gates": {"A_boundary": true, "B_localization": "True", "C_quantization": "True", "passed": "True"}, "accepted": true, "shape": [9, 9]}, {"atomic": "<Transform Reflect_h>", "score": 98.0, "residue": 98.0, "energy": 2352.0, "gates": {"A_boundary": true, "B_localization": "True", "C_quantization": "True", "passed": "True"}, "accepted": true, "shape": [9, 9]}, {"atomic": "<Transform tile_to_target>", "score": 98.0, "residue": 98.0, "energy": 2352.0, "gates": {"A_boundary": true, "B_localization": "True", "C_quantization": "True", "passed": "True"}, "accepted": true, "shape": [9, 9]}, {"atomic": "<Transform FillEnclosedHarmonic>", "score": 98.0, "residue": 98.0, "energy": 2352.0, "gates": {"A_boundary": true, "B_localization": "True", "C_quantization": "True", "passed": "True"}, "accepted": true, "shape": [9, 9]}, {"atomic": "<Transform Rotate_90>", "score": 98.0, "residue": 98.0, "energy": 2352.0, "gates": {"A_boundary": true, "B_localization": "True", "C_quantization": "True", "passed": "True"}, "accepted": true, "shape": [9, 9]}, {"atomic": "<Transform Reflect_h>", "score": 98.0, "residue": 98.0, "energy": 2352.0, "gates": {"A_boundary": true, "B_localization": "True", "C_quantization": "True", "passed": "True"}, "accepted": true, "shape": [9, 9]}, {"atomic": "<Transform tile_to_target>", "score": 98.0, "residue": 98.0, "energy": 2352.0, "gates": {"A_boundary": true, "B_localization": "True", "C_quantization": "True", "passed": "True"}, "accepted": true, "shape": [9, 9]}, {"atomic": "<Transform FillEnclosedHarmonic>", "score": 98.0, "residue": 98.0, "energy": 2352.0, "gates": {"A_boundary": true, "B_localization": "True", "C_quantization": "True", "passed": "True"}, "accepted": true, "shape": [9, 9]}, {"atomic": "<Transform Rotate_90>", "score": 98.0, "residue": 98.0, "energy": 2352.0, "gates": {"A_boundary": true, "B_localization": "True", "C_quantization": "True", "passed": "True"}, "accepted": true, "shape": [9, 9]}, {"atomic": "<Transform Reflect_h>", "score": 98.0, "residue": 98.0, "energy": 2352.0, "gates": {"A_boundary": true, "B_localization": "True", "C_quantization": "True", "passed": "True"}, "accepted": true, "shape": [9, 9]}], [{"atomic": "<Transform tile_to_target>", "score": 98.0, "residue": 98.0, "energy": 2352.0, "gates": {"A_boundary": true, "B_localization": "True", "C_quantization": "True", "passed": "True"}, "accepted": true, "shape": [9, 9]}, {"atomic": "<Transform FillEnclosedHarmonic>", "score": 98.0, "residue": 98.0, "energy": 2352.0, "gates": {"A_boundary": true, "B_localization": "True", "C_quantization": "True", "passed": "True"}, "accepted": true, "shape": [9, 9]}, {"atomic": "<Transform Rotate_90>", "score": 98.0, "residue": 98.0, "energy": 2352.0, "gates": {"A_boundary": true, "B_localization": "True", "C_quantization": "True", "passed": "True"}, "accepted": true, "shape": [9, 9]}, {"atomic": "<Transform Reflect_h>", "score": 98.0, "residue": 98.0, "energy": 2352.0, "gates": {"A_boundary": true, "B_localization": "True", "C_quantization": "True", "passed": "True"}, "accepted": true, "shape": [9, 9]}, {"atomic": "<Transform tile_to_target>", "score": 98.0, "residue": 98.0, "energy": 2352.0, "gates": {"A_boundary": true, "B_localization": "True", "C_quantization": "True", "passed": "True"}, "accepted": true, "shape": [9, 9]}, {"atomic": "<Transform FillEnclosedHarmonic>", "score": 98.0, "residue": 98.0, "energy": 2352.0, "gates": {"A_boundary": true, "B_localization": "True", "C_quantization": "True", "passed": "True"}, "accepted": true, "shape": [9, 9]}, {"atomic": "<Transform Rotate_90>", "score": 98.0, "residue": 98.0, "energy": 2352.0, "gates": {"A_boundary": true, "B_localization": "True", "C_quantization": "True", "passed": "True"}, "accepted": true, "shape": [9, 9]}, {"atomic": "<Transform Reflect_h>", "score": 98.0, "residue": 98.0, "energy": 2352.0, "gates": {"A_boundary": true, "B_localization": "True", "C_quantization": "True", "passed": "True"}, "accepted": true, "shape": [9, 9]}, {"atomic": "<Transform tile_to_target>", "score": 98.0, "residue": 98.0, "energy": 2352.0, "gates": {"A_boundary": true, "B_localization": "True", "C_quantization": "True", "passed": "True"}, "accepted": true, "shape": [9, 9]}, {"atomic": "<Transform FillEnclosedHarmonic>", "score": 98.0, "residue": 98.0, "energy": 2352.0, "gates": {"A_boundary": true, "B_localization": "True", "C_quantization": "True", "passed": "True"}, "accepted": true, "shape": [9, 9]}, {"atomic": "<Transform Rotate_90>", "score": 98.0, "residue": 98.0, "energy": 2352.0, "gates": {"A_boundary": true, "B_localization": "True", "C_quantization": "True", "passed": "True"}, "accepted": true, "shape": [9, 9]}, {"atomic": "<Transform Reflect_h>", "score": 98.0, "residue": 98.0, "energy": 2352.0, "gates": {"A_boundary": true, "B_localization": "True", "C_quantization": "True", "passed": "True"}, "accepted": true, "shape": [9, 9]}, {"atomic": "<Transform tile_to_target>", "score": 98.0, "residue": 98.0, "energy": 2352.0, "gates": {"A_boundary": true, "B_localization": "True", "C_quantization": "True", "passed": "True"}, "accepted": true, "shape": [9, 9]}, {"atomic": "<Transform FillEnclosedHarmonic>", "score": 98.0, "residue": 98.0, "energy": 2352.0, "gates": {"A_boundary": true, "B_localization": "True", "C_quantization": "True", "passed": "True"}, "accepted": true, "shape": [9, 9]}, {"atomic": "<Transform Rotate_90>", "score": 98.0, "residue": 98.0, "energy": 2352.0, "gates": {"A_boundary": true, "B_localization": "True", "C_quantization": "True", "passed": "True"}, "accepted": true, "shape": [9, 9]}, {"atomic": "<Transform Reflect_h>", "score": 98.0, "residue": 98.0, "energy": 2352.0, "gates": {"A_boundary": true, "B_localization": "True", "C_quantization": "True", "passed": "True"}, "accepted": true, "shape": [9, 9]}, {"atomic": "<Transform tile_to_target>", "score": 98.0, "residue": 98.0, "energy": 2352.0, "gates": {"A_boundary": true, "B_localization": "True", "C_quantization": "True", "passed": "True"}, "accepted": true, "shape": [9, 9]}, {"atomic": "<Transform FillEnclosedHarmonic>", "score": 98.0, "residue": 98.0, "energy": 2352.0, "gates": {"A_boundary": true, "B_localization": "True", "C_quantization": "True", "passed": "True"}, "accepted": true, "shape": [9, 9]}, {"atomic": "<Transform Rotate_90>", "score": 98.0, "residue": 98.0, "energy": 2352.0, "gates": {"A_boundary": true, "B_localization": "True", "C_quantization": "True", "passed": "True"}, "accepted": true, "shape": [9, 9]}, {"atomic": "<Transform Reflect_h>", "score": 98.0, "residue": 98.0, "energy": 2352.0, "gates": {"A_boundary": true, "B_localization": "True", "C_quantization": "True", "passed": "True"}, "accepted": true, "shape": [9, 9]}, {"atomic": "<Transform tile_to_target>", "score": 98.0, "residue": 98.0, "energy": 2352.0, "gates": {"A_boundary": true, "B_localization": "True", "C_quantization": "True", "passed": "True"}, "accepted": true, "shape": [9, 9]}, {"atomic": "<Transform FillEnclosedHarmonic>", "score": 98.0, "residue": 98.0, "energy": 2352.0, "gates": {"A_boundary": true, "B_localization": "True", "C_quantization": "True", "passed": "True"}, "accepted": true, "shape": [9, 9]}, {"atomic": "<Transform Rotate_90>", "score": 98.0, "residue": 98.0, "energy": 2352.0, "gates": {"A_boundary": true, "B_localization": "True", "C_quantization": "True", "passed": "True"}, "accepted": true, "shape": [9, 9]}, {"atomic": "<Transform Reflect_h>", "score": 98.0, "residue": 98.0, "energy": 2352.0, "gates": {"A_boundary": true, "B_localization": "True", "C_quantization": "True", "passed": "True"}, "accepted": true, "shape": [9, 9]}]]

pemf/experiments/example1_20260428T172250Z_phi_best.npy

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:660ada98c4dfce4cdf016cac4f3432f7e589a0c758e0a74a97f5719f4972caee
+size 776

pemf/experiments/example1_20260428T172250Z_result.json

@@ -0,0 +1,23 @@
+{
+  "task_name": "example1",
+  "params": {
+    "beam_width": 6,
+    "max_depth": 3,
+    "lock_coeff": 0.0,
+    "max_fraction": 1.0,
+    "enable_layer_minus_one": true,
+    "boundary_source": "target",
+    "wandb_project": "itt_solver",
+    "wandb_anonymous": "allow"
+  },
+  "final_sigma": 98.0,
+  "sigma_trace": [
+    98.0,
+    98.0,
+    98.0,
+    98.0
+  ],
+  "time_s": 0.008741617202758789,
+  "transform": "<Transform Id\u2218tile_to_target\u2218tile_to_target\u2218tile_to_target>",
+  "states_count": 4
+}

pemf/experiments/example1_20260428T172311Z_logs.json

@@ -0,0 +1 @@
+[[{"atomic": "<Transform tile_to_target>", "score": 98.0, "residue": 98.0, "energy": 2352.0, "gates": {"A_boundary": true, "B_localization": "True", "C_quantization": "True", "passed": "True"}, "accepted": true, "shape": [9, 9]}, {"atomic": "<Transform FillEnclosedHarmonic>", "score": 98.0, "residue": 98.0, "energy": 2352.0, "gates": {"A_boundary": true, "B_localization": "True", "C_quantization": "True", "passed": "True"}, "accepted": true, "shape": [9, 9]}], [{"atomic": "<Transform tile_to_target>", "score": 98.0, "residue": 98.0, "energy": 2352.0, "gates": {"A_boundary": true, "B_localization": "True", "C_quantization": "True", "passed": "True"}, "accepted": true, "shape": [9, 9]}, {"atomic": "<Transform FillEnclosedHarmonic>", "score": 98.0, "residue": 98.0, "energy": 2352.0, "gates": {"A_boundary": true, "B_localization": "True", "C_quantization": "True", "passed": "True"}, "accepted": true, "shape": [9, 9]}, {"atomic": "<Transform tile_to_target>", "score": 98.0, "residue": 98.0, "energy": 2352.0, "gates": {"A_boundary": true, "B_localization": "True", "C_quantization": "True", "passed": "True"}, "accepted": true, "shape": [9, 9]}, {"atomic": "<Transform FillEnclosedHarmonic>", "score": 98.0, "residue": 98.0, "energy": 2352.0, "gates": {"A_boundary": true, "B_localization": "True", "C_quantization": "True", "passed": "True"}, "accepted": true, "shape": [9, 9]}]]

pemf/experiments/example1_20260428T172311Z_phi_best.npy

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:660ada98c4dfce4cdf016cac4f3432f7e589a0c758e0a74a97f5719f4972caee
+size 776

pemf/experiments/example1_20260428T172311Z_result.json

@@ -0,0 +1,21 @@
+{
+  "task_name": "example1",
+  "params": {
+    "beam_width": 4,
+    "max_depth": 2,
+    "lock_coeff": 0.0,
+    "max_fraction": 0.5,
+    "enable_layer_minus_one": true,
+    "boundary_source": "target",
+    "use_symmetry": false
+  },
+  "final_sigma": 98.0,
+  "sigma_trace": [
+    98.0,
+    98.0,
+    98.0
+  ],
+  "time_s": 0.0020961761474609375,
+  "transform": "<Transform Id\u2218tile_to_target\u2218tile_to_target>",
+  "states_count": 3
+}

pemf/experiments/results.csv

@@ -0,0 +1,5 @@
+task_name,params,final_sigma,time_s,transform,sigma_trace
+example1,"{""beam_width"": 4, ""max_depth"": 2, ""lock_coeff"": 0.0, ""max_fraction"": 0.5, ""enable_layer_minus_one"": false, ""boundary_source"": ""target"", ""use_symmetry"": true}",98.0,0.003506183624267578,<Transform Id∘tile_to_target∘tile_to_target>,"[98.0, 98.0, 98.0]"
+example1,"{""beam_width"": 4, ""max_depth"": 2, ""lock_coeff"": 0.0, ""max_fraction"": 0.5, ""enable_layer_minus_one"": false, ""boundary_source"": ""target"", ""use_symmetry"": false}",98.0,0.0017173290252685547,<Transform Id∘tile_to_target∘tile_to_target>,"[98.0, 98.0, 98.0]"
+example1,"{""beam_width"": 4, ""max_depth"": 2, ""lock_coeff"": 0.0, ""max_fraction"": 0.5, ""enable_layer_minus_one"": true, ""boundary_source"": ""target"", ""use_symmetry"": true}",98.0,0.0046575069427490234,<Transform Id∘tile_to_target∘tile_to_target>,"[98.0, 98.0, 98.0]"
+example1,"{""beam_width"": 4, ""max_depth"": 2, ""lock_coeff"": 0.0, ""max_fraction"": 0.5, ""enable_layer_minus_one"": true, ""boundary_source"": ""target"", ""use_symmetry"": false}",98.0,0.0020961761474609375,<Transform Id∘tile_to_target∘tile_to_target>,"[98.0, 98.0, 98.0]"

pemf/experiments_analysis.py

@@ -0,0 +1,154 @@
+"""
+Quick diagnostics for itt_solver experiments.
+
+Usage (from notebook or shell):
+  python experiments_analysis.py
+
+It will:
+ - list recent files in experiments/
+ - print the latest result.json
+ - print depth-0 logs (candidates, gates, residues)
+ - load the latest phi_best and compute L1 vs a provided target (if you set TARGET_GRID below)
+ - test atomic transforms from default_atomic_factory to see if they change the input
+"""
+
+import os
+import glob
+import json
+import numpy as np
+from pprint import pprint
+
+# === Corrected target from real ARC task 007bbfb7 (Kronecker self-similar) ===
+TARGET_GRID = [
+    [0,0,0,0,7,7,0,7,7],
+    [0,0,0,7,7,7,7,7,7],
+    [0,0,0,0,7,7,0,7,7],
+    [0,7,7,0,7,7,0,7,7],
+    [7,7,7,7,7,7,7,7,7],
+    [0,7,7,0,7,7,0,7,7],
+    [0,0,0,0,7,7,0,7,7],
+    [0,0,0,7,7,7,7,7,7],
+    [0,0,0,0,7,7,0,7,7],
+]
+
+EXPERIMENTS_DIR = "experiments"
+
+def list_recent_files(n=20):
+    files = sorted(glob.glob(os.path.join(EXPERIMENTS_DIR, "*")))
+    print(f"Recent files (last {n}):")
+    for f in files[-n:]:
+        print(" ", f)
+    return files
+
+def load_latest_result():
+    res_files = sorted(glob.glob(os.path.join(EXPERIMENTS_DIR, "*_result.json")))
+    if not res_files:
+        print("No result.json files found in experiments/")
+        return None, None
+    latest = res_files[-1]
+    print("\nLatest result file:", latest)
+    with open(latest) as fh:
+        data = json.load(fh)
+    pprint(data)
+    return latest, data
+
+def load_latest_logs():
+    logs_files = sorted(glob.glob(os.path.join(EXPERIMENTS_DIR, "*_logs.json")))
+    if not logs_files:
+        print("No logs.json files found in experiments/")
+        return None, None
+    latest = logs_files[-1]
+    print("\nLatest logs file:", latest)
+    with open(latest) as fh:
+        logs = json.load(fh)
+    if logs and isinstance(logs, list) and len(logs) > 0:
+        print("\nDepth 0 log entries (summary):")
+        for i, entry in enumerate(logs[0]):
+            atomic = entry.get('atomic')
+            accepted = entry.get('accepted')
+            residue = entry.get('residue')
+            energy = entry.get('energy')
+            gates = entry.get('gates')
+            print(f"{i}: {atomic} | accepted={accepted} | residue={residue} | energy={energy} | gates={gates}")
+    else:
+        print("Logs format unexpected or empty.")
+    return latest, logs
+
+def load_latest_phi():
+    phi_files = sorted(glob.glob(os.path.join(EXPERIMENTS_DIR, "*_phi_best.npy")))
+    if not phi_files:
+        print("No phi_best.npy files found in experiments/")
+        return None, None
+    latest = phi_files[-1]
+    print("\nLatest phi_best file:", latest)
+    phi = np.load(latest)
+    print("phi_best shape:", phi.shape, "unique values:", np.unique(phi))
+    return latest, phi
+
+def l1_residue_check(phi, target_grid):
+    if phi is None:
+        print("No phi provided for residue check.")
+        return
+    target = np.array(target_grid, dtype=phi.dtype)
+    if phi.shape != target.shape:
+        print("phi and target shapes differ:", phi.shape, target.shape)
+        try:
+            from itt_solver.solver_core import tile_transform
+            target_resized = tile_transform(target, phi.shape)
+            print("Resized target to phi shape for comparison.")
+        except Exception:
+            print("Could not resize target automatically.")
+            return
+    else:
+        target_resized = target
+    l1 = float(np.sum(np.abs(phi - target_resized)))
+    print("L1 residue between phi_best and target:", l1)
+    return l1
+
+def test_atomic_effects():
+    print("\nTesting atomic transforms from default_atomic_factory...")
+    try:
+        from itt_solver.experiment_driver import default_atomic_factory
+        from itt_solver.solver_core import initialize_potential, tile_transform
+    except Exception as e:
+        print("Could not import default_atomic_factory or solver_core:", e)
+        return
+    params = {'beam_width':6,'max_depth':3,'lock_coeff':0.0,'max_fraction':1.0,'enable_layer_minus_one':True,'boundary_source':'target'}
+    task_stub = {'target_shape': (9,9)}
+    atomic_library = default_atomic_factory(params, task_stub)
+    phi_in = initialize_potential([[0,7,7],[7,7,7],[0,7,7]])
+    print("Input shape:", phi_in.shape, "unique:", np.unique(phi_in))
+    for T in atomic_library:
+        try:
+            out = T.apply(phi_in.copy())
+        except Exception as e:
+            print(repr(T), "apply() raised:", e)
+            continue
+        out_resized = out
+        if out.shape != phi_in.shape:
+            try:
+                out_resized = tile_transform(out, phi_in.shape)
+            except Exception:
+                try:
+                    out_resized = np.broadcast_to(out, phi_in.shape)
+                except Exception:
+                    out_resized = None
+        if out_resized is None:
+            changed = None
+        else:
+            changed = int(np.sum(out_resized != phi_in))
+        print(repr(T), "-> out shape", out.shape, "changed cells (compared to input):", changed)
+
+def main():
+    print("=== experiments_analysis.py diagnostics ===")
+    list_recent_files()
+    load_latest_result()
+    load_latest_logs()
+    _, phi = load_latest_phi()
+    if phi is not None:
+        l1_residue_check(phi, TARGET_GRID)
+    test_atomic_effects()
+    print("\nDone.")
+
+if __name__ == "__main__":
+    main()

pemf/notebooks/pemf_llm_lightning.ipynb

@@ -0,0 +1,303 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# PEMF ARC-AGI — LLM Solver (Lightning.ai / Multi-GPU)\n",
+    "\n",
+    "Runs Ollama with auto multi-GPU sharding for local inference.\n",
+    "\n",
+    "| GPU Config | Model | VRAM | Quality |\n",
+    "|---|---|---|---|\n",
+    "| 2xA10G (48GB) | qwen2.5-coder:32b | ~20GB q4 | Best |\n",
+    "| 2xL4 (48GB) | qwen2.5-coder:32b | ~20GB q4 | Best |\n",
+    "| 2xT4 (32GB) | qwen2.5-coder:14b | ~10GB q4 | Good |\n",
+    "| 1xA10G (24GB) | qwen2.5-coder:14b | ~10GB | Good |\n",
+    "| 4xA10G (96GB) | qwen2.5-coder:32b fp16 | ~65GB | Best+fast |"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# ============ CONFIGURATION ============\n",
+    "MODEL = 'qwen2.5-coder:32b'\n",
+    "# MODEL = 'qwen2.5-coder:14b'  # fallback for less VRAM\n",
+    "N_CANDIDATES = 8"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import subprocess, os, time, json, re, glob\n",
+    "import numpy as np, urllib.request\n",
+    "from collections import Counter\n",
+    "\n",
+    "# Check GPUs\n",
+    "!nvidia-smi --query-gpu=index,name,memory.total --format=csv,noheader\n",
+    "gpu_count = len(subprocess.run(['nvidia-smi','-L'], capture_output=True, text=True).stdout.strip().split('\\n'))\n",
+    "print(f'GPUs: {gpu_count}')"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# Install Ollama\n",
+    "try:\n",
+    "    subprocess.run(['ollama','--version'], capture_output=True, check=True)\n",
+    "    print('Ollama installed')\n",
+    "except: \n",
+    "    !curl -fsSL https://ollama.com/install.sh | sh\n",
+    "\n",
+    "# Start server (auto-detects all GPUs)\n",
+    "subprocess.run(['pkill','-f','ollama'], capture_output=True)\n",
+    "time.sleep(2)\n",
+    "env = os.environ.copy()\n",
+    "env['CUDA_VISIBLE_DEVICES'] = ','.join(str(i) for i in range(gpu_count))\n",
+    "server = subprocess.Popen(['ollama','serve'],\n",
+    "    stdout=open('/tmp/ollama.log','w'), stderr=subprocess.STDOUT, env=env)\n",
+    "time.sleep(5)\n",
+    "print(f'Server PID {server.pid}, GPUs: {env[\"CUDA_VISIBLE_DEVICES\"]}')\n",
+    "\n",
+    "# Pull model\n",
+    "print(f'Pulling {MODEL}...')\n",
+    "r = subprocess.run(['ollama','pull',MODEL], capture_output=True, text=True, timeout=3600)\n",
+    "if r.returncode != 0:\n",
+    "    print(f'Failed, trying 14b...'); MODEL='qwen2.5-coder:14b'\n",
+    "    subprocess.run(['ollama','pull',MODEL], capture_output=True, text=True, timeout=3600)\n",
+    "print(f'{MODEL} ready')\n",
+    "\n",
+    "# Test\n",
+    "r = subprocess.run(['ollama','run',MODEL,'Say hello'], capture_output=True, text=True, timeout=60)\n",
+    "print(f'Test: {r.stdout.strip()[:80]}')\n",
+    "!nvidia-smi --query-gpu=index,memory.used,memory.total --format=csv,noheader"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# Download ARC data\n",
+    "if not os.path.exists('arc_data/training'):\n",
+    "    !git clone --depth 1 https://github.com/fchollet/ARC-AGI.git /tmp/arc\n",
+    "    os.makedirs('arc_data', exist_ok=True)\n",
+    "    !cp -r /tmp/arc/data/training arc_data/training\n",
+    "print(f'Tasks: {len(glob.glob(\"arc_data/training/*.json\"))}')\n",
+    "\n",
+    "ALREADY_SOLVED = {\n",
+    "    '007bbfb7','00d62c1b','0d3d703e','1190e5a7','1cf80156','1e0a9b12','1f85a75f',\n",
+    "    '2013d3e2','22168020','22eb0ac0','239be575','23b5c85d','28bf18c6','2dee498d',\n",
+    "    '3618c87e','3906de3d','3aa6fb7a','3af2c5a8','3c9b0459','42a50994','4347f46a',\n",
+    "    '50cb2852','6150a2bd','62c24649','67385a82','67a3c6ac','67e8384a','68b16354',\n",
+    "    '6d0aefbc','6f8cd79b','6fa7a44f','746b3537','74dd1130','7b7f7511','7e0986d6',\n",
+    "    '7f4411dc','868de0fa','8be77c9e','8d5021e8','91714a58','9172f3a0','9565186b',\n",
+    "    '9dfd6313','a416b8f3','a5313dff','a699fb00','aabf363d','aedd82e4','b1948b0a',\n",
+    "    'b6afb2da','ba97ae07','bb43febb','bda2d7a6','be94b721','c0f76784','c59eb873',\n",
+    "    'c8f0f002','c9e6f938','d10ecb37','d23f8c26','d511f180','d631b094','d90796e8',\n",
+    "    'd9fac9be','de1cd16c','ded97339','e26a3af2','eb5a1d5d','ed36ccf7','f76d97a5',\n",
+    "}\n",
+    "task_files = sorted(glob.glob('arc_data/training/*.json'))\n",
+    "unsolved = [(os.path.basename(f).replace('.json',''),f) for f in task_files\n",
+    "            if os.path.basename(f).replace('.json','') not in ALREADY_SOLVED]\n",
+    "print(f'Symbolic: {len(ALREADY_SOLVED)}, LLM to try: {len(unsolved)}')"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# LLM Engine\n",
+    "def call_ollama(prompt, model, temperature=0.7):\n",
+    "    payload = {'model':model,'prompt':prompt,'stream':False,\n",
+    "               'options':{'temperature':temperature,'num_predict':2048}}\n",
+    "    req = urllib.request.Request('http://localhost:11434/api/generate',\n",
+    "        data=json.dumps(payload).encode(), headers={'Content-Type':'application/json'}, method='POST')\n",
+    "    try:\n",
+    "        with urllib.request.urlopen(req, timeout=180) as resp:\n",
+    "            return json.loads(resp.read().decode()).get('response','')\n",
+    "    except Exception as e: return f'ERROR: {e}'\n",
+    "\n",
+    "def build_prompt(task):\n",
+    "    pairs = task.get('train',[])\n",
+    "    ex = '\\n'.join(f\"Example {i+1}:\\n  Input:  {json.dumps(p['input'])}\\n  Output: {json.dumps(p['output'])}\"\n",
+    "                    for i,p in enumerate(pairs))\n",
+    "    inps = [np.array(p['input']) for p in pairs]\n",
+    "    outs = [np.array(p['output']) for p in pairs]\n",
+    "    same = all(i.shape==o.shape for i,o in zip(inps,outs))\n",
+    "    ic = sorted(set(c for i in inps for c in np.unique(i).tolist()))\n",
+    "    oc = sorted(set(c for o in outs for c in np.unique(o).tolist()))\n",
+    "    a = f\"  Same shape: {same}\\n  Colors in: {ic}, out: {oc}\\n\"\n",
+    "    if not same: a += f\"  Shape: {inps[0].shape} -> {outs[0].shape}\\n\"\n",
+    "    return f\"\"\"Solve this ARC-AGI puzzle. Write ONLY a Python function, no explanations.\n",
+    "\n",
+    "{ex}\n",
+    "\n",
+    "Analysis:\n",
+    "{a}\n",
+    "```python\n",
+    "import numpy as np\n",
+    "from collections import Counter, deque\n",
+    "\n",
+    "def transform(grid: list[list[int]]) -> list[list[int]]:\n",
+    "    grid = np.array(grid)\n",
+    "\"\"\"\n",
+    "\n",
+    "def extract_code(resp):\n",
+    "    for pat in [r'```python\\s*(.*?)```', r'```\\s*(.*?)```']:\n",
+    "        for m in re.findall(pat, resp, re.DOTALL):\n",
+    "            if 'def transform' in m: return m.strip()\n",
+    "    idx = resp.find('def transform')\n",
+    "    if idx >= 0:\n",
+    "        before = resp[:idx]\n",
+    "        s = max(before.rfind('import '), before.rfind('from '))\n",
+    "        code = resp[s if s>=0 else idx:]\n",
+    "        end = code.find('```')\n",
+    "        if end>0: code=code[:end]\n",
+    "        return code.strip()\n",
+    "    s = resp.strip()\n",
+    "    if s.startswith(('import','def transform','from')): return s\n",
+    "    return None\n",
+    "\n",
+    "def verify(code, pairs):\n",
+    "    ns = {'np':np,'numpy':np,'Counter':Counter,'deque':__import__('collections').deque}\n",
+    "    try:\n",
+    "        import scipy.ndimage; ns['scipy']=__import__('scipy')\n",
+    "    except: pass\n",
+    "    try: exec(code, ns)\n",
+    "    except: return False\n",
+    "    if 'transform' not in ns: return False\n",
+    "    fn = ns['transform']\n",
+    "    for p in pairs:\n",
+    "        try:\n",
+    "            r = np.array(fn([row[:] for row in p['input']]), dtype=int)\n",
+    "            e = np.array(p['output'], dtype=int)\n",
+    "            if r.shape!=e.shape or not np.array_equal(r,e): return False\n",
+    "        except: return False\n",
+    "    return True\n",
+    "\n",
+    "def apply_prog(code, inp):\n",
+    "    ns = {'np':np,'numpy':np,'Counter':Counter,'deque':__import__('collections').deque}\n",
+    "    try:\n",
+    "        import scipy.ndimage; ns['scipy']=__import__('scipy')\n",
+    "    except: pass\n",
+    "    try:\n",
+    "        exec(code, ns)\n",
+    "        r = ns['transform']([row[:] for row in inp])\n",
+    "        if r is not None: return np.array(r,dtype=int).tolist()\n",
+    "    except: pass\n",
+    "    return None\n",
+    "\n",
+    "print('Engine ready')"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# Quick test\n",
+    "with open(f'arc_data/training/{unsolved[0][0]}.json') as f: t=json.load(f)\n",
+    "print(f'Test on {unsolved[0][0]}...')\n",
+    "s=time.time(); r=call_ollama(build_prompt(t),MODEL,0.1); e=time.time()-s\n",
+    "code=extract_code(r)\n",
+    "if code: print(f'{e:.1f}s, {len(code)}ch, verified: {\"Y\" if verify(code,t[\"train\"]) else \"N\"}')\n",
+    "else: print(f'{e:.1f}s, no code')\n",
+    "est = e*N_CANDIDATES*len(unsolved)/3600\n",
+    "print(f'Est total: {est:.1f}h for {len(unsolved)} tasks x {N_CANDIDATES} candidates')"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# === MAIN LOOP (crash-safe, resumable) ===\n",
+    "results = {}\n",
+    "solved = 0\n",
+    "total_time = 0\n",
+    "\n",
+    "if os.path.exists('llm_results.json'):\n",
+    "    with open('llm_results.json') as f: prev=json.load(f)\n",
+    "    results=prev.get('results',{})\n",
+    "    solved=sum(1 for r in results.values() if r['status']=='solved')\n",
+    "    total_time=prev.get('total_time_s',0)\n",
+    "    print(f'Resuming: {solved} LLM-solved, {len(results)} attempted')\n",
+    "\n",
+    "for idx,(tid,tf) in enumerate(unsolved):\n",
+    "    if tid in results: continue\n",
+    "    with open(tf) as f: task=json.load(f)\n",
+    "    print(f'[{idx+1:3d}/{len(unsolved)}] {tid}:',end=' ',flush=True)\n",
+    "    s=time.time(); prompt=build_prompt(task); ok=False\n",
+    "    for i in range(N_CANDIDATES):\n",
+    "        temp=0.1 if i==0 else min(0.4+0.15*i,1.2)\n",
+    "        resp=call_ollama(prompt,MODEL,temp)\n",
+    "        if resp.startswith('ERROR:'): continue\n",
+    "        code=extract_code(resp)\n",
+    "        if code and verify(code,task['train']):\n",
+    "            e=time.time()-s; total_time+=e; solved+=1\n",
+    "            to=[apply_prog(code,t['input']) for t in task.get('test',[])]\n",
+    "            results[tid]={'status':'solved','rule':f'llm_c{i+1}','code':code,\n",
+    "                         'test_outputs':to,'time_s':round(e,2)}\n",
+    "            print(f'✅ c{i+1} ({e:.1f}s) [{len(ALREADY_SOLVED)+solved}/{len(task_files)}]')\n",
+    "            ok=True; break\n",
+    "    if not ok:\n",
+    "        e=time.time()-s; total_time+=e\n",
+    "        results[tid]={'status':'failed','time_s':round(e,2)}\n",
+    "        print(f'❌ ({e:.1f}s)')\n",
+    "    if (idx+1)%5==0 or ok:\n",
+    "        with open('llm_results.json','w') as f:\n",
+    "            json.dump({'model':MODEL,'n_candidates':N_CANDIDATES,'llm_solved':solved,\n",
+    "                'attempted':len(results),'symbolic_solved':len(ALREADY_SOLVED),\n",
+    "                'total_solved':len(ALREADY_SOLVED)+solved,'total_tasks':len(task_files),\n",
+    "                'solve_rate':round(100*(len(ALREADY_SOLVED)+solved)/len(task_files),2),\n",
+    "                'total_time_s':round(total_time,1),'results':results},f,indent=2)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# Final save + summary\n",
+    "with open('llm_results.json','w') as f:\n",
+    "    json.dump({'model':MODEL,'n_candidates':N_CANDIDATES,'llm_solved':solved,\n",
+    "        'attempted':len(results),'symbolic_solved':len(ALREADY_SOLVED),\n",
+    "        'total_solved':len(ALREADY_SOLVED)+solved,'total_tasks':len(task_files),\n",
+    "        'solve_rate':round(100*(len(ALREADY_SOLVED)+solved)/len(task_files),2),\n",
+    "        'total_time_s':round(total_time,1),'results':results},f,indent=2)\n",
+    "\n",
+    "print(f'\\n{\"=\"*60}')\n",
+    "print(f'LLM solved:     {solved}')\n",
+    "print(f'Symbolic:       {len(ALREADY_SOLVED)}')\n",
+    "print(f'TOTAL:          {len(ALREADY_SOLVED)+solved}/{len(task_files)} ({100*(len(ALREADY_SOLVED)+solved)/len(task_files):.1f}%)')\n",
+    "print(f'Time:           {total_time/3600:.1f}h')\n",
+    "print(f'\\nDownload llm_results.json, then run:')\n",
+    "print(f'  python scripts/merge_results.py arc_results/summary_v4.json llm_results.json')\n",
+    "\n",
+    "subprocess.run(['pkill','-f','ollama'], capture_output=True)"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {"display_name":"Python 3","language":"python","name":"python3"},
+  "language_info": {"name":"python","version":"3.10.0"}
+ },
+ "nbformat": 4,
+ "nbformat_minor": 4
+}

pemf/notebooks/pemf_llm_solver.ipynb

@@ -0,0 +1,490 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# PEMF ARC-AGI — LLM Program Synthesis\n",
+    "\n",
+    "Uses NVIDIA NIM (free) with GLM 4.7 / DeepSeek V4 to solve ARC tasks.\n",
+    "\n",
+    "**Pipeline:** For each unsolved task → build prompt → LLM generates Python `transform()` → verify against ALL training pairs → apply to test.\n",
+    "\n",
+    "**Prerequisites:**\n",
+    "- NVIDIA NIM API key from https://build.nvidia.com/settings/api-keys\n",
+    "- Internet access enabled"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## 1. Setup"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# ============================================================\n",
+    "# CONFIGURATION — EDIT THESE\n",
+    "# ============================================================\n",
+    "\n",
+    "NVIDIA_API_KEY = \"nvapi-YOUR-KEY-HERE\"  # Get from https://build.nvidia.com/settings/api-keys\n",
+    "\n",
+    "MODEL = \"z-ai/glm4.7\"                  # Default: GLM 4.7\n",
+    "# MODEL = \"deepseek-ai/deepseek-v4-pro\"  # Alternative: DeepSeek V4\n",
+    "\n",
+    "N_CANDIDATES = 8      # Candidates per task (more = better but slower)\n",
+    "RATE_LIMIT_SLEEP = 2  # Seconds between API calls"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# Download ARC dataset\n",
+    "import os, subprocess\n",
+    "\n",
+    "if not os.path.exists('arc_data/training'):\n",
+    "    print('Downloading ARC dataset...')\n",
+    "    subprocess.run(['git', 'clone', '--depth', '1', 'https://github.com/fchollet/ARC-AGI.git', '/tmp/arc'], \n",
+    "                   capture_output=True)\n",
+    "    os.makedirs('arc_data', exist_ok=True)\n",
+    "    subprocess.run(['cp', '-r', '/tmp/arc/data/training', 'arc_data/training'], capture_output=True)\n",
+    "    print(f'Downloaded {len(os.listdir(\"arc_data/training\"))} tasks')\n",
+    "else:\n",
+    "    print(f'ARC data already present: {len(os.listdir(\"arc_data/training\"))} tasks')"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# Already solved by symbolic pipeline (70 tasks)\n",
+    "ALREADY_SOLVED = {\n",
+    "    \"007bbfb7\",\"00d62c1b\",\"0d3d703e\",\"1190e5a7\",\"1cf80156\",\"1e0a9b12\",\"1f85a75f\",\n",
+    "    \"2013d3e2\",\"22168020\",\"22eb0ac0\",\"239be575\",\"23b5c85d\",\"28bf18c6\",\"2dee498d\",\n",
+    "    \"3618c87e\",\"3906de3d\",\"3aa6fb7a\",\"3af2c5a8\",\"3c9b0459\",\"42a50994\",\"4347f46a\",\n",
+    "    \"50cb2852\",\"6150a2bd\",\"62c24649\",\"67385a82\",\"67a3c6ac\",\"67e8384a\",\"68b16354\",\n",
+    "    \"6d0aefbc\",\"6f8cd79b\",\"6fa7a44f\",\"746b3537\",\"74dd1130\",\"7b7f7511\",\"7e0986d6\",\n",
+    "    \"7f4411dc\",\"868de0fa\",\"8be77c9e\",\"8d5021e8\",\"91714a58\",\"9172f3a0\",\"9565186b\",\n",
+    "    \"9dfd6313\",\"a416b8f3\",\"a5313dff\",\"a699fb00\",\"aabf363d\",\"aedd82e4\",\"b1948b0a\",\n",
+    "    \"b6afb2da\",\"ba97ae07\",\"bb43febb\",\"bda2d7a6\",\"be94b721\",\"c0f76784\",\"c59eb873\",\n",
+    "    \"c8f0f002\",\"c9e6f938\",\"d10ecb37\",\"d23f8c26\",\"d511f180\",\"d631b094\",\"d90796e8\",\n",
+    "    \"d9fac9be\",\"de1cd16c\",\"ded97339\",\"e26a3af2\",\"eb5a1d5d\",\"ed36ccf7\",\"f76d97a5\",\n",
+    "}\n",
+    "print(f'Already solved by symbolic pipeline: {len(ALREADY_SOLVED)} tasks')"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## 2. LLM Engine"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import json\n",
+    "import time\n",
+    "import re\n",
+    "import glob\n",
+    "import numpy as np\n",
+    "import urllib.request\n",
+    "from collections import Counter\n",
+    "\n",
+    "\n",
+    "def call_nvidia(prompt, api_key, model=\"z-ai/glm4.7\", temperature=0.7):\n",
+    "    \"\"\"Call NVIDIA NIM API.\"\"\"\n",
+    "    url = \"https://integrate.api.nvidia.com/v1/chat/completions\"\n",
+    "    payload = {\n",
+    "        \"model\": model,\n",
+    "        \"messages\": [{\"role\": \"user\", \"content\": prompt}],\n",
+    "        \"max_tokens\": 2048,\n",
+    "        \"temperature\": temperature,\n",
+    "    }\n",
+    "    data = json.dumps(payload).encode('utf-8')\n",
+    "    req = urllib.request.Request(url, data=data,\n",
+    "                                headers={\"Content-Type\": \"application/json\",\n",
+    "                                         \"Authorization\": f\"Bearer {api_key}\"},\n",
+    "                                method='POST')\n",
+    "    try:\n",
+    "        with urllib.request.urlopen(req, timeout=120) as resp:\n",
+    "            result = json.loads(resp.read().decode())\n",
+    "            return result['choices'][0]['message']['content']\n",
+    "    except Exception as e:\n",
+    "        return f\"ERROR: {e}\"\n",
+    "\n",
+    "\n",
+    "def build_prompt(task):\n",
+    "    \"\"\"Build prompt for ARC task.\"\"\"\n",
+    "    train_pairs = task.get('train', [])\n",
+    "    examples = []\n",
+    "    for i, pair in enumerate(train_pairs):\n",
+    "        examples.append(\n",
+    "            f\"Example {i+1}:\\n\"\n",
+    "            f\"  Input:  {json.dumps(pair['input'])}\\n\"\n",
+    "            f\"  Output: {json.dumps(pair['output'])}\"\n",
+    "        )\n",
+    "    examples_str = \"\\n\".join(examples)\n",
+    "\n",
+    "    inputs = [np.array(p['input']) for p in train_pairs]\n",
+    "    outputs = [np.array(p['output']) for p in train_pairs]\n",
+    "    same_shape = all(i.shape == o.shape for i, o in zip(inputs, outputs))\n",
+    "    in_colors = sorted(set(c for i in inputs for c in np.unique(i).tolist()))\n",
+    "    out_colors = sorted(set(c for o in outputs for c in np.unique(o).tolist()))\n",
+    "\n",
+    "    analysis = f\"  Same input/output shape: {same_shape}\\n\"\n",
+    "    analysis += f\"  Input colors: {in_colors}, Output colors: {out_colors}\\n\"\n",
+    "    if not same_shape:\n",
+    "        for i, o in zip(inputs[:1], outputs[:1]):\n",
+    "            analysis += f\"  Shape: {i.shape} -> {o.shape}\\n\"\n",
+    "\n",
+    "    return f\"\"\"Solve this ARC-AGI puzzle. Write ONLY a Python function, no explanations.\n",
+    "\n",
+    "{examples_str}\n",
+    "\n",
+    "Analysis:\n",
+    "{analysis}\n",
+    "```python\n",
+    "import numpy as np\n",
+    "from collections import Counter, deque\n",
+    "\n",
+    "def transform(grid: list[list[int]]) -> list[list[int]]:\n",
+    "    grid = np.array(grid)\n",
+    "\"\"\"\n",
+    "\n",
+    "\n",
+    "def extract_code(response):\n",
+    "    \"\"\"Extract Python function from LLM response.\"\"\"\n",
+    "    for pattern in [r'```python\\s*(.*?)```', r'```\\s*(.*?)```']:\n",
+    "        matches = re.findall(pattern, response, re.DOTALL)\n",
+    "        for match in matches:\n",
+    "            if 'def transform' in match:\n",
+    "                return match.strip()\n",
+    "    idx = response.find('def transform')\n",
+    "    if idx >= 0:\n",
+    "        before = response[:idx]\n",
+    "        import_start = max(before.rfind('import '), before.rfind('from '))\n",
+    "        start = import_start if import_start >= 0 else idx\n",
+    "        code = response[start:]\n",
+    "        end = code.find('```')\n",
+    "        if end > 0:\n",
+    "            code = code[:end]\n",
+    "        return code.strip()\n",
+    "    stripped = response.strip()\n",
+    "    if stripped.startswith(('import', 'def transform', 'from')):\n",
+    "        return stripped\n",
+    "    return None\n",
+    "\n",
+    "\n",
+    "def verify_program(code, train_pairs):\n",
+    "    \"\"\"Execute program and verify against all training pairs.\"\"\"\n",
+    "    namespace = {'np': np, 'numpy': np, 'Counter': Counter,\n",
+    "                 'deque': __import__('collections').deque}\n",
+    "    try:\n",
+    "        import scipy.ndimage\n",
+    "        namespace['scipy'] = __import__('scipy')\n",
+    "    except ImportError:\n",
+    "        pass\n",
+    "    try:\n",
+    "        exec(code, namespace)\n",
+    "    except Exception:\n",
+    "        return False\n",
+    "    if 'transform' not in namespace:\n",
+    "        return False\n",
+    "    fn = namespace['transform']\n",
+    "    for pair in train_pairs:\n",
+    "        try:\n",
+    "            result = fn([row[:] for row in pair['input']])\n",
+    "            if result is None:\n",
+    "                return False\n",
+    "            r = np.array(result, dtype=int)\n",
+    "            e = np.array(pair['output'], dtype=int)\n",
+    "            if r.shape != e.shape or not np.array_equal(r, e):\n",
+    "                return False\n",
+    "        except Exception:\n",
+    "            return False\n",
+    "    return True\n",
+    "\n",
+    "\n",
+    "def apply_program(code, test_input):\n",
+    "    \"\"\"Apply verified program to test input.\"\"\"\n",
+    "    namespace = {'np': np, 'numpy': np, 'Counter': Counter,\n",
+    "                 'deque': __import__('collections').deque}\n",
+    "    try:\n",
+    "        import scipy.ndimage\n",
+    "        namespace['scipy'] = __import__('scipy')\n",
+    "    except ImportError:\n",
+    "        pass\n",
+    "    try:\n",
+    "        exec(code, namespace)\n",
+    "        result = namespace['transform']([row[:] for row in test_input])\n",
+    "        if result is not None:\n",
+    "            return np.array(result, dtype=int).tolist()\n",
+    "    except Exception:\n",
+    "        pass\n",
+    "    return None\n",
+    "\n",
+    "\n",
+    "print('LLM engine ready.')"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## 3. Quick Test (1 task)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# Quick test — verify API works before running all 330 tasks\n",
+    "test_tid = '0520fde7'\n",
+    "with open(f'arc_data/training/{test_tid}.json') as f:\n",
+    "    test_task = json.load(f)\n",
+    "\n",
+    "print(f'Testing on {test_tid}...')\n",
+    "for i, p in enumerate(test_task['train']):\n",
+    "    inp = np.array(p['input']); out = np.array(p['output'])\n",
+    "    print(f'  Pair {i}: {inp.shape} -> {out.shape}')\n",
+    "\n",
+    "prompt = build_prompt(test_task)\n",
+    "print(f'Prompt: {len(prompt)} chars')\n",
+    "\n",
+    "response = call_nvidia(prompt, NVIDIA_API_KEY, MODEL, temperature=0.1)\n",
+    "if response.startswith('ERROR:'):\n",
+    "    print(f'\\n❌ API Error: {response}')\n",
+    "    print('Check your NVIDIA_API_KEY and MODEL settings above.')\n",
+    "else:\n",
+    "    code = extract_code(response)\n",
+    "    if code:\n",
+    "        ok = verify_program(code, test_task['train'])\n",
+    "        print(f'\\nCode extracted: {len(code)} chars')\n",
+    "        print(f'Verified: {\"✅\" if ok else \"❌\"}')\n",
+    "        if ok:\n",
+    "            print('API working and generating correct code!')\n",
+    "        else:\n",
+    "            print('API working but code failed verification (normal — will try more candidates in full run)')\n",
+    "    else:\n",
+    "        print(f'\\nNo code extracted from response ({len(response)} chars)')\n",
+    "        print('API working but response format unexpected. Will retry with different temperatures in full run.')"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## 4. Run on All Unsolved Tasks"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# Load all unsolved tasks\n",
+    "task_files = sorted(glob.glob('arc_data/training/*.json'))\n",
+    "unsolved = []\n",
+    "for tf in task_files:\n",
+    "    tid = os.path.basename(tf).replace('.json', '')\n",
+    "    if tid not in ALREADY_SOLVED:\n",
+    "        unsolved.append((tid, tf))\n",
+    "\n",
+    "print(f'Total tasks: {len(task_files)}')\n",
+    "print(f'Already solved (symbolic): {len(ALREADY_SOLVED)}')\n",
+    "print(f'To attempt with LLM: {len(unsolved)}')\n",
+    "print(f'Model: {MODEL}')\n",
+    "print(f'Candidates per task: {N_CANDIDATES}')\n",
+    "print(f'\\nStarting...')"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# Main loop\n",
+    "results = {}\n",
+    "solved = 0\n",
+    "total_time = 0\n",
+    "\n",
+    "# Resume from previous run if exists\n",
+    "if os.path.exists('llm_results.json'):\n",
+    "    with open('llm_results.json') as f:\n",
+    "        prev = json.load(f)\n",
+    "    results = prev.get('results', {})\n",
+    "    solved = sum(1 for r in results.values() if r['status'] == 'solved')\n",
+    "    print(f'Resuming from previous run: {solved} already solved by LLM')\n",
+    "\n",
+    "for idx, (tid, tf) in enumerate(unsolved):\n",
+    "    # Skip if already attempted\n",
+    "    if tid in results:\n",
+    "        continue\n",
+    "    \n",
+    "    with open(tf) as f:\n",
+    "        task = json.load(f)\n",
+    "    \n",
+    "    print(f'[{idx+1:3d}/{len(unsolved)}] {tid}:', end=' ', flush=True)\n",
+    "    start = time.time()\n",
+    "    \n",
+    "    prompt = build_prompt(task)\n",
+    "    task_solved = False\n",
+    "    \n",
+    "    for i in range(N_CANDIDATES):\n",
+    "        temp = 0.1 if i == 0 else min(0.4 + 0.15 * i, 1.2)\n",
+    "        response = call_nvidia(prompt, NVIDIA_API_KEY, MODEL, temp)\n",
+    "        \n",
+    "        if response.startswith('ERROR:'):\n",
+    "            if '429' in response or 'rate' in response.lower():\n",
+    "                time.sleep(10)  # Rate limit — wait longer\n",
+    "            continue\n",
+    "        \n",
+    "        code = extract_code(response)\n",
+    "        if code is None:\n",
+    "            continue\n",
+    "        \n",
+    "        if verify_program(code, task['train']):\n",
+    "            elapsed = time.time() - start\n",
+    "            total_time += elapsed\n",
+    "            solved += 1\n",
+    "            \n",
+    "            test_outputs = [apply_program(code, t['input']) for t in task.get('test', [])]\n",
+    "            results[tid] = {\n",
+    "                'status': 'solved', 'rule': f'llm_c{i+1}_t{temp:.1f}',\n",
+    "                'code': code, 'test_outputs': test_outputs,\n",
+    "                'time_s': round(elapsed, 2),\n",
+    "            }\n",
+    "            print(f'✅ c{i+1} ({elapsed:.1f}s)  [total: {len(ALREADY_SOLVED)+solved}/{len(task_files)}]')\n",
+    "            task_solved = True\n",
+    "            break\n",
+    "        \n",
+    "        time.sleep(RATE_LIMIT_SLEEP)\n",
+    "    \n",
+    "    if not task_solved:\n",
+    "        elapsed = time.time() - start\n",
+    "        total_time += elapsed\n",
+    "        results[tid] = {'status': 'failed', 'time_s': round(elapsed, 2)}\n",
+    "        print(f'❌ ({elapsed:.1f}s)')\n",
+    "    \n",
+    "    # Save progress every 10 tasks\n",
+    "    if (idx + 1) % 10 == 0:\n",
+    "        with open('llm_results.json', 'w') as f:\n",
+    "            json.dump({\n",
+    "                'model': MODEL, 'n_candidates': N_CANDIDATES,\n",
+    "                'llm_solved': solved, 'attempted': sum(1 for r in results.values()),\n",
+    "                'symbolic_solved': len(ALREADY_SOLVED),\n",
+    "                'total_solved': len(ALREADY_SOLVED) + solved,\n",
+    "                'total_tasks': len(task_files),\n",
+    "                'solve_rate': round(100 * (len(ALREADY_SOLVED) + solved) / len(task_files), 2),\n",
+    "                'total_time_s': round(total_time, 1),\n",
+    "                'results': results,\n",
+    "            }, f, indent=2)\n",
+    "        print(f'  [Saved: {len(ALREADY_SOLVED)+solved}/{len(task_files)} total]')"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# Final save\n",
+    "with open('llm_results.json', 'w') as f:\n",
+    "    json.dump({\n",
+    "        'model': MODEL, 'n_candidates': N_CANDIDATES,\n",
+    "        'llm_solved': solved, 'attempted': sum(1 for r in results.values()),\n",
+    "        'symbolic_solved': len(ALREADY_SOLVED),\n",
+    "        'total_solved': len(ALREADY_SOLVED) + solved,\n",
+    "        'total_tasks': len(task_files),\n",
+    "        'solve_rate': round(100 * (len(ALREADY_SOLVED) + solved) / len(task_files), 2),\n",
+    "        'total_time_s': round(total_time, 1),\n",
+    "        'results': results,\n",
+    "    }, f, indent=2)\n",
+    "\n",
+    "print(f'\\n{\"=\"*60}')\n",
+    "print(f'FINAL RESULTS')\n",
+    "print(f'{\"=\"*60}')\n",
+    "print(f'LLM solved:      {solved}')\n",
+    "print(f'Symbolic solved:  {len(ALREADY_SOLVED)}')\n",
+    "print(f'TOTAL SOLVED:     {len(ALREADY_SOLVED)+solved}/{len(task_files)} ({100*(len(ALREADY_SOLVED)+solved)/len(task_files):.1f}%)')\n",
+    "print(f'Time:             {total_time:.0f}s')\n",
+    "print(f'\\nResults saved to: llm_results.json')"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## 5. Results Analysis"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# Load and analyze results\n",
+    "with open('llm_results.json') as f:\n",
+    "    data = json.load(f)\n",
+    "\n",
+    "print(f'Model: {data[\"model\"]}')\n",
+    "print(f'Candidates per task: {data[\"n_candidates\"]}')\n",
+    "print(f'\\nSymbolic solved: {data[\"symbolic_solved\"]}')\n",
+    "print(f'LLM solved: {data[\"llm_solved\"]}')\n",
+    "print(f'TOTAL: {data[\"total_solved\"]}/{data[\"total_tasks\"]} ({data[\"solve_rate\"]}%)')\n",
+    "\n",
+    "llm_solved_tasks = [tid for tid, r in data['results'].items() if r['status'] == 'solved']\n",
+    "print(f'\\nLLM-solved tasks ({len(llm_solved_tasks)}):')\n",
+    "for tid in sorted(llm_solved_tasks):\n",
+    "    rule = data['results'][tid].get('rule', '?')\n",
+    "    t = data['results'][tid].get('time_s', 0)\n",
+    "    print(f'  {tid}: {rule} ({t}s)')"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## 6. Download Results\n",
+    "\n",
+    "Download `llm_results.json` from the notebook output, then merge with symbolic results:\n",
+    "\n",
+    "```bash\n",
+    "python scripts/merge_results.py arc_results/summary_v4.json llm_results.json\n",
+    "```"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "name": "python",
+   "version": "3.10.0"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 4
+}

pemf/pyproject.toml

@@ -0,0 +1,42 @@
+[project]
+name = "pemf-arc-agi"
+version = "0.4.0"
+description = "Pre-Emergence Mechanics Framework (PEMF) solver for ARC-AGI"
+requires-python = ">=3.10"
+license = {text = "MIT"}
+
+dependencies = [
+    "numpy>=1.24",
+    "scipy>=1.10",
+]
+
+[project.optional-dependencies]
+viz = [
+    "matplotlib>=3.7",
+]
+wandb = [
+    "wandb>=0.15",
+    "matplotlib>=3.7",
+]
+llm = [
+    "huggingface-hub>=0.20",
+]
+all = [
+    "numpy>=1.24",
+    "scipy>=1.10",
+    "matplotlib>=3.7",
+    "wandb>=0.15",
+    "huggingface-hub>=0.20",
+]
+
+[build-system]
+requires = ["hatchling"]
+build-backend = "hatchling.build"
+
+[tool.hatch.build.targets.wheel]
+packages = ["itt_solver"]
+
+[dependency-groups]
+dev = [
+    "pytest>=7.0",
+]

pemf/scripts/entrypoint.py

@@ -0,0 +1,84 @@
+"""
+Headless entrypoint for running a single experiment or a sweep.
+
+Usage:
+  python scripts/entrypoint.py --task example1 --out_dir experiments
+  python scripts/entrypoint.py --task example1 --out_dir experiments --use_wandb
+
+By default Weights & Biases logging is disabled. Use --use_wandb to enable it.
+"""
+import argparse
+import json
+import os
+import importlib
+
+def main():
+    parser = argparse.ArgumentParser(description="Run ARC-AGI experiment (headless).")
+    parser.add_argument("--task", type=str, required=True, help="Task name or path to task JSON")
+    parser.add_argument("--out_dir", type=str, default="experiments", help="Output directory")
+    parser.add_argument("--use_wandb", action="store_true", help="Enable Weights & Biases logging (default: off)")
+    parser.add_argument("--params", type=str, default=None, help="Optional JSON string of params")
+    args = parser.parse_args()
+
+    os.makedirs(args.out_dir, exist_ok=True)
+
+    # lazy imports to avoid heavy startup cost
+    import itt_solver.experiment_driver as ed
+    import itt_solver.solver_core as sc
+
+    # load task: if args.task is a JSON file path, load it; otherwise expect a built-in name
+    if os.path.exists(args.task):
+        with open(args.task) as fh:
+            task = json.load(fh)
+    else:
+        # minimal built-in example if user passed 'example1'
+        # Corrected target from real ARC task 007bbfb7 (Kronecker self-similar)
+        if args.task == "example1":
+            task = {
+              'name': 'example1',
+              'input': [[0,7,7],[7,7,7],[0,7,7]],
+              'target': [
+                [0,0,0,0,7,7,0,7,7],
+                [0,0,0,7,7,7,7,7,7],
+                [0,0,0,0,7,7,0,7,7],
+                [0,7,7,0,7,7,0,7,7],
+                [7,7,7,7,7,7,7,7,7],
+                [0,7,7,0,7,7,0,7,7],
+                [0,0,0,0,7,7,0,7,7],
+                [0,0,0,7,7,7,7,7,7],
+                [0,0,0,0,7,7,0,7,7],
+              ],
+              'target_shape': (9,9)
+            }
+        else:
+            raise SystemExit(f"Unknown task identifier: {args.task}")
+
+    # parse params if provided
+    params = {}
+    if args.params:
+        try:
+            params = json.loads(args.params)
+        except Exception:
+            print("Warning: could not parse --params JSON; ignoring.")
+
+    # build atomic library using default factory
+    atomic_library = ed.default_atomic_factory(params, task)
+
+    # run single experiment
+    result = ed.run_single(task, atomic_library, params, out_dir=args.out_dir)
+
+    # optionally run W&B logging externally (only if requested)
+    if args.use_wandb:
+        try:
+            from itt_solver.wandb_runner import run_and_log_wandb
+            run_and_log_wandb(task, atomic_library, params, out_dir=args.out_dir,
+                              wandb_project=params.get('wandb_project','itt_solver'),
+                              wandb_entity=None, resume="allow")
+        except Exception as e:
+            print("W&B logging failed or not configured:", e)
+
+    print("Run finished. Result summary:")
+    print(json.dumps(result, indent=2))
+
+if __name__ == "__main__":
+    main()

pemf/scripts/fix_and_inspect_logs.py

@@ -0,0 +1,104 @@
+import glob, json, numpy as np, os
+from pprint import pprint
+
+def load_latest(pattern):
+    files = sorted(glob.glob(pattern))
+    return files[-1] if files else None
+
+logs_path = load_latest("experiments/*_logs.json")
+phi_path = load_latest("experiments/*_phi_best.npy")
+res_path = load_latest("experiments/*_result.json")
+
+print("logs:", logs_path)
+print("phi_best:", phi_path)
+print("result:", res_path)
+
+if not logs_path:
+    raise SystemExit("No logs file found")
+
+logs = json.load(open(logs_path))
+res = json.load(open(res_path)) if res_path else {}
+
+# coerce gate values to booleans for all depth entries
+def coerce_gates(g):
+    if not isinstance(g, dict):
+        return g
+    out = {}
+    for k,v in g.items():
+        if isinstance(v, str):
+            lv = v.strip().lower()
+            if lv in ("true","1","yes"):
+                out[k] = True
+            elif lv in ("false","0","no"):
+                out[k] = False
+            else:
+                try:
+                    out[k] = bool(int(v))
+                except Exception:
+                    out[k] = v
+        else:
+            out[k] = v
+    return out
+
+for depth_idx, depth in enumerate(logs):
+    for entry in depth:
+        if 'gates' in entry:
+            entry['gates'] = coerce_gates(entry['gates'])
+
+# attach phi_best into the first accepted entry (if not present)
+accepted_entry = None
+for entry in logs[0]:
+    if entry.get('accepted'):
+        accepted_entry = entry
+        break
+
+phi = np.load(phi_path) if phi_path else None
+if accepted_entry is not None:
+    if 'candidate_array' not in accepted_entry:
+        accepted_entry['candidate_array'] = phi.tolist() if phi is not None else None
+
+# Corrected target from real ARC task 007bbfb7 (Kronecker self-similar)
+TARGET_GRID = [
+  [0,0,0,0,7,7,0,7,7],
+  [0,0,0,7,7,7,7,7,7],
+  [0,0,0,0,7,7,0,7,7],
+  [0,7,7,0,7,7,0,7,7],
+  [7,7,7,7,7,7,7,7,7],
+  [0,7,7,0,7,7,0,7,7],
+  [0,0,0,0,7,7,0,7,7],
+  [0,0,0,7,7,7,7,7,7],
+  [0,0,0,0,7,7,0,7,7],
+]
+TARGET = np.array(TARGET_GRID, dtype=int)
+
+def tile_transform(phi, out_shape):
+    a = np.array(phi)
+    h_out, w_out = out_shape
+    h_in, w_in = a.shape
+    reps_h = (h_out + h_in - 1) // h_in
+    reps_w = (w_out + w_in - 1) // w_in
+    tiled = np.tile(a, (reps_h, reps_w))
+    return tiled[:h_out, :w_out]
+
+if accepted_entry is not None and accepted_entry.get('candidate_array') is not None:
+    cand = np.array(accepted_entry['candidate_array'], dtype=float)
+    if cand.shape != TARGET.shape:
+        cand_resized = tile_transform(cand, TARGET.shape)
+    else:
+        cand_resized = cand
+    cand_q = np.rint(cand_resized).astype(int)
+    l1 = float(np.sum(np.abs(cand_q - TARGET)))
+    print("Recomputed L1 residue for first accepted candidate:", l1)
+    print("Candidate unique values:", np.unique(cand_q))
+    diff = (cand_q != TARGET).astype(int)
+    print("Changed cells count:", int(diff.sum()))
+    print("Diff map (1=diff):")
+    print(diff)
+else:
+    print("No candidate array available in logs or phi_best missing.")
+
+# write fixed logs copy
+fixed_path = logs_path.replace("_logs.json", "_logs.fixed.json")
+with open(fixed_path, "w") as fh:
+    json.dump(logs, fh, indent=2)
+print("Wrote fixed logs to", fixed_path)

pemf/scripts/kaggle_llm_solver.py

@@ -0,0 +1,452 @@
+"""
+PEMF ARC-AGI — LLM Program Synthesis via Ollama (Kaggle Edition)
+================================================================
+
+Self-contained script for Kaggle GPU notebooks.
+Pulls a model via Ollama, runs LLM synthesis on unsolved ARC tasks.
+
+Usage on Kaggle:
+  1. Enable GPU (T4 x2 or P100)
+  2. Enable internet access
+  3. Upload this file + arc_data/ + already_solved.json
+  4. Run all cells
+
+The script:
+  - Installs Ollama
+  - Pulls the model (qwen2.5-coder:32b or smaller)
+  - Loads ARC tasks
+  - For each unsolved task: generates Python transform(), verifies against training pairs
+  - Saves results to llm_results.json
+"""
+
+import subprocess
+import sys
+import os
+import json
+import time
+import re
+import signal
+import numpy as np
+from typing import Dict, List, Optional, Tuple
+from collections import Counter
+from pathlib import Path
+
+
+# =============================================================================
+# 1. OLLAMA SETUP
+# =============================================================================
+
+def install_ollama():
+    """Install Ollama on Kaggle/Linux."""
+    print("Installing Ollama...")
+    subprocess.run("curl -fsSL https://ollama.com/install.sh | sh", 
+                   shell=True, check=True, capture_output=True)
+    print("Ollama installed.")
+
+
+def start_ollama():
+    """Start Ollama server in background."""
+    print("Starting Ollama server...")
+    proc = subprocess.Popen(
+        ["ollama", "serve"],
+        stdout=subprocess.DEVNULL,
+        stderr=subprocess.DEVNULL,
+    )
+    time.sleep(3)  # Wait for server to start
+    print(f"Ollama server started (PID {proc.pid})")
+    return proc
+
+
+def pull_model(model_name: str):
+    """Pull a model via Ollama."""
+    print(f"Pulling model {model_name}... (this may take several minutes)")
+    result = subprocess.run(
+        ["ollama", "pull", model_name],
+        capture_output=True, text=True, timeout=1800
+    )
+    if result.returncode != 0:
+        print(f"Pull failed: {result.stderr}")
+        raise RuntimeError(f"Failed to pull {model_name}")
+    print(f"Model {model_name} ready.")
+
+
+def call_ollama(prompt: str, model: str = "qwen2.5-coder:32b",
+                temperature: float = 0.7, timeout_s: int = 120) -> str:
+    """Call Ollama API and return response text."""
+    import urllib.request
+
+    payload = {
+        "model": model,
+        "prompt": prompt,
+        "stream": False,
+        "options": {
+            "temperature": temperature,
+            "num_predict": 2048,
+        }
+    }
+
+    data = json.dumps(payload).encode('utf-8')
+    req = urllib.request.Request(
+        "http://localhost:11434/api/generate",
+        data=data,
+        headers={"Content-Type": "application/json"},
+        method='POST'
+    )
+
+    try:
+        with urllib.request.urlopen(req, timeout=timeout_s) as resp:
+            result = json.loads(resp.read().decode())
+            return result.get('response', '')
+    except Exception as e:
+        return f"ERROR: {e}"
+
+
+# =============================================================================
+# 2. PROMPT BUILDING
+# =============================================================================
+
+def build_prompt(task: Dict) -> str:
+    """Build prompt for ARC task."""
+    train_pairs = task.get('train', [])
+
+    examples = []
+    for i, pair in enumerate(train_pairs):
+        examples.append(
+            f"Example {i+1}:\n"
+            f"  Input:  {json.dumps(pair['input'])}\n"
+            f"  Output: {json.dumps(pair['output'])}"
+        )
+    examples_str = "\n".join(examples)
+
+    # Basic analysis
+    inputs = [np.array(p['input']) for p in train_pairs]
+    outputs = [np.array(p['output']) for p in train_pairs]
+    same_shape = all(i.shape == o.shape for i, o in zip(inputs, outputs))
+    in_colors = sorted(set(c for i in inputs for c in np.unique(i).tolist()))
+    out_colors = sorted(set(c for o in outputs for c in np.unique(o).tolist()))
+
+    analysis = f"  Same input/output shape: {same_shape}\n"
+    analysis += f"  Input colors: {in_colors}\n"
+    analysis += f"  Output colors: {out_colors}\n"
+    if not same_shape:
+        ratios = [(o.shape[0]/i.shape[0], o.shape[1]/i.shape[1]) 
+                  for i, o in zip(inputs, outputs)]
+        analysis += f"  Shape ratios (h,w): {ratios}\n"
+
+    prompt = f"""Solve this ARC-AGI puzzle. Write ONLY a Python function, no explanations.
+
+{examples_str}
+
+Analysis:
+{analysis}
+Write a complete Python function that transforms any input grid to its output.
+The function MUST work correctly for ALL examples above.
+
+```python
+import numpy as np
+from collections import Counter
+
+def transform(grid: list[list[int]]) -> list[list[int]]:
+    grid = np.array(grid)
+"""
+    return prompt
+
+
+# =============================================================================
+# 3. CODE EXTRACTION AND VERIFICATION
+# =============================================================================
+
+def extract_code(response: str) -> Optional[str]:
+    """Extract Python function from LLM response."""
+    # Try ```python blocks
+    for pattern in [r'```python\s*(.*?)```', r'```\s*(.*?)```']:
+        matches = re.findall(pattern, response, re.DOTALL)
+        for match in matches:
+            if 'def transform' in match:
+                return match.strip()
+
+    # Try finding def transform directly
+    idx = response.find('def transform')
+    if idx >= 0:
+        # Look backwards for imports
+        before = response[:idx]
+        import_start = before.rfind('import ')
+        if import_start >= 0:
+            code = response[import_start:]
+        else:
+            code = response[idx:]
+        # Trim at next ``` or double newline after function ends
+        end = code.find('```')
+        if end > 0:
+            code = code[:end]
+        return code.strip()
+
+    # If response itself looks like code (starts with import or def)
+    stripped = response.strip()
+    if stripped.startswith('import') or stripped.startswith('def transform'):
+        return stripped
+
+    return None
+
+
+def verify_program(code: str, train_pairs: List[Dict]) -> bool:
+    """Execute program and verify against all training pairs."""
+    namespace = {'np': np, 'numpy': np, 'Counter': Counter,
+                 'collections': __import__('collections')}
+
+    try:
+        exec(code, namespace)
+    except Exception:
+        return False
+
+    if 'transform' not in namespace:
+        return False
+
+    transform_fn = namespace['transform']
+
+    for pair in train_pairs:
+        try:
+            inp = [row[:] for row in pair['input']]  # deep copy
+            result = transform_fn(inp)
+            if result is None:
+                return False
+            result_arr = np.array(result, dtype=int)
+            expected_arr = np.array(pair['output'], dtype=int)
+            if result_arr.shape != expected_arr.shape:
+                return False
+            if not np.array_equal(result_arr, expected_arr):
+                return False
+        except Exception:
+            return False
+
+    return True
+
+
+def apply_program(code: str, test_input: List[List[int]]) -> Optional[List[List[int]]]:
+    """Apply verified program to test input."""
+    namespace = {'np': np, 'numpy': np, 'Counter': Counter,
+                 'collections': __import__('collections')}
+    try:
+        exec(code, namespace)
+        result = namespace['transform']([row[:] for row in test_input])
+        if result is not None:
+            return [list(row) for row in np.array(result, dtype=int).tolist()]
+    except Exception:
+        pass
+    return None
+
+
+# =============================================================================
+# 4. SYNTHESIS ENGINE
+# =============================================================================
+
+def synthesize_task(task: Dict, model: str = "qwen2.5-coder:32b",
+                    n_candidates: int = 8, verbose: bool = False) -> Optional[Tuple[str, str]]:
+    """
+    Try to solve a task via LLM.
+    Returns (rule_name, code) if successful, None otherwise.
+    """
+    train_pairs = task.get('train', [])
+    if not train_pairs:
+        return None
+
+    prompt = build_prompt(task)
+
+    for i in range(n_candidates):
+        temp = 0.1 if i == 0 else 0.5 + 0.1 * i  # first try low temp, then increase
+        response = call_ollama(prompt, model=model, temperature=min(temp, 1.0))
+
+        if response.startswith("ERROR:"):
+            if verbose:
+                print(f"    Candidate {i+1}: API error")
+            continue
+
+        code = extract_code(response)
+        if code is None:
+            if verbose:
+                print(f"    Candidate {i+1}: No code extracted")
+            continue
+
+        if verbose:
+            print(f"    Candidate {i+1}: {len(code)} chars", end="")
+
+        if verify_program(code, train_pairs):
+            if verbose:
+                print(f" ✅")
+            return (f"llm_c{i+1}_t{temp:.1f}", code)
+        else:
+            if verbose:
+                print(f" ❌")
+
+    return None
+
+
+# =============================================================================
+# 5. MAIN RUNNER
+# =============================================================================
+
+def main():
+    # --- Configuration ---
+    MODEL = os.environ.get("OLLAMA_MODEL", "qwen2.5-coder:32b")
+    # For smaller GPUs, use:
+    #   MODEL = "qwen2.5-coder:14b"   (fits T4 16GB)
+    #   MODEL = "qwen2.5-coder:7b"    (fits any GPU)
+
+    N_CANDIDATES = int(os.environ.get("N_CANDIDATES", "8"))
+    ARC_DIR = os.environ.get("ARC_DIR", "arc_data/training")
+    ALREADY_SOLVED_FILE = os.environ.get("ALREADY_SOLVED", "already_solved.json")
+    OUTPUT_FILE = os.environ.get("OUTPUT_FILE", "llm_results.json")
+
+    print("=" * 60)
+    print("PEMF ARC-AGI — LLM Program Synthesis (Kaggle/Ollama)")
+    print("=" * 60)
+    print(f"Model: {MODEL}")
+    print(f"Candidates per task: {N_CANDIDATES}")
+    print(f"ARC data: {ARC_DIR}")
+    print()
+
+    # --- Install & start Ollama ---
+    try:
+        subprocess.run(["ollama", "--version"], capture_output=True, check=True)
+        print("Ollama already installed.")
+    except (FileNotFoundError, subprocess.CalledProcessError):
+        install_ollama()
+
+    server = start_ollama()
+
+    try:
+        pull_model(MODEL)
+    except Exception as e:
+        print(f"Failed to pull {MODEL}: {e}")
+        print("Trying smaller model...")
+        MODEL = "qwen2.5-coder:7b"
+        pull_model(MODEL)
+
+    # --- Load already solved tasks ---
+    already_solved = set()
+    if os.path.exists(ALREADY_SOLVED_FILE):
+        with open(ALREADY_SOLVED_FILE) as f:
+            already_solved = set(json.load(f))
+        print(f"Already solved (symbolic): {len(already_solved)} tasks")
+
+    # --- Load ARC tasks ---
+    import glob
+    task_files = sorted(glob.glob(os.path.join(ARC_DIR, "*.json")))
+    print(f"Total ARC tasks: {len(task_files)}")
+
+    unsolved_files = []
+    for tf in task_files:
+        tid = os.path.basename(tf).replace('.json', '')
+        if tid not in already_solved:
+            unsolved_files.append((tid, tf))
+    print(f"Unsolved tasks to try: {len(unsolved_files)}")
+    print()
+
+    # --- Run synthesis ---
+    results = {}
+    solved = 0
+    total_time = 0
+
+    for idx, (tid, tf) in enumerate(unsolved_files):
+        with open(tf) as f:
+            task = json.load(f)
+
+        print(f"[{idx+1:3d}/{len(unsolved_files)}] {tid}:", end=" ", flush=True)
+        start = time.time()
+
+        result = synthesize_task(task, model=MODEL, n_candidates=N_CANDIDATES, verbose=False)
+        elapsed = time.time() - start
+        total_time += elapsed
+
+        if result:
+            rule_name, code = result
+            solved += 1
+
+            # Apply to test pairs
+            test_outputs = []
+            for test in task.get('test', []):
+                out = apply_program(code, test['input'])
+                test_outputs.append(out)
+
+            results[tid] = {
+                'status': 'solved',
+                'rule': rule_name,
+                'code': code,
+                'test_outputs': test_outputs,
+                'time_s': round(elapsed, 2),
+            }
+            print(f"✅ {rule_name} ({elapsed:.1f}s)")
+        else:
+            results[tid] = {
+                'status': 'failed',
+                'time_s': round(elapsed, 2),
+            }
+            print(f"❌ ({elapsed:.1f}s)")
+
+        # Save progress periodically
+        if (idx + 1) % 10 == 0:
+            with open(OUTPUT_FILE, 'w') as f:
+                json.dump({
+                    'model': MODEL,
+                    'n_candidates': N_CANDIDATES,
+                    'solved': solved,
+                    'attempted': idx + 1,
+                    'total_time_s': round(total_time, 1),
+                    'results': results,
+                }, f, indent=2)
+            print(f"  [Progress saved: {solved}/{idx+1} solved]")
+
+    # --- Final save ---
+    with open(OUTPUT_FILE, 'w') as f:
+        json.dump({
+            'model': MODEL,
+            'n_candidates': N_CANDIDATES,
+            'solved': solved,
+            'attempted': len(unsolved_files),
+            'total_time_s': round(total_time, 1),
+            'already_solved_symbolic': len(already_solved),
+            'total_solved': len(already_solved) + solved,
+            'total_tasks': len(task_files),
+            'solve_rate': round(100 * (len(already_solved) + solved) / len(task_files), 2),
+            'results': results,
+        }, f, indent=2)
+
+    # --- Summary ---
+    print()
+    print("=" * 60)
+    print("FINAL RESULTS")
+    print("=" * 60)
+    print(f"LLM solved:        {solved}/{len(unsolved_files)} unsolved tasks")
+    print(f"Symbolic solved:   {len(already_solved)}")
+    print(f"TOTAL SOLVED:      {len(already_solved) + solved}/{len(task_files)} ({100*(len(already_solved)+solved)/len(task_files):.1f}%)")
+    print(f"Total LLM time:    {total_time:.0f}s ({total_time/max(1,len(unsolved_files)):.1f}s/task)")
+    print(f"Results saved to:  {OUTPUT_FILE}")
+
+    # Cleanup
+    server.terminate()
+
+
+# =============================================================================
+# 6. GENERATE already_solved.json FROM SYMBOLIC RESULTS
+# =============================================================================
+
+def generate_already_solved(summary_file: str, output_file: str = "already_solved.json"):
+    """
+    Generate already_solved.json from a v4 summary file.
+    Run this BEFORE running on Kaggle.
+    """
+    with open(summary_file) as f:
+        data = json.load(f)
+    solved = [r['task_id'] for r in data['results'] if r.get('all_train_solved')]
+    with open(output_file, 'w') as f:
+        json.dump(solved, f)
+    print(f"Wrote {len(solved)} solved task IDs to {output_file}")
+
+
+if __name__ == "__main__":
+    # If run with --generate-solved, create the already_solved.json
+    if len(sys.argv) > 1 and sys.argv[1] == "--generate-solved":
+        summary = sys.argv[2] if len(sys.argv) > 2 else "arc_results/summary_v4.json"
+        generate_already_solved(summary)
+    else:
+        main()

pemf/scripts/llm_solver_cloud.py

@@ -0,0 +1,483 @@
+"""
+PEMF ARC-AGI — LLM Program Synthesis (Multi-Provider)
+=====================================================
+
+Supports:
+  - NVIDIA NIM (free — DeepSeek V4 Pro, GLM-4, Qwen, Llama)
+  - Google Gemini (free tier: 15 RPM)
+  - DeepSeek direct API (very cheap)
+  - GLM/Zhipu direct API (free tier)
+  - Ollama local (any model)
+
+Usage:
+  # NVIDIA NIM — FREE, best option (GLM 4.7 default)
+  export LLM_PROVIDER=nvidia
+  export NVIDIA_API_KEY=nvapi-xxxxx
+  python llm_solver_cloud.py
+  # Get key: https://build.nvidia.com/settings/api-keys
+  # Default model: z-ai/glm4.7
+
+  # NVIDIA NIM with DeepSeek V4
+  export LLM_PROVIDER=nvidia
+  export NVIDIA_API_KEY=nvapi-xxxxx
+  export LLM_MODEL=deepseek-ai/deepseek-v4-pro
+  python llm_solver_cloud.py
+
+  # Gemini (free)
+  export LLM_PROVIDER=gemini
+  export GEMINI_API_KEY=your_key
+  python llm_solver_cloud.py
+
+  # Ollama local
+  export LLM_PROVIDER=ollama
+  export OLLAMA_MODEL=qwen2.5-coder:32b
+  python llm_solver_cloud.py
+"""
+
+import os
+import sys
+import json
+import time
+import re
+import glob
+import numpy as np
+from typing import Dict, List, Optional, Tuple
+from collections import Counter
+import urllib.request
+
+
+# =============================================================================
+# PROVIDER CONFIGS
+# =============================================================================
+
+PROVIDERS = {
+    "nvidia": {
+        "name": "NVIDIA NIM (free — DeepSeek V4, GLM 4.7, Qwen, Llama)",
+        "base_url": "https://integrate.api.nvidia.com/v1/chat/completions",
+        "default_model": "z-ai/glm4.7",
+        "env_key": "NVIDIA_API_KEY",
+        "free_tier": "Free for NVIDIA Developer Program members",
+        "get_key_url": "https://build.nvidia.com/settings/api-keys",
+        "models": {
+            "glm4.7": "z-ai/glm4.7",
+            "deepseek-v4": "deepseek-ai/deepseek-v4-pro",
+        },
+    },
+    "gemini": {
+        "name": "Google Gemini",
+        "base_url": "https://generativelanguage.googleapis.com/v1beta/models/{model}:generateContent",
+        "default_model": "gemini-2.0-flash",
+        "env_key": "GEMINI_API_KEY",
+        "free_tier": "15 RPM, 1M tokens/day",
+        "get_key_url": "https://aistudio.google.com/apikey",
+    },
+    "deepseek": {
+        "name": "DeepSeek (direct API)",
+        "base_url": "https://api.deepseek.com/v1/chat/completions",
+        "default_model": "deepseek-chat",
+        "env_key": "DEEPSEEK_API_KEY",
+        "free_tier": "$0.07/M input, $0.27/M output",
+        "get_key_url": "https://platform.deepseek.com/api_keys",
+    },
+    "glm": {
+        "name": "GLM (Zhipu AI direct)",
+        "base_url": "https://open.bigmodel.cn/api/paas/v4/chat/completions",
+        "default_model": "glm-4-flash",
+        "env_key": "GLM_API_KEY",
+        "free_tier": "glm-4-flash is free",
+        "get_key_url": "https://open.bigmodel.cn/usercenter/apikeys",
+    },
+    "ollama": {
+        "name": "Ollama (local)",
+        "base_url": "http://localhost:11434/api/generate",
+        "default_model": "qwen2.5-coder:32b",
+        "env_key": None,
+    },
+}
+
+
+# =============================================================================
+# API CALLERS
+# =============================================================================
+
+def call_nvidia(prompt: str, api_key: str, model: str = "deepseek-ai/deepseek-v4-pro",
+                temperature: float = 0.7) -> str:
+    """Call NVIDIA NIM API (OpenAI-compatible). Hosts DeepSeek V4, GLM, Qwen, Llama."""
+    url = "https://integrate.api.nvidia.com/v1/chat/completions"
+    payload = {
+        "model": model,
+        "messages": [{"role": "user", "content": prompt}],
+        "max_tokens": 2048,
+        "temperature": temperature,
+    }
+    data = json.dumps(payload).encode('utf-8')
+    req = urllib.request.Request(url, data=data,
+                                headers={"Content-Type": "application/json",
+                                         "Authorization": f"Bearer {api_key}"},
+                                method='POST')
+    try:
+        with urllib.request.urlopen(req, timeout=120) as resp:
+            result = json.loads(resp.read().decode())
+            return result['choices'][0]['message']['content']
+    except Exception as e:
+        return f"ERROR: {e}"
+
+
+def call_gemini(prompt: str, api_key: str, model: str = "gemini-2.0-flash",
+                temperature: float = 0.7) -> str:
+    """Call Google Gemini API."""
+    url = f"https://generativelanguage.googleapis.com/v1beta/models/{model}:generateContent?key={api_key}"
+    payload = {
+        "contents": [{"parts": [{"text": prompt}]}],
+        "generationConfig": {
+            "temperature": temperature,
+            "maxOutputTokens": 2048,
+        }
+    }
+    data = json.dumps(payload).encode('utf-8')
+    req = urllib.request.Request(url, data=data,
+                                headers={"Content-Type": "application/json"},
+                                method='POST')
+    try:
+        with urllib.request.urlopen(req, timeout=120) as resp:
+            result = json.loads(resp.read().decode())
+            candidates = result.get('candidates', [])
+            if candidates:
+                parts = candidates[0].get('content', {}).get('parts', [])
+                if parts:
+                    return parts[0].get('text', '')
+        return "ERROR: No response content"
+    except Exception as e:
+        return f"ERROR: {e}"
+
+
+def call_deepseek(prompt: str, api_key: str, model: str = "deepseek-chat",
+                  temperature: float = 0.7) -> str:
+    """Call DeepSeek API (OpenAI-compatible)."""
+    url = "https://api.deepseek.com/v1/chat/completions"
+    payload = {
+        "model": model,
+        "messages": [{"role": "user", "content": prompt}],
+        "max_tokens": 2048,
+        "temperature": temperature,
+    }
+    data = json.dumps(payload).encode('utf-8')
+    req = urllib.request.Request(url, data=data,
+                                headers={"Content-Type": "application/json",
+                                         "Authorization": f"Bearer {api_key}"},
+                                method='POST')
+    try:
+        with urllib.request.urlopen(req, timeout=120) as resp:
+            result = json.loads(resp.read().decode())
+            return result['choices'][0]['message']['content']
+    except Exception as e:
+        return f"ERROR: {e}"
+
+
+def call_glm(prompt: str, api_key: str, model: str = "glm-4-flash",
+             temperature: float = 0.7) -> str:
+    """Call GLM/Zhipu API (OpenAI-compatible)."""
+    url = "https://open.bigmodel.cn/api/paas/v4/chat/completions"
+    payload = {
+        "model": model,
+        "messages": [{"role": "user", "content": prompt}],
+        "max_tokens": 2048,
+        "temperature": temperature,
+    }
+    data = json.dumps(payload).encode('utf-8')
+    req = urllib.request.Request(url, data=data,
+                                headers={"Content-Type": "application/json",
+                                         "Authorization": f"Bearer {api_key}"},
+                                method='POST')
+    try:
+        with urllib.request.urlopen(req, timeout=120) as resp:
+            result = json.loads(resp.read().decode())
+            return result['choices'][0]['message']['content']
+    except Exception as e:
+        return f"ERROR: {e}"
+
+
+def call_ollama(prompt: str, model: str = "qwen2.5-coder:32b",
+                temperature: float = 0.7) -> str:
+    """Call local Ollama."""
+    url = "http://localhost:11434/api/generate"
+    payload = {
+        "model": model,
+        "prompt": prompt,
+        "stream": False,
+        "options": {"temperature": temperature, "num_predict": 2048},
+    }
+    data = json.dumps(payload).encode('utf-8')
+    req = urllib.request.Request(url, data=data,
+                                headers={"Content-Type": "application/json"},
+                                method='POST')
+    try:
+        with urllib.request.urlopen(req, timeout=180) as resp:
+            result = json.loads(resp.read().decode())
+            return result.get('response', '')
+    except Exception as e:
+        return f"ERROR: {e}"
+
+
+def call_llm(prompt: str, provider: str, api_key: str = "",
+             model: str = "", temperature: float = 0.7) -> str:
+    """Unified LLM caller."""
+    if provider == "nvidia":
+        return call_nvidia(prompt, api_key, model or "deepseek-ai/deepseek-v4-pro", temperature)
+    elif provider == "gemini":
+        return call_gemini(prompt, api_key, model or "gemini-2.0-flash", temperature)
+    elif provider == "deepseek":
+        return call_deepseek(prompt, api_key, model or "deepseek-chat", temperature)
+    elif provider == "glm":
+        return call_glm(prompt, api_key, model or "glm-4-flash", temperature)
+    elif provider == "ollama":
+        return call_ollama(prompt, model or "qwen2.5-coder:32b", temperature)
+    else:
+        return f"ERROR: Unknown provider {provider}"
+
+
+# =============================================================================
+# PROMPT, EXTRACTION, VERIFICATION (same as before)
+# =============================================================================
+
+def build_prompt(task: Dict) -> str:
+    train_pairs = task.get('train', [])
+    examples = []
+    for i, pair in enumerate(train_pairs):
+        examples.append(
+            f"Example {i+1}:\n"
+            f"  Input:  {json.dumps(pair['input'])}\n"
+            f"  Output: {json.dumps(pair['output'])}"
+        )
+    examples_str = "\n".join(examples)
+
+    inputs = [np.array(p['input']) for p in train_pairs]
+    outputs = [np.array(p['output']) for p in train_pairs]
+    same_shape = all(i.shape == o.shape for i, o in zip(inputs, outputs))
+    in_colors = sorted(set(c for i in inputs for c in np.unique(i).tolist()))
+    out_colors = sorted(set(c for o in outputs for c in np.unique(o).tolist()))
+
+    analysis = f"  Same input/output shape: {same_shape}\n"
+    analysis += f"  Input colors: {in_colors}, Output colors: {out_colors}\n"
+    if not same_shape:
+        for i, o in zip(inputs[:1], outputs[:1]):
+            analysis += f"  Shape: {i.shape} -> {o.shape}\n"
+
+    return f"""Solve this ARC-AGI puzzle. Write ONLY a Python function, no explanations.
+
+{examples_str}
+
+Analysis:
+{analysis}
+```python
+import numpy as np
+from collections import Counter, deque
+from scipy.ndimage import label
+
+def transform(grid: list[list[int]]) -> list[list[int]]:
+    grid = np.array(grid)
+"""
+
+
+def extract_code(response: str) -> Optional[str]:
+    for pattern in [r'```python\s*(.*?)```', r'```\s*(.*?)```']:
+        matches = re.findall(pattern, response, re.DOTALL)
+        for match in matches:
+            if 'def transform' in match:
+                return match.strip()
+    idx = response.find('def transform')
+    if idx >= 0:
+        before = response[:idx]
+        import_start = max(before.rfind('import '), before.rfind('from '))
+        start = import_start if import_start >= 0 else idx
+        code = response[start:]
+        end = code.find('```')
+        if end > 0:
+            code = code[:end]
+        return code.strip()
+    stripped = response.strip()
+    if stripped.startswith(('import', 'def transform', 'from')):
+        return stripped
+    return None
+
+
+def verify_program(code: str, train_pairs: List[Dict]) -> bool:
+    namespace = {'np': np, 'numpy': np, 'Counter': Counter,
+                 'deque': __import__('collections').deque}
+    try:
+        # Allow scipy import in generated code
+        try:
+            import scipy.ndimage
+            namespace['scipy'] = __import__('scipy')
+        except ImportError:
+            pass
+        exec(code, namespace)
+    except Exception:
+        return False
+    if 'transform' not in namespace:
+        return False
+    fn = namespace['transform']
+    for pair in train_pairs:
+        try:
+            result = fn([row[:] for row in pair['input']])
+            if result is None:
+                return False
+            r = np.array(result, dtype=int)
+            e = np.array(pair['output'], dtype=int)
+            if r.shape != e.shape or not np.array_equal(r, e):
+                return False
+        except Exception:
+            return False
+    return True
+
+
+def apply_program(code: str, test_input):
+    namespace = {'np': np, 'numpy': np, 'Counter': Counter,
+                 'deque': __import__('collections').deque}
+    try:
+        import scipy.ndimage
+        namespace['scipy'] = __import__('scipy')
+    except ImportError:
+        pass
+    try:
+        exec(code, namespace)
+        result = namespace['transform']([row[:] for row in test_input])
+        if result is not None:
+            return np.array(result, dtype=int).tolist()
+    except Exception:
+        pass
+    return None
+
+
+# =============================================================================
+# SYNTHESIS + MAIN
+# =============================================================================
+
+def synthesize_task(task, provider, api_key, model, n_candidates=8, verbose=False):
+    prompt = build_prompt(task)
+    for i in range(n_candidates):
+        temp = 0.1 if i == 0 else min(0.4 + 0.15 * i, 1.2)
+        response = call_llm(prompt, provider, api_key, model, temp)
+        if response.startswith("ERROR:"):
+            if verbose: print(f"    C{i+1}: {response[:60]}")
+            # Rate limit — wait and retry
+            if "429" in response or "rate" in response.lower():
+                time.sleep(5)
+            continue
+        code = extract_code(response)
+        if code is None:
+            if verbose: print(f"    C{i+1}: no code")
+            continue
+        if verbose: print(f"    C{i+1}: {len(code)}ch", end="")
+        if verify_program(code, task['train']):
+            if verbose: print(" ✅")
+            return (f"llm_c{i+1}", code)
+        else:
+            if verbose: print(" ❌")
+    return None
+
+
+def main():
+    PROVIDER = os.environ.get("LLM_PROVIDER", "gemini")
+    config = PROVIDERS.get(PROVIDER, {})
+    API_KEY = os.environ.get(config.get("env_key", ""), "") if config.get("env_key") else ""
+    MODEL = os.environ.get("LLM_MODEL", config.get("default_model", ""))
+    N_CANDIDATES = int(os.environ.get("N_CANDIDATES", "8"))
+    ARC_DIR = os.environ.get("ARC_DIR", "arc_data/training")
+    ALREADY_SOLVED = os.environ.get("ALREADY_SOLVED", "already_solved.json")
+    OUTPUT = os.environ.get("OUTPUT_FILE", "llm_results.json")
+
+    print("=" * 60)
+    print(f"PEMF ARC-AGI — LLM Synthesis ({config.get('name', PROVIDER)})")
+    print("=" * 60)
+    print(f"Provider: {PROVIDER}")
+    print(f"Model: {MODEL}")
+    print(f"Candidates/task: {N_CANDIDATES}")
+    if not API_KEY and PROVIDER != "ollama":
+        print(f"\n⚠️  No API key! Set {config.get('env_key', '???')}")
+        print(f"   Get key: {config.get('get_key_url', '?')}")
+        return
+    print()
+
+    # Load already solved
+    already_solved = set()
+    if os.path.exists(ALREADY_SOLVED):
+        with open(ALREADY_SOLVED) as f:
+            already_solved = set(json.load(f))
+        print(f"Symbolic solved: {len(already_solved)}")
+
+    # Load tasks
+    task_files = sorted(glob.glob(os.path.join(ARC_DIR, "*.json")))
+    unsolved = [(os.path.basename(tf).replace('.json',''), tf)
+                for tf in task_files
+                if os.path.basename(tf).replace('.json','') not in already_solved]
+    print(f"Total tasks: {len(task_files)}, unsolved: {len(unsolved)}")
+    print()
+
+    # Run
+    results = {}
+    solved = 0
+    total_time = 0
+
+    for idx, (tid, tf) in enumerate(unsolved):
+        with open(tf) as f:
+            task = json.load(f)
+        print(f"[{idx+1:3d}/{len(unsolved)}] {tid}:", end=" ", flush=True)
+        start = time.time()
+        result = synthesize_task(task, PROVIDER, API_KEY, MODEL, N_CANDIDATES, verbose=False)
+        elapsed = time.time() - start
+        total_time += elapsed
+
+        if result:
+            rule, code = result
+            solved += 1
+            test_outputs = [apply_program(code, t['input']) for t in task.get('test', [])]
+            results[tid] = {'status': 'solved', 'rule': rule, 'code': code,
+                           'test_outputs': test_outputs, 'time_s': round(elapsed, 2)}
+            print(f"✅ ({elapsed:.1f}s)")
+        else:
+            results[tid] = {'status': 'failed', 'time_s': round(elapsed, 2)}
+            print(f"❌ ({elapsed:.1f}s)")
+
+        # Rate limit respect
+        if PROVIDER == "gemini":
+            time.sleep(4)  # 15 RPM = 1 every 4s
+        elif PROVIDER == "nvidia":
+            time.sleep(2)  # NIM free tier: ~30 RPM
+        elif PROVIDER in ("deepseek", "glm"):
+            time.sleep(1)
+
+        # Save every 10
+        if (idx + 1) % 10 == 0:
+            _save(OUTPUT, PROVIDER, MODEL, N_CANDIDATES, solved, idx+1,
+                  total_time, already_solved, len(task_files), results)
+            print(f"  [Saved: {solved}/{idx+1}, total {len(already_solved)+solved}/{len(task_files)}]")
+
+    # Final save
+    _save(OUTPUT, PROVIDER, MODEL, N_CANDIDATES, solved, len(unsolved),
+          total_time, already_solved, len(task_files), results)
+
+    print(f"\n{'='*60}")
+    print(f"LLM solved:      {solved}/{len(unsolved)}")
+    print(f"Symbolic:        {len(already_solved)}")
+    print(f"TOTAL:           {len(already_solved)+solved}/{len(task_files)} ({100*(len(already_solved)+solved)/len(task_files):.1f}%)")
+    print(f"Saved: {OUTPUT}")
+
+
+def _save(path, provider, model, n_cand, solved, attempted, total_time,
+          already_solved, total_tasks, results):
+    with open(path, 'w') as f:
+        json.dump({
+            'provider': provider, 'model': model, 'n_candidates': n_cand,
+            'llm_solved': solved, 'attempted': attempted,
+            'total_time_s': round(total_time, 1),
+            'symbolic_solved': len(already_solved),
+            'total_solved': len(already_solved) + solved,
+            'total_tasks': total_tasks,
+            'solve_rate': round(100*(len(already_solved)+solved)/total_tasks, 2),
+            'results': results,
+        }, f, indent=2)
+
+
+if __name__ == "__main__":
+    main()

pemf/scripts/merge_results.py

@@ -0,0 +1,53 @@
+"""
+Merge LLM results with symbolic results to get final solve count.
+
+Usage:
+  python merge_results.py arc_results/summary_v4.json llm_results.json
+"""
+import json
+import sys
+
+
+def merge(symbolic_file: str, llm_file: str, output_file: str = "arc_results/summary_final.json"):
+    with open(symbolic_file) as f:
+        symbolic = json.load(f)
+    with open(llm_file) as f:
+        llm = json.load(f)
+
+    symbolic_solved = {r['task_id'] for r in symbolic['results'] if r.get('all_train_solved')}
+    llm_solved = {tid for tid, r in llm['results'].items() if r['status'] == 'solved'}
+
+    total_solved = symbolic_solved | llm_solved
+    new_from_llm = llm_solved - symbolic_solved
+
+    print(f"Symbolic solved: {len(symbolic_solved)}")
+    print(f"LLM solved:      {len(llm_solved)}")
+    print(f"New from LLM:    {len(new_from_llm)}")
+    print(f"TOTAL SOLVED:    {len(total_solved)}/{symbolic['total_tasks']} ({100*len(total_solved)/symbolic['total_tasks']:.1f}%)")
+
+    print(f"\nNew tasks solved by LLM:")
+    for tid in sorted(new_from_llm):
+        rule = llm['results'][tid].get('rule', '?')
+        print(f"  {tid}: {rule}")
+
+    # Save merged
+    merged = {
+        'total_tasks': symbolic['total_tasks'],
+        'symbolic_solved': len(symbolic_solved),
+        'llm_solved': len(llm_solved),
+        'new_from_llm': len(new_from_llm),
+        'total_solved': len(total_solved),
+        'solve_rate': round(100 * len(total_solved) / symbolic['total_tasks'], 2),
+        'symbolic_tasks': sorted(symbolic_solved),
+        'llm_tasks': sorted(llm_solved),
+        'new_llm_tasks': sorted(new_from_llm),
+    }
+    with open(output_file, 'w') as f:
+        json.dump(merged, f, indent=2)
+    print(f"\nMerged results saved to {output_file}")
+
+
+if __name__ == "__main__":
+    sym = sys.argv[1] if len(sys.argv) > 1 else "arc_results/summary_v4.json"
+    llm = sys.argv[2] if len(sys.argv) > 2 else "llm_results.json"
+    merge(sym, llm)

pemf/scripts/run_all_arc.py

@@ -0,0 +1,183 @@
+"""
+Run the PEMF solver on all ARC-AGI tasks and report solve rates.
+
+For each task, the solver tries every training pair. A task is "solved"
+if the solver achieves σ=0 on ALL training pairs.
+
+Usage:
+  1. Download the ARC dataset into arc_data/training/:
+       git clone https://github.com/fchollet/ARC-AGI.git /tmp/arc
+       cp -r /tmp/arc/data/training arc_data/training
+  2. Run:
+       python scripts/run_all_arc.py
+
+Outputs:
+  arc_results/summary.json   — per-task results
+  arc_results/report.txt     — human-readable report
+"""
+import os, json, time, glob
+
+import numpy as np
+from itt_solver.solver_core import initialize_potential, sigma_l1
+from itt_solver.beam_logging import beam_minimize_with_log
+from itt_solver.experiment_driver import default_atomic_factory
+
+ARC_DIR = os.environ.get("ARC_DIR", "arc_data/training")
+OUT_DIR = os.environ.get("OUT_DIR", "arc_results")
+os.makedirs(OUT_DIR, exist_ok=True)
+
+PARAMS = {
+    'beam_width': 8,
+    'max_depth': 2,
+    'lock_coeff': 0.0,
+    'max_fraction': 1.0,
+    'use_symmetry': True,
+    'use_gravity': True,
+    'use_color_ops': True,
+    'boundary_source': 'target',
+}
+
+def solve_pair(inp, out, params):
+    """Run solver on one input→output pair. Returns (sigma, transform_name, time_s)."""
+    h, w = len(out), len(out[0])
+    task = {
+        'name': 'pair',
+        'input': inp,
+        'target': out,
+        'target_shape': (h, w),
+    }
+    atomic_lib = default_atomic_factory(params, task)
+    phi_in = initialize_potential(inp)
+    phi_target = initialize_potential(out)
+
+    start = time.time()
+    T_best, phi_best, states, sigmas, logs = beam_minimize_with_log(
+        phi_in, phi_target, atomic_lib,
+        beam_width=params['beam_width'],
+        max_depth=params['max_depth'],
+        lock_coeff=params['lock_coeff'],
+        max_fraction=params['max_fraction'],
+        allowed_symbols=list(range(10)),
+        enable_layer_minus_one=False,
+        boundary_source=params['boundary_source'],
+    )
+    elapsed = time.time() - start
+    final_sigma = float(sigmas[-1]) if sigmas else float('inf')
+    return final_sigma, repr(T_best), elapsed
+
+def run_all():
+    task_files = sorted(glob.glob(os.path.join(ARC_DIR, "*.json")))
+    print(f"Running solver on {len(task_files)} ARC training tasks...")
+    print(f"Params: beam_width={PARAMS['beam_width']}, max_depth={PARAMS['max_depth']}")
+    print()
+
+    results = []
+    solved_count = 0
+    partial_count = 0
+    total_time = 0
+
+    for ti, tf in enumerate(task_files):
+        task_id = os.path.basename(tf).replace('.json', '')
+        with open(tf) as fh:
+            task_data = json.load(fh)
+
+        train_pairs = task_data.get('train', [])
+        test_pairs = task_data.get('test', [])
+
+        pair_results = []
+        all_zero = True
+        best_sigma = float('inf')
+        best_transform = None
+
+        for pi, pair in enumerate(train_pairs):
+            sigma, transform, elapsed = solve_pair(pair['input'], pair['output'], PARAMS)
+            total_time += elapsed
+            pair_results.append({
+                'pair': pi, 'sigma': sigma,
+                'transform': transform, 'time_s': round(elapsed, 4),
+            })
+            if sigma > 0:
+                all_zero = False
+            if sigma < best_sigma:
+                best_sigma = sigma
+                best_transform = transform
+
+        test_results = []
+        test_solved = None
+        for pi, pair in enumerate(test_pairs):
+            if 'output' in pair:
+                sigma, transform, elapsed = solve_pair(pair['input'], pair['output'], PARAMS)
+                total_time += elapsed
+                test_results.append({
+                    'pair': pi, 'sigma': sigma,
+                    'transform': transform, 'time_s': round(elapsed, 4),
+                })
+                if test_solved is None:
+                    test_solved = True
+                if sigma > 0:
+                    test_solved = False
+
+        status = "SOLVED" if all_zero else "PARTIAL" if best_sigma < float('inf') and best_sigma > 0 else "FAILED"
+        if all_zero:
+            solved_count += 1
+        elif best_sigma < float('inf'):
+            partial_count += 1
+
+        results.append({
+            'task_id': task_id, 'status': status,
+            'train_pairs': len(train_pairs), 'all_train_solved': all_zero,
+            'best_sigma': best_sigma, 'best_transform': best_transform,
+            'pair_results': pair_results,
+            'test_results': test_results, 'test_solved': test_solved,
+        })
+
+        if (ti + 1) % 20 == 0 or all_zero:
+            marker = "✅" if all_zero else "  "
+            print(f"[{ti+1:3d}/{len(task_files)}] {task_id}: {status} (best σ={best_sigma:.1f}) {marker}")
+
+    failed_count = len(task_files) - solved_count - partial_count
+    print(f"\n{'='*60}")
+    print(f"RESULTS: {len(task_files)} tasks")
+    print(f"  SOLVED (σ=0 all train pairs): {solved_count} ({100*solved_count/len(task_files):.1f}%)")
+    print(f"  PARTIAL (σ>0 but finite):     {partial_count}")
+    print(f"  FAILED:                        {failed_count}")
+    print(f"  Total time: {total_time:.1f}s ({total_time/len(task_files):.2f}s/task)")
+
+    summary = {
+        'total_tasks': len(task_files), 'solved': solved_count,
+        'partial': partial_count, 'failed': failed_count,
+        'solve_rate': round(100 * solved_count / len(task_files), 2),
+        'params': PARAMS, 'total_time_s': round(total_time, 2),
+        'results': results,
+    }
+    with open(os.path.join(OUT_DIR, 'summary.json'), 'w') as fh:
+        json.dump(summary, fh, indent=2)
+
+    solved_tasks = [r for r in results if r['all_train_solved']]
+    print(f"\nSolved tasks:")
+    for r in solved_tasks:
+        print(f"  {r['task_id']}: {r['best_transform']}")
+
+    partial_tasks = sorted(
+        [r for r in results if not r['all_train_solved'] and r['best_sigma'] < float('inf')],
+        key=lambda r: r['best_sigma']
+    )
+    print(f"\nTop 20 closest-to-solving:")
+    for r in partial_tasks[:20]:
+        print(f"  {r['task_id']}: σ={r['best_sigma']:.1f} ({r['best_transform']})")
+
+    with open(os.path.join(OUT_DIR, 'report.txt'), 'w') as fh:
+        fh.write(f"PEMF Solver — ARC-AGI Training Set Results\n{'='*60}\n")
+        fh.write(f"Total tasks: {len(task_files)}\n")
+        fh.write(f"Solved:      {solved_count} ({100*solved_count/len(task_files):.1f}%)\n")
+        fh.write(f"Partial:     {partial_count}\nFailed:      {failed_count}\n")
+        fh.write(f"Time:        {total_time:.1f}s\n\n")
+        fh.write(f"Params: {json.dumps(PARAMS, indent=2)}\n\n")
+        fh.write(f"Solved tasks:\n")
+        for r in solved_tasks:
+            fh.write(f"  {r['task_id']}: {r['best_transform']}\n")
+
+    print(f"\nResults saved to {OUT_DIR}/")
+
+if __name__ == '__main__':
+    run_all()

pemf/tests/test_transforms.py

@@ -0,0 +1,156 @@
+"""
+Unit tests for all transforms in itt_solver.transforms.
+
+Usage:
+  python tests/test_transforms.py
+
+40 tests covering: Kronecker, mirror tiles, upscale, downscale, stack,
+rotate, reflect, color ops, gravity, crop, transpose, shifted tile,
+fill enclosed.
+"""
+import numpy as np
+from itt_solver import transforms as tr
+
+INP = np.array([[0,7,7],[7,7,7],[0,7,7]], dtype=float)
+
+tests_passed = 0
+tests_failed = 0
+
+def check(name, condition):
+    global tests_passed, tests_failed
+    if condition:
+        print(f"  ✅ {name}")
+        tests_passed += 1
+    else:
+        print(f"  ❌ {name}")
+        tests_failed += 1
+
+print("=== Kronecker Self-Similar ===")
+T = tr.KroneckerSelfSimilar()
+out = T.apply(INP)
+check("Output shape is 9x9", out.shape == (9, 9))
+check("σ=0 vs known target", np.array_equal(out, np.kron((INP!=0).astype(float), INP)))
+
+print("\n=== KroneckerSelfSimilarInv ===")
+T = tr.KroneckerSelfSimilarInv()
+out = T.apply(INP)
+check("Output shape is 9x9", out.shape == (9, 9))
+
+print("\n=== MirrorTileH ===")
+T = tr.MirrorTileH()
+out = T.apply(INP)
+check("Shape is 3x6", out.shape == (3, 6))
+check("Left half is input", np.array_equal(out[:, :3], INP))
+check("Right half is fliplr(input)", np.array_equal(out[:, 3:], np.fliplr(INP)))
+
+print("\n=== MirrorTileV ===")
+T = tr.MirrorTileV()
+out = T.apply(INP)
+check("Shape is 6x3", out.shape == (6, 3))
+check("Top half is input", np.array_equal(out[:3, :], INP))
+check("Bottom half is flipud(input)", np.array_equal(out[3:, :], np.flipud(INP)))
+
+print("\n=== MirrorTile4Way ===")
+T = tr.MirrorTile4Way()
+out = T.apply(INP)
+check("Shape is 6x6", out.shape == (6, 6))
+
+print("\n=== Upscale 2x ===")
+T = tr.Upscale(2)
+out = T.apply(INP)
+check("Shape is 6x6", out.shape == (6, 6))
+check("Top-left 2x2 block is INP[0,0]", np.all(out[:2, :2] == INP[0, 0]))
+
+print("\n=== Upscale 3x ===")
+T = tr.Upscale(3)
+out = T.apply(INP)
+check("Shape is 9x9", out.shape == (9, 9))
+check("Top-left 3x3 block is INP[0,0]", np.all(out[:3, :3] == INP[0, 0]))
+
+print("\n=== Downscale 2x ===")
+T = tr.Downscale(2)
+big = np.kron(INP, np.ones((2, 2)))
+out = T.apply(big)
+check("Downscale of upscaled recovers original", np.array_equal(out, INP))
+
+print("\n=== StackH 3 ===")
+T = tr.StackH(3)
+out = T.apply(INP)
+check("Shape is 3x9", out.shape == (3, 9))
+check("First third is input", np.array_equal(out[:, :3], INP))
+
+print("\n=== StackV 3 ===")
+T = tr.StackV(3)
+out = T.apply(INP)
+check("Shape is 9x3", out.shape == (9, 3))
+check("First third is input", np.array_equal(out[:3, :], INP))
+
+print("\n=== Rotate 90/180/270 ===")
+for k in [1, 2, 3]:
+    T = tr.Rotate(k)
+    out = T.apply(INP)
+    check(f"Rotate_{90*k} matches np.rot90", np.array_equal(out, np.rot90(INP, k)))
+
+print("\n=== Reflect h/v ===")
+T = tr.Reflect('h')
+check("Reflect_h matches flipud", np.array_equal(T.apply(INP), np.flipud(INP)))
+T = tr.Reflect('v')
+check("Reflect_v matches fliplr", np.array_equal(T.apply(INP), np.fliplr(INP)))
+
+print("\n=== RetainColor ===")
+T = tr.RetainColor(7)
+out = T.apply(INP)
+check("Only 7s remain", np.all(out[INP == 7] == 7))
+check("Non-7 positions are 0", np.all(out[INP != 7] == 0))
+
+print("\n=== RemoveColor ===")
+T = tr.RemoveColor(7)
+out = T.apply(INP)
+check("7s are removed", np.all(out[INP == 7] == 0))
+check("0s stay 0", np.all(out[INP == 0] == 0))
+
+print("\n=== InvertColors ===")
+T = tr.InvertColors()
+out = T.apply(INP)
+check("0→7 swap", np.all(out[INP == 0] == 7))
+check("7→0 swap", np.all(out[INP == 7] == 0))
+
+print("\n=== GravityDown ===")
+T = tr.GravityDown()
+col_in = np.array([[0,7,0],[0,0,7],[7,0,0]], dtype=float)
+out = T.apply(col_in)
+check("Col 0: 7 at bottom", out[2, 0] == 7 and out[0, 0] == 0 and out[1, 0] == 0)
+check("Col 1: 7 at bottom", out[2, 1] == 7 and out[0, 1] == 0)
+
+print("\n=== GravityUp ===")
+T = tr.GravityUp()
+out = T.apply(col_in)
+check("Col 0: 7 at top", out[0, 0] == 7 and out[1, 0] == 0 and out[2, 0] == 0)
+
+print("\n=== CropToContent ===")
+T = tr.CropToContent()
+padded = np.array([[0,0,0,0],[0,7,7,0],[0,7,7,0],[0,0,0,0]], dtype=float)
+out = T.apply(padded)
+check("Crops to 2x2", out.shape == (2, 2))
+check("All 7s", np.all(out == 7))
+
+print("\n=== Transpose ===")
+T = tr.Transpose()
+out = T.apply(INP)
+check("Shape is transposed", out.shape == (3, 3))
+check("Values match transpose", np.array_equal(out, INP.T))
+
+print("\n=== ShiftedTile ===")
+T = tr.tile_to_target_shifted(shift=(1, 1), tile_factor=3)
+out = T.apply(INP)
+check("Shape is 9x9", out.shape == (9, 9))
+check("Differs from vanilla tile", not np.array_equal(out, np.tile(INP, (3, 3))))
+
+print("\n=== FillEnclosedHarmonic ===")
+T = tr.FillEnclosedHarmonic()
+enclosed = np.array([[7,7,7],[7,0,7],[7,7,7]], dtype=float)
+out = T.apply(enclosed)
+check("Center hole filled", out[1, 1] == 7)
+
+print(f"\n{'='*50}")
+print(f"Results: {tests_passed} passed, {tests_failed} failed")