Add blog.md: accurate systems table, SFT+GRPO plots

docs/blog.md

@@ -0,0 +1,139 @@
+# PhysiX — Equation Discovery from Noisy Trajectories via RLVR
+
+**OpenEnv India Hackathon 2026 submission** · [Live Space](https://huggingface.co/spaces/Pratyush-01/physix-live) · [Trained Model](https://huggingface.co/Pratyush-01/physix-3b-rl) · [W&B Runs](https://wandb.ai/pratyush01/physix-live)
+
+---
+
+## The Problem
+
+Given a short, noisy trajectory of a physical system — positions and velocities over time — can a language model discover the underlying equation of motion?
+
+This is symbolic regression meets agentic RL. The task is hard because:
+- The equation space is combinatorially large
+- Noise means no trajectory perfectly matches any ODE
+- The agent has to learn to iterate — propose, simulate, compare residuals, refine
+
+Classical symbolic regression tools (GP, sparse regression) can do this, but they aren't language models and they can't use natural language hints. We train a 3B LLM to do it iteratively using RLVR.
+
+---
+
+## The Environment
+
+**PhysiXEnvironment** presents the agent with a noisy trajectory from a physical system and asks it to output an ODE that reproduces the motion. All reward comes from `scipy.odeint` — no LLM-as-judge.
+
+### Systems trained on (GRPO)
+
+Three systems were used for GRPO training:
+
+| System | Ground-truth ODE | Difficulty |
+|--------|-----------------|------------|
+| Free Fall | `d2y/dt2 = -g` | 1 param |
+| Simple Pendulum | `d2theta/dt2 = -(g/L)*sin(theta)` | transcendental |
+| Damped Spring | `d2x/dt2 = -(k/m)*x - (b/m)*dx` | damped oscillation, 3 params |
+
+Parameters and initial conditions are randomised per episode.
+
+The SFT warm-start used all 6 Tier 1–2 systems (above three plus Free Fall with Drag, Damped Pendulum, Spring-Mass) to give the model broad format coverage before RL focused it on the three above.
+
+### Episode flow
+
+Each episode:
+1. `reset()` → agent receives a noisy trajectory + a one-sentence hint
+2. Agent proposes an ODE in JSON: `{"equation": "...", "params": {...}}`
+3. Environment simulates the hypothesis via `scipy.odeint` and computes R²
+4. Agent receives residual summary in English + numeric reward breakdown
+5. Repeat up to 8 turns, or until R² ≥ 0.93
+
+---
+
+## Reward Design
+
+All reward is computed from `scipy.odeint` — no model-in-the-loop scoring. Five components:
+
+**R²** (coefficient of determination): R² = 1 is a perfect match, R² = 0 means no better than predicting the mean, R² < 0 is actively wrong.
+
+| Component | Formula | What it rewards | Why it's needed |
+|-----------|---------|-----------------|-----------------|
+| `match` | R² | Continuous fit quality | Primary learning signal |
+| `match_dense` | √R² | Same, stretched | R² ≈ 0 early on; √R² gives non-zero gradient (√0.05 ≈ 0.22) so GRPO isn't blind in early steps |
+| `correctness` | 1 if R² ≥ 0.70 else 0 | Binary "good enough" | Creates a cliff the policy climbs; helps escape plateaus where the dense signal flattens |
+| `simplicity` | 1 − operators/12, gated on R² ≥ 0.10 | Shorter equations | Without the gate, `d2y/dt2 = 0` scores simplicity = 1 for free despite being completely wrong |
+| `format` | 1 if parses **and** `odeint` succeeds without NaN | Valid, simulatable output | Without the NaN check, explosive equations like `exp(vy**10)` claim format reward |
+
+---
+
+## Training: SFT → GRPO
+
+### Why SFT first
+
+Qwen2.5-3B is a small, cold base model. Out of the box ~80% of its completions are LaTeX, prose, or malformed JSON that the verifier can't parse. GRPO needs *variance in reward* across rollouts to estimate advantages; if every rollout scores ~0 because nothing parses, the gradient is zero and nothing learns.
+
+SFT on 384 synthetic `(prompt, ground_truth)` pairs from the environment (64 per system × 6 systems) teaches the model the output format before RL begins. 4 epochs, ~5 min on L40S.
+
+After SFT: >90% of completions parse and simulate successfully — GRPO now has a signal to work with.
+
+### SFT results
+
+| SFT Loss (↓) |
+|:---:|
+| ![SFT loss](plots/sft_loss.png) |
+
+Loss drops from 2.66 → 1.68 over 4 epochs. The smooth monotonic descent confirms the format is being learned reliably with no overfitting signs.
+
+### GRPO
+
+- **Model:** Qwen/Qwen2.5-3B-Instruct + LoRA-32
+- **Systems:** free_fall, simple_pendulum, damped_spring
+- **Steps:** 200 (with early stopping on reward convergence)
+- **LR:** 1e-5
+- **Generations:** 4 per prompt
+- **Framework:** Unsloth + TRL GRPOTrainer
+
+LR selection:
+- `3e-6` → near-zero gradient for 67+ steps
+- `3e-5` → hit ceiling too fast (~250 steps)
+- `1e-5` → smooth, steadily rising curve ✓
+
+---
+
+## Results
+
+| SFT Loss (↓) | GRPO Loss (↓) |
+|:---:|:---:|
+| ![SFT loss](plots/sft_loss.png) | ![GRPO loss](plots/loss.png) |
+
+| GRPO Total Reward (↑) |
+|:---:|
+| ![reward](plots/reward.png) |
+
+| Per-component reward breakdown |
+|:---:|
+| ![reward components](plots/reward_components.png) |
+
+Key observations:
+- `reward_format` jumps to ~0.9 in the first 10 steps — the SFT warm-start establishes output format immediately
+- `reward_match_dense` (√R²) and `reward_correctness` climb from ~0.6 → ~0.95 over 200 steps
+- `reward_match` (raw R²) converges to ~0.95+ by step 150
+- Total mean reward rises from ~3.3 → ~4.8 (+45%) with ±1σ variance shrinking
+- GRPO loss near zero is **expected** — it's the KL regularisation term; the real signal is the reward curves
+
+Full runs: [wandb.ai/pratyush01/physix-live](https://wandb.ai/pratyush01/physix-live)
+
+---
+
+## What's Novel
+
+1. **Verifiable reward without a judge** — R² from `scipy.odeint` is ground truth, not a proxy
+2. **Iterative refinement loop** — the environment returns residual summaries in English so the agent can reason about what went wrong
+3. **Reward hacking case study** — three exploits found and patched during development (parse-but-crash, trivial-equation simplicity, progress signal duplication)
+4. **SFT → GRPO pipeline** — shows how a cold 3B model can be made RL-trainable in under 10 minutes of SFT
+
+---
+
+## Links
+
+- **Live demo:** https://huggingface.co/spaces/Pratyush-01/physix-live
+- **Trained model:** https://huggingface.co/Pratyush-01/physix-3b-rl
+- **Checkpoint repo:** https://huggingface.co/Pratyush-01/physix-3b-rl-ckpt
+- **Training notebook:** https://huggingface.co/spaces/Pratyush-01/physix-live/blob/main/train/physix_train_colab.ipynb
+- **W&B project:** https://wandb.ai/pratyush01/physix-live