docs: expand reward section, remove held-out mention

docs/writeup.md

@@ -29,8 +29,6 @@ Classical symbolic regression tools (GP, sparse regression) can do this, but the
 | 2 | Damped Pendulum | `d2theta/dt2 = -(g/L)*sin(theta) - b*dtheta` |
 | 2 | Spring-Mass | `d2x/dt2 = -(k/m)*x` |
 | 2 | Damped Spring | `d2x/dt2 = -(k/m)*x - (b/m)*dx` |
-| 3 *(held-out)* | Projectile with Drag | 2-D coupled ODE |
-| 3 *(held-out)* | Charged Particle in B-field | 2-D Lorentz force |
 
 Each episode: the environment samples random parameters and initial conditions, simulates a noisy trajectory, and presents it to the agent with a one-sentence hint. The agent proposes an equation + parameters; the environment simulates the hypothesis via `scipy.odeint` and computes per-step R².
 
@@ -40,20 +38,28 @@ Each episode: the environment samples random parameters and initial conditions,
 
 ## Reward Design
 
-Five components, summed per rollout:
+Five components are summed per rollout. All are computed from `scipy.odeint` output — no model-in-the-loop scoring.
 
-| Component | Weight | Signal |
-|-----------|--------|--------|
-| `match` | R² (linear) | Primary accuracy |
-| `match_dense` | √R² | Dense gradient near zero |
-| `correctness` | Binary at R² ≥ 0.70 | Cliff signal |
-| `simplicity` | 1 − operators/12 | Prefer shorter equations |
-| `format` | Parse + simulate succeeds | Syntactic validity |
+### `reward_match` — R² (coefficient of determination)
+The primary signal. Compares the agent's simulated trajectory against the observed one using R². R² = 1 means perfect fit; R² = 0 means no better than predicting the mean. This is a continuous, differentiable signal that gives GRPO a smooth gradient to follow.
 
-**Three reward-hacking mitigations discovered during development:**
-1. `format = 1` only if `odeint` completes without NaN (prevents explosive equations)
-2. `simplicity = 0` unless R² ≥ 0.10 (prevents `d2y/dt2 = 0` gaming)
-3. `progress` removed from GRPO set (was a duplicate of `match` in single-turn)
+### `reward_match_dense` — √R²
+R² is near-zero for most early rollouts, making the gradient vanishingly small. Taking the square root stretches the low end of the range: `√0.05 ≈ 0.22` instead of `0.05`. This gives GRPO a non-trivial gradient even when the model is far from the right equation, speeding up early learning.
+
+### `reward_correctness` — binary cliff at R² ≥ 0.70
+A binary 0/1 bonus that fires when the trajectory is "good enough". This creates a sharp cliff in reward space that the policy is incentivised to climb. In practice it helps push past local plateaus where `match` and `match_dense` flatten out — the model learns to specifically target the 0.70 threshold rather than settling for partial credit.
+
+### `reward_simplicity` — prefer shorter equations
+Computed as `1 − (operator_count / 12)`. Shorter equations that explain the data are physically preferable to long, overfit ones. **Gated on R² ≥ 0.10** — if the equation is wrong, simplicity doesn't score. Without this gate the model would learn to output `d2y/dt2 = 0` (zero operators, simplicity = 1) and farm 20% reward for a completely wrong trajectory.
+
+### `reward_format` — syntactic and numerical validity
+1.0 only if the output (a) parses against the whitelisted ODE grammar, **and** (b) `odeint` integrates it to completion without NaN or overflow. Without the NaN check, a valid-looking but explosive equation like `d2y/dt2 = exp(vy**10)` would earn format reward despite being physically nonsense.
+
+---
+
+### Why these five together
+
+The three correctness-shaped signals (`match`, `match_dense`, `correctness`) dominate the GRPO advantage, ensuring physical accuracy drives the gradient. `simplicity` prevents equation bloat. `format` ensures the output is always usable by the verifier. Together they make reward hacking significantly harder than any single signal would.
 
 ---