docs/PHASE2_IMPLEMENTATION.md

# Phase 2 Implementation Notes

Phase 2 goal: Curriculum PPO across easy, medium, and hard tasks with deterministic evaluation discipline.

## Implemented Components

- `rl/curriculum.py`
  - `CurriculumScheduler` with staged task sampling:
    - Stage 1 (0%-30%): easy only
    - Stage 2 (30%-70%): easy + medium
    - Stage 3 (70%-100%): all 3 tasks with configurable weights
- `rl/configs/curriculum.yaml`
  - curriculum fractions and weights
  - PPO hyperparameters for Phase 2
- `rl/train_ppo.py`
  - `--phase 2` training path wired to curriculum scheduler
  - default config path uses `rl/configs/curriculum.yaml`
  - backward compatibility fallback to `rl/configs/ppo_curriculum.yaml`
  - explicit CLI args: `--phase1-config`, `--phase2-config`
- `tests/test_curriculum.py`
  - stage transitions
  - stage-1 easy-only enforcement
  - stage-3 all-task sampling
  - deterministic task seed invariants

## Operational Notes

- Existing 28-action design is preserved.
- Existing task IDs and grader logic are unchanged.
- No files were deleted as part of structure cleanup.

## Commands (using existing .venv313)

- Train Phase 1:
  - `.\\.venv313\\Scripts\\python.exe -m rl.train_ppo --phase 1 --timesteps 200000 --n-envs 4 --seed 42`
- Train Phase 2:
  - `.\\.venv313\\Scripts\\python.exe -m rl.train_ppo --phase 2 --timesteps 500000 --n-envs 4 --seed 42 --phase2-config rl/configs/curriculum.yaml`
- Train Phase 2 (tuned continuation):
  - `.\\.venv313\\Scripts\\python.exe -m rl.train_ppo --phase 2 --timesteps 300000 --n-envs 4 --seed 42 --phase2-config rl/configs/curriculum_tuned.yaml`
- Evaluate trained model:
  - `.\\.venv313\\Scripts\\python.exe -m rl.evaluate --model results/best_model/phase2_final.zip --episodes 3`