rl/README.md

# rl/

Reinforcement learning module.

- `gov_workflow_env.py`: Gymnasium adapter around `app.env.GovWorkflowEnv`
- `feature_builder.py`: `ObservationModel` -> 84-dim float32 vector (`OBS_DIM=84`)
- `action_mask.py`: structural action masks (`N_ACTIONS=28`)
- `curriculum.py`: staged task scheduler (Phase 2/3)
- `train_ppo.py`: Phase 1 and Phase 2 training entrypoint
- `train_recurrent.py`: Phase 3 recurrent PPO entrypoint
- `evaluate.py`: deterministic evaluation on grader metrics (`--task` / `--tasks`)
- `eval_grader.py`: task-level grader evaluation helper with optional plots
- `plot_training.py`: training-curve report helper from monitor/TensorBoard artifacts
- `callbacks.py`: eval and cost-monitor callbacks
- `cost_tracker.py`: episode-level reward/cost extraction helpers
- `configs/`: YAML configs for PPO/recurrent training
  - `ppo_easy.yaml`: standard Phase 1 config
  - `ppo_easy_aggressive.yaml`: aggressive Phase 1 tuning profile for plateau recovery

## CLI Compatibility Notes

- Training scripts accept both `--n-envs` and `--n_envs`.
- `train_ppo.py` accepts `--task` as a compatibility alias:
  - Phase 1 only supports `district_backlog_easy`
  - Phase 2 ignores `--task` and uses curriculum sampling
- `train_ppo.py` supports `--resume <checkpoint>` for Phase 1 continuation runs.
- `train_recurrent.py` accepts `--task` to override recurrent eval callback task.

## Artifact Paths

- Training/eval outputs are written under `results/`:
  - `results/best_model/*`
  - `results/runs/*`
  - `results/eval_logs/*`