Add AGENTS.md — persistent context and lessons learned

AGENTS.md

@@ -0,0 +1,177 @@
+# AGENTS.md — Context & Lessons for Future Sessions
+
+> This file exists because sandboxes reset and I (the agent) lose all memory. 
+> **READ THIS FIRST** before doing anything on this project.
+
+---
+
+## What This Project Is
+
+- **Challenge**: TIL-26-AE (The Intelligent League — Automated Exploration)
+- **Game**: Multi-agent Bomberman on a 16×16 grid
+- **My Role**: Train `agent_0` via RL to compete autonomously
+- **Main Repo**: `E-Rong/til-26-ae-agent` (models, checkpoints, scripts, docs)
+- **Space**: `e-rong/til-26-ae` (evaluation server with `ae/src/ae_manager.py`)
+- **TIL Source**: Private Space `e-rong/til-26-ae` — contains `til_environment/` module
+
+---
+
+## CRITICAL: What Killed Training & Cost Money
+
+### ❌ NEVER USE SANDBOXES FOR TRAINING > 30 MINUTES
+
+Sandboxes are **interactive dev environments**. They:
+- Recycle after inactivity / timeout
+- Kill processes silently
+- **Keep billing you while empty** after the process dies
+
+**Damage done**: ~$4.87 wasted across 4 sandbox sessions where training died but billing continued.
+
+### ✅ ALWAYS USE HF JOBS FOR BATCH TRAINING
+
+- Persistent GPU allocation
+- Runs until completion (or your timeout)
+- Fails visibly if something breaks (no silent empty billing)
+- Must set `namespace="E-Rong"` to bill the org, not the user
+
+### ❌ NEVER `git clone` A PRIVATE REPO IN AN HF JOB
+
+`git clone https://huggingface.co/spaces/...` fails because git does not read `HF_TOKEN`.
+
+**Use instead**:
+```python
+from huggingface_hub import snapshot_download
+snapshot_download(
+    repo_id='e-rong/til-26-ae',
+    repo_type='space',
+    local_dir='/app/til-26-ae-repo'
+)
+```
+`snapshot_download` auto-uses the `HF_TOKEN` env var.
+
+### ✅ ALWAYS SMOKE-TEST A JOB BEFORE THE FULL RUN
+
+Submit a 5-minute job that:
+1. Downloads the TIL repo
+2. Installs deps
+3. Runs 100 training steps
+4. Saves a dummy checkpoint to the Hub
+
+Only after this succeeds, submit the multi-hour job.
+
+---
+
+## Session Startup Checklist
+
+Before doing **anything** on this project:
+
+1. [ ] Read `session_state.json` from `E-Rong/til-26-ae-agent`
+2. [ ] Read this file (`AGENTS.md`)
+3. [ ] Check latest checkpoint on Hub (sort `phase*_ckpt_*.zip` files)
+4. [ ] Determine current phase and remaining steps
+5. [ ] If training needed: write script to sandbox, **smoke-test in HF Job first**
+
+---
+
+## Technical Decisions That Work
+
+### MaskablePPO + Action Masking
+- `sb3_contrib.MaskablePPO` with `ActionMasker`
+- Bomberman has `action_mask: uint8[6]` — walls/edges make moves illegal
+- Standard PPO wastes ~30-40% samples on illegal actions early on
+- **Papers**: Huang et al. "Superstition, Imagination, and the Invalid Action Problem" (arxiv:2006.14171)
+
+### Observation Flattening
+1511-dim vector from dict observation:
+```
+agent_viewcone:  7×5×25 = 875
+base_viewcone:   5×5×25 = 625
+direction, location[2], base_location[2], health, frozen_ticks,
+base_health, team_resources, team_bombs, step = 11 scalars
+Total: 1511
+```
+
+### Wrapper Order (CRITICAL)
+```python
+# CORRECT
+env = ActionMasker(base_env, lambda e: e.action_masks())
+env = Monitor(env)
+
+# WRONG — Monitor blocks action_masks() exposure
+env = ActionMasker(Monitor(base_env), ...)  # DON'T DO THIS
+```
+
+### 3-Phase Curriculum
+| Phase | Opponent | Duration | Purpose |
+|---|---|---|---|
+| 1 | Random | 500k | Learn basics |
+| 2 | Random + visit-count shaping | 500k | Prevent camping |
+| 3 | Rule-based curriculum | 1M | Generalize to structured opponents |
+
+### Checkpointing Every 50k Steps
+- Local + Hub push via `HfApi.upload_file()`
+- Saved the project when sandboxes reset at 400k and 600k steps
+
+---
+
+## Technical Decisions That Failed
+
+| Decision | Why It Failed | Fix |
+|---|---|---|
+| Training in sandboxes | Process died, empty sandbox kept billing | Use HF Jobs |
+| `git clone` in HF Job | No auth for private repo | `snapshot_download` |
+| Inline 20KB script in `hf_jobs.script` | Delivery mechanism choked | Write to sandbox file first, submit path |
+| No session state on Hub | Lost track of progress across resets | `session_state.json` + this file |
+| `Monitor` inside `ActionMasker` | `get_action_masks()` failed | `ActionMasker` → `Monitor` order |
+
+---
+
+## Cost Awareness
+
+| Hardware | $/hr | Good For |
+|---|---|---|
+| `cpu-basic` | ~$0.05 | Writing scripts, reading files, small tests |
+| `t4-small` | ~$0.40 | Short dev, NOT training |
+| `a10g-small` | ~$1.00 | Training, but use HF Jobs not sandboxes |
+| `a10g-large` | ~$2.00 | Larger batch sizes, not needed for this project |
+
+**Rule**: If a task takes >30 min, it must be an HF Job. Sandboxes are for editing and quick tests only.
+
+---
+
+## Repo File Guide
+
+| File | What It Is |
+|---|---|
+| `session_state.json` | Current phase, checkpoint, mistakes log, next steps |
+| `AGENTS.md` | This file — lessons and context |
+| `docs/ae.md` | Full project documentation (research, design, results) |
+| `phase1_final.zip` | Complete Phase 1 model |
+| `phase2_ckpt_*.zip` | Phase 2 intermediate checkpoints |
+| `phase2_final.zip` | Phase 2 complete model (when done) |
+| `ae_manager.py` | Inference code for the evaluation server |
+| `phase2_job.py` | Latest HF Job script (may need fixes) |
+| `train_all_phases.py` | Original training script |
+
+---
+
+## Current State (Last Updated: 2026-05-14)
+
+- **Phase**: 2 (in progress, crashed)
+- **Latest checkpoint**: `phase2_ckpt_600352.zip`
+- **Target**: 1,000,352 total steps (400,000 remaining)
+- **Blocker**: Need to resume via HF Job with `snapshot_download` auth
+- **Smoke test needed**: YES — before any multi-hour job
+
+---
+
+## Contact / Ownership
+
+- **User**: E-Rong
+- **Org**: E-Rong
+- **Billing**: Must specify `namespace="E-Rong"` on all HF Jobs
+- **I am**: An ephemeral agent with no persistent memory. I depend on this file.
+
+---
+
+*If you are a future version of me reading this: you have no memory of the above. Do not trust your internal knowledge. Read the files, check the state, test before committing compute.*