docs: add initial project brief and architecture documentation for CyberSecurity_OWASP environment

00_PROJECT_BRIEF.md

@@ -0,0 +1,145 @@
+# 00_PROJECT_BRIEF.md
+
+# CyberSecurity_OWASP — Project Brief
+
+## 1. One-line summary
+
+`CyberSecurity_OWASP` is an OpenEnv reinforcement-learning environment where a **single LLM agent learns the full defensive workflow for OWASP access-control bugs**: understand the intended authorization policy, discover a broken access-control path in a local synthetic app, patch the code, and prove that the fix blocks unauthorized access without breaking valid user flows.
+
+## 2. Problem
+
+Broken access control remains one of the most important web-application security risks because the correct behavior is usually **application-specific**. Generic scanners can find some missing checks, but they often lack enough context to answer the real engineering question:
+
+> “Given this app’s policy, users, roles, tenants, routes, and data model, is this behavior intended or a security bug?”
+
+Modern LLMs can read code, reason about tests, and propose patches, but they still struggle with:
+
+- distinguishing intended public/feature behavior from accidental over-permission;
+- following authorization logic across routes, middleware, ORM queries, tenants, roles, and ownership checks;
+- validating that a patch fixes the bug without introducing regressions;
+- avoiding reward hacking when tests are visible or too narrow;
+- generalizing across app templates instead of memorizing one codebase.
+
+`CyberSecurity_OWASP` turns this into a trainable environment.
+
+## 3. What the environment trains
+
+The environment trains **one agent**, not a separate red-team and blue-team pair. The same model must perform the entire secure-repair loop:
+
+1. **Understand policy** — read the policy graph, user roles, route intent, tenant rules, and allowed operations.
+2. **Discover evidence** — use safe local requests, logs, route metadata, and visible tests to identify the likely access-control failure.
+3. **Patch** — edit application code, middleware, route guards, query filters, or policy mappings.
+4. **Validate** — run public tests, policy checks, and regression tests.
+5. **Submit** — final answer is judged by deterministic hidden tests and reward logic.
+
+## 4. Scope for MVP
+
+The MVP should focus on **OWASP A01: Broken Access Control** with ASVS-inspired access-control requirements.
+
+Initial scenario families:
+
+1. Missing route-level authorization check.
+2. Insecure direct object reference / object ownership bug.
+3. Cross-tenant data leakage.
+4. Role confusion: user/admin/support/editor boundary error.
+5. Client-side-only authorization assumption.
+6. Query filter omission in list/search/export endpoint.
+7. Over-broad update/delete permission.
+8. Feature route intentionally public, so the agent must not over-secure it.
+
+Recommended MVP size: **8 scenario families × 3 app templates × 25 seeds = 600 trainable scenarios**, with separate held-out families and hidden seeds for evaluation.
+
+## 5. Why this is useful
+
+This environment is useful because it targets a real gap between today’s scanners and useful defensive agents:
+
+- **Scanners detect patterns.** This environment trains policy-aware reasoning.
+- **Unit tests check known cases.** This environment includes hidden authorization invariants.
+- **Static repair can overfit.** This environment forces the model to preserve valid business behavior.
+- **One-app benchmarks are easy to memorize.** This environment compiles many equivalent-but-different apps from policy graphs, templates, route shapes, schema names, and hidden test seeds.
+
+The outcome is a model that becomes better at a practical DevSecOps workflow: safely reviewing and repairing authorization logic in small-to-medium web apps.
+
+## 6. What success looks like
+
+A successful submission should show **measurable reward improvement** and better held-out security behavior after RL training.
+
+### Minimum success criteria
+
+- Environment runs through OpenEnv `reset`, `step`, and `state` APIs.
+- Hosted on Hugging Face Spaces.
+- Provides a minimal GRPO/TRL or Unsloth training script.
+- Tracks training/eval metrics with Trackio or equivalent.
+- Shows reward curves and before/after agent behavior.
+- Uses deterministic reward as the primary reward source.
+- Keeps hidden tests hidden from the agent.
+
+### Target metrics
+
+| Metric | MVP target |
+|---|---:|
+| Valid episode completion rate | ≥ 85% |
+| Hidden authorization test pass rate | ≥ 65% after initial RL run |
+| Regression preservation rate | ≥ 80% |
+| Held-out scenario success lift vs base model | ≥ +15 percentage points |
+| Reward-hacking incidents found in eval | 0 critical |
+| Median patch size | ≤ 3 files changed |
+
+## 7. Core design principle
+
+The environment should reward **correct defensive repair**, not exploit creativity. The discovery stage exists only to help the agent gather enough local evidence to make a safe patch. The reward engine must never reward real-world misuse, data exfiltration, persistence, credential theft, or evasion behavior.
+
+## 8. Deliverables for engineers
+
+Initial implementation should produce:
+
+```text
+CyberSecurity_OWASP/
+├── 00_PROJECT_BRIEF.md
+├── 01_ARCHITECTURE.md
+├── README.md
+├── pyproject.toml
+├── openenv.yaml
+├── cybersecurity_owasp/
+│   ├── __init__.py
+│   ├── models.py
+│   ├── client.py
+│   ├── rewards.py
+│   ├── scenarios/
+│   │   ├── compiler.py
+│   │   ├── policy_graph.py
+│   │   ├── templates/
+│   │   └── seeds/
+│   ├── apps/
+│   │   ├── fastapi_basic/
+│   │   ├── express_basic/
+│   │   └── django_basic/
+│   ├── evals/
+│   │   ├── public_tests.py
+│   │   ├── hidden_invariants.py
+│   │   └── heldout_eval.py
+│   └── server/
+│       ├── environment.py
+│       ├── app.py
+│       ├── requirements.txt
+│       └── Dockerfile
+├── training/
+│   ├── train_grpo.py
+│   ├── rollout.py
+│   └── eval_before_after.py
+└── outputs/
+    ├── logs/
+    ├── evals/
+    └── reward_curves/
+```
+
+## 9. Source notes and credibility
+
+| Source | How it informs this project | Credibility |
+|---|---|---:|
+| OWASP Top 10 2025 / A01 Broken Access Control | Confirms current relevance of Broken Access Control as a top web-app risk. | 10/10 |
+| OWASP ASVS | Provides security-control requirements that can be translated into policy invariants and hidden tests. | 9.5/10 |
+| OpenEnv build/deploy docs | Defines the required OpenEnv structure: models, server, client, Docker, HF Spaces deployment. | 8.5/10 |
+| Hackathon judging criteria | Aligns deliverables with scoring: innovation, storytelling, reward improvement, and training pipeline. | 9/10 |
+| TRL/OpenEnv GRPO example | Shows a practical pattern for environment rollouts, reward functions, and Trackio logging. | 8/10 |
+

01_ARCHITECTURE.md

@@ -0,0 +1,479 @@
+# 01_ARCHITECTURE.md
+
+# CyberSecurity_OWASP — Architecture
+
+## 1. System goal
+
+`CyberSecurity_OWASP` is an OpenEnv environment for training a **single LLM policy** to perform a complete defensive authorization-repair workflow:
+
+```text
+Understand policy → discover local evidence → patch code → validate → submit
+```
+
+The environment is intentionally not a two-agent red-team/blue-team setup. The agent is one model with one trajectory. It must learn both sides of the defensive workflow: finding the policy violation and fixing it safely.
+
+## 2. Final architecture diagram
+
+```mermaid
+flowchart TB
+    %% =========================
+    %% Offline Build Layer
+    %% =========================
+    subgraph A[Offline Scenario Factory]
+        A1[Policy Graph Generator\nroles, users, tenants, ownership, route intent]
+        A2[App Template Library\nFastAPI, Express, Django MVP templates]
+        A3[Bug Injector\nmissing guard, IDOR, tenant leak, role confusion, query omission]
+        A4[Scenario Compiler\nmaterializes app + DB + public tests + hidden invariants]
+        A5[Split Manager\ntrain seeds, validation seeds, hidden held-out seeds]
+        A1 --> A4
+        A2 --> A4
+        A3 --> A4
+        A5 --> A4
+    end
+
+    %% =========================
+    %% OpenEnv Runtime
+    %% =========================
+    subgraph B[CyberSecurity_OWASP OpenEnv Server]
+        B1[reset\(\)\nselect scenario + start sandbox]
+        B2[Sandbox App Runtime\nlocal app, DB fixture, logs, route map]
+        B3[Tool API exposed through step\(action\)\nReadFile, ListRoutes, SendLocalRequest, RunTests, ApplyPatch, SubmitFix]
+        B4[State Store\nepisode_id, step_count, scenario_id, patch diff, test history]
+        B5[Deterministic Reward Engine\npolicy tests + hidden tests + regression tests + penalties]
+        B6[state\(\)\nstructured metadata for debugging/eval]
+        B1 --> B2
+        B2 --> B3
+        B3 --> B4
+        B4 --> B5
+        B4 --> B6
+    end
+
+    %% =========================
+    %% Agent + Training
+    %% =========================
+    subgraph C[Single LLM Agent]
+        C1[Observation Parser]
+        C2[Planner\npolicy reasoning + patch strategy]
+        C3[Action Generator\nchooses next OpenEnv action]
+        C1 --> C2 --> C3
+    end
+
+    subgraph D[Training + Evaluation]
+        D1[Rollout Loop\nreset → step* → final reward]
+        D2[GRPO / TRL / Unsloth Training]
+        D3[Trackio Metrics\nreward curves, pass rates, patch size, steps]
+        D4[Held-out Eval Suite\nunseen templates, seeds, names, route structures]
+        D5[Demo Artifacts\nbefore/after traces, mini-blog, 2-minute video]
+        D1 --> D2 --> D3
+        D3 --> D4 --> D5
+    end
+
+    A4 --> B1
+    C3 -->|typed action| B3
+    B3 -->|observation + reward + done| C1
+    B5 --> D1
+    D2 --> C1
+    B5 --> D4
+```
+
+## 3. Component responsibilities
+
+### 3.1 Scenario Factory
+
+The scenario factory generates many small but realistic web apps from a structured authorization policy.
+
+It should output:
+
+- application code;
+- route map;
+- database fixture;
+- user/session/token fixtures;
+- policy graph;
+- intentionally injected access-control bug;
+- public tests visible to the agent;
+- hidden tests invisible to the agent;
+- metadata for eval and debugging.
+
+The scenario compiler is the main anti-overfitting mechanism. It should vary:
+
+- route names;
+- schema names;
+- ORM query structure;
+- framework template;
+- role names;
+- tenant IDs;
+- object ownership patterns;
+- file layout;
+- visible test coverage;
+- hidden invariant seeds.
+
+### 3.2 Policy Graph Generator
+
+The policy graph is the ground truth for intended behavior.
+
+Example internal representation:
+
+```yaml
+resources:
+  invoice:
+    owner_field: owner_user_id
+    tenant_field: tenant_id
+roles:
+  user:
+    can:
+      - read:invoice where owner_user_id == actor.user_id
+      - update:invoice where owner_user_id == actor.user_id and status != locked
+  support:
+    can:
+      - read:invoice where tenant_id == actor.tenant_id
+  admin:
+    can:
+      - read:any_invoice where tenant_id == actor.tenant_id
+      - update:any_invoice where tenant_id == actor.tenant_id
+public_routes:
+  - GET /health
+  - GET /pricing
+forbidden:
+  - cross_tenant_read
+  - cross_tenant_update
+  - user_reads_other_user_invoice
+```
+
+The policy graph prevents false rewards for over-securing intentionally public or intentionally allowed routes.
+
+### 3.3 Bug Injector
+
+The bug injector creates controlled, defensive lab scenarios. It should only generate bugs inside local synthetic apps.
+
+MVP bug classes:
+
+| Bug class | Example failure mode | Expected fix type |
+|---|---|---|
+| Missing route guard | Protected endpoint lacks authorization middleware | Add policy check/middleware |
+| IDOR / ownership bug | User can access another user’s object by changing ID | Add owner check in query/policy |
+| Tenant leak | Tenant A can list Tenant B records | Add tenant filter |
+| Role confusion | Support/editor/admin boundary is wrong | Correct role-to-permission mapping |
+| Client-side-only auth | Server trusts UI to hide forbidden action | Enforce server-side authorization |
+| Query omission | List/export/search endpoint lacks auth filter | Filter query by actor permissions |
+| Over-broad mutation | User can update/delete forbidden object | Add mutation permission check |
+| Public route decoy | Agent may wrongly lock down intended public endpoint | Preserve intended public behavior |
+
+### 3.4 OpenEnv Server
+
+The OpenEnv server should implement the standard lifecycle:
+
+- `reset()` — initialize a fresh scenario instance.
+- `step(action)` — execute one typed action and return observation, reward, and done.
+- `state()` — expose episode metadata for debugging and evaluation.
+
+Recommended package/class names:
+
+```text
+Repo name:      CyberSecurity_OWASP
+Python package: cybersecurity_owasp
+Client class:   CyberSecurityOWASPEnv
+Action class:   CyberSecurityOWASPAction
+Observation:    CyberSecurityOWASPObservation
+State:          CyberSecurityOWASPState
+```
+
+### 3.5 Tool API
+
+The agent should interact through typed actions. Keep the interface small enough for RL but expressive enough for realistic repair.
+
+```python
+@dataclass
+class CyberSecurityOWASPAction(Action):
+    action_type: Literal[
+        "read_file",
+        "list_files",
+        "list_routes",
+        "inspect_policy",
+        "send_local_request",
+        "run_public_tests",
+        "apply_patch",
+        "submit_fix",
+    ]
+    arguments: dict
+```
+
+Recommended actions:
+
+| Action | Purpose | Safety boundary |
+|---|---|---|
+| `inspect_policy` | Read intended authorization rules. | Only synthetic policy. |
+| `list_routes` | See local app route map. | No internet target. |
+| `read_file` | Inspect selected source file. | Sandbox allowlist only. |
+| `send_local_request` | Validate behavior against local app. | Local generated app only. |
+| `run_public_tests` | Run visible tests. | No hidden test disclosure. |
+| `apply_patch` | Modify source through unified diff. | Patch size and file allowlist limits. |
+| `submit_fix` | End episode and trigger hidden eval. | Final hidden score only, no leaked test details. |
+
+### 3.6 Observation schema
+
+Observations should be compact and structured.
+
+```python
+@dataclass
+class CyberSecurityOWASPObservation(Observation):
+    message: str
+    visible_policy_summary: str
+    route_summary: list[dict]
+    last_action_result: dict
+    public_test_summary: dict
+    patch_summary: dict
+    done_reason: str | None = None
+```
+
+Do not expose hidden test bodies, hidden expected outputs, or seed-specific solution hints.
+
+### 3.7 State schema
+
+State should support debugging and training analytics.
+
+```python
+@dataclass
+class CyberSecurityOWASPState(State):
+    episode_id: str
+    scenario_id: str
+    split: Literal["train", "validation", "heldout"]
+    step_count: int = 0
+    max_steps: int = 30
+    scenario_family: str = ""
+    app_template: str = ""
+    files_touched: list[str] = field(default_factory=list)
+    public_tests_passed: int = 0
+    public_tests_total: int = 0
+    hidden_tests_passed: int = 0
+    hidden_tests_total: int = 0
+    accumulated_reward: float = 0.0
+```
+
+## 4. Episode lifecycle
+
+```text
+1. reset()
+   - sample train/validation scenario seed
+   - compile app from policy graph + template + injected bug
+   - start local sandbox app and DB fixture
+   - return initial observation
+
+2. agent loop
+   - inspect policy/routes/files
+   - send local requests only inside sandbox
+   - run public tests
+   - apply one or more patches
+   - rerun public tests
+
+3. submit_fix
+   - freeze patch
+   - run public tests
+   - run hidden authorization invariants
+   - run regression tests
+   - compute deterministic reward
+   - return final observation, reward, done=True
+
+4. logging
+   - record scenario_id, action trace, patch diff, reward components
+   - send metrics to Trackio during training/eval
+```
+
+## 5. Reward design
+
+The reward should be deterministic, decomposed, and resistant to reward hacking.
+
+Recommended reward formula:
+
+```text
+R = 0.35 * public_policy_tests
+  + 0.30 * hidden_authz_invariants
+  + 0.15 * regression_preservation
+  + 0.10 * evidence_quality
+  + 0.05 * patch_minimality
+  + 0.05 * efficiency
+  - penalties
+```
+
+### Reward components
+
+| Component | Weight | What it rewards |
+|---|---:|---|
+| Public policy tests | 0.35 | Agent fixes known failing behavior. |
+| Hidden authz invariants | 0.30 | Patch generalizes beyond visible tests. |
+| Regression preservation | 0.15 | Valid user flows and intended public routes still work. |
+| Evidence quality | 0.10 | Agent gathered relevant policy/test/file evidence before patching. |
+| Patch minimality | 0.05 | Small focused patches instead of broad rewrites. |
+| Efficiency | 0.05 | Fewer wasted steps and repeated actions. |
+
+### Penalties
+
+| Penalty | Trigger |
+|---|---|
+| `-0.25` | Breaks public route intentionally marked public. |
+| `-0.25` | Deletes tests, policy file, or route instead of fixing authorization. |
+| `-0.20` | Hardcodes seed-specific IDs, users, tenants, or hidden assumptions. |
+| `-0.15` | Over-broad denial that blocks legitimate authorized users. |
+| `-0.10` | Patch exceeds file or diff-size budget. |
+| `-1.00` | Attempts external network access, credential extraction, persistence, or unsafe behavior. |
+
+The LLM judge, if used at all, should only annotate trace quality for analysis. It must not decide security-critical reward.
+
+## 6. Hidden tests and anti-overfitting
+
+Hidden tests are necessary because visible tests can be gamed or memorized. They should test policy invariants rather than exact implementation details.
+
+Use **4 anti-overfitting layers**:
+
+1. **Seed diversity** — route names, user IDs, tenant IDs, object names, and schemas change every episode.
+2. **Template diversity** — same policy bug appears in different frameworks and file layouts.
+3. **Hidden invariant tests** — final reward uses unseen authorization cases.
+4. **Held-out eval split** — at least 20% of scenario families/seeds are never used in training.
+
+Recommended split:
+
+```text
+Train:      70%
+Validation: 10%
+Held-out:   20%
+```
+
+## 7. Evaluation plan
+
+Run before/after evaluation on the same held-out suite.
+
+### Metrics
+
+| Metric | Meaning |
+|---|---|
+| `episode_success_rate` | Public + hidden + regression tests pass. |
+| `hidden_authz_pass_rate` | Security-critical hidden checks pass. |
+| `regression_pass_rate` | Normal valid behavior remains intact. |
+| `oversecure_rate` | Agent blocks intended legitimate/public behavior. |
+| `patch_compile_rate` | Patch applies and app still runs. |
+| `median_steps_to_submit` | Efficiency of the repair workflow. |
+| `median_files_changed` | Patch focus/minimality. |
+| `reward_hacking_rate` | Attempts to delete tests, hardcode fixtures, or bypass eval. |
+
+### Eval table template
+
+| Model | Split | Success | Hidden authz | Regression | Oversecure | Median steps | Median files changed |
+|---|---|---:|---:|---:|---:|---:|---:|
+| Base model | heldout | TBD | TBD | TBD | TBD | TBD | TBD |
+| RL-trained model | heldout | TBD | TBD | TBD | TBD | TBD | TBD |
+
+## 8. Training flow
+
+```text
+1. Build CyberSecurity_OWASP OpenEnv server.
+2. Generate 600 MVP scenarios.
+3. Run baseline eval with the base model.
+4. Train with GRPO/TRL or Unsloth using rollout episodes.
+5. Log reward components to Trackio.
+6. Run held-out eval every N training steps.
+7. Inspect failure clusters.
+8. Add scenario mutations only if failures reveal overfitting.
+9. Produce final demo: before/after trace + reward curve + held-out eval table.
+```
+
+Recommended initial training setup:
+
+```text
+Model: Qwen/Qwen3-1.7B or similar small instruct model
+Algorithm: GRPO via TRL or Unsloth-compatible loop
+Dataset prompt: repeated task instruction with randomized scenario IDs
+Max steps per episode: 30
+Rollouts per prompt: 2-4
+Logging: Trackio
+Primary eval: held-out deterministic test pass rate
+```
+
+## 9. Deployment architecture
+
+The environment should be runnable in 3 modes:
+
+| Mode | Purpose |
+|---|---|
+| Local Uvicorn | Fast engineer iteration. |
+| Docker | Reproducible local training/eval. |
+| Hugging Face Spaces | Public hackathon demo and OpenEnv-compliant hosting. |
+
+Expected endpoints:
+
+```text
+/ws       OpenEnv client session
+/health   health check
+/reset    debug reset
+/step     debug step
+/state    debug state
+/docs     FastAPI docs
+/web      optional web UI
+```
+
+## 10. Implementation milestones
+
+### Milestone 1 — Skeleton environment
+
+- `models.py`
+- `client.py`
+- `server/environment.py`
+- `server/app.py`
+- `server/Dockerfile`
+- `openenv.yaml`
+- health check
+- one hand-written scenario
+
+### Milestone 2 — Scenario compiler
+
+- policy graph format
+- app template renderer
+- bug injector
+- DB fixture generator
+- public and hidden test generator
+
+### Milestone 3 — Reward engine
+
+- public test score
+- hidden invariant score
+- regression score
+- patch minimality score
+- safety/reward-hacking penalties
+- reward component logging
+
+### Milestone 4 — Training script
+
+- rollout loop
+- GRPO/TRL or Unsloth training script
+- Trackio logging
+- checkpoint save/push
+- baseline and post-training eval
+
+### Milestone 5 — Hackathon demo
+
+- HF Spaces deployment
+- mini-blog
+- 2-minute video
+- before/after traces
+- reward curve
+- held-out eval table
+
+## 11. Engineering notes
+
+- Keep scenario apps small: ideally 5-15 files each.
+- Prefer deterministic tests over LLM judging.
+- Hide final hidden test details from observations.
+- Log enough trace data to debug failures but never leak hidden tests to the agent.
+- Include intentionally public routes and allowed cross-role cases so the model does not learn “add auth everywhere.”
+- The best demo is not just “agent finds bug,” but “agent learns not to break valid business behavior.”
+
+## 12. Source notes and credibility
+
+| Source | How it informs this architecture | Credibility |
+|---|---|---:|
+| OWASP Top 10 2025 / A01 Broken Access Control | Confirms why access control is the right security focus. | 10/10 |
+| OWASP ASVS access-control guidance | Informs policy invariants and server-side authorization checks. | 9.5/10 |
+| OpenEnv environment-building docs | Defines required models, reset/step/state, FastAPI server, Docker, and client. | 8.5/10 |
+| OpenEnv quickstart/architecture docs | Informs WebSocket client/server design, typed EnvClient, and container isolation. | 8.5/10 |
+| OpenEnv deployment docs | Informs HF Spaces deployment, endpoints, Docker workflow, and installable client package. | 8.5/10 |
+| Hackathon judging criteria | Informs demo priorities: innovation, storytelling, reward improvement, and training pipeline. | 9/10 |
+| TRL/OpenEnv training example | Informs rollout function, decomposed reward functions, and Trackio logging pattern. | 8/10 |
+

AGENTS.md

@@ -0,0 +1,1197 @@
+# AGENTS.md — CyberSecurity_OWASP Builder Instructions
+
+## Purpose
+
+This repository implements **CyberSecurity_OWASP**, an OpenEnv-compliant RL environment for training a **single LLM agent** to perform a defensive application-security workflow:
+
+```text
+inspect generated app + policy -> discover authorization bug -> submit safe finding -> patch code -> preserve intended behavior
+```
+
+The environment must train the model to do real interactive work, not answer static security questions. The model must act step by step through typed OpenEnv actions, observe consequences, receive deterministic reward, and improve through RL.
+
+The canonical repository and OpenEnv environment name is **`CyberSecurity_OWASP`**. Use this exact name in `openenv.yaml`, `pyproject.toml`, HF Spaces repo naming, Docker image tags, Trackio run names, command examples, and documentation.
+
+The target stack is:
+
+```text
+CyberSecurity_OWASP OpenEnv environment
+  -> deterministic verifier + hidden tests
+  -> rollout loop
+  -> HF TRL / Unsloth GRPO
+  -> Trackio logging
+  -> held-out evaluation
+  -> HF Spaces deployment
+```
+
+The final project must show measurable improvement in reward, exploit-block rate, regression-preservation rate, and held-out generalization after training.
+
+---
+
+## Product definition
+
+CyberSecurity_OWASP generates a new local application scenario every `reset(seed)`. Each episode contains:
+
+- a policy graph describing users, roles, tenants, resources, ownership, permissions, and public routes;
+- a generated FastAPI-style application workspace;
+- exactly one injected OWASP A01-style authorization defect;
+- visible tests for normal behavior;
+- hidden invariant tests for authorization correctness, regression protection, public-route preservation, and anti-cheat checks.
+
+The environment has **one LLM agent**, not separate red-team and blue-team LLMs. The environment itself acts as the scenario generator, tool server, verifier, and judge.
+
+---
+
+## Highest-priority objectives
+
+When making implementation decisions, optimize in this order:
+
+1. **Verifier correctness**: deterministic tests must decide whether the patch actually fixes the authorization defect.
+2. **Reward integrity**: reward must be hard to hack and must punish insecure or regressive patches.
+3. **Anti-overfitting**: the model must generalize across apps, layouts, policies, domains, names, and bug families.
+4. **OpenEnv compliance**: expose typed `Action`, `Observation`, and `State`; implement `reset()`, `step(action)`, and `state` correctly.
+5. **Trainability**: baseline model should sometimes get partial reward; curriculum should make early learning possible.
+6. **Real-world usefulness**: the workflow should resemble secure code review / AppSec authorization repair.
+7. **Demo clarity**: show before/after rollouts, reward curves, and why the trained model improved.
+8. **Hackathon competitiveness**: prioritize a novel, interactive, professionally useful environment with a coherent training pipeline.
+
+Do not train before the environment, verifier, anti-cheat tests, and before/after evaluation are stable.
+
+---
+
+## Hackathon alignment requirements
+
+The implementation must satisfy these minimum requirements:
+
+- use the latest OpenEnv release;
+- include a minimal HF TRL or Unsloth training script;
+- use Trackio as the default tracker for training and evaluation;
+- be deployable as an OpenEnv-compliant Hugging Face Space;
+- include a README / mini-blog style explanation;
+- show baseline-vs-trained improvement.
+
+Optimize for judging:
+
+| Criterion | Weight | CyberSecurity_OWASP evidence |
+|---|---:|---|
+| Environment innovation | 40% | procedural OWASP authorization-repair environment with generated code, policy, and hidden verifier |
+| Storytelling | 30% | single LLM learns discover + patch, before/after security behavior |
+| Showing improvement in rewards | 20% | reward curves, exploit-block pass rate, regression-preservation rate |
+| Reward/training pipeline | 10% | deterministic reward, GRPO/PPO rollout loop, Trackio metrics |
+
+---
+
+## Non-negotiable environment design
+
+CyberSecurity_OWASP must be a **single-agent** environment:
+
+```text
+phase = discover -> patch -> done
+```
+
+Do not implement a two-LLM red-team/blue-team setup. The single model must learn both discovery and repair.
+
+The environment must be defensive and local only. It must never target real systems or teach unauthorized exploitation. All probing must be limited to the generated local workspace controlled by the environment.
+
+---
+
+## Required repository structure
+
+Prefer this structure:
+
+```text
+.
+├── AGENTS.md
+├── README.md
+├── 00_PROJECT_BRIEF.md
+├── 01_ARCHITECTURE.md
+├── pyproject.toml
+├── openenv.yaml
+├── envs/
+│   └── CyberSecurity_OWASP/
+│       ├── __init__.py
+│       ├── models.py
+│       ├── client.py
+│       ├── README.md
+│       ├── rewards.py
+│       ├── validators.py
+│       ├── safety.py
+│       ├── evals.py
+│       ├── server/
+│       │   ├── __init__.py
+│       │   ├── app.py
+│       │   ├── environment.py
+│       │   ├── scenario_compiler.py
+│       │   ├── policy_graph.py
+│       │   ├── template_renderer.py
+│       │   ├── bug_mutator.py
+│       │   ├── fixture_generator.py
+│       │   ├── reward_engine.py
+│       │   ├── requirements.txt
+│       │   └── Dockerfile
+│       ├── templates/
+│       │   └── fastapi_basic/
+│       ├── scenario_cache/
+│       │   ├── train/
+│       │   ├── validation/
+│       │   └── hidden_eval/
+│       └── tests/
+│           ├── test_models.py
+│           ├── test_reset_step_state.py
+│           ├── test_rewards.py
+│           ├── test_anti_cheat.py
+│           ├── test_seed_reproducibility.py
+│           ├── test_invalid_actions.py
+│           └── test_rollouts.py
+├── training/
+│   ├── train_grpo.py
+│   ├── rollout.py
+│   ├── reward_funcs.py
+│   ├── eval_before_after.py
+│   ├── trackio_utils.py
+│   └── configs/
+│       └── grpo_small.yaml
+├── scripts/
+│   ├── run_local.sh
+│   ├── docker_build.sh
+│   ├── docker_run.sh
+│   ├── smoke_test.sh
+│   ├── generate_scenarios.sh
+│   └── push_space.sh
+├── assets/
+│   └── anti_overfitting_training_flow_diagram.png
+└── outputs/
+    ├── logs/
+    ├── evals/
+    └── rollouts/
+```
+
+If `openenv init CyberSecurity_OWASP` creates a different structure, preserve the generated structure and add the missing files around it.
+
+---
+
+## Architecture overview
+
+CyberSecurity_OWASP has 7 main components:
+
+1. **Policy Graph + Domain Sampler** — samples users, roles, tenants, ownership, public routes, and business exceptions.
+2. **Template / Framework Randomizer** — renders FastAPI-style apps with randomized layouts and naming.
+3. **A01 Bug Mutator** — injects one authorization defect per scenario.
+4. **Fixture + Hidden Test Generator** — creates users, resources, visible tests, and hidden invariant tests.
+5. **OpenEnv Server** — exposes typed `Action`, `Observation`, and `State` through `reset`, `step`, and `state`.
+6. **LLM Agent + LoRA** — one model performs discover + patch.
+7. **Deterministic Reward Engine** — hidden tests score exploit blocking, normal-flow preservation, patch quality, and anti-cheat.
+
+An optional LLM reviewer may score rationale quality and ASVS/OWASP mapping only. It must not provide the primary reward.
+
+---
+
+## Scenario compiler requirements
+
+### Policy Graph + Domain Sampler
+
+The policy graph is the source of truth. It must define:
+
+- users;
+- tenants;
+- roles;
+- resources;
+- ownership relationships;
+- role permissions;
+- public routes;
+- business exceptions.
+
+Initial domains:
+
+| Domain | Example resources | Example policy rule |
+|---|---|---|
+| invoices | invoices, payments, accounts | owner or billing admin can read invoice |
+| support | tickets, comments, customer records | assigned agent can update ticket |
+| projects | projects, documents, milestones | project member can read project docs |
+| marketplace | orders, returns, seller records | buyer owns own orders; seller owns own listings |
+| HR | employee profiles, reviews, payroll records | HR admin can read employee records |
+
+### Template / Framework Randomizer
+
+First version: FastAPI only. Still randomize structure so the model cannot memorize one app.
+
+Randomize:
+
+- path naming;
+- parameter names;
+- helper names;
+- folder layout;
+- route/service/auth split;
+- fixture names;
+- error messages within valid policy bounds.
+
+Examples:
+
+```text
+/routes/invoices.py
+/api/billing.py
+/controllers/accounts.py
+/services/access.py
+/authz/guards.py
+```
+
+### A01 Bug Mutator
+
+Inject exactly one primary bug per scenario.
+
+Initial bug families:
+
+| Bug family | Defect | Desired repair pattern |
+|---|---|---|
+| BOLA/IDOR | resource ID lookup lacks owner/tenant check | check server-side owner/tenant relation |
+| BFLA | privileged route lacks role/function check | add reusable role or permission guard |
+| tenant leak | request/header tenant ID is trusted | derive tenant from authenticated principal or server-side mapping |
+| JWT claim trust | mutable claim is treated as authoritative | verify against server-side user/role record |
+| public-route trap | route is intentionally public | do not over-secure public allowlisted route |
+
+### Fixture + Hidden Test Generator
+
+Visible tests should check that the app boots and normal happy paths work.
+
+Hidden tests must check:
+
+- exploit request is blocked;
+- legitimate owner flow still works;
+- legitimate admin/support flow still works;
+- public routes remain public;
+- cross-tenant access is denied;
+- randomized IDs/names defeat hardcoded patches;
+- hidden tests, fixtures, oracle, and reward files are not modified.
+
+### Scenario Cache + Seeded Reset
+
+Training should use cached scenarios for speed.
+
+Recommended first cache:
+
+| Split | Seeds | Purpose |
+|---|---:|---|
+| train | 500–1,000 | RL rollouts |
+| validation | 100–200 | checkpoint selection and curriculum signal |
+| hidden_eval | 100–200 | final generalization proof |
+
+### Hold-Out Generalization Splitter
+
+Hold out at least 4 dimensions:
+
+1. domains;
+2. policy graph shapes;
+3. code layouts;
+4. bug-family/domain combinations.
+
+Example: train on invoices/support/projects, evaluate on marketplace/HR.
+
+---
+
+## OpenEnv model definitions
+
+Implement these in `envs/CyberSecurity_OWASP/models.py`.
+
+```python
+from dataclasses import dataclass, field
+from typing import Any, Literal
+from openenv.core.env_server import Action, Observation, State
+
+CyberSecurityOWASPPhase = Literal["discover", "patch", "done"]
+CyberSecurityOWASPSplit = Literal["train", "validation", "hidden_eval"]
+
+@dataclass
+class CyberSecurityOWASPAction(Action):
+    tool_name: Literal[
+        "inspect_policy_graph",
+        "list_routes",
+        "read_openapi",
+        "read_file",
+        "search_code",
+        "send_local_request",
+        "compare_identities",
+        "submit_finding",
+        "patch_file",
+        "run_visible_tests",
+        "submit_fix",
+        "noop",
+    ]
+    arguments: dict[str, Any] = field(default_factory=dict)
+
+@dataclass
+class CyberSecurityOWASPObservation(Observation):
+    phase: CyberSecurityOWASPPhase
+    message: str
+    task_brief: str
+    visible_policy_hint: dict[str, Any] = field(default_factory=dict)
+    workspace_summary: dict[str, Any] = field(default_factory=dict)
+    available_actions: list[str] = field(default_factory=list)
+    last_tool_result: str = ""
+    last_action_valid: bool = True
+    last_action_error: str | None = None
+    visible_test_result: str | None = None
+    reward_breakdown: dict[str, float] = field(default_factory=dict)
+    done_reason: str | None = None
+
+@dataclass
+class CyberSecurityOWASPState(State):
+    episode_id: str = ""
+    task_id: str = ""
+    seed: int = 0
+    split: CyberSecurityOWASPSplit = "train"
+    difficulty: int = 0
+    domain: str = ""
+    bug_family: str = ""
+    phase: CyberSecurityOWASPPhase = "discover"
+    step_count: int = 0
+    max_steps: int = 40
+    done: bool = False
+    success: bool = False
+    failure_reason: str | None = None
+    finding_submitted: bool = False
+    patch_submitted: bool = False
+    accumulated_reward: float = 0.0
+    last_reward: float = 0.0
+    action_history: list[dict[str, Any]] = field(default_factory=list)
+    reward_history: list[dict[str, float]] = field(default_factory=list)
+    visible_facts: dict[str, Any] = field(default_factory=dict)
+    hidden_facts: dict[str, Any] = field(default_factory=dict)
+    metrics: dict[str, Any] = field(default_factory=dict)
+    anti_cheat_flags: list[str] = field(default_factory=list)
+```
+
+---
+
+## Action design and phase gating
+
+Actions must be explicit, typed, serializable, and constrained. Invalid actions must not crash the server.
+
+### Phase-gated tools
+
+| Phase | Allowed tools |
+|---|---|
+| discover | `inspect_policy_graph`, `list_routes`, `read_openapi`, `read_file`, `search_code`, `send_local_request`, `compare_identities`, `submit_finding`, `noop` |
+| patch | `read_file`, `search_code`, `patch_file`, `run_visible_tests`, `send_local_request`, `submit_fix`, `noop` |
+| done | no state-changing tools; return stable done observation |
+
+### Tool contracts
+
+`inspect_policy_graph`
+: Returns public policy hints. Must not reveal hidden bug labels or hidden tests.
+
+`list_routes`
+: Returns route method/path summaries from the generated app.
+
+`read_openapi`
+: Returns generated OpenAPI metadata.
+
+`read_file`
+: Reads editable workspace files only. Must block hidden tests, reward files, oracle files, and host files.
+
+`search_code`
+: Searches editable workspace files only.
+
+`send_local_request`
+: Sends a request to the local generated app only. Must block external URLs and host network access.
+
+`compare_identities`
+: Runs the same local request as two generated users and summarizes behavioral differences.
+
+`submit_finding`
+: Accepts structured evidence of the suspected authorization bug. Required before patch phase unless curriculum level explicitly allows blind patching.
+
+`patch_file`
+: Applies a bounded unified diff to editable app files only.
+
+`run_visible_tests`
+: Runs visible tests only. Must not run or reveal hidden tests.
+
+`submit_fix`
+: Triggers hidden evaluation.
+
+---
+
+## Observation rules
+
+Observations should provide enough information to act but must not leak the answer.
+
+Include:
+
+- current phase;
+- task brief;
+- visible policy hints;
+- workspace summary;
+- available tools;
+- previous tool output;
+- visible test output;
+- public reward breakdown after terminal evaluation.
+
+Do not include:
+
+- hidden bug family if not meant to be visible;
+- hidden test contents;
+- hidden oracle;
+- exact exploit path labels;
+- hidden seed split labels that allow memorization;
+- reward implementation details that allow proxy hacking.
+
+---
+
+## State rules
+
+State is the source of truth for deterministic replay and debugging.
+
+Required state properties:
+
+- `reset(seed)` must create a fresh independent state;
+- same seed + same action sequence should produce same result;
+- each WebSocket session must be isolated;
+- `step_count` increments once per processed action;
+- terminal states return stable done observations;
+- hidden facts never appear in observations;
+- all actions and reward breakdowns are stored for debugging.
+
+---
+
+## Environment API contract
+
+Implement in `envs/CyberSecurity_OWASP/server/environment.py`.
+
+```python
+from openenv.core.env_server import Environment
+from ..models import CyberSecurityOWASPAction, CyberSecurityOWASPObservation, CyberSecurityOWASPState
+
+class CyberSecurityOWASPEnvironment(Environment):
+    def __init__(self):
+        super().__init__()
+        self._state = CyberSecurityOWASPState()
+
+    def reset(self) -> CyberSecurityOWASPObservation:
+        ...
+
+    def step(self, action: CyberSecurityOWASPAction) -> CyberSecurityOWASPObservation:
+        ...
+
+    @property
+    def state(self) -> CyberSecurityOWASPState:
+        return self._state
+```
+
+`step(action)` must follow this order:
+
+1. If done, return stable done observation.
+2. Validate action and phase permissions.
+3. Increment step count.
+4. Execute the tool.
+5. Update state/history.
+6. Run verifier if `submit_finding`, `run_visible_tests`, or `submit_fix`.
+7. Compute reward components.
+8. Check terminal conditions.
+9. Return observation, reward, and done through OpenEnv step result handling.
+
+---
+
+## FastAPI server and client
+
+`envs/CyberSecurity_OWASP/server/app.py` must use the OpenEnv FastAPI helper.
+
+```python
+from openenv.core.env_server import create_fastapi_app
+from ..models import CyberSecurityOWASPAction, CyberSecurityOWASPObservation
+from .environment import CyberSecurityOWASPEnvironment
+
+env = CyberSecurityOWASPEnvironment()
+app = create_fastapi_app(env, CyberSecurityOWASPAction, CyberSecurityOWASPObservation)
+```
+
+`envs/CyberSecurity_OWASP/client.py` must parse `StepResult`, observations, rewards, done flags, and state into typed classes.
+
+---
+
+## Deterministic reward engine
+
+The reward engine is the task specification. Build it before training.
+
+Maximum reward: **15.0**.
+
+| Component | Reward |
+|---|---:|
+| valid finding with reproducible local evidence | +2.0 |
+| finding tied to violated policy rule | +1.0 |
+| hidden exploit blocked | +5.0 |
+| legitimate owner/admin/support flows preserved | +3.0 |
+| public routes preserved | +1.0 |
+| patch is localized, reusable, and policy-aligned | +2.0 |
+| visible tests pass and app boots | +1.0 |
+| exploit still works | -6.0 |
+| legitimate flow regression | -5.0 |
+| public route incorrectly locked | -3.0 |
+| hardcoded user/tenant/resource fix | -4.0 |
+| modified tests, fixtures, oracle, or hidden files | -8.0 |
+| app does not parse or boot | -6.0 |
+
+The reward breakdown must use stable keys:
+
+```python
+reward_breakdown = {
+    "discovery": discovery_reward,
+    "security": security_reward,
+    "regression": regression_reward,
+    "public_routes": public_route_reward,
+    "patch_quality": patch_quality_reward,
+    "visible_tests": visible_tests_reward,
+    "safety": safety_reward,
+    "anti_cheat": anti_cheat_reward,
+    "total": total_reward,
+}
+```
+
+Do not let explanation quality dominate the reward. If using an LLM reviewer, keep it separate from `total` or cap it at a small non-primary auxiliary signal.
+
+---
+
+## Verifier requirements
+
+Create `envs/CyberSecurity_OWASP/validators.py` and `envs/CyberSecurity_OWASP/server/reward_engine.py`.
+
+Required verifier functions:
+
+```python
+def verify_finding(state: CyberSecurityOWASPState, finding: dict) -> dict:
+    ...
+
+
+def run_hidden_security_tests(state: CyberSecurityOWASPState) -> dict:
+    ...
+
+
+def run_hidden_regression_tests(state: CyberSecurityOWASPState) -> dict:
+    ...
+
+
+def detect_cheating(state: CyberSecurityOWASPState, action: CyberSecurityOWASPAction) -> list[str]:
+    ...
+
+
+def compute_reward(state: CyberSecurityOWASPState, action: CyberSecurityOWASPAction, verifier_result: dict) -> dict[str, float]:
+    ...
+```
+
+Verifier tests must prove:
+
+- correct patch receives high reward;
+- exploit-only finding without patch does not complete the episode;
+- deny-all patch fails regression tests;
+- hardcoded patch fails randomized hidden tests;
+- modified hidden files produce anti-cheat penalty;
+- visible-test-only patch does not guarantee high reward;
+- repeated intermediate actions cannot inflate reward indefinitely.
+
+---
+
+## Anti-overfitting requirements
+
+CyberSecurity_OWASP must prevent overfitting to one app or scenario.
+
+Use all of these defenses:
+
+| Risk | Required defense |
+|---|---|
+| memorizes one app | many domains and templates |
+| memorizes route names | randomized path, resource, parameter, helper names |
+| memorizes bug location | vary route/service/auth layer placement |
+| learns deny-all patch | hidden positive-flow and public-route tests |
+| learns hardcoded patch | randomized users, tenants, resource IDs, role names |
+| overfits visible tests | hidden invariant tests and held-out eval |
+| overfits one bug family | curriculum-sampled bug mix |
+| overfits one code layout | hold out entire layouts and domains |
+| optimizes explanation only | deterministic reward is primary |
+
+Acceptance target: at least **20%** of domain/layout/bug combinations must be held out from training.
+
+---
+
+## Safety and cybersecurity boundaries
+
+This is a defensive AppSec training environment.
+
+Allowed:
+
+- local generated app probing;
+- authorization reasoning;
+- secure patching;
+- visible and hidden test execution;
+- policy-to-code mapping;
+- defensive vulnerability validation in sandbox.
+
+Forbidden:
+
+- real-world exploitation;
+- credential theft;
+- persistence/evasion/malware behavior;
+- scanning external targets;
+- bypassing real services;
+- writing exploit instructions for systems outside the local generated lab.
+
+`send_local_request` must only target the generated local app.
+
+---
+
+## Curriculum controller
+
+RL needs partial successes. Implement at least 3 difficulty levels.
+
+```text
+level_0: BOLA/IDOR, small app, direct route, obvious policy hint
+level_1: BFLA or tenant bug, moderate app, realistic distractors
+level_2: JWT trust or nested tenant/resource route, multiple files, false-positive traps
+level_3: held-out domain/layout/bug combo, harder naming, fewer hints
+```
+
+Curriculum signal:
+
+```text
+if exploit_block_rate < 60%:
+    increase level_0 and level_1 tasks
+elif regression_rate > 20%:
+    increase positive-flow and public-route traps
+elif public_route_false_positive_rate > 10%:
+    increase intentionally public route examples
+elif validation_reward plateaus:
+    increase unseen layouts and nested resources
+else:
+    increase difficulty by 1
+```
+
+---
+
+## Training requirements
+
+Create a runnable minimal training script using HF TRL or Unsloth.
+
+Required files:
+
+```text
+training/train_grpo.py
+training/rollout.py
+training/reward_funcs.py
+training/eval_before_after.py
+training/trackio_utils.py
+training/configs/grpo_small.yaml
+```
+
+Recommended first model:
+
+```text
+Qwen/Qwen3-1.7B
+```
+
+Acceptable alternatives:
+
+```text
+Qwen2.5-Coder-1.5B-Instruct
+Qwen2.5-Coder-3B-Instruct
+```
+
+Use LoRA / QLoRA. Do not full-finetune unless explicitly required.
+
+---
+
+## Rollout function requirements
+
+`training/rollout.py` must run a full OpenEnv episode.
+
+```python
+def rollout_once(trainer, env, tokenizer, dataset_prompt: str, max_steps: int = 40) -> dict:
+    result = env.reset()
+    observation = result.observation
+
+    prompt_ids = []
+    completion_ids = []
+    logprobs = []
+    reward_trace = []
+    action_trace = []
+    observation_trace = []
+
+    for _ in range(max_steps):
+        if result.done:
+            break
+
+        prompt = build_cybersecurity_owasp_prompt(observation, action_trace, observation_trace)
+        rollout_output = generate_rollout_completions(trainer, [prompt])[0]
+        action = parse_action_json(rollout_output["text"])
+
+        result = env.step(action)
+        observation = result.observation
+
+        prompt_ids.extend(rollout_output["prompt_ids"])
+        completion_ids.extend(rollout_output["completion_ids"])
+        logprobs.extend(rollout_output["logprobs"])
+        reward_trace.append(float(result.reward or 0.0))
+        action_trace.append(action)
+        observation_trace.append(observation)
+
+    final_breakdown = getattr(observation, "reward_breakdown", {}) or {}
+    return {
+        "prompt_ids": prompt_ids,
+        "completion_ids": completion_ids,
+        "logprobs": logprobs,
+        "reward_total": float(final_breakdown.get("total", sum(reward_trace))),
+        "reward_discovery": float(final_breakdown.get("discovery", 0.0)),
+        "reward_security": float(final_breakdown.get("security", 0.0)),
+        "reward_regression": float(final_breakdown.get("regression", 0.0)),
+        "reward_patch_quality": float(final_breakdown.get("patch_quality", 0.0)),
+        "reward_anti_cheat": float(final_breakdown.get("anti_cheat", 0.0)),
+        "success": bool(getattr(env.state(), "success", False)),
+        "episode_length": len(action_trace),
+    }
+```
+
+The prompt must require the model to output exactly one JSON action at a time.
+
+Example action format:
+
+```json
+{"tool_name":"read_file","arguments":{"path":"app/routes/invoices.py"}}
+```
+
+---
+
+## Reward functions for TRL
+
+`training/reward_funcs.py` must expose separate reward functions for GRPO/PPO logging.
+
+```python
+def reward_total(completions, **kwargs):
+    return [float(x) for x in kwargs.get("reward_total", [0.0] * len(completions))]
+
+
+def reward_security(completions, **kwargs):
+    return [float(x) for x in kwargs.get("reward_security", [0.0] * len(completions))]
+
+
+def reward_regression(completions, **kwargs):
+    return [float(x) for x in kwargs.get("reward_regression", [0.0] * len(completions))]
+
+
+def reward_patch_quality(completions, **kwargs):
+    return [float(x) for x in kwargs.get("reward_patch_quality", [0.0] * len(completions))]
+
+
+def reward_anti_cheat(completions, **kwargs):
+    return [float(x) for x in kwargs.get("reward_anti_cheat", [0.0] * len(completions))]
+```
+
+---
+
+## GRPO training config
+
+Use Trackio in `GRPOConfig`.
+
+```python
+import os
+from trl import GRPOConfig
+
+output_dir = os.getenv("OUTPUT_DIR", "CyberSecurity_OWASP-qwen3-1.7b-grpo")
+trackio_space_id = os.getenv("TRACKIO_SPACE_ID", output_dir)
+
+grpo_config = GRPOConfig(
+    output_dir=output_dir,
+    report_to="trackio",
+    trackio_space_id=trackio_space_id,
+    logging_steps=1,
+    save_steps=25,
+    learning_rate=5e-6,
+    num_train_epochs=1,
+    per_device_train_batch_size=1,
+    gradient_accumulation_steps=32,
+    num_generations=2,
+    max_prompt_length=4096,
+    max_completion_length=768,
+    use_vllm=True,
+    vllm_mode="colocate",
+    vllm_gpu_memory_utilization=0.2,
+    gradient_checkpointing=True,
+    gradient_checkpointing_kwargs={"use_reentrant": False},
+    push_to_hub=False,
+)
+```
+
+Start with small debug runs before scaling.
+
+---
+
+## Trackio logging requirements
+
+Trackio is mandatory for training and evaluation visibility.
+
+Run naming convention:
+
+```text
+CyberSecurity_OWASP-<model>-<algo>-level<difficulty>-<YYYYMMDD-HHMM>-<git_sha>
+```
+
+Log these training metrics:
+
+```text
+train/reward_total_mean
+train/reward_discovery_mean
+train/reward_security_mean
+train/reward_regression_mean
+train/reward_public_routes_mean
+train/reward_patch_quality_mean
+train/reward_visible_tests_mean
+train/reward_safety_mean
+train/reward_anti_cheat_mean
+train/success_rate
+train/exploit_block_rate
+train/regression_preservation_rate
+train/public_route_preservation_rate
+train/invalid_action_rate
+train/timeout_rate
+train/safety_violation_rate
+train/reward_hacking_suspected_rate
+train/episode_length_mean
+train/episode_length_p95
+train/rollouts_per_second
+train/tokens_per_second
+train/loss
+train/learning_rate
+train/kl
+train/grad_norm
+```
+
+Log these evaluation metrics:
+
+```text
+eval/baseline_success_rate
+eval/trained_success_rate
+eval/absolute_success_improvement
+eval/baseline_mean_reward
+eval/trained_mean_reward
+eval/absolute_reward_improvement
+eval/heldout_success_rate
+eval/heldout_mean_reward
+eval/exploit_block_rate
+eval/regression_preservation_rate
+eval/public_route_preservation_rate
+eval/anti_cheat_pass_rate
+eval/invalid_action_rate
+eval/timeout_rate
+eval/safety_violation_rate
+eval/mean_episode_length
+```
+
+Log these environment metrics:
+
+```text
+env/reset_latency_ms
+env/step_latency_ms
+env/verifier_latency_ms
+env/reward_latency_ms
+env/scenario_compile_latency_ms
+env/error_rate
+env/task_difficulty
+env/task_seed
+```
+
+---
+
+## Rollout artifact requirements
+
+Save sampled rollouts under `outputs/rollouts/`.
+
+Each rollout JSON must include:
+
+```json
+{
+  "run_name": "...",
+  "episode_id": "...",
+  "task_id": "...",
+  "seed": 123,
+  "split": "validation",
+  "difficulty": 1,
+  "domain": "invoices",
+  "bug_family": "bola_idor",
+  "actions": [],
+  "observations": [],
+  "reward_breakdown_by_step": [],
+  "final_reward_breakdown": {},
+  "total_reward": 0.0,
+  "success": false,
+  "failure_reason": null,
+  "safety_violations": [],
+  "anti_cheat_flags": []
+}
+```
+
+Minimum artifacts:
+
+- 10 baseline rollouts;
+- 10 mid-training rollouts;
+- 10 trained rollouts;
+- 10 held-out evaluation rollouts.
+
+---
+
+## Evaluation requirements
+
+Create `training/eval_before_after.py`.
+
+It must evaluate:
+
+| Metric | Required |
+|---|---:|
+| baseline success rate | yes |
+| trained success rate | yes |
+| absolute success improvement | yes |
+| baseline mean reward | yes |
+| trained mean reward | yes |
+| absolute reward improvement | yes |
+| held-out success rate | yes |
+| exploit-block rate | yes |
+| regression-preservation rate | yes |
+| public-route preservation rate | yes |
+| invalid action rate | yes |
+| anti-cheat pass rate | yes |
+
+Save output:
+
+```text
+outputs/evals/<run_name>_eval_summary.json
+```
+
+Minimum hackathon target:
+
+```text
+>= 50 evaluation episodes
+>= 3 independently logged reward components
+>= 1 held-out split
+>= 1 baseline-vs-trained comparison
+>= 1 anti-cheat evaluation
+```
+
+Preferred demo target:
+
+```text
+mean reward improvement >= 30%
+hidden exploit-block pass rate >= 70%
+regression-preservation pass rate >= 80%
+public-route preservation pass rate >= 90%
+anti-cheat pass rate >= 95%
+```
+
+---
+
+## Testing requirements
+
+Before training, all tests must pass.
+
+Required tests:
+
+```text
+test_models.py
+test_reset_step_state.py
+test_rewards.py
+test_anti_cheat.py
+test_seed_reproducibility.py
+test_invalid_actions.py
+test_rollouts.py
+```
+
+Implement at least 3 scripted policies:
+
+```text
+random_policy: explores action space; should usually fail but not crash
+bad_policy: tries invalid/cheating actions; should be penalized
+oracle_policy: uses internal test-only access to solve; should get high reward
+```
+
+The oracle policy is only for tests and must never be exposed to the model during training.
+
+---
+
+## Deployment requirements
+
+The environment must run in these modes:
+
+1. local Python / Uvicorn;
+2. Docker container;
+3. Hugging Face Space;
+4. OpenEnv client over WebSocket.
+
+Required commands:
+
+```bash
+# initialize if not already scaffolded
+openenv init CyberSecurity_OWASP
+
+# local development
+uv sync
+uv run server
+curl http://localhost:8000/health
+
+# Docker
+openenv build -t CyberSecurity_OWASP:latest
+# or:
+docker build -t CyberSecurity_OWASP:latest -f envs/CyberSecurity_OWASP/server/Dockerfile .
+docker run -p 8000:8000 CyberSecurity_OWASP:latest
+
+# HF Spaces
+openenv push --repo-id <username>/CyberSecurity_OWASP
+
+# client install from Space
+pip install git+https://huggingface.co/spaces/<username>/CyberSecurity_OWASP
+```
+
+Use WebSocket mode for training rollouts. HTTP endpoints are acceptable for debugging only.
+
+---
+
+## Scaling rules
+
+Before scaling training, confirm:
+
+1. one manual episode works;
+2. scripted oracle can solve easy seeds;
+3. random policy does not crash;
+4. 10 validation rollouts complete;
+5. reward distributions make sense;
+6. Trackio receives metrics;
+7. rollout artifacts are saved.
+
+Then scale gradually:
+
+```text
+1 episode -> 10 episodes -> 50 episodes -> 100+ rollouts -> training run
+```
+
+For high-volume rollouts, prefer local Docker or Uvicorn over remote HF Spaces because local WebSocket sessions reduce latency and avoid Space limits.
+
+---
+
+## README requirements
+
+The README must explain:
+
+- what CyberSecurity_OWASP models;
+- why authorization repair is useful for LLM RL;
+- action space;
+- observation space;
+- state fields;
+- scenario generation;
+- reward components;
+- hidden tests;
+- anti-overfitting safeguards;
+- anti-cheat safeguards;
+- curriculum;
+- local/Docker/HF Spaces commands;
+- training with TRL/Unsloth;
+- before/after evaluation.
+
+Include a demo narrative:
+
+```text
+1. Baseline model attempts a generated A01 authorization repair episode.
+2. Verifier shows whether it discovered the bug and whether the patch regressed normal flows.
+3. RL training improves reward and pass rates.
+4. Trained model handles held-out domain/layout seeds.
+5. Anti-cheat tests prove it is not using deny-all, hardcoding, or fixture tampering.
+```
+
+---
+
+## Implementation workflow for Codex
+
+When implementing this repo, follow this exact order:
+
+1. Inspect existing structure and tests.
+2. Create/update `00_PROJECT_BRIEF.md` and `01_ARCHITECTURE.md` if missing.
+3. Define `CyberSecurityOWASPAction`, `CyberSecurityOWASPObservation`, and `CyberSecurityOWASPState`.
+4. Implement a dummy OpenEnv server and client.
+5. Implement scenario compiler with 1 domain and 1 BOLA/IDOR mutator.
+6. Implement editable workspace generation.
+7. Implement local request tool.
+8. Implement visible tests.
+9. Implement hidden verifier and reward engine.
+10. Add anti-cheat checks.
+11. Add tests for normal, failing, and cheating rollouts.
+12. Add oracle, random, and bad scripted policies.
+13. Add scenario cache and seeded splits.
+14. Add 3 domains and 3 bug families.
+15. Add GRPO training script.
+16. Add Trackio logging.
+17. Add before/after evaluation script.
+18. Add HF Spaces deployment config.
+19. Run tests and smoke tests.
+20. Produce demo artifacts and README results.
+
+Do not jump to training code before environment and verifier are correct.
+
+---
+
+## Definition of done
+
+CyberSecurity_OWASP is done only when all are true:
+
+- `reset()`, `step(action)`, and `state` work;
+- actions, observations, and state are typed dataclasses;
+- the environment runs locally;
+- the environment runs in Docker;
+- the environment is deployable to HF Spaces;
+- there are at least 5 meaningful reward components;
+- reward components are logged separately;
+- hidden tests exist;
+- anti-cheat tests exist and pass;
+- scenario cache has train/validation/hidden-eval splits;
+- at least 3 bug families exist;
+- at least 3 domains exist;
+- at least 3 scripted policies exist;
+- Trackio is configured for training and evaluation;
+- before/after evaluation exists;
+- held-out evaluation exists;
+- at least 40 rollout artifacts are saved;
+- README explains environment, reward, training, and demo story;
+- demo shows baseline behavior, trained behavior, reward improvement, and safeguards.
+
+---
+
+## Final PR checklist
+
+Every PR summary must answer:
+
+1. What real-world workflow does this implement?
+2. What does the agent observe?
+3. What actions can the agent take?
+4. What hidden state exists and why is it hidden?
+5. What terminates an episode?
+6. What exact checks prove success?
+7. What are the reward components and ranges?
+8. How could the model hack the reward?
+9. What anti-cheat checks prevent that?
+10. What tests prove the reward cannot be trivially hacked?
+11. What baseline success rate did we observe?
+12. What trained success rate did we observe?
+13. What held-out success rate did we observe?
+14. What Trackio run contains the evidence?
+15. Does behavior improve, or only the reward proxy?
+16. Is the environment ready for HF Spaces deployment?
+
+---
+
+## Source grounding and credibility
+
+| Source | Why used | Credibility |
+|---|---|---:|
+| OWASP Top 10 A01 Broken Access Control | Authorization bug taxonomy and prevention framing | 8.5/10 |
+| OWASP ASVS | Access-control verification grounding | 9/10 |
+| NIST SP 800-218 SSDF | Secure software development lifecycle grounding | 9.5/10 |
+| Smith et al., ESEC/FSE 2015, “Is the Cure Worse Than the Disease?” | Peer-reviewed basis for hidden tests and repair-overfitting risk | 9/10 |
+| OpenEnv build/deploy/training docs | Typed model, server, client, deployment, and training mechanics | 8/10 |
+| Meta OpenEnv Hackathon criteria | Judging alignment and minimum requirements | 8/10 |
+
+---
+
+## Non-negotiable rule
+
+A reward that can be hacked is worse than no reward. Build the verifier, hidden tests, anti-cheat tests, and held-out evaluation before scaling training.