00_PROJECT_BRIEF.md

00_PROJECT_BRIEF.md

CyberSecurity_OWASP — Project Brief

1. One-line summary

CyberSecurity_OWASP is an OpenEnv reinforcement-learning environment where a single LLM agent learns the full defensive workflow for OWASP access-control bugs: understand the intended authorization policy, discover a broken access-control path in a local synthetic app, patch the code, and prove that the fix blocks unauthorized access without breaking valid user flows.

2. Problem

Broken access control remains one of the most important web-application security risks because the correct behavior is usually application-specific. Generic scanners can find some missing checks, but they often lack enough context to answer the real engineering question:

“Given this app’s policy, users, roles, tenants, routes, and data model, is this behavior intended or a security bug?”

Modern LLMs can read code, reason about tests, and propose patches, but they still struggle with:

• distinguishing intended public/feature behavior from accidental over-permission;
• following authorization logic across routes, middleware, ORM queries, tenants, roles, and ownership checks;
• validating that a patch fixes the bug without introducing regressions;
• avoiding reward hacking when tests are visible or too narrow;
• generalizing across app templates instead of memorizing one codebase.

CyberSecurity_OWASP turns this into a trainable environment.

3. What the environment trains

The environment trains one agent, not a separate red-team and blue-team pair. The same model must perform the entire secure-repair loop:

1. Understand policy — read the policy graph, user roles, route intent, tenant rules, and allowed operations.
2. Discover evidence — use safe local requests, logs, route metadata, and visible tests to identify the likely access-control failure.
3. Patch — edit application code, middleware, route guards, query filters, or policy mappings.
4. Validate — run public tests, policy checks, and regression tests.
5. Submit — final answer is judged by deterministic hidden tests and reward logic.

4. Scope for MVP

The MVP should focus on OWASP A01: Broken Access Control with ASVS-inspired access-control requirements.

Initial scenario families:

1. Missing route-level authorization check.
2. Insecure direct object reference / object ownership bug.
3. Cross-tenant data leakage.
4. Role confusion: user/admin/support/editor boundary error.
5. Client-side-only authorization assumption.
6. Query filter omission in list/search/export endpoint.
7. Over-broad update/delete permission.
8. Feature route intentionally public, so the agent must not over-secure it.

Recommended MVP size: 8 scenario families × 3 app templates × 25 seeds = 600 trainable scenarios, with separate held-out families and hidden seeds for evaluation.

5. Why this is useful

This environment is useful because it targets a real gap between today’s scanners and useful defensive agents:

• Scanners detect patterns. This environment trains policy-aware reasoning.
• Unit tests check known cases. This environment includes hidden authorization invariants.
• Static repair can overfit. This environment forces the model to preserve valid business behavior.
• One-app benchmarks are easy to memorize. This environment prepares and caches many equivalent-but-different apps from policy graphs, templates, route shapes, schema names, and hidden test seeds, then keeps runtime reset() deterministic and fast.

The outcome is a model that becomes better at a practical DevSecOps workflow: safely reviewing and repairing authorization logic in small-to-medium web apps.

6. What success looks like

A successful submission should show measurable reward improvement and better held-out security behavior after RL training.

Minimum success criteria

• Environment runs through OpenEnv reset, step, and state APIs.
• Hosted on Hugging Face Spaces.
• Provides a minimal GRPO/TRL or Unsloth training script.
• Tracks training/eval metrics with Trackio or equivalent.
• Shows reward curves and before/after agent behavior.
• Uses deterministic reward as the primary reward source.
• Keeps hidden tests hidden from the agent.

Target metrics

Metric	MVP target
Valid episode completion rate	≥ 85%
Hidden authorization test pass rate	≥ 65% after initial RL run
Regression preservation rate	≥ 80%
Held-out scenario success lift vs base model	≥ +15 percentage points
Reward-hacking incidents found in eval	0 critical
Median patch size	≤ 3 files changed

7. Core design principle

The environment should reward correct defensive repair, not exploit creativity. The discovery stage exists only to help the agent gather enough local evidence to make a safe patch. The reward engine must never reward real-world misuse, data exfiltration, persistence, credential theft, or evasion behavior.

8. Deliverables for engineers

Initial implementation should produce:

CyberSecurity_OWASP/
├── 00_PROJECT_BRIEF.md
├── 01_ARCHITECTURE.md
├── README.md
├── pyproject.toml
├── openenv.yaml
├── cybersecurity_owasp/
│   ├── __init__.py
│   ├── models.py
│   ├── client.py
│   ├── rewards.py
│   ├── scenarios/
│   │   ├── compiler.py
│   │   ├── policy_graph.py
│   │   ├── templates/
│   │   └── seeds/
│   ├── apps/
│   │   ├── fastapi_basic/
│   │   ├── express_basic/
│   │   └── django_basic/
│   ├── evals/
│   │   ├── public_tests.py
│   │   ├── hidden_invariants.py
│   │   └── heldout_eval.py
│   └── server/
│       ├── environment.py
│       ├── app.py
│       ├── requirements.txt
│       └── Dockerfile
├── training/
│   ├── train_grpo.py
│   ├── rollout.py
│   └── eval_before_after.py
└── outputs/
    ├── logs/
    ├── evals/
    └── reward_curves/

9. Source notes and credibility

Source	How it informs this project	Credibility
OWASP Top 10 2025 / A01 Broken Access Control	Confirms current relevance of Broken Access Control as a top web-app risk.	10/10
OWASP ASVS	Provides security-control requirements that can be translated into policy invariants and hidden tests.	9.5/10
OpenEnv build/deploy docs	Defines the required OpenEnv structure: models, server, client, Docker, HF Spaces deployment.	8.5/10
Hackathon judging criteria	Aligns deliverables with scoring: innovation, storytelling, reward improvement, and training pipeline.	9/10
TRL/OpenEnv GRPO example	Shows a practical pattern for environment rollouts, reward functions, and Trackio logging.	8/10