| --- |
| title: ForgeEnv |
| emoji: 🔧 |
| colorFrom: indigo |
| colorTo: green |
| sdk: docker |
| app_port: 7860 |
| pinned: true |
| license: apache-2.0 |
| tags: |
| - openenv |
| - self-play |
| - self-improvement |
| - code-repair |
| - schema-drift |
| - reinforcement-learning |
| - huggingface |
| short_description: Self-improving RL env for HF library-drift repair |
| --- |
| |
| # ForgeEnv — OpenEnv Server |
|
|
| This Space hosts the **ForgeEnv** OpenEnv-compliant environment as a FastAPI |
| service. It exposes the standard `reset`, `step`, and `state` endpoints and is |
| the runtime that training notebooks (TRL + Unsloth) connect to. |
|
|
| > **Theme:** Self-Improvement (Hackathon Theme #4) — Challenger / Solver |
| > co-evolution via R-Zero, SPIRAL, and Absolute Zero Reasoner techniques. |
|
|
| ## What it does |
|
|
| ForgeEnv simulates **HuggingFace library version drift**. A *Drift Generator* |
| proposes a realistic breakage to a working training script (renamed APIs, |
| deprecated imports, changed argument signatures, etc.). A *Repair Agent* then |
| emits a unified diff that should restore the script. Reward is computed by an |
| execution simulator + AST checker + held-out evaluator (multi-component to |
| resist reward hacking). |
|
|
| ## API |
|
|
| The server uses [`openenv-core`](https://pypi.org/project/openenv-core/) and |
| follows the Gym-style contract: |
|
|
| | Endpoint | Method | Purpose | |
| | -------- | ------ | -------------------------------------------------- | |
| | `/reset` | POST | Sample a fresh task, return drift-gen observation | |
| | `/step` | POST | Apply a `ForgeAction` (breakage or repair) | |
| | `/state` | GET | Inspect the current internal state | |
| | `/health`| GET | Health probe (used by the container HEALTHCHECK) | |
|
|
| `ForgeAction` is a discriminated union of `BreakageAction` (used in phase 1) |
| and `RepairAction` (used in phase 2). See |
| [`forgeenv/env/actions.py`](forgeenv/env/actions.py). |
|
|
| ## Quick test |
|
|
| ```bash |
| curl -X POST https://akhiilll-forgeenv.hf.space/reset |
| curl https://akhiilll-forgeenv.hf.space/state |
| ``` |
|
|
| ```python |
| from openenv.core.env_client import EnvClient |
| |
| async with EnvClient(base_url="https://akhiilll-forgeenv.hf.space") as client: |
| obs = await client.reset() |
| print(obs.observation.current_phase, obs.observation.task_id) |
| ``` |
|
|
| ## Project links |
|
|
| - **Main repo / training notebooks / plots:** |
| <https://github.com/akhiilll/forgeenv> |
| - **Repair Agent model (LoRA):** |
| <https://huggingface.co/akhiilll/forgeenv-repair-agent> |
| - **Demo (Gradio + ZeroGPU):** |
| <https://huggingface.co/spaces/akhiilll/forgeenv-demo> |
|
|
| ## Citations |
|
|
| - Huang et al., *R-Zero: Self-Evolving Reasoning LLM From Zero Data* (2025) |
| - Zhao et al., *Absolute Zero: Reinforced Self-play Reasoning with Zero Data* (2025) |
| - Liu et al., *SPIRAL: Self-Play on Zero-Sum Games* (2025) |
| - [arXiv:2408.10215](https://arxiv.org/abs/2408.10215) — Reward engineering & shaping |
| - [arXiv:2601.19100](https://arxiv.org/abs/2601.19100) — Reward engineering for RL in software tasks |
|
|
