Spaces:
Sleeping
Sleeping
| title: LandscapeForge | |
| emoji: 🏔️ | |
| colorFrom: purple | |
| colorTo: blue | |
| sdk: docker | |
| pinned: true | |
| app_port: 8000 | |
| tags: | |
| - openenv | |
| - reinforcement-learning | |
| - optimization | |
| - llm-agents | |
| - self-improvement | |
| - gradio | |
| license: apache-2.0 | |
| short_description: LLM agent designs optimizers via a probe-draft-commit REPL. | |
| # 🏔️ LandscapeForge | |
| **An OpenEnv where an LLM agent designs optimization algorithms through a probe-draft-commit REPL, trained against a Goldilocks-regulated landscape adversary.** | |
| Target: **OpenEnv Hackathon, April 2026**. Theme 4 (Self-Improvement), secondary Theme 1 (Multi-Agent). | |
| --- | |
| ## What this Space gives you | |
| Two things, in one container: | |
| | Path | What it is | | |
| |---|---| | |
| | **`/web`** | **Interactive Gradio demo** — landscape explorer + baseline race + paste-your-own-optimizer arena. Visual-first, meant to make the env legible to judges. | | |
| | **`/reset`, `/step`, `/schema`, WebSocket** | **OpenEnv FastAPI endpoints** — wire the env into a TRL / Unsloth GRPO training loop. | | |
| Same process, no second container required. | |
| --- | |
| ## How the env works (in 90 seconds) | |
| **OptCoder** is the LLM policy. Each episode: | |
| 1. **LandscapeForge** (an internal template picker in v1) chooses a loss landscape `f: ℝⁿ → ℝ` at a tier-appropriate difficulty: quadratic / Rosenbrock / Styblinski-Tang / Gaussian-mix / Himmelblau / plateau / cliff. | |
| 2. **OptCoder runs a 4-action REPL** with a budget of 12 units: | |
| - `run_baseline(name)` — run SGD / Momentum / Adam / L-BFGS on the hidden landscape and see their trajectory (cost: 2) | |
| - `draft(code)` — submit a full `Optimizer` class; the env auto-tests it for 20 steps (cost: 2) | |
| - `inspect(draft_idx, step_range)` — zoom into a prior draft's per-step `(x, f, grad, update_norm, step_size_eff)` to diagnose failures (cost: 1) | |
| - `commit` — evaluate the latest draft on the full **Phase-D arena**: 10 fresh seeds × 200 steps (cost: 0) | |
| 3. **Reward** (terminal only; stepwise is feedback-only): | |
| - `r_regret` — **Adam-relative progress** (tuned Adam LR per landscape; no `f_min` dependency, generalises directly to NN training) | |
| - `r_convergence`, `r_robustness`, `r_novelty` (gated), minus `r_budget`, minus `r_eval_failures` | |
| 4. **GRPO** can then train the policy; arena wall-clock is ~50 ms so ~36k episodes/hour on one H100. | |
| See `IMPLEMENTATION.md` and `LANDSCAPEFORGE_DESIGN.md` in this repo for the full spec, staged bootstrap (SFT → solo RL → adversarial unfreezing), and the anti-reward-hacking table. | |
| --- | |
| ## Quick-start (Python client) | |
| ```python | |
| from landscapeforge import LandscapeforgeEnv, LandscapeforgeAction | |
| with LandscapeforgeEnv.from_docker_image("landscapeforge-env:latest") as env: | |
| obs = env.reset() | |
| env.step(LandscapeforgeAction(kind="run_baseline", baseline_name="adam")) | |
| env.step(LandscapeforgeAction(kind="draft", code=""" | |
| class Optimizer: | |
| def __init__(self, dim): | |
| self.lr = 0.05; self.beta = 0.9 | |
| self.v = np.zeros(dim) | |
| def step(self, x, f_val, grad): | |
| self.v = self.beta * self.v - self.lr * grad | |
| return x + self.v | |
| """)) | |
| result = env.step(LandscapeforgeAction(kind="commit")) | |
| print(result.observation.r_optcoder_breakdown) | |
| # {'r_regret': ..., 'r_convergence': ..., 'r_robustness': ..., | |
| # 'r_novelty': ..., 'r_budget': ..., 'r_eval_failures': ..., | |
| # 'my_progress': ..., 'adam_progress': ..., 'speedup_vs_adam': ...} | |
| ``` | |
| ## Quick-start (drive with any OpenAI-compat LLM) | |
| The repo ships `run_llm_episode.py` that drives one episode against any `/v1/chat/completions` endpoint (HuggingFace router, Ollama, vLLM, …): | |
| ```bash | |
| # Ollama local | |
| API_BASE_URL=http://localhost:11434/v1 MODEL_NAME=qwen2.5:3b \ | |
| python -m landscapeforge.run_llm_episode | |
| # HuggingFace router | |
| HF_TOKEN=hf_xxx MODEL_NAME=Qwen/Qwen2.5-7B-Instruct \ | |
| python -m landscapeforge.run_llm_episode | |
| ``` | |
| Full turn transcripts (prompt, raw reply, parsed action, env feedback, reward breakdown) are written to `episode_logs/*.jsonl` + `*.md`. | |
| --- | |
| ## What to click first in `/web` | |
| 1. **Baseline Race** tab → pick Rosenbrock → hit "🏁 Race!" to see how default-SGD, default-Momentum, **tuned-Adam**, and crude-L-BFGS actually perform on the classic stiff valley. | |
| 2. **Optimizer Arena** tab → keep the sample SGD+Momentum optimizer, hit "⚔️ Run arena" to see the reward breakdown vs tuned Adam. | |
| 3. **Landscape Explorer** tab → browse the 9 template families with contour plots + structural hints. | |
| --- | |
| ## Repo structure | |
| ``` | |
| landscapeforge/ | |
| ├── LANDSCAPEFORGE_DESIGN.md # Full design doc (v0.2) | |
| ├── IMPLEMENTATION.md # What's in the code today + constants | |
| ├── models.py # Action + Observation (pydantic) | |
| ├── landscapes.py # 9 analytic template builders with gradients | |
| ├── reference_optimizers.py # SGD / Momentum / Adam / L-BFGS + LR tuner | |
| ├── sandbox.py # AST-strip + restricted exec + timeout | |
| ├── arena.py # Phase-D runner + auto_test_draft | |
| ├── rewards.py # Terminal reward + stepwise feedback | |
| ├── prompts.py # obs → prompt / response → action | |
| ├── run_llm_episode.py # LLM-in-the-loop runner (OpenAI-compat) | |
| ├── server/ | |
| │ ├── app.py # FastAPI + mounted Gradio at /web | |
| │ └── landscapeforge_environment.py # OpenEnv Environment class | |
| ├── demo/ui.py # Gradio UI source | |
| ├── tests/test_episode.py # Scripted end-to-end tests | |
| └── episode_logs/ # Per-episode JSONL + Markdown transcripts | |
| ``` | |
| --- | |
| ## Research anchors | |
| LandscapeForge sits at the intersection of five established research threads: | |
| - **Thread 1** — LLMs as optimizer designers: [Lion (NeurIPS 2023)](https://arxiv.org/abs/2302.06675), [FunSearch (Nature 2024)](https://www.nature.com/articles/s41586-023-06924-6) | |
| - **Thread 2** — Adversarial / co-evolutionary LLM-env: Coevolve, [GenEnv (ICLR 2026)](https://arxiv.org/html/2512.19682v1) | |
| - **Thread 3** — Iterative code refinement: [Self-Refine](https://arxiv.org/abs/2303.17651) | |
| - **Thread 4** — GRPO with measurable rewards: [HPC GFLOPS reward paper](https://arxiv.org/abs/2602.12049v1) | |
| - **Thread 5** — Analytical landscape benchmarks: [BBOB/COCO](https://inria.hal.science/hal-00362649/document), [POET](https://arxiv.org/abs/1901.01753) | |
| Every ingredient has prior work; the combination — LLM-generated optimizers + LLM-picked landscapes + iterative REPL + GRPO on Adam-relative progress — is novel. | |
| --- | |
| ## License | |
| Apache-2.0. | |