README.md

---
title: Python Code Review Environment Server
sdk: docker
app_port: 8000
base_path: /web
pinned: false
tags:
  - openenv
  - code-review
---

# Python Code Review Environment

A production-grade OpenEnv environment for Python code review, repair, and optimization tasks. This environment simulates real-world developer workflows where an AI agent reviews, fixes, and improves Python code.

## Overview

**`python_code_review_env`** is a deterministic benchmark environment featuring:

- ✅ **3 real-world tasks** with increasing difficulty (Syntax, Bug Fix, Optimization)
- ✅ **Deterministic graders** using AST analysis, pytest execution, and performance benchmarking
- ✅ **OpenAI-compatible API** supporting free/open models (Gemini, DeepSeek, Together, OpenRouter)
- ✅ **Production-ready Docker** deployment for Hugging Face Spaces
- ✅ **Structured Observations & Actions** following OpenEnv spec
- ✅ **Rich reward shaping** with bonuses for syntax fixes, test passes, and optimization

## Tasks

### 1. 🟢 Easy: Syntax Fixing

**Task ID**: `syntax-fix-easy`

Fix broken Python code with syntax errors.

- **Difficulty**: Easy
- **Goal**: Repair syntax errors to make code compile
- **Starter Code**: Function with missing closing parenthesis
- **Grading**: Compilation check + code similarity to reference
- **Score Range**: 0.0–1.0

### 2. 🟡 Medium: Bug Fixing

**Task ID**: `bug-fix-medium`

Fix logic bugs with visible and hidden test cases.

- **Difficulty**: Medium  
- **Goal**: Repair a logic error in invoice calculation
- **Starter Code**: Function that returns wrong total (returns subtotal instead of discounted)
- **Grading**: Test pass fraction (visible & hidden)
- **Score Range**: 0.0–1.0

### 3. 🔴 Hard: Optimization & Refactoring

**Task ID**: `optimization-hard`

Optimize inefficient code while maintaining correctness.

- **Difficulty**: Hard
- **Goal**: Convert O(n²) duplicate removal to O(n) with set
- **Starter Code**: Slow nested-loop implementation
- **Grading**: 50% correctness + 30% speedup + 15% code quality + 5% style
- **Score Range**: 0.0–1.0
- **Bonus**: Runtime benchmarking against reference implementation

## Quick Start

### Run Locally

```bash
cd python-code-review-env
pip install -r server/requirements.txt
python -m server.app
```

Visit http://localhost:8000/docs for interactive API

### Run with Docker

```bash
docker build -f server/Dockerfile -t python_code_review_env:latest .
docker run -p 8000:8000 python_code_review_env:latest
```

### Run Inference

```bash
python inference.py --model "gpt-3.5-turbo" --base-url "http://localhost:8000/v1"
```

## OpenEnv Specification

### Observation

```json
{
  "task_id": "syntax-fix-easy",
  "difficulty": "easy",
  "task_description": "Fix syntax errors...",
  "current_code": "def normalize_username(raw_name: str) -> str:\n    cleaned = raw_name.strip().lower(\n    ...",
  "errors": "invalid syntax ( line 2, column 40 )",
  "test_results": "Not run yet.",
  "visible_tests": ["normalize_username('  Alice Smith  ') == 'alice_smith'"],
  "history": [],
  "attempts_remaining": 8,
  "score": 0.0,
  "reward": {
    "value": 0.0,
    "reason": "Episode reset."
  }
}
```

### Action

```json
{
  "action_type": "edit_code",
  "code": "def normalize_username(raw_name: str) -> str:\n    cleaned = raw_name.strip().lower()\n    if not cleaned:\n        return \"anonymous\"\n    return cleaned.replace(\" \", \"_\")"
}
```

### Reward Details

- **+0.2**: Syntax fixed (one-time per episode)
- **+0.15**: Passing additional test (cumulative per test)
- **+0.1**: Code quality improvement  
- **+0.5**: Full correctness (100% hidden tests, one-time)
- **-0.1**: Invalid action

## Architecture

```
python_code_review_env/
├── models.py          # Pydantic models (Observation, Action, Reward)
├── server/
│   ├── app.py         # FastAPI server  
│   ├── env.py         # OpenEnv environment
│   ├── Dockerfile     # Docker config
│   └── requirements.txt
├── graders/
│   ├── common.py      # Shared utilities
│   ├── syntax.py      # Syntax/bug graders
│   ├── optimization.py# Optimization grader
│   └── pytest_runner.py
├── tasks/
│   ├── task_bank.py   # 3 deterministic tasks
│   └── __init__.py
├── inference.py       # Baseline evaluation script
├── openenv.yaml       # OpenEnv spec
├── pyproject.toml     # Project metadata
└── README.md
```

## FastAPI Endpoints

- `GET /health` – Health check
- `GET /tasks` – List all tasks
- `GET /tasks/{task_id}` – Get task details
- `POST /tasks/{task_id}/grade` – Grade code offline
- Standard OpenEnv endpoints (`/reset`, `/step`, `/state`)

## Deterministic Graders

### Syntax Fix
```
if code compiles:
  score = 1.0
else:
  score = 0.15 + 0.55 * similarity_to_reference
```

### Bug Fix  
```
score = test_pass_fraction (0.0 to 1.0)
```

### Optimization
```
score = (
  0.5 * test_fraction +
  0.3 * speedup_score +
  0.15 * code_quality +
  0.05 * pep8_style
)
```

## Examples

### Using Python

```python
from server.env import PythonCodeReviewEnvironment
from models import PythonCodeReviewAction

env = PythonCodeReviewEnvironment()
obs = env.reset(task_id="syntax-fix-easy")

action = PythonCodeReviewAction(
    action_type="edit_code",
    code="""def normalize_username(raw_name: str) -> str:
    cleaned = raw_name.strip().lower()
    if not cleaned:
        return "anonymous"
    return cleaned.replace(" ", "_")
"""
)

obs = env.step(action)
print(f"Score: {obs.score}")
print(f"Reward: {obs.reward.value:+.3f}")
```

### Using cURL

```bash
# Check health
curl http://localhost:8000/health

# List tasks
curl http://localhost:8000/tasks

# Grade code
curl -X POST http://localhost:8000/tasks/syntax-fix-easy/grade \
  -H "Content-Type: application/json" \
  -d '{"action_type": "edit_code", "code": "..."}'
```

## Deployment

### Hugging Face Spaces

1. Create Space > Docker
2. Upload files + `server/Dockerfile`
3. Space auto-deploys on CPU
4. Monitor `/health` endpoint

### Local Docker

```bash
docker build -f server/Dockerfile -t python_code_review_env .
docker run -p 8000:8000 \
  -e MAX_CONCURRENT_ENVS=16 \
  python_code_review_env
```

## Performance

- Startup: < 5s
- Reset: < 100ms
- Step: 50ms–3s (depends on action)
- Inference (3 tasks): < 20 minutes
- CPU: Works on 2 vCPU, 8GB RAM

## Validation Checklist

- ✅ 3 deterministic tasks
- ✅ Deterministic graders (AST, pytest, benchmarks)
- ✅ `/health` → 200
- ✅ Scores vary per task (not constant)
- ✅ Docker builds successfully
- ✅ OpenEnv spec compliant
- ✅ Reward shaping working
- ✅ All tests deterministic and reproducible

## License

MIT

---

**Built for production. Deterministic. Deployable. Extensible.**