Spaces:
Running
Running
ajaxwin commited on
Commit Β·
17ed3a7
1
Parent(s): 0304fd3
feat: Add landing page and API documentation
Browse files- Implemented a landing page that serves HTML for browsers and JSON for API clients.
- Added a new index.html file containing detailed information about the Smart Contract Audit RL Environment, including tasks, data sources, and API endpoints.
- Updated the root endpoint to return the landing page or JSON summary based on the request's Accept header.
- Enhanced the project description and versioning in the root JSON response.
- .gitignore +2 -1
- Docs.md +0 -295
- LICENSE.txt +21 -0
- README.md +384 -20
- server/app.py +39 -3
- server/index.html +708 -0
.gitignore
CHANGED
|
@@ -12,4 +12,5 @@ baseline_scores.json
|
|
| 12 |
*.log
|
| 13 |
.pytest_cache/
|
| 14 |
MySolution.md
|
| 15 |
-
nltk_data
|
|
|
|
|
|
| 12 |
*.log
|
| 13 |
.pytest_cache/
|
| 14 |
MySolution.md
|
| 15 |
+
nltk_data
|
| 16 |
+
uv.lock
|
Docs.md
DELETED
|
@@ -1,295 +0,0 @@
|
|
| 1 |
-
# Smart Contract Audit RL Environment
|
| 2 |
-
|
| 3 |
-
> **OpenEnv-compliant reinforcement learning environment for smart contract security analysis.**
|
| 4 |
-
> Three fully implemented tasks covering the core workflow of a professional Solidity auditor.
|
| 5 |
-
|
| 6 |
-
[](openenv.yaml)
|
| 7 |
-
[](https://python.org)
|
| 8 |
-
[](LICENSE)
|
| 9 |
-
|
| 10 |
-
---
|
| 11 |
-
|
| 12 |
-
## Motivation
|
| 13 |
-
|
| 14 |
-
Smart contract auditing is a $500M+ industry where human experts identify security flaws, write formal properties, and check whether code satisfies those properties. This environment lets agents practise exactly those three tasks using real Solidity contracts from Certora-audited DeFi projects.
|
| 15 |
-
|
| 16 |
-
---
|
| 17 |
-
|
| 18 |
-
## Tasks at a Glance
|
| 19 |
-
|
| 20 |
-
| # | Name | Difficulty | Status | One-line description |
|
| 21 |
-
|---|------|-----------|--------|---------------------|
|
| 22 |
-
| 1 | Targeted Vulnerability Detection | Medium | β
Active | Find which function is vulnerable and name the vulnerability |
|
| 23 |
-
| 2 | Property Discovery | Hard | β
Active | Write the natural-language postcondition for a given function |
|
| 24 |
-
| 3 | Rule Checker | Easy | β
Active | Identify which function violates a given property |
|
| 25 |
-
|
| 26 |
-
---
|
| 27 |
-
|
| 28 |
-
## Task 1 β Targeted Vulnerability Detection *(Medium)*
|
| 29 |
-
|
| 30 |
-
**Setup:** A Solidity contract (4β6 functions) is shown. One function contains a critical vulnerability.
|
| 31 |
-
|
| 32 |
-
**Objective:** Name the vulnerable function and describe its vulnerability type in 2β3 words.
|
| 33 |
-
|
| 34 |
-
### Actions
|
| 35 |
-
|
| 36 |
-
| Action | Params | Reward |
|
| 37 |
-
|--------|--------|--------|
|
| 38 |
-
| `list_functions` | β | β0.05 |
|
| 39 |
-
| `get_function_code` | `function_name` | +0.05 if target / β0.10 if other |
|
| 40 |
-
| `get_function_summary` | `function_name` | +0.03 if target / β0.05 if other |
|
| 41 |
-
| `get_file_metadata` | β | β0.04 |
|
| 42 |
-
| `get_state_variable` | `variable_name` (opt.) | β0.05 |
|
| 43 |
-
| `get_call_graph` | β | β0.08 |
|
| 44 |
-
| `submit` | `function_name`, `vulnerability_type` | **+5.0 / +1.0 / β1.5** |
|
| 45 |
-
|
| 46 |
-
Repeated queries: **β0.40**
|
| 47 |
-
|
| 48 |
-
### Grader
|
| 49 |
-
|
| 50 |
-
- **1.0** β correct function + correct vulnerability keyword β reward **+5.0**
|
| 51 |
-
- **0.5** β correct function, vague/wrong vulnerability type β reward **+1.0**
|
| 52 |
-
- **0.0** β wrong function β reward **β1.5**
|
| 53 |
-
|
| 54 |
-
### Vulnerability types covered
|
| 55 |
-
Reentrancy Β· Missing access control Β· Integer overflow Β· tx.origin authentication Β·
|
| 56 |
-
Front-running Β· Timestamp dependence Β· Denial of service Β· Unchecked return value
|
| 57 |
-
|
| 58 |
-
---
|
| 59 |
-
|
| 60 |
-
## Task 2 β Property Discovery *(Hard)*
|
| 61 |
-
|
| 62 |
-
**Setup:** A single Solidity function is shown. The agent must discover its natural-language correctness property.
|
| 63 |
-
|
| 64 |
-
**Objective:** Write a precise 2β4 sentence postcondition describing what the function guarantees on success.
|
| 65 |
-
|
| 66 |
-
### Actions
|
| 67 |
-
|
| 68 |
-
| Action | Params | Reward |
|
| 69 |
-
|--------|--------|--------|
|
| 70 |
-
| `get_function_code` | β | β0.06 |
|
| 71 |
-
| `get_function_natspec` | β | β0.08 |
|
| 72 |
-
| `get_file_natspec` | β | β0.03 |
|
| 73 |
-
| `get_related_functions` | β | β0.06 |
|
| 74 |
-
| `get_signature` | β | β0.04 |
|
| 75 |
-
| `get_similar_rule` | β | β0.20 |
|
| 76 |
-
| `submit_property` | `property` (string) | **0.0β5.0** scored, ONE attempt |
|
| 77 |
-
|
| 78 |
-
### Grader (keyword-weighted)
|
| 79 |
-
|
| 80 |
-
```
|
| 81 |
-
score = 0.70 Γ (key_phrases_matched / total_key)
|
| 82 |
-
+ 0.30 Γ (bonus_phrases_matched / total_bonus)
|
| 83 |
-
reward = score Γ 5.0
|
| 84 |
-
```
|
| 85 |
-
|
| 86 |
-
Matching uses **word-set containment + synonym expansion** β words don't need to be adjacent.
|
| 87 |
-
|
| 88 |
-
---
|
| 89 |
-
|
| 90 |
-
## Task 3 β Rule Checker *(Easy)*
|
| 91 |
-
|
| 92 |
-
**Setup:** A Solidity contract is shown alongside a violated property in natural English. One function breaks that property.
|
| 93 |
-
|
| 94 |
-
**Objective:** Identify which function violates the property.
|
| 95 |
-
|
| 96 |
-
### Actions
|
| 97 |
-
|
| 98 |
-
| Action | Params | Reward |
|
| 99 |
-
|--------|--------|--------|
|
| 100 |
-
| `list_functions` | β | β0.05 |
|
| 101 |
-
| `get_function_metadata` | `function_name` | β0.05 |
|
| 102 |
-
| `get_function_code` | `function_name` | β0.10 |
|
| 103 |
-
| `get_state_variable` | `variable_name` (opt.) | β0.05 |
|
| 104 |
-
| `get_call_graph` | β | β0.08 |
|
| 105 |
-
| `get_property_specification` | β | **β0.03** (cheapest β read this first!) |
|
| 106 |
-
| `submit_function` | `function_name` | **+5.0 / +1.5 / β1.5**, ONE attempt |
|
| 107 |
-
|
| 108 |
-
### Grader (three-tier deterministic)
|
| 109 |
-
|
| 110 |
-
- **1.0** β exact target function (case-insensitive) β reward **+5.0**
|
| 111 |
-
- **0.3** β a direct internal subfunction of the target β reward **+1.5**
|
| 112 |
-
- **0.0** β anything else β reward **β1.5**
|
| 113 |
-
|
| 114 |
-
`get_property_specification` returns the precise pre/post-condition (`rule_broken_specs`). Reading it costs only β0.03 and usually provides enough information to identify the violating function without inspecting all code.
|
| 115 |
-
|
| 116 |
-
---
|
| 117 |
-
|
| 118 |
-
## Observation Space
|
| 119 |
-
|
| 120 |
-
All tasks share the same `Observation` structure:
|
| 121 |
-
|
| 122 |
-
```json
|
| 123 |
-
{
|
| 124 |
-
"task_id": "task3_rule_checker",
|
| 125 |
-
"contract_name": "SimpleVault",
|
| 126 |
-
"contract_description": "An ETH vault that allows users to deposit...",
|
| 127 |
-
"available_actions": ["list_functions", "get_function_metadata", "..."],
|
| 128 |
-
"last_action": "get_property_specification",
|
| 129 |
-
"last_action_result": "Formal property:\nPre: caller != owner...",
|
| 130 |
-
"step_count": 1,
|
| 131 |
-
"cumulative_reward": -0.03,
|
| 132 |
-
"done": false,
|
| 133 |
-
"extra": {
|
| 134 |
-
"property_english": "Only the owner should be able to drain the vault...",
|
| 135 |
-
"solidity_version": "0.8.0",
|
| 136 |
-
"hint": "Find the function that violates this property..."
|
| 137 |
-
}
|
| 138 |
-
}
|
| 139 |
-
```
|
| 140 |
-
|
| 141 |
-
For Task 2, `extra` contains `target_function` and `target_signature`.
|
| 142 |
-
For Task 3, `extra` contains `property_english`.
|
| 143 |
-
|
| 144 |
-
---
|
| 145 |
-
|
| 146 |
-
## Project Structure
|
| 147 |
-
|
| 148 |
-
```
|
| 149 |
-
|
| 150 |
-
```
|
| 151 |
-
|
| 152 |
-
---
|
| 153 |
-
|
| 154 |
-
## Setup
|
| 155 |
-
|
| 156 |
-
### Local Python
|
| 157 |
-
|
| 158 |
-
```bash
|
| 159 |
-
pip install -r requirements.txt
|
| 160 |
-
|
| 161 |
-
# Start the server
|
| 162 |
-
python app.py # β http://localhost:7860
|
| 163 |
-
|
| 164 |
-
# Interactive / scripted demos
|
| 165 |
-
python demo.py --auto # Task 1 scripted demo
|
| 166 |
-
python demo.py --auto --seed 42 # Task 2 (same flag, different env seed)
|
| 167 |
-
|
| 168 |
-
# Full evaluation harness (no LLM required)
|
| 169 |
-
python eval.py # All 3 tasks, 8 episodes each
|
| 170 |
-
python eval.py --task 3 # Task 3 only
|
| 171 |
-
python eval.py --episodes 16 --verbose
|
| 172 |
-
|
| 173 |
-
# Pre-submission validation
|
| 174 |
-
python validate-submission.py # 23/23 checks
|
| 175 |
-
```
|
| 176 |
-
|
| 177 |
-
### Docker
|
| 178 |
-
|
| 179 |
-
```bash
|
| 180 |
-
docker build -t sc-audit-env .
|
| 181 |
-
docker run -p 7860:7860 sc-audit-env
|
| 182 |
-
```
|
| 183 |
-
|
| 184 |
-
### Direct Python API
|
| 185 |
-
|
| 186 |
-
```python
|
| 187 |
-
# Task 3 example
|
| 188 |
-
from tasks.task3.environment import Task3Environment
|
| 189 |
-
from env.schemas import Action, ActionType
|
| 190 |
-
|
| 191 |
-
env = Task3Environment()
|
| 192 |
-
r = env.reset(seed=42)
|
| 193 |
-
print(r.observation.extra["property_english "])
|
| 194 |
-
# "Only the owner should be able to drain the vault..."
|
| 195 |
-
|
| 196 |
-
s = env.step(Action(action_type=ActionType.GET_PROPERTY_SPECIFICATION))
|
| 197 |
-
s = env.step(Action(action_type=ActionType.SUBMIT_FUNCTION,
|
| 198 |
-
params={"function_name": "emergencyDrain"}))
|
| 199 |
-
print(s.reward.value) # +5.0
|
| 200 |
-
```
|
| 201 |
-
|
| 202 |
-
---
|
| 203 |
-
|
| 204 |
-
## HTTP API
|
| 205 |
-
|
| 206 |
-
| Method | Endpoint | Description |
|
| 207 |
-
|--------|----------|-------------|
|
| 208 |
-
| `GET` | `/health` | Liveness probe |
|
| 209 |
-
| `GET` | `/tasks` | All tasks + status |
|
| 210 |
-
| `POST` | `/reset` | Start episode (`task_id`, `seed`) |
|
| 211 |
-
| `POST` | `/step` | Take action (`action_type`, `params`) |
|
| 212 |
-
| `GET` | `/state` | Internal debug state |
|
| 213 |
-
| `GET` | `/action_space?task_id=...` | Action schema |
|
| 214 |
-
| `GET` | `/observation_space` | Observation schema |
|
| 215 |
-
|
| 216 |
-
```bash
|
| 217 |
-
# Full Task 3 episode
|
| 218 |
-
curl -X POST localhost:7860/reset \
|
| 219 |
-
-H "Content-Type: application/json" \
|
| 220 |
-
-d '{"task_id":"task3_rule_checker","seed":42}'
|
| 221 |
-
|
| 222 |
-
curl -X POST localhost:7860/step \
|
| 223 |
-
-H "Content-Type: application/json" \
|
| 224 |
-
-d '{"action_type":"get_property_specification","params":{}}'
|
| 225 |
-
|
| 226 |
-
curl -X POST localhost:7860/step \
|
| 227 |
-
-H "Content-Type: application/json" \
|
| 228 |
-
-d '{"action_type":"submit_function","params":{"function_name":"emergencyDrain"}}'
|
| 229 |
-
```
|
| 230 |
-
|
| 231 |
-
---
|
| 232 |
-
|
| 233 |
-
## Baseline Inference
|
| 234 |
-
|
| 235 |
-
```bash
|
| 236 |
-
export API_BASE_URL="https://api.openai.com/v1"
|
| 237 |
-
export MODEL_NAME="gpt-4o-mini"
|
| 238 |
-
export HF_TOKEN="sk-..."
|
| 239 |
-
python inference.py
|
| 240 |
-
```
|
| 241 |
-
|
| 242 |
-
### Expected scores (gpt-4o-mini, 3 episodes per task)
|
| 243 |
-
|
| 244 |
-
| Task | Avg Grader Score | Notes |
|
| 245 |
-
|------|-----------------|-------|
|
| 246 |
-
| Task 1 | ~0.67 | Good at classic vulns; struggles with subtle ones |
|
| 247 |
-
| Task 2 | ~0.55 | Reasonable properties; misses specific variable names |
|
| 248 |
-
| Task 3 | ~0.78 | Property text gives strong signal; usually correct in 3β4 steps |
|
| 249 |
-
|
| 250 |
-
---
|
| 251 |
-
|
| 252 |
-
## Evaluation Summary
|
| 253 |
-
|
| 254 |
-
Deterministic oracle / partial / floor tiers verified on 8 episodes (seeds 42β49):
|
| 255 |
-
|
| 256 |
-
| Task | Oracle | Partial/Sub | Floor | Ordering |
|
| 257 |
-
|------|--------|-------------|-------|----------|
|
| 258 |
-
| Task 1 | **1.000** | 0.500 | 0.000 | β
1.0 > 0.5 > 0.0 |
|
| 259 |
-
| Task 2 | **0.775** | 0.034 | 0.000 | β
0.775 > 0.034 > 0.0 |
|
| 260 |
-
| Task 3 | **1.000** | 0.037 | 0.000 | β
1.0 > 0.037 > 0.0 |
|
| 261 |
-
|
| 262 |
-
The clear separation across all three tasks confirms the graders provide **meaningful gradient signal** across the full reward range β a core requirement for RL training environments.
|
| 263 |
-
|
| 264 |
-
---
|
| 265 |
-
|
| 266 |
-
## OpenEnv Spec Compliance
|
| 267 |
-
|
| 268 |
-
| Requirement | Status |
|
| 269 |
-
|-------------|--------|
|
| 270 |
-
| Typed `Observation`, `Action`, `Reward` Pydantic models | β
|
|
| 271 |
-
| `step(action) β StepResult(obs, reward, done, info)` | β
|
|
| 272 |
-
| `reset() β ResetResult` | β
|
|
| 273 |
-
| `state() β StateResult` | β
|
|
| 274 |
-
| `openenv.yaml` metadata | β
|
|
| 275 |
-
| 3 tasks, all active | β
|
|
| 276 |
-
| Grader scores in [0.0, 1.0] | β
|
|
| 277 |
-
| Shaped rewards (non-binary signal) | β
|
|
| 278 |
-
| Dockerfile + port 7860 | β
|
|
| 279 |
-
| `inference.py` with OpenAI client | β
|
|
| 280 |
-
|
| 281 |
-
---
|
| 282 |
-
|
| 283 |
-
## Deploying to Hugging Face Spaces
|
| 284 |
-
|
| 285 |
-
```bash
|
| 286 |
-
# Copy the HF frontmatter into README.md, then:
|
| 287 |
-
git remote add hf https://huggingface.co/spaces/<user>/<space>
|
| 288 |
-
git push hf main
|
| 289 |
-
```
|
| 290 |
-
|
| 291 |
-
---
|
| 292 |
-
|
| 293 |
-
## License
|
| 294 |
-
|
| 295 |
-
MIT. Contract vulnerability patterns adapted from Certora audits on production DeFi protocols.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
LICENSE.txt
ADDED
|
@@ -0,0 +1,21 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
The MIT License (MIT)
|
| 2 |
+
|
| 3 |
+
Copyright (c) 2011-2026 The Bootstrap Authors
|
| 4 |
+
|
| 5 |
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
| 6 |
+
of this software and associated documentation files (the "Software"), to deal
|
| 7 |
+
in the Software without restriction, including without limitation the rights
|
| 8 |
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
| 9 |
+
copies of the Software, and to permit persons to whom the Software is
|
| 10 |
+
furnished to do so, subject to the following conditions:
|
| 11 |
+
|
| 12 |
+
The above copyright notice and this permission notice shall be included in
|
| 13 |
+
all copies or substantial portions of the Software.
|
| 14 |
+
|
| 15 |
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
| 16 |
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
| 17 |
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
| 18 |
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
| 19 |
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
| 20 |
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
|
| 21 |
+
THE SOFTWARE.
|
README.md
CHANGED
|
@@ -12,46 +12,410 @@ tags:
|
|
| 12 |
- solidity
|
| 13 |
- security
|
| 14 |
- evaluation
|
|
|
|
| 15 |
license: mit
|
| 16 |
short_description: OpenEnv RL environment for smart contract security auditing
|
| 17 |
---
|
| 18 |
|
| 19 |
-
# Smart Contract Audit RL Environment
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 20 |
|
| 21 |
-
|
| 22 |
|
| 23 |
-
|
| 24 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 25 |
|
| 26 |
-
## Quick
|
| 27 |
|
| 28 |
```bash
|
| 29 |
-
|
|
|
|
|
|
|
| 30 |
curl -X POST $SPACE_URL/reset \
|
| 31 |
-H "Content-Type: application/json" \
|
| 32 |
-d '{"task_id": "task1_vuln_detection", "seed": 42}'
|
| 33 |
|
| 34 |
-
#
|
| 35 |
curl -X POST $SPACE_URL/step \
|
| 36 |
-H "Content-Type: application/json" \
|
| 37 |
-d '{"action_type": "list_functions", "params": {}}'
|
| 38 |
|
| 39 |
-
#
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 40 |
curl -X POST $SPACE_URL/step \
|
| 41 |
-H "Content-Type: application/json" \
|
| 42 |
-d '{"action_type": "submit", "params": {"function_name": "withdraw", "vulnerability_type": "reentrancy"}}'
|
| 43 |
```
|
| 44 |
|
| 45 |
-
|
| 46 |
|
| 47 |
-
|
| 48 |
-
|
| 49 |
-
|
| 50 |
-
|
| 51 |
-
|
| 52 |
-
|
| 53 |
-
|
| 54 |
-
|
| 55 |
-
|
| 56 |
-
|
| 57 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 12 |
- solidity
|
| 13 |
- security
|
| 14 |
- evaluation
|
| 15 |
+
- openenv
|
| 16 |
license: mit
|
| 17 |
short_description: OpenEnv RL environment for smart contract security auditing
|
| 18 |
---
|
| 19 |
|
| 20 |
+
# π Smart Contract Audit RL Environment
|
| 21 |
+
|
| 22 |
+
> An OpenEnv-compliant reinforcement learning environment for training and evaluating AI agents on real-world Solidity smart contract security auditing tasks.
|
| 23 |
+
|
| 24 |
+
---
|
| 25 |
+
|
| 26 |
+
## Overview
|
| 27 |
+
|
| 28 |
+
Smart contract auditing is a high-stakes, expert-level task performed by professional security researchers. Mistakes cost millions β the Ethereum ecosystem has lost over **$3 billion** to exploits in audited and unaudited contracts alike. This environment simulates the core reasoning loop of a smart contract auditor, enabling RL agents to learn structured exploration strategies for vulnerability detection, property discovery, and rule checking.
|
| 29 |
+
|
| 30 |
+
The dataset is derived from real audit reports published by **[Certora](https://www.certora.com/)**, covering three production-grade DeFi protocols:
|
| 31 |
+
|
| 32 |
+
| Source | Protocol |
|
| 33 |
+
|---|---|
|
| 34 |
+
| Certora Audit | AaveVault |
|
| 35 |
+
| Certora Audit | AaveVaultV2 |
|
| 36 |
+
| Certora Audit | Lido Finance |
|
| 37 |
+
|
| 38 |
+
Each episode exposes a fragment of a real Solidity contract. The agent must use a structured action API β mirroring how a human auditor would methodically inspect a codebase β to accomplish a defined objective within a fixed step budget.
|
| 39 |
+
|
| 40 |
+
---
|
| 41 |
+
|
| 42 |
+
## Environment Architecture
|
| 43 |
+
|
| 44 |
+
```
|
| 45 |
+
SmartContractEnv/
|
| 46 |
+
βββ agents/
|
| 47 |
+
β βββ task1.py
|
| 48 |
+
β βββ task2.py
|
| 49 |
+
β βββ task3.py
|
| 50 |
+
βββ data/
|
| 51 |
+
β βββ __init__.py
|
| 52 |
+
β βββ contracts.json
|
| 53 |
+
β βββ data_loader.py
|
| 54 |
+
β βββ properties.csv
|
| 55 |
+
β βββ Template.json
|
| 56 |
+
β βββ vulnerabilities.json
|
| 57 |
+
β βββ vulnerabilities.md
|
| 58 |
+
βββ env/
|
| 59 |
+
β βββ __init__.py
|
| 60 |
+
β βββ base_env.py
|
| 61 |
+
β βββ schemas.py
|
| 62 |
+
βββ server/
|
| 63 |
+
β βββ tasks/
|
| 64 |
+
β β βββ task1/
|
| 65 |
+
β β β βββ __init__.py
|
| 66 |
+
β β β βββ actions.py
|
| 67 |
+
β β β βββ environment.py
|
| 68 |
+
β β β βββ grader.py
|
| 69 |
+
β β βββ task2/
|
| 70 |
+
β β β βββ __init__.py
|
| 71 |
+
β β β βββ actions.py
|
| 72 |
+
β β β βββ environment.py
|
| 73 |
+
β β β βββ grader.py
|
| 74 |
+
β β βββ task3/
|
| 75 |
+
β β β βββ __init__.py
|
| 76 |
+
β β β βββ actions.py
|
| 77 |
+
β β β βββ environment.py
|
| 78 |
+
β β β βββ grader.py
|
| 79 |
+
β β βββ __init__.py
|
| 80 |
+
β βββ __init__.py
|
| 81 |
+
β βββ app.py
|
| 82 |
+
βββ utils/
|
| 83 |
+
β βββ __init__.py
|
| 84 |
+
β βββ prompts.py
|
| 85 |
+
β βββ propertyretriever.py
|
| 86 |
+
β βββ semanticmatcher.py
|
| 87 |
+
βββ .env
|
| 88 |
+
βββ .gitignore
|
| 89 |
+
βββ demo.py
|
| 90 |
+
βββ Dockerfile
|
| 91 |
+
βββ Docs.md
|
| 92 |
+
βββ eval.py
|
| 93 |
+
βββ inference.py
|
| 94 |
+
βββ LICENSE.txt
|
| 95 |
+
βββ openenv.yaml
|
| 96 |
+
βββ pyproject.toml
|
| 97 |
+
βββ README.md
|
| 98 |
+
βββ requirements.txt
|
| 99 |
+
βββ validate-submission.sh
|
| 100 |
+
```
|
| 101 |
+
|
| 102 |
+
---
|
| 103 |
+
|
| 104 |
+
## Tasks
|
| 105 |
+
|
| 106 |
+
### Task 1 β Targeted Vulnerability Detection *(Medium)*
|
| 107 |
+
|
| 108 |
+
**Real-world analogue:** A security auditor is handed a Solidity file and asked to pinpoint the vulnerable function and describe the class of bug.
|
| 109 |
+
|
| 110 |
+
**Setup:** The agent receives a single Solidity file. The episode selects one vulnerable function at random from the dataset (7β8 available) on each `reset()`.
|
| 111 |
+
|
| 112 |
+
**Objective:** Identify the vulnerable function and describe its issue in 2β3 words (e.g., `"reentrancy"`, `"integer overflow"`, `"unchecked return value"`). Submit `"NO"` if no vulnerability exists.
|
| 113 |
+
|
| 114 |
+
**Action Space:**
|
| 115 |
+
|
| 116 |
+
| Action | Reward | Notes |
|
| 117 |
+
|---|---|---|
|
| 118 |
+
| `list_functions` | β0.05 | Returns all function signatures in the file |
|
| 119 |
+
| `get_function_code` | β0.10 (wrong fn) / +0.05 (correct fn) | Returns raw Solidity source of one function |
|
| 120 |
+
| `get_function_summary` | β0.05 (wrong) / +0.03 (correct) | Returns NatSpec comments for a function |
|
| 121 |
+
| `get_file_metadata` | β0.04 | Returns the file's header comment / pragma / imports |
|
| 122 |
+
| `get_state_variables` | β0.05 | Returns all contract-level state variable declarations |
|
| 123 |
+
| `get_call_graph` | β0.08 | Returns the inter-function call graph |
|
| 124 |
+
| `get_task_state` | 0.00 | Returns current step count and cumulative reward |
|
| 125 |
+
| `submit` | +5.00 (correct) / β1.50 (wrong) | One submission allowed per episode |
|
| 126 |
+
| *(repeated query)* | β0.40 | Penalty for querying the exact same action+params twice |
|
| 127 |
+
| *(unknown action)* | β0.20 | Any unrecognised action type |
|
| 128 |
+
|
| 129 |
+
**Episode terminates** on `submit` or when the step budget is exhausted.
|
| 130 |
+
|
| 131 |
+
---
|
| 132 |
+
|
| 133 |
+
### Task 2 β Property Discovery *(Hard)*
|
| 134 |
+
|
| 135 |
+
**Real-world analogue:** A formal verification engineer must derive an invariant or safety property for a contract function β the kind written as a Certora Verification Language (CVL) spec.
|
| 136 |
+
|
| 137 |
+
**Setup:** The agent receives a single function extracted from a Solidity file, along with a brief description of the broader contract. The episode targets a function that has a known, labelled property in the dataset.
|
| 138 |
+
|
| 139 |
+
**Objective:** Produce a natural-language description of the function's key safety property (e.g., *"The total shares minted must never exceed the total underlying assets deposited"*).
|
| 140 |
+
|
| 141 |
+
**Action Space:**
|
| 142 |
+
|
| 143 |
+
| Action | Reward | Notes |
|
| 144 |
+
|---|---|---|
|
| 145 |
+
| `get_file_natspec` | β0.03 | File-level NatSpec documentation |
|
| 146 |
+
| `get_function_natspec` | β0.08 | Function-level NatSpec comments |
|
| 147 |
+
| `get_function_code` | β0.06 | Raw Solidity source of the target function |
|
| 148 |
+
| `get_related_functions` | β0.06 | Functions that call or are called by the target |
|
| 149 |
+
| `get_input_output` | β0.04 | Parameter names/types and return values |
|
| 150 |
+
| `get_similar_property` | β0.20 | Hard-coded reference property from a different contract |
|
| 151 |
+
| `submit_property` | 0β5 (graded) | **One attempt per episode.** Scored by deterministic similarity checker |
|
| 152 |
+
|
| 153 |
+
**Grading:** Submission reward is computed by a deterministic checker that combines keyword overlap and structural similarity against the ground-truth property. Score is normalised to `[0, 5]` and then scaled to `[0.0, 1.0]` for the episode return.
|
| 154 |
+
|
| 155 |
+
---
|
| 156 |
+
|
| 157 |
+
### Task 3 β Rule Checker *(Easy)*
|
| 158 |
+
|
| 159 |
+
**Real-world analogue:** Given a known security rule (e.g., *"functions that transfer funds must emit a Transfer event"*), identify which function in the contract violates it.
|
| 160 |
+
|
| 161 |
+
**Setup:** The agent receives a Solidity file and a natural-language description of a property/rule. At least one function in the file violates this rule.
|
| 162 |
+
|
| 163 |
+
**Objective:** Identify the name of the rule-breaking function.
|
| 164 |
+
|
| 165 |
+
**Action Space:**
|
| 166 |
+
|
| 167 |
+
| Action | Reward | Notes |
|
| 168 |
+
|---|---|---|
|
| 169 |
+
| `get_property_specification` | β0.03 | Returns a pseudo-formal (CVL-like) version of the property |
|
| 170 |
+
| `list_functions` | β0.05 | All function signatures in the file |
|
| 171 |
+
| `get_function_metadata` | β0.05 | Visibility, modifiers, and signature for a function |
|
| 172 |
+
| `get_function_code` | β0.10 | Raw Solidity source of one function |
|
| 173 |
+
| `get_state_variables` | β0.05 | Contract-level state variable declarations |
|
| 174 |
+
| `get_call_graph` | β0.08 | Inter-function call graph |
|
| 175 |
+
| `submit` | +5.00 (exact) / +1.50 (sub-caller) / β1.50 (wrong) | One submission per episode |
|
| 176 |
+
|
| 177 |
+
**Partial credit:** If the agent names a function that *calls* the true violating function, it receives +1.50 rather than the full +5.00. This rewards reasoning that reaches the right area of the call graph.
|
| 178 |
+
|
| 179 |
+
---
|
| 180 |
+
|
| 181 |
+
## Reward Design
|
| 182 |
+
|
| 183 |
+
Rewards are shaped to encourage **efficient, targeted exploration** and discourage two failure modes: aimless browsing and brute-force guessing.
|
| 184 |
+
|
| 185 |
+
```
|
| 186 |
+
R_episode = Ξ£(step_rewards) + final_submission_reward
|
| 187 |
+
```
|
| 188 |
+
|
| 189 |
+
- **Exploration costs** are small and graduated by information value β cheap actions (metadata) cost less than expensive ones (full code retrieval).
|
| 190 |
+
- **Correct-direction bonuses** on `get_function_code` in Task 1 reward navigating toward the vulnerable function before committing.
|
| 191 |
+
- **Repetition penalty** (β0.40) discourages looping over the same queries.
|
| 192 |
+
- **Wrong submission** (β1.50) is painful enough to deter random guessing but recoverable through efficient prior exploration.
|
| 193 |
+
- **Episode score** is normalised to `[0.0, 1.0]` for the OpenEnv grader: `score = max(0, R_episode) / 5.0`.
|
| 194 |
+
|
| 195 |
+
---
|
| 196 |
+
|
| 197 |
+
## Observation Space
|
| 198 |
+
|
| 199 |
+
Every `step()` and `reset()` returns a typed `Observation` object:
|
| 200 |
+
|
| 201 |
+
```python
|
| 202 |
+
class Observation(BaseModel):
|
| 203 |
+
task_id: str # "task1_vuln_detection" | "task2_property_discovery" | "task3_rule_checker"
|
| 204 |
+
step: int # Current step index (0-indexed)
|
| 205 |
+
max_steps: int # Episode step budget
|
| 206 |
+
cumulative_reward: float # Running reward total
|
| 207 |
+
done: bool # Episode terminal flag
|
| 208 |
+
content: str # Main textual payload (code, summary, error, etc.)
|
| 209 |
+
metadata: dict[str, Any] # Extra context (function name, contract name, etc.)
|
| 210 |
+
initial_description: str # Persistent contract/task description shown every step
|
| 211 |
+
```
|
| 212 |
+
|
| 213 |
+
---
|
| 214 |
+
|
| 215 |
+
## Action Space
|
| 216 |
|
| 217 |
+
Actions are typed `Action` objects passed to `step()`:
|
| 218 |
|
| 219 |
+
```python
|
| 220 |
+
class Action(BaseModel):
|
| 221 |
+
action_type: str # One of the action names listed per task above
|
| 222 |
+
params: dict[str, str] # e.g. {"function_name": "withdraw"}
|
| 223 |
+
```
|
| 224 |
+
|
| 225 |
+
All unknown `action_type` values return a penalty observation without terminating the episode.
|
| 226 |
+
|
| 227 |
+
---
|
| 228 |
+
|
| 229 |
+
## OpenEnv Interface
|
| 230 |
+
|
| 231 |
+
The environment exposes a standard HTTP API:
|
| 232 |
+
|
| 233 |
+
| Method | Path | Description |
|
| 234 |
+
|---|---|---|
|
| 235 |
+
| `GET` | `/health` | Liveness probe β returns `{"status": "ok"}` |
|
| 236 |
+
| `GET` | `/tasks` | Lists all tasks with ID, difficulty, and status |
|
| 237 |
+
| `POST` | `/reset` | Starts a new episode. Body: `{"task_id": str, "seed": int}` |
|
| 238 |
+
| `POST` | `/step` | Takes one action. Body: `{"action_type": str, "params": {}}` |
|
| 239 |
+
| `GET` | `/state` | Returns full internal episode state (debug) |
|
| 240 |
+
| `GET` | `/action_space` | Returns JSON schema of valid actions |
|
| 241 |
+
| `GET` | `/observation_space` | Returns JSON schema of observation structure |
|
| 242 |
|
| 243 |
+
### Quick Start
|
| 244 |
|
| 245 |
```bash
|
| 246 |
+
SPACE_URL=http://localhost:7860
|
| 247 |
+
|
| 248 |
+
# Start a new episode for Task 1
|
| 249 |
curl -X POST $SPACE_URL/reset \
|
| 250 |
-H "Content-Type: application/json" \
|
| 251 |
-d '{"task_id": "task1_vuln_detection", "seed": 42}'
|
| 252 |
|
| 253 |
+
# List all functions in the contract
|
| 254 |
curl -X POST $SPACE_URL/step \
|
| 255 |
-H "Content-Type: application/json" \
|
| 256 |
-d '{"action_type": "list_functions", "params": {}}'
|
| 257 |
|
| 258 |
+
# Inspect a specific function
|
| 259 |
+
curl -X POST $SPACE_URL/step \
|
| 260 |
+
-H "Content-Type: application/json" \
|
| 261 |
+
-d '{"action_type": "get_function_code", "params": {"function_name": "withdraw"}}'
|
| 262 |
+
|
| 263 |
+
# Submit your answer
|
| 264 |
curl -X POST $SPACE_URL/step \
|
| 265 |
-H "Content-Type: application/json" \
|
| 266 |
-d '{"action_type": "submit", "params": {"function_name": "withdraw", "vulnerability_type": "reentrancy"}}'
|
| 267 |
```
|
| 268 |
|
| 269 |
+
---
|
| 270 |
|
| 271 |
+
## Setup & Installation
|
| 272 |
+
|
| 273 |
+
### Prerequisites
|
| 274 |
+
|
| 275 |
+
- Docker β₯ 20.10
|
| 276 |
+
- Python 3.11+ (for local development)
|
| 277 |
+
- `OPENAI_API_KEY`, `API_BASE_URL`, `MODEL_NAME`, `HF_TOKEN` environment variables
|
| 278 |
+
|
| 279 |
+
### Run with Docker
|
| 280 |
+
|
| 281 |
+
```bash
|
| 282 |
+
# Build the image
|
| 283 |
+
docker build -t sc-audit-env .
|
| 284 |
+
|
| 285 |
+
# Run the container
|
| 286 |
+
docker run -p 7860:7860 \
|
| 287 |
+
-e OPENAI_API_KEY=$OPENAI_API_KEY \
|
| 288 |
+
-e API_BASE_URL=$API_BASE_URL \
|
| 289 |
+
-e MODEL_NAME=$MODEL_NAME \
|
| 290 |
+
sc-audit-env
|
| 291 |
+
|
| 292 |
+
# Verify it's running
|
| 293 |
+
curl http://localhost:7860/health
|
| 294 |
+
```
|
| 295 |
+
|
| 296 |
+
### Run Locally (Development)
|
| 297 |
+
|
| 298 |
+
```bash
|
| 299 |
+
pip install -r requirements.txt
|
| 300 |
+
uvicorn api.app:app --host 0.0.0.0 --port 7860 --reload
|
| 301 |
+
```
|
| 302 |
+
|
| 303 |
+
### Validate OpenEnv Compliance
|
| 304 |
+
|
| 305 |
+
```bash
|
| 306 |
+
openenv validate openenv.yaml
|
| 307 |
+
```
|
| 308 |
+
|
| 309 |
+
---
|
| 310 |
+
|
| 311 |
+
## Baseline Inference Script
|
| 312 |
+
|
| 313 |
+
The `inference.py` script runs an OpenAI-compatible model against all three tasks and reports episode scores. It reads credentials from environment variables and completes in under 20 minutes on a 2 vCPU / 8 GB machine.
|
| 314 |
+
|
| 315 |
+
```bash
|
| 316 |
+
export OPENAI_API_KEY=your_key
|
| 317 |
+
export API_BASE_URL=https://api.openai.com/v1 # or your custom endpoint
|
| 318 |
+
export MODEL_NAME=gpt-4o-mini
|
| 319 |
+
|
| 320 |
+
python inference.py
|
| 321 |
+
```
|
| 322 |
+
|
| 323 |
+
**Expected output:**
|
| 324 |
+
|
| 325 |
+
```
|
| 326 |
+
=== Smart Contract Audit RL β Baseline Evaluation ===
|
| 327 |
+
|
| 328 |
+
Task 1 | Targeted Vulnerability Detection | Score: 0.41 | Steps used: 8/15
|
| 329 |
+
Task 2 | Property Discovery | Score: 0.28 | Steps used: 6/10
|
| 330 |
+
Task 3 | Rule Checker | Score: 0.72 | Steps used: 4/10
|
| 331 |
+
|
| 332 |
+
Overall average: 0.47
|
| 333 |
+
```
|
| 334 |
+
|
| 335 |
+
> **Note:** Scores are stochastic due to random episode selection on `reset()`. Run with a fixed seed (`--seed 42`) for reproducible results.
|
| 336 |
+
|
| 337 |
+
### Agent System Prompt
|
| 338 |
+
|
| 339 |
+
The inference script injects the following system prompt to guide output format:
|
| 340 |
+
|
| 341 |
+
```
|
| 342 |
+
You are a smart contract security auditor. You will be given access to a Solidity
|
| 343 |
+
contract via a structured action API. Use the available actions to investigate the
|
| 344 |
+
contract, then submit your answer.
|
| 345 |
+
|
| 346 |
+
Always respond with a single JSON object:
|
| 347 |
+
{"action_type": "<action>", "params": {"<key>": "<value>"}}
|
| 348 |
+
|
| 349 |
+
Do not include any other text outside the JSON object.
|
| 350 |
+
```
|
| 351 |
+
|
| 352 |
+
---
|
| 353 |
+
|
| 354 |
+
## openenv.yaml
|
| 355 |
+
|
| 356 |
+
```yaml
|
| 357 |
+
name: smart-contract-audit-env
|
| 358 |
+
version: "1.0.0"
|
| 359 |
+
description: >
|
| 360 |
+
OpenEnv RL environment for Solidity smart contract security auditing.
|
| 361 |
+
Agents explore real-world DeFi contracts using a structured action API
|
| 362 |
+
to detect vulnerabilities, discover properties, and check rule compliance.
|
| 363 |
+
tasks:
|
| 364 |
+
- id: task1_vuln_detection
|
| 365 |
+
name: Targeted Vulnerability Detection
|
| 366 |
+
difficulty: medium
|
| 367 |
+
max_steps: 15
|
| 368 |
+
max_score: 5.0
|
| 369 |
+
- id: task2_property_discovery
|
| 370 |
+
name: Property Discovery
|
| 371 |
+
difficulty: hard
|
| 372 |
+
max_steps: 10
|
| 373 |
+
max_score: 5.0
|
| 374 |
+
- id: task3_rule_checker
|
| 375 |
+
name: Rule Checker
|
| 376 |
+
difficulty: easy
|
| 377 |
+
max_steps: 10
|
| 378 |
+
max_score: 5.0
|
| 379 |
+
observation_schema: models/observation.py
|
| 380 |
+
action_schema: models/action.py
|
| 381 |
+
app_port: 7860
|
| 382 |
+
```
|
| 383 |
+
|
| 384 |
+
---
|
| 385 |
+
|
| 386 |
+
## Data
|
| 387 |
+
|
| 388 |
+
The dataset (`data/dataset.json`) contains **7β8 labelled entries** per contract, each with format accoding to `data/template.json`:
|
| 389 |
+
Ground truth is **never exposed** to the agent via any action. The `submit` action is the only path to positive reward.
|
| 390 |
+
|
| 391 |
+
---
|
| 392 |
+
|
| 393 |
+
## Design Notes & Known Limitations
|
| 394 |
+
|
| 395 |
+
- **Reward calibration:** Step penalties and submission rewards may need tuning based on empirical agent performance. Current values are derived from initial design rationale, not from extensive ablation.
|
| 396 |
+
- **Call graph granularity:** The current `get_call_graph` action returns the entire graph at once. A future revision could expose it incrementally (per-function neighbours) to make the action more informative and cost-proportional.
|
| 397 |
+
- **Vulnerability naming:** Vulnerability types do not follow a fixed taxonomy. Grading uses keyword + semantic matching against a curated synonym list (e.g., `"re-entrancy"` β‘ `"reentrancy"`).
|
| 398 |
+
- **Dataset size:** The current dataset covers 3 contracts with 7β8 vulnerabilities each. Expanding to more Certora audit reports would improve task diversity and reduce overfitting risk.
|
| 399 |
+
- **`get_function_code` decomposition:** This action could be split into finer-grained sub-actions (`get_parameters`, `get_return_values`, `get_modifiers`) to give agents a more gradual information ladder.
|
| 400 |
+
- **Property similarity scoring (Task 2):** Sentence transformer models cannot be used in the containerised environment due to memory constraints. The checker instead uses TF-IDF cosine similarity combined with keyword matching against the ground-truth property.
|
| 401 |
+
|
| 402 |
+
---
|
| 403 |
+
|
| 404 |
+
## License
|
| 405 |
+
|
| 406 |
+
MIT β see `LICENSE` for details.
|
| 407 |
+
|
| 408 |
+
Data sourced from public Certora audit reports. Solidity source files are reproduced for research and evaluation purposes.
|
| 409 |
+
|
| 410 |
+
---
|
| 411 |
+
|
| 412 |
+
## Citation
|
| 413 |
+
|
| 414 |
+
```bibtex
|
| 415 |
+
@misc{sc-audit-openenv-2025,
|
| 416 |
+
title = {Smart Contract Audit RL Environment},
|
| 417 |
+
year = {2025},
|
| 418 |
+
note = {OpenEnv-compliant RL environment for Solidity security analysis.
|
| 419 |
+
Data sourced from Certora audit reports (AaveVault, AaveVaultV2, Lido Finance).}
|
| 420 |
+
}
|
| 421 |
+
```
|
server/app.py
CHANGED
|
@@ -17,8 +17,10 @@ If omitted, "default" is used (fine for sequential single-agent runs).
|
|
| 17 |
"""
|
| 18 |
|
| 19 |
from typing import Dict, Optional, Union
|
|
|
|
| 20 |
|
| 21 |
-
from fastapi import FastAPI, HTTPException, Query
|
|
|
|
| 22 |
from pydantic import BaseModel
|
| 23 |
|
| 24 |
from env.schemas import Action, ActionType, TaskInfo
|
|
@@ -75,6 +77,32 @@ class ResetRequest(BaseModel):
|
|
| 75 |
class StepRequest(BaseModel):
|
| 76 |
action_type: str
|
| 77 |
params: dict = {}
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 78 |
|
| 79 |
|
| 80 |
# βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
|
@@ -82,8 +110,16 @@ class StepRequest(BaseModel):
|
|
| 82 |
# βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
| 83 |
|
| 84 |
@app.get("/")
|
| 85 |
-
def root():
|
| 86 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 87 |
|
| 88 |
@app.get("/health")
|
| 89 |
def health():
|
|
|
|
| 17 |
"""
|
| 18 |
|
| 19 |
from typing import Dict, Optional, Union
|
| 20 |
+
from zipfile import Path
|
| 21 |
|
| 22 |
+
from fastapi import FastAPI, HTTPException, Query, Request
|
| 23 |
+
from fastapi.responses import FileResponse, JSONResponse
|
| 24 |
from pydantic import BaseModel
|
| 25 |
|
| 26 |
from env.schemas import Action, ActionType, TaskInfo
|
|
|
|
| 77 |
class StepRequest(BaseModel):
|
| 78 |
action_type: str
|
| 79 |
params: dict = {}
|
| 80 |
+
|
| 81 |
+
|
| 82 |
+
_ROOT_JSON = {
|
| 83 |
+
"name": "Smart Contract Audit RL Environment",
|
| 84 |
+
"version": "1.2.0",
|
| 85 |
+
"description": (
|
| 86 |
+
"OpenEnv-compliant RL environment for Solidity smart contract security analysis. "
|
| 87 |
+
"Train and evaluate agents on real-world DeFi audit tasks from Certora reports."
|
| 88 |
+
),
|
| 89 |
+
"tasks": [
|
| 90 |
+
{"id": "task1_vuln_detection", "name": "Targeted Vulnerability Detection", "difficulty": "medium"},
|
| 91 |
+
{"id": "task2_property_discovery", "name": "Property Discovery", "difficulty": "hard"},
|
| 92 |
+
{"id": "task3_rule_checker", "name": "Rule Checker", "difficulty": "easy"},
|
| 93 |
+
],
|
| 94 |
+
"endpoints": {
|
| 95 |
+
"reset": "POST /reset",
|
| 96 |
+
"step": "POST /step",
|
| 97 |
+
"state": "GET /state",
|
| 98 |
+
"tasks": "GET /tasks",
|
| 99 |
+
"health": "GET /health",
|
| 100 |
+
"action_space": "GET /action_space",
|
| 101 |
+
"observation_space": "GET /observation_space",
|
| 102 |
+
"docs": "GET /docs",
|
| 103 |
+
},
|
| 104 |
+
"data_sources": ["AaveVault", "AaveVaultV2", "Lido Finance"],
|
| 105 |
+
}
|
| 106 |
|
| 107 |
|
| 108 |
# βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
|
|
|
| 110 |
# βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
| 111 |
|
| 112 |
@app.get("/")
|
| 113 |
+
def root(request: Request):
|
| 114 |
+
"""
|
| 115 |
+
Landing page.
|
| 116 |
+
- Browsers (Accept: text/html) β polished HTML UI
|
| 117 |
+
- API clients (Accept: */*) β JSON summary
|
| 118 |
+
"""
|
| 119 |
+
accept = request.headers.get("accept", "")
|
| 120 |
+
if "text/html" in accept and Path("./index.html").is_file():
|
| 121 |
+
return FileResponse("./index.html", media_type="text/html", status_code=200)
|
| 122 |
+
return JSONResponse(content=_ROOT_JSON, status_code=200)
|
| 123 |
|
| 124 |
@app.get("/health")
|
| 125 |
def health():
|
server/index.html
ADDED
|
@@ -0,0 +1,708 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
|
| 2 |
+
# βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
| 3 |
+
# Landing page HTML
|
| 4 |
+
# βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
| 5 |
+
|
| 6 |
+
LANDING_HTML = """<!DOCTYPE html>
|
| 7 |
+
<html lang="en">
|
| 8 |
+
<head>
|
| 9 |
+
<meta charset="UTF-8" />
|
| 10 |
+
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
|
| 11 |
+
<title>SC Audit RL Environment</title>
|
| 12 |
+
<link rel="preconnect" href="https://fonts.googleapis.com" />
|
| 13 |
+
<link href="https://fonts.googleapis.com/css2?family=Space+Mono:wght@400;700&family=DM+Sans:wght@300;400;500&display=swap" rel="stylesheet" />
|
| 14 |
+
<style>
|
| 15 |
+
*, *::before, *::after { box-sizing: border-box; margin: 0; padding: 0; }
|
| 16 |
+
|
| 17 |
+
:root {
|
| 18 |
+
--bg: #0a0c0f;
|
| 19 |
+
--surface: #111418;
|
| 20 |
+
--border: #1e2530;
|
| 21 |
+
--accent: #00ff88;
|
| 22 |
+
--accent2: #00b8ff;
|
| 23 |
+
--warn: #ffaa00;
|
| 24 |
+
--danger: #ff4455;
|
| 25 |
+
--text: #c8d0dc;
|
| 26 |
+
--muted: #4a5568;
|
| 27 |
+
--mono: 'Space Mono', monospace;
|
| 28 |
+
--sans: 'DM Sans', sans-serif;
|
| 29 |
+
}
|
| 30 |
+
|
| 31 |
+
html { scroll-behavior: smooth; }
|
| 32 |
+
|
| 33 |
+
body {
|
| 34 |
+
background: var(--bg);
|
| 35 |
+
color: var(--text);
|
| 36 |
+
font-family: var(--sans);
|
| 37 |
+
font-size: 15px;
|
| 38 |
+
line-height: 1.6;
|
| 39 |
+
min-height: 100vh;
|
| 40 |
+
overflow-x: hidden;
|
| 41 |
+
}
|
| 42 |
+
|
| 43 |
+
/* ββ noise overlay ββ */
|
| 44 |
+
body::before {
|
| 45 |
+
content: '';
|
| 46 |
+
position: fixed; inset: 0;
|
| 47 |
+
background-image: url("data:image/svg+xml,%3Csvg viewBox='0 0 256 256' xmlns='http://www.w3.org/2000/svg'%3E%3Cfilter id='n'%3E%3CfeTurbulence type='fractalNoise' baseFrequency='0.9' numOctaves='4' stitchTiles='stitch'/%3E%3C/filter%3E%3Crect width='100%25' height='100%25' filter='url(%23n)' opacity='0.04'/%3E%3C/svg%3E");
|
| 48 |
+
pointer-events: none;
|
| 49 |
+
z-index: 0;
|
| 50 |
+
opacity: .5;
|
| 51 |
+
}
|
| 52 |
+
|
| 53 |
+
/* ββ grid lines ββ */
|
| 54 |
+
body::after {
|
| 55 |
+
content: '';
|
| 56 |
+
position: fixed; inset: 0;
|
| 57 |
+
background-image:
|
| 58 |
+
linear-gradient(var(--border) 1px, transparent 1px),
|
| 59 |
+
linear-gradient(90deg, var(--border) 1px, transparent 1px);
|
| 60 |
+
background-size: 40px 40px;
|
| 61 |
+
pointer-events: none;
|
| 62 |
+
z-index: 0;
|
| 63 |
+
opacity: .35;
|
| 64 |
+
}
|
| 65 |
+
|
| 66 |
+
/* βββ layout βββ */
|
| 67 |
+
.wrap { position: relative; z-index: 1; max-width: 1100px; margin: 0 auto; padding: 0 28px; }
|
| 68 |
+
|
| 69 |
+
/* βββ header βββ */
|
| 70 |
+
header {
|
| 71 |
+
border-bottom: 1px solid var(--border);
|
| 72 |
+
padding: 18px 0;
|
| 73 |
+
}
|
| 74 |
+
.nav {
|
| 75 |
+
display: flex;
|
| 76 |
+
align-items: center;
|
| 77 |
+
justify-content: space-between;
|
| 78 |
+
}
|
| 79 |
+
.logo {
|
| 80 |
+
font-family: var(--mono);
|
| 81 |
+
font-size: 13px;
|
| 82 |
+
color: var(--accent);
|
| 83 |
+
letter-spacing: .05em;
|
| 84 |
+
display: flex;
|
| 85 |
+
align-items: center;
|
| 86 |
+
gap: 8px;
|
| 87 |
+
}
|
| 88 |
+
.logo-dot {
|
| 89 |
+
width: 8px; height: 8px;
|
| 90 |
+
border-radius: 50%;
|
| 91 |
+
background: var(--accent);
|
| 92 |
+
box-shadow: 0 0 8px var(--accent);
|
| 93 |
+
animation: pulse 2s ease-in-out infinite;
|
| 94 |
+
}
|
| 95 |
+
@keyframes pulse {
|
| 96 |
+
0%,100% { opacity: 1; }
|
| 97 |
+
50% { opacity: .35; }
|
| 98 |
+
}
|
| 99 |
+
.nav-links {
|
| 100 |
+
display: flex;
|
| 101 |
+
gap: 24px;
|
| 102 |
+
list-style: none;
|
| 103 |
+
}
|
| 104 |
+
.nav-links a {
|
| 105 |
+
font-family: var(--mono);
|
| 106 |
+
font-size: 11px;
|
| 107 |
+
color: var(--muted);
|
| 108 |
+
text-decoration: none;
|
| 109 |
+
letter-spacing: .08em;
|
| 110 |
+
text-transform: uppercase;
|
| 111 |
+
transition: color .2s;
|
| 112 |
+
}
|
| 113 |
+
.nav-links a:hover { color: var(--accent); }
|
| 114 |
+
|
| 115 |
+
/* βββ hero βββ */
|
| 116 |
+
.hero {
|
| 117 |
+
padding: 90px 0 70px;
|
| 118 |
+
position: relative;
|
| 119 |
+
}
|
| 120 |
+
.hero-tag {
|
| 121 |
+
font-family: var(--mono);
|
| 122 |
+
font-size: 11px;
|
| 123 |
+
color: var(--accent);
|
| 124 |
+
letter-spacing: .12em;
|
| 125 |
+
text-transform: uppercase;
|
| 126 |
+
margin-bottom: 22px;
|
| 127 |
+
display: flex;
|
| 128 |
+
align-items: center;
|
| 129 |
+
gap: 10px;
|
| 130 |
+
}
|
| 131 |
+
.hero-tag::before {
|
| 132 |
+
content: '';
|
| 133 |
+
display: inline-block;
|
| 134 |
+
width: 28px; height: 1px;
|
| 135 |
+
background: var(--accent);
|
| 136 |
+
}
|
| 137 |
+
h1 {
|
| 138 |
+
font-family: var(--mono);
|
| 139 |
+
font-size: clamp(28px, 5vw, 52px);
|
| 140 |
+
font-weight: 700;
|
| 141 |
+
line-height: 1.15;
|
| 142 |
+
color: #fff;
|
| 143 |
+
margin-bottom: 26px;
|
| 144 |
+
letter-spacing: -.02em;
|
| 145 |
+
}
|
| 146 |
+
h1 span {
|
| 147 |
+
color: var(--accent);
|
| 148 |
+
text-shadow: 0 0 30px rgba(0,255,136,.3);
|
| 149 |
+
}
|
| 150 |
+
.hero-sub {
|
| 151 |
+
font-size: 16px;
|
| 152 |
+
color: var(--muted);
|
| 153 |
+
max-width: 560px;
|
| 154 |
+
line-height: 1.7;
|
| 155 |
+
margin-bottom: 40px;
|
| 156 |
+
}
|
| 157 |
+
.hero-badges {
|
| 158 |
+
display: flex;
|
| 159 |
+
flex-wrap: wrap;
|
| 160 |
+
gap: 10px;
|
| 161 |
+
margin-bottom: 44px;
|
| 162 |
+
}
|
| 163 |
+
.badge {
|
| 164 |
+
font-family: var(--mono);
|
| 165 |
+
font-size: 11px;
|
| 166 |
+
padding: 5px 12px;
|
| 167 |
+
border-radius: 3px;
|
| 168 |
+
letter-spacing: .06em;
|
| 169 |
+
}
|
| 170 |
+
.badge-green { background: rgba(0,255,136,.08); color: var(--accent); border: 1px solid rgba(0,255,136,.25); }
|
| 171 |
+
.badge-blue { background: rgba(0,184,255,.08); color: var(--accent2); border: 1px solid rgba(0,184,255,.25); }
|
| 172 |
+
.badge-yellow { background: rgba(255,170,0,.08); color: var(--warn); border: 1px solid rgba(255,170,0,.25); }
|
| 173 |
+
|
| 174 |
+
.cta-row { display: flex; gap: 14px; flex-wrap: wrap; }
|
| 175 |
+
.btn {
|
| 176 |
+
font-family: var(--mono);
|
| 177 |
+
font-size: 12px;
|
| 178 |
+
padding: 11px 22px;
|
| 179 |
+
border-radius: 4px;
|
| 180 |
+
text-decoration: none;
|
| 181 |
+
letter-spacing: .06em;
|
| 182 |
+
transition: all .2s;
|
| 183 |
+
cursor: pointer;
|
| 184 |
+
border: none;
|
| 185 |
+
}
|
| 186 |
+
.btn-primary {
|
| 187 |
+
background: var(--accent);
|
| 188 |
+
color: #000;
|
| 189 |
+
font-weight: 700;
|
| 190 |
+
}
|
| 191 |
+
.btn-primary:hover {
|
| 192 |
+
background: #fff;
|
| 193 |
+
box-shadow: 0 0 24px rgba(0,255,136,.4);
|
| 194 |
+
}
|
| 195 |
+
.btn-outline {
|
| 196 |
+
background: transparent;
|
| 197 |
+
color: var(--text);
|
| 198 |
+
border: 1px solid var(--border);
|
| 199 |
+
}
|
| 200 |
+
.btn-outline:hover {
|
| 201 |
+
border-color: var(--accent);
|
| 202 |
+
color: var(--accent);
|
| 203 |
+
}
|
| 204 |
+
|
| 205 |
+
/* βββ section βββ */
|
| 206 |
+
section { padding: 64px 0; border-top: 1px solid var(--border); }
|
| 207 |
+
.section-label {
|
| 208 |
+
font-family: var(--mono);
|
| 209 |
+
font-size: 10px;
|
| 210 |
+
color: var(--muted);
|
| 211 |
+
letter-spacing: .14em;
|
| 212 |
+
text-transform: uppercase;
|
| 213 |
+
margin-bottom: 36px;
|
| 214 |
+
}
|
| 215 |
+
h2 {
|
| 216 |
+
font-family: var(--mono);
|
| 217 |
+
font-size: 22px;
|
| 218 |
+
font-weight: 700;
|
| 219 |
+
color: #fff;
|
| 220 |
+
margin-bottom: 8px;
|
| 221 |
+
}
|
| 222 |
+
h3 {
|
| 223 |
+
font-family: var(--mono);
|
| 224 |
+
font-size: 14px;
|
| 225 |
+
color: #fff;
|
| 226 |
+
margin-bottom: 10px;
|
| 227 |
+
}
|
| 228 |
+
|
| 229 |
+
/* βββ task cards βββ */
|
| 230 |
+
.task-grid {
|
| 231 |
+
display: grid;
|
| 232 |
+
grid-template-columns: repeat(auto-fit, minmax(300px, 1fr));
|
| 233 |
+
gap: 18px;
|
| 234 |
+
margin-top: 32px;
|
| 235 |
+
}
|
| 236 |
+
.task-card {
|
| 237 |
+
background: var(--surface);
|
| 238 |
+
border: 1px solid var(--border);
|
| 239 |
+
border-radius: 8px;
|
| 240 |
+
padding: 26px;
|
| 241 |
+
position: relative;
|
| 242 |
+
overflow: hidden;
|
| 243 |
+
transition: border-color .25s, transform .25s;
|
| 244 |
+
}
|
| 245 |
+
.task-card:hover {
|
| 246 |
+
border-color: var(--accent);
|
| 247 |
+
transform: translateY(-3px);
|
| 248 |
+
}
|
| 249 |
+
.task-card::before {
|
| 250 |
+
content: '';
|
| 251 |
+
position: absolute;
|
| 252 |
+
top: 0; left: 0; right: 0;
|
| 253 |
+
height: 2px;
|
| 254 |
+
}
|
| 255 |
+
.task-card.easy::before { background: var(--accent); }
|
| 256 |
+
.task-card.medium::before { background: var(--warn); }
|
| 257 |
+
.task-card.hard::before { background: var(--danger); }
|
| 258 |
+
|
| 259 |
+
.task-header {
|
| 260 |
+
display: flex;
|
| 261 |
+
justify-content: space-between;
|
| 262 |
+
align-items: flex-start;
|
| 263 |
+
margin-bottom: 14px;
|
| 264 |
+
}
|
| 265 |
+
.task-num {
|
| 266 |
+
font-family: var(--mono);
|
| 267 |
+
font-size: 11px;
|
| 268 |
+
color: var(--muted);
|
| 269 |
+
}
|
| 270 |
+
.diff-badge {
|
| 271 |
+
font-family: var(--mono);
|
| 272 |
+
font-size: 10px;
|
| 273 |
+
padding: 3px 9px;
|
| 274 |
+
border-radius: 99px;
|
| 275 |
+
letter-spacing: .07em;
|
| 276 |
+
text-transform: uppercase;
|
| 277 |
+
}
|
| 278 |
+
.diff-easy { background: rgba(0,255,136,.1); color: var(--accent); }
|
| 279 |
+
.diff-medium { background: rgba(255,170,0,.1); color: var(--warn); }
|
| 280 |
+
.diff-hard { background: rgba(255,68,85,.1); color: var(--danger); }
|
| 281 |
+
|
| 282 |
+
.task-desc {
|
| 283 |
+
font-size: 13px;
|
| 284 |
+
color: var(--muted);
|
| 285 |
+
line-height: 1.65;
|
| 286 |
+
margin-bottom: 18px;
|
| 287 |
+
}
|
| 288 |
+
.task-actions {
|
| 289 |
+
display: flex;
|
| 290 |
+
flex-wrap: wrap;
|
| 291 |
+
gap: 6px;
|
| 292 |
+
}
|
| 293 |
+
.action-chip {
|
| 294 |
+
font-family: var(--mono);
|
| 295 |
+
font-size: 10px;
|
| 296 |
+
padding: 3px 8px;
|
| 297 |
+
border-radius: 3px;
|
| 298 |
+
background: rgba(255,255,255,.04);
|
| 299 |
+
color: var(--muted);
|
| 300 |
+
border: 1px solid var(--border);
|
| 301 |
+
}
|
| 302 |
+
|
| 303 |
+
/* βββ endpoints table βββ */
|
| 304 |
+
.endpoint-list { margin-top: 28px; display: flex; flex-direction: column; gap: 2px; }
|
| 305 |
+
.endpoint-row {
|
| 306 |
+
display: grid;
|
| 307 |
+
grid-template-columns: 70px 220px 1fr;
|
| 308 |
+
gap: 16px;
|
| 309 |
+
align-items: center;
|
| 310 |
+
padding: 12px 18px;
|
| 311 |
+
background: var(--surface);
|
| 312 |
+
border: 1px solid var(--border);
|
| 313 |
+
border-radius: 4px;
|
| 314 |
+
font-size: 13px;
|
| 315 |
+
transition: border-color .2s;
|
| 316 |
+
}
|
| 317 |
+
.endpoint-row:hover { border-color: var(--muted); }
|
| 318 |
+
.method {
|
| 319 |
+
font-family: var(--mono);
|
| 320 |
+
font-size: 11px;
|
| 321 |
+
font-weight: 700;
|
| 322 |
+
padding: 3px 8px;
|
| 323 |
+
border-radius: 3px;
|
| 324 |
+
text-align: center;
|
| 325 |
+
letter-spacing: .04em;
|
| 326 |
+
}
|
| 327 |
+
.method-get { background: rgba(0,184,255,.12); color: var(--accent2); }
|
| 328 |
+
.method-post { background: rgba(0,255,136,.12); color: var(--accent); }
|
| 329 |
+
.path { font-family: var(--mono); font-size: 13px; color: #fff; }
|
| 330 |
+
.ep-desc { color: var(--muted); font-size: 13px; }
|
| 331 |
+
|
| 332 |
+
/* βββ code block βββ */
|
| 333 |
+
.code-wrap {
|
| 334 |
+
margin-top: 28px;
|
| 335 |
+
background: var(--surface);
|
| 336 |
+
border: 1px solid var(--border);
|
| 337 |
+
border-radius: 8px;
|
| 338 |
+
overflow: hidden;
|
| 339 |
+
}
|
| 340 |
+
.code-header {
|
| 341 |
+
display: flex;
|
| 342 |
+
align-items: center;
|
| 343 |
+
justify-content: space-between;
|
| 344 |
+
padding: 10px 18px;
|
| 345 |
+
border-bottom: 1px solid var(--border);
|
| 346 |
+
}
|
| 347 |
+
.code-title { font-family: var(--mono); font-size: 11px; color: var(--muted); }
|
| 348 |
+
.copy-btn {
|
| 349 |
+
font-family: var(--mono);
|
| 350 |
+
font-size: 10px;
|
| 351 |
+
color: var(--muted);
|
| 352 |
+
background: none;
|
| 353 |
+
border: 1px solid var(--border);
|
| 354 |
+
border-radius: 3px;
|
| 355 |
+
padding: 3px 10px;
|
| 356 |
+
cursor: pointer;
|
| 357 |
+
transition: all .2s;
|
| 358 |
+
}
|
| 359 |
+
.copy-btn:hover { color: var(--accent); border-color: var(--accent); }
|
| 360 |
+
pre {
|
| 361 |
+
margin: 0;
|
| 362 |
+
padding: 22px;
|
| 363 |
+
overflow-x: auto;
|
| 364 |
+
font-family: var(--mono);
|
| 365 |
+
font-size: 12px;
|
| 366 |
+
line-height: 1.75;
|
| 367 |
+
color: var(--text);
|
| 368 |
+
}
|
| 369 |
+
.kw { color: var(--accent2); }
|
| 370 |
+
.str { color: var(--accent); }
|
| 371 |
+
.cm { color: var(--muted); }
|
| 372 |
+
.pn { color: var(--warn); }
|
| 373 |
+
|
| 374 |
+
/* βββ data sources βββ */
|
| 375 |
+
.source-grid {
|
| 376 |
+
display: grid;
|
| 377 |
+
grid-template-columns: repeat(auto-fit, minmax(240px, 1fr));
|
| 378 |
+
gap: 16px;
|
| 379 |
+
margin-top: 28px;
|
| 380 |
+
}
|
| 381 |
+
.source-card {
|
| 382 |
+
background: var(--surface);
|
| 383 |
+
border: 1px solid var(--border);
|
| 384 |
+
border-radius: 8px;
|
| 385 |
+
padding: 22px;
|
| 386 |
+
display: flex;
|
| 387 |
+
gap: 16px;
|
| 388 |
+
align-items: flex-start;
|
| 389 |
+
}
|
| 390 |
+
.source-icon {
|
| 391 |
+
font-size: 22px;
|
| 392 |
+
flex-shrink: 0;
|
| 393 |
+
line-height: 1;
|
| 394 |
+
}
|
| 395 |
+
.source-name {
|
| 396 |
+
font-family: var(--mono);
|
| 397 |
+
font-size: 13px;
|
| 398 |
+
color: #fff;
|
| 399 |
+
margin-bottom: 4px;
|
| 400 |
+
}
|
| 401 |
+
.source-meta { font-size: 12px; color: var(--muted); }
|
| 402 |
+
|
| 403 |
+
/* βββ footer βββ */
|
| 404 |
+
footer {
|
| 405 |
+
border-top: 1px solid var(--border);
|
| 406 |
+
padding: 28px 0;
|
| 407 |
+
margin-top: 40px;
|
| 408 |
+
}
|
| 409 |
+
.footer-inner {
|
| 410 |
+
display: flex;
|
| 411 |
+
justify-content: space-between;
|
| 412 |
+
align-items: center;
|
| 413 |
+
flex-wrap: wrap;
|
| 414 |
+
gap: 14px;
|
| 415 |
+
}
|
| 416 |
+
.footer-left { font-family: var(--mono); font-size: 11px; color: var(--muted); }
|
| 417 |
+
.footer-links { display: flex; gap: 20px; }
|
| 418 |
+
.footer-links a {
|
| 419 |
+
font-family: var(--mono);
|
| 420 |
+
font-size: 11px;
|
| 421 |
+
color: var(--muted);
|
| 422 |
+
text-decoration: none;
|
| 423 |
+
transition: color .2s;
|
| 424 |
+
}
|
| 425 |
+
.footer-links a:hover { color: var(--accent); }
|
| 426 |
+
|
| 427 |
+
/* βββ responsive βββ */
|
| 428 |
+
@media (max-width: 600px) {
|
| 429 |
+
.endpoint-row { grid-template-columns: 60px 1fr; }
|
| 430 |
+
.ep-desc { display: none; }
|
| 431 |
+
.nav-links { display: none; }
|
| 432 |
+
}
|
| 433 |
+
|
| 434 |
+
/* βββ fade-in βββ */
|
| 435 |
+
.fade-in {
|
| 436 |
+
opacity: 0;
|
| 437 |
+
transform: translateY(18px);
|
| 438 |
+
animation: fadeUp .55s ease forwards;
|
| 439 |
+
}
|
| 440 |
+
@keyframes fadeUp {
|
| 441 |
+
to { opacity: 1; transform: none; }
|
| 442 |
+
}
|
| 443 |
+
.fade-in:nth-child(1) { animation-delay: .05s; }
|
| 444 |
+
.fade-in:nth-child(2) { animation-delay: .12s; }
|
| 445 |
+
.fade-in:nth-child(3) { animation-delay: .19s; }
|
| 446 |
+
.fade-in:nth-child(4) { animation-delay: .26s; }
|
| 447 |
+
</style>
|
| 448 |
+
</head>
|
| 449 |
+
<body>
|
| 450 |
+
|
| 451 |
+
<!-- ββ HEADER ββββββββββββββββββββββββββββββββββββββββββββββββ -->
|
| 452 |
+
<header>
|
| 453 |
+
<div class="wrap">
|
| 454 |
+
<nav class="nav">
|
| 455 |
+
<div class="logo">
|
| 456 |
+
<span class="logo-dot"></span>
|
| 457 |
+
SC-AUDIT-ENV v1.2.0
|
| 458 |
+
</div>
|
| 459 |
+
<ul class="nav-links">
|
| 460 |
+
<li><a href="#tasks">Tasks</a></li>
|
| 461 |
+
<li><a href="#api">API</a></li>
|
| 462 |
+
<li><a href="#quickstart">Quickstart</a></li>
|
| 463 |
+
<li><a href="/docs">Swagger</a></li>
|
| 464 |
+
</ul>
|
| 465 |
+
</nav>
|
| 466 |
+
</div>
|
| 467 |
+
</header>
|
| 468 |
+
|
| 469 |
+
<!-- ββ HERO ββββββββββββββββββββββββββββββββββββββββββββββββββ -->
|
| 470 |
+
<div class="wrap">
|
| 471 |
+
<div class="hero">
|
| 472 |
+
<div class="hero-tag fade-in">OpenEnv Β· Reinforcement Learning Β· Smart Contract Security</div>
|
| 473 |
+
<h1 class="fade-in">Audit Solidity.<br><span>Train smarter agents.</span></h1>
|
| 474 |
+
<p class="hero-sub fade-in">
|
| 475 |
+
An OpenEnv-compliant RL environment for smart contract security analysis.
|
| 476 |
+
Agents explore real-world DeFi contracts from Certora audit reports β
|
| 477 |
+
detecting vulnerabilities, deriving properties, and checking invariants.
|
| 478 |
+
</p>
|
| 479 |
+
<div class="hero-badges fade-in">
|
| 480 |
+
<span class="badge badge-green">openenv</span>
|
| 481 |
+
<span class="badge badge-blue">solidity</span>
|
| 482 |
+
<span class="badge badge-blue">aave</span>
|
| 483 |
+
<span class="badge badge-blue">lido</span>
|
| 484 |
+
<span class="badge badge-yellow">certora</span>
|
| 485 |
+
<span class="badge badge-green">3 tasks</span>
|
| 486 |
+
</div>
|
| 487 |
+
<div class="cta-row fade-in">
|
| 488 |
+
<a class="btn btn-primary" href="/docs">Explore API Docs</a>
|
| 489 |
+
<a class="btn btn-outline" href="#quickstart">Quick Start β</a>
|
| 490 |
+
</div>
|
| 491 |
+
</div>
|
| 492 |
+
</div>
|
| 493 |
+
|
| 494 |
+
<!-- ββ TASKS βββββββββββββββββββββββββββββββββββββββββββββββββ -->
|
| 495 |
+
<section id="tasks">
|
| 496 |
+
<div class="wrap">
|
| 497 |
+
<div class="section-label">// three tasks</div>
|
| 498 |
+
<h2>What can agents learn here?</h2>
|
| 499 |
+
<p style="color:var(--muted); max-width:520px; margin-top:6px;">
|
| 500 |
+
Each task mirrors a real step in a professional smart contract audit.
|
| 501 |
+
Difficulty is graded easy β medium β hard.
|
| 502 |
+
</p>
|
| 503 |
+
|
| 504 |
+
<div class="task-grid">
|
| 505 |
+
<!-- Task 1 -->
|
| 506 |
+
<div class="task-card medium">
|
| 507 |
+
<div class="task-header">
|
| 508 |
+
<span class="task-num">TASK 01</span>
|
| 509 |
+
<span class="diff-badge diff-medium">medium</span>
|
| 510 |
+
</div>
|
| 511 |
+
<h3>Targeted Vulnerability Detection</h3>
|
| 512 |
+
<p class="task-desc">
|
| 513 |
+
Given a Solidity file, identify the vulnerable function and name the
|
| 514 |
+
vulnerability class in 2β3 words. Submit <code>NO</code> if no vulnerability exists.
|
| 515 |
+
</p>
|
| 516 |
+
<div class="task-actions">
|
| 517 |
+
<span class="action-chip">list_functions</span>
|
| 518 |
+
<span class="action-chip">get_function_code</span>
|
| 519 |
+
<span class="action-chip">get_state_variable</span>
|
| 520 |
+
<span class="action-chip">get_call_graph</span>
|
| 521 |
+
<span class="action-chip">submit</span>
|
| 522 |
+
</div>
|
| 523 |
+
</div>
|
| 524 |
+
|
| 525 |
+
<!-- Task 2 -->
|
| 526 |
+
<div class="task-card hard">
|
| 527 |
+
<div class="task-header">
|
| 528 |
+
<span class="task-num">TASK 02</span>
|
| 529 |
+
<span class="diff-badge diff-hard">hard</span>
|
| 530 |
+
</div>
|
| 531 |
+
<h3>Property Discovery</h3>
|
| 532 |
+
<p class="task-desc">
|
| 533 |
+
Given a single function, derive its natural-language safety property β
|
| 534 |
+
the kind a formal verification engineer would write as a CVL invariant.
|
| 535 |
+
One submission attempt per episode, scored 0β5.
|
| 536 |
+
</p>
|
| 537 |
+
<div class="task-actions">
|
| 538 |
+
<span class="action-chip">get_function_natspec</span>
|
| 539 |
+
<span class="action-chip">get_related_functions</span>
|
| 540 |
+
<span class="action-chip">get_similar_rule</span>
|
| 541 |
+
<span class="action-chip">submit_property</span>
|
| 542 |
+
</div>
|
| 543 |
+
</div>
|
| 544 |
+
|
| 545 |
+
<!-- Task 3 -->
|
| 546 |
+
<div class="task-card easy">
|
| 547 |
+
<div class="task-header">
|
| 548 |
+
<span class="task-num">TASK 03</span>
|
| 549 |
+
<span class="diff-badge diff-easy">easy</span>
|
| 550 |
+
</div>
|
| 551 |
+
<h3>Rule Checker</h3>
|
| 552 |
+
<p class="task-desc">
|
| 553 |
+
A property is given in plain English. At least one function in the
|
| 554 |
+
contract breaks it. Find the offender. Partial credit for naming a
|
| 555 |
+
direct caller of the true violating function.
|
| 556 |
+
</p>
|
| 557 |
+
<div class="task-actions">
|
| 558 |
+
<span class="action-chip">list_functions</span>
|
| 559 |
+
<span class="action-chip">get_function_code</span>
|
| 560 |
+
<span class="action-chip">get_property_specification</span>
|
| 561 |
+
<span class="action-chip">submit_function</span>
|
| 562 |
+
</div>
|
| 563 |
+
</div>
|
| 564 |
+
</div>
|
| 565 |
+
</div>
|
| 566 |
+
</section>
|
| 567 |
+
|
| 568 |
+
<!-- ββ DATA SOURCES ββββββββββββββββββββββββββββββββββββββββββ -->
|
| 569 |
+
<section>
|
| 570 |
+
<div class="wrap">
|
| 571 |
+
<div class="section-label">// dataset</div>
|
| 572 |
+
<h2>Grounded in real audits</h2>
|
| 573 |
+
<p style="color:var(--muted); max-width:520px; margin-top:6px;">
|
| 574 |
+
All contracts and vulnerabilities are sourced from public Certora audit
|
| 575 |
+
reports on production DeFi protocols.
|
| 576 |
+
</p>
|
| 577 |
+
<div class="source-grid">
|
| 578 |
+
<div class="source-card">
|
| 579 |
+
<div class="source-icon">π¦</div>
|
| 580 |
+
<div>
|
| 581 |
+
<div class="source-name">AaveVault</div>
|
| 582 |
+
<div class="source-meta">Certora Audit Β· ERC-4626 vault Β· Access control, share accounting</div>
|
| 583 |
+
</div>
|
| 584 |
+
</div>
|
| 585 |
+
<div class="source-card">
|
| 586 |
+
<div class="source-icon">π¦</div>
|
| 587 |
+
<div>
|
| 588 |
+
<div class="source-name">AaveVaultV2</div>
|
| 589 |
+
<div class="source-meta">Certora Audit Β· Upgraded vault Β· Fee logic, reentrancy surfaces</div>
|
| 590 |
+
</div>
|
| 591 |
+
</div>
|
| 592 |
+
<div class="source-card">
|
| 593 |
+
<div class="source-icon">π·</div>
|
| 594 |
+
<div>
|
| 595 |
+
<div class="source-name">Lido Finance</div>
|
| 596 |
+
<div class="source-meta">Certora Audit Β· Liquid staking Β· Rebasing token invariants</div>
|
| 597 |
+
</div>
|
| 598 |
+
</div>
|
| 599 |
+
</div>
|
| 600 |
+
</div>
|
| 601 |
+
</section>
|
| 602 |
+
|
| 603 |
+
<!-- ββ API REFERENCE βββββββββββββββββββββββββββββββββββββββββ -->
|
| 604 |
+
<section id="api">
|
| 605 |
+
<div class="wrap">
|
| 606 |
+
<div class="section-label">// http interface</div>
|
| 607 |
+
<h2>API Endpoints</h2>
|
| 608 |
+
<div class="endpoint-list">
|
| 609 |
+
<div class="endpoint-row">
|
| 610 |
+
<span class="method method-get">GET</span>
|
| 611 |
+
<span class="path">/health</span>
|
| 612 |
+
<span class="ep-desc">Liveness probe</span>
|
| 613 |
+
</div>
|
| 614 |
+
<div class="endpoint-row">
|
| 615 |
+
<span class="method method-get">GET</span>
|
| 616 |
+
<span class="path">/tasks</span>
|
| 617 |
+
<span class="ep-desc">List all tasks with difficulty and status</span>
|
| 618 |
+
</div>
|
| 619 |
+
<div class="endpoint-row">
|
| 620 |
+
<span class="method method-post">POST</span>
|
| 621 |
+
<span class="path">/reset</span>
|
| 622 |
+
<span class="ep-desc">Start a new episode — body: <code>{task_id, seed}</code></span>
|
| 623 |
+
</div>
|
| 624 |
+
<div class="endpoint-row">
|
| 625 |
+
<span class="method method-post">POST</span>
|
| 626 |
+
<span class="path">/step</span>
|
| 627 |
+
<span class="ep-desc">Take one action — body: <code>{action_type, params}</code></span>
|
| 628 |
+
</div>
|
| 629 |
+
<div class="endpoint-row">
|
| 630 |
+
<span class="method method-get">GET</span>
|
| 631 |
+
<span class="path">/state</span>
|
| 632 |
+
<span class="ep-desc">Full internal episode state (debug)</span>
|
| 633 |
+
</div>
|
| 634 |
+
<div class="endpoint-row">
|
| 635 |
+
<span class="method method-get">GET</span>
|
| 636 |
+
<span class="path">/action_space</span>
|
| 637 |
+
<span class="ep-desc">Valid actions for a task — query: <code>?task_id=</code></span>
|
| 638 |
+
</div>
|
| 639 |
+
<div class="endpoint-row">
|
| 640 |
+
<span class="method method-get">GET</span>
|
| 641 |
+
<span class="path">/observation_space</span>
|
| 642 |
+
<span class="ep-desc">Observation schema (all tasks share this)</span>
|
| 643 |
+
</div>
|
| 644 |
+
<div class="endpoint-row">
|
| 645 |
+
<span class="method method-get">GET</span>
|
| 646 |
+
<span class="path">/docs</span>
|
| 647 |
+
<span class="ep-desc">Interactive Swagger UI</span>
|
| 648 |
+
</div>
|
| 649 |
+
</div>
|
| 650 |
+
</div>
|
| 651 |
+
</section>
|
| 652 |
+
|
| 653 |
+
<!-- ββ QUICK START βββββββββββββββββββββββββββββββββββββββββββ -->
|
| 654 |
+
<section id="quickstart">
|
| 655 |
+
<div class="wrap">
|
| 656 |
+
<div class="section-label">// quick start</div>
|
| 657 |
+
<h2>Run an episode in 3 calls</h2>
|
| 658 |
+
<div class="code-wrap">
|
| 659 |
+
<div class="code-header">
|
| 660 |
+
<span class="code-title">bash</span>
|
| 661 |
+
<button class="copy-btn" onclick="copyCode(this)">copy</button>
|
| 662 |
+
</div>
|
| 663 |
+
<pre id="snippet"><span class="cm"># 1. Reset β start Task 1 with a fixed seed</span>
|
| 664 |
+
<span class="kw">curl</span> -X POST $SPACE_URL/reset \
|
| 665 |
+
-H <span class="str">"Content-Type: application/json"</span> \
|
| 666 |
+
-d <span class="str">'{"task_id": "task1_vuln_detection", "seed": 42}'</span>
|
| 667 |
+
|
| 668 |
+
<span class="cm"># 2. Explore β list all functions in the contract</span>
|
| 669 |
+
<span class="kw">curl</span> -X POST $SPACE_URL/step \
|
| 670 |
+
-H <span class="str">"Content-Type: application/json"</span> \
|
| 671 |
+
-d <span class="str">'{"action_type": "list_functions", "params": {}}'</span>
|
| 672 |
+
|
| 673 |
+
<span class="cm"># 3. Submit β name the vulnerable function</span>
|
| 674 |
+
<span class="kw">curl</span> -X POST $SPACE_URL/step \
|
| 675 |
+
-H <span class="str">"Content-Type: application/json"</span> \
|
| 676 |
+
-d <span class="str">'{"action_type": "submit", "params": {"function_name": "withdraw", "vulnerability_type": "reentrancy"}}'</span></pre>
|
| 677 |
+
</div>
|
| 678 |
+
</div>
|
| 679 |
+
</section>
|
| 680 |
+
|
| 681 |
+
<!-- ββ FOOTER ββββββββββββββββββββββββββββββββββββββββββββββββ -->
|
| 682 |
+
<footer>
|
| 683 |
+
<div class="wrap">
|
| 684 |
+
<div class="footer-inner">
|
| 685 |
+
<div class="footer-left">
|
| 686 |
+
SC-AUDIT-ENV Β· MIT License Β· Data from Certora public audits
|
| 687 |
+
</div>
|
| 688 |
+
<div class="footer-links">
|
| 689 |
+
<a href="/docs">Swagger</a>
|
| 690 |
+
<a href="/tasks">Tasks JSON</a>
|
| 691 |
+
<a href="/health">Health</a>
|
| 692 |
+
</div>
|
| 693 |
+
</div>
|
| 694 |
+
</div>
|
| 695 |
+
</footer>
|
| 696 |
+
|
| 697 |
+
<script>
|
| 698 |
+
function copyCode(btn) {
|
| 699 |
+
const pre = document.getElementById('snippet');
|
| 700 |
+
const text = pre.innerText;
|
| 701 |
+
navigator.clipboard.writeText(text).then(() => {
|
| 702 |
+
btn.textContent = 'copied!';
|
| 703 |
+
setTimeout(() => btn.textContent = 'copy', 2000);
|
| 704 |
+
});
|
| 705 |
+
}
|
| 706 |
+
</script>
|
| 707 |
+
</body>
|
| 708 |
+
</html>"""
|