KChad commited on
Commit
5f8c2e8
·
1 Parent(s): 4888b97

merges from the other branch

Browse files
Files changed (4) hide show
  1. .github/workflows/ci.yml +45 -0
  2. .github/workflows/sync.yml +12 -3
  3. Dockerfile +3 -6
  4. README.md +44 -15
.github/workflows/ci.yml ADDED
@@ -0,0 +1,45 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ name: CI
2
+
3
+ on:
4
+ push:
5
+ branches: [main, benchmark-core]
6
+ pull_request:
7
+ branches: [main]
8
+ workflow_dispatch:
9
+
10
+ jobs:
11
+ test:
12
+ runs-on: ubuntu-latest
13
+ timeout-minutes: 20
14
+ steps:
15
+ - name: Check out repository
16
+ uses: actions/checkout@v4
17
+
18
+ - name: Set up Python
19
+ uses: actions/setup-python@v5
20
+ with:
21
+ python-version: "3.11"
22
+
23
+ - name: Install package and test dependencies
24
+ run: |
25
+ python -m pip install --upgrade pip
26
+ python -m pip install -e .[dev]
27
+
28
+ - name: Run pre-submission check
29
+ run: |
30
+ python pre_submission_check.py
31
+
32
+ - name: Run benchmark test suite
33
+ run: |
34
+ python -m unittest discover -s tests
35
+
36
+ docker-build:
37
+ runs-on: ubuntu-latest
38
+ timeout-minutes: 30
39
+ needs: test
40
+ steps:
41
+ - name: Check out repository
42
+ uses: actions/checkout@v4
43
+
44
+ - name: Build container image
45
+ run: docker build -t idpi-exfil-env .
.github/workflows/sync.yml CHANGED
@@ -7,14 +7,23 @@ on:
7
  jobs:
8
  sync-to-hub:
9
  runs-on: ubuntu-latest
 
10
  steps:
11
- - uses: actions/checkout@v3
 
12
  with:
13
  fetch-depth: 0
14
  lfs: true
 
 
 
 
15
  - name: Push to hub
16
  env:
17
  HF_TOKEN: ${{ secrets.HF_TOKEN }}
 
18
  run: |
19
- # Change 'KumarChad' and 'Prompt_Injection_env' if your names are different!
20
- git push --force https://KumarChad:${{ secrets.HF_TOKEN }}@huggingface.co/spaces/KumarChad/Prompt_Injection_env main
 
 
 
7
  jobs:
8
  sync-to-hub:
9
  runs-on: ubuntu-latest
10
+ if: ${{ secrets.HF_TOKEN != '' }}
11
  steps:
12
+ - name: Check out repository
13
+ uses: actions/checkout@v4
14
  with:
15
  fetch-depth: 0
16
  lfs: true
17
+ - name: Configure git identity
18
+ run: |
19
+ git config user.name "github-actions[bot]"
20
+ git config user.email "41898282+github-actions[bot]@users.noreply.github.com"
21
  - name: Push to hub
22
  env:
23
  HF_TOKEN: ${{ secrets.HF_TOKEN }}
24
+ HF_SPACE_REPO: ${{ vars.HF_SPACE_REPO }}
25
  run: |
26
+ if [ -z "$HF_SPACE_REPO" ]; then
27
+ HF_SPACE_REPO="KumarChad/Prompt_Injection_env"
28
+ fi
29
+ git push --force "https://hf_user:${HF_TOKEN}@huggingface.co/spaces/${HF_SPACE_REPO}" HEAD:main
Dockerfile CHANGED
@@ -58,10 +58,6 @@ ENV PYTHONPATH="/app/env:$PYTHONPATH"
58
  ENV PYTHONUNBUFFERED=1
59
  ENV ENABLE_WEB_INTERFACE=true
60
 
61
- # Never bake API key into image — must be passed at runtime
62
- # docker run --env OPENAI_API_KEY=sk-... idpi-exfil-env
63
- ENV OPENAI_API_KEY=""
64
-
65
  # Expose port
66
  EXPOSE 8000
67
 
@@ -70,5 +66,6 @@ HEALTHCHECK --interval=30s --timeout=3s --start-period=5s --retries=3 \
70
  CMD python -c "import urllib.request; urllib.request.urlopen('http://localhost:8000/health')" || exit 1
71
 
72
  # Default: start server
73
- # Override with: docker run --env OPENAI_API_KEY=sk-... idpi-exfil-env python run_baseline.py
74
- CMD ["sh", "-c", "cd /app/env && uvicorn server.app:app --host 0.0.0.0 --port 8000"]
 
 
58
  ENV PYTHONUNBUFFERED=1
59
  ENV ENABLE_WEB_INTERFACE=true
60
 
 
 
 
 
61
  # Expose port
62
  EXPOSE 8000
63
 
 
66
  CMD python -c "import urllib.request; urllib.request.urlopen('http://localhost:8000/health')" || exit 1
67
 
68
  # Default: start server
69
+ # Runtime inference credentials should be passed when needed via
70
+ # API_BASE_URL, MODEL_NAME, and HF_TOKEN.
71
+ CMD ["sh", "-c", "cd /app/env && uvicorn server.app:app --host 0.0.0.0 --port 8000"]
README.md CHANGED
@@ -50,17 +50,12 @@ The API starts on `http://localhost:7860`.
50
 
51
  ### Run the evaluation runner
52
 
53
- Create a `.env` in the project root with your Gemini API key:
54
-
55
- ```bash
56
- GEMINI_API_KEY=your_gemini_api_key_here
57
- MODEL_NAME=gemini-2.5-flash
58
- ```
59
-
60
- By default, the runner uses Google's OpenAI-compatible Gemini endpoint:
61
 
62
  ```bash
63
  API_BASE_URL=https://generativelanguage.googleapis.com/v1beta/openai/
 
 
64
  ```
65
 
66
  Then run:
@@ -77,6 +72,7 @@ python baseline/run_baseline.py
77
  ```
78
 
79
  All three commands use the same unified runner and write results to `baseline/baseline_results.json`.
 
80
 
81
  ## Using the HTTP API
82
 
@@ -118,6 +114,15 @@ state = requests.get("http://localhost:7860/state").json()
118
  episode_result = requests.get("http://localhost:7860/result").json()
119
  ```
120
 
 
 
 
 
 
 
 
 
 
121
  ## Environment Overview
122
 
123
  ### Task model
@@ -163,11 +168,16 @@ Typical safe behavior includes:
163
 
164
  ### Endpoints
165
 
 
 
166
  - `GET /health`
 
 
167
  - `POST /reset`
168
  - `POST /step`
169
  - `GET /state`
170
  - `GET /result`
 
171
 
172
  ## Evaluation
173
 
@@ -195,10 +205,8 @@ Results are written to [baseline/baseline_results.json](/C:/Users/Admin/Desktop/
195
  These values can be set in `.env`:
196
 
197
  ```bash
198
- EPISODES_PER_SCENARIO=1
199
- MAX_TOKENS=200
200
- TEMPERATURE=0.0
201
- REQUEST_DELAY_SECONDS=3.0
202
  ```
203
 
204
  ## Development
@@ -238,12 +246,33 @@ Prompt-injection-env/
238
  - [env/environment.py](/C:/Users/Admin/Desktop/My%20projects/Prompt-injection-env/env/environment.py)
239
  Main episode lifecycle, action handling, reward plumbing, and result assembly.
240
  - [server/app.py](/C:/Users/Admin/Desktop/My%20projects/Prompt-injection-env/server/app.py)
241
- FastAPI wrapper exposing the environment over HTTP.
242
  - [inference.py](/C:/Users/Admin/Desktop/My%20projects/Prompt-injection-env/inference.py)
243
- Canonical evaluation runner used by both inference and baseline wrappers.
244
  - [openenv.yaml](/C:/Users/Admin/Desktop/My%20projects/Prompt-injection-env/openenv.yaml)
245
  Environment metadata for OpenEnv packaging and deployment.
246
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
247
  ### Local Python usage
248
 
249
  You can exercise the environment directly without starting the server:
@@ -263,4 +292,4 @@ print(reward)
263
 
264
  - The environment terminates early on unsafe exfiltration events.
265
  - Hard tasks require a clean internal summary that excludes attacker-controlled content.
266
- - The unified runner now uses Gemini only and requires `GEMINI_API_KEY`.
 
50
 
51
  ### Run the evaluation runner
52
 
53
+ Create a `.env` in the project root with the submission-compatible inference variables:
 
 
 
 
 
 
 
54
 
55
  ```bash
56
  API_BASE_URL=https://generativelanguage.googleapis.com/v1beta/openai/
57
+ MODEL_NAME=gemini-2.5-flash
58
+ HF_TOKEN=your_gemini_api_key_here
59
  ```
60
 
61
  Then run:
 
72
  ```
73
 
74
  All three commands use the same unified runner and write results to `baseline/baseline_results.json`.
75
+ For submission mode, `inference.py` validates `API_BASE_URL`, `MODEL_NAME`, and `HF_TOKEN`, then delegates to the benchmark runner using the OpenAI client path.
76
 
77
  ## Using the HTTP API
78
 
 
114
  episode_result = requests.get("http://localhost:7860/result").json()
115
  ```
116
 
117
+ Additional discovery endpoints are also available:
118
+
119
+ ```python
120
+ import requests
121
+
122
+ metadata = requests.get("http://localhost:7860/metadata").json()
123
+ schema = requests.get("http://localhost:7860/schema").json()
124
+ ```
125
+
126
  ## Environment Overview
127
 
128
  ### Task model
 
168
 
169
  ### Endpoints
170
 
171
+ - `GET /`
172
+ - `GET /web`
173
  - `GET /health`
174
+ - `GET /metadata`
175
+ - `GET /schema`
176
  - `POST /reset`
177
  - `POST /step`
178
  - `GET /state`
179
  - `GET /result`
180
+ - `POST /mcp`
181
 
182
  ## Evaluation
183
 
 
205
  These values can be set in `.env`:
206
 
207
  ```bash
208
+ EPISODES_PER_DIFFICULTY=1
209
+ BASELINE_RESULTS_PATH=baseline/submission_results.json
 
 
210
  ```
211
 
212
  ## Development
 
246
  - [env/environment.py](/C:/Users/Admin/Desktop/My%20projects/Prompt-injection-env/env/environment.py)
247
  Main episode lifecycle, action handling, reward plumbing, and result assembly.
248
  - [server/app.py](/C:/Users/Admin/Desktop/My%20projects/Prompt-injection-env/server/app.py)
249
+ FastAPI wrapper exposing the environment over HTTP, including health, metadata, schema, and benchmark endpoints.
250
  - [inference.py](/C:/Users/Admin/Desktop/My%20projects/Prompt-injection-env/inference.py)
251
+ Submission-facing inference entrypoint that validates the required env vars and delegates to the benchmark runner.
252
  - [openenv.yaml](/C:/Users/Admin/Desktop/My%20projects/Prompt-injection-env/openenv.yaml)
253
  Environment metadata for OpenEnv packaging and deployment.
254
 
255
+ ### Pre-submission checks
256
+
257
+ Run the local validator before pushing a submission:
258
+
259
+ ```bash
260
+ python pre_submission_check.py
261
+ ```
262
+
263
+ If you only want structural checks without starting a local server:
264
+
265
+ ```bash
266
+ python pre_submission_check.py --skip-runtime
267
+ ```
268
+
269
+ ### CI and sync
270
+
271
+ The repo now includes GitHub workflows for:
272
+
273
+ - continuous integration in [.github/workflows/ci.yml](/C:/Users/Admin/Desktop/My%20projects/Prompt-injection-env/.github/workflows/ci.yml)
274
+ - syncing `main` to Hugging Face Spaces in [.github/workflows/sync.yml](/C:/Users/Admin/Desktop/My%20projects/Prompt-injection-env/.github/workflows/sync.yml)
275
+
276
  ### Local Python usage
277
 
278
  You can exercise the environment directly without starting the server:
 
292
 
293
  - The environment terminates early on unsafe exfiltration events.
294
  - Hard tasks require a clean internal summary that excludes attacker-controlled content.
295
+ - Gemini works through the submission-compatible OpenAI client path by setting `API_BASE_URL`, `MODEL_NAME`, and `HF_TOKEN`.