Refactor: Restore intrinsic detector to fallback logic, rewrite README.md, and polish all codebase comments for final submission

.env.example

@@ -0,0 +1,10 @@
+# LLM Configuration
+USE_LLM=1
+API_BASE_URL=https://router.huggingface.co/v1
+MODEL_NAME=Qwen/Qwen2.5-72B-Instruct
+# Get your token from https://huggingface.co/settings/tokens
+HF_TOKEN=your_huggingface_token_here
+
+# Environment Configuration
+# Use http://0.0.0.0:7860 or http://localhost:7860 locally
+ENV_URL=http://localhost:7860

README.md

@@ -18,137 +18,145 @@ pinned: false
 
 ## Overview
 
-Phase-Locked Loops (PLLs) are critical components in grid-connected power converters that synchronize the inverter's output with the utility grid. The Synchronous Reference Frame PLL (SRF-PLL) estimates grid frequency and phase angle by tracking the q-axis voltage component — making it a high-value target for **False Data Injection (FDI)** cyberattacks.
+Phase-Locked Loops (PLLs) are critical components in grid-connected power converters, responsible for synchronizing the inverter's output with the utility grid. The Synchronous Reference Frame PLL (SRF-PLL) estimates grid frequency and phase angle by tracking the q-axis voltage component. Because of its critical role and reliance on sensor data, the SRF-PLL is a high-value target for **False Data Injection (FDI)** cyberattacks.
 
-This OpenEnv environment simulates an SRF-PLL under various FDI attack scenarios. An AI agent monitors time-windowed sensor observations (voltages, frequency deviations) and must detect, classify, and respond to attacks in real time before they cause loss of grid synchronization.
+This OpenEnv environment simulates an SRF-PLL subjected to varied FDI attack scenarios. An AI agent acts as a cyber-guard: it monitors arriving time-windowed sensor observations—such as voltages and frequency deviations—and must accurately detect, classify, and mitigate attacks in real-time before grid synchronization is lost.
 
 ## Architecture
 
-```
+The environment relies on a discrete-time SRF-PLL simulation running at a 1 ms step size. A streamlined view of the signal flow is below:
+
+```text
 Grid Voltage (50Hz)
      │
      ▼
-[FDI Attack Injection] ◄── Attacker injects false signal on va
+[FDI Attack Injection]  ◄── Attacker injects a malicious signal on phase `va`
      │
      ▼
 Clarke Transform (αβ)
      │
      ▼
-Park Transform (dq) ◄── uses estimated angle θ̂
+Park Transform (dq)     ◄── Uses the currently estimated angle θ̂
      │
      ▼
-PI Controller ──► ω̂, θ̂ updated
+PI Controller           ──► ω̂, θ̂ are updated continuously
      │
      ▼
-Agent observes: vq_window, omega_deviation_window, raw_voltages
+Agent Observation       ──► Agent receives: `vq_window`, `omega_deviation_window`, `raw_voltages`
      │
      ▼
-Agent outputs: attack_detected, attack_type, confidence
+Agent Action            ──► Agent outputs: `attack_detected`, `attack_type`, `confidence`
 ```
 
-## Inference & Detection Strategy
+## Inference Flow & Detector Walkthrough
 
-The environment natively features an **Adaptive Physics-Informed Detector** (`src/detector.py`) that calibrates anomaly residuals (R1, R3, R4, R5) during the PLL warm-up phase to identify stealthy voltage and frequency deviations.
+To balance speed and accuracy across thousands of steps, the standard inference client (`inference.py`) deploys a **Smart Blending Strategy**:
 
-The default inference client (`inference.py`) deploys a **Smart Blending Agent** strategy:
-1. It relies primarily on the environment's `AdaptiveDetector` output passed via `info["detector"]`.
-2. As a **safety net**, if the detector's classification confidence drops below 50% (`< 0.5`) on ambiguous anomalies, the client dynamically falls back to an independent, cumulative **Rule-Based Heuristic Agent**.
-3. Optionally, an LLM agent (e.g., `Qwen/Qwen2.5-72B-Instruct`) can be enabled natively via the `USE_LLM=1` environment variable.
+1. **Environment Simulation (`env.py`)**: 
+   Every step, the PLL updates its internal math based on potential attack injections. It yields a rich observation window of the last 20 frames for variables like $V_q$ and $\omega_{dev}$.
+2. **Adaptive Physics-Informed Detector (`src/detector.py`)**: 
+   Before returning the observation to the client, the environment evaluates the data using an intrinsic physics-based detector. This detector calibrates anomaly residuals during the first 20 "healthy" warm-up steps. It tracks variances and symmetry to identify stealthy voltage anomalies, providing a baseline `confidence` score.
+3. **Smart Blending Client (`inference.py`)**: 
+   The client receives the observation and the detector's baseline prediction. 
+   * If the intrinsic detector has high confidence (> 50%), the client adopts its recommendation.
+   * If the anomaly is ambiguous (confidence < 50%), the client queries its own **Rule-Based Heuristic Agent**, which monitors historical $V_q$ growth, monotonicity, and zero-crossing density.
+   * *Optional*: If `USE_LLM=1` is set, the client uses an LLM (e.g., `Qwen2.5-72B`) for advanced reasoning. A resilient "circuit breaker" automatically transitions to the heuristic model if network or authentication failures occur.
 
 ## Tasks
 
-| Task | ID | Difficulty | Attack Type | Objective | Score |
+The environment supports three sequentially evaluated difficulty levels:
+
+| Task | ID | Difficulty | Attack Type | Objective | Score Metric |
 |------|----|-----------|-------------|-----------|-------|
-| Sinusoidal FDI Detection | 0 | Easy | Sinusoidal injection | Detect within 100 steps | Time-based decay |
-| Multi-Attack Classification | 1 | Medium | Sinusoidal/Ramp/Pulse | Classify attack type | Accuracy + speed |
-| Stealthy Attack Detection | 2 | Hard | Low-amplitude phase drift | Detect before lock loss | Prevention score |
+| Sinusoidal FDI | 0 | Easy | Sinusoidal Injection | Detect attack within 100 steps of initiation. | Time-decaying detection reward. |
+| Multi-Attack Class. | 1 | Medium | Sinusoidal, Ramp, Pulse | Safely and correctly classify the specific attack type. | Accuracy and speed aggregate. |
+| Stealthy Detection | 2 | Hard | Low-amplitude phase drift | Detect slow deviations before the PLL loses lock (θ_error > 5°). | Preventative lock-loss metric. |
 
 ## Observation Space
 
-Each step provides a JSON observation with the following fields:
+At each step, the environment provides a JSON observation containing:
 
 | Field | Shape | Description |
 |-------|-------|-------------|
-| `vq_window` | `[20]` | q-axis voltage error signal (pu) |
-| `vd_window` | `[20]` | d-axis voltage (pu) |
-| `omega_window` | `[20]` | Normalized frequency deviation from nominal |
-| `omega_deviation_window` | `[20]` | Frequency deviation from nominal (rad/s) |
-| `raw_voltages` | `[3]` | Raw three-phase voltages `[va, vb, vc]` (pu) |
-| `step` | scalar | Current simulation step |
-| `task_id` | scalar | Task identifier (0, 1, or 2) |
+| `vq_window` | `[20]` | q-axis voltage error signal (pu). |
+| `vd_window` | `[20]` | d-axis voltage (pu). |
+| `omega_window` | `[20]` | Normalized frequency deviation from nominal. |
+| `omega_deviation_window` | `[20]` | Frequency deviation from nominal (rad/s). |
+| `raw_voltages` | `[3]` | Raw three-phase voltages `[va, vb, vc]` (pu). |
+| `step` | `scalar` | Current simulation time step. |
+| `task_id` | `scalar` | Current task identifier (0, 1, or 2). |
 
-**Total observation dimension**: 83 (20+20+20+20+3)
+**Total observation dimension**: 83 ($20 \times 4 + 3$)
 
 ## Action Space
 
-Agents return a JSON action each step:
+Agents must return a structured JSON response predicting the system state:
 
 | Field | Type | Range | Description |
 |-------|------|-------|-------------|
-| `attack_detected` | `bool` | — | Whether an attack is detected |
-| `attack_type` | `int` | 0–4 | 0=none, 1=sinusoidal, 2=ramp, 3=pulse, 4=stealthy |
-| `confidence` | `float` | 0.0–1.0 | Agent's confidence in its classification |
-| `protective_action` | `int` | 0–3 | 0=none, 1=alert, 2=reduce power, 3=disconnect |
+| `attack_detected` | `bool` | — | True if malicious injection is suspected. |
+| `attack_type` | `int` | 0–4 | 0=None, 1=Sinusoidal, 2=Ramp, 3=Pulse, 4=Stealthy. |
+| `confidence` | `float` | 0.0–1.0 | Absolute predictive certainty. |
+| `protective_action` | `int` | 0–3 | Suggested mitigation: 0=None, 1=Alert, 2=Reduce Power, 3=Disconnect. |
 
-## API Endpoints
+## Setup & API Usage
 
-### Reset Environment
-```bash
-curl -X POST http://localhost:7860/reset \
-  -H "Content-Type: application/json" \
-  -d '{"task_id": 0, "seed": 42}'
-```
+The system acts as a standard REST API server over port `7860`.
 
-### Step
-```bash
-curl -X POST http://localhost:7860/step \
-  -H "Content-Type: application/json" \
-  -d '{"attack_detected": false, "attack_type": 0, "confidence": 0.5, "protective_action": 0}'
-```
+### Local Setup
 
-### Get State
+**Via Python (Recommended)**:
 ```bash
-curl http://localhost:7860/state
+pip install -r requirements.txt
+uvicorn src.api:app --host 0.0.0.0 --port 7860
 ```
 
-### Health Check
+**Via Docker**:
 ```bash
-curl http://localhost:7860/health
+docker build -t pll-cyberattack-env .
+docker run -p 7860:7860 pll-cyberattack-env
 ```
 
-## Quick Start
+### Environment Variables
 
-### With Docker
+Configure execution behavior locally via a `.env` file (see `.env.example`).
 
-```bash
-docker build -t pll-cyberattack-env .
-docker run -p 7860:7860 pll-cyberattack-env
-```
+| Variable | Default | Description |
+|----------|---------|-------------|
+| `API_BASE_URL` | `https://router.huggingface.co/v1` | Custom endpoint for Language Models. |
+| `MODEL_NAME` | `Qwen/Qwen2.5-72B-Instruct` | Internal Model identifier. |
+| `HF_TOKEN` | — | HuggingFace or valid proxy API key. |
+| `USE_LLM` | `1` | Set to `1` to run the active LLM agent, `0` for pure heuristics. |
 
-### Without Docker
+### REST Endpoints
 
-```bash
-pip install -r requirements.txt
-uvicorn src.api:app --host 0.0.0.0 --port 7860
-```
+1. **POST `/reset`**
+   Initializes the environment for a specific task.
+   ```bash
+   curl -X POST http://localhost:7860/reset \
+     -H "Content-Type: application/json" \
+     -d '{"task_id": 0, "seed": 42}'
+   ```
 
-## Environment Variables
+2. **POST `/step`**
+   Submit an action based on recent observations and advance by one tick.
+   ```bash
+   curl -X POST http://localhost:7860/step \
+     -H "Content-Type: application/json" \
+     -d '{"attack_detected": false, "attack_type": 0, "confidence": 0.5, "protective_action": 0}'
+   ```
 
-| Variable | Required | Default | Description |
-|----------|----------|---------|-------------|
-| `API_BASE_URL` | No | `https://router.huggingface.co/v1` | LLM API endpoint |
-| `MODEL_NAME` | No | `Qwen/Qwen2.5-72B-Instruct` | Model identifier |
-| `HF_TOKEN` | Yes | — | HuggingFace API token |
+3. **GET `/health`**
+   Returns operational status and step numbers.
 
 ## Baseline Performance
 
-The default hybrid strategy (Adaptive Detector + Heuristic Fallback) achieves the following baseline scores evaluated locally over 500-step episodes:
+The default hybrid strategy outlined in `inference.py` consistently yields the following evaluation bounds across a full 500-step envelope:
 
 * **Task 0 (Sinusoidal FDI):** 1.0000 
-* **Task 1 (Multi-Attack Classification):** 0.8720
-* **Task 2 (Stealthy Drift):** 0.1639
-* **Average Score:** `0.6786`
+* **Task 1 (Multi-Attack Classification):** ~0.8720
+* **Task 2 (Stealthy Drift):** ~0.1639
+* **Aggregate System Average:** `0.6786`
 
-## Live Demo
-
-🚀 **HuggingFace Space**: [https://huggingface.co/spaces/krishuggingface/CyberAttack-PLL](https://huggingface.co/spaces/krishuggingface/CyberAttack-PLL)
+---
+🚀 **Live Environment Hosted on HuggingFace Spaces**: [krishuggingface/CyberAttack-PLL](https://huggingface.co/spaces/krishuggingface/CyberAttack-PLL)

inference.py

@@ -21,6 +21,12 @@ import requests
 from typing import List, Optional
 from openai import OpenAI
 
+try:
+    from dotenv import load_dotenv
+    load_dotenv()
+except ImportError:
+    pass
+
 # ── Config — always read from environment, never hardcode ─────────────────────
 # The judging sandbox injects API_BASE_URL and API_KEY via their LiteLLM proxy.
 # All LLM calls MUST go through these values or the submission will be rejected.
@@ -106,7 +112,35 @@ def log_end(success: bool, steps: int, score: float, rewards: List[float]) -> No
         flush=True,
     )
 
-# ── Heuristic agent (FALLBACK ONLY — used when LLM call fails) ────────────────
+# ── Detector Agent & Smart Blending ───────────────────────────────────────────
+
+def detector_agent(prev_info: dict) -> Optional[dict]:
+    """Reads the environment's intrinsic physics-based detector output."""
+    det = prev_info.get("detector", {})
+    if not det or "attack_detected" not in det:
+        return None
+        
+    return {
+        "attack_detected": det.get("attack_detected", False),
+        "attack_type": det.get("attack_type", 0),
+        "confidence": det.get("confidence", 0.5),
+        "protective_action": det.get("protective_action", 0),
+    }
+
+def smart_blend_agent(obs: dict, prev_info: dict) -> dict:
+    """Uses detector if confident, else falls back to robust heuristic."""
+    heur_action = heuristic_agent(obs)
+    det_action = detector_agent(prev_info)
+    
+    if not det_action:
+        return heur_action
+    if det_action["confidence"] < 0.5:
+        return heur_action
+    
+    return det_action
+
+
+# ── Rule-Based Heuristic Agent ────────────────────────────────────────────────
 
 class HeuristicState:
     """Tracks running state for the heuristic agent across steps."""
@@ -301,15 +335,15 @@ def format_observation(obs: dict) -> str:
 _llm_disabled = False  # circuit breaker — flips True after first LLM failure
 
 
-def llm_agent(obs: dict) -> dict:
+def llm_agent(obs: dict, prev_info: dict) -> dict:
     """Primary agent — calls the LLM through the injected proxy.
-    Falls back to heuristic only if the API call itself raises an exception.
+    Falls back to smart blending if the API call itself raises an exception.
     Uses a circuit breaker: after the first failure, all future calls skip the
-    network request and go straight to heuristic (restoring ~10s runtime).
+    network request and go straight to blending (restoring ~10s runtime).
     """
     global _llm_disabled
     if _llm_disabled:
-        return heuristic_agent(obs)
+        return smart_blend_agent(obs, prev_info)
 
     try:
         completion = client.chat.completions.create(
@@ -324,9 +358,9 @@ def llm_agent(obs: dict) -> dict:
         )
         return parse_llm_response(completion.choices[0].message.content or "")
     except Exception as e:
-        print(f"[DEBUG] LLM error ({type(e).__name__}: {e}), disabling LLM for remaining steps", file=sys.stderr, flush=True)
+        print(f"[WARN] LLM error ({type(e).__name__}: {e}), disabling LLM for remaining steps", file=sys.stderr, flush=True)
         _llm_disabled = True
-        return heuristic_agent(obs)
+        return smart_blend_agent(obs, prev_info)
 
 # ── Episode runner ────────────────────────────────────────────────────────────
 
@@ -362,9 +396,9 @@ def run_episode(task_id: int) -> float:
             # This caps LLM calls at ~150 total across 3 tasks, keeping runtime
             # well under the 20-min judging limit even with 3s/call latency.
             if step_count % 10 == 0:
-                action = llm_agent(obs)
+                action = llm_agent(obs, info)
             else:
-                action = heuristic_agent(obs)
+                action = smart_blend_agent(obs, info)
 
             step_resp = _session.post(
                 f"{ENV_URL}/step",

requirements.txt

@@ -5,3 +5,4 @@ numpy==1.26.4
 openai>=1.0.0
 requests>=2.31.0
 openenv-core>=0.2.0
+python-dotenv>=1.0.0

src/attacks.py

@@ -109,7 +109,7 @@ class AttackGenerator:
         return 0.0
 
     def is_active(self, current_step: int) -> bool:
-        """Checking if the attack is currently active at this step."""
+        """Check whether the attack is currently active at this specific step."""
         if current_step < self.attack_start_step:
             return False
 
@@ -123,7 +123,7 @@ class AttackGenerator:
 
 
 def get_attack_type_id(attack_type_str: str) -> int:
-    """Mapping attack type string to integer ID."""
+    """Map an attack type string to its corresponding integer ID."""
     mapping = {
         "none": 0,
         "sinusoidal": 1,

src/env.py

@@ -72,7 +72,7 @@ class PLLAttackEnv:
         self.vq_window: deque = deque(maxlen=WINDOW_SIZE)
         self.vd_window: deque = deque(maxlen=WINDOW_SIZE)
         self.omega_window: deque = deque(maxlen=WINDOW_SIZE)
-        self.omega_deviation_window: deque = deque(maxlen=WINDOW_SIZE)  # Fix 8
+        self.omega_deviation_window: deque = deque(maxlen=WINDOW_SIZE)
 
         # Detector
         self.detector = AdaptiveDetector()
@@ -116,7 +116,7 @@ class PLLAttackEnv:
         # Reset history
         self.history = []
 
-        # Reset observation windows (Fix 6: no theta_err_window)
+        # Reset observation windows
         self.vq_window = deque(maxlen=WINDOW_SIZE)
         self.vd_window = deque(maxlen=WINDOW_SIZE)
         self.omega_window = deque(maxlen=WINDOW_SIZE)
@@ -205,12 +205,11 @@ class PLLAttackEnv:
         # --- Advance step counter ----------------------------------------
         self.step_count += 1
 
-        # --- Episode termination -----------------------------------------
-        # Fix 4: Task 2 terminates early on lock-loss, not just at MAX_STEPS
+        # Terminate Task 2 early upon losing lock to save computational steps
         if self.step_count >= MAX_STEPS:
             self.done = True
         elif self.task_id == 2 and self.lock_lost:
-            self.done = True  # early termination — no point continuing
+            self.done = True
 
         # --- Physics-informed detector (evaluation/debug only) ------------
         detector_output = self.detector.detect(self._get_observation())
@@ -350,7 +349,7 @@ class PLLAttackEnv:
             vq_window=list(self.vq_window),
             vd_window=list(self.vd_window),
             omega_window=list(self.omega_window),
-            omega_deviation_window=list(self.omega_deviation_window),  # Fix 5
+            omega_deviation_window=list(self.omega_deviation_window),
             raw_voltages=[self.pll.va_m, self.pll.vb_m, self.pll.vc_m],
             task_id=self.task_id,
             step=self.step_count,

src/graders.py

@@ -112,14 +112,14 @@ def grade_task_hard(
         attack_active = entry["attack_active"]
         attack_detected = entry["attack_detected"]
 
-        # Only counting false alarms before the attack starts
+        # Only count false alarms before the attack starts
         if attack_detected and not attack_active and step < attack_start_step:
             false_alarm_count += 1
 
         if attack_detected and attack_active and first_detection_step is None:
             first_detection_step = step
 
-    # Computing base score
+    # Compute base score
     if first_detection_step is None:
         score = 0.0
     elif loss_of_lock_step is not None and first_detection_step < loss_of_lock_step:
@@ -130,7 +130,7 @@ def grade_task_hard(
         # No loss of lock occurred but attack was detected
         score = 0.3
 
-    # Applying false alarm penalty
+    # Apply false alarm penalty
     penalty = 0.2 * false_alarm_count
     score = max(0.01, score - penalty)