Update README with v2 adaptive features

README.md

@@ -1,123 +1,108 @@
-# 🛸 Orbit Wars - Kaggle Competition Agent
+# 🛸 Orbit Wars - Kaggle Competition Agent (v2 Adaptive)
 
-**Competition:** [Orbit Wars](https://www.kaggle.com/competitions/orbit-wars) ($50,000 prize pool)
-**Deadline:** June 23, 2026
+**Competition:** [Orbit Wars](https://www.kaggle.com/competitions/orbit-wars) ($50,000 prize pool)  
+**Deadline:** June 23, 2026  
+**Current ELO:** ~1100 and climbing
 
 ## Overview
 
-This is a highly competitive rule-based agent for the Orbit Wars Kaggle competition — a real-time strategy game where 2 or 4 AI agents compete to conquer planets orbiting a central sun in continuous 2D space.
+This is a highly competitive **adaptive** agent for the Orbit Wars Kaggle competition — a real-time strategy game where 2 or 4 AI agents compete to conquer planets orbiting a central sun in continuous 2D space.
 
-### Game Rules Summary
-- **Board:** 100×100 continuous 2D space with a sun at center (radius 10)
-- **Planets:** Produce 1-5 ships/turn; inner ones orbit the sun, outer ones are static
-- **Fleets:** Speed scales logarithmically with size; crossing the sun destroys them
-- **Comets:** Spawn at steps 50/150/250/350/450 as temporary extra planets
-- **Win condition:** Most total ships (on planets + in flight) at step 500, or last player standing
-
-### Agent Actions
-Each turn, the agent returns: `[[from_planet_id, angle, num_ships], ...]`
+**v2 adds real-time in-match opponent profiling and adaptive parameter tuning — the agent learns opponent playstyle during each game and adjusts its strategy accordingly.**
 
 ## Architecture
 
-This agent is a **composite super-agent** that combines the best strategies from the top-rated public agents, enhanced with novel features:
+### Core: Composite Rule-Based Engine
+Combined from the **5 top public agents** on the leaderboard:
+
+| Source | LB Rating | Key Feature |
+|--------|-----------|-------------|
+| tamrazov-starwars (base) | LB 1224 | Gang-up attacks, weakest enemy targeting, elimination missions |
+| ykhnkf | LB #1 | Hostile reinforcement prediction |
+| pascal v14 | High-rated | 4-source coordinated swarm attacks |
+| pilkwang | LB ~1000 | Structured decision architecture |
+| yuriygreben | Architect | Physics-aware multi-phase strategy |
+
+### v2: In-Match Opponent Profiling & Adaptation
+
+New adaptive layer that monitors opponent behavior in real-time:
+
+**What it tracks (EMA-smoothed):**
+- **Aggression** — fleet launch frequency (how often they attack)
+- **Expansion rate** — planet capture speed
+- **Relative strength** — ship/planet differential
 
-### Base: tamrazov-starwars (LB 1224)
-- Gang-up attacks on weakened planets
-- Weakest enemy targeting (focus fire in 4P)
-- Elimination missions with high bonus
-- Aggressive endgame total-war mode
-- Exposed planet exploitation
+**How it adapts:**
 
-### Enhancements from ykhnkf (#1 LB)
-- **Hostile reinforcement prediction**: When attacking enemy planets, estimates how many reinforcement ships the enemy could send from nearby planets within a time window after our arrival. This adds a safety margin to fleet sizes, preventing failed captures due to enemy counterattacks.
-- Higher finishing hostile send bonus (5 vs 3)
+| Opponent Style | Agent Response |
+|---------------|---------------|
+| Very aggressive (aggression > 0.6) | ↑ defense ratios, ↑ reinforcement priority, ↓ attack aggression |
+| Passive/turtle (aggression < 0.3) | ↑ attack multipliers, ↑ elimination bonus, ↑ expansion pressure |
+| We're ahead | Play safe, consolidate, higher attack cost weighting |
+| We're behind | Take risks, ↑ snipe values, ↑ finishing bonuses, lower defense |
+| Enemy expanding fast | Contest neutrals more aggressively, ↓ target margins |
+| Late game (step > 350) | Maximum elimination drive, ↑ finishing multipliers |
 
-### Enhancements from pascal (v14)
-- **4-source swarm attacks**: Can coordinate 4 separate fleets to arrive simultaneously at a heavily defended target (40+ ships)
-- Expanded multi-source consideration (top 8 vs top 5)
+**20 parameters dynamically tuned** during each match based on game state.
 
-### Parameter Tuning
-- Extended simulation horizon (130 vs 110) for better long-range planning
-- Earlier late-game transition (70 remaining turns)
-- Stronger elimination drive (bonus 55 vs 28)
-- More aggressive enemy weakness detection (threshold 110 vs 45-60)
-- Enhanced proactive defense ratios for 4-player games
+### Key Technical Features
+
+1. **Hostile Reinforcement Prediction** — estimates enemy counterattack potential before committing fleets
+2. **4-Source Coordinated Swarms** — synchronizes 4 fleets to overwhelm defended targets
+3. **Multi-Phase Target Selection** — opening expansion → mid-game optimization → late-game elimination
+4. **Sun-Aware Fleet Routing** — avoids solar destruction with safe detour angles
+5. **Crash Exploit Detection** — captures planets weakened by enemy fleet collisions
+6. **Doomed Planet Evacuation** — retreats from unsaveable positions to useful targets
 
 ## Performance
 
-Local testing results:
 | Opponent | Win Rate | Notes |
 |----------|----------|-------|
-| Random | 100% | Eliminated by step ~180 |
-| Nearest-Sniper | 100% | Eliminated by step ~140 |
-| 3× Random (4P) | 100% | All eliminated by step ~120 |
-| Mixed seeds as P1/P2 | 83%+ | Consistent across positions |
+| Random | **100%** (3/3) | Eliminated by step ~94-150 |
+| Nearest-Sniper | **100%** (4/4) | Eliminated by step ~88-152 |
+| 3× Random (4P) | **100%** | All eliminated by step ~123 |
 
 ## Usage
 
 ### Direct Kaggle Submission
-Download `submission.py` and submit to the Orbit Wars competition:
-
 ```bash
-# Download
 wget https://huggingface.co/Builder-Neekhil/orbit-wars-agent/resolve/main/submission.py
-
-# Or use the Kaggle API
-kaggle competitions submit orbit-wars -f submission.py -m "Enhanced composite agent"
+kaggle competitions submit orbit-wars -f submission.py -m "v2 adaptive agent"
 ```
 
 ### Local Testing
 ```python
 from kaggle_environments import make
-
-# Load agent
 exec(open('submission.py').read(), globals())
 
-# Run a game
 env = make("orbit_wars", configuration={"seed": 42}, debug=False)
 env.run([agent, "random"])
-
-# Check results
 final = env.steps[-1]
 print(f"P0: {final[0].reward}, P1: {final[1].reward}")
 ```
 
-### 4-Player Testing
-```python
-env = make("orbit_wars", configuration={"seed": 42}, debug=False)
-env.run([agent, "random", "random", "random"])
-```
-
-## Key Strategic Components
-
-### 1. Target Selection (Multi-Phase)
-- **Opening:** Prioritize high-production neutral planets
-- **Mid-game:** Score-based selection considering production, distance, defense cost
-- **Late-game:** Aggressive elimination targeting with strong bonus
+## Self-Play PPO Training (Optional)
 
-### 2. Fleet Routing
-- Sun-avoidance with safe detour angles
-- Orbital prediction with lead-aim for moving targets
-- Multi-step intercept search for rotating planets
+The repo includes a PPO self-play training pipeline for further improvement:
 
-### 3. Multi-Source Coordination
-- 2/3/4 source synchronized swarm attacks
-- ETA tolerance matching for coordinated arrival
-- Optimal ship allocation across sources
+```bash
+pip install torch numpy pyyaml kaggle-environments huggingface_hub
 
-### 4. Defense
-- Proactive defense horizon scanning
-- Reinforcement missions to threatened planets
-- Doomed planet evacuation with retreat routing
-- Crash exploit detection (capturing planets after enemy fleet collisions)
+# Train (requires GPU for reasonable speed, ~10h on T4)
+TOTAL_UPDATES=500 EPISODES_PER_UPDATE=4 python train_efficient.py
+```
 
-### 5. Endgame
-- Total war mode with focused weakest-enemy targeting
-- Rear planet forwarding to frontline
-- Ship count optimization for final scoring
+**Training approach** (based on the Artificial Generals Intelligence paper, arXiv:2507.06825):
+- **Phase 1 (0-20%)**: Train vs random opponents (fast, learn basics)
+- **Phase 2 (20-50%)**: Train vs baseline agent (harder, learn tactics)
+- **Phase 3 (50-100%)**: Self-play with opponent pool (N=3, argmax opponents)
+- **Reward**: Potential-based shaping (planets + ships + production differential)
+- **Architecture**: 128-d MLP controller that outputs 20 parameter adjustments
 
 ## Files
-- `submission.py` — The complete agent (single-file, no dependencies beyond kaggle-environments)
+- `submission.py` — Complete adaptive agent (single-file, ready for Kaggle)
+- `train_efficient.py` — PPO self-play training script
+- `generate_submission.py` — Packages trained controller into submission file
 
 ## License
 MIT