stevenkhan commited on
Commit
3f8e91d
Β·
verified Β·
1 Parent(s): f2d7808

Upload REWRITE_PLAN.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. REWRITE_PLAN.md +116 -0
REWRITE_PLAN.md ADDED
@@ -0,0 +1,116 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # ClashCR Rewrite Plan
2
+
3
+ ## 1. Research Summary
4
+
5
+ ### Evaluated Projects
6
+ | Project | URL | Credibility | Visual Mode | Approach | Gaps |
7
+ |---------|-----|-------------|-------------|----------|------|
8
+ | KataCR | https://github.com/wty-yy/katacr | β˜…β˜…β˜…β˜…β˜… Academic paper | Live + Replay | YOLO unit detection + ResNet hand classifier + RL policy | Does NOT track opponent cards; focuses on self-play AI. Dataset stale (May 2024). |
9
+ | CR Vision | https://github.com/bytkim/cr_vision | β˜…β˜…β˜…β˜…β˜† Active 2026 | Live (BlueStacks) | YOLO troop detection + grid tracker + wall-clock elixir + cycle inference | Correct architecture for opponent tracking. Missing evolutions/heroes/spell detection. ~60 cards in cost table. |
10
+ | TV Royale Dataset | https://huggingface.co/datasets/chrisrca/clash-royale-tv-replays | β˜…β˜…β˜…β˜†β˜† | Replay/TV | Frame-level card crops with x,y positions | Not usable for live opponent tracking; no opponent hand info. |
11
+ | RoyaleAPI cr-api-data | https://royaleapi.github.io/cr-api-data/json/cards.json | β˜…β˜…β˜…β˜†β˜† | Static data | 120 cards as of snapshot | Stale: missing 2025-2026 cards (Vines, Spirit Empress, Hero cards, etc.). |
12
+ | Official Clash Royale API | https://developer.clashroyale.com/ | β˜…β˜…β˜…β˜…β˜… | Official | /v1/cards endpoint | Requires API token; best source of truth. |
13
+
14
+ ### Why Current Whole-Frame Classifier Fails
15
+ - **Opponent card art is never visible in live gameplay.** Only own hand cards are shown.
16
+ - A whole-frame classifier trained on card images will hallucinate predictions when shown combat frames.
17
+ - The correct signal is **unit spawns and spell effects on the arena**, not card art.
18
+ - KataCR paper explicitly splits perception into: arena (YOLO units), hand cards (classifier), time/elixir (OCR). Opponent cards are inferred from arena detections, never classified directly.
19
+
20
+ ## 2. Recommended Architecture
21
+
22
+ ```
23
+ Emulator Capture (Win32 / mss)
24
+ |
25
+ v
26
+ Battle Gater (lobby vs battle detection, ROI computation)
27
+ |
28
+ v
29
+ Event Detector (temporal differencing, persistence suppression, side separation)
30
+ |
31
+ v
32
+ Evidence Model (YOLO unit detector + heuristic spell effect detectors)
33
+ |
34
+ v
35
+ Card Resolver (unit->card mapping, multi-unit rules, evolution/hero handling, ambiguity->UNKNOWN)
36
+ |
37
+ v
38
+ Elixir Tracker (wall-clock regen with mode schedule)
39
+ Deck Tracker (cycle inference, meta priors)
40
+ ```
41
+
42
+ ## 3. Concrete Rewrite Plan for F:\ClashCR
43
+
44
+ 1. **Delete** old whole-frame card classifier code and stale card JSON dumps.
45
+ 2. **Add** `clashcr/utils/card_registry.py` with sync from RoyaleAPI + Official API.
46
+ 3. **Add** `clashcr/core/capture.py` for MuMu/BlueStacks window capture.
47
+ 4. **Add** `clashcr/core/battle_gating.py` for battle detection and ROI separation.
48
+ 5. **Add** `clashcr/core/event_detector.py` for temporal event detection.
49
+ 6. **Add** `clashcr/core/recorder.py` for evidence recording and labeling.
50
+ 7. **Add** `clashcr/models/evidence_model.py` for YOLO + spell heuristics.
51
+ 8. **Add** `clashcr/models/card_resolver.py` for unit->card mapping with ambiguity handling.
52
+ 9. **Add** `clashcr/game/elixir_tracker.py` for elixir estimation.
53
+ 10. **Add** `clashcr/game/deck_tracker.py` for deck/cycle inference.
54
+ 11. **Add** `clashcr/core/evaluator.py` for offline metrics.
55
+ 12. **Add** `clashcr/cli.py` for all CLI commands.
56
+
57
+ ## 4. Card Registry Sync
58
+ - Primary: RoyaleAPI static JSON (fallback)
59
+ - Secondary: Official API `/v1/cards` (requires token)
60
+ - Logs: source URL, sync timestamp, card count, IDs, costs, types, rarities, evolution/hero/tower-troop flags.
61
+ - Fails loudly if registry is stale or missing expected new cards.
62
+
63
+ ## 5. Dataset Collection and Labeling Plan
64
+ - `clashcr record-battle --output data/live-recordings/session-001 --seconds 180 --fps 8`
65
+ - Saves frames, metadata.jsonl, and creates empty labels.csv / predictions.csv.
66
+ - User manually labels opponent card plays in labels.csv.
67
+ - Required validation set:
68
+ - >= 20 full battles
69
+ - Multiple arenas/maps
70
+ - Multiple resolutions
71
+ - Single/double/triple elixir periods
72
+ - All card categories (troops, spells, buildings, champions, evolutions, heroes, tower troops)
73
+ - Negative recordings (lobby/menu, quiet battle periods)
74
+
75
+ ## 6. Training Plan
76
+ - **Unit detector**: Fine-tune YOLOv8 on KataCR detection dataset + custom labeled spawns.
77
+ - **Spell effects**: Heuristic color/motion detectors (no training required for MVP).
78
+ - **Future**: Train a lightweight spell-effect classifier on recorded spell crops.
79
+
80
+ ## 7. CLI Commands
81
+ ```bash
82
+ # Setup
83
+ pip install -e .
84
+ clashcr sync-cards --config config.yaml
85
+
86
+ # Recording
87
+ clashcr record-battle --config config.yaml --output data/live-recordings/session-001 --seconds 180 --fps 8
88
+
89
+ # Labeling
90
+ clashcr label-recording --recording data/live-recordings/session-001
91
+ # (Then edit labels.csv manually or with a GUI tool)
92
+
93
+ # Training (placeholder)
94
+ clashcr train-event-model --config config.yaml --data data/live-recordings
95
+
96
+ # Evaluation
97
+ clashcr evaluate-recording --config config.yaml --recording data/live-recordings/session-heldout --labels data/live-recordings/session-heldout/labels.csv
98
+
99
+ # Live run
100
+ clashcr run --config config.yaml --raw --debug-frame debug-frame.jpg --debug-dir debug-events
101
+ ```
102
+
103
+ ## 8. Debug Checklist for Live Failures
104
+ - [ ] Emulator window detected? Run `python -m clashcr.core.capture` to list windows.
105
+ - [ ] Battle detected? Check battle confidence in logs.
106
+ - [ ] Events firing? Check debug-events/ directory for crops.
107
+ - [ ] YOLO model loaded? Verify path in config.yaml.
108
+ - [ ] Side attribution correct? Check opponent ROI vs own ROI.
109
+ - [ ] Elixir tracking reasonable? Compare to wall-clock regen.
110
+ - [ ] False positives? Increase `min_event_area` or `temporal_confirmation`.
111
+ - [ ] Missed spells? Verify spell heuristics or add new color ranges.
112
+
113
+ ## 9. Verification Report Requirements
114
+ - Run evaluation on held-out labeled recordings.
115
+ - Report: precision, recall, F1, FP/min, missed events, mean timing error.
116
+ - If 100% not achievable, state plainly and identify missing visual signal.