Hugging Face's logo

stevenkhan
/
clashcr

ml-intern
Model card Files
xet
 Community
clashcr / REWRITE_PLAN.md
stevenkhan's picture
stevenkhan
Upload REWRITE_PLAN.md with huggingface_hub
3f8e91d verified
preview code
|
raw
history blame contribute delete
6.01 kB

ClashCR Rewrite Plan

1. Research Summary

Evaluated Projects

Project URL Credibility Visual Mode Approach Gaps
KataCR https://github.com/wty-yy/katacr ★★★★★ Academic paper Live + Replay YOLO unit detection + ResNet hand classifier + RL policy Does NOT track opponent cards; focuses on self-play AI. Dataset stale (May 2024).
CR Vision https://github.com/bytkim/cr_vision ★★★★☆ Active 2026 Live (BlueStacks) YOLO troop detection + grid tracker + wall-clock elixir + cycle inference Correct architecture for opponent tracking. Missing evolutions/heroes/spell detection. ~60 cards in cost table.
TV Royale Dataset https://huggingface.co/datasets/chrisrca/clash-royale-tv-replays ★★★☆☆ Replay/TV Frame-level card crops with x,y positions Not usable for live opponent tracking; no opponent hand info.
RoyaleAPI cr-api-data https://royaleapi.github.io/cr-api-data/json/cards.json ★★★☆☆ Static data 120 cards as of snapshot Stale: missing 2025-2026 cards (Vines, Spirit Empress, Hero cards, etc.).
Official Clash Royale API https://developer.clashroyale.com/ ★★★★★ Official /v1/cards endpoint Requires API token; best source of truth.

Why Current Whole-Frame Classifier Fails

  • Opponent card art is never visible in live gameplay. Only own hand cards are shown.
  • A whole-frame classifier trained on card images will hallucinate predictions when shown combat frames.
  • The correct signal is unit spawns and spell effects on the arena, not card art.
  • KataCR paper explicitly splits perception into: arena (YOLO units), hand cards (classifier), time/elixir (OCR). Opponent cards are inferred from arena detections, never classified directly.

2. Recommended Architecture 

Emulator Capture (Win32 / mss)
    |
    v
Battle Gater (lobby vs battle detection, ROI computation)
    |
    v
Event Detector (temporal differencing, persistence suppression, side separation)
    |
    v
Evidence Model (YOLO unit detector + heuristic spell effect detectors)
    |
    v
Card Resolver (unit->card mapping, multi-unit rules, evolution/hero handling, ambiguity->UNKNOWN)
    |
    v
Elixir Tracker (wall-clock regen with mode schedule)
Deck Tracker (cycle inference, meta priors)

3. Concrete Rewrite Plan for F:\ClashCR

  1. Delete old whole-frame card classifier code and stale card JSON dumps.
  2. Add clashcr/utils/card_registry.py with sync from RoyaleAPI + Official API.
  3. Add clashcr/core/capture.py for MuMu/BlueStacks window capture.
  4. Add clashcr/core/battle_gating.py for battle detection and ROI separation.
  5. Add clashcr/core/event_detector.py for temporal event detection.
  6. Add clashcr/core/recorder.py for evidence recording and labeling.
  7. Add clashcr/models/evidence_model.py for YOLO + spell heuristics.
  8. Add clashcr/models/card_resolver.py for unit->card mapping with ambiguity handling.
  9. Add clashcr/game/elixir_tracker.py for elixir estimation.
  10. Add clashcr/game/deck_tracker.py for deck/cycle inference.
  11. Add clashcr/core/evaluator.py for offline metrics.
  12. Add clashcr/cli.py for all CLI commands.

4. Card Registry Sync

  • Primary: RoyaleAPI static JSON (fallback)
  • Secondary: Official API /v1/cards (requires token)
  • Logs: source URL, sync timestamp, card count, IDs, costs, types, rarities, evolution/hero/tower-troop flags.
  • Fails loudly if registry is stale or missing expected new cards.

5. Dataset Collection and Labeling Plan

  • clashcr record-battle --output data/live-recordings/session-001 --seconds 180 --fps 8
  • Saves frames, metadata.jsonl, and creates empty labels.csv / predictions.csv.
  • User manually labels opponent card plays in labels.csv.
  • Required validation set:

    • = 20 full battles

    • Multiple arenas/maps
    • Multiple resolutions
    • Single/double/triple elixir periods
    • All card categories (troops, spells, buildings, champions, evolutions, heroes, tower troops)
    • Negative recordings (lobby/menu, quiet battle periods)

6. Training Plan

  • Unit detector: Fine-tune YOLOv8 on KataCR detection dataset + custom labeled spawns.
  • Spell effects: Heuristic color/motion detectors (no training required for MVP).
  • Future: Train a lightweight spell-effect classifier on recorded spell crops.

7. CLI Commands 

# Setup
pip install -e .
clashcr sync-cards --config config.yaml

# Recording
clashcr record-battle --config config.yaml --output data/live-recordings/session-001 --seconds 180 --fps 8

# Labeling
clashcr label-recording --recording data/live-recordings/session-001
# (Then edit labels.csv manually or with a GUI tool)

# Training (placeholder)
clashcr train-event-model --config config.yaml --data data/live-recordings

# Evaluation
clashcr evaluate-recording --config config.yaml --recording data/live-recordings/session-heldout --labels data/live-recordings/session-heldout/labels.csv

# Live run
clashcr run --config config.yaml --raw --debug-frame debug-frame.jpg --debug-dir debug-events

8. Debug Checklist for Live Failures

  • Emulator window detected? Run python -m clashcr.core.capture to list windows.
  • Battle detected? Check battle confidence in logs.
  • Events firing? Check debug-events/ directory for crops.
  • YOLO model loaded? Verify path in config.yaml.
  • Side attribution correct? Check opponent ROI vs own ROI.
  • Elixir tracking reasonable? Compare to wall-clock regen.
  • False positives? Increase min_event_area or temporal_confirmation.
  • Missed spells? Verify spell heuristics or add new color ranges.

9. Verification Report Requirements

  • Run evaluation on held-out labeled recordings.
  • Report: precision, recall, F1, FP/min, missed events, mean timing error.
  • If 100% not achievable, state plainly and identify missing visual signal.