final_plots/README.md

Final plots — submission-ready figures for judging

Maps directly to the 4 judging criteria:

• Reward curves → training trajectory of the v2 GRPO run
• Metrics → per-difficulty, threshold sweep, calibration, ablation, leakage
• Before/after behavior → v1 vs v2 fix, scripted vs trained, co-evolution rounds

Regenerate with: python3 eval/build_final_plots.py (or re-run the source notebooks then eval/plot_coevolution.py).

Inventory

File	Category	What it shows
training_curves_v2.png	Reward curve	v2 Analyzer GRPO training: mean reward + std band, KL divergence, loss, gradient norm across 619 steps. THE training reward curve.
reward_hacking_diagnostic.png	Before/after	v1 LoRA's uniform 100% detection across all difficulty buckets — the visible signature of reward hacking that triggered the v1→v2 fix.
v2_per_difficulty_check.png	Metrics	Per-difficulty detection of v2 LoRA vs scripted: 100/100/100/97% across easy/medium/hard/novel.
baseline_vs_trained_overall.png	Before/after	Aggregate detection: scripted baseline vs v2 LoRA on the 174-scenario bench.
baseline_vs_trained_per_category.png	Before/after	Per-category detection: scripted vs v2 LoRA, broken out by scam category.
v1_vs_v2_fingerprint.png	Before/after	B.1 controlled experiment: SFT baseline vs v2 GRPO, same LoRA + same training data, only algorithm differs.
ece_reliability.png	Metrics	Calibration: Expected Calibration Error + reliability diagram for v2 (B.6).
ablation_per_rubric.png	Metrics	Per-rubric ablation: contribution of each of 8 reward rubrics to final v2 detection/FPR.
leakage_clean_slice.png	Metrics	Leakage-clean OOD slice: v2 detection on cosine<0.70 subset (50 scenarios) — generalization, not memorization.
semantic_leakage_histogram.png	Metrics	Cosine-similarity histogram between bench and training corpus — honest disclosure of 44.8% high-leakage.
temporal_gap_closure.png	Before/after	Detection gap closure on post-2024 novel attacks: scripted (76.5%) vs v2 LoRA (97.1%).
rubric_decomposition.png	Metrics	Per-rubric reward decomposition over training — which rubrics dominated learning.
coevolution_headline.png	Before/after	B.2 co-evolution headline: bypass rate for ScriptedAnalyzer vs v2 LoRA across train/held-out splits.
coevolution_per_category.png	Before/after	B.2 per-category bypass: where v2 LoRA holds and where it has known gaps (vaccine, customer-support, EMI).
scammer_phase1_per_category.png	Before/after	B.2 phase 1 Scammer LoRA: per-category bypass of rule-based defense, single-shot vs best-of-8.
headline_v1_vs_v2_reward_fix.png	Before/after	THE dramatic v1→v2 headline: detection stable at 99.3%+, FPR drops 36% → 6.7% (5× better). Asymmetric improvement = real learning vs reward-hacking.
metrics_v2_threshold_sweep.png	Metrics	Threshold-degeneracy plot: 12 of 13 thresholds yield identical metrics. Demonstrates v2 produces a bimodal score distribution — model is confident, not borderline.
metrics_b1_sft_vs_grpo.png	Metrics	B.1 controlled experiment: SFT baseline ties v2 GRPO within Wilson CIs. Honest research-rigor signal — answers 'did GRPO actually help?'