results/PAPER_GATE_INTERPRETATION.md

Paper Gate Interpretation for Temporal Twins

This document is the paper-facing interpretation layer for the final Temporal Twins suite. It does not alter the raw diagnostic output in results/paper_suite_20260503_202810/paper_suite_failed_checks.csv. Instead, it reclassifies which checks are true hard gates versus descriptive or advisory findings for the paper.

Gate Categories

A. Hard Gates for oracle_calib

These are true benchmark validity gates for temporal_twins_oracle_calib:

• matched_eval_pairs >= required threshold
• positive_rate = 0.5
• benign_motif_hit_rate = 0
• static_agg_auc near 0.5
• shortcut AUCs near 0.5
• XGBoost near 0.5
• StaticGNN near chance
• AuditOracle near 1.0
• RawMotifOracle near 1.0
• SeqGRU high
• SeqGRU shuffle delta strongly negative

B. Hard Gates for Standard easy / medium / hard

For the standard temporal_twins difficulty ladder, the hard gates are the matched static-control checks:

• matched_eval_pairs >= required threshold
• positive_rate = 0.5
• benign_motif_hit_rate = 0
• static_agg_auc near 0.5
• shortcut AUCs near 0.5
• XGBoost near 0.5
• StaticGNN near chance

These conditions verify that the benchmark remains shortcut-resistant and that fraud and benign twins are properly matched at evaluation.

C. Advisory / Descriptive Checks for Standard easy / medium / hard

The following are not hard validity gates for the standard difficulty ladder:

• MotifProbe
• RawMotifProbe
• SeqGRU difficulty trend
• SeqGRU shuffle delta
• temporal-GNN performance
• temporal-GNN shuffle delta

These measurements are descriptive benchmark outcomes. They characterize difficulty and inductive bias; they do not determine whether the dataset itself is valid.

Reclassified Final Paper-Suite Status

oracle_calib

• hard gate passes: 5/5
• AuditOracle = 1.0000 ± 0.0000
• RawMotifOracle = 1.0000 ± 0.0000
• XGBoost = 0.5000 ± 0.0000
• StaticGNN = 0.5222 ± 0.0235
• SeqGRU = 1.0000 ± 0.0000
• SeqGRU delta = -0.5032 ± 0.0043

Interpretation: oracle_calib passes the intended hard benchmark validation. The oracle/probe alignment is correct, static shortcuts are eliminated, and a causal sequence model recovers the signal with a large negative shuffle delta.

easy

• static-control hard gates pass: 5/5
• XGBoost = 0.5000 ± 0.0000
• StaticGNN = 0.4946 ± 0.0128
• SeqGRU = 1.0000 ± 0.0000
• SeqGRU delta = -0.5003 ± 0.0096

Interpretation: easy is a valid standard benchmark slice. Static shortcuts remain suppressed, and the temporal sequence signal is strong.

medium

• static-control hard gates pass: 5/5
• XGBoost = 0.5000 ± 0.0000
• StaticGNN = 0.4922 ± 0.0203
• SeqGRU = 0.8391 ± 0.0174
• SeqGRU delta = -0.3337 ± 0.0191
• MotifProbe and RawMotifProbe are lower by design and should not be treated as hard-gate failures

Interpretation: medium is not a failed dataset. It passes the static-control hard gates and shows the intended increase in temporal difficulty.

hard

• static-control hard gates pass: 5/5
• XGBoost = 0.5000 ± 0.0000
• StaticGNN = 0.5026 ± 0.0198
• SeqGRU = 0.6876 ± 0.0128
• SeqGRU delta = -0.1883 ± 0.0111
• lower probe and SeqGRU scores reflect intended difficulty

Interpretation: hard is not a failed dataset. It passes the static-control hard gates and intentionally weakens temporal recoverability relative to easy and medium.

Reclassified Status Table

Benchmark	Static-control hard gates	Probe/oracle status	SeqGRU status	Temporal-GNN status	Paper interpretation
oracle_calib	Pass 5/5	AuditOracle and RawMotifOracle both near 1.0; oracle behavior validated	1.0000 ± 0.0000, delta -0.5032 ± 0.0043	Underperformance is advisory only	Valid calibration benchmark with correct motif-label alignment and dead static shortcuts
easy	Pass 5/5	MotifProbe / RawMotifProbe high, descriptive	1.0000 ± 0.0000, delta -0.5003 ± 0.0096	Advisory underperformance only	Valid standard benchmark with strong temporal signal
medium	Pass 5/5	Lower probes are expected and descriptive, not failures	0.8391 ± 0.0174, delta -0.3337 ± 0.0191	Advisory underperformance only	Valid medium-difficulty benchmark with increased temporal challenge
hard	Pass 5/5	Lower probes are expected and descriptive, not failures	0.6876 ± 0.0128, delta -0.1883 ± 0.0111	Advisory underperformance only	Valid hard benchmark with intentionally reduced temporal recoverability

Temporal-GNN advisory failures are not benchmark failures. They support the paper finding that current temporal GNNs may not exploit order-sensitive temporal structure as effectively as a causal sequence model under matched static controls.

medium and hard are not failed datasets. They are intended difficulty levels in the Temporal Twins ladder. Their lower MotifProbe, RawMotifProbe, and SeqGRU values show increasing temporal difficulty rather than invalid benchmark construction.

Notes on the Raw Diagnostic File

• results/paper_suite_20260503_202810/paper_suite_failed_checks.csv is retained unchanged as the raw diagnostic output.
• The raw file still reflects older gate semantics in which standard-mode probe thresholds and temporal-GNN thresholds appeared in failure columns.
• This document is the corrected paper-facing interpretation layer and should be cited when describing benchmark validity in the manuscript.