# Final Paper Suite Summary

Rows: 20

## Dataset and Audit
| Benchmark | Matched Pairs | Positives | Negatives | Fraud Users | Benign Users | Templates | Positive Rate | Benign Motif Hit | static_agg_auc | Txn AUC | Idx AUC | Prefix AUC | Time AUC | Account AUC | Active AUC |
|---|---:|---:|---:|---:|---:|---:|---:|---:|---:|---:|---:|---:|---:|---:|---:|
| easy | 2222.2000 ± 128.4434 | 2222.2000 ± 128.4434 | 2222.2000 ± 128.4434 | 304.6000 ± 23.2551 | 304.6000 ± 23.2551 | 300.2000 ± 24.2425 | 0.5000 ± 0.0000 | 0.0000 ± 0.0000 | 0.5000 ± 0.0000 | 0.5000 ± 0.0000 | 0.5000 ± 0.0000 | 0.5000 ± 0.0000 | 0.5000 ± 0.0000 | 0.5000 ± 0.0000 | 0.5000 ± 0.0000 |
| hard | 2317.6000 ± 21.9727 | 2317.6000 ± 21.9727 | 2317.6000 ± 21.9727 | 315.0000 ± 0.0000 | 315.0000 ± 0.0000 | 315.0000 ± 0.0000 | 0.5000 ± 0.0000 | 0.0000 ± 0.0000 | 0.5000 ± 0.0000 | 0.5000 ± 0.0000 | 0.5000 ± 0.0000 | 0.5000 ± 0.0000 | 0.5000 ± 0.0000 | 0.5000 ± 0.0000 | 0.5000 ± 0.0000 |
| medium | 2356.6000 ± 18.0361 | 2356.6000 ± 18.0361 | 2356.6000 ± 18.0361 | 263.0000 ± 0.0000 | 263.0000 ± 0.0000 | 263.0000 ± 0.0000 | 0.5000 ± 0.0000 | 0.0000 ± 0.0000 | 0.5000 ± 0.0000 | 0.5000 ± 0.0000 | 0.5000 ± 0.0000 | 0.5000 ± 0.0000 | 0.5000 ± 0.0000 | 0.5000 ± 0.0000 | 0.5000 ± 0.0000 |
| oracle_calib | 2606.6000 ± 454.3449 | 2606.6000 ± 454.3449 | 2606.6000 ± 454.3449 | 409.8000 ± 86.5402 | 409.8000 ± 86.5402 | 409.8000 ± 86.5402 | 0.5000 ± 0.0000 | 0.0000 ± 0.0000 | 0.5000 ± 0.0000 | 0.5000 ± 0.0000 | 0.5000 ± 0.0000 | 0.5000 ± 0.0000 | 0.5000 ± 0.0000 | 0.5000 ± 0.0000 | 0.5000 ± 0.0000 |

## Probes and Models
| Benchmark | Primary Probe | Secondary Probe | XGB ROC/PR | StaticGNN ROC/PR | SeqGRU Clean ROC/PR | SeqGRU Shuf ROC/PR | SeqGRU Delta |
|---|---:|---:|---:|---:|---:|---:|---:|
| easy | MotifProbe 1.0000 ± 0.0000 | RawMotifProbe 0.9983 ± 0.0011 | 0.5000 ± 0.0000 / 0.5000 ± 0.0000 | 0.4946 ± 0.0128 / 0.5001 ± 0.0099 | 1.0000 ± 0.0000 / 1.0000 ± 0.0000 | 0.4997 ± 0.0096 / 0.5008 ± 0.0077 | -0.5003 ± 0.0096 |
| hard | MotifProbe 0.5790 ± 0.0045 | RawMotifProbe 0.5910 ± 0.0105 | 0.5000 ± 0.0000 / 0.5000 ± 0.0000 | 0.5026 ± 0.0198 / 0.5081 ± 0.0128 | 0.6876 ± 0.0128 / 0.7058 ± 0.0250 | 0.4994 ± 0.0033 / 0.4981 ± 0.0031 | -0.1883 ± 0.0111 |
| medium | MotifProbe 0.6374 ± 0.0069 | RawMotifProbe 0.6482 ± 0.0058 | 0.5000 ± 0.0000 / 0.5000 ± 0.0000 | 0.4922 ± 0.0203 / 0.4956 ± 0.0231 | 0.8391 ± 0.0174 / 0.8520 ± 0.0110 | 0.5053 ± 0.0071 / 0.5054 ± 0.0078 | -0.3337 ± 0.0191 |
| oracle_calib | AuditOracle 1.0000 ± 0.0000 | RawMotifOracle 1.0000 ± 0.0000 | 0.5000 ± 0.0000 / 0.5000 ± 0.0000 | 0.5222 ± 0.0235 / 0.5141 ± 0.0202 | 1.0000 ± 0.0000 / 1.0000 ± 0.0000 | 0.4968 ± 0.0043 / 0.5008 ± 0.0057 | -0.5032 ± 0.0043 |

## Temporal GNNs
| Benchmark | TGN ROC/PR/Delta | TGAT ROC/PR/Delta | DyRep ROC/PR/Delta | JODIE ROC/PR/Delta |
|---|---:|---:|---:|---:|
| easy | 0.5660 ± 0.0123 / 0.5566 ± 0.0107 / -0.0668 ± 0.0126 | 0.5200 ± 0.0119 / 0.5177 ± 0.0086 / -0.0294 ± 0.0244 | 0.5320 ± 0.0090 / 0.5245 ± 0.0100 / -0.0306 ± 0.0150 | 0.5282 ± 0.0085 / 0.5210 ± 0.0079 / -0.0420 ± 0.0231 |
| hard | 0.5060 ± 0.0081 / 0.5069 ± 0.0062 / -0.0006 ± 0.0131 | 0.5157 ± 0.0135 / 0.5133 ± 0.0163 / -0.0145 ± 0.0173 | 0.5229 ± 0.0050 / 0.5172 ± 0.0072 / -0.0171 ± 0.0147 | 0.5137 ± 0.0085 / 0.5087 ± 0.0076 / -0.0231 ± 0.0139 |
| medium | 0.5177 ± 0.0063 / 0.5133 ± 0.0034 / -0.0124 ± 0.0094 | 0.5199 ± 0.0146 / 0.5121 ± 0.0105 / -0.0334 ± 0.0211 | 0.5285 ± 0.0246 / 0.5252 ± 0.0144 / -0.0290 ± 0.0181 | 0.5298 ± 0.0057 / 0.5235 ± 0.0103 / -0.0299 ± 0.0061 |
| oracle_calib | 0.6270 ± 0.0103 / 0.6107 ± 0.0144 / -0.1288 ± 0.0173 | 0.6327 ± 0.0465 / 0.6104 ± 0.0381 / -0.1403 ± 0.0654 | 0.6135 ± 0.0481 / 0.6010 ± 0.0400 / -0.1203 ± 0.0600 | 0.5903 ± 0.0170 / 0.5720 ± 0.0146 / -0.0947 ± 0.0227 |

## Runtime
| Benchmark | Run Time (sec) | StaticGNN Eval Time (sec) |
|---|---:|---:|
| easy | 1345.9431 ± 135.9206 | 198.4129 ± 45.3445 |
| hard | 2613.7203 ± 231.4288 | 272.9988 ± 70.8384 |
| medium | 2181.9480 ± 228.0777 | 184.2332 ± 57.5306 |
| oracle_calib | 1136.5994 ± 283.1015 | 170.9515 ± 34.9402 |

## Failed Gate Checks
| Benchmark | Seed | Gate Pass | Volume Failures | Hard Gate Failures | Advisory Failures |
|---|---:|---:|---|---|---|
| easy | 0 | 1 | - | - | TGN ROC-AUC: 0.5835 (>=0.75) | TGN shuffle delta: -0.0859 (<=-0.1) | TGAT ROC-AUC: 0.5060 (>=0.75) | TGAT shuffle delta: 0.0037 (<=-0.1) | DyRep ROC-AUC: 0.5403 (>=0.75) | DyRep shuffle delta: -0.0398 (<=-0.1) | JODIE ROC-AUC: 0.5382 (>=0.75) | JODIE shuffle delta: -0.0824 (<=-0.1) |
| easy | 1 | 1 | - | - | TGN ROC-AUC: 0.5495 (>=0.75) | TGN shuffle delta: -0.0536 (<=-0.1) | TGAT ROC-AUC: 0.5254 (>=0.75) | TGAT shuffle delta: -0.0422 (<=-0.1) | DyRep ROC-AUC: 0.5251 (>=0.75) | DyRep shuffle delta: -0.0090 (<=-0.1) | JODIE ROC-AUC: 0.5186 (>=0.75) | JODIE shuffle delta: -0.0257 (<=-0.1) |
| easy | 2 | 1 | - | - | TGN ROC-AUC: 0.5638 (>=0.75) | TGN shuffle delta: -0.0725 (<=-0.1) | TGAT ROC-AUC: 0.5103 (>=0.75) | TGAT shuffle delta: -0.0200 (<=-0.1) | DyRep ROC-AUC: 0.5372 (>=0.75) | DyRep shuffle delta: -0.0433 (<=-0.1) | JODIE ROC-AUC: 0.5357 (>=0.75) | JODIE shuffle delta: -0.0324 (<=-0.1) |
| easy | 3 | 1 | - | - | TGN ROC-AUC: 0.5693 (>=0.75) | TGN shuffle delta: -0.0604 (<=-0.1) | TGAT ROC-AUC: 0.5229 (>=0.75) | TGAT shuffle delta: -0.0271 (<=-0.1) | DyRep ROC-AUC: 0.5375 (>=0.75) | DyRep shuffle delta: -0.0401 (<=-0.1) | JODIE ROC-AUC: 0.5259 (>=0.75) | JODIE shuffle delta: -0.0310 (<=-0.1) |
| easy | 4 | 1 | - | - | TGN ROC-AUC: 0.5639 (>=0.75) | TGN shuffle delta: -0.0617 (<=-0.1) | TGAT ROC-AUC: 0.5353 (>=0.75) | TGAT shuffle delta: -0.0616 (<=-0.1) | DyRep ROC-AUC: 0.5198 (>=0.75) | DyRep shuffle delta: -0.0208 (<=-0.1) | JODIE ROC-AUC: 0.5225 (>=0.75) | JODIE shuffle delta: -0.0386 (<=-0.1) |
| hard | 0 | 0 | - | MotifProbe ROC-AUC: 0.5838 (>=0.99) | MotifProbe pair-sep: 0.1676 (>=0.99) | RawMotifProbe ROC-AUC: 0.5946 (>=0.95) | RawMotifProbe pair-sep: 0.2200 (>=0.9) | SeqGRU ROC-AUC: 0.7069 (>=0.8) | TGN ROC-AUC: 0.5072 (>=0.75) | TGN shuffle delta: 0.0107 (<=-0.1) | TGAT ROC-AUC: 0.5192 (>=0.75) | TGAT shuffle delta: -0.0276 (<=-0.1) | DyRep ROC-AUC: 0.5277 (>=0.75) | DyRep shuffle delta: -0.0311 (<=-0.1) | JODIE ROC-AUC: 0.5097 (>=0.75) | JODIE shuffle delta: -0.0098 (<=-0.1) |
| hard | 1 | 0 | - | MotifProbe ROC-AUC: 0.5771 (>=0.99) | MotifProbe pair-sep: 0.1543 (>=0.99) | RawMotifProbe ROC-AUC: 0.5873 (>=0.95) | RawMotifProbe pair-sep: 0.2181 (>=0.9) | SeqGRU ROC-AUC: 0.6795 (>=0.8) | TGN ROC-AUC: 0.4995 (>=0.75) | TGN shuffle delta: 0.0021 (<=-0.1) | TGAT ROC-AUC: 0.5159 (>=0.75) | TGAT shuffle delta: -0.0116 (<=-0.1) | DyRep ROC-AUC: 0.5243 (>=0.75) | DyRep shuffle delta: -0.0118 (<=-0.1) | JODIE ROC-AUC: 0.5218 (>=0.75) | JODIE shuffle delta: -0.0348 (<=-0.1) |
| hard | 2 | 0 | - | MotifProbe ROC-AUC: 0.5824 (>=0.99) | MotifProbe pair-sep: 0.1648 (>=0.99) | RawMotifProbe ROC-AUC: 0.6057 (>=0.95) | RawMotifProbe pair-sep: 0.2333 (>=0.9) | SeqGRU ROC-AUC: 0.6946 (>=0.8) | TGN ROC-AUC: 0.5187 (>=0.75) | TGN shuffle delta: -0.0215 (<=-0.1) | TGAT ROC-AUC: 0.5278 (>=0.75) | TGAT shuffle delta: -0.0359 (<=-0.1) | DyRep ROC-AUC: 0.5257 (>=0.75) | DyRep shuffle delta: -0.0312 (<=-0.1) | JODIE ROC-AUC: 0.5231 (>=0.75) | JODIE shuffle delta: -0.0260 (<=-0.1) |
| hard | 3 | 0 | - | MotifProbe ROC-AUC: 0.5724 (>=0.99) | MotifProbe pair-sep: 0.1448 (>=0.99) | RawMotifProbe ROC-AUC: 0.5768 (>=0.95) | RawMotifProbe pair-sep: 0.2038 (>=0.9) | SeqGRU ROC-AUC: 0.6799 (>=0.8) | TGN ROC-AUC: 0.4986 (>=0.75) | TGN shuffle delta: 0.0097 (<=-0.1) | TGAT ROC-AUC: 0.4929 (>=0.75) | TGAT shuffle delta: 0.0066 (<=-0.1) | DyRep ROC-AUC: 0.5146 (>=0.75) | DyRep shuffle delta: -0.0155 (<=-0.1) | JODIE ROC-AUC: 0.5109 (>=0.75) | JODIE shuffle delta: -0.0075 (<=-0.1) |
| hard | 4 | 0 | - | MotifProbe ROC-AUC: 0.5790 (>=0.99) | MotifProbe pair-sep: 0.1581 (>=0.99) | RawMotifProbe ROC-AUC: 0.5904 (>=0.95) | RawMotifProbe pair-sep: 0.2238 (>=0.9) | SeqGRU ROC-AUC: 0.6773 (>=0.8) | TGN ROC-AUC: 0.5059 (>=0.75) | TGN shuffle delta: -0.0038 (<=-0.1) | TGAT ROC-AUC: 0.5225 (>=0.75) | TGAT shuffle delta: -0.0040 (<=-0.1) | DyRep ROC-AUC: 0.5225 (>=0.75) | DyRep shuffle delta: 0.0039 (<=-0.1) | JODIE ROC-AUC: 0.5032 (>=0.75) | JODIE shuffle delta: -0.0375 (<=-0.1) |
| medium | 0 | 0 | - | MotifProbe ROC-AUC: 0.6463 (>=0.99) | MotifProbe pair-sep: 0.2926 (>=0.99) | RawMotifProbe ROC-AUC: 0.6473 (>=0.95) | RawMotifProbe pair-sep: 0.4194 (>=0.9) | TGN ROC-AUC: 0.5194 (>=0.75) | TGN shuffle delta: -0.0238 (<=-0.1) | TGAT ROC-AUC: 0.5019 (>=0.75) | TGAT shuffle delta: -0.0047 (<=-0.1) | DyRep ROC-AUC: 0.4964 (>=0.75) | DyRep shuffle delta: -0.0249 (<=-0.1) | JODIE ROC-AUC: 0.5229 (>=0.75) | JODIE shuffle delta: -0.0293 (<=-0.1) |
| medium | 1 | 0 | - | MotifProbe ROC-AUC: 0.6303 (>=0.99) | MotifProbe pair-sep: 0.2606 (>=0.99) | RawMotifProbe ROC-AUC: 0.6470 (>=0.95) | RawMotifProbe pair-sep: 0.3909 (>=0.9) | TGN ROC-AUC: 0.5228 (>=0.75) | TGN shuffle delta: -0.0197 (<=-0.1) | TGAT ROC-AUC: 0.5341 (>=0.75) | TGAT shuffle delta: -0.0414 (<=-0.1) | DyRep ROC-AUC: 0.5343 (>=0.75) | DyRep shuffle delta: -0.0539 (<=-0.1) | JODIE ROC-AUC: 0.5375 (>=0.75) | JODIE shuffle delta: -0.0232 (<=-0.1) |
| medium | 2 | 0 | - | MotifProbe ROC-AUC: 0.6423 (>=0.99) | MotifProbe pair-sep: 0.2846 (>=0.99) | RawMotifProbe ROC-AUC: 0.6578 (>=0.95) | RawMotifProbe pair-sep: 0.4114 (>=0.9) | TGN ROC-AUC: 0.5101 (>=0.75) | TGN shuffle delta: -0.0055 (<=-0.1) | TGAT ROC-AUC: 0.5268 (>=0.75) | TGAT shuffle delta: -0.0349 (<=-0.1) | DyRep ROC-AUC: 0.5555 (>=0.75) | DyRep shuffle delta: -0.0263 (<=-0.1) | JODIE ROC-AUC: 0.5293 (>=0.75) | JODIE shuffle delta: -0.0392 (<=-0.1) |
| medium | 3 | 0 | - | MotifProbe ROC-AUC: 0.6314 (>=0.99) | MotifProbe pair-sep: 0.2629 (>=0.99) | RawMotifProbe ROC-AUC: 0.6422 (>=0.95) | RawMotifProbe pair-sep: 0.3806 (>=0.9) | TGN ROC-AUC: 0.5241 (>=0.75) | TGN shuffle delta: -0.0116 (<=-0.1) | TGAT ROC-AUC: 0.5299 (>=0.75) | TGAT shuffle delta: -0.0618 (<=-0.1) | DyRep ROC-AUC: 0.5460 (>=0.75) | DyRep shuffle delta: -0.0360 (<=-0.1) | JODIE ROC-AUC: 0.5261 (>=0.75) | JODIE shuffle delta: -0.0314 (<=-0.1) |
| medium | 4 | 0 | - | MotifProbe ROC-AUC: 0.6366 (>=0.99) | MotifProbe pair-sep: 0.2731 (>=0.99) | RawMotifProbe ROC-AUC: 0.6467 (>=0.95) | RawMotifProbe pair-sep: 0.4023 (>=0.9) | TGN ROC-AUC: 0.5121 (>=0.75) | TGN shuffle delta: -0.0013 (<=-0.1) | TGAT ROC-AUC: 0.5067 (>=0.75) | TGAT shuffle delta: -0.0239 (<=-0.1) | DyRep ROC-AUC: 0.5105 (>=0.75) | DyRep shuffle delta: -0.0041 (<=-0.1) | JODIE ROC-AUC: 0.5331 (>=0.75) | JODIE shuffle delta: -0.0263 (<=-0.1) |
| oracle_calib | 0 | 1 | nan | nan | TGN ROC-AUC: 0.6135 (>=0.75) | TGAT ROC-AUC: 0.6152 (>=0.75) | TGAT shuffle delta: -0.0937 (<=-0.1) | DyRep ROC-AUC: 0.5936 (>=0.75) | DyRep shuffle delta: -0.0774 (<=-0.1) | JODIE ROC-AUC: 0.5632 (>=0.75) | JODIE shuffle delta: -0.0554 (<=-0.1) |
| oracle_calib | 1 | 1 | - | - | TGN ROC-AUC: 0.6250 (>=0.75) | TGAT ROC-AUC: 0.5882 (>=0.75) | TGAT shuffle delta: -0.0837 (<=-0.1) | DyRep ROC-AUC: 0.6369 (>=0.75) | JODIE ROC-AUC: 0.6069 (>=0.75) |
| oracle_calib | 2 | 1 | - | - | TGN ROC-AUC: 0.6353 (>=0.75) | TGAT ROC-AUC: 0.7061 (>=0.75) | DyRep ROC-AUC: 0.6658 (>=0.75) | JODIE ROC-AUC: 0.6025 (>=0.75) |
| oracle_calib | 3 | 1 | - | - | TGN ROC-AUC: 0.6223 (>=0.75) | TGAT ROC-AUC: 0.6055 (>=0.75) | DyRep ROC-AUC: 0.5408 (>=0.75) | DyRep shuffle delta: -0.0406 (<=-0.1) | JODIE ROC-AUC: 0.5901 (>=0.75) | JODIE shuffle delta: -0.0993 (<=-0.1) |
| oracle_calib | 4 | 1 | - | - | TGN ROC-AUC: 0.6391 (>=0.75) | TGAT ROC-AUC: 0.6486 (>=0.75) | DyRep ROC-AUC: 0.6303 (>=0.75) | JODIE ROC-AUC: 0.5890 (>=0.75) | JODIE shuffle delta: -0.0988 (<=-0.1) |