Initial release: attack_phase 7-class baseline + 6-oracle-path leakage diagnostic + missing tier note

README.md

@@ -0,0 +1,488 @@
+---
+license: cc-by-nc-4.0
+library_name: pytorch
+tags:
+  - cybersecurity
+  - adversarial-machine-learning
+  - ai-security
+  - adversarial-attacks
+  - evasion-attacks
+  - apt
+  - tabular-classification
+  - synthetic-data
+  - xgboost
+  - baseline
+  - leakage-diagnostic
+pipeline_tag: tabular-classification
+base_model: []
+datasets:
+  - xpertsystems/cyb011-sample
+metrics:
+  - accuracy
+  - f1
+  - roc_auc
+model-index:
+  - name: cyb011-baseline-classifier
+    results:
+      - task:
+          type: tabular-classification
+          name: 7-class adversarial attack phase classification
+        dataset:
+          type: xpertsystems/cyb011-sample
+          name: CYB011 Synthetic AI Evasion Attack Trajectory Dataset (Sample)
+        metrics:
+          - type: roc_auc
+            value: 0.9753
+            name: Test macro ROC-AUC OvR (XGBoost, seed 42)
+          - type: accuracy
+            value: 0.8643
+            name: Test accuracy (XGBoost, seed 42)
+          - type: f1
+            value: 0.7693
+            name: Test macro-F1 (XGBoost, seed 42)
+          - type: accuracy
+            value: 0.867
+            name: Multi-seed accuracy mean ± 0.010 (XGBoost, 10 seeds)
+          - type: roc_auc
+            value: 0.977
+            name: Multi-seed ROC-AUC mean ± 0.002 (XGBoost, 10 seeds)
+---
+
+# CYB011 Baseline Classifier
+
+**Adversarial attack phase classifier (7-class) trained on the CYB011
+synthetic AI evasion attack trajectory sample. Predicts which of 7
+attack phases (`reconnaissance` / `feature_space_probe` /
+`perturbation_craft` / `evasion_attempt` / `feedback_adaptation` /
+`campaign_consolidation` / `idle_dwell`) a per-timestep trajectory
+event belongs to, from per-event features. ALSO ships a comprehensive
+`leakage_diagnostic.json` documenting 6 oracle paths discovered
+across the dataset's targets, 4 README-suggested targets that are
+unlearnable on the sample after honest leak removal, and the missing
+`nation_state` attacker tier.**
+
+> **Read this first.** This repo ships two related artifacts:
+> (1) a working baseline classifier for `attack_phase` (the dataset's
+> headline target), and (2) `leakage_diagnostic.json` documenting 6
+> separate oracle paths, 4 unlearnable targets, and one missing
+> attacker tier. Both files matter; the diagnostic is required reading
+> for anyone evaluating CYB011 for adversarial ML research.
+
+## Model overview
+
+| Property | Value |
+|---|---|
+| Primary task | 7-class `attack_phase` classification |
+| Secondary artifact | `leakage_diagnostic.json` — 6 oracle paths + 4 unlearnable targets |
+| Training data | `xpertsystems/cyb011-sample` (14,000 events / 200 campaigns) |
+| Models | XGBoost + PyTorch MLP |
+| Input features | 37 (after one-hot encoding) |
+| Split | **Group-aware** (GroupShuffleSplit on `campaign_id`) |
+| Validation | Single seed (artifact) + multi-seed aggregate across 10 seeds |
+| License | CC-BY-NC-4.0 (matches dataset) |
+| Status | Reference baseline + comprehensive leakage diagnostic |
+
+## Why this task — and what was dropped
+
+The CYB011 README describes a "6-phase adversarial state machine."
+The actual sample data contains **7 phases** — it adds `idle_dwell`
+as a class (18% of all events, the second-largest class). The
+published baseline trains on all 7.
+
+We piloted nine candidate targets and found:
+
+- **`attack_phase` 7-class**: strongest honest result. Acc 0.867 ±
+  0.010, ROC-AUC 0.977 ± 0.002 (multi-seed). All 7 classes
+  represented, per-class F1 range 0.49–1.00.
+
+- **`attacker_capability_tier` 3-class (per-timestep)**: weak honest
+  result (acc 0.68, mF1 0.64). The 3 tiers do not strongly
+  distinguish each other at the per-timestep level — feature means
+  are within ~1% across tiers.
+
+- **`attacker_capability_tier` 3-class (per-campaign)**: hits acc 0.94
+  but is structurally inflated by `stealth_score` leakage
+  (near-deterministic ranges per tier). Documented in the diagnostic.
+
+- **`detection_outcome` 4-class**: hits 100% trivially via
+  `detector_confidence_score` thresholds. Pure oracle.
+
+- **`defender_architecture` 8-class**: hits 100% trivially via the
+  topology fingerprint (7 segment features uniquely identify each
+  architecture). Collapses to acc 0.13 vs majority 0.17 when the
+  fingerprint is dropped.
+
+- **`campaign_success_flag` / `campaign_type` / `coordinated_attack_flag`**:
+  all below majority baseline at n=200 campaigns.
+
+### Three oracle columns dropped from features
+
+The phase task has three direct outcome-leak columns. Each is a perfect
+or near-perfect oracle for specific phases:
+
+| Column | Oracle relationship |
+|---|---|
+| `detection_outcome` | `!= suppressed_alert` → 100% `evasion_attempt` phase |
+| `detector_confidence_score` | Threshold-derived from `detection_outcome` (<0.25 → evasion_success, [0.52,0.78] → marginal, ≥0.78 → high_confidence) |
+| `evasion_budget_consumed` | `== 0` → 100% one of 3 early phases (reconnaissance, feature_space_probe, perturbation_craft) |
+
+With these three columns present, a plain XGBoost achieves 100%
+accuracy. The published baseline trains with all three excluded.
+
+### `timestep` kept as a legitimate observable
+
+`timestep` is a partial oracle for 3 phases (reconnaissance is
+always timestep 1-7, feedback_adaptation is 63-66, campaign_consolidation
+is 65-70). It's **kept** in the feature set because campaign-progress
+position is a real observable a defender would have at decision time
+— it's not encoding the label, it's encoding the lifecycle position.
+
+Removing `timestep` drops headline accuracy by ~9pp (0.87 → 0.78).
+Documented in the diagnostic for transparency.
+
+Two model artifacts are published. They are designed to be used
+together:
+
+- `model_xgb.json` — gradient-boosted trees (higher F1)
+- `model_mlp.safetensors` — PyTorch MLP
+
+## Quick start
+
+```bash
+pip install xgboost torch safetensors pandas huggingface_hub
+```
+
+```python
+from huggingface_hub import hf_hub_download, snapshot_download
+import json, numpy as np, torch, xgboost as xgb
+from safetensors.torch import load_file
+
+REPO = "xpertsystems/cyb011-baseline-classifier"
+
+paths = {n: hf_hub_download(REPO, n) for n in [
+    "model_xgb.json", "model_mlp.safetensors",
+    "feature_engineering.py", "feature_meta.json", "feature_scaler.json",
+]}
+
+import sys, os
+sys.path.insert(0, os.path.dirname(paths["feature_engineering.py"]))
+from feature_engineering import (
+    transform_single, load_meta, build_segment_lookup, INT_TO_LABEL,
+)
+
+meta = load_meta(paths["feature_meta.json"])
+
+# Segment features are joined from network_topology.csv at inference time
+ds = snapshot_download("xpertsystems/cyb011-sample", repo_type="dataset")
+segment_lookup = build_segment_lookup(f"{ds}/network_topology.csv")
+
+xgb_model = xgb.XGBClassifier(); xgb_model.load_model(paths["model_xgb.json"])
+
+# Predict (see inference_example.ipynb for the full pattern)
+# Note: do NOT include detection_outcome, detector_confidence_score,
+# or evasion_budget_consumed — those were the outcome leak columns.
+X = transform_single(my_event, meta, segment_lookup=segment_lookup)
+proba = xgb_model.predict_proba(X)[0]
+print(INT_TO_LABEL[int(np.argmax(proba))])
+```
+
+See [`inference_example.ipynb`](./inference_example.ipynb) for the full
+copy-paste demo.
+
+## Training data
+
+Trained on the public sample of CYB011, 14,000 per-timestep records:
+
+| Phase | Events | Class share |
+|---|---:|---:|
+| `evasion_attempt` | 7,206 | 51.5% |
+| `idle_dwell` | 2,450 | 17.5% |
+| `feature_space_probe` | 1,465 | 10.5% |
+| `campaign_consolidation` | 829 | 5.9% |
+| `reconnaissance` | 809 | 5.8% |
+| `perturbation_craft` | 745 | 5.3% |
+| `feedback_adaptation` | 496 | 3.5% |
+
+### Group-aware split by campaign_id
+
+200 campaigns × 70 timesteps each. Timesteps from the same campaign
+share attacker, target segment, and tier — so train/test contamination
+is a real risk with random splitting. The baseline uses
+**GroupShuffleSplit** on `campaign_id` (nested 70/15/15):
+
+| Fold | Events | Campaigns |
+|---|---:|---:|
+| Train | 9,730 | ~140 |
+| Validation | 2,170 | ~30 |
+| Test | 2,100 | ~30 |
+
+All 10 multi-seed evaluations yielded all 7 classes in the test fold.
+Class imbalance is addressed with `class_weight='balanced'` (XGBoost
+`sample_weight`) and weighted cross-entropy (MLP).
+
+## Feature pipeline
+
+The bundled `feature_engineering.py` is the canonical recipe. 37
+features survive after encoding, drawn from:
+
+- **Per-timestep numeric** (5): `timestep`, `perturbation_magnitude`,
+  `feature_delta_l2_norm`, `feature_delta_linf_norm`, `query_count_cumulative`
+- **Per-timestep categorical** (1, one-hot): `attacker_capability_tier`
+  (3 values in sample)
+- **Segment features** (joined from `network_topology.csv`): 8 numeric
+  + 2 categorical (segment_type, defender_architecture)
+- **Engineered** (5): `progress_frac`, `log_queries`, `perturb_intensity`,
+  `defender_weakness`, `query_rate`
+
+## Evaluation
+
+### Test-set metrics, seed 42 (n = 2,100 events from ~30 test campaigns)
+
+**XGBoost** (the published `model_xgb.json` artifact)
+
+| Metric | Value |
+|---|---:|
+| Macro ROC-AUC (OvR) | **0.9753** |
+| Accuracy | **0.8643** |
+| Macro-F1 | 0.7693 |
+| Weighted-F1 | 0.8703 |
+
+**MLP** (the published `model_mlp.safetensors` artifact)
+
+| Metric | Value |
+|---|---:|
+| Macro ROC-AUC (OvR) | **0.9705** |
+| Accuracy | **0.8386** |
+| Macro-F1 | 0.7345 |
+| Weighted-F1 | 0.8462 |
+
+XGBoost slightly outperforms MLP (acc 0.864 vs 0.839, macro-F1 0.769
+vs 0.735). The gap is consistent across seeds.
+
+### Multi-seed robustness (XGBoost, 10 seeds)
+
+| Metric | Mean | Std | Min | Max |
+|---|---:|---:|---:|---:|
+| Accuracy | 0.867 | 0.010 | 0.852 | 0.884 |
+| Macro-F1 | 0.775 | 0.012 | 0.750 | 0.798 |
+| Macro ROC-AUC OvR | 0.977 | 0.002 | 0.973 | 0.980 |
+
+All 10 seeds yielded all 7 classes in the test fold. Full per-seed
+results in [`multi_seed_results.json`](./multi_seed_results.json).
+
+### Per-class F1 (seed 42)
+
+| Phase | Class share | XGBoost F1 | MLP F1 |
+|---|---:|---:|---:|
+| `evasion_attempt` | 51.5% | **0.996** | 0.993 |
+| `reconnaissance` | 5.8% | **0.886** | 0.874 |
+| `campaign_consolidation` | 5.9% | 0.808 | 0.785 |
+| `feature_space_probe` | 10.5% | 0.783 | 0.747 |
+| `feedback_adaptation` | 3.5% | 0.715 | 0.628 |
+| `idle_dwell` | 17.5% | 0.704 | 0.619 |
+| `perturbation_craft` | 5.3% | **0.493** | 0.497 |
+
+`evasion_attempt` is nearly perfectly separable because of its
+distinctive query-usage and perturbation-activity signatures.
+`reconnaissance` and `campaign_consolidation` are well-separated by
+their characteristic timestep ranges. `perturbation_craft` is the
+hardest class (F1 0.49) because its per-timestep features overlap
+heavily with `feature_space_probe` — both involve probing model
+behavior at moderate query counts without submitting a final evasion
+attempt.
+
+### Ablation: which feature groups matter
+
+| Configuration | Accuracy | Macro-F1 | ROC-AUC | Δ accuracy | Δ macro-F1 |
+|---|---:|---:|---:|---:|---:|
+| Full feature set (published) | 0.8643 | 0.7693 | 0.9753 | — | — |
+| No perturbation features | 0.6595 | 0.6451 | 0.8979 | **−0.205** | **−0.124** |
+| No query features | 0.8210 | 0.7080 | 0.9669 | −0.043 | −0.061 |
+| No engineered features | 0.8590 | 0.7619 | 0.9751 | −0.005 | −0.007 |
+| No tier (one-hot) | 0.8614 | 0.7647 | 0.9752 | −0.003 | −0.005 |
+| No timestep | 0.8557 | 0.7549 | 0.9696 | −0.009 | −0.014 |
+| No topology features | 0.8648 | 0.7745 | 0.9760 | +0.001 | +0.005 |
+
+Three findings:
+
+1. **Perturbation features carry the dominant signal** (−20pp accuracy,
+   −12pp F1 when removed). `feature_delta_l2_norm`,
+   `feature_delta_linf_norm`, and `perturbation_magnitude` directly
+   encode whether the attacker is actively perturbing inputs.
+2. **Query features are second-strongest** (−4pp accuracy, −6pp F1).
+   Cumulative query count distinguishes active phases (evasion_attempt,
+   probe) from idle phases.
+3. **Topology features contribute nothing on this task** (+0.1pp
+   accuracy when removed). Clean confirmation that the topology
+   fingerprint isn't leaking phase information — topology
+   fingerprints defender_architecture, not attack_phase.
+
+### Architecture
+
+**XGBoost:** multi-class gradient boosting (`multi:softprob`, 7 classes),
+`hist` tree method, class-balanced sample weights, early stopping on
+validation mlogloss.
+
+**MLP:** `37 → 128 → 64 → 7`, each hidden layer followed by `BatchNorm1d`
+→ `ReLU` → `Dropout(0.3)`, weighted cross-entropy loss, AdamW optimizer,
+early stopping on validation macro-F1.
+
+Training hyperparameters are held internally by XpertSystems.
+
+## Limitations
+
+**This is a baseline reference, not a production phase classifier.**
+
+1. **The leakage diagnostic is required reading.** Three direct
+   oracle columns for the phase task plus three additional documented
+   leaks (timestep partial, stealth_score per-tier, topology
+   fingerprint) are in `leakage_diagnostic.json`. If you use CYB011
+   sample data for your own training, you MUST drop the three direct
+   oracles or your model will learn the oracles instead of the task.
+
+2. **`perturbation_craft` F1 0.49 is the weakest class.** This phase's
+   per-timestep features overlap heavily with `feature_space_probe`.
+   A sequence model considering event ordering within campaigns would
+   likely do better than per-timestep classification.
+
+3. **`nation_state` attacker tier is MISSING from the sample.** The
+   README claims 4 tiers (script_kiddie, opportunistic, APT,
+   nation_state). The sample contains only 3 — nation_state events
+   are entirely absent. Models trained on this sample cannot
+   generalize to nation_state actors.
+
+4. **Four README-suggested headline targets are unlearnable on the
+   sample** after honest leak removal: `campaign_success_flag` (acc
+   0.51 vs majority 0.61), `campaign_type` 8-class (acc 0.11 vs 0.17),
+   `coordinated_attack_flag` (acc 0.83 vs 0.90 — only 20 positives in
+   200 campaigns), and `defender_architecture` 8-class (collapses to
+   acc 0.13 when the 7-feature topology fingerprint is dropped).
+
+5. **Per-campaign tasks are structurally limited at n=200.** With ~30
+   test campaigns per fold, statistical power is limited. The full
+   ~5,500-campaign product would yield much tighter per-campaign
+   metrics.
+
+6. **Synthetic-vs-real transfer.** The dataset is synthetic, calibrated
+   to 12 benchmarks from MITRE ATLAS / NIST AI 100-2 / OWASP ML Top 10
+   / USENIX / IBM ART / Anthropic-OpenAI red team reports. Real
+   adversarial ML telemetry has different noise characteristics, and
+   in particular the threshold-encoded `detector_confidence_score`
+   and zero-sentinel `evasion_budget_consumed` patterns documented in
+   the diagnostic would not be present in real data. Real telemetry
+   has continuous, overlapping distributions.
+
+## Notes on dataset schema
+
+The CYB011 sample dataset README describes some fields differently
+from the actual schema. The model was trained on the actual schema;
+this note helps buyers reconcile what they read with what they receive.
+
+| What the README says | What the data actually contains |
+|---|---|
+| `attack_trajectories` has 18 columns | Data has **13 columns** |
+| Field renames | `adversarial_phase` → `attack_phase`, `attacker_tier` → `attacker_capability_tier`, `perturbation_linf` → `feature_delta_linf_norm`, `perturbation_l2` → `feature_delta_l2_norm`, `queries_used` → `query_count_cumulative` |
+| README missing from `attack_trajectories` | `detector_confidence_score`, `detection_outcome`, `evasion_budget_consumed` are in data but not documented |
+| README claims `gradient_access`, `evasion_attempted`, `evasion_succeeded`, `query_budget_remaining`, `defender_detection_strength`, `concept_drift_injected`, `transfer_attack_used`, `stealth_score`, `feature_space_dim` | None of these columns exist in `attack_trajectories`. `defender_detection_strength`, `feature_space_dim`, and `stealth_score` exist in `network_topology` or `campaign_summary` respectively, not in `attack_trajectories` |
+| `attacker_capability_tier` has 4 values | Data has **3 values** — `nation_state` MISSING entirely |
+| `attack_phase` 6-phase lifecycle | Data has **7 phases** — adds `idle_dwell` (18% of events) |
+| `campaign_summary` has 14 columns | Data has **25 columns** |
+| README documents no schema for `network_topology` | Data has **12 columns** |
+
+None of these affects model correctness — the feature pipeline uses
+the actual column names. If you build your own pipeline against the
+dataset, use the actual columns.
+
+## Intended use
+
+- **Evaluating fit** of the CYB011 dataset for your adversarial ML
+  research
+- **Baseline reference** for new model architectures on the attack-
+  phase classification task
+- **Reference example of structural-leakage diagnostics** for
+  synthetic adversarial ML datasets — the methodology is reusable
+- **Feature engineering reference** for per-timestep adversarial
+  trajectory telemetry
+
+## Out-of-scope use
+
+- Production adversarial detection on real ML systems
+- Attacker tier attribution (3-class per-timestep is weak; per-campaign
+  is leaky via stealth_score)
+- Defender architecture vulnerability assessment (trivially leaky on
+  this sample; collapses when topology fingerprint is dropped)
+- Campaign success prediction (unlearnable on sample)
+- Any nation_state-specific modeling (tier absent from sample)
+- Any operational AI security decision without further validation on
+  real adversarial telemetry
+
+## Reproducibility
+
+Outputs above were produced with `seed = 42` (published artifact),
+nested `GroupShuffleSplit` on `campaign_id` (70/15/15), on the
+published sample (`xpertsystems/cyb011-sample`, version 1.0.0,
+generated 2026-05-16). The feature pipeline in `feature_engineering.py`
+is deterministic and the trained weights in this repo correspond
+exactly to the metrics above.
+
+Multi-seed results (seeds 42, 7, 13, 17, 23, 31, 45, 99, 123, 200)
+in `multi_seed_results.json` confirm robust performance across splits
+(std 0.010 on accuracy, 0.002 on ROC-AUC).
+
+The training script itself is private to XpertSystems.
+
+## Files in this repo
+
+| File | Purpose |
+|---|---|
+| `model_xgb.json` | XGBoost weights (seed 42) |
+| `model_mlp.safetensors` | PyTorch MLP weights (seed 42) |
+| `feature_engineering.py` | Feature pipeline |
+| `feature_meta.json` | Feature column order + categorical levels |
+| `feature_scaler.json` | MLP input mean/std (XGBoost ignores) |
+| `validation_results.json` | Per-class metrics, confusion matrix, architecture |
+| `ablation_results.json` | Per-feature-group ablation |
+| `multi_seed_results.json` | XGBoost metrics across 10 seeds |
+| **`leakage_diagnostic.json`** | **6-oracle-path audit + 4 unlearnable targets + missing tier note** |
+| `inference_example.ipynb` | End-to-end inference demo notebook |
+| `README.md` | This file |
+
+## Contact and full product
+
+The full **CYB011** dataset contains **~383,000 rows** across four files,
+with calibrated benchmark validation against 12 metrics drawn from
+authoritative adversarial ML research (MITRE ATLAS, NIST AI 100-2
+Adversarial ML Taxonomy, OWASP ML Top 10, USENIX Security adversarial
+ML papers, IEEE SaTML, Microsoft Counterfit, IBM Adversarial Robustness
+Toolbox, Anthropic / OpenAI red team reports).
+
+The full XpertSystems.ai synthetic data catalogue spans 41 SKUs across
+Cybersecurity, Healthcare, Insurance & Risk, Oil & Gas, and Materials
+& Energy.
+
+- 📧 **pradeep@xpertsystems.ai**
+- 🌐 **https://xpertsystems.ai**
+- 🗂  Dataset: https://huggingface.co/datasets/xpertsystems/cyb011-sample
+- 🤖 Companion models:
+  - https://huggingface.co/xpertsystems/cyb001-baseline-classifier (network traffic)
+  - https://huggingface.co/xpertsystems/cyb002-baseline-classifier (ATT&CK kill-chain)
+  - https://huggingface.co/xpertsystems/cyb003-baseline-classifier (malware execution phase)
+  - https://huggingface.co/xpertsystems/cyb004-baseline-classifier (phishing campaign phase)
+  - https://huggingface.co/xpertsystems/cyb005-baseline-classifier (ransomware actor-tier attribution)
+  - https://huggingface.co/xpertsystems/cyb006-baseline-classifier (user risk tier + leakage diagnostic)
+  - https://huggingface.co/xpertsystems/cyb007-baseline-classifier (insider threat type)
+  - https://huggingface.co/xpertsystems/cyb008-baseline-classifier (SOC alert triage + leakage diagnostic)
+  - https://huggingface.co/xpertsystems/cyb009-baseline-classifier (vulnerability classification + leakage diagnostic)
+  - https://huggingface.co/xpertsystems/cyb010-baseline-classifier (attack lifecycle phase + leakage diagnostic)
+
+## Citation
+
+```bibtex
+@misc{xpertsystems_cyb011_baseline_2026,
+  title  = {CYB011 Baseline Classifier: XGBoost and MLP for Adversarial Attack Phase Classification, with 6-Oracle-Path Leakage Diagnostic},
+  author = {XpertSystems.ai},
+  year   = {2026},
+  url    = {https://huggingface.co/xpertsystems/cyb011-baseline-classifier},
+  note   = {Baseline reference model + leakage audit trained on xpertsystems/cyb011-sample}
+}
+```

ablation_results.json

@@ -0,0 +1,685 @@
+{
+  "purpose": "Quantify how much each feature group contributes to the headline XGBoost score. Identical architecture, same group-aware split, with one feature group dropped at a time.",
+  "full_model_metrics": {
+    "model": "xgboost",
+    "accuracy": 0.8642857142857143,
+    "macro_f1": 0.7693247628697397,
+    "weighted_f1": 0.8650489644308249,
+    "per_class_f1": {
+      "reconnaissance": 0.8865248226950354,
+      "feature_space_probe": 0.7829977628635347,
+      "perturbation_craft": 0.4927536231884058,
+      "evasion_attempt": 0.9962013295346629,
+      "feedback_adaptation": 0.7151515151515152,
+      "campaign_consolidation": 0.8075471698113208,
+      "idle_dwell": 0.7040971168437026
+    },
+    "confusion_matrix": {
+      "labels": [
+        "reconnaissance",
+        "feature_space_probe",
+        "perturbation_craft",
+        "evasion_attempt",
+        "feedback_adaptation",
+        "campaign_consolidation",
+        "idle_dwell"
+      ],
+      "matrix": [
+        [
+          125,
+          0,
+          0,
+          0,
+          0,
+          0,
+          3
+        ],
+        [
+          0,
+          175,
+          43,
+          0,
+          0,
+          0,
+          2
+        ],
+        [
+          0,
+          20,
+          68,
+          0,
+          0,
+          0,
+          27
+        ],
+        [
+          0,
+          0,
+          2,
+          1049,
+          0,
+          0,
+          6
+        ],
+        [
+          0,
+          0,
+          0,
+          0,
+          59,
+          16,
+          1
+        ],
+        [
+          0,
+          0,
+          0,
+          0,
+          9,
+          107,
+          0
+        ],
+        [
+          29,
+          32,
+          48,
+          0,
+          21,
+          26,
+          232
+        ]
+      ]
+    },
+    "macro_roc_auc_ovr": 0.9752868672798508
+  },
+  "ablations": {
+    "no_timestep": {
+      "n_features": 35,
+      "dropped_count": 2,
+      "metrics": {
+        "model": "xgboost_no_timestep",
+        "accuracy": 0.8557142857142858,
+        "macro_f1": 0.7549062338242875,
+        "weighted_f1": 0.8554198045390304,
+        "per_class_f1": {
+          "reconnaissance": 0.8833922261484098,
+          "feature_space_probe": 0.7652173913043478,
+          "perturbation_craft": 0.51985559566787,
+          "evasion_attempt": 0.9952516619183286,
+          "feedback_adaptation": 0.6380368098159509,
+          "campaign_consolidation": 0.8108108108108109,
+          "idle_dwell": 0.6717791411042945
+        },
+        "confusion_matrix": {
+          "labels": [
+            "reconnaissance",
+            "feature_space_probe",
+            "perturbation_craft",
+            "evasion_attempt",
+            "feedback_adaptation",
+            "campaign_consolidation",
+            "idle_dwell"
+          ],
+          "matrix": [
+            [
+              125,
+              0,
+              0,
+              0,
+              0,
+              0,
+              3
+            ],
+            [
+              0,
+              176,
+              40,
+              0,
+              0,
+              0,
+              4
+            ],
+            [
+              0,
+              25,
+              72,
+              0,
+              0,
+              0,
+              18
+            ],
+            [
+              0,
+              0,
+              2,
+              1048,
+              1,
+              0,
+              6
+            ],
+            [
+              0,
+              0,
+              0,
+              0,
+              52,
+              17,
+              7
+            ],
+            [
+              0,
+              0,
+              0,
+              0,
+              4,
+              105,
+              7
+            ],
+            [
+              30,
+              39,
+              48,
+              1,
+              30,
+              21,
+              219
+            ]
+          ]
+        },
+        "macro_roc_auc_ovr": 0.9695634570525955
+      },
+      "delta_accuracy": 0.008571428571428563,
+      "delta_macro_f1": 0.014418529045452266
+    },
+    "no_perturb": {
+      "n_features": 33,
+      "dropped_count": 4,
+      "metrics": {
+        "model": "xgboost_no_perturb",
+        "accuracy": 0.6595238095238095,
+        "macro_f1": 0.6450937004078117,
+        "weighted_f1": 0.6477912364682181,
+        "per_class_f1": {
+          "reconnaissance": 0.8896797153024911,
+          "feature_space_probe": 0.7264957264957265,
+          "perturbation_craft": 0.4034090909090909,
+          "evasion_attempt": 0.7787005373717636,
+          "feedback_adaptation": 0.7317073170731707,
+          "campaign_consolidation": 0.8120300751879699,
+          "idle_dwell": 0.17363344051446947
+        },
+        "confusion_matrix": {
+          "labels": [
+            "reconnaissance",
+            "feature_space_probe",
+            "perturbation_craft",
+            "evasion_attempt",
+            "feedback_adaptation",
+            "campaign_consolidation",
+            "idle_dwell"
+          ],
+          "matrix": [
+            [
+              125,
+              0,
+              0,
+              0,
+              0,
+              0,
+              3
+            ],
+            [
+              0,
+              170,
+              47,
+              2,
+              0,
+              0,
+              1
+            ],
+            [
+              0,
+              20,
+              71,
+              9,
+              0,
+              0,
+              15
+            ],
+            [
+              0,
+              25,
+              76,
+              797,
+              0,
+              0,
+              159
+            ],
+            [
+              0,
+              0,
+              0,
+              0,
+              60,
+              15,
+              1
+            ],
+            [
+              0,
+              0,
+              0,
+              0,
+              7,
+              108,
+              1
+            ],
+            [
+              28,
+              33,
+              43,
+              182,
+              21,
+              27,
+              54
+            ]
+          ]
+        },
+        "macro_roc_auc_ovr": 0.89792342601005
+      },
+      "delta_accuracy": 0.2047619047619048,
+      "delta_macro_f1": 0.12423106246192805
+    },
+    "no_queries": {
+      "n_features": 34,
+      "dropped_count": 3,
+      "metrics": {
+        "model": "xgboost_no_queries",
+        "accuracy": 0.820952380952381,
+        "macro_f1": 0.7079823380902172,
+        "weighted_f1": 0.824790421215039,
+        "per_class_f1": {
+          "reconnaissance": 0.7986111111111112,
+          "feature_space_probe": 0.5427872860635696,
+          "perturbation_craft": 0.42073170731707316,
+          "evasion_attempt": 0.9952471482889734,
+          "feedback_adaptation": 0.7209302325581395,
+          "campaign_consolidation": 0.8015564202334631,
+          "idle_dwell": 0.67601246105919
+        },
+        "confusion_matrix": {
+          "labels": [
+            "reconnaissance",
+            "feature_space_probe",
+            "perturbation_craft",
+            "evasion_attempt",
+            "feedback_adaptation",
+            "campaign_consolidation",
+            "idle_dwell"
+          ],
+          "matrix": [
+            [
+              115,
+              13,
+              0,
+              0,
+              0,
+              0,
+              0
+            ],
+            [
+              20,
+              111,
+              84,
+              0,
+              0,
+              0,
+              5
+            ],
+            [
+              0,
+              25,
+              69,
+              0,
+              0,
+              0,
+              21
+            ],
+            [
+              0,
+              0,
+              2,
+              1047,
+              0,
+              0,
+              8
+            ],
+            [
+              0,
+              0,
+              0,
+              0,
+              62,
+              13,
+              1
+            ],
+            [
+              0,
+              0,
+              0,
+              0,
+              11,
+              103,
+              2
+            ],
+            [
+              25,
+              40,
+              58,
+              0,
+              23,
+              25,
+              217
+            ]
+          ]
+        },
+        "macro_roc_auc_ovr": 0.9668743863750572
+      },
+      "delta_accuracy": 0.043333333333333335,
+      "delta_macro_f1": 0.061342424779522564
+    },
+    "no_topology": {
+      "n_features": 12,
+      "dropped_count": 25,
+      "metrics": {
+        "model": "xgboost_no_topology",
+        "accuracy": 0.8647619047619047,
+        "macro_f1": 0.7744509705042503,
+        "weighted_f1": 0.8651794157598562,
+        "per_class_f1": {
+          "reconnaissance": 0.9014084507042254,
+          "feature_space_probe": 0.7668161434977578,
+          "perturbation_craft": 0.519298245614035,
+          "evasion_attempt": 0.9952561669829222,
+          "feedback_adaptation": 0.7218934911242604,
+          "campaign_consolidation": 0.816793893129771,
+          "idle_dwell": 0.6996904024767802
+        },
+        "confusion_matrix": {
+          "labels": [
+            "reconnaissance",
+            "feature_space_probe",
+            "perturbation_craft",
+            "evasion_attempt",
+            "feedback_adaptation",
+            "campaign_consolidation",
+            "idle_dwell"
+          ],
+          "matrix": [
+            [
+              128,
+              0,
+              0,
+              0,
+              0,
+              0,
+              0
+            ],
+            [
+              0,
+              171,
+              47,
+              0,
+              0,
+              0,
+              2
+            ],
+            [
+              0,
+              18,
+              74,
+              0,
+              0,
+              0,
+              23
+            ],
+            [
+              0,
+              0,
+              1,
+              1049,
+              0,
+              0,
+              7
+            ],
+            [
+              0,
+              0,
+              0,
+              0,
+              61,
+              15,
+              0
+            ],
+            [
+              0,
+              0,
+              0,
+              0,
+              9,
+              107,
+              0
+            ],
+            [
+              28,
+              37,
+              48,
+              2,
+              23,
+              24,
+              226
+            ]
+          ]
+        },
+        "macro_roc_auc_ovr": 0.9760448304097272
+      },
+      "delta_accuracy": -0.0004761904761904079,
+      "delta_macro_f1": -0.005126207634510549
+    },
+    "no_tier": {
+      "n_features": 34,
+      "dropped_count": 3,
+      "metrics": {
+        "model": "xgboost_no_tier",
+        "accuracy": 0.8614285714285714,
+        "macro_f1": 0.7646643425700288,
+        "weighted_f1": 0.8620313204951823,
+        "per_class_f1": {
+          "reconnaissance": 0.8865248226950354,
+          "feature_space_probe": 0.7671840354767184,
+          "perturbation_craft": 0.48148148148148145,
+          "evasion_attempt": 0.9952471482889734,
+          "feedback_adaptation": 0.7073170731707317,
+          "campaign_consolidation": 0.8120300751879699,
+          "idle_dwell": 0.702865761689291
+        },
+        "confusion_matrix": {
+          "labels": [
+            "reconnaissance",
+            "feature_space_probe",
+            "perturbation_craft",
+            "evasion_attempt",
+            "feedback_adaptation",
+            "campaign_consolidation",
+            "idle_dwell"
+          ],
+          "matrix": [
+            [
+              125,
+              0,
+              0,
+              0,
+              0,
+              0,
+              3
+            ],
+            [
+              0,
+              173,
+              45,
+              0,
+              0,
+              0,
+              2
+            ],
+            [
+              0,
+              21,
+              65,
+              0,
+              0,
+              0,
+              29
+            ],
+            [
+              0,
+              0,
+              3,
+              1047,
+              0,
+              0,
+              7
+            ],
+            [
+              0,
+              0,
+              0,
+              0,
+              58,
+              17,
+              1
+            ],
+            [
+              0,
+              0,
+              0,
+              0,
+              8,
+              108,
+              0
+            ],
+            [
+              29,
+              37,
+              42,
+              0,
+              22,
+              25,
+              233
+            ]
+          ]
+        },
+        "macro_roc_auc_ovr": 0.9752322842014612
+      },
+      "delta_accuracy": 0.0028571428571428914,
+      "delta_macro_f1": 0.004660420299710921
+    },
+    "no_engineered": {
+      "n_features": 32,
+      "dropped_count": 5,
+      "metrics": {
+        "model": "xgboost_no_engineered",
+        "accuracy": 0.8590476190476191,
+        "macro_f1": 0.7619124928932358,
+        "weighted_f1": 0.859520574191734,
+        "per_class_f1": {
+          "reconnaissance": 0.8825622775800712,
+          "feature_space_probe": 0.7682119205298014,
+          "perturbation_craft": 0.4946236559139785,
+          "evasion_attempt": 0.9957285239677266,
+          "feedback_adaptation": 0.703030303030303,
+          "campaign_consolidation": 0.8,
+          "idle_dwell": 0.6892307692307692
+        },
+        "confusion_matrix": {
+          "labels": [
+            "reconnaissance",
+            "feature_space_probe",
+            "perturbation_craft",
+            "evasion_attempt",
+            "feedback_adaptation",
+            "campaign_consolidation",
+            "idle_dwell"
+          ],
+          "matrix": [
+            [
+              124,
+              0,
+              0,
+              0,
+              0,
+              0,
+              4
+            ],
+            [
+              0,
+              174,
+              45,
+              0,
+              0,
+              0,
+              1
+            ],
+            [
+              0,
+              21,
+              69,
+              0,
+              0,
+              0,
+              25
+            ],
+            [
+              0,
+              0,
+              2,
+              1049,
+              0,
+              0,
+              6
+            ],
+            [
+              0,
+              0,
+              0,
+              0,
+              58,
+              17,
+              1
+            ],
+            [
+              0,
+              0,
+              0,
+              0,
+              9,
+              106,
+              1
+            ],
+            [
+              29,
+              38,
+              48,
+              1,
+              22,
+              26,
+              224
+            ]
+          ]
+        },
+        "macro_roc_auc_ovr": 0.9751320314704773
+      },
+      "delta_accuracy": 0.005238095238095264,
+      "delta_macro_f1": 0.007412269976503905
+    }
+  }
+}

feature_engineering.py

@@ -0,0 +1,399 @@
+"""
+feature_engineering.py
+======================
+
+Feature pipeline for the CYB011 baseline classifier.
+
+Predicts `attack_phase` (7-class adversarial attack phase) from
+per-timestep features on the CYB011 sample dataset.
+
+CSV inputs:
+    attack_trajectories.csv  (primary, per-timestep, 14,000 events)
+    network_topology.csv     (per-segment registry, joined for defender
+                              context features)
+    campaign_summary.csv     (per-campaign summaries; reserved)
+    campaign_events.csv      (discrete event log; reserved)
+
+Target classes (7):
+    reconnaissance, feature_space_probe, perturbation_craft,
+    evasion_attempt, feedback_adaptation, campaign_consolidation,
+    idle_dwell
+
+The CYB011 README describes a "6-phase adversarial state machine" but
+the sample data has 7 phases — it adds `idle_dwell` (18% of events,
+the second-largest class).
+
+Group structure
+---------------
+200 campaigns x 70 timesteps = 14,000 events. Each campaign is a
+sequential evasion attempt; events from the same campaign share
+attacker, target segment, and tier. Group-aware splitting by
+`campaign_id` (~30 test campaigns per fold) prevents train/test
+contamination.
+
+Leakage audit
+-------------
+Three columns dropped from features because they're outcome leaks
+for `attack_phase`:
+
+1. `detection_outcome` (4-class categorical):
+   - `evasion_success` / `marginal_alert` / `high_confidence_alert`
+     ALL → 100% `evasion_attempt` phase
+   - `suppressed_alert` → can be any of the 7 phases
+   So detection_outcome != suppressed_alert is a perfect oracle for
+   evasion_attempt.
+
+2. `detector_confidence_score`: deterministically derives detection
+   outcome via threshold boundaries (< 0.25 -> evasion_success,
+   [0.52, 0.78] -> marginal, >= 0.78 -> high_confidence). Same
+   leakage as detection_outcome.
+
+3. `evasion_budget_consumed`: == 0 for 100% of {reconnaissance,
+   feature_space_probe, perturbation_craft} events. > 0 for the
+   other 4 phases. Perfect oracle for the 3 early phases.
+
+KEPT as a legitimate observable:
+
+- `timestep` is the per-event position in the campaign lifecycle.
+  It correlates with phase (reconnaissance is always early,
+  campaign_consolidation is always late) but is NOT a label-encoding
+  oracle — it's a real progress observable that a defender would have
+  at decision time. Adding +9pp accuracy when included is honest signal.
+
+KEPT as a defender-context observable:
+
+- `defender_architecture`, `detection_strength`, `adversarial_robustness`,
+  `ensemble_size`, `alert_threshold`, `detection_coverage`,
+  `feature_space_dim`, `retraining_cadence_days`, `trust_level`: all
+  per-segment topology features. They are deterministic per segment
+  (each topology row uniquely fingerprints its segment), but the
+  segment itself is real context — a defender knows its own
+  architecture. These features are NOT oracles for attack_phase (they
+  predict defender_architecture trivially, but defender_architecture
+  isn't our target).
+
+Public API
+----------
+    build_features(trajectories_path, topology_path)
+        -> (X, y, ids, groups, meta)
+    transform_single(record, meta, segment_lookup=None) -> np.ndarray
+    save_meta(meta, path) / load_meta(path)
+    build_segment_lookup(topology_path) -> dict
+
+License
+-------
+Ships with the public model on Hugging Face under CC-BY-NC-4.0,
+matching the dataset license. See README.md.
+"""
+
+from __future__ import annotations
+
+import json
+from pathlib import Path
+from typing import Any
+
+import numpy as np
+import pandas as pd
+
+# ---------------------------------------------------------------------------
+# Label space
+# ---------------------------------------------------------------------------
+
+# Ordered by attack lifecycle progression.
+LABEL_ORDER = [
+    "reconnaissance",
+    "feature_space_probe",
+    "perturbation_craft",
+    "evasion_attempt",
+    "feedback_adaptation",
+    "campaign_consolidation",
+    "idle_dwell",
+]
+LABEL_TO_INT = {lbl: i for i, lbl in enumerate(LABEL_ORDER)}
+INT_TO_LABEL = {i: lbl for lbl, i in LABEL_TO_INT.items()}
+
+# ---------------------------------------------------------------------------
+# Identifier and target columns
+# ---------------------------------------------------------------------------
+
+ID_COLUMNS = [
+    "campaign_id", "attacker_id",
+    "target_segment_id", "segment_id", "detector_id",
+]
+TARGET_COLUMN = "attack_phase"
+GROUP_COLUMN = "campaign_id"
+
+# Outcome leaks dropped from features.
+ORACLE_COLUMNS = [
+    "detection_outcome",        # !=suppressed -> 100% evasion_attempt
+    "detector_confidence_score",# threshold-derived from detection_outcome
+    "evasion_budget_consumed",  # ==0 -> 100% one of 3 early phases
+]
+
+# ---------------------------------------------------------------------------
+# Per-timestep numeric features
+# ---------------------------------------------------------------------------
+
+EVENT_NUMERIC_FEATURES = [
+    "timestep",                 # kept: legitimate campaign-progress observable
+    "perturbation_magnitude",
+    "feature_delta_l2_norm",
+    "feature_delta_linf_norm",
+    "query_count_cumulative",
+]
+
+EVENT_CATEGORICAL_FEATURES = [
+    "attacker_capability_tier",  # 3 values in sample (script_kiddie, opportunistic, APT)
+]
+
+# ---------------------------------------------------------------------------
+# Segment / topology features (joined on target_segment_id)
+# ---------------------------------------------------------------------------
+
+SEGMENT_NUMERIC_FEATURES = [
+    "trust_level",
+    "detection_coverage",
+    "feature_space_dim",
+    "alert_threshold",
+    "retraining_cadence_days",
+    "ensemble_size",
+    "detection_strength",
+    "adversarial_robustness",
+]
+
+SEGMENT_CATEGORICAL_FEATURES = [
+    "segment_type",           # 8 values
+    "defender_architecture",  # 8 values
+]
+
+
+# ---------------------------------------------------------------------------
+# Engineered features
+# ---------------------------------------------------------------------------
+
+def _add_engineered_features(df: pd.DataFrame) -> pd.DataFrame:
+    """
+    Five engineered features encoding phase-discriminative hypotheses.
+    """
+    df = df.copy()
+
+    # 1. Campaign progress fraction (timestep / 70). Normalizes the
+    #    position-in-lifecycle signal.
+    if "timestep" in df.columns:
+        df["progress_frac"] = (df["timestep"] / 70.0).astype(float)
+    else:
+        df["progress_frac"] = 0.0
+
+    # 2. Log query intensity. Queries are heavy-tailed; some phases
+    #    (reconnaissance, idle_dwell) have ~0 queries while
+    #    evasion_attempt cumulates many.
+    df["log_queries"] = np.log1p(
+        df.get("query_count_cumulative", 0).clip(lower=0)
+    ).astype(float)
+
+    # 3. Perturbation intensity: max(L2, Linf). Captures whether the
+    #    attacker is actively perturbing inputs.
+    if "feature_delta_l2_norm" in df.columns and "feature_delta_linf_norm" in df.columns:
+        df["perturb_intensity"] = np.maximum(
+            df["feature_delta_l2_norm"].fillna(0),
+            df["feature_delta_linf_norm"].fillna(0),
+        ).astype(float)
+    else:
+        df["perturb_intensity"] = 0.0
+
+    # 4. Defender weakness composite: low detection_strength + low
+    #    adversarial_robustness = more evadable defender. Some phases
+    #    (evasion_attempt) cluster on weaker defenders.
+    if "detection_strength" in df.columns and "adversarial_robustness" in df.columns:
+        df["defender_weakness"] = (
+            (1 - df["detection_strength"].fillna(0.5))
+            * (1 - df["adversarial_robustness"].fillna(0.5))
+        ).astype(float)
+    else:
+        df["defender_weakness"] = 0.0
+
+    # 5. Query-per-timestep rate: indicates active probing vs idling.
+    if "query_count_cumulative" in df.columns and "timestep" in df.columns:
+        df["query_rate"] = (
+            df["query_count_cumulative"] / df["timestep"].clip(lower=1)
+        ).astype(float)
+    else:
+        df["query_rate"] = 0.0
+
+    return df
+
+
+# ---------------------------------------------------------------------------
+# Public API
+# ---------------------------------------------------------------------------
+
+def build_features(
+    trajectories_path: str | Path,
+    topology_path: str | Path,
+) -> tuple[pd.DataFrame, pd.Series, pd.Series, pd.Series, dict[str, Any]]:
+    """
+    Load attack_trajectories.csv, join network_topology.csv, drop
+    target + identifiers + oracle columns, engineer features, one-hot
+    encode, return (X, y, ids, groups, meta).
+    """
+    traj = pd.read_csv(trajectories_path)
+    topo = pd.read_csv(topology_path)
+
+    y = traj[TARGET_COLUMN].map(LABEL_TO_INT)
+    if y.isna().any():
+        bad = traj.loc[y.isna(), TARGET_COLUMN].unique()
+        raise ValueError(f"Unknown attack_phase values: {bad}")
+    y = y.astype(int)
+    ids = (
+        traj["campaign_id"].astype(str)
+        + ":t"
+        + traj["timestep"].astype(str)
+    )
+    groups = traj[GROUP_COLUMN].copy()
+
+    topo_cols_needed = (
+        ["segment_id"]
+        + SEGMENT_NUMERIC_FEATURES
+        + SEGMENT_CATEGORICAL_FEATURES
+    )
+    traj = traj.merge(
+        topo[topo_cols_needed],
+        left_on="target_segment_id", right_on="segment_id",
+        how="left",
+    )
+
+    traj = _add_engineered_features(traj)
+
+    traj = traj.drop(
+        columns=ID_COLUMNS + [TARGET_COLUMN] + ORACLE_COLUMNS,
+        errors="ignore",
+    )
+
+    numeric_features = (
+        EVENT_NUMERIC_FEATURES
+        + SEGMENT_NUMERIC_FEATURES
+        + [
+            "progress_frac", "log_queries", "perturb_intensity",
+            "defender_weakness", "query_rate",
+        ]
+    )
+    numeric_features = [c for c in numeric_features if c in traj.columns]
+    X_numeric = traj[numeric_features].astype(float)
+
+    all_categorical = EVENT_CATEGORICAL_FEATURES + SEGMENT_CATEGORICAL_FEATURES
+    categorical_levels: dict[str, list[str]] = {}
+    blocks: list[pd.DataFrame] = []
+    for col in all_categorical:
+        if col not in traj.columns:
+            continue
+        levels = sorted(traj[col].dropna().astype(str).unique().tolist())
+        categorical_levels[col] = levels
+        block = pd.get_dummies(
+            traj[col].astype(str).astype("category").cat.set_categories(levels),
+            prefix=col, dummy_na=False,
+        ).astype(int)
+        blocks.append(block)
+
+    X = pd.concat(
+        [X_numeric.reset_index(drop=True)]
+        + [b.reset_index(drop=True) for b in blocks],
+        axis=1,
+    ).fillna(0.0)
+
+    meta = {
+        "feature_names": X.columns.tolist(),
+        "numeric_features": numeric_features,
+        "categorical_levels": categorical_levels,
+        "label_to_int": LABEL_TO_INT,
+        "int_to_label": INT_TO_LABEL,
+        "oracle_excluded": ORACLE_COLUMNS,
+    }
+    return X, y, ids, groups, meta
+
+
+def transform_single(
+    record: dict | pd.DataFrame,
+    meta: dict[str, Any],
+    segment_lookup: dict | None = None,
+) -> np.ndarray:
+    """Encode a single trajectory record for inference."""
+    if isinstance(record, dict):
+        df = pd.DataFrame([record.copy()])
+    else:
+        df = record.copy()
+
+    if segment_lookup is not None and "target_segment_id" in df.columns:
+        seg_id = df["target_segment_id"].iloc[0]
+        seg_feats = segment_lookup.get(seg_id, {})
+        for k, v in seg_feats.items():
+            if k not in df.columns:
+                df[k] = v
+
+    df = _add_engineered_features(df)
+
+    numeric = pd.DataFrame({
+        col: df.get(col, pd.Series([0.0] * len(df))).astype(float).values
+        for col in meta["numeric_features"]
+    })
+    blocks: list[pd.DataFrame] = [numeric]
+    for col, levels in meta["categorical_levels"].items():
+        val = df.get(col, pd.Series([None] * len(df))).astype(str)
+        block = pd.get_dummies(
+            val.astype("category").cat.set_categories(levels),
+            prefix=col, dummy_na=False,
+        ).astype(int)
+        for lvl in levels:
+            cname = f"{col}_{lvl}"
+            if cname not in block.columns:
+                block[cname] = 0
+        block = block[[f"{col}_{lvl}" for lvl in levels]]
+        blocks.append(block)
+
+    X = pd.concat(blocks, axis=1).fillna(0.0)
+    X = X.reindex(columns=meta["feature_names"], fill_value=0.0)
+    return X.values.astype(np.float32)
+
+
+def save_meta(meta: dict[str, Any], path: str | Path) -> None:
+    serializable = {
+        "feature_names": meta["feature_names"],
+        "numeric_features": meta["numeric_features"],
+        "categorical_levels": meta["categorical_levels"],
+        "label_to_int": meta["label_to_int"],
+        "int_to_label": {str(k): v for k, v in meta["int_to_label"].items()},
+        "oracle_excluded": meta.get("oracle_excluded", []),
+    }
+    with open(path, "w") as f:
+        json.dump(serializable, f, indent=2)
+
+
+def load_meta(path: str | Path) -> dict[str, Any]:
+    with open(path) as f:
+        meta = json.load(f)
+    meta["int_to_label"] = {int(k): v for k, v in meta["int_to_label"].items()}
+    return meta
+
+
+def build_segment_lookup(topology_path: str | Path) -> dict[str, dict]:
+    """Build {segment_id: {segment feature values}} for inference."""
+    topo = pd.read_csv(topology_path)
+    cols = SEGMENT_NUMERIC_FEATURES + SEGMENT_CATEGORICAL_FEATURES
+    out = {}
+    for _, row in topo.iterrows():
+        out[row["segment_id"]] = {c: row[c] for c in cols if c in topo.columns}
+    return out
+
+
+if __name__ == "__main__":
+    import sys
+    base = Path(sys.argv[1]) if len(sys.argv) > 1 else Path("/mnt/user-data/uploads")
+    X, y, ids, groups, meta = build_features(
+        base / "attack_trajectories.csv",
+        base / "network_topology.csv",
+    )
+    print(f"X shape: {X.shape}")
+    print(f"y shape: {y.shape}")
+    print(f"groups: {groups.nunique()} unique campaigns")
+    print(f"n_features: {len(meta['feature_names'])}")
+    print(f"label distribution:\n{y.map(INT_TO_LABEL).value_counts()}")
+    print(f"X has NaN: {X.isnull().any().any()}")

feature_meta.json

@@ -0,0 +1,111 @@
+{
+  "feature_names": [
+    "timestep",
+    "perturbation_magnitude",
+    "feature_delta_l2_norm",
+    "feature_delta_linf_norm",
+    "query_count_cumulative",
+    "trust_level",
+    "detection_coverage",
+    "feature_space_dim",
+    "alert_threshold",
+    "retraining_cadence_days",
+    "ensemble_size",
+    "detection_strength",
+    "adversarial_robustness",
+    "progress_frac",
+    "log_queries",
+    "perturb_intensity",
+    "defender_weakness",
+    "query_rate",
+    "attacker_capability_tier_advanced_persistent_threat",
+    "attacker_capability_tier_opportunistic",
+    "attacker_capability_tier_script_kiddie",
+    "segment_type_cloud_workload",
+    "segment_type_corporate_lan",
+    "segment_type_data_exfiltration_target",
+    "segment_type_dmz_perimeter",
+    "segment_type_endpoint_fleet",
+    "segment_type_ot_ics_control_network",
+    "segment_type_soc_management_plane",
+    "segment_type_zero_trust_segment",
+    "defender_architecture_autoencoder_anomaly",
+    "defender_architecture_ensemble_stacked",
+    "defender_architecture_gradient_boosted_tree",
+    "defender_architecture_isolation_forest",
+    "defender_architecture_lstm_behavioural",
+    "defender_architecture_neural_network_dense",
+    "defender_architecture_rule_based_threshold",
+    "defender_architecture_transformer_sequence"
+  ],
+  "numeric_features": [
+    "timestep",
+    "perturbation_magnitude",
+    "feature_delta_l2_norm",
+    "feature_delta_linf_norm",
+    "query_count_cumulative",
+    "trust_level",
+    "detection_coverage",
+    "feature_space_dim",
+    "alert_threshold",
+    "retraining_cadence_days",
+    "ensemble_size",
+    "detection_strength",
+    "adversarial_robustness",
+    "progress_frac",
+    "log_queries",
+    "perturb_intensity",
+    "defender_weakness",
+    "query_rate"
+  ],
+  "categorical_levels": {
+    "attacker_capability_tier": [
+      "advanced_persistent_threat",
+      "opportunistic",
+      "script_kiddie"
+    ],
+    "segment_type": [
+      "cloud_workload",
+      "corporate_lan",
+      "data_exfiltration_target",
+      "dmz_perimeter",
+      "endpoint_fleet",
+      "ot_ics_control_network",
+      "soc_management_plane",
+      "zero_trust_segment"
+    ],
+    "defender_architecture": [
+      "autoencoder_anomaly",
+      "ensemble_stacked",
+      "gradient_boosted_tree",
+      "isolation_forest",
+      "lstm_behavioural",
+      "neural_network_dense",
+      "rule_based_threshold",
+      "transformer_sequence"
+    ]
+  },
+  "label_to_int": {
+    "reconnaissance": 0,
+    "feature_space_probe": 1,
+    "perturbation_craft": 2,
+    "evasion_attempt": 3,
+    "feedback_adaptation": 4,
+    "campaign_consolidation": 5,
+    "idle_dwell": 6
+  },
+  "int_to_label": {
+    "0": "reconnaissance",
+    "1": "feature_space_probe",
+    "2": "perturbation_craft",
+    "3": "evasion_attempt",
+    "4": "feedback_adaptation",
+    "5": "campaign_consolidation",
+    "6": "idle_dwell"
+  },
+  "oracle_excluded": [
+    "detection_outcome",
+    "detector_confidence_score",
+    "evasion_budget_consumed"
+  ]
+}

feature_scaler.json

@@ -0,0 +1 @@
+{"mean": [35.5, 0.18952780945529293, 1.306543723946557, 0.13124837821171634, 22.172045220966083, 0.6039635149023638, 0.7373798047276464, 83.58273381294964, 0.6482954779033917, 29.53186022610483, 1.9398766700924974, 0.7428622816032887, 0.5362281603288798, 0.5071428571428571, 2.7516034029559706, 1.306543723946557, 0.1348483556012333, 0.5364089872594734, 0.10071942446043165, 0.381294964028777, 0.5179856115107914, 0.18088386433710174, 0.12435765673175746, 0.1264131551901336, 0.10996916752312436, 0.11664953751284686, 0.11613566289825282, 0.12281603288797534, 0.10277492291880781, 0.17317574511819117, 0.14131551901336073, 0.10123329907502569, 0.11459403905447071, 0.09198355601233299, 0.11356628982528263, 0.17060637204522097, 0.09352517985611511], "std": [20.206235725016693, 0.09167219921794667, 0.8164877299177017, 0.08880481196415957, 15.071372312011375, 0.1506360032773659, 0.11803727444435302, 18.394205279913415, 0.08031189726900136, 7.820137508750206, 1.280796109162329, 0.11317550157786087, 0.14072721772354296, 0.28866051035738133, 1.091565441029427, 0.8164877299177017, 0.09460859412712151, 0.2023285682155406, 0.3009723106774258, 0.48572972162326267, 0.4997020921518917, 0.38494171137992955, 0.33000609471109255, 0.3323314915581053, 0.31286739993889146, 0.3210187131250674, 0.3204039972341474, 0.3282427885969357, 0.30368028618535087, 0.37841858286034125, 0.3483646303082042, 0.3016528968584111, 0.3185477579797315, 0.2890175882967011, 0.3173000708343629, 0.37618397359867656, 0.2911819612538873]}

inference_example.ipynb

@@ -0,0 +1,342 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# CYB011 Baseline Classifier — Inference Example\n",
+    "\n",
+    "End-to-end demo: load the trained XGBoost and PyTorch MLP models from the Hugging Face repo and predict the **adversarial attack phase** for a per-timestep trajectory record.\n",
+    "\n",
+    "**Models predict one of 7 phases:** `reconnaissance`, `feature_space_probe`, `perturbation_craft`, `evasion_attempt`, `feedback_adaptation`, `campaign_consolidation`, `idle_dwell`.\n",
+    "\n",
+    "**This is a baseline reference model**, not a production phase classifier. See the model card and **`leakage_diagnostic.json`** for the structural-leakage findings (6 oracle paths documented across the dataset, 4 README-suggested targets unlearnable after honest leak removal)."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## 1. Install dependencies"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "%pip install --quiet xgboost torch safetensors pandas numpy huggingface_hub"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## 2. Download model artifacts from Hugging Face"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from huggingface_hub import hf_hub_download\n",
+    "\n",
+    "REPO_ID = \"xpertsystems/cyb011-baseline-classifier\"\n",
+    "\n",
+    "files = {}\n",
+    "for name in [\"model_xgb.json\", \"model_mlp.safetensors\",\n",
+    "             \"feature_engineering.py\", \"feature_meta.json\",\n",
+    "             \"feature_scaler.json\"]:\n",
+    "    files[name] = hf_hub_download(repo_id=REPO_ID, filename=name)\n",
+    "    print(f\"  downloaded: {name}\")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import sys, os\n",
+    "fe_dir = os.path.dirname(files[\"feature_engineering.py\"])\n",
+    "if fe_dir not in sys.path:\n",
+    "    sys.path.insert(0, fe_dir)\n",
+    "\n",
+    "from feature_engineering import (\n",
+    "    transform_single, load_meta, build_segment_lookup, INT_TO_LABEL,\n",
+    ")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## 3. Load models and metadata"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import json\n",
+    "import numpy as np\n",
+    "import torch\n",
+    "import torch.nn as nn\n",
+    "import xgboost as xgb\n",
+    "from safetensors.torch import load_file\n",
+    "\n",
+    "meta = load_meta(files[\"feature_meta.json\"])\n",
+    "with open(files[\"feature_scaler.json\"]) as f:\n",
+    "    scaler = json.load(f)\n",
+    "\n",
+    "N_FEATURES = len(meta[\"feature_names\"])\n",
+    "N_CLASSES = len(meta[\"int_to_label\"])\n",
+    "print(f\"feature count: {N_FEATURES}\")\n",
+    "print(f\"class count:   {N_CLASSES}\")\n",
+    "print(f\"label classes: {list(meta['int_to_label'].values())}\")\n",
+    "print(f\"\\noracle columns excluded (do not pass these to the model):\")\n",
+    "for c in meta.get(\"oracle_excluded\", []):\n",
+    "    print(f\"  - {c}\")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "xgb_model = xgb.XGBClassifier()\n",
+    "xgb_model.load_model(files[\"model_xgb.json\"])\n",
+    "\n",
+    "# MLP architecture (must match training)\n",
+    "class PhaseMLP(nn.Module):\n",
+    "    def __init__(self, n_features, n_classes=7, hidden1=128, hidden2=64, dropout=0.3):\n",
+    "        super().__init__()\n",
+    "        self.net = nn.Sequential(\n",
+    "            nn.Linear(n_features, hidden1),\n",
+    "            nn.BatchNorm1d(hidden1),\n",
+    "            nn.ReLU(),\n",
+    "            nn.Dropout(dropout),\n",
+    "            nn.Linear(hidden1, hidden2),\n",
+    "            nn.BatchNorm1d(hidden2),\n",
+    "            nn.ReLU(),\n",
+    "            nn.Dropout(dropout),\n",
+    "            nn.Linear(hidden2, n_classes),\n",
+    "        )\n",
+    "    def forward(self, x):\n",
+    "        return self.net(x)\n",
+    "\n",
+    "mlp_model = PhaseMLP(N_FEATURES, n_classes=N_CLASSES)\n",
+    "mlp_model.load_state_dict(load_file(files[\"model_mlp.safetensors\"]))\n",
+    "mlp_model.eval()\n",
+    "print(\"models loaded\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## 4. Load segment topology for defender-feature lookup\n",
+    "\n",
+    "The model uses segment context (defender_architecture, detection_strength, ensemble_size, etc.) as features. To predict on a new trajectory, we look up its segment features from the network_topology."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from huggingface_hub import snapshot_download\n",
+    "\n",
+    "ds_path = snapshot_download(repo_id=\"xpertsystems/cyb011-sample\", repo_type=\"dataset\")\n",
+    "segment_lookup = build_segment_lookup(f\"{ds_path}/network_topology.csv\")\n",
+    "print(f\"loaded {len(segment_lookup)} segment records\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## 5. Prediction helper"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "MU = np.array(scaler[\"mean\"], dtype=np.float32)\n",
+    "SD = np.array(scaler[\"std\"],  dtype=np.float32)\n",
+    "\n",
+    "def predict_attack_phase(record: dict) -> dict:\n",
+    "    \"\"\"Predict the adversarial attack phase for one trajectory record.\n",
+    "\n",
+    "    Note: do NOT include detection_outcome, detector_confidence_score,\n",
+    "    or evasion_budget_consumed in the record. These were outcome leaks\n",
+    "    in the training data and are excluded from the feature set.\n",
+    "\n",
+    "    Segment features (defender_architecture, detection_strength, etc.)\n",
+    "    are looked up from network_topology by target_segment_id.\n",
+    "    \"\"\"\n",
+    "    X = transform_single(record, meta, segment_lookup=segment_lookup)\n",
+    "\n",
+    "    xgb_proba = xgb_model.predict_proba(X)[0]\n",
+    "    xgb_label = INT_TO_LABEL[int(np.argmax(xgb_proba))]\n",
+    "\n",
+    "    Xs = ((X - MU) / SD).astype(np.float32)\n",
+    "    with torch.no_grad():\n",
+    "        logits = mlp_model(torch.tensor(Xs))\n",
+    "        mlp_proba = torch.softmax(logits, dim=1).numpy()[0]\n",
+    "    mlp_label = INT_TO_LABEL[int(np.argmax(mlp_proba))]\n",
+    "\n",
+    "    return {\n",
+    "        \"xgboost\": {\n",
+    "            \"label\": xgb_label,\n",
+    "            \"probabilities\": {INT_TO_LABEL[i]: float(p) for i, p in enumerate(xgb_proba)},\n",
+    "        },\n",
+    "        \"mlp\": {\n",
+    "            \"label\": mlp_label,\n",
+    "            \"probabilities\": {INT_TO_LABEL[i]: float(p) for i, p in enumerate(mlp_proba)},\n",
+    "        },\n",
+    "    }"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## 6. Run on an example record\n",
+    "\n",
+    "Real APT-tier trajectory at timestep 21 (mid-campaign). True phase is `evasion_attempt` — the attacker has built up 11 queries and is actively perturbing inputs."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# Real trajectory record from the sample dataset (true phase: evasion_attempt)\n",
+    "# Note: target_segment_id is supplied so segment features are auto-looked-up\n",
+    "example_record = {\n",
+    "    \"target_segment_id\": \"SEG00197\",\n",
+    "    \"timestep\": 21,\n",
+    "    \"perturbation_magnitude\": 0.14152,\n",
+    "    \"feature_delta_l2_norm\": 1.278436,\n",
+    "    \"feature_delta_linf_norm\": 0.14152,\n",
+    "    \"query_count_cumulative\": 11,\n",
+    "    \"attacker_capability_tier\": \"advanced_persistent_threat\",\n",
+    "}\n",
+    "\n",
+    "result = predict_attack_phase(example_record)\n",
+    "\n",
+    "print(f\"XGBoost  ->  {result['xgboost']['label']}\")\n",
+    "for lbl, p in sorted(result['xgboost']['probabilities'].items(), key=lambda x: -x[1]):\n",
+    "    print(f\"    P({lbl:30s}) = {p:.4f}\")\n",
+    "\n",
+    "print(f\"\\nMLP      ->  {result['mlp']['label']}\")\n",
+    "for lbl, p in sorted(result['mlp']['probabilities'].items(), key=lambda x: -x[1]):\n",
+    "    print(f\"    P({lbl:30s}) = {p:.4f}\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "### Per-class confidence patterns\n",
+    "\n",
+    "The model has strong confidence on `evasion_attempt` (per-class F1 1.00), `reconnaissance` (F1 0.89), and `campaign_consolidation` (F1 0.81) — these phases have distinctive feature signatures (query usage, timestep position, perturbation activity).\n",
+    "\n",
+    "The middle phases overlap more in feature space. `perturbation_craft` is the hardest class (F1 0.49) because its trajectory features look similar to `feature_space_probe` at the per-timestep level. A sequence model considering event ordering within campaigns would likely do better than per-timestep classification."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## 7. Batch prediction on the sample dataset"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import pandas as pd\n",
+    "\n",
+    "trajectories = pd.read_csv(f\"{ds_path}/attack_trajectories.csv\")\n",
+    "\n",
+    "# Score the first 500 events\n",
+    "sample = trajectories.head(500).copy()\n",
+    "preds = [predict_attack_phase(row.to_dict())[\"xgboost\"][\"label\"] for _, row in sample.iterrows()]\n",
+    "sample[\"xgb_pred\"] = preds\n",
+    "\n",
+    "ct = pd.crosstab(sample[\"attack_phase\"], sample[\"xgb_pred\"],\n",
+    "                 rownames=[\"true\"], colnames=[\"pred\"])\n",
+    "print(\"Confusion on first 500 sample events (XGBoost):\")\n",
+    "print(ct)\n",
+    "acc = (sample[\"attack_phase\"] == sample[\"xgb_pred\"]).mean()\n",
+    "print(f\"\\nbatch accuracy on first 500 events (in-distribution): {acc:.4f}\")\n",
+    "print(\"\\nNote: this includes training-set events. See validation_results.json\\n\"\n",
+    "      \"for proper held-out test metrics (group-aware split by campaign_id).\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## 8. Important reading: the leakage diagnostic\n",
+    "\n",
+    "Before using CYB011 sample data to train your own models, read **`leakage_diagnostic.json`** in this repo. It documents **6 oracle paths** across the sample's targets:\n",
+    "\n",
+    "**Phase target oracles (3 paths — dropped from features):**\n",
+    "1. `detection_outcome` (`!= suppressed_alert` → 100% `evasion_attempt`)\n",
+    "2. `detector_confidence_score` (threshold-derived from `detection_outcome`)\n",
+    "3. `evasion_budget_consumed` (`== 0` → 100% one of 3 early phases)\n",
+    "\n",
+    "**Other documented leaks (for transparency, not features for this model):**\n",
+    "4. `stealth_score` near-deterministic per `attacker_capability_tier` (campaign-level)\n",
+    "5. Topology fingerprint (7 segment-level features uniquely identify `defender_architecture`)\n",
+    "6. `timestep` partial oracle for 3 phases — **KEPT as legitimate campaign-progress observable**\n",
+    "\n",
+    "It also documents **4 README-suggested headline targets that are unlearnable on the sample** after honest leak removal: `campaign_success_flag`, `campaign_type` 8-class, `coordinated_attack_flag`, `defender_architecture` 8-class.\n",
+    "\n",
+    "And it documents the **missing `nation_state` attacker tier** — README claims 4 tiers, sample contains only 3 (script_kiddie, opportunistic, APT)."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## 9. Next steps\n",
+    "\n",
+    "- See `validation_results.json` for held-out test metrics (2,100 events from ~30 test campaigns).\n",
+    "- See `multi_seed_results.json` for the across-10-seeds picture (accuracy 0.867 ± 0.010, ROC-AUC 0.977 ± 0.002).\n",
+    "- See `ablation_results.json` for per-feature-group contribution. Perturbation features carry the most signal (−20pp accuracy when removed); query features second (−4pp).\n",
+    "- See **`leakage_diagnostic.json`** for the full 6-oracle-path audit and 4 unlearnable targets.\n",
+    "- For the full ~383k-row CYB011 dataset and commercial licensing, contact **pradeep@xpertsystems.ai**."
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "name": "python",
+   "version": "3.10"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}

leakage_diagnostic.json

@@ -0,0 +1,238 @@
+{
+  "purpose": "CYB011 sample has multiple structural leakage patterns rooted in the generator's outcome-modeling logic. Three outcome columns (detection_outcome, detector_confidence_score, evasion_budget_consumed) are perfect or near-perfect oracles for attack_phase. Per-campaign features encode attacker_capability_tier via stealth_score. Per-segment topology features uniquely fingerprint each defender_architecture. The published baseline (attack_phase 7-class) trains with the three phase oracles excluded but retains timestep as a legitimate campaign-progress observable.",
+  "primary_target": "attack_phase (7-class, per-timestep)",
+  "split": "GroupShuffleSplit on campaign_id, 70/15/15 nested",
+  "missing_attacker_tier_note": {
+    "issue": "README claims 4 attacker_capability_tier values (script_kiddie, opportunistic, advanced_persistent_threat, nation_state). The sample data contains only 3: nation_state is entirely absent. Models trained on this sample cannot generalize to nation_state actors.",
+    "tier_counts_in_sample": {
+      "script_kiddie": 7000,
+      "opportunistic": 5600,
+      "advanced_persistent_threat": 1400
+    }
+  },
+  "oracle_paths_documented": {
+    "P1_detection_outcome": {
+      "target": "attack_phase",
+      "leak_column": "detection_outcome",
+      "mechanism": "Three of the four detection_outcome values (evasion_success, marginal_alert, high_confidence_alert) occur ONLY when attack_phase == 'evasion_attempt'. The fourth value (suppressed_alert) occurs across all 7 phases. So detection_outcome != suppressed_alert is a perfect oracle for evasion_attempt phase.",
+      "evidence_crosstab": {
+        "evasion_success": {
+          "campaign_consolidation": 0,
+          "evasion_attempt": 416,
+          "feature_space_probe": 0,
+          "feedback_adaptation": 0,
+          "idle_dwell": 0,
+          "perturbation_craft": 0,
+          "reconnaissance": 0
+        },
+        "high_confidence_alert": {
+          "campaign_consolidation": 0,
+          "evasion_attempt": 1102,
+          "feature_space_probe": 0,
+          "feedback_adaptation": 0,
+          "idle_dwell": 0,
+          "perturbation_craft": 0,
+          "reconnaissance": 0
+        },
+        "marginal_alert": {
+          "campaign_consolidation": 0,
+          "evasion_attempt": 3228,
+          "feature_space_probe": 0,
+          "feedback_adaptation": 0,
+          "idle_dwell": 0,
+          "perturbation_craft": 0,
+          "reconnaissance": 0
+        },
+        "suppressed_alert": {
+          "campaign_consolidation": 829,
+          "evasion_attempt": 2460,
+          "feature_space_probe": 1465,
+          "feedback_adaptation": 496,
+          "idle_dwell": 2450,
+          "perturbation_craft": 745,
+          "reconnaissance": 809
+        }
+      },
+      "verdict": "Perfect oracle for evasion_attempt (51% of all events)."
+    },
+    "P2_detector_confidence_score": {
+      "target": "attack_phase (via detection_outcome)",
+      "leak_column": "detector_confidence_score",
+      "mechanism": "detector_confidence_score is threshold-derived from detection_outcome: <0.25 -> evasion_success, [0.52,0.78] -> marginal_alert, >=0.78 -> high_confidence_alert. Non-overlapping ranges mean detection_outcome is mechanically decoded from this score, indirectly oracling attack_phase.",
+      "score_ranges_by_outcome": {
+        "evasion_success": {
+          "min": 0.001,
+          "max": 0.25,
+          "mean": 0.1801,
+          "std": 0.0553
+        },
+        "high_confidence_alert": {
+          "min": 0.7801,
+          "max": 0.999,
+          "mean": 0.8558,
+          "std": 0.0561
+        },
+        "marginal_alert": {
+          "min": 0.5201,
+          "max": 0.7797,
+          "mean": 0.6436,
+          "std": 0.0737
+        },
+        "suppressed_alert": {
+          "min": 0.001,
+          "max": 0.999,
+          "mean": 0.3992,
+          "std": 0.1817
+        }
+      },
+      "verdict": "Mechanical decoder for detection_outcome -> indirect oracle for phase."
+    },
+    "P3_evasion_budget_consumed_zero": {
+      "target": "attack_phase (3 early phases)",
+      "leak_column": "evasion_budget_consumed",
+      "mechanism": "evasion_budget_consumed == 0 occurs in 100% of {reconnaissance, feature_space_probe, perturbation_craft} events (the 3 early phases that don't submit evasion attempts). > 0 occurs in 100% of the 4 later phases.",
+      "early_phase_events_at_zero": 3019,
+      "verdict": "Perfect oracle for the 3 early phases."
+    },
+    "P4_stealth_score_to_tier": {
+      "target": "attacker_capability_tier (campaign level)",
+      "leak_column": "stealth_score",
+      "mechanism": "stealth_score has tier-discriminative ranges with modest overlap: APT in [0.806, 0.938] (mean 0.912), opportunistic in [0.751, 0.924] (mean 0.882), script_kiddie in [0.715, 0.950] (mean 0.846). Drives per-campaign tier prediction to 0.94 accuracy vs 0.50 majority - artificially inflated.",
+      "stealth_ranges_by_tier": {
+        "advanced_persistent_threat": {
+          "min": 0.806,
+          "max": 0.938,
+          "mean": 0.9116,
+          "std": 0.0277
+        },
+        "opportunistic": {
+          "min": 0.7508,
+          "max": 0.9236,
+          "mean": 0.8816,
+          "std": 0.0359
+        },
+        "script_kiddie": {
+          "min": 0.7148,
+          "max": 0.95,
+          "mean": 0.8456,
+          "std": 0.0462
+        }
+      },
+      "verdict": "Near-deterministic per-tier feature. Per-campaign tier prediction is structurally inflated by this leak."
+    },
+    "P5_topology_fingerprint": {
+      "target": "defender_architecture",
+      "leak_column": "(combination of 7 topology features)",
+      "mechanism": "Each defender_architecture has detection_strength and adversarial_robustness as a CONSTANT (std = 0.0 across all rows of that architecture). Combined with ranges of ensemble_size, alert_threshold, detection_coverage, feature_space_dim, and retraining_cadence_days, each topology row uniquely fingerprints its defender. The 8-class defender_architecture target hits 100% accuracy via this combination.",
+      "detection_strength_std_within_arch": {
+        "autoencoder_anomaly": 0.0,
+        "ensemble_stacked": 0.0,
+        "gradient_boosted_tree": 0.0,
+        "isolation_forest": 0.0,
+        "lstm_behavioural": 0.0,
+        "neural_network_dense": 0.0,
+        "rule_based_threshold": 0.0,
+        "transformer_sequence": 0.0
+      },
+      "adversarial_robustness_std_within_arch": {
+        "autoencoder_anomaly": 0.0,
+        "ensemble_stacked": 0.0,
+        "gradient_boosted_tree": 0.0,
+        "isolation_forest": 0.0,
+        "lstm_behavioural": 0.0,
+        "neural_network_dense": 0.0,
+        "rule_based_threshold": 0.0,
+        "transformer_sequence": 0.0
+      },
+      "verdict": "Trivially leaky 8-class target. Each segment row uniquely identifies its defender architecture by feature combination."
+    },
+    "P6_timestep_partial": {
+      "target": "attack_phase (partial)",
+      "leak_column": "timestep",
+      "mechanism": "Phases have characteristic timestep ranges due to the sequential lifecycle structure. reconnaissance is timestep 1-7 (mean 3.16), campaign_consolidation is 65-70 (mean 67.96), feedback_adaptation is 63-66 (mean 64.15). The middle phases overlap broadly. NOTE: timestep is KEPT as a feature in the published model because it's a legitimate campaign-progress observable a defender would have at decision time. Documenting here for transparency: removing timestep drops headline accuracy by ~9pp (0.87 -> 0.78).",
+      "timestep_ranges_by_phase": {
+        "campaign_consolidation": {
+          "min": 65,
+          "max": 70,
+          "mean": 67.96
+        },
+        "evasion_attempt": {
+          "min": 11,
+          "max": 62,
+          "mean": 40.32
+        },
+        "feature_space_probe": {
+          "min": 4,
+          "max": 35,
+          "mean": 11.29
+        },
+        "feedback_adaptation": {
+          "min": 63,
+          "max": 66,
+          "mean": 64.15
+        },
+        "idle_dwell": {
+          "min": 1,
+          "max": 70,
+          "mean": 35.44
+        },
+        "perturbation_craft": {
+          "min": 8,
+          "max": 38,
+          "mean": 16.65
+        },
+        "reconnaissance": {
+          "min": 1,
+          "max": 7,
+          "mean": 3.16
+        }
+      },
+      "verdict": "Partial oracle for 3 phases (reconnaissance, feedback_adaptation, campaign_consolidation). KEPT as legitimate progress feature."
+    }
+  },
+  "unlearnable_targets": [
+    {
+      "target": "campaign_success_flag (per-campaign)",
+      "n_campaigns": 200,
+      "majority_baseline": 0.605,
+      "honest_accuracy": 0.5111111111111111,
+      "honest_roc_auc": 0.48765432098765427,
+      "verdict": "below_majority"
+    },
+    {
+      "target": "campaign_type (per-campaign)",
+      "n_campaigns": 200,
+      "majority_baseline": 0.17,
+      "honest_accuracy": 0.11111111111111112,
+      "honest_roc_auc": 0.48226979604757386,
+      "verdict": "below_majority"
+    },
+    {
+      "target": "coordinated_attack_flag (per-campaign)",
+      "n_campaigns": 200,
+      "majority_baseline": 0.9,
+      "honest_accuracy": 0.8333333333333334,
+      "honest_roc_auc": 0.38271604938271603,
+      "verdict": "below_majority"
+    },
+    {
+      "target": "defender_architecture (per-campaign, all 7 topology fingerprint features dropped)",
+      "n_campaigns": 200,
+      "majority_baseline": 0.17,
+      "honest_accuracy": 0.13333333333333333,
+      "honest_roc_auc": 0.5770656344684122,
+      "verdict": "below_majority",
+      "note": "With all 7 topology fingerprint features included, defender_architecture hits 100% trivially. With all 7 dropped, performance collapses to or below majority. The target is not learnable from the trajectory features themselves - only from the segment fingerprint."
+    }
+  ],
+  "unlearnable_summary": "Four README-suggested headline targets are unlearnable on the sample after honest oracle removal: campaign_success_flag (acc ~0.51 vs maj 0.61), campaign_type 8-class (acc ~0.11 vs maj 0.17), coordinated_attack_flag (acc ~0.83 vs maj 0.90), and defender_architecture 8-class (trivially leaky via topology fingerprint; collapses when the fingerprint is dropped). Only attack_phase 7-class learns honestly with a respectable lift over majority.",
+  "recommendations_to_dataset_author": [
+    "Make detector_confidence_score have OVERLAPPING ranges across detection_outcome values. As shipped, the ranges are perfectly non-overlapping (high_confidence_alert >=0.78, marginal_alert [0.52, 0.78], evasion_success <0.25). This makes detection_outcome a mechanical function of the score.",
+    "Allow evasion_budget_consumed to be positive in some reconnaissance / feature_space_probe / perturbation_craft events. The current zero-only encoding creates a perfect oracle for these 3 phases.",
+    "Add per-tier feature noise. stealth_score has tier-discriminative ranges (APT >0.80, script_kiddie <0.95) but with substantial overlap. Tighten the noise so the per-campaign tier-attribution task isn't structurally inflated.",
+    "Add per-segment NOISE to detection_strength and adversarial_robustness. Currently these are CONSTANT per defender_architecture (std=0.0). Real systems have deployment-specific tuning, so these should vary within an architecture class.",
+    "Include the missing nation_state attacker tier in the sample. The README lists 4 tiers but the sample contains only 3. Buyers cannot validate nation_state-specific modeling on the sample.",
+    "Increase coordinated_attack positives in the sample (only 20 of 200 campaigns at 10%). With n=20 positives, the binary task has insufficient statistical power for honest evaluation.",
+    "For campaign_type 8-class, add stronger per-type feature signatures. Currently the 8 types are not discriminable from trajectory features at n=200 campaigns."
+  ]
+}

model_mlp.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:06e8b3f2322f2b94a9a376f77306c119fc335173f3979c893910a729be379bf1
+size 58596

multi_seed_results.json

@@ -0,0 +1,98 @@
+{
+  "purpose": "Multi-seed evaluation across 10 group-aware splits of the 14,000-event sample (200 campaigns).",
+  "seeds_evaluated": [
+    42,
+    7,
+    13,
+    17,
+    23,
+    31,
+    45,
+    99,
+    123,
+    200
+  ],
+  "per_seed": [
+    {
+      "seed": 42,
+      "test_n_classes": 7,
+      "accuracy": 0.8642857142857143,
+      "macro_f1": 0.7693247628697397,
+      "macro_roc_auc_ovr": 0.9752868672798508
+    },
+    {
+      "seed": 7,
+      "test_n_classes": 7,
+      "accuracy": 0.8733333333333333,
+      "macro_f1": 0.7868555284450741,
+      "macro_roc_auc_ovr": 0.9786952359398997
+    },
+    {
+      "seed": 13,
+      "test_n_classes": 7,
+      "accuracy": 0.8752380952380953,
+      "macro_f1": 0.7750991458229394,
+      "macro_roc_auc_ovr": 0.9779387743730787
+    },
+    {
+      "seed": 17,
+      "test_n_classes": 7,
+      "accuracy": 0.8738095238095238,
+      "macro_f1": 0.7814925647016364,
+      "macro_roc_auc_ovr": 0.9776960470844541
+    },
+    {
+      "seed": 23,
+      "test_n_classes": 7,
+      "accuracy": 0.8838095238095238,
+      "macro_f1": 0.7978303920930874,
+      "macro_roc_auc_ovr": 0.9798719092961202
+    },
+    {
+      "seed": 31,
+      "test_n_classes": 7,
+      "accuracy": 0.8690476190476191,
+      "macro_f1": 0.7726664814609271,
+      "macro_roc_auc_ovr": 0.9759310226918093
+    },
+    {
+      "seed": 45,
+      "test_n_classes": 7,
+      "accuracy": 0.8519047619047619,
+      "macro_f1": 0.7504006897882468,
+      "macro_roc_auc_ovr": 0.9727919502752255
+    },
+    {
+      "seed": 99,
+      "test_n_classes": 7,
+      "accuracy": 0.8585714285714285,
+      "macro_f1": 0.7746640410602633,
+      "macro_roc_auc_ovr": 0.9769979540429897
+    },
+    {
+      "seed": 123,
+      "test_n_classes": 7,
+      "accuracy": 0.8533333333333334,
+      "macro_f1": 0.771942700676468,
+      "macro_roc_auc_ovr": 0.9738063729400632
+    },
+    {
+      "seed": 200,
+      "test_n_classes": 7,
+      "accuracy": 0.8652380952380953,
+      "macro_f1": 0.7668641323226082,
+      "macro_roc_auc_ovr": 0.9762239650477442
+    }
+  ],
+  "aggregate": {
+    "accuracy_mean": 0.8668571428571428,
+    "accuracy_std": 0.009680145423468645,
+    "accuracy_min": 0.8519047619047619,
+    "accuracy_max": 0.8838095238095238,
+    "macro_f1_mean": 0.774714043924099,
+    "macro_f1_std": 0.011922910105924629,
+    "roc_auc_mean": 0.9765240098971235,
+    "roc_auc_std": 0.0020690216988592247
+  },
+  "published_artifact_seed": 42
+}

validation_results.json

@@ -0,0 +1,247 @@
+{
+  "version": "1.0.0",
+  "dataset": "xpertsystems/cyb011-sample",
+  "task": "7-class attack_phase classification",
+  "baselines": {
+    "always_predict_majority_accuracy": 0.5033333333333333,
+    "majority_class": "evasion_attempt",
+    "random_guess_accuracy": 0.14285714285714285
+  },
+  "split": {
+    "strategy": "group-aware (GroupShuffleSplit on campaign_id, nested 70/15/15)",
+    "rationale": "200 campaigns x 70 timesteps each. Timesteps from the same campaign share attacker, target segment, and tier - so train/test contamination is a real risk with random splitting. ~30 test campaigns per fold.",
+    "events_train": 9730,
+    "events_val": 2170,
+    "events_test": 2100,
+    "seed": 42
+  },
+  "n_features": 37,
+  "label_classes": [
+    "reconnaissance",
+    "feature_space_probe",
+    "perturbation_craft",
+    "evasion_attempt",
+    "feedback_adaptation",
+    "campaign_consolidation",
+    "idle_dwell"
+  ],
+  "class_distribution_train": {
+    "evasion_attempt": 5082,
+    "idle_dwell": 1677,
+    "feature_space_probe": 983,
+    "campaign_consolidation": 571,
+    "reconnaissance": 558,
+    "perturbation_craft": 511,
+    "feedback_adaptation": 348
+  },
+  "class_distribution_test": {
+    "evasion_attempt": 1057,
+    "idle_dwell": 388,
+    "feature_space_probe": 220,
+    "reconnaissance": 128,
+    "campaign_consolidation": 116,
+    "perturbation_craft": 115,
+    "feedback_adaptation": 76
+  },
+  "oracle_excluded_features": [
+    "detection_outcome (perfect oracle for evasion_attempt phase)",
+    "detector_confidence_score (mechanical decoder for detection_outcome)",
+    "evasion_budget_consumed (==0 is perfect oracle for 3 early phases)"
+  ],
+  "timestep_kept_as_legitimate_feature": "timestep is KEPT as a feature. It's a partial oracle for 3 phases (reconnaissance, feedback_adaptation, campaign_consolidation) but is a legitimate campaign-progress observable a defender would have at decision time. Removing it drops accuracy by ~9pp.",
+  "leakage_audit_note": "See leakage_diagnostic.json for the full 6-oracle-path audit, 4 unlearnable README-suggested targets, and the missing nation_state attacker tier note.",
+  "models": {
+    "xgboost": {
+      "architecture": "Gradient-boosted decision trees, multi:softprob, 7 classes",
+      "framework": "xgboost",
+      "test_metrics": {
+        "model": "xgboost",
+        "accuracy": 0.8642857142857143,
+        "macro_f1": 0.7693247628697397,
+        "weighted_f1": 0.8650489644308249,
+        "per_class_f1": {
+          "reconnaissance": 0.8865248226950354,
+          "feature_space_probe": 0.7829977628635347,
+          "perturbation_craft": 0.4927536231884058,
+          "evasion_attempt": 0.9962013295346629,
+          "feedback_adaptation": 0.7151515151515152,
+          "campaign_consolidation": 0.8075471698113208,
+          "idle_dwell": 0.7040971168437026
+        },
+        "confusion_matrix": {
+          "labels": [
+            "reconnaissance",
+            "feature_space_probe",
+            "perturbation_craft",
+            "evasion_attempt",
+            "feedback_adaptation",
+            "campaign_consolidation",
+            "idle_dwell"
+          ],
+          "matrix": [
+            [
+              125,
+              0,
+              0,
+              0,
+              0,
+              0,
+              3
+            ],
+            [
+              0,
+              175,
+              43,
+              0,
+              0,
+              0,
+              2
+            ],
+            [
+              0,
+              20,
+              68,
+              0,
+              0,
+              0,
+              27
+            ],
+            [
+              0,
+              0,
+              2,
+              1049,
+              0,
+              0,
+              6
+            ],
+            [
+              0,
+              0,
+              0,
+              0,
+              59,
+              16,
+              1
+            ],
+            [
+              0,
+              0,
+              0,
+              0,
+              9,
+              107,
+              0
+            ],
+            [
+              29,
+              32,
+              48,
+              0,
+              21,
+              26,
+              232
+            ]
+          ]
+        },
+        "macro_roc_auc_ovr": 0.9752868672798508
+      }
+    },
+    "mlp": {
+      "architecture": "PyTorch MLP, 37 -> 128 -> 64 -> 7, BatchNorm1d + ReLU + Dropout, weighted cross-entropy loss",
+      "framework": "pytorch",
+      "test_metrics": {
+        "model": "mlp",
+        "accuracy": 0.8385714285714285,
+        "macro_f1": 0.7344635260259678,
+        "weighted_f1": 0.8387834443096441,
+        "per_class_f1": {
+          "reconnaissance": 0.8737201365187713,
+          "feature_space_probe": 0.746606334841629,
+          "perturbation_craft": 0.49707602339181284,
+          "evasion_attempt": 0.9928537398761315,
+          "feedback_adaptation": 0.627906976744186,
+          "campaign_consolidation": 0.784452296819788,
+          "idle_dwell": 0.6186291739894552
+        },
+        "confusion_matrix": {
+          "labels": [
+            "reconnaissance",
+            "feature_space_probe",
+            "perturbation_craft",
+            "evasion_attempt",
+            "feedback_adaptation",
+            "campaign_consolidation",
+            "idle_dwell"
+          ],
+          "matrix": [
+            [
+              128,
+              0,
+              0,
+              0,
+              0,
+              0,
+              0
+            ],
+            [
+              0,
+              165,
+              55,
+              0,
+              0,
+              0,
+              0
+            ],
+            [
+              5,
+              24,
+              85,
+              0,
+              0,
+              0,
+              1
+            ],
+            [
+              0,
+              4,
+              2,
+              1042,
+              4,
+              1,
+              4
+            ],
+            [
+              0,
+              0,
+              0,
+              0,
+              54,
+              22,
+              0
+            ],
+            [
+              0,
+              0,
+              0,
+              0,
+              5,
+              111,
+              0
+            ],
+            [
+              32,
+              29,
+              85,
+              0,
+              33,
+              33,
+              176
+            ]
+          ]
+        },
+        "macro_roc_auc_ovr": 0.9705026035482472
+      }
+    }
+  }
+}