Initial FireWx-FM artifact release

.gitignore

@@ -0,0 +1,6 @@
+__pycache__/
+*.py[cod]
+.DS_Store
+.ipynb_checkpoints/
+*.log
+tmp/

LICENSE

@@ -0,0 +1,9 @@
+This release is provided for scholarly reproducibility of the associated paper.
+
+Code files are released under the MIT License.
+Paper-output tables, figures, and bundled summary artifacts are released for
+non-commercial research and review use with attribution.
+
+Raw data from NOAA, NASA, LANDFIRE, Wildfire Risk to Communities, LandScan,
+WFIGS, MTBS, and model providers are not redistributed here. Users must obtain
+those data from the original providers and comply with their terms.

README.md

@@ -0,0 +1,146 @@
+---
+license: mit
+tags:
+  - wildfire
+  - geospatial
+  - earth-observation
+  - foundation-models
+  - evaluation
+  - reproducibility
+  - paper-artifact
+pretty_name: FireWx-FM and Wildfire Evaluation Contracts
+---
+
+# FireWx-FM and Wildfire Evaluation Contracts
+
+This repository contains the public code and lightweight artifacts for the
+paper *Does Your Wildfire Prediction Model Actually Work, or Just Score Well?*
+
+The release has two parts:
+
+- **FireWx-FM reference backbone artifacts.** FireWx-FM is a wildfire-specialized
+  reference model used as an in-domain comparator in the paper.
+- **Fixed-contract evaluation artifacts.** The paper evaluates wildfire transfer
+  under fixed task, metric, matching-rule, scope, and head-family choices.
+
+This is a paper-artifact repository. It includes scripts, compact summary files,
+paper tables, and paper figures needed to inspect and reproduce reported
+outputs. It does **not** redistribute raw weather, fire, fuel, exposure,
+incident, perimeter, feature-cache, or private model files.
+
+## Key Results in the Release
+
+The bundled paper outputs reproduce the main results reported in the manuscript.
+
+| Check | Paper artifact | What it shows |
+|---|---|---|
+| Matching-rule sensitivity (RQ1) | `paper_outputs/figures/fig_fireprone_contract_progression_compact.pdf` | The same occupancy outputs can move sharply from exact to tolerated to union \(F_1\), especially under fire-prone scopes. |
+| Fixed-feature head selection (RQ2) | `paper_outputs/figures/fig_selection_regret_rq2.tikz` | Selecting a head by ranking evidence can lose decision performance relative to direct decision selection. |
+| Same-contract transfer matrix (RQ3) | `paper_outputs/tables/tab_primary_results.tex` | FireWx-FM, Prithvi-WxC, Aurora, ClimaX, StormCast, DLWP, FCN, FengWu, FuXi, Pangu-Weather, and AlphaEarth are compared under fixed occupancy and spread contracts. |
+| Supporting task forms (RQ4) | `paper_outputs/tables/tab_supporting_results.tex` and `paper_outputs/figures/fig_task_rank_map.pdf` | Backbone ranking changes across burned area, analog retrieval, smoke PM2.5, and extreme heat task forms. |
+
+Selected displayed values from the paper:
+
+- FireWx-FM reference occupancy union \(F_1\): `59.0656 ± 2.7372`.
+- ClimaX occupancy union \(F_1\): `60.1506 ± 7.5865`.
+- FireWx-FM fire-spread AP: `30.0900 ± 1.2500`.
+- FireWx-FM smoke PM2.5 RMSE: `4.4646 ± 0.0060`.
+- AlphaEarth smoke PM2.5 RMSE: `4.4403 ± 0.0488`.
+
+Values are stored as TeX table cells under `paper_outputs/tables/`. The
+corresponding compact CSV/JSON summaries are under `artifacts/results/`.
+
+## Repository Layout
+
+```text
+artifacts/
+  manifests/              table/figure provenance metadata and SHA-256 hashes
+  results/                compact CSV/JSON summaries used by paper outputs
+data_sources/             source list and download notes for raw data
+docs/                     artifact map and Hugging Face release notes
+experiments/              sanitized raw-rerun scripts and Slurm templates
+paper_outputs/
+  figures/                PDF/TikZ figures used by the manuscript
+  tables/                 TeX table blocks used by the manuscript
+scripts/                  release rebuild and audit scripts
+```
+
+## Quick Reproduction
+
+The paper-output path uses only the Python standard library.
+
+```bash
+python3 scripts/reproduce_paper_outputs.py
+```
+
+This command:
+
+- rebuilds the RQ1 fire-prone progression figure from summary JSON;
+- rebuilds the RQ2 selection-regret TikZ figure from CSV;
+- rebuilds the RQ4 rank-map PDF from the released main tables;
+- checks SHA-256 hashes for all final paper outputs;
+- audits that stale labels, local paths, and incomplete placeholders are absent.
+
+Expected terminal tail:
+
+```text
+Paper-output checksum check passed.
+Release audit passed.
+Rebuilt reproducible outputs and passed release audit.
+```
+
+## What Is Included
+
+- Final paper table TeX files under `paper_outputs/tables/`.
+- Final paper figures under `paper_outputs/figures/`.
+- Small released CSV/JSON summary artifacts under `artifacts/results/`.
+- Builder scripts for reproducible paper-output figures under `scripts/`.
+- Sanitized raw-rerun reference scripts under `experiments/raw_reference/`.
+- Data-source documentation under `data_sources/DATA_SOURCES.md`.
+- A table/figure provenance map under `docs/artifact_map.md`.
+
+## What Is Not Included
+
+Raw data are not bundled. The paper uses public or provider-hosted resources,
+including NOAA HRRR, NASA FIRMS, LANDFIRE, Wildfire Risk to Communities,
+LandScan, WFIGS, MTBS, and external Earth-FM/backbone sources.
+
+See `data_sources/DATA_SOURCES.md` for the role of each source and public access
+entry points. Full raw-data reruns require users to obtain those sources
+independently and rebuild local feature caches.
+
+## Reproducibility Scope
+
+There are two levels of reproducibility:
+
+1. **Paper-output reproduction from bundled artifacts.** This is lightweight and
+   does not require raw data, GPUs, or Slurm. It verifies the exact files used by
+   the manuscript.
+2. **Raw-data reruns.** These require separately downloaded source data, local
+   preprocessing, model dependencies, and compute resources. The repository
+   provides sanitized scripts and Slurm templates, but not the raw inputs.
+
+## Intended Use
+
+Use this repository to inspect paper values, reproduce released figures from
+summary artifacts, audit table/figure provenance, or adapt the fixed-contract
+evaluation workflow for wildfire transfer studies.
+
+Do not use this repository as a raw dataset mirror. Do not treat the included
+summary artifacts as a substitute for the original data sources.
+
+## Citation
+
+If you use this release, please cite:
+
+```bibtex
+@misc{wildfire_fm_evaluation_contracts_2026,
+  title = {Does Your Wildfire Prediction Model Actually Work, or Just Score Well?},
+  author = {Anonymous},
+  year = {2026},
+  note = {FireWx-FM and fixed-contract wildfire evaluation code and artifacts}
+}
+```
+
+The BibTeX entry will be updated with arXiv metadata after the preprint is
+public.

artifacts/manifests/paper_outputs.sha256

@@ -0,0 +1,20 @@
+b369d13e0419fa8272ccdc994b6642f3b141248a879c030218e387c583537eb2  paper_outputs/figures/fig_fireprone_contract_progression_compact.pdf
+b2e56403e2774c457dd12c4685e2dc7492e22e32df46fcc5c37b3087110f2439  paper_outputs/figures/fig_selection_regret_rq2.tikz
+bc4d35ad9cb4c1f9ba8f31c7c340d9684c9dd2d55f5a2e60604a2b58b90cbe40  paper_outputs/figures/fig_task_contract_tiles.pdf
+c382f5d69f25cc2f5db174601a33d0fd0928b44910a2a4b1c131954bd42113d9  paper_outputs/figures/fig_task_rank_map.pdf
+015ab951b0af5c130e4894092a5dd0bb0fd62e710467163a9df8246d8cf369f4  paper_outputs/figures/matching.pdf
+e8abbd2668517f5cae14933ed943fe103e74132886b0ff48ecd1685978549504  paper_outputs/tables/tab_app_analog_rank_depth.tex
+81db28aace3366625f1cfd5935892eb5af672d5ecd8327e6dcba00b7b04e2b3c  paper_outputs/tables/tab_app_burned_area_median_acre.tex
+4a93401ef355c02eb0cc6b2e9a1506f9ed9d912301ec6829581247e40991bdfb  paper_outputs/tables/tab_app_contract_params_full.tex
+3c5398c28e6243b1784b27d2e9eab1a5c60e6e6d2cfd14a79aa6fd1e0499b871  paper_outputs/tables/tab_app_head_architectures.tex
+f740b8f076490e852efa88fa8180ca08bb6b12901ff3ec3687c7e5c0b236da4e  paper_outputs/tables/tab_app_heat_event_pr.tex
+86e97a394ceae8cc6eafd6d1021b44d13a117378ead87bfee662cc90a1e0e54b  paper_outputs/tables/tab_app_matching_rule_params.tex
+0b1ad4587dd440fdabf771000b1c971daa9222e946a3404c9beae10dd7ea67c6  paper_outputs/tables/tab_app_occupancy_ppr_scope.tex
+4e79672c28a938cd9ba1bc0e423e7169eca389251a22357aff6fe84d3cbfa889  paper_outputs/tables/tab_app_scope_params.tex
+6850ee131e203f66392c79f17f59214672b362274f42285b252b83ac0ede1eb3  paper_outputs/tables/tab_app_seed_robustness.tex
+1ca91ca451f846e59cb62ea64a616780c698b9dee80918a05467bd6c40df2dd5  paper_outputs/tables/tab_app_smoke_high_event.tex
+cd65372622e8dd388adb1122a3e93b22d2090fba836405b08a078d5159b182de  paper_outputs/tables/tab_app_spread_ap_by_scope.tex
+a31d4a4e0f2f1c7f90a5610acea77aef5a48e63c754ab2159a42473dce2c3b94  paper_outputs/tables/tab_appendix_selection_regret_tolerance.tex
+22614e90568cc562c023c540bdfdec14c0923ecf55d432fffa2619625b856092  paper_outputs/tables/tab_fireprone_contract_progression.tex
+6672c62a150d83a351f4fa23ac04537d9aaae01af6056f689437d9b7d8bcee40  paper_outputs/tables/tab_primary_results.tex
+717555b2584658c936aa8fc27b63f1068dc5f796a297bcef0576cf020b3ddaf8  paper_outputs/tables/tab_supporting_results.tex

artifacts/manifests/paper_outputs.yml

@@ -0,0 +1,60 @@
+figures:
+  fig:toy_occupancy_contract:
+    output: paper_outputs/figures/matching.pdf
+    type: static_vector
+  fig:task_contract_tiles:
+    output: paper_outputs/figures/fig_task_contract_tiles.pdf
+    type: static_vector
+  fig:selection_regret_diagnostic:
+    output: paper_outputs/figures/fig_selection_regret_rq2.tikz
+    builder: scripts/build_selection_regret_rq2_figure.py
+    inputs:
+      - artifacts/results/selection_regret_scope_sweep_20260505.csv
+  fig:fireprone_contract_progression:
+    output: paper_outputs/figures/fig_fireprone_contract_progression_compact.pdf
+    builder: scripts/build_fireprone_contract_progression_figure.py
+    inputs:
+      - artifacts/results/fireprone_contract_progression_summary.json
+  fig:task_comparator_normalized_map:
+    output: paper_outputs/figures/fig_task_rank_map.pdf
+    builder: scripts/build_task_rank_map.py
+    inputs:
+      - paper_outputs/tables/tab_primary_results.tex
+      - paper_outputs/tables/tab_supporting_results.tex
+tables:
+  tab:primary_results:
+    output: paper_outputs/tables/tab_primary_results.tex
+  tab:supporting_results:
+    output: paper_outputs/tables/tab_supporting_results.tex
+  tab:app_matching_rule_params:
+    output: paper_outputs/tables/tab_app_matching_rule_params.tex
+  tab:app_contract_params_full:
+    output: paper_outputs/tables/tab_app_contract_params_full.tex
+  tab:app_scope_params:
+    output: paper_outputs/tables/tab_app_scope_params.tex
+  tab:fireprone_contract_progression:
+    output: paper_outputs/tables/tab_fireprone_contract_progression.tex
+    inputs:
+      - artifacts/results/fireprone_contract_progression_summary.json
+  tab:appendix_selection_regret_tolerance:
+    output: paper_outputs/tables/tab_appendix_selection_regret_tolerance.tex
+    inputs:
+      - artifacts/results/selection_regret_all_backbones_20260504.csv
+  tab:app_occupancy_ppr_scope:
+    output: paper_outputs/tables/tab_app_occupancy_ppr_scope.tex
+    inputs:
+      - artifacts/results/fireprone_contract_progression_summary.json
+  tab:app_spread_ap_by_scope:
+    output: paper_outputs/tables/tab_app_spread_ap_by_scope.tex
+  tab:app_burned_area_median_acre:
+    output: paper_outputs/tables/tab_app_burned_area_median_acre.tex
+  tab:app_analog_rank_depth:
+    output: paper_outputs/tables/tab_app_analog_rank_depth.tex
+  tab:app_smoke_high_event:
+    output: paper_outputs/tables/tab_app_smoke_high_event.tex
+  tab:app_heat_event_pr:
+    output: paper_outputs/tables/tab_app_heat_event_pr.tex
+  tab:app_seed_robustness:
+    output: paper_outputs/tables/tab_app_seed_robustness.tex
+  tab:app_head_architectures:
+    output: paper_outputs/tables/tab_app_head_architectures.tex

artifacts/results/fireprone_contract_progression_table.generated.tex

@@ -0,0 +1,69 @@
+\begin{table*}[t]
+    \centering
+    \scriptsize
+    \setlength{\tabcolsep}{4pt}
+    \caption{Occupancy scores across global and fire-prone scopes. Global uses the full validation/test domain; top-\(k\) rows use train-defined fire-prone masks from historical fire frequency. Values are \(F_1\) percentages from the same validation-selected strict threshold. Tolerance is spatial-only; union adds temporal and spatial matching. Difference is union minus strict. Rows report five-seed mean with small std. Values use four decimals.}
+    \label{tab:fireprone_contract_progression}
+    \begin{adjustbox}{max width=\textwidth}
+    \begin{tabular}{@{}llcccc@{}}
+        \toprule
+        Backbone & Scope & Strict \(F_1\uparrow\) & Tolerance \(F_1\uparrow\) & Union \(F_1\uparrow\) & Difference \(\uparrow\) \\
+        \midrule
+        \textcolor{blue}{FireWx-FM ref.} & global & \ms{0.4550}{0.1410} & \ms{29.7480}{1.2870} & \ms{59.0660}{2.7370} & \ms{58.6110}{2.6950} \\
+         & top 5\% & \ms{3.5600}{0.8810} & \ms{39.2620}{1.4010} & \ms{72.8280}{2.5780} & \ms{69.2680}{1.9960} \\
+         & top 10\% & \ms{3.5580}{0.8800} & \ms{39.1660}{1.3910} & \ms{72.5200}{2.5670} & \ms{68.9630}{1.9890} \\
+         & top 20\% & \ms{3.5300}{0.8700} & \ms{38.2850}{1.2950} & \ms{69.7230}{2.4660} & \ms{66.1930}{1.9270} \\
+        \addlinespace[1pt]
+        Prithvi-WxC & global & \ms{0.0550}{0.0040} & \ms{7.1600}{0.6600} & \ms{20.1900}{1.8300} & \ms{20.1300}{1.8300} \\
+         & top 5\% & \ms{1.4100}{1.1600} & \ms{19.2600}{4.5000} & \ms{42.5800}{4.5500} & \ms{41.1700}{3.4800} \\
+         & top 10\% & \ms{1.2400}{1.3200} & \ms{14.8800}{8.4400} & \ms{32.6900}{13.2100} & \ms{31.4500}{11.9100} \\
+         & top 20\% & \ms{1.1500}{1.3800} & \ms{13.1500}{9.4600} & \ms{28.1300}{15.2900} & \ms{26.9800}{13.9200} \\
+        \addlinespace[1pt]
+        Aurora & global & \ms{0.0700}{0.0100} & \ms{8.5000}{1.9600} & \ms{23.1000}{4.9400} & \ms{23.0400}{4.9300} \\
+         & top 5\% & \ms{0.9900}{0.9300} & \ms{15.1300}{6.0800} & \ms{35.4800}{11.0200} & \ms{34.5000}{10.3700} \\
+         & top 10\% & \ms{0.7800}{1.0500} & \ms{12.7400}{6.5600} & \ms{30.5300}{10.8800} & \ms{29.7500}{9.8700} \\
+         & top 20\% & \ms{0.6700}{1.1000} & \ms{10.5300}{7.4300} & \ms{24.9400}{12.5800} & \ms{24.2800}{11.4900} \\
+        \addlinespace[1pt]
+        ClimaX & global & \ms{0.3500}{0.0800} & \ms{29.7500}{3.6100} & \ms{60.1500}{7.5900} & \ms{59.8000}{7.5500} \\
+         & top 5\% & \ms{1.2900}{0.1100} & \ms{34.5800}{2.3800} & \ms{69.2200}{5.7200} & \ms{67.9200}{5.7300} \\
+         & top 10\% & \ms{1.2500}{0.1600} & \ms{34.3300}{2.2900} & \ms{68.5700}{5.5400} & \ms{67.3200}{5.5500} \\
+         & top 20\% & \ms{1.0300}{0.2700} & \ms{30.2100}{4.2900} & \ms{60.0600}{7.5700} & \ms{59.0400}{7.5900} \\
+        \addlinespace[1pt]
+        StormCast & global & \ms{0.0560}{0.0110} & \ms{8.2000}{2.1900} & \ms{22.3800}{5.4300} & \ms{22.3200}{5.4200} \\
+         & top 5\% & \ms{0.9600}{0.8000} & \ms{15.3200}{5.5300} & \ms{36.1900}{9.7300} & \ms{35.2300}{9.1800} \\
+         & top 10\% & \ms{0.7300}{0.9300} & \ms{12.6700}{6.3300} & \ms{30.4700}{10.6500} & \ms{29.7500}{9.7500} \\
+         & top 20\% & \ms{0.5800}{0.9100} & \ms{10.4200}{7.3400} & \ms{24.6600}{12.4000} & \ms{24.0800}{11.5000} \\
+        \addlinespace[1pt]
+        AlphaEarth & global & \ms{2.0600}{0.4400} & \ms{29.4500}{6.0100} & \ms{37.4300}{9.9500} & \ms{35.3700}{10.0300} \\
+         & top 5\% & \ms{6.9100}{0.8500} & \ms{42.8800}{4.6100} & \ms{51.7400}{8.7300} & \ms{44.8300}{9.0800} \\
+         & top 10\% & \ms{6.6400}{0.9900} & \ms{41.9000}{5.9500} & \ms{50.5700}{10.0100} & \ms{43.9300}{9.9200} \\
+         & top 20\% & \ms{6.1900}{1.1300} & \ms{38.8300}{7.5000} & \ms{46.3800}{12.1700} & \ms{40.1900}{11.6800} \\
+        \addlinespace[1pt]
+        DLWP & global & \ms{0.1700}{0.0400} & \ms{14.9100}{3.2400} & \ms{28.1900}{6.9700} & \ms{28.0200}{6.9300} \\
+         & top 5\% & \ms{1.8100}{0.4800} & \ms{31.7200}{3.2900} & \ms{55.4600}{5.2900} & \ms{53.6500}{5.4800} \\
+         & top 10\% & \ms{1.6100}{0.6000} & \ms{27.6600}{5.9200} & \ms{47.1300}{8.0100} & \ms{45.5200}{7.7900} \\
+         & top 20\% & \ms{1.5200}{0.9000} & \ms{20.9400}{4.8000} & \ms{34.9300}{7.8500} & \ms{33.4100}{7.8800} \\
+        \addlinespace[1pt]
+        FCN & global & \ms{0.2800}{0.0800} & \ms{19.5100}{3.3400} & \ms{40.0600}{9.3700} & \ms{39.7800}{9.3400} \\
+         & top 5\% & \ms{1.6200}{0.5100} & \ms{29.3800}{2.7600} & \ms{54.3000}{7.4100} & \ms{52.6800}{7.4400} \\
+         & top 10\% & \ms{1.1800}{0.5100} & \ms{22.4200}{3.9800} & \ms{43.4500}{9.2500} & \ms{42.2700}{9.0300} \\
+         & top 20\% & \ms{1.0000}{0.4300} & \ms{16.9800}{3.9400} & \ms{34.0900}{8.2600} & \ms{33.0900}{7.9300} \\
+        \addlinespace[1pt]
+        FengWu & global & \ms{0.2600}{0.0800} & \ms{12.0000}{6.0200} & \ms{24.1000}{13.6300} & \ms{23.8400}{13.5700} \\
+         & top 5\% & \ms{1.5700}{0.3600} & \ms{16.2800}{3.7000} & \ms{30.1100}{5.0100} & \ms{28.5400}{4.7700} \\
+         & top 10\% & \ms{1.2400}{0.5300} & \ms{12.9500}{5.6100} & \ms{24.1900}{8.6900} & \ms{22.9400}{8.1900} \\
+         & top 20\% & \ms{1.1200}{0.5000} & \ms{11.9500}{5.0700} & \ms{22.7900}{7.9100} & \ms{21.6700}{7.4400} \\
+        \addlinespace[1pt]
+        FuXi & global & \ms{0.3800}{0.1200} & \ms{21.0300}{4.8200} & \ms{37.2900}{9.4500} & \ms{36.9100}{9.4300} \\
+         & top 5\% & \ms{2.0300}{0.6800} & \ms{31.8900}{4.7300} & \ms{53.9300}{8.3800} & \ms{51.9000}{8.6900} \\
+         & top 10\% & \ms{1.6500}{0.7300} & \ms{24.0100}{5.7800} & \ms{40.2100}{9.9300} & \ms{38.5600}{9.7700} \\
+         & top 20\% & \ms{1.3600}{0.6800} & \ms{21.9500}{5.8600} & \ms{36.7300}{10.0300} & \ms{35.3700}{9.9200} \\
+        \addlinespace[1pt]
+        Pangu-Weather & global & \ms{0.2800}{0.1100} & \ms{17.0900}{4.0500} & \ms{35.6400}{9.0300} & \ms{35.3600}{9.0800} \\
+         & top 5\% & \ms{1.3700}{0.3100} & \ms{22.2200}{6.8600} & \ms{43.4200}{13.2400} & \ms{42.0600}{13.0600} \\
+         & top 10\% & \ms{1.0900}{0.3500} & \ms{18.9300}{5.9300} & \ms{38.5300}{11.7200} & \ms{37.4400}{11.5300} \\
+         & top 20\% & \ms{0.8800}{0.3600} & \ms{17.0200}{5.4900} & \ms{34.5700}{10.2900} & \ms{33.6800}{10.1300} \\
+        \bottomrule
+    \end{tabular}
+    \end{adjustbox}
+\end{table*}

artifacts/results/selection_regret_all_backbones_20260504.csv

@@ -0,0 +1,25 @@
+model_tag,label,scope,n,seeds,exact_regret_mean,exact_regret_std,tolerated_regret_mean,tolerated_regret_std,union_regret_mean,union_regret_std
+reference,Reference,global,5,1 7 42 99 123,0.0,0.0,0.08783024981138902,0.09670495645481135,0.08783024981138902,0.09670495645481135
+reference,Reference,fire_prone,5,1 7 42 99 123,0.0,0.0,0.03402707057672223,0.032044658643147844,0.03402707057672223,0.032044658643147844
+prithvi_wxc,Prithvi-WxC,global,5,1 7 42 99 123,0.0,0.0,0.0,0.0,0.0,0.0
+prithvi_wxc,Prithvi-WxC,fire_prone,5,1 7 42 99 123,0.0,0.0,0.0,0.0,0.0,0.0
+aurora,Aurora,global,5,1 7 42 99 123,0.00020004882767231798,0.00026703384456332115,0.09851983041506818,0.1298781980037557,0.09851983041506818,0.1298781980037557
+aurora,Aurora,fire_prone,5,1 7 42 99 123,0.008202508825959588,0.01834136732088763,0.14391889430974364,0.32121904665016227,0.14391889430974364,0.32121904665016227
+climax,ClimaX,global,5,1 7 42 99 123,3.0287686240700486e-06,4.147312242167625e-06,0.0012959969982639485,0.0017746169760203706,0.0012959969982639485,0.0017746169760203706
+climax,ClimaX,fire_prone,5,1 7 42 99 123,0.0,0.0,0.0,0.0,0.0,0.0
+stormcast,StormCast,global,5,1 7 42 99 123,0.0,0.0,0.0,0.0,0.0,0.0
+stormcast,StormCast,fire_prone,5,1 7 42 99 123,0.0,0.0,0.0,0.0,0.0,0.0
+pangu_weather,Pangu-Weather,global,5,1 7 42 99 123,0.00013033979247265275,0.0002685372203690466,0.048806713097574374,0.10733308684741971,0.048806713097574374,0.10733308684741971
+pangu_weather,Pangu-Weather,fire_prone,5,1 7 42 99 123,0.027875386332505546,0.02348779386900393,0.43111948243387105,0.39355644251497235,0.43111948243387105,0.39355644251497235
+dlwp,DLWP,global,5,1 7 42 99 123,0.0,0.0,0.0,0.0,0.0,0.0
+dlwp,DLWP,fire_prone,5,1 7 42 99 123,0.0007702319787454587,0.0010995336594539604,0.043265915053601556,0.04332331365579739,0.043265915053601556,0.04332331365579739
+fcn,FCN,global,5,1 7 42 99 123,0.0,0.0,0.0,0.0,0.0,0.0
+fcn,FCN,fire_prone,5,1 7 42 99 123,5.960229415004348e-06,1.3327478133443526e-05,0.011679805987441694,0.019872372458657642,0.011679805987441694,0.019872372458657642
+fengwu,FengWu,global,5,1 7 42 99 123,0.0,0.0,0.0,0.0,0.0,0.0
+fengwu,FengWu,fire_prone,5,1 7 42 99 123,0.0006908222234409067,0.0011910586589384115,0.005222389249812243,0.0062394095558402415,0.005222389249812243,0.0062394095558402415
+fuxi,FuXi,global,5,1 7 42 99 123,0.0,0.0,0.0,0.0,0.0,0.0
+fuxi,FuXi,fire_prone,5,1 7 42 99 123,0.0,0.0,0.0010839188523199318,0.0017288780545672386,0.0010839188523199318,0.0017288780545672386
+pangu6,Pangu-Weather,global,5,1 7 42 99 123,0.0,0.0,0.0,0.0,0.0,0.0
+pangu6,Pangu-Weather,fire_prone,5,1 7 42 99 123,0.0007280423771922354,0.001178746460551365,0.0018491271881979853,0.0032630386057089294,0.0018491271881979853,0.0032630386057089294
+alphaearth,AlphaEarth,global,5,1 7 42 99 123,0.0,0.0,0.1722171037486726,0.08849214830495522,0.1722171037486726,0.08849214830495522
+alphaearth,AlphaEarth,fire_prone,5,1 7 42 99 123,0.0,0.0,0.038803552655092256,0.0594825313313219,0.038803552655092256,0.0594825313313219

artifacts/results/selection_regret_full_head_table.generated.tex

@@ -0,0 +1,2 @@
+% Full per-head rows are kept in the supplementary CSV files.
+% The manuscript uses the all-backbone selection-regret summaries instead.

artifacts/results/selection_regret_head_metrics.csv

@@ -0,0 +1,241 @@
+family,model_tag,scope,seed,selected_by,head_label,test_pr_auc,exact_f1,tolerated_f1,union_f1
+AlphaEarth,alphaearth,fire_prone,1,PR-AUC,shallow spatial adapter,0.04390614036305261,0.09715762273901808,0.7863702028280513,0.7863702028280513
+AlphaEarth,alphaearth,fire_prone,1,decision,shallow spatial adapter,,,,0.7863702028280513
+AlphaEarth,alphaearth,fire_prone,7,PR-AUC,shallow spatial adapter,0.05955397140893137,0.12514898688915374,0.8294499693616161,0.8294499693616161
+AlphaEarth,alphaearth,fire_prone,7,decision,shallow spatial adapter,,,,0.8294499693616161
+AlphaEarth,alphaearth,fire_prone,42,PR-AUC,shallow spatial adapter,0.038083070948941686,0.08702469619756958,0.7112901458230849,0.7112901458230849
+AlphaEarth,alphaearth,fire_prone,42,decision,pixel MLP head,,,,0.8461131676361712
+AlphaEarth,alphaearth,fire_prone,99,PR-AUC,shallow spatial adapter,0.0458102699699856,0.10365251727541955,0.7758298037709835,0.7758298037709835
+AlphaEarth,alphaearth,fire_prone,99,decision,pixel MLP head,,,,0.8350245452333586
+AlphaEarth,alphaearth,fire_prone,123,PR-AUC,shallow spatial adapter,0.045809049876129763,0.10531544957774468,0.7789089693560928,0.7789089693560928
+AlphaEarth,alphaearth,fire_prone,123,decision,shallow spatial adapter,,,,0.7789089693560928
+AlphaEarth,alphaearth,global,1,PR-AUC,shallow spatial adapter,0.0006549130347299629,0.004193290734824281,0.40561891947698747,0.40561891947698747
+AlphaEarth,alphaearth,global,1,decision,pixel MLP head,,,,0.6337627266658229
+AlphaEarth,alphaearth,global,7,PR-AUC,shallow spatial adapter,0.001005722733868245,0.010460251046025104,0.6184842128568402,0.6184842128568402
+AlphaEarth,alphaearth,global,7,decision,pixel MLP head,,,,0.6691395427484861
+AlphaEarth,alphaearth,global,42,PR-AUC,shallow spatial adapter,0.0005634701573991865,0.004809747755451047,0.4087444681515033,0.4087444681515033
+AlphaEarth,alphaearth,global,42,decision,pixel MLP head,,,,0.6812131506751973
+AlphaEarth,alphaearth,global,99,PR-AUC,shallow spatial adapter,0.0006577120081349608,0.006780481898534931,0.3921547570095426,0.3921547570095426
+AlphaEarth,alphaearth,global,99,decision,pixel MLP head,,,,0.5842714676652996
+AlphaEarth,alphaearth,global,123,PR-AUC,shallow spatial adapter,0.0007047712457371991,0.006959088991986505,0.4427625907752311,0.4427625907752311
+AlphaEarth,alphaearth,global,123,decision,pixel MLP head,,,,0.5604635792586619
+Aurora,aurora,fire_prone,1,PR-AUC,linear probe,0.02093558282208589,0.04101254412979794,0.7185324707231184,0.7185324707231184
+Aurora,aurora,fire_prone,1,decision,linear probe,,,,0.7185324707231184
+Aurora,aurora,fire_prone,7,PR-AUC,linear probe,0.024802820904513342,0.0413500618483831,0.7184857293868923,0.7184857293868923
+Aurora,aurora,fire_prone,7,decision,shallow spatial adapter,,,,0.7185324707231184
+Aurora,aurora,fire_prone,42,PR-AUC,linear probe,0.02613792907867929,0.04101254412979794,0.7185324707231184,0.7185324707231184
+Aurora,aurora,fire_prone,42,decision,linear probe,,,,0.7185324707231184
+Aurora,aurora,fire_prone,99,PR-AUC,linear probe,0.02093558282208589,0.0,0.0,0.0
+Aurora,aurora,fire_prone,99,decision,shallow spatial adapter,,,,0.7185324707231184
+Aurora,aurora,fire_prone,123,PR-AUC,pixel MLP head,0.03014151567817997,0.04541038665445361,0.7175172112337449,0.7175172112337449
+Aurora,aurora,fire_prone,123,decision,linear probe,,,,0.7185324707231184
+Aurora,aurora,global,1,PR-AUC,linear probe,0.00024254792826221397,0.00048497822606044473,0.23755358049655212,0.23755358049655212
+Aurora,aurora,global,1,decision,shallow spatial adapter,,,,0.240793572992212
+Aurora,aurora,global,7,PR-AUC,linear probe,0.00027660331739269843,0.0,0.23734400297937533,0.23734400297937533
+Aurora,aurora,global,7,decision,shallow spatial adapter,,,,0.240793572992212
+Aurora,aurora,global,42,PR-AUC,linear probe,0.0002876372030063385,0.00048497822606044473,0.0,0.0
+Aurora,aurora,global,42,decision,shallow spatial adapter,,,,0.240793572992212
+Aurora,aurora,global,99,PR-AUC,linear probe,0.00024254792826221397,0.0,0.0,0.0
+Aurora,aurora,global,99,decision,shallow spatial adapter,,,,0.240793572992212
+Aurora,aurora,global,123,PR-AUC,pixel MLP head,0.00031683315961488916,0.0005311562430574265,0.23647112940979162,0.23647112940979162
+Aurora,aurora,global,123,decision,shallow spatial adapter,,,,0.240793572992212
+ClimaX,climax,fire_prone,1,PR-AUC,linear probe,0.02281272244151735,0.04101254412979794,0.7185324707231184,0.7185324707231184
+ClimaX,climax,fire_prone,1,decision,linear probe,,,,0.7185324707231184
+ClimaX,climax,fire_prone,7,PR-AUC,linear probe,0.021317405351800135,0.04101254412979794,0.7185324707231184,0.7185324707231184
+ClimaX,climax,fire_prone,7,decision,linear probe,,,,0.7185324707231184
+ClimaX,climax,fire_prone,42,PR-AUC,linear probe,0.021516770872035896,0.04101254412979794,0.7185324707231184,0.7185324707231184
+ClimaX,climax,fire_prone,42,decision,linear probe,,,,0.7185324707231184
+ClimaX,climax,fire_prone,99,PR-AUC,shallow spatial adapter,0.02099123219536693,0.04101254412979794,0.7185324707231184,0.7185324707231184
+ClimaX,climax,fire_prone,99,decision,shallow spatial adapter,,,,0.7185324707231184
+ClimaX,climax,fire_prone,123,PR-AUC,shallow spatial adapter,0.024930358757410707,0.04101254412979794,0.7185324707231184,0.7185324707231184
+ClimaX,climax,fire_prone,123,decision,shallow spatial adapter,,,,0.7185324707231184
+ClimaX,climax,global,1,PR-AUC,linear probe,0.0002543464414550104,0.00048497822606044473,0.23755358049655212,0.23755358049655212
+ClimaX,climax,global,1,decision,shallow spatial adapter,,,,0.240793572992212
+ClimaX,climax,global,7,PR-AUC,pixel MLP head,0.00025423546642937565,0.00048497822606044473,0.23755358049655212,0.23755358049655212
+ClimaX,climax,global,7,decision,shallow spatial adapter,,,,0.240793572992212
+ClimaX,climax,global,42,PR-AUC,shallow spatial adapter,0.00023723426605001756,0.0004925501476206198,0.240793572992212,0.240793572992212
+ClimaX,climax,global,42,decision,shallow spatial adapter,,,,0.240793572992212
+ClimaX,climax,global,99,PR-AUC,shallow spatial adapter,0.0002340075376021003,0.00048696535779102983,0.2384111045733381,0.2384111045733381
+ClimaX,climax,global,99,decision,shallow spatial adapter,,,,0.2384111045733381
+ClimaX,climax,global,123,PR-AUC,shallow spatial adapter,0.00025952340213823634,0.0004925501476206198,0.240793572992212,0.240793572992212
+ClimaX,climax,global,123,decision,shallow spatial adapter,,,,0.240793572992212
+DLWP,dlwp,fire_prone,1,PR-AUC,linear probe,0.019747545663020845,0.043506471331489265,0.7280364139105968,0.7280364139105968
+DLWP,dlwp,fire_prone,1,decision,linear probe,,,,0.7280364139105968
+DLWP,dlwp,fire_prone,7,PR-AUC,pixel MLP head,0.018519739310339497,0.04101254412979794,0.7185324707231184,0.7185324707231184
+DLWP,dlwp,fire_prone,7,decision,shallow spatial adapter,,,,0.762536895087842
+DLWP,dlwp,fire_prone,42,PR-AUC,pixel MLP head,0.020762153794205103,0.04637177602565815,0.6606900017471591,0.6606900017471591
+DLWP,dlwp,fire_prone,42,decision,shallow spatial adapter,,,,0.7728400679088107
+DLWP,dlwp,fire_prone,99,PR-AUC,linear probe,0.02136633936888583,0.04819843096725701,0.7187730423241409,0.7187730423241409
+DLWP,dlwp,fire_prone,99,decision,shallow spatial adapter,,,,0.7653346239087763
+DLWP,dlwp,fire_prone,123,PR-AUC,pixel MLP head,0.021118517500793188,0.04101254412979794,0.7185324707231184,0.7185324707231184
+DLWP,dlwp,fire_prone,123,decision,linear probe,,,,0.7321459738801157
+DLWP,dlwp,global,1,PR-AUC,shallow spatial adapter,0.0006257446338466172,0.00487022180273714,0.38023285660836226,0.38023285660836226
+DLWP,dlwp,global,1,decision,shallow spatial adapter,,,,0.38023285660836226
+DLWP,dlwp,global,7,PR-AUC,shallow spatial adapter,0.0005264872646085452,0.0,0.3432315705541329,0.3432315705541329
+DLWP,dlwp,global,7,decision,shallow spatial adapter,,,,0.3432315705541329
+DLWP,dlwp,global,42,PR-AUC,shallow spatial adapter,0.0006203713852571992,0.0,0.3405125814370199,0.3405125814370199
+DLWP,dlwp,global,42,decision,shallow spatial adapter,,,,0.3405125814370199
+DLWP,dlwp,global,99,PR-AUC,shallow spatial adapter,0.0007477128447471452,0.0,0.3979559626836394,0.3979559626836394
+DLWP,dlwp,global,99,decision,shallow spatial adapter,,,,0.3979559626836394
+DLWP,dlwp,global,123,PR-AUC,shallow spatial adapter,0.0007129763973023342,0.0,0.3797689460796109,0.3797689460796109
+DLWP,dlwp,global,123,decision,shallow spatial adapter,,,,0.3797689460796109
+FCN,fcn,fire_prone,1,PR-AUC,shallow spatial adapter,0.01844011667219625,0.042068766252528166,0.7185324707231184,0.7185324707231184
+FCN,fcn,fire_prone,1,decision,linear probe,,,,0.7182175622542595
+FCN,fcn,fire_prone,7,PR-AUC,pixel MLP head,0.02050876208409485,0.04101254412979794,0.7185324707231184,0.7185324707231184
+FCN,fcn,fire_prone,7,decision,linear probe,,,,0.7644129739607127
+FCN,fcn,fire_prone,42,PR-AUC,shallow spatial adapter,0.018030815062946615,0.0414596444738876,0.7197180735022655,0.7197180735022655
+FCN,fcn,fire_prone,42,decision,shallow spatial adapter,,,,0.7197180735022655
+FCN,fcn,fire_prone,99,PR-AUC,linear probe,0.029098665712304895,0.042822140550172624,0.726408418760773,0.726408418760773
+FCN,fcn,fire_prone,99,decision,linear probe,,,,0.726408418760773
+FCN,fcn,fire_prone,123,PR-AUC,pixel MLP head,0.019943278646881796,0.04101254412979794,0.7185324707231184,0.7185324707231184
+FCN,fcn,fire_prone,123,decision,linear probe,,,,0.7310509974227326
+FCN,fcn,global,1,PR-AUC,shallow spatial adapter,0.00037256097806901117,0.0009319664492078285,0.31167484413093016,0.31167484413093016
+FCN,fcn,global,1,decision,shallow spatial adapter,,,,0.31167484413093016
+FCN,fcn,global,7,PR-AUC,shallow spatial adapter,0.0003268363416054406,0.001086071137659517,0.3051941376005135,0.3051941376005135
+FCN,fcn,global,7,decision,shallow spatial adapter,,,,0.3051941376005135
+FCN,fcn,global,42,PR-AUC,shallow spatial adapter,0.00041063897933390575,0.0007027406886858749,0.31987973649439366,0.31987973649439366
+FCN,fcn,global,42,decision,pixel MLP head,,,,0.2870596305028149
+FCN,fcn,global,99,PR-AUC,shallow spatial adapter,0.00038120453362995967,0.0018159806295399514,0.3054145960271247,0.3054145960271247
+FCN,fcn,global,99,decision,shallow spatial adapter,,,,0.3054145960271247
+FCN,fcn,global,123,PR-AUC,shallow spatial adapter,0.000387412312932838,0.0006535947712418301,0.3096850885545486,0.3096850885545486
+FCN,fcn,global,123,decision,shallow spatial adapter,,,,0.3096850885545486
+FengWu,fengwu,fire_prone,1,PR-AUC,shallow spatial adapter,0.022736128443885992,0.04452825597664091,0.7269980510116651,0.7269980510116651
+FengWu,fengwu,fire_prone,1,decision,shallow spatial adapter,,,,0.7269980510116651
+FengWu,fengwu,fire_prone,7,PR-AUC,pixel MLP head,0.016801706970978273,0.04101254412979794,0.7185324707231184,0.7185324707231184
+FengWu,fengwu,fire_prone,7,decision,shallow spatial adapter,,,,0.722901134194411
+FengWu,fengwu,fire_prone,42,PR-AUC,shallow spatial adapter,0.021165185967854567,0.04574838388861263,0.7265705731122933,0.7265705731122933
+FengWu,fengwu,fire_prone,42,decision,shallow spatial adapter,,,,0.7265705731122933
+FengWu,fengwu,fire_prone,99,PR-AUC,linear probe,0.02199779031689959,0.0,0.7185324707231184,0.7185324707231184
+FengWu,fengwu,fire_prone,99,decision,shallow spatial adapter,,,,0.7336829717908426
+FengWu,fengwu,fire_prone,123,PR-AUC,pixel MLP head,0.020495144103834257,0.04101254412979794,0.7185324707231184,0.7185324707231184
+FengWu,fengwu,fire_prone,123,decision,shallow spatial adapter,,,,0.7251252524331628
+FengWu,fengwu,global,1,PR-AUC,shallow spatial adapter,0.000398077435365184,0.0,0.31071390711162444,0.31071390711162444
+FengWu,fengwu,global,1,decision,shallow spatial adapter,,,,0.31071390711162444
+FengWu,fengwu,global,7,PR-AUC,shallow spatial adapter,0.00036963872610239895,0.0,0.30641904273669174,0.30641904273669174
+FengWu,fengwu,global,7,decision,shallow spatial adapter,,,,0.30641904273669174
+FengWu,fengwu,global,42,PR-AUC,shallow spatial adapter,0.00036987379624400454,0.0,0.31219058559732665,0.31219058559732665
+FengWu,fengwu,global,42,decision,shallow spatial adapter,,,,0.31219058559732665
+FengWu,fengwu,global,99,PR-AUC,shallow spatial adapter,0.00042782651526874734,0.0,0.3111214819309595,0.3111214819309595
+FengWu,fengwu,global,99,decision,shallow spatial adapter,,,,0.3111214819309595
+FengWu,fengwu,global,123,PR-AUC,shallow spatial adapter,0.0004116035724473925,0.0,0.3145618361221859,0.3145618361221859
+FengWu,fengwu,global,123,decision,shallow spatial adapter,,,,0.3145618361221859
+FuXi,fuxi,fire_prone,1,PR-AUC,shallow spatial adapter,0.01904942261850253,0.043196160341303,0.7246636456247585,0.7246636456247585
+FuXi,fuxi,fire_prone,1,decision,linear probe,,,,0.7261195534617713
+FuXi,fuxi,fire_prone,7,PR-AUC,shallow spatial adapter,0.018717347905238046,0.044572342126298965,0.7235687421646468,0.7235687421646468
+FuXi,fuxi,fire_prone,7,decision,shallow spatial adapter,,,,0.7235687421646468
+FuXi,fuxi,fire_prone,42,PR-AUC,shallow spatial adapter,0.020363596550511454,0.04603830266616599,0.7054843984273774,0.7054843984273774
+FuXi,fuxi,fire_prone,42,decision,shallow spatial adapter,,,,0.7054843984273774
+FuXi,fuxi,fire_prone,99,PR-AUC,pixel MLP head,0.02225455497934009,0.030284377692970574,0.7203102915557309,0.7203102915557309
+FuXi,fuxi,fire_prone,99,decision,shallow spatial adapter,,,,0.7183426482806338
+FuXi,fuxi,fire_prone,123,PR-AUC,pixel MLP head,0.021045021724179047,0.018543768748295608,0.7185324707231184,0.7185324707231184
+FuXi,fuxi,fire_prone,123,decision,linear probe,,,,0.7224961571477053
+FuXi,fuxi,global,1,PR-AUC,shallow spatial adapter,0.0003596048414560045,0.0006866311182961118,0.3091927111996169,0.3091927111996169
+FuXi,fuxi,global,1,decision,linear probe,,,,0.21516044416153002
+FuXi,fuxi,global,7,PR-AUC,shallow spatial adapter,0.00037118462783325537,0.0008450613619815959,0.3062654921252336,0.3062654921252336
+FuXi,fuxi,global,7,decision,shallow spatial adapter,,,,0.3062654921252336
+FuXi,fuxi,global,42,PR-AUC,shallow spatial adapter,0.0003830459807690152,0.0003188165529554294,0.316668570748256,0.316668570748256
+FuXi,fuxi,global,42,decision,shallow spatial adapter,,,,0.316668570748256
+FuXi,fuxi,global,99,PR-AUC,shallow spatial adapter,0.0003758107890486513,0.0007771032641515431,0.3108110703043797,0.3108110703043797
+FuXi,fuxi,global,99,decision,shallow spatial adapter,,,,0.3108110703043797
+FuXi,fuxi,global,123,PR-AUC,shallow spatial adapter,0.00039999784430104664,0.0009888839378897987,0.3096924811079745,0.3096924811079745
+FuXi,fuxi,global,123,decision,shallow spatial adapter,,,,0.3096924811079745
+Pangu-Weather,pangu_weather,fire_prone,1,PR-AUC,pixel MLP head,0.014813375233921266,0.04101254412979794,0.7185324707231184,0.7185324707231184
+Pangu-Weather,pangu_weather,fire_prone,1,decision,pixel MLP head,,,,0.7185324707231184
+Pangu-Weather,pangu_weather,fire_prone,7,PR-AUC,pixel MLP head,0.029883397445312192,0.0,0.0,0.0
+Pangu-Weather,pangu_weather,fire_prone,7,decision,shallow spatial adapter,,,,0.7185324707231184
+Pangu-Weather,pangu_weather,fire_prone,42,PR-AUC,pixel MLP head,0.02188826028256454,0.0,0.0,0.0
+Pangu-Weather,pangu_weather,fire_prone,42,decision,shallow spatial adapter,,,,0.7185324707231184
+Pangu-Weather,pangu_weather,fire_prone,99,PR-AUC,linear probe,0.02093558282208589,0.0,0.0,0.0
+Pangu-Weather,pangu_weather,fire_prone,99,decision,shallow spatial adapter,,,,0.7185324707231184
+Pangu-Weather,pangu_weather,fire_prone,123,PR-AUC,linear probe,0.02093558282208589,0.04101254412979794,0.7185324707231184,0.7185324707231184
+Pangu-Weather,pangu_weather,fire_prone,123,decision,linear probe,,,,0.7185324707231184
+Pangu-Weather,pangu_weather,global,1,PR-AUC,linear probe,0.00024254792826221397,0.00048497822606044473,0.23755358049655212,0.23755358049655212
+Pangu-Weather,pangu_weather,global,1,decision,shallow spatial adapter,,,,0.240793572992212
+Pangu-Weather,pangu_weather,global,7,PR-AUC,shallow spatial adapter,0.00028045576992618193,0.0004925501476206198,0.240793572992212,0.240793572992212
+Pangu-Weather,pangu_weather,global,7,decision,shallow spatial adapter,,,,0.240793572992212
+Pangu-Weather,pangu_weather,global,42,PR-AUC,pixel MLP head,0.00021817709863716954,0.0,0.0,0.0
+Pangu-Weather,pangu_weather,global,42,decision,shallow spatial adapter,,,,0.240793572992212
+Pangu-Weather,pangu_weather,global,99,PR-AUC,shallow spatial adapter,0.0003408299850116487,0.0004887421693148924,0.23917716245227552,0.23917716245227552
+Pangu-Weather,pangu_weather,global,99,decision,pixel MLP head,,,,0.229804399271568
+Pangu-Weather,pangu_weather,global,123,PR-AUC,shallow spatial adapter,0.0003300754798792237,0.0004925501476206198,0.240793572992212,0.240793572992212
+Pangu-Weather,pangu_weather,global,123,decision,shallow spatial adapter,,,,0.240793572992212
+Pangu-Weather,pangu6,fire_prone,1,PR-AUC,shallow spatial adapter,0.02328829578760066,0.04418397717161179,0.7231388400090598,0.7231388400090598
+Pangu-Weather,pangu6,fire_prone,1,decision,shallow spatial adapter,,,,0.7231388400090598
+Pangu-Weather,pangu6,fire_prone,7,PR-AUC,pixel MLP head,0.018664183428008234,0.04101254412979794,0.7185324707231184,0.7185324707231184
+Pangu-Weather,pangu6,fire_prone,7,decision,shallow spatial adapter,,,,0.719461376864001
+Pangu-Weather,pangu6,fire_prone,42,PR-AUC,shallow spatial adapter,0.021420904714594617,0.0430065972893146,0.7241279907754397,0.7241279907754397
+Pangu-Weather,pangu6,fire_prone,42,decision,shallow spatial adapter,,,,0.7241279907754397
+Pangu-Weather,pangu6,fire_prone,99,PR-AUC,linear probe,0.024740444457605402,0.0044004400440044,0.7183382629739177,0.7183382629739177
+Pangu-Weather,pangu6,fire_prone,99,decision,shallow spatial adapter,,,,0.719015307962107
+Pangu-Weather,pangu6,fire_prone,123,PR-AUC,pixel MLP head,0.02254610440161657,0.04101254412979794,0.7185324707231184,0.7185324707231184
+Pangu-Weather,pangu6,fire_prone,123,decision,linear probe,,,,0.7261721555350364
+Pangu-Weather,pangu6,global,1,PR-AUC,shallow spatial adapter,0.00042881385578730365,0.0,0.32066974948800614,0.32066974948800614
+Pangu-Weather,pangu6,global,1,decision,shallow spatial adapter,,,,0.32066974948800614
+Pangu-Weather,pangu6,global,7,PR-AUC,shallow spatial adapter,0.00038395193539280824,0.0,0.31120670593952293,0.31120670593952293
+Pangu-Weather,pangu6,global,7,decision,shallow spatial adapter,,,,0.31120670593952293
+Pangu-Weather,pangu6,global,42,PR-AUC,shallow spatial adapter,0.00040651309469793043,0.0,0.32154249424354786,0.32154249424354786
+Pangu-Weather,pangu6,global,42,decision,shallow spatial adapter,,,,0.32154249424354786
+Pangu-Weather,pangu6,global,99,PR-AUC,shallow spatial adapter,0.0004450373086921994,0.0,0.3214547875801752,0.3214547875801752
+Pangu-Weather,pangu6,global,99,decision,shallow spatial adapter,,,,0.3214547875801752
+Pangu-Weather,pangu6,global,123,PR-AUC,shallow spatial adapter,0.00043738577276574255,0.0,0.31812983009681495,0.31812983009681495
+Pangu-Weather,pangu6,global,123,decision,shallow spatial adapter,,,,0.31812983009681495
+Prithvi-WxC,prithvi_wxc,fire_prone,1,PR-AUC,shallow spatial adapter,0.02093558282208589,0.04101254412979794,0.7185324707231184,0.7185324707231184
+Prithvi-WxC,prithvi_wxc,fire_prone,1,decision,shallow spatial adapter,,,,0.7185324707231184
+Prithvi-WxC,prithvi_wxc,fire_prone,7,PR-AUC,shallow spatial adapter,0.02093558282208589,0.04101254412979794,0.7185324707231184,0.7185324707231184
+Prithvi-WxC,prithvi_wxc,fire_prone,7,decision,shallow spatial adapter,,,,0.7185324707231184
+Prithvi-WxC,prithvi_wxc,fire_prone,42,PR-AUC,pixel MLP head,0.02093558282208589,0.04101254412979794,0.7185324707231184,0.7185324707231184
+Prithvi-WxC,prithvi_wxc,fire_prone,42,decision,shallow spatial adapter,,,,0.7185324707231184
+Prithvi-WxC,prithvi_wxc,fire_prone,99,PR-AUC,linear probe,0.02093558282208589,0.04101254412979794,0.7185324707231184,0.7185324707231184
+Prithvi-WxC,prithvi_wxc,fire_prone,99,decision,shallow spatial adapter,,,,0.7185324707231184
+Prithvi-WxC,prithvi_wxc,fire_prone,123,PR-AUC,shallow spatial adapter,0.02093558282208589,0.04101254412979794,0.7185324707231184,0.7185324707231184
+Prithvi-WxC,prithvi_wxc,fire_prone,123,decision,shallow spatial adapter,,,,0.7185324707231184
+Prithvi-WxC,prithvi_wxc,global,1,PR-AUC,shallow spatial adapter,0.0002463357401629007,0.0004925501476206198,0.240793572992212,0.240793572992212
+Prithvi-WxC,prithvi_wxc,global,1,decision,shallow spatial adapter,,,,0.240793572992212
+Prithvi-WxC,prithvi_wxc,global,7,PR-AUC,shallow spatial adapter,0.0002463357401629007,0.0004925501476206198,0.240793572992212,0.240793572992212
+Prithvi-WxC,prithvi_wxc,global,7,decision,shallow spatial adapter,,,,0.240793572992212
+Prithvi-WxC,prithvi_wxc,global,42,PR-AUC,shallow spatial adapter,0.0002463357401629007,0.0004925501476206198,0.240793572992212,0.240793572992212
+Prithvi-WxC,prithvi_wxc,global,42,decision,shallow spatial adapter,,,,0.240793572992212
+Prithvi-WxC,prithvi_wxc,global,99,PR-AUC,shallow spatial adapter,0.0002454330184381399,0.00048497822606044473,0.23755358049655212,0.23755358049655212
+Prithvi-WxC,prithvi_wxc,global,99,decision,shallow spatial adapter,,,,0.23755358049655212
+Prithvi-WxC,prithvi_wxc,global,123,PR-AUC,shallow spatial adapter,0.0002463357401629007,0.0004925501476206198,0.240793572992212,0.240793572992212
+Prithvi-WxC,prithvi_wxc,global,123,decision,shallow spatial adapter,,,,0.240793572992212
+Reference,reference,fire_prone,1,PR-AUC,shallow spatial adapter,0.10204224118176683,0.1611624834874505,0.799032457577039,0.799032457577039
+Reference,reference,fire_prone,1,decision,linear probe,,,,0.8685988450931266
+Reference,reference,fire_prone,7,PR-AUC,shallow spatial adapter,0.1323902067230726,0.22885572139303484,0.8027807546359551,0.8027807546359551
+Reference,reference,fire_prone,7,decision,shallow spatial adapter,,,,0.8027807546359551
+Reference,reference,fire_prone,42,PR-AUC,shallow spatial adapter,0.12048427320762313,0.19225806451612906,0.8358151221553631,0.8358151221553631
+Reference,reference,fire_prone,42,decision,shallow spatial adapter,,,,0.8358151221553631
+Reference,reference,fire_prone,99,PR-AUC,shallow spatial adapter,0.11947246238005743,0.19477124183006533,0.8534759193943048,0.8534759193943048
+Reference,reference,fire_prone,99,decision,linear probe,,,,0.9039923296574045
+Reference,reference,fire_prone,123,PR-AUC,shallow spatial adapter,0.11551882470432043,0.19165378670788252,0.8066689866810086,0.8066689866810086
+Reference,reference,fire_prone,123,decision,pixel MLP head,,,,0.8567215417854325
+Reference,reference,global,1,PR-AUC,shallow spatial adapter,0.002624341503354088,0.017134572294714646,0.6186566066408326,0.6186566066408326
+Reference,reference,global,1,decision,linear probe,,,,0.7286109603326707
+Reference,reference,global,7,PR-AUC,shallow spatial adapter,0.0030317345997110698,0.02287675150128682,0.5002193417313746,0.5002193417313746
+Reference,reference,global,7,decision,shallow spatial adapter,,,,0.5002193417313746
+Reference,reference,global,42,PR-AUC,shallow spatial adapter,0.0032729435045747413,0.025515210991167808,0.7239943741463363,0.7239943741463363
+Reference,reference,global,42,decision,shallow spatial adapter,,,,0.7239943741463363
+Reference,reference,global,99,PR-AUC,shallow spatial adapter,0.0031953098868323193,0.021876258220373104,0.6698005926442897,0.6698005926442897
+Reference,reference,global,99,decision,linear probe,,,,0.7647466876509988
+Reference,reference,global,123,PR-AUC,shallow spatial adapter,0.0025603887793133394,0.019320660641944532,0.5009707461135111,0.5009707461135111
+Reference,reference,global,123,decision,pixel MLP head,,,,0.735221546471909
+StormCast,stormcast,fire_prone,1,PR-AUC,linear probe,0.02093558282208589,0.04101254412979794,0.7185324707231184,0.7185324707231184
+StormCast,stormcast,fire_prone,1,decision,shallow spatial adapter,,,,0.7185324707231184
+StormCast,stormcast,fire_prone,7,PR-AUC,linear probe,0.02093558282208589,0.04101254412979794,0.7185324707231184,0.7185324707231184
+StormCast,stormcast,fire_prone,7,decision,shallow spatial adapter,,,,0.7185324707231184
+StormCast,stormcast,fire_prone,42,PR-AUC,linear probe,0.02093558282208589,0.04101254412979794,0.7185324707231184,0.7185324707231184
+StormCast,stormcast,fire_prone,42,decision,shallow spatial adapter,,,,0.7185324707231184
+StormCast,stormcast,fire_prone,99,PR-AUC,pixel MLP head,0.02093558282208589,0.04101254412979794,0.7185324707231184,0.7185324707231184
+StormCast,stormcast,fire_prone,99,decision,shallow spatial adapter,,,,0.7185324707231184
+StormCast,stormcast,fire_prone,123,PR-AUC,linear probe,0.02093558282208589,0.04101254412979794,0.7185324707231184,0.7185324707231184
+StormCast,stormcast,fire_prone,123,decision,shallow spatial adapter,,,,0.7185324707231184
+StormCast,stormcast,global,1,PR-AUC,shallow spatial adapter,0.0002463357401629007,0.0004925501476206198,0.240793572992212,0.240793572992212
+StormCast,stormcast,global,1,decision,shallow spatial adapter,,,,0.240793572992212
+StormCast,stormcast,global,7,PR-AUC,shallow spatial adapter,0.0002463357401629007,0.0004925501476206198,0.240793572992212,0.240793572992212
+StormCast,stormcast,global,7,decision,shallow spatial adapter,,,,0.240793572992212
+StormCast,stormcast,global,42,PR-AUC,shallow spatial adapter,0.0002463357401629007,0.0004925501476206198,0.240793572992212,0.240793572992212
+StormCast,stormcast,global,42,decision,shallow spatial adapter,,,,0.240793572992212
+StormCast,stormcast,global,99,PR-AUC,shallow spatial adapter,0.0002463357401629007,0.0004905319945803844,0.2399251001974223,0.2399251001974223
+StormCast,stormcast,global,99,decision,shallow spatial adapter,,,,0.2399251001974223
+StormCast,stormcast,global,123,PR-AUC,shallow spatial adapter,0.0002463357401629007,0.0004925501476206198,0.240793572992212,0.240793572992212
+StormCast,stormcast,global,123,decision,shallow spatial adapter,,,,0.240793572992212

artifacts/results/selection_regret_main_table.generated.tex

@@ -0,0 +1,24 @@
+\begin{table*}[!t]
+    \centering
+    \small
+    \setlength{\tabcolsep}{4pt}
+    \caption{Fixed-feature selection-regret check across evaluation scopes. Values are percentage-point regret \(\delta = D(h_D)-D(h_R)\) under union-\(F_1\), where \(h_R\) is selected by PR-AUC and \(h_D\) by the decision metric. Top-\(k\) columns use train-defined fire-prone scopes. Rows report mean with small std over five seeds; \(0.0000\) means the two selectors give the same decision score for all seeds.}
+    \label{tab:selection_regret_diagnostic}
+    \begin{tabular}{lcccc}
+        \toprule
+        \textbf{Feature source} & \textbf{\(\Omega=\)global} & \textbf{\(\Omega=\)top 5\%} & \textbf{\(\Omega=\)top 10\%} & \textbf{\(\Omega=\)top 20\%} \\
+        \midrule
+        \textcolor{blue}{FireWx-FM ref.} & \ms{7.3831}{7.4536} & \ms{0.3664}{0.6812} & \ms{1.2275}{1.2665} & \ms{2.9385}{2.7513} \\
+        Prithvi-WxC & 0.0000 & 0.0000 & 0.0000 & 0.0000 \\
+        Aurora & \ms{4.9455}{10.6974} & \ms{15.4283}{34.4987} & \ms{13.9934}{31.2903} & \ms{14.3706}{32.1337} \\
+        ClimaX & \ms{0.1296}{0.1775} & 0.0000 & 0.0000 & 0.0000 \\
+        StormCast & 0.0000 & 0.0000 & 0.0000 & 0.0000 \\
+        DLWP & 0.0000 & \ms{1.6716}{1.6079} & \ms{2.8465}{2.6938} & \ms{4.4634}{4.3561} \\
+        FCN & 0.0000 & \ms{0.4510}{1.0071} & \ms{0.4200}{0.9390} & \ms{1.1680}{1.9872} \\
+        FengWu & 0.0000 & \ms{0.8796}{0.5532} & \ms{0.4023}{0.5511} & \ms{0.5222}{0.6239} \\
+        FuXi & 0.0000 & \ms{1.3545}{2.0970} & \ms{0.1656}{0.3703} & \ms{0.2833}{0.3681} \\
+        Pangu-Weather & 0.0000 & \ms{0.7593}{0.8974} & \ms{0.3048}{0.5054} & \ms{0.1868}{0.3255} \\
+        AlphaEarth & \ms{17.2217}{8.8492} & \ms{6.3846}{4.9653} & \ms{6.5738}{6.8970} & \ms{3.8804}{5.9483} \\
+        \bottomrule
+    \end{tabular}
+\end{table*}

artifacts/results/selection_regret_per_seed.csv

@@ -0,0 +1,121 @@
+family,model_tag,scope,seed,ranking_head,decision_head,ranking_pr_auc,ranking_union_f1,decision_union_f1,regret,top1_agreement
+AlphaEarth,alphaearth,fire_prone,1,shallow spatial adapter,shallow spatial adapter,0.04390614036305261,0.7863702028280513,0.7863702028280513,0.0,True
+AlphaEarth,alphaearth,fire_prone,7,shallow spatial adapter,shallow spatial adapter,0.05955397140893137,0.8294499693616161,0.8294499693616161,0.0,True
+AlphaEarth,alphaearth,fire_prone,42,shallow spatial adapter,pixel MLP head,0.038083070948941686,0.7112901458230849,0.8461131676361712,0.13482302181308625,False
+AlphaEarth,alphaearth,fire_prone,99,shallow spatial adapter,pixel MLP head,0.0458102699699856,0.7758298037709835,0.8350245452333586,0.05919474146237502,False
+AlphaEarth,alphaearth,fire_prone,123,shallow spatial adapter,shallow spatial adapter,0.045809049876129763,0.7789089693560928,0.7789089693560928,0.0,True
+AlphaEarth,alphaearth,global,1,shallow spatial adapter,pixel MLP head,0.0006549130347299629,0.40561891947698747,0.6337627266658229,0.22814380718883542,False
+AlphaEarth,alphaearth,global,7,shallow spatial adapter,pixel MLP head,0.001005722733868245,0.6184842128568402,0.6691395427484861,0.050655329891645895,False
+AlphaEarth,alphaearth,global,42,shallow spatial adapter,pixel MLP head,0.0005634701573991865,0.4087444681515033,0.6812131506751973,0.272468682523694,False
+AlphaEarth,alphaearth,global,99,shallow spatial adapter,pixel MLP head,0.0006577120081349608,0.3921547570095426,0.5842714676652996,0.19211671065575697,False
+AlphaEarth,alphaearth,global,123,shallow spatial adapter,pixel MLP head,0.0007047712457371991,0.4427625907752311,0.5604635792586619,0.11770098848343075,False
+Aurora,aurora,fire_prone,1,linear probe,linear probe,0.02093558282208589,0.7185324707231184,0.7185324707231184,0.0,True
+Aurora,aurora,fire_prone,7,linear probe,shallow spatial adapter,0.024802820904513342,0.7184857293868923,0.7185324707231184,4.674133622617482e-05,False
+Aurora,aurora,fire_prone,42,linear probe,linear probe,0.02613792907867929,0.7185324707231184,0.7185324707231184,0.0,True
+Aurora,aurora,fire_prone,99,linear probe,shallow spatial adapter,0.02093558282208589,0.0,0.7185324707231184,0.7185324707231184,False
+Aurora,aurora,fire_prone,123,pixel MLP head,linear probe,0.03014151567817997,0.7175172112337449,0.7185324707231184,0.0010152594893735323,False
+Aurora,aurora,global,1,linear probe,shallow spatial adapter,0.00024254792826221397,0.23755358049655212,0.240793572992212,0.0032399924956598714,False
+Aurora,aurora,global,7,linear probe,shallow spatial adapter,0.00027660331739269843,0.23734400297937533,0.240793572992212,0.0034495700128366613,False
+Aurora,aurora,global,42,linear probe,shallow spatial adapter,0.0002876372030063385,0.0,0.240793572992212,0.240793572992212,False
+Aurora,aurora,global,99,linear probe,shallow spatial adapter,0.00024254792826221397,0.0,0.240793572992212,0.240793572992212,False
+Aurora,aurora,global,123,pixel MLP head,shallow spatial adapter,0.00031683315961488916,0.23647112940979162,0.240793572992212,0.004322443582420371,False
+ClimaX,climax,fire_prone,1,linear probe,linear probe,0.02281272244151735,0.7185324707231184,0.7185324707231184,0.0,True
+ClimaX,climax,fire_prone,7,linear probe,linear probe,0.021317405351800135,0.7185324707231184,0.7185324707231184,0.0,True
+ClimaX,climax,fire_prone,42,linear probe,linear probe,0.021516770872035896,0.7185324707231184,0.7185324707231184,0.0,True
+ClimaX,climax,fire_prone,99,shallow spatial adapter,shallow spatial adapter,0.02099123219536693,0.7185324707231184,0.7185324707231184,0.0,True
+ClimaX,climax,fire_prone,123,shallow spatial adapter,shallow spatial adapter,0.024930358757410707,0.7185324707231184,0.7185324707231184,0.0,True
+ClimaX,climax,global,1,linear probe,shallow spatial adapter,0.0002543464414550104,0.23755358049655212,0.240793572992212,0.0032399924956598714,False
+ClimaX,climax,global,7,pixel MLP head,shallow spatial adapter,0.00025423546642937565,0.23755358049655212,0.240793572992212,0.0032399924956598714,False
+ClimaX,climax,global,42,shallow spatial adapter,shallow spatial adapter,0.00023723426605001756,0.240793572992212,0.240793572992212,0.0,True
+ClimaX,climax,global,99,shallow spatial adapter,shallow spatial adapter,0.0002340075376021003,0.2384111045733381,0.2384111045733381,0.0,True
+ClimaX,climax,global,123,shallow spatial adapter,shallow spatial adapter,0.00025952340213823634,0.240793572992212,0.240793572992212,0.0,True
+DLWP,dlwp,fire_prone,1,linear probe,linear probe,0.019747545663020845,0.7280364139105968,0.7280364139105968,0.0,True
+DLWP,dlwp,fire_prone,7,pixel MLP head,shallow spatial adapter,0.018519739310339497,0.7185324707231184,0.762536895087842,0.044004424364723516,False
+DLWP,dlwp,fire_prone,42,pixel MLP head,shallow spatial adapter,0.020762153794205103,0.6606900017471591,0.7728400679088107,0.11215006616165157,False
+DLWP,dlwp,fire_prone,99,linear probe,shallow spatial adapter,0.02136633936888583,0.7187730423241409,0.7653346239087763,0.046561581584635414,False
+DLWP,dlwp,fire_prone,123,pixel MLP head,linear probe,0.021118517500793188,0.7185324707231184,0.7321459738801157,0.013613503156997275,False
+DLWP,dlwp,global,1,shallow spatial adapter,shallow spatial adapter,0.0006257446338466172,0.38023285660836226,0.38023285660836226,0.0,True
+DLWP,dlwp,global,7,shallow spatial adapter,shallow spatial adapter,0.0005264872646085452,0.3432315705541329,0.3432315705541329,0.0,True
+DLWP,dlwp,global,42,shallow spatial adapter,shallow spatial adapter,0.0006203713852571992,0.3405125814370199,0.3405125814370199,0.0,True
+DLWP,dlwp,global,99,shallow spatial adapter,shallow spatial adapter,0.0007477128447471452,0.3979559626836394,0.3979559626836394,0.0,True
+DLWP,dlwp,global,123,shallow spatial adapter,shallow spatial adapter,0.0007129763973023342,0.3797689460796109,0.3797689460796109,0.0,True
+FCN,fcn,fire_prone,1,shallow spatial adapter,linear probe,0.01844011667219625,0.7185324707231184,0.7182175622542595,0.0,False
+FCN,fcn,fire_prone,7,pixel MLP head,linear probe,0.02050876208409485,0.7185324707231184,0.7644129739607127,0.045880503237594294,False
+FCN,fcn,fire_prone,42,shallow spatial adapter,shallow spatial adapter,0.018030815062946615,0.7197180735022655,0.7197180735022655,0.0,True
+FCN,fcn,fire_prone,99,linear probe,linear probe,0.029098665712304895,0.726408418760773,0.726408418760773,0.0,True
+FCN,fcn,fire_prone,123,pixel MLP head,linear probe,0.019943278646881796,0.7185324707231184,0.7310509974227326,0.012518526699614174,False
+FCN,fcn,global,1,shallow spatial adapter,shallow spatial adapter,0.00037256097806901117,0.31167484413093016,0.31167484413093016,0.0,True
+FCN,fcn,global,7,shallow spatial adapter,shallow spatial adapter,0.0003268363416054406,0.3051941376005135,0.3051941376005135,0.0,True
+FCN,fcn,global,42,shallow spatial adapter,pixel MLP head,0.00041063897933390575,0.31987973649439366,0.2870596305028149,0.0,False
+FCN,fcn,global,99,shallow spatial adapter,shallow spatial adapter,0.00038120453362995967,0.3054145960271247,0.3054145960271247,0.0,True
+FCN,fcn,global,123,shallow spatial adapter,shallow spatial adapter,0.000387412312932838,0.3096850885545486,0.3096850885545486,0.0,True
+FengWu,fengwu,fire_prone,1,shallow spatial adapter,shallow spatial adapter,0.022736128443885992,0.7269980510116651,0.7269980510116651,0.0,True
+FengWu,fengwu,fire_prone,7,pixel MLP head,shallow spatial adapter,0.016801706970978273,0.7185324707231184,0.722901134194411,0.004368663471292611,False
+FengWu,fengwu,fire_prone,42,shallow spatial adapter,shallow spatial adapter,0.021165185967854567,0.7265705731122933,0.7265705731122933,0.0,True
+FengWu,fengwu,fire_prone,99,linear probe,shallow spatial adapter,0.02199779031689959,0.7185324707231184,0.7336829717908426,0.015150501067724198,False
+FengWu,fengwu,fire_prone,123,pixel MLP head,shallow spatial adapter,0.020495144103834257,0.7185324707231184,0.7251252524331628,0.006592781710044404,False
+FengWu,fengwu,global,1,shallow spatial adapter,shallow spatial adapter,0.000398077435365184,0.31071390711162444,0.31071390711162444,0.0,True
+FengWu,fengwu,global,7,shallow spatial adapter,shallow spatial adapter,0.00036963872610239895,0.30641904273669174,0.30641904273669174,0.0,True
+FengWu,fengwu,global,42,shallow spatial adapter,shallow spatial adapter,0.00036987379624400454,0.31219058559732665,0.31219058559732665,0.0,True
+FengWu,fengwu,global,99,shallow spatial adapter,shallow spatial adapter,0.00042782651526874734,0.3111214819309595,0.3111214819309595,0.0,True
+FengWu,fengwu,global,123,shallow spatial adapter,shallow spatial adapter,0.0004116035724473925,0.3145618361221859,0.3145618361221859,0.0,True
+FuXi,fuxi,fire_prone,1,shallow spatial adapter,linear probe,0.01904942261850253,0.7246636456247585,0.7261195534617713,0.001455907837012771,False
+FuXi,fuxi,fire_prone,7,shallow spatial adapter,shallow spatial adapter,0.018717347905238046,0.7235687421646468,0.7235687421646468,0.0,True
+FuXi,fuxi,fire_prone,42,shallow spatial adapter,shallow spatial adapter,0.020363596550511454,0.7054843984273774,0.7054843984273774,0.0,True
+FuXi,fuxi,fire_prone,99,pixel MLP head,shallow spatial adapter,0.02225455497934009,0.7203102915557309,0.7183426482806338,0.0,False
+FuXi,fuxi,fire_prone,123,pixel MLP head,linear probe,0.021045021724179047,0.7185324707231184,0.7224961571477053,0.0039636864245868875,False
+FuXi,fuxi,global,1,shallow spatial adapter,linear probe,0.0003596048414560045,0.3091927111996169,0.21516044416153002,0.0,False
+FuXi,fuxi,global,7,shallow spatial adapter,shallow spatial adapter,0.00037118462783325537,0.3062654921252336,0.3062654921252336,0.0,True
+FuXi,fuxi,global,42,shallow spatial adapter,shallow spatial adapter,0.0003830459807690152,0.316668570748256,0.316668570748256,0.0,True
+FuXi,fuxi,global,99,shallow spatial adapter,shallow spatial adapter,0.0003758107890486513,0.3108110703043797,0.3108110703043797,0.0,True
+FuXi,fuxi,global,123,shallow spatial adapter,shallow spatial adapter,0.00039999784430104664,0.3096924811079745,0.3096924811079745,0.0,True
+Pangu-Weather,pangu_weather,fire_prone,1,pixel MLP head,pixel MLP head,0.014813375233921266,0.7185324707231184,0.7185324707231184,0.0,True
+Pangu-Weather,pangu_weather,fire_prone,7,pixel MLP head,shallow spatial adapter,0.029883397445312192,0.0,0.7185324707231184,0.7185324707231184,False
+Pangu-Weather,pangu_weather,fire_prone,42,pixel MLP head,shallow spatial adapter,0.02188826028256454,0.0,0.7185324707231184,0.7185324707231184,False
+Pangu-Weather,pangu_weather,fire_prone,99,linear probe,shallow spatial adapter,0.02093558282208589,0.0,0.7185324707231184,0.7185324707231184,False
+Pangu-Weather,pangu_weather,fire_prone,123,linear probe,linear probe,0.02093558282208589,0.7185324707231184,0.7185324707231184,0.0,True
+Pangu-Weather,pangu_weather,global,1,linear probe,shallow spatial adapter,0.00024254792826221397,0.23755358049655212,0.240793572992212,0.0032399924956598714,False
+Pangu-Weather,pangu_weather,global,7,shallow spatial adapter,shallow spatial adapter,0.00028045576992618193,0.240793572992212,0.240793572992212,0.0,True
+Pangu-Weather,pangu_weather,global,42,pixel MLP head,shallow spatial adapter,0.00021817709863716954,0.0,0.240793572992212,0.240793572992212,False
+Pangu-Weather,pangu_weather,global,99,shallow spatial adapter,pixel MLP head,0.0003408299850116487,0.23917716245227552,0.229804399271568,0.0,False
+Pangu-Weather,pangu_weather,global,123,shallow spatial adapter,shallow spatial adapter,0.0003300754798792237,0.240793572992212,0.240793572992212,0.0,True
+Pangu-Weather,pangu6,fire_prone,1,shallow spatial adapter,shallow spatial adapter,0.02328829578760066,0.7231388400090598,0.7231388400090598,0.0,True
+Pangu-Weather,pangu6,fire_prone,7,pixel MLP head,shallow spatial adapter,0.018664183428008234,0.7185324707231184,0.719461376864001,0.0009289061408825905,False
+Pangu-Weather,pangu6,fire_prone,42,shallow spatial adapter,shallow spatial adapter,0.021420904714594617,0.7241279907754397,0.7241279907754397,0.0,True
+Pangu-Weather,pangu6,fire_prone,99,linear probe,shallow spatial adapter,0.024740444457605402,0.7183382629739177,0.719015307962107,0.0006770449881893237,False
+Pangu-Weather,pangu6,fire_prone,123,pixel MLP head,linear probe,0.02254610440161657,0.7185324707231184,0.7261721555350364,0.007639684811918013,False
+Pangu-Weather,pangu6,global,1,shallow spatial adapter,shallow spatial adapter,0.00042881385578730365,0.32066974948800614,0.32066974948800614,0.0,True
+Pangu-Weather,pangu6,global,7,shallow spatial adapter,shallow spatial adapter,0.00038395193539280824,0.31120670593952293,0.31120670593952293,0.0,True
+Pangu-Weather,pangu6,global,42,shallow spatial adapter,shallow spatial adapter,0.00040651309469793043,0.32154249424354786,0.32154249424354786,0.0,True
+Pangu-Weather,pangu6,global,99,shallow spatial adapter,shallow spatial adapter,0.0004450373086921994,0.3214547875801752,0.3214547875801752,0.0,True
+Pangu-Weather,pangu6,global,123,shallow spatial adapter,shallow spatial adapter,0.00043738577276574255,0.31812983009681495,0.31812983009681495,0.0,True
+Prithvi-WxC,prithvi_wxc,fire_prone,1,shallow spatial adapter,shallow spatial adapter,0.02093558282208589,0.7185324707231184,0.7185324707231184,0.0,True
+Prithvi-WxC,prithvi_wxc,fire_prone,7,shallow spatial adapter,shallow spatial adapter,0.02093558282208589,0.7185324707231184,0.7185324707231184,0.0,True
+Prithvi-WxC,prithvi_wxc,fire_prone,42,pixel MLP head,shallow spatial adapter,0.02093558282208589,0.7185324707231184,0.7185324707231184,0.0,False
+Prithvi-WxC,prithvi_wxc,fire_prone,99,linear probe,shallow spatial adapter,0.02093558282208589,0.7185324707231184,0.7185324707231184,0.0,False
+Prithvi-WxC,prithvi_wxc,fire_prone,123,shallow spatial adapter,shallow spatial adapter,0.02093558282208589,0.7185324707231184,0.7185324707231184,0.0,True
+Prithvi-WxC,prithvi_wxc,global,1,shallow spatial adapter,shallow spatial adapter,0.0002463357401629007,0.240793572992212,0.240793572992212,0.0,True
+Prithvi-WxC,prithvi_wxc,global,7,shallow spatial adapter,shallow spatial adapter,0.0002463357401629007,0.240793572992212,0.240793572992212,0.0,True
+Prithvi-WxC,prithvi_wxc,global,42,shallow spatial adapter,shallow spatial adapter,0.0002463357401629007,0.240793572992212,0.240793572992212,0.0,True
+Prithvi-WxC,prithvi_wxc,global,99,shallow spatial adapter,shallow spatial adapter,0.0002454330184381399,0.23755358049655212,0.23755358049655212,0.0,True
+Prithvi-WxC,prithvi_wxc,global,123,shallow spatial adapter,shallow spatial adapter,0.0002463357401629007,0.240793572992212,0.240793572992212,0.0,True
+Reference,reference,fire_prone,1,shallow spatial adapter,linear probe,0.10204224118176683,0.799032457577039,0.8685988450931266,0.06956638751608757,False
+Reference,reference,fire_prone,7,shallow spatial adapter,shallow spatial adapter,0.1323902067230726,0.8027807546359551,0.8027807546359551,0.0,True
+Reference,reference,fire_prone,42,shallow spatial adapter,shallow spatial adapter,0.12048427320762313,0.8358151221553631,0.8358151221553631,0.0,True
+Reference,reference,fire_prone,99,shallow spatial adapter,linear probe,0.11947246238005743,0.8534759193943048,0.9039923296574045,0.05051641026309972,False
+Reference,reference,fire_prone,123,shallow spatial adapter,pixel MLP head,0.11551882470432043,0.8066689866810086,0.8567215417854325,0.05005255510442386,False
+Reference,reference,global,1,shallow spatial adapter,linear probe,0.002624341503354088,0.6186566066408326,0.7286109603326707,0.10995435369183815,False
+Reference,reference,global,7,shallow spatial adapter,shallow spatial adapter,0.0030317345997110698,0.5002193417313746,0.5002193417313746,0.0,True
+Reference,reference,global,42,shallow spatial adapter,shallow spatial adapter,0.0032729435045747413,0.7239943741463363,0.7239943741463363,0.0,True
+Reference,reference,global,99,shallow spatial adapter,linear probe,0.0031953098868323193,0.6698005926442897,0.7647466876509988,0.09494609500670914,False
+Reference,reference,global,123,shallow spatial adapter,pixel MLP head,0.0025603887793133394,0.5009707461135111,0.735221546471909,0.23425080035839785,False
+StormCast,stormcast,fire_prone,1,linear probe,shallow spatial adapter,0.02093558282208589,0.7185324707231184,0.7185324707231184,0.0,False
+StormCast,stormcast,fire_prone,7,linear probe,shallow spatial adapter,0.02093558282208589,0.7185324707231184,0.7185324707231184,0.0,False
+StormCast,stormcast,fire_prone,42,linear probe,shallow spatial adapter,0.02093558282208589,0.7185324707231184,0.7185324707231184,0.0,False
+StormCast,stormcast,fire_prone,99,pixel MLP head,shallow spatial adapter,0.02093558282208589,0.7185324707231184,0.7185324707231184,0.0,False
+StormCast,stormcast,fire_prone,123,linear probe,shallow spatial adapter,0.02093558282208589,0.7185324707231184,0.7185324707231184,0.0,False
+StormCast,stormcast,global,1,shallow spatial adapter,shallow spatial adapter,0.0002463357401629007,0.240793572992212,0.240793572992212,0.0,True
+StormCast,stormcast,global,7,shallow spatial adapter,shallow spatial adapter,0.0002463357401629007,0.240793572992212,0.240793572992212,0.0,True
+StormCast,stormcast,global,42,shallow spatial adapter,shallow spatial adapter,0.0002463357401629007,0.240793572992212,0.240793572992212,0.0,True
+StormCast,stormcast,global,99,shallow spatial adapter,shallow spatial adapter,0.0002463357401629007,0.2399251001974223,0.2399251001974223,0.0,True
+StormCast,stormcast,global,123,shallow spatial adapter,shallow spatial adapter,0.0002463357401629007,0.240793572992212,0.240793572992212,0.0,True

artifacts/results/selection_regret_rq2_figure_values.csv

@@ -0,0 +1,12 @@
+feature_source,global_mean_pp,global_std_pp,top20_mean_pp,top20_std_pp
+FireWx-FM ref.,7.3831,7.4536,2.9385,2.7513
+Prithvi-WxC,0.0000,0.0000,0.0000,0.0000
+Aurora,4.9455,10.6974,14.3706,32.1337
+ClimaX,0.1296,0.1775,0.0000,0.0000
+StormCast,0.0000,0.0000,0.0000,0.0000
+DLWP,0.0000,0.0000,4.4634,4.3561
+FCN,0.0000,0.0000,1.1680,1.9872
+FengWu,0.0000,0.0000,0.5222,0.6239
+FuXi,0.0000,0.0000,0.2833,0.3681
+Pangu-Weather,0.0000,0.0000,0.1868,0.3255
+AlphaEarth,17.2217,8.8492,3.8804,5.9483

artifacts/results/selection_regret_scope_sweep_20260505.csv

@@ -0,0 +1,45 @@
+model_tag,label,scope,scope_label,n,seeds,exact_regret_mean,exact_regret_std,exact_regret_min,exact_regret_max,tolerated_regret_mean,tolerated_regret_std,tolerated_regret_min,tolerated_regret_max,union_regret_mean,union_regret_std,union_regret_min,union_regret_max
+reference,FireWx-FM ref.,global,\(\Omega=\)global,5,1 7 42 99 123,0.0,0.0,0.0,0.0,0.07383089600948442,0.07453636071372995,0.0,0.17497107865629713,0.07383089600948442,0.07453636071372995,0.0,0.17497107865629713
+reference,FireWx-FM ref.,top5,\(\Omega=\)top 5\%,5,1 7 42 99 123,0.0,0.0,0.0,0.0,0.003663718115532055,0.006812231244812292,0.0,0.015676762201120686,0.003663718115532055,0.006812231244812292,0.0,0.015676762201120686
+reference,FireWx-FM ref.,top10,\(\Omega=\)top 10\%,5,1 7 42 99 123,0.0,0.0,0.0,0.0,0.012275489592085752,0.012665162001740834,0.0,0.02670880922526031,0.012275489592085752,0.012665162001740834,0.0,0.02670880922526031
+reference,FireWx-FM ref.,top20,\(\Omega=\)top 20\%,5,1 7 42 99 123,0.0,0.0,0.0,0.0,0.029384646387840017,0.02751315335001922,0.0,0.05675140203555318,0.029384646387840017,0.02751315335001922,0.0,0.05675140203555318
+prithvi_wxc,Prithvi-WxC,global,\(\Omega=\)global,5,1 7 42 99 123,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0
+prithvi_wxc,Prithvi-WxC,top5,\(\Omega=\)top 5\%,5,1 7 42 99 123,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0
+prithvi_wxc,Prithvi-WxC,top10,\(\Omega=\)top 10\%,5,1 7 42 99 123,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0
+prithvi_wxc,Prithvi-WxC,top20,\(\Omega=\)top 20\%,5,1 7 42 99 123,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0
+aurora,Aurora,global,\(\Omega=\)global,5,1 7 42 99 123,0.00010153879814819402,0.00021861477435572763,0.0,0.0004925501476206198,0.04945471159670635,0.10697394238964528,0.0,0.240793572992212,0.04945471159670635,0.10697394238964528,0.0,0.240793572992212
+aurora,Aurora,top5,\(\Omega=\)top 5\%,5,1 7 42 99 123,0.023667154505540456,0.05292136630837888,0.0,0.11833577252770228,0.1542829840966487,0.34498724021162547,0.0,0.7714149204832434,0.1542829840966487,0.34498724021162547,0.0,0.7714149204832434
+aurora,Aurora,top10,\(\Omega=\)top 10\%,5,1 7 42 99 123,0.014651279478173606,0.032761256870543834,0.0,0.07325639739086803,0.1399343063221699,0.31290262132065055,0.0,0.6996715316108496,0.1399343063221699,0.31290262132065055,0.0,0.6996715316108496
+aurora,Aurora,top20,\(\Omega=\)top 20\%,5,1 7 42 99 123,0.008202508825959588,0.01834136732088763,0.0,0.04101254412979794,0.1437064941446237,0.32133748971555404,0.0,0.7185324707231184,0.1437064941446237,0.32133748971555404,0.0,0.7185324707231184
+climax,ClimaX,global,\(\Omega=\)global,5,1 7 42 99 123,3.0287686240700486e-06,4.147312242167625e-06,0.0,7.571921560175121e-06,0.0012959969982639485,0.0017746169760203706,0.0,0.0032399924956598714,0.0012959969982639485,0.0017746169760203706,0.0,0.0032399924956598714
+climax,ClimaX,top5,\(\Omega=\)top 5\%,5,1 7 42 99 123,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0
+climax,ClimaX,top10,\(\Omega=\)top 10\%,5,1 7 42 99 123,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0
+climax,ClimaX,top20,\(\Omega=\)top 20\%,5,1 7 42 99 123,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0
+stormcast,StormCast,global,\(\Omega=\)global,5,1 7 42 99 123,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0
+stormcast,StormCast,top5,\(\Omega=\)top 5\%,5,1 7 42 99 123,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0
+stormcast,StormCast,top10,\(\Omega=\)top 10\%,5,1 7 42 99 123,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0
+stormcast,StormCast,top20,\(\Omega=\)top 20\%,5,1 7 42 99 123,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0
+dlwp,DLWP,global,\(\Omega=\)global,5,1 7 42 99 123,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0
+dlwp,DLWP,top5,\(\Omega=\)top 5\%,5,1 7 42 99 123,0.0048037709215293075,0.006217185202152866,0.0,0.015203005078871956,0.016716228534155796,0.016079313546074458,0.0,0.03305057342744666,0.016716228534155796,0.016079313546074458,0.0,0.03305057342744666
+dlwp,DLWP,top10,\(\Omega=\)top 10\%,5,1 7 42 99 123,0.0017281632798742507,0.002514722758075371,0.0,0.005523780499856246,0.02846514801700826,0.026938012702643194,0.0,0.053927677500854476,0.02846514801700826,0.026938012702643194,0.0,0.053927677500854476
+dlwp,DLWP,top20,\(\Omega=\)top 20\%,5,1 7 42 99 123,0.0007702319787454587,0.0010995336594539604,0.0,0.0023651634514294945,0.04463354681768479,0.04356064433532197,0.0,0.11215006616165157,0.04463354681768479,0.04356064433532197,0.0,0.11215006616165157
+fcn,FCN,global,\(\Omega=\)global,5,1 7 42 99 123,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0
+fcn,FCN,top5,\(\Omega=\)top 5\%,5,1 7 42 99 123,0.0006342898232943345,0.0009899554165032742,0.0,0.002257520679520411,0.004509624980300697,0.010070611656609236,0.0,0.022524473456150496,0.004509624980300697,0.010070611656609236,0.0,0.022524473456150496
+fcn,FCN,top10,\(\Omega=\)top 10\%,5,1 7 42 99 123,0.00021156854817603877,0.0004730816556225618,0.0,0.0010578427408801938,0.004199537050817615,0.009390450319657174,0.0,0.020997685254088072,0.004199537050817615,0.009390450319657174,0.0,0.020997685254088072
+fcn,FCN,top20,\(\Omega=\)top 20\%,5,1 7 42 99 123,5.754560074337778e-06,1.2867587506825515e-05,0.0,2.877280037168889e-05,0.011679805987441694,0.019872372458657642,0.0,0.045880503237594294,0.011679805987441694,0.019872372458657642,0.0,0.045880503237594294
+fengwu,FengWu,global,\(\Omega=\)global,5,1 7 42 99 123,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0
+fengwu,FengWu,top5,\(\Omega=\)top 5\%,5,1 7 42 99 123,0.0005029843170376968,0.0008109166521114917,0.0,0.0018628094907809783,0.008795951947678215,0.005532321338017505,0.0,0.01484136735033148,0.008795951947678215,0.005532321338017505,0.0,0.01484136735033148
+fengwu,FengWu,top10,\(\Omega=\)top 10\%,5,1 7 42 99 123,0.000495228089292582,0.0007349190216431337,0.0,0.0016387212062008855,0.00402300475984525,0.005510851442075993,0.0,0.010273937098576491,0.00402300475984525,0.005510851442075993,0.0,0.010273937098576491
+fengwu,FengWu,top20,\(\Omega=\)top 20\%,5,1 7 42 99 123,0.0006908222234409067,0.0011910586589384115,0.0,0.0027505832409660327,0.005222389249812243,0.0062394095558402415,0.0,0.015150501067724198,0.005222389249812243,0.0062394095558402415,0.0,0.015150501067724198
+fuxi,FuXi,global,\(\Omega=\)global,5,1 7 42 99 123,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0
+fuxi,FuXi,top5,\(\Omega=\)top 5\%,5,1 7 42 99 123,0.002973545331200933,0.0023946274991058026,0.0010927807990139538,0.007024214143542151,0.013545122545609134,0.02097023683418404,0.0,0.050156261654859424,0.013545122545609134,0.02097023683418404,0.0,0.050156261654859424
+fuxi,FuXi,top10,\(\Omega=\)top 10\%,5,1 7 42 99 123,0.001383793743586542,0.0019248128430711165,0.0,0.003938013087198336,0.0016559834970027332,0.0037028916689159307,0.0,0.008279917485013666,0.0016559834970027332,0.0037028916689159307,0.0,0.008279917485013666
+fuxi,FuXi,top20,\(\Omega=\)top 20\%,5,1 7 42 99 123,0.0,0.0,0.0,0.0,0.00283318355751887,0.0036808289681375247,0.0,0.008746323525994693,0.00283318355751887,0.0036808289681375247,0.0,0.008746323525994693
+pangu6,Pangu-Weather,global,\(\Omega=\)global,5,1 7 42 99 123,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0
+pangu6,Pangu-Weather,top5,\(\Omega=\)top 5\%,5,1 7 42 99 123,0.003154674487241463,0.002546125713211599,0.0,0.005711594157251587,0.007592888149777122,0.00897418737588444,0.0,0.019790633919317124,0.007592888149777122,0.00897418737588444,0.0,0.019790633919317124
+pangu6,Pangu-Weather,top10,\(\Omega=\)top 10\%,5,1 7 42 99 123,0.0017345627303725214,0.0019305189318827886,0.0,0.004535321555179647,0.003047840737004992,0.005053805614558161,0.0,0.011660780793438352,0.003047840737004992,0.005053805614558161,0.0,0.011660780793438352
+pangu6,Pangu-Weather,top20,\(\Omega=\)top 20\%,5,1 7 42 99 123,0.0007280423771922354,0.001178746460551365,0.0,0.0027096086413018403,0.0018679847512695024,0.0032548337047755126,0.0,0.007639684811918013,0.0018679847512695024,0.0032548337047755126,0.0,0.007639684811918013
+alphaearth,AlphaEarth,global,\(\Omega=\)global,5,1 7 42 99 123,0.0,0.0,0.0,0.0,0.1722171037486726,0.08849214830495522,0.050655329891645895,0.272468682523694,0.1722171037486726,0.08849214830495522,0.050655329891645895,0.272468682523694
+alphaearth,AlphaEarth,top5,\(\Omega=\)top 5\%,5,1 7 42 99 123,0.0,0.0,0.0,0.0,0.06384618090125256,0.04965276403138872,0.0,0.1365277562230962,0.06384618090125256,0.04965276403138872,0.0,0.1365277562230962
+alphaearth,AlphaEarth,top10,\(\Omega=\)top 10\%,5,1 7 42 99 123,0.0,0.0,0.0,0.0,0.06573776411084173,0.06897015340160571,0.0,0.1615566566666954,0.06573776411084173,0.06897015340160571,0.0,0.1615566566666954
+alphaearth,AlphaEarth,top20,\(\Omega=\)top 20\%,5,1 7 42 99 123,0.0,0.0,0.0,0.0,0.038803552655092256,0.0594825313313219,0.0,0.13482302181308625,0.038803552655092256,0.0594825313313219,0.0,0.13482302181308625

artifacts/results/selection_regret_scope_sweep_20260505.generated.tex

@@ -0,0 +1,24 @@
+\begin{table*}[!t]
+    \centering
+    \small
+    \setlength{\tabcolsep}{4pt}
+    \caption{Fixed-feature selection-regret sweep across evaluation scopes. Values are percentage-point regret \(\delta = D(h_D)-D(h_R)\) under union-\(F_1\). Top-\(k\) scopes are train-defined fire-prone masks. Rows report mean with small std over five seeds.}
+    \label{tab:selection_regret_scope_sweep}
+    \begin{tabular}{lcccc}
+        \toprule
+        \textbf{Feature source} & \textbf{\(\Omega=\)global} & \textbf{\(\Omega=\)top 5\%} & \textbf{\(\Omega=\)top 10\%} & \textbf{\(\Omega=\)top 20\%} \\
+        \midrule
+        \textcolor{blue}{FireWx-FM ref.} & \ms{7.3831}{7.4536} & \ms{0.3664}{0.6812} & \ms{1.2275}{1.2665} & \ms{2.9385}{2.7513} \\
+        Prithvi-WxC & 0.0000 & 0.0000 & 0.0000 & 0.0000 \\
+        Aurora & \ms{4.9455}{10.6974} & \ms{15.4283}{34.4987} & \ms{13.9934}{31.2903} & \ms{14.3706}{32.1337} \\
+        ClimaX & \ms{0.1296}{0.1775} & 0.0000 & 0.0000 & 0.0000 \\
+        StormCast & 0.0000 & 0.0000 & 0.0000 & 0.0000 \\
+        DLWP & 0.0000 & \ms{1.6716}{1.6079} & \ms{2.8465}{2.6938} & \ms{4.4634}{4.3561} \\
+        FCN & 0.0000 & \ms{0.4510}{1.0071} & \ms{0.4200}{0.9390} & \ms{1.1680}{1.9872} \\
+        FengWu & 0.0000 & \ms{0.8796}{0.5532} & \ms{0.4023}{0.5511} & \ms{0.5222}{0.6239} \\
+        FuXi & 0.0000 & \ms{1.3545}{2.0970} & \ms{0.1656}{0.3703} & \ms{0.2833}{0.3681} \\
+        Pangu-Weather & 0.0000 & \ms{0.7593}{0.8974} & \ms{0.3048}{0.5054} & \ms{0.1868}{0.3255} \\
+        AlphaEarth & \ms{17.2217}{8.8492} & \ms{6.3846}{4.9653} & \ms{6.5738}{6.8970} & \ms{3.8804}{5.9483} \\
+        \bottomrule
+    \end{tabular}
+\end{table*}

artifacts/results/selection_regret_summary.csv

@@ -0,0 +1,25 @@
+model_tag,label,scope,n,seeds,exact_regret_mean,exact_regret_std,tolerated_regret_mean,tolerated_regret_std,union_regret_mean,union_regret_std
+reference,Reference,global,5,1 7 42 99 123,0.0,0.0,0.08783024981138902,0.09670495645481135,0.08783024981138902,0.09670495645481135
+reference,Reference,fire_prone,5,1 7 42 99 123,0.0,0.0,0.03402707057672223,0.032044658643147844,0.03402707057672223,0.032044658643147844
+prithvi_wxc,Prithvi-WxC,global,5,1 7 42 99 123,0.0,0.0,0.0,0.0,0.0,0.0
+prithvi_wxc,Prithvi-WxC,fire_prone,5,1 7 42 99 123,0.0,0.0,0.0,0.0,0.0,0.0
+aurora,Aurora,global,5,1 7 42 99 123,0.00020004882767231798,0.00026703384456332115,0.09851983041506818,0.1298781980037557,0.09851983041506818,0.1298781980037557
+aurora,Aurora,fire_prone,5,1 7 42 99 123,0.008202508825959588,0.01834136732088763,0.14391889430974364,0.32121904665016227,0.14391889430974364,0.32121904665016227
+climax,ClimaX,global,5,1 7 42 99 123,3.0287686240700486e-06,4.147312242167625e-06,0.0012959969982639485,0.0017746169760203706,0.0012959969982639485,0.0017746169760203706
+climax,ClimaX,fire_prone,5,1 7 42 99 123,0.0,0.0,0.0,0.0,0.0,0.0
+stormcast,StormCast,global,5,1 7 42 99 123,0.0,0.0,0.0,0.0,0.0,0.0
+stormcast,StormCast,fire_prone,5,1 7 42 99 123,0.0,0.0,0.0,0.0,0.0,0.0
+pangu_weather,Pangu-Weather,global,5,1 7 42 99 123,0.00013033979247265275,0.0002685372203690466,0.048806713097574374,0.10733308684741971,0.048806713097574374,0.10733308684741971
+pangu_weather,Pangu-Weather,fire_prone,5,1 7 42 99 123,0.027875386332505546,0.02348779386900393,0.43111948243387105,0.39355644251497235,0.43111948243387105,0.39355644251497235
+dlwp,DLWP,global,5,1 7 42 99 123,0.0,0.0,0.0,0.0,0.0,0.0
+dlwp,DLWP,fire_prone,5,1 7 42 99 123,0.0007702319787454587,0.0010995336594539604,0.043265915053601556,0.04332331365579739,0.043265915053601556,0.04332331365579739
+fcn,FCN,global,5,1 7 42 99 123,0.0,0.0,0.0,0.0,0.0,0.0
+fcn,FCN,fire_prone,5,1 7 42 99 123,5.960229415004348e-06,1.3327478133443526e-05,0.011679805987441694,0.019872372458657642,0.011679805987441694,0.019872372458657642
+fengwu,FengWu,global,5,1 7 42 99 123,0.0,0.0,0.0,0.0,0.0,0.0
+fengwu,FengWu,fire_prone,5,1 7 42 99 123,0.0006908222234409067,0.0011910586589384115,0.005222389249812243,0.0062394095558402415,0.005222389249812243,0.0062394095558402415
+fuxi,FuXi,global,5,1 7 42 99 123,0.0,0.0,0.0,0.0,0.0,0.0
+fuxi,FuXi,fire_prone,5,1 7 42 99 123,0.0,0.0,0.0010839188523199318,0.0017288780545672386,0.0010839188523199318,0.0017288780545672386
+pangu6,Pangu-Weather,global,5,1 7 42 99 123,0.0,0.0,0.0,0.0,0.0,0.0
+pangu6,Pangu-Weather,fire_prone,5,1 7 42 99 123,0.0007280423771922354,0.001178746460551365,0.0018491271881979853,0.0032630386057089294,0.0018491271881979853,0.0032630386057089294
+alphaearth,AlphaEarth,global,5,1 7 42 99 123,0.0,0.0,0.1722171037486726,0.08849214830495522,0.1722171037486726,0.08849214830495522
+alphaearth,AlphaEarth,fire_prone,5,1 7 42 99 123,0.0,0.0,0.038803552655092256,0.0594825313313219,0.038803552655092256,0.0594825313313219

artifacts/results/selection_regret_tolerance_family_table.generated.tex

@@ -0,0 +1,2 @@
+% Replaced by the all-backbone value table in sections/appendix.tex
+% (Table~\ref{tab:appendix_selection_regret_tolerance}).

data_sources/DATA_SOURCES.md

@@ -0,0 +1,27 @@
+# Data Sources
+
+This repository does not redistribute raw data. The table below records the
+resources used by the paper, their role in the experiments, and public access
+entry points. Users must obtain each source under its own terms.
+
+| Source | Role in paper | Public access entry point |
+|---|---|---|
+| NOAA High-Resolution Rapid Refresh (HRRR) | Dynamic weather fields for the California regional gridded occupancy inputs. | NOAA/NCEI HRRR product page: <https://www.ncei.noaa.gov/products/weather-climate-models/high-resolution-rapid-refresh>; AWS Open Data archive: <https://registry.opendata.aws/noaa-hrrr-pds/>. |
+| NASA FIRMS active-fire detections | Active-fire detections used to derive gridded occupancy labels. | FIRMS download and API services: <https://firms.modaps.eosdis.nasa.gov/download/> and <https://firms.modaps.eosdis.nasa.gov/api/>. |
+| LANDFIRE 40 Fire Behavior Fuel Models | Static fuel layer used in the FireWx-FM gridded input. | LANDFIRE data access portal: <https://landfire.gov/data>. |
+| LANDFIRE Forest Canopy Cover | Static canopy layer used in the FireWx-FM gridded input. | LANDFIRE data access portal: <https://landfire.gov/data>. |
+| Wildfire Risk to Communities housing-unit density | Static exposure layer used in the FireWx-FM gridded input. | Wildfire Risk to Communities data access: <https://wildfirerisk.org/download/>. |
+| LandScan Global 2024 | Static population layer used in the FireWx-FM gridded input. | Oak Ridge National Laboratory LandScan access: <https://landscan.ornl.gov/>. |
+| WFIGS incident/perimeter attributes | Event-level incident metadata for supporting burned-area and analog tasks. | NIFC Open Data portal for WFIGS layers: <https://data-nifc.opendata.arcgis.com/>. |
+| MTBS burned area and burn severity | Event-scale burned-area and burn-severity records for supporting tasks. | MTBS data access and direct download pages: <https://www.mtbs.gov/> and <https://www.mtbs.gov/direct-download>. |
+| Earth-FM/backbone sources | Frozen feature sources for transferred Earth-FM comparisons. | Original model providers and their terms. Examples include Hugging Face model cards, model-provider GitHub repositories, and provider-hosted model files. |
+
+## Notes
+
+- The paper places gridded resources on a projected 5 km EPSG:5070 grid.
+- The bundled artifacts contain summary values only. They are not a substitute
+  for the original data.
+- Full raw-data reruns require users to obtain each source independently and to
+  construct the intermediate grids/features described in the paper.
+- Access mechanisms and licensing can change. The links above are entry points,
+  not redistributed copies.

docs/artifact_map.md

@@ -0,0 +1,56 @@
+# Paper Artifact Map
+
+This map links every table and figure label in the current manuscript to the
+public release artifact and its provenance. Final output checksums are stored in
+`artifacts/manifests/paper_outputs.sha256`.
+
+## Figures
+
+| Paper label | Release file | Provenance |
+|---|---|---|
+| `fig:toy_occupancy_contract` | `paper_outputs/figures/matching.pdf` | Static vector schematic used by the manuscript. |
+| `fig:task_contract_tiles` | `paper_outputs/figures/fig_task_contract_tiles.pdf` | Static contract-map figure used by the manuscript. |
+| `fig:selection_regret_diagnostic` | `paper_outputs/figures/fig_selection_regret_rq2.tikz` | Rebuilt by `scripts/build_selection_regret_rq2_figure.py` from `artifacts/results/selection_regret_scope_sweep_20260505.csv`. |
+| `fig:fireprone_contract_progression` | `paper_outputs/figures/fig_fireprone_contract_progression_compact.pdf` | Rebuilt by `scripts/build_fireprone_contract_progression_figure.py` from `artifacts/results/fireprone_contract_progression_summary.json`. |
+| `fig:task_comparator_normalized_map` | `paper_outputs/figures/fig_task_rank_map.pdf` | Rebuilt by `scripts/build_task_rank_map.py` from `tab_primary_results.tex` and `tab_supporting_results.tex`. |
+
+## Main Tables
+
+| Paper label | Release file | Provenance |
+|---|---|---|
+| `tab:primary_results` | `paper_outputs/tables/tab_primary_results.tex` | Frozen paper-output TeX extracted from the current manuscript source and verified by checksum. Raw reruns require the task scripts and non-redistributed feature caches. |
+| `tab:supporting_results` | `paper_outputs/tables/tab_supporting_results.tex` | Frozen paper-output TeX extracted from the current manuscript source and verified by checksum. Raw reruns require the task scripts and non-redistributed feature caches. |
+
+## Appendix Tables
+
+| Paper label | Release file | Provenance |
+|---|---|---|
+| `tab:app_matching_rule_params` | `paper_outputs/tables/tab_app_matching_rule_params.tex` | Contract parameter table from manuscript source, verified by checksum. |
+| `tab:app_contract_params_full` | `paper_outputs/tables/tab_app_contract_params_full.tex` | Contract parameter table from manuscript source, verified by checksum. |
+| `tab:app_scope_params` | `paper_outputs/tables/tab_app_scope_params.tex` | Scope parameter table from manuscript source, verified by checksum. |
+| `tab:fireprone_contract_progression` | `paper_outputs/tables/tab_fireprone_contract_progression.tex` | Values from `artifacts/results/fireprone_contract_progression_summary.json`. |
+| `tab:appendix_selection_regret_tolerance` | `paper_outputs/tables/tab_appendix_selection_regret_tolerance.tex` | Values from selection-regret summary artifacts. |
+| `tab:app_occupancy_ppr_scope` | `paper_outputs/tables/tab_app_occupancy_ppr_scope.tex` | Values from `artifacts/results/fireprone_contract_progression_summary.json`. |
+| `tab:app_spread_ap_by_scope` | `paper_outputs/tables/tab_app_spread_ap_by_scope.tex` | Frozen paper-output TeX extracted from current manuscript source, verified by checksum. |
+| `tab:app_burned_area_median_acre` | `paper_outputs/tables/tab_app_burned_area_median_acre.tex` | Frozen paper-output TeX extracted from current manuscript source, verified by checksum. |
+| `tab:app_analog_rank_depth` | `paper_outputs/tables/tab_app_analog_rank_depth.tex` | Frozen paper-output TeX extracted from current manuscript source, verified by checksum. |
+| `tab:app_smoke_high_event` | `paper_outputs/tables/tab_app_smoke_high_event.tex` | Frozen paper-output TeX extracted from current manuscript source, verified by checksum. |
+| `tab:app_heat_event_pr` | `paper_outputs/tables/tab_app_heat_event_pr.tex` | Frozen paper-output TeX extracted from current manuscript source, verified by checksum. |
+| `tab:app_seed_robustness` | `paper_outputs/tables/tab_app_seed_robustness.tex` | Seed summary table from manuscript source, verified by checksum. |
+| `tab:app_head_architectures` | `paper_outputs/tables/tab_app_head_architectures.tex` | Architecture description table from manuscript source, verified by checksum. |
+
+## Reproduction Commands
+
+```bash
+python3 scripts/reproduce_paper_outputs.py
+```
+
+This command rebuilds the outputs that depend only on released summary files,
+checks all final paper-output hashes, and runs the release audit.
+
+## Raw Rerun Boundary
+
+Some tables depend on raw gridded data, event data, or backbone feature caches
+that are not redistributed. For public release, we provide the compact summary
+artifacts used to reproduce the displayed paper values and document the raw data
+sources separately.

docs/huggingface_release_design.md

@@ -0,0 +1,16 @@
+# Hugging Face Release Design
+
+This release follows the common Hugging Face pattern for research artifacts:
+
+- `README.md` is the public card. It contains YAML metadata, intended use,
+  limitations, data provenance, reproduction commands, and citation text.
+- `paper_outputs/` stores the final TeX, TikZ, and PDF artifacts used by the
+  manuscript.
+- `artifacts/results/` stores compact CSV/JSON summaries that can be public.
+- `artifacts/manifests/` maps paper labels to files and records output hashes.
+- `data_sources/` documents external data resources without redistributing them.
+- `experiments/` contains raw-rerun reference scripts and Slurm templates.
+
+The repository is intentionally a paper-artifact release rather than a dataset
+mirror or model-weight release. Full raw-data reruns require separately obtained
+source data and local feature caches.

experiments/README.md

@@ -0,0 +1,25 @@
+# Raw Rerun Notes
+
+This directory documents the raw rerun boundary. The public artifact release does
+not include local Slurm scripts with machine paths, raw wildfire inputs, or local
+feature caches. Full raw reruns require users to obtain the source data listed in
+`data_sources/DATA_SOURCES.md` and adapt the templates below to their own cluster.
+
+The bundled paper-output reproduction path does not require these raw reruns.
+
+## Reference Scripts
+
+The scripts under `raw_reference/` are sanitized references for the task-level
+runs used in the paper. They preserve the command-line interfaces and evaluation
+logic, but they require user-provided data tables, feature caches, and model
+dependencies.
+
+If a script imports local project modules from an external preprocessing tree,
+set `WILDFIRE_FM_EXTRA_PYTHONPATH` before running it:
+
+```bash
+export WILDFIRE_FM_EXTRA_PYTHONPATH=/path/to/your/project/src:/path/to/extra/site-packages
+```
+
+The Slurm file in `slurm/` is a template only. Replace all placeholder paths
+before submitting jobs on your own cluster.

experiments/raw_reference/run_selection_regret_scope_sweep_20260505.py

@@ -0,0 +1,335 @@
+#!/usr/bin/env python3
+"""Run fixed-feature head-selection regret for global and top-k fire-prone scopes."""
+
+from __future__ import annotations
+
+import argparse
+import csv
+import importlib.util
+import json
+import math
+from pathlib import Path
+from typing import Any
+
+import numpy as np
+
+
+BASE_RUNNER = Path(__file__).resolve().parent / "task_scripts" / "run_all_backbone_selection_regret_20260504.py"
+spec = importlib.util.spec_from_file_location("selection_regret_base_20260504", BASE_RUNNER)
+if spec is None or spec.loader is None:
+    raise RuntimeError(f"Cannot import base runner: {BASE_RUNNER}")
+base = importlib.util.module_from_spec(spec)
+spec.loader.exec_module(base)
+
+head_control = base.head_control
+
+SCOPE_FRACS = (0.05, 0.10, 0.20)
+SCOPE_ORDER = ("global", "top5", "top10", "top20")
+SCOPE_LABELS = {
+    "global": "global",
+    "top5": "top 5%",
+    "top10": "top 10%",
+    "top20": "top 20%",
+}
+
+
+def parse_args() -> argparse.Namespace:
+    parser = argparse.ArgumentParser(description="Selection-regret scope sweep.")
+    parser.add_argument("--source-kind", choices=("reference", "attached", "spatial", "alphaearth"), required=True)
+    parser.add_argument("--feature-root", type=Path, required=True)
+    parser.add_argument("--daily-rows-csv", type=Path)
+    parser.add_argument("--support-dir", type=Path)
+    parser.add_argument("--alphaearth-cache-root", type=Path)
+    parser.add_argument("--output-dir", type=Path, required=True)
+    parser.add_argument("--fm-family", type=str, required=True)
+    parser.add_argument("--model-tag", type=str, required=True)
+    parser.add_argument("--seed", type=int, required=True)
+    parser.add_argument("--heads", nargs="+", choices=base.HEADS, default=["linear", "pixel_mlp", "shallow"])
+    parser.add_argument("--batch-size", type=int, default=8)
+    parser.add_argument("--epochs", type=int, default=2)
+    parser.add_argument("--learning-rate", type=float, default=8e-4)
+    parser.add_argument("--weight-decay", type=float, default=1e-5)
+    parser.add_argument("--pos-weight-cap", type=float, default=150.0)
+    parser.add_argument("--device", choices=("cpu", "cuda", "auto"), default="cpu")
+    parser.add_argument(
+        "--metric-thresholds",
+        nargs="+",
+        type=float,
+        default=[
+            1e-5,
+            2e-5,
+            5e-5,
+            1e-4,
+            2e-4,
+            5e-4,
+            1e-3,
+            2e-3,
+            5e-3,
+            1e-2,
+            2e-2,
+            5e-2,
+            8e-2,
+            1e-1,
+            1.5e-1,
+            2e-1,
+            3e-1,
+            5e-1,
+        ],
+    )
+    parser.add_argument("--variants", nargs="+", default=["identity"])
+    parser.add_argument("--fire-prone-top-fracs", nargs="+", type=float, default=list(SCOPE_FRACS))
+    parser.add_argument("--temporal-steps", type=int, default=3)
+    parser.add_argument("--spatial-radius", type=int, default=8)
+    parser.add_argument("--buffer-radius", type=int, default=8)
+    parser.add_argument("--boundary-radius", type=int, default=8)
+    parser.add_argument("--coarse-factor", type=int, default=8)
+    parser.add_argument("--time-step-hours", type=int, default=6)
+    return parser.parse_args()
+
+
+def scope_name(top_frac: float) -> str:
+    pct = int(round(float(top_frac) * 100.0))
+    return f"top{pct}"
+
+
+def scope_label(top_frac: float) -> str:
+    pct = int(round(float(top_frac) * 100.0))
+    return f"top {pct}%"
+
+
+def build_scope_masks(
+    split_rows: dict[str, list[dict[str, str]]],
+    store: Any,
+    top_fracs: list[float],
+) -> tuple[dict[str, np.ndarray | None], dict[str, dict[str, Any]]]:
+    masks: dict[str, np.ndarray | None] = {"global": None}
+    meta: dict[str, dict[str, Any]] = {
+        "global": {
+            "scope_name": "global",
+            "reported_as": "global",
+            "top_fraction": None,
+        }
+    }
+    for frac in top_fracs:
+        name = scope_name(frac)
+        mask, mask_meta = head_control.build_fire_prone_mask(split_rows["train"], store, float(frac))
+        masks[name] = mask
+        meta[name] = {
+            "scope_name": name,
+            "reported_as": scope_label(frac),
+            **mask_meta,
+        }
+    return masks, meta
+
+
+def build_posthoc_rows_for_scopes(
+    probs: np.ndarray,
+    targets: np.ndarray,
+    sample_times: np.ndarray,
+    split: str,
+    scope_masks: dict[str, np.ndarray | None],
+    args: argparse.Namespace,
+) -> list[dict[str, object]]:
+    rows_out: list[dict[str, object]] = []
+    for threshold in [float(v) for v in args.metric_thresholds]:
+        base_binary = probs >= threshold
+        for variant in args.variants:
+            binary = head_control.apply_variant(base_binary, variant)
+            tensors = head_control.evaluate_threshold_variant(
+                binary_np=binary,
+                target_np=targets,
+                sample_times=sample_times,
+                time_step_hours=args.time_step_hours,
+                temporal_steps=args.temporal_steps,
+                spatial_radius=args.spatial_radius,
+                buffer_radius=args.buffer_radius,
+                boundary_radius=args.boundary_radius,
+                coarse_factor=args.coarse_factor,
+                tolerance_hours=args.temporal_steps * args.time_step_hours,
+            )
+            for scope, region_mask in scope_masks.items():
+                row: dict[str, object] = {
+                    "split": split,
+                    "scope": scope,
+                    "threshold": float(threshold),
+                    "variant": variant,
+                    "time_step_hours": int(args.time_step_hours),
+                    "temporal_steps": int(args.temporal_steps),
+                    "tolerance_hours": int(args.temporal_steps * args.time_step_hours),
+                    "spatial_radius": int(args.spatial_radius),
+                    "buffer_radius": int(args.buffer_radius),
+                    "boundary_radius": int(args.boundary_radius),
+                    "coarse_factor": int(args.coarse_factor),
+                }
+                row.update(head_control.metrics_for_scope(tensors, region_mask))
+                rows_out.append(row)
+    return rows_out
+
+
+def read_csv(path: Path) -> list[dict[str, str]]:
+    with path.open("r", encoding="utf-8", newline="") as fh:
+        return list(csv.DictReader(fh))
+
+
+def load_head_summary(
+    head_dir: Path,
+    head_arch: str,
+    scopes: tuple[str, ...],
+) -> tuple[list[dict[str, object]], dict[str, dict[str, float]], dict[str, object]] | None:
+    posthoc_path = head_dir / "posthoc_rows.csv"
+    summary_path = head_dir / "summary.json"
+    if not posthoc_path.exists() or not summary_path.exists():
+        return None
+    rows = [dict(row) for row in read_csv(posthoc_path)]
+    if not rows:
+        return None
+    try:
+        summary = json.loads(summary_path.read_text(encoding="utf-8"))
+    except json.JSONDecodeError:
+        return None
+    if str(summary.get("head_arch")) != str(head_arch):
+        return None
+    raw_pr_auc = summary.get("raw_pr_auc")
+    if not isinstance(raw_pr_auc, dict):
+        return None
+    try:
+        parsed_pr_auc = {
+            split: {scope: float(raw_pr_auc[split][scope]) for scope in scopes}
+            for split in ("val", "test")
+        }
+    except Exception:
+        return None
+    return rows, parsed_pr_auc, summary
+
+
+def finite_json(value: Any) -> Any:
+    if isinstance(value, float):
+        return value if math.isfinite(value) else None
+    if isinstance(value, dict):
+        return {key: finite_json(val) for key, val in value.items()}
+    if isinstance(value, list):
+        return [finite_json(val) for val in value]
+    return value
+
+
+def main() -> None:
+    args = parse_args()
+    args.output_dir.mkdir(parents=True, exist_ok=True)
+    base.set_seed(int(args.seed))
+    device = base.choose_device(args.device)
+
+    top_fracs = sorted({float(v) for v in args.fire_prone_top_fracs})
+    scope_order = ("global",) + tuple(scope_name(frac) for frac in top_fracs)
+    base.SCOPE_ORDER = scope_order
+
+    split_rows = {
+        split: base.read_rows(args.feature_root / "splits" / f"{split}.csv")
+        for split in ("train", "val", "test")
+    }
+    if args.source_kind == "reference":
+        store = base.build_reference_store(split_rows)
+    elif args.source_kind == "attached":
+        store = base.build_attached_store(args, split_rows)
+    elif args.source_kind == "spatial":
+        store = base.build_spatial_store(args, split_rows)
+    else:
+        store = base.build_alphaearth_store(args, split_rows)
+
+    loaders = base.make_loaders(split_rows, store, int(args.batch_size), device, int(args.seed))
+    first = next(iter(loaders["train"]))
+    in_ch = int(first["x"].shape[1])
+    prior_prob = base.total_positive_rate(split_rows["train"])
+    scope_masks, scope_meta = build_scope_masks(split_rows, store, top_fracs)
+
+    head_metrics: list[dict[str, object]] = []
+    head_artifacts: dict[str, str] = {}
+    for head_index, head_arch in enumerate(args.heads):
+        head_dir = args.output_dir / head_arch
+        head_dir.mkdir(parents=True, exist_ok=True)
+        cached = load_head_summary(head_dir, head_arch, scope_order)
+        if cached is not None:
+            posthoc_rows, raw_pr_auc, _ = cached
+            print(f"[scope-sweep] reuse {args.fm_family} seed={args.seed} head={head_arch}", flush=True)
+        else:
+            print(f"[scope-sweep] training {args.fm_family} seed={args.seed} head={head_arch}", flush=True)
+            model, history = base.train_one_head(
+                head_arch=head_arch,
+                in_ch=in_ch,
+                prior_prob=prior_prob,
+                loaders=loaders,
+                args=args,
+                device=device,
+                seed_offset=1009 * (head_index + 1),
+            )
+            posthoc_rows = []
+            raw_pr_auc: dict[str, dict[str, float]] = {}
+            for split in ("val", "test"):
+                probs, targets = base.collect_predictions(model, loaders[split], device)
+                sample_times = base.build_sample_times(split_rows[split])
+                raw_pr_auc[split] = {
+                    scope: head_control._masked_average_precision(probs, targets, region_mask=mask)
+                    for scope, mask in scope_masks.items()
+                }
+                posthoc_rows.extend(
+                    build_posthoc_rows_for_scopes(
+                        probs=probs,
+                        targets=targets,
+                        sample_times=sample_times,
+                        split=split,
+                        scope_masks=scope_masks,
+                        args=args,
+                    )
+                )
+            base.write_csv(posthoc_rows, head_dir / "posthoc_rows.csv")
+            head_summary = {
+                "head_arch": head_arch,
+                "head_label": head_control.HEAD_LABELS[head_arch],
+                "history": history,
+                "raw_pr_auc": raw_pr_auc,
+                "scope_meta": scope_meta,
+                "posthoc_rows_csv": str(head_dir / "posthoc_rows.csv"),
+            }
+            (head_dir / "summary.json").write_text(json.dumps(finite_json(head_summary), indent=2), encoding="utf-8")
+        head_artifacts[head_arch] = str(head_dir / "summary.json")
+        base.append_head_metrics(head_metrics, posthoc_rows, raw_pr_auc, head_arch, args)
+
+    selection_rows = base.summarize_head_scores(head_metrics)
+    for row in selection_rows:
+        row["model_tag"] = args.model_tag
+        row["family"] = args.fm_family
+        row["seed"] = int(args.seed)
+
+    base.write_csv(head_metrics, args.output_dir / "head_metrics.csv")
+    base.write_csv(selection_rows, args.output_dir / "selection_rows.csv")
+    summary = {
+        "experiment": "fixed-feature head-selection regret scope sweep",
+        "task": "wildfire_occupancy",
+        "model_tag": args.model_tag,
+        "fm_family": args.fm_family,
+        "source_kind": args.source_kind,
+        "seed": int(args.seed),
+        "feature_root": str(args.feature_root),
+        "daily_rows_csv": str(args.daily_rows_csv) if args.daily_rows_csv else None,
+        "support_dir": str(args.support_dir) if args.support_dir else None,
+        "alphaearth_cache_root": str(args.alphaearth_cache_root) if args.alphaearth_cache_root else None,
+        "device": str(device),
+        "heads": list(args.heads),
+        "scope_order": list(scope_order),
+        "scope_meta": scope_meta,
+        "input_channels": int(in_ch),
+        "prior_prob": float(prior_prob),
+        "metrics": base.METRICS,
+        "head_metrics": head_metrics,
+        "selection_rows": selection_rows,
+        "head_artifacts": head_artifacts,
+        "artifacts": {
+            "head_metrics_csv": str(args.output_dir / "head_metrics.csv"),
+            "selection_rows_csv": str(args.output_dir / "selection_rows.csv"),
+        },
+    }
+    (args.output_dir / "summary.json").write_text(json.dumps(finite_json(summary), indent=2), encoding="utf-8")
+    print(json.dumps(finite_json(summary), indent=2), flush=True)
+
+
+if __name__ == "__main__":
+    main()

experiments/raw_reference/task_scripts/run_all_backbone_selection_regret_20260504.py

@@ -0,0 +1,656 @@
+#!/usr/bin/env python3
+"""Run one fixed-feature head-selection regret job for one backbone and seed."""
+
+from __future__ import annotations
+
+import argparse
+import csv
+import json
+import math
+import random
+import sys
+from pathlib import Path
+from typing import Any
+
+import numpy as np
+import torch
+import torch.nn as nn
+from torch.utils.data import DataLoader, Dataset
+
+
+import os
+
+for _p in os.environ.get("WILDFIRE_FM_EXTRA_PYTHONPATH", "").split(os.pathsep):
+    if _p and _p not in sys.path:
+        sys.path.insert(0, _p)
+
+import run_alphaearth_occupancy_benchmark as alpha_runner  # noqa: E402
+import run_attached_daily_occupancy_head_control as head_control  # noqa: E402
+
+
+HEADS = ("constant", "linear", "pixel_mlp", "shallow", "shallow_wide")
+METRICS = {
+    "exact": "strict_f1",
+    "tolerated": "ts_f1",
+    "union": "comprehensive_union_f1",
+}
+SCOPE_ORDER = ("global", "fire_prone")
+
+
+def parse_args() -> argparse.Namespace:
+    parser = argparse.ArgumentParser(description="All-backbone fixed-feature head-selection regret.")
+    parser.add_argument("--source-kind", choices=("reference", "attached", "spatial", "alphaearth"), required=True)
+    parser.add_argument("--feature-root", type=Path, required=True)
+    parser.add_argument("--daily-rows-csv", type=Path)
+    parser.add_argument("--support-dir", type=Path)
+    parser.add_argument("--alphaearth-cache-root", type=Path)
+    parser.add_argument("--output-dir", type=Path, required=True)
+    parser.add_argument("--fm-family", type=str, required=True)
+    parser.add_argument("--model-tag", type=str, required=True)
+    parser.add_argument("--seed", type=int, required=True)
+    parser.add_argument("--heads", nargs="+", choices=HEADS, default=list(HEADS))
+    parser.add_argument("--batch-size", type=int, default=4)
+    parser.add_argument("--epochs", type=int, default=4)
+    parser.add_argument("--learning-rate", type=float, default=8e-4)
+    parser.add_argument("--weight-decay", type=float, default=1e-5)
+    parser.add_argument("--pos-weight-cap", type=float, default=150.0)
+    parser.add_argument("--device", choices=("cpu", "cuda", "auto"), default="auto")
+    parser.add_argument(
+        "--metric-thresholds",
+        nargs="+",
+        type=float,
+        default=[
+            1e-5,
+            2e-5,
+            5e-5,
+            1e-4,
+            2e-4,
+            5e-4,
+            1e-3,
+            2e-3,
+            5e-3,
+            1e-2,
+            2e-2,
+            5e-2,
+            8e-2,
+            1e-1,
+            1.5e-1,
+            2e-1,
+            3e-1,
+            5e-1,
+        ],
+    )
+    parser.add_argument(
+        "--variants",
+        nargs="+",
+        default=["identity", "erode_r1", "close_r1"],
+    )
+    parser.add_argument("--fire-prone-top-frac", type=float, default=0.20)
+    parser.add_argument("--temporal-steps", type=int, default=3)
+    parser.add_argument("--spatial-radius", type=int, default=8)
+    parser.add_argument("--buffer-radius", type=int, default=8)
+    parser.add_argument("--boundary-radius", type=int, default=8)
+    parser.add_argument("--coarse-factor", type=int, default=8)
+    parser.add_argument("--time-step-hours", type=int, default=6)
+    return parser.parse_args()
+
+
+def read_rows(path: Path) -> list[dict[str, str]]:
+    with path.open("r", encoding="utf-8", newline="") as fh:
+        return list(csv.DictReader(fh))
+
+
+def choose_device(value: str) -> torch.device:
+    if value == "auto":
+        return torch.device("cuda" if torch.cuda.is_available() else "cpu")
+    device = torch.device(value)
+    if device.type == "cuda" and not torch.cuda.is_available():
+        raise RuntimeError("CUDA requested but not available.")
+    return device
+
+
+def set_seed(seed: int) -> None:
+    random.seed(seed)
+    np.random.seed(seed)
+    torch.manual_seed(seed)
+    torch.cuda.manual_seed_all(seed)
+
+
+def total_positive_rate(rows: list[dict[str, str]]) -> float:
+    pos = float(sum(int(row["pos_cells"]) for row in rows))
+    arr = np.load(rows[0]["feature_path"], allow_pickle=True)
+    try:
+        total = float(len(rows) * np.squeeze(arr["y_occ"]).size)
+    finally:
+        arr.close()
+    return float(pos / total) if total > 0 else 0.0
+
+
+class Normalizer:
+    def __init__(self, mean: np.ndarray, std: np.ndarray) -> None:
+        self.mean = mean.astype(np.float32)
+        self.std = np.maximum(std.astype(np.float32), 1e-6)
+
+    def apply(self, x: np.ndarray) -> np.ndarray:
+        return (x.astype(np.float32) - self.mean[:, None, None]) / self.std[:, None, None]
+
+
+def compute_map_normalizer(rows: list[dict[str, str]], map_path_fn: Any) -> Normalizer:
+    sum_c: np.ndarray | None = None
+    sumsq_c: np.ndarray | None = None
+    count = 0
+    for row in rows:
+        arr = np.load(map_path_fn(row), allow_pickle=True)
+        try:
+            x = np.nan_to_num(arr["features"].astype(np.float32), nan=0.0, posinf=0.0, neginf=0.0)
+        finally:
+            arr.close()
+        flat = x.reshape(x.shape[0], -1)
+        if sum_c is None:
+            sum_c = np.zeros(x.shape[0], dtype=np.float64)
+            sumsq_c = np.zeros(x.shape[0], dtype=np.float64)
+        sum_c += flat.sum(axis=1)
+        sumsq_c += np.square(flat, dtype=np.float64).sum(axis=1)
+        count += flat.shape[1]
+    if sum_c is None or sumsq_c is None or count == 0:
+        raise RuntimeError("Cannot compute feature normalizer.")
+    mean = sum_c / float(count)
+    var = np.maximum(sumsq_c / float(count) - mean * mean, 1e-12)
+    return Normalizer(mean=mean.astype(np.float32), std=np.sqrt(var).astype(np.float32))
+
+
+def load_support_manifest(path: Path) -> dict[str, dict[str, str]]:
+    rows = read_rows(path)
+    support: dict[str, dict[str, str]] = {}
+    for row in rows:
+        support_path = Path(row["support_path"])
+        if row.get("status") in {"generated", "existing"} and support_path.exists():
+            support[str(row["sample_id"])] = row
+    return support
+
+
+class ReferenceFeatureStore:
+    def __init__(self, rows: list[dict[str, str]], normalizer: Normalizer) -> None:
+        self.rows_by_id = {str(row["sample_id"]): row for row in rows}
+        self.normalizer = normalizer
+
+    def get(self, sample_id: str) -> dict[str, np.ndarray | str]:
+        row = self.rows_by_id[str(sample_id)]
+        arr = np.load(row["feature_path"], allow_pickle=True)
+        try:
+            x = np.nan_to_num(arr["features"].astype(np.float32), nan=0.0, posinf=0.0, neginf=0.0)
+            y = np.nan_to_num(arr["y_occ"].astype(np.float32), nan=0.0, posinf=0.0, neginf=0.0)
+        finally:
+            arr.close()
+        return {"x": self.normalizer.apply(x), "y_occ": y, "target_timestamp": row["target_timestamp"]}
+
+
+class SpatialSupportStore:
+    def __init__(
+        self,
+        rows: list[dict[str, str]],
+        support: dict[str, dict[str, str]],
+        normalizer: Normalizer,
+    ) -> None:
+        self.rows_by_id = {str(row["sample_id"]): row for row in rows}
+        self.support = support
+        self.normalizer = normalizer
+
+    def get(self, sample_id: str) -> dict[str, np.ndarray | str]:
+        sid = str(sample_id)
+        row = self.rows_by_id[sid]
+        sarr = np.load(self.support[sid]["support_path"], allow_pickle=True)
+        farr = np.load(row["feature_path"], allow_pickle=True)
+        try:
+            x = np.nan_to_num(sarr["features"].astype(np.float32), nan=0.0, posinf=0.0, neginf=0.0)
+            y = np.nan_to_num(farr["y_occ"].astype(np.float32), nan=0.0, posinf=0.0, neginf=0.0)
+        finally:
+            sarr.close()
+            farr.close()
+        return {"x": self.normalizer.apply(x), "y_occ": y, "target_timestamp": row["target_timestamp"]}
+
+
+class FullMapDataset(Dataset):
+    def __init__(self, rows: list[dict[str, str]], store: Any) -> None:
+        self.rows = rows
+        self.store = store
+
+    def __len__(self) -> int:
+        return len(self.rows)
+
+    def __getitem__(self, idx: int) -> dict[str, Any]:
+        row = self.rows[idx]
+        sample = self.store.get(str(row["sample_id"]))
+        return {
+            "x": torch.from_numpy(np.asarray(sample["x"], dtype=np.float32)),
+            "y": torch.from_numpy(np.asarray(sample["y_occ"], dtype=np.float32)),
+            "sample_id": str(row["sample_id"]),
+            "target_timestamp": str(sample["target_timestamp"]),
+        }
+
+
+def make_loaders(
+    split_rows: dict[str, list[dict[str, str]]],
+    store: Any,
+    batch_size: int,
+    device: torch.device,
+    seed: int,
+) -> dict[str, DataLoader]:
+    loaders: dict[str, DataLoader] = {}
+    for split, rows in split_rows.items():
+        kwargs: dict[str, Any] = {}
+        if split == "train":
+            kwargs["generator"] = torch.Generator().manual_seed(int(seed))
+        loaders[split] = DataLoader(
+            FullMapDataset(rows, store),
+            batch_size=int(batch_size),
+            shuffle=(split == "train"),
+            num_workers=0,
+            pin_memory=device.type == "cuda",
+            **kwargs,
+        )
+    return loaders
+
+
+def build_attached_store(args: argparse.Namespace, split_rows: dict[str, list[dict[str, str]]]) -> Any:
+    if args.daily_rows_csv is None:
+        raise ValueError("--daily-rows-csv is required for attached source.")
+    daily_lookup, ordered_times, ordered_features = head_control.build_daily_lookup(args.daily_rows_csv)
+    return head_control.FeatureStore(
+        split_rows["train"] + split_rows["val"] + split_rows["test"],
+        daily_lookup,
+        ordered_times,
+        ordered_features,
+    )
+
+
+def build_alphaearth_store(args: argparse.Namespace, split_rows: dict[str, list[dict[str, str]]]) -> Any:
+    if args.alphaearth_cache_root is None:
+        raise ValueError("--alphaearth-cache-root is required for AlphaEarth source.")
+    grid_cache = alpha_runner.GridCache(args.alphaearth_cache_root)
+    return alpha_runner.FeatureStore(split_rows["train"] + split_rows["val"] + split_rows["test"], grid_cache)
+
+
+def build_spatial_store(args: argparse.Namespace, split_rows: dict[str, list[dict[str, str]]]) -> SpatialSupportStore:
+    if args.support_dir is None:
+        raise ValueError("--support-dir is required for spatial source.")
+    support = load_support_manifest(args.support_dir / "support_manifest.csv")
+    missing = [
+        row["sample_id"]
+        for rows in split_rows.values()
+        for row in rows
+        if str(row["sample_id"]) not in support
+    ]
+    if missing:
+        raise RuntimeError(f"Missing spatial support maps for {len(missing)} samples; first={missing[:5]}")
+    normalizer = compute_map_normalizer(split_rows["train"], lambda row: support[str(row["sample_id"])]["support_path"])
+    return SpatialSupportStore(split_rows["train"] + split_rows["val"] + split_rows["test"], support, normalizer)
+
+
+def build_reference_store(split_rows: dict[str, list[dict[str, str]]]) -> ReferenceFeatureStore:
+    normalizer = compute_map_normalizer(split_rows["train"], lambda row: row["feature_path"])
+    return ReferenceFeatureStore(split_rows["train"] + split_rows["val"] + split_rows["test"], normalizer)
+
+
+def build_head(head_arch: str, in_ch: int, prior_prob: float) -> nn.Module:
+    if head_arch == "constant":
+        return head_control.ConstantHead(prior_prob=prior_prob)
+    if head_arch == "linear":
+        return head_control.LinearHead(in_ch=in_ch, prior_prob=prior_prob)
+    if head_arch == "pixel_mlp":
+        return head_control.PixelMLPHead(in_ch=in_ch, hidden=16, dropout=0.05, prior_prob=prior_prob)
+    if head_arch == "shallow_wide":
+        return head_control.WildfireHead(
+            in_ch=in_ch,
+            hidden=64,
+            dropout=0.10,
+            norm_type="group",
+            norm_groups=8,
+            prior_prob=prior_prob,
+        )
+    return head_control.WildfireHead(
+        in_ch=in_ch,
+        hidden=32,
+        dropout=0.05,
+        norm_type="group",
+        norm_groups=8,
+        prior_prob=prior_prob,
+    )
+
+
+@torch.no_grad()
+def collect_predictions(model: nn.Module, loader: DataLoader, device: torch.device) -> tuple[np.ndarray, np.ndarray]:
+    model.eval()
+    probs: list[np.ndarray] = []
+    targets: list[np.ndarray] = []
+    for batch in loader:
+        x = batch["x"].to(device, non_blocking=True)
+        y = batch["y"].to(device, non_blocking=True)
+        logits = model(x)
+        probs.append(np.nan_to_num(torch.sigmoid(logits).detach().cpu().numpy()[:, 0], nan=0.0, posinf=1.0, neginf=0.0))
+        targets.append(np.nan_to_num(y.detach().cpu().numpy()[:, 0], nan=0.0, posinf=0.0, neginf=0.0))
+    return np.concatenate(probs, axis=0), np.concatenate(targets, axis=0)
+
+
+def train_one_head(
+    head_arch: str,
+    in_ch: int,
+    prior_prob: float,
+    loaders: dict[str, DataLoader],
+    args: argparse.Namespace,
+    device: torch.device,
+    seed_offset: int,
+) -> tuple[nn.Module, list[dict[str, float]]]:
+    set_seed(int(args.seed) + int(seed_offset))
+    model = build_head(head_arch, in_ch=in_ch, prior_prob=prior_prob).to(device)
+    optimizer = torch.optim.AdamW(model.parameters(), lr=float(args.learning_rate), weight_decay=float(args.weight_decay))
+    raw_weight = (1.0 - float(prior_prob)) / max(float(prior_prob), 1e-9)
+    pos_weight = float(min(float(args.pos_weight_cap), raw_weight))
+    criterion = nn.BCEWithLogitsLoss(pos_weight=torch.tensor([pos_weight], dtype=torch.float32, device=device))
+    history: list[dict[str, float]] = []
+    for epoch in range(1, int(args.epochs) + 1):
+        model.train()
+        losses: list[float] = []
+        for batch in loaders["train"]:
+            x = batch["x"].to(device, non_blocking=True)
+            y = batch["y"].to(device, non_blocking=True)
+            optimizer.zero_grad(set_to_none=True)
+            logits = model(x)
+            loss = criterion(logits, y)
+            if not torch.isfinite(loss):
+                raise RuntimeError(f"Non-finite loss for head={head_arch}")
+            loss.backward()
+            optimizer.step()
+            losses.append(float(loss.item()))
+        history.append({"epoch": epoch, "train_bce": float(np.mean(losses)), "pos_weight": pos_weight})
+    return model, history
+
+
+def build_sample_times(rows: list[dict[str, str]]) -> np.ndarray:
+    return np.array([row["target_timestamp"] for row in rows], dtype="datetime64[h]")
+
+
+def select_val_posthoc(
+    rows: list[dict[str, object]],
+    scope: str,
+    metric: str,
+) -> dict[str, object]:
+    prefix = metric.rsplit("_", 1)[0]
+    precision_key = f"{prefix}_precision"
+    recall_key = f"{prefix}_recall"
+    selected = [row for row in rows if row["split"] == "val" and row["scope"] == scope]
+    if not selected:
+        raise RuntimeError(f"No validation rows for scope={scope}")
+    return max(
+        selected,
+        key=lambda row: (
+            float(row.get(metric, 0.0)),
+            float(row.get(precision_key, 0.0)),
+            float(row.get(recall_key, 0.0)),
+            -abs(float(row["threshold"]) - 0.5),
+        ),
+    )
+
+
+def matching_test_row(rows: list[dict[str, object]], scope: str, selected: dict[str, object]) -> dict[str, object]:
+    threshold = float(selected["threshold"])
+    variant = str(selected["variant"])
+    for row in rows:
+        if (
+            row["split"] == "test"
+            and row["scope"] == scope
+            and str(row["variant"]) == variant
+            and abs(float(row["threshold"]) - threshold) < 1e-12
+        ):
+            return row
+    raise RuntimeError(f"No matching test row for scope={scope}, threshold={threshold}, variant={variant}")
+
+
+def summarize_head_scores(
+    head_metrics: list[dict[str, object]],
+) -> list[dict[str, object]]:
+    selection_rows: list[dict[str, object]] = []
+    for scope in SCOPE_ORDER:
+        candidates = [row for row in head_metrics if row["scope"] == scope]
+        if not candidates:
+            continue
+        ranking_selected = max(
+            candidates,
+            key=lambda row: (
+                float(row["val_pr_auc"]),
+                float(row["val_union_f1"]),
+                float(row["val_tolerated_f1"]),
+                float(row["val_exact_f1"]),
+            ),
+        )
+        out: dict[str, object] = {
+            "scope": scope,
+            "ranking_selected_head": ranking_selected["head_label"],
+            "ranking_selected_head_arch": ranking_selected["head_arch"],
+            "ranking_selected_val_pr_auc": float(ranking_selected["val_pr_auc"]),
+            "ranking_selected_test_pr_auc": float(ranking_selected["test_pr_auc"]),
+        }
+        for short, column in (("exact", "exact_f1"), ("tolerated", "tolerated_f1"), ("union", "union_f1")):
+            val_column = f"val_{column}"
+            test_column = f"test_{column}"
+            decision_selected = max(
+                candidates,
+                key=lambda row: (
+                    float(row[val_column]),
+                    float(row["val_pr_auc"]),
+                    str(row["head_arch"]),
+                ),
+            )
+            val_gap = float(decision_selected[val_column]) - float(ranking_selected[val_column])
+            test_gap = float(decision_selected[test_column]) - float(ranking_selected[test_column])
+            out[f"{short}_val_ranking_score"] = float(ranking_selected[val_column])
+            out[f"{short}_val_decision_score"] = float(decision_selected[val_column])
+            out[f"{short}_val_gap"] = float(max(0.0, val_gap))
+            out[f"{short}_ranking_score"] = float(ranking_selected[test_column])
+            out[f"{short}_decision_score"] = float(decision_selected[test_column])
+            out[f"{short}_test_gap"] = float(test_gap)
+            out[f"{short}_regret"] = float(max(0.0, test_gap))
+            out[f"{short}_decision_head"] = decision_selected["head_label"]
+            out[f"{short}_decision_head_arch"] = decision_selected["head_arch"]
+        selection_rows.append(out)
+    return selection_rows
+
+
+def write_csv(rows: list[dict[str, object]], path: Path) -> None:
+    path.parent.mkdir(parents=True, exist_ok=True)
+    fieldnames = sorted({key for row in rows for key in row})
+    with path.open("w", newline="", encoding="utf-8") as fh:
+        writer = csv.DictWriter(fh, fieldnames=fieldnames)
+        writer.writeheader()
+        writer.writerows(rows)
+
+
+def load_head_summary(head_dir: Path, head_arch: str) -> tuple[list[dict[str, object]], dict[str, dict[str, float]], dict[str, object]] | None:
+    posthoc_path = head_dir / "posthoc_rows.csv"
+    summary_path = head_dir / "summary.json"
+    if not posthoc_path.exists() or not summary_path.exists():
+        return None
+    rows = [dict(row) for row in read_rows(posthoc_path)]
+    if not rows:
+        return None
+    try:
+        summary = json.loads(summary_path.read_text(encoding="utf-8"))
+    except json.JSONDecodeError:
+        return None
+    if str(summary.get("head_arch")) != str(head_arch):
+        return None
+    raw_pr_auc = summary.get("raw_pr_auc")
+    if not isinstance(raw_pr_auc, dict):
+        return None
+    try:
+        parsed_pr_auc = {
+            split: {
+                scope: float(raw_pr_auc[split][scope])
+                for scope in SCOPE_ORDER
+            }
+            for split in ("val", "test")
+        }
+    except Exception:
+        return None
+    return rows, parsed_pr_auc, summary
+
+
+def append_head_metrics(
+    head_metrics: list[dict[str, object]],
+    posthoc_rows: list[dict[str, object]],
+    raw_pr_auc: dict[str, dict[str, float]],
+    head_arch: str,
+    args: argparse.Namespace,
+) -> None:
+    for scope in SCOPE_ORDER:
+        metric_scores: dict[str, float] = {}
+        selected_thresholds: dict[str, float] = {}
+        selected_variants: dict[str, str] = {}
+        for short, metric in METRICS.items():
+            selected = select_val_posthoc(posthoc_rows, scope, metric)
+            test_row = matching_test_row(posthoc_rows, scope, selected)
+            metric_scores[f"val_{short}_f1"] = float(selected[metric])
+            metric_scores[f"test_{short}_f1"] = float(test_row[metric])
+            selected_thresholds[short] = float(selected["threshold"])
+            selected_variants[short] = str(selected["variant"])
+        head_metrics.append(
+            {
+                "model_tag": args.model_tag,
+                "family": args.fm_family,
+                "seed": int(args.seed),
+                "scope": scope,
+                "head_arch": head_arch,
+                "head_label": head_control.HEAD_LABELS[head_arch],
+                "val_pr_auc": float(raw_pr_auc["val"][scope]),
+                "test_pr_auc": float(raw_pr_auc["test"][scope]),
+                **metric_scores,
+                "selected_thresholds": selected_thresholds,
+                "selected_variants": selected_variants,
+            }
+        )
+
+
+def main() -> None:
+    args = parse_args()
+    args.output_dir.mkdir(parents=True, exist_ok=True)
+    set_seed(int(args.seed))
+    device = choose_device(args.device)
+
+    split_rows = {
+        split: read_rows(args.feature_root / "splits" / f"{split}.csv")
+        for split in ("train", "val", "test")
+    }
+    if args.source_kind == "reference":
+        store = build_reference_store(split_rows)
+    elif args.source_kind == "attached":
+        store = build_attached_store(args, split_rows)
+    elif args.source_kind == "spatial":
+        store = build_spatial_store(args, split_rows)
+    else:
+        store = build_alphaearth_store(args, split_rows)
+
+    loaders = make_loaders(split_rows, store, int(args.batch_size), device, int(args.seed))
+    first = next(iter(loaders["train"]))
+    in_ch = int(first["x"].shape[1])
+    prior_prob = total_positive_rate(split_rows["train"])
+    fire_prone_mask, fire_prone_meta = head_control.build_fire_prone_mask(
+        split_rows["train"],
+        store,
+        float(args.fire_prone_top_frac),
+    )
+
+    head_metrics: list[dict[str, object]] = []
+    head_artifacts: dict[str, str] = {}
+    for head_index, head_arch in enumerate(args.heads):
+        head_dir = args.output_dir / head_arch
+        head_dir.mkdir(parents=True, exist_ok=True)
+        cached = load_head_summary(head_dir, head_arch)
+        if cached is not None:
+            posthoc_rows, raw_pr_auc, _ = cached
+            print(f"[selection-regret] reuse {args.fm_family} seed={args.seed} head={head_arch}", flush=True)
+        else:
+            print(f"[selection-regret] training {args.fm_family} seed={args.seed} head={head_arch}", flush=True)
+            model, history = train_one_head(
+                head_arch=head_arch,
+                in_ch=in_ch,
+                prior_prob=prior_prob,
+                loaders=loaders,
+                args=args,
+                device=device,
+                seed_offset=1009 * (head_index + 1),
+            )
+            posthoc_rows = []
+            raw_pr_auc = {}
+            for split in ("val", "test"):
+                probs, targets = collect_predictions(model, loaders[split], device)
+                sample_times = build_sample_times(split_rows[split])
+                raw_pr_auc[split] = {
+                    "global": head_control._masked_average_precision(probs, targets, region_mask=None),
+                    "fire_prone": head_control._masked_average_precision(probs, targets, region_mask=fire_prone_mask),
+                }
+                posthoc_rows.extend(
+                    head_control.build_posthoc_rows(
+                        probs=probs,
+                        targets=targets,
+                        sample_times=sample_times,
+                        split=split,
+                        fire_prone_mask=fire_prone_mask,
+                        args=args,
+                    )
+                )
+
+            write_csv(posthoc_rows, head_dir / "posthoc_rows.csv")
+            head_summary = {
+                "head_arch": head_arch,
+                "head_label": head_control.HEAD_LABELS[head_arch],
+                "history": history,
+                "raw_pr_auc": raw_pr_auc,
+                "posthoc_rows_csv": str(head_dir / "posthoc_rows.csv"),
+            }
+            (head_dir / "summary.json").write_text(json.dumps(head_summary, indent=2), encoding="utf-8")
+        head_artifacts[head_arch] = str(head_dir / "summary.json")
+        append_head_metrics(head_metrics, posthoc_rows, raw_pr_auc, head_arch, args)
+
+    selection_rows = summarize_head_scores(head_metrics)
+    for row in selection_rows:
+        row["model_tag"] = args.model_tag
+        row["family"] = args.fm_family
+        row["seed"] = int(args.seed)
+
+    write_csv(head_metrics, args.output_dir / "head_metrics.csv")
+    write_csv(selection_rows, args.output_dir / "selection_rows.csv")
+    summary = {
+        "experiment": "all-backbone fixed-feature head-selection regret",
+        "task": "wildfire_occupancy",
+        "model_tag": args.model_tag,
+        "fm_family": args.fm_family,
+        "source_kind": args.source_kind,
+        "seed": int(args.seed),
+        "feature_root": str(args.feature_root),
+        "daily_rows_csv": str(args.daily_rows_csv) if args.daily_rows_csv else None,
+        "support_dir": str(args.support_dir) if args.support_dir else None,
+        "alphaearth_cache_root": str(args.alphaearth_cache_root) if args.alphaearth_cache_root else None,
+        "device": str(device),
+        "heads": list(args.heads),
+        "input_channels": int(in_ch),
+        "prior_prob": float(prior_prob),
+        "fire_prone_scope": {
+            "scope_name": "fire_prone",
+            "reported_as": "top 20%",
+            **fire_prone_meta,
+        },
+        "metrics": METRICS,
+        "head_metrics": head_metrics,
+        "selection_rows": selection_rows,
+        "head_artifacts": head_artifacts,
+        "artifacts": {
+            "head_metrics_csv": str(args.output_dir / "head_metrics.csv"),
+            "selection_rows_csv": str(args.output_dir / "selection_rows.csv"),
+        },
+    }
+    (args.output_dir / "summary.json").write_text(json.dumps(summary, indent=2), encoding="utf-8")
+    print(json.dumps(summary, indent=2), flush=True)
+
+
+if __name__ == "__main__":
+    main()

experiments/raw_reference/task_scripts/run_analog_extended_retrieval_sweep_seeded.py

@@ -0,0 +1,333 @@
+#!/usr/bin/env python3
+from __future__ import annotations
+
+import argparse
+import json
+import sys
+from pathlib import Path
+from typing import Dict, List, Tuple
+
+
+import os
+
+for _p in os.environ.get("WILDFIRE_FM_EXTRA_PYTHONPATH", "").split(os.pathsep):
+    if _p and _p not in sys.path:
+        sys.path.insert(0, _p)
+
+import numpy as np
+import pandas as pd
+from sklearn.compose import ColumnTransformer
+from sklearn.impute import SimpleImputer
+from sklearn.linear_model import ElasticNet, Ridge
+from sklearn.metrics.pairwise import cosine_similarity
+from sklearn.pipeline import Pipeline
+from sklearn.preprocessing import OneHotEncoder, StandardScaler
+
+
+DROP_COLUMNS = {
+    "Event_ID",
+    "Incid_Name",
+    "incident_name_norm",
+    "wfigs_name",
+    "Ig_Date",
+    "weather_date",
+    "BurnBndAc",
+    "target_log_burn_acres",
+}
+CATEGORICAL_COLUMNS = ["Incid_Type", "state_abbr", "county_name", "wfigs_match_type"]
+
+
+def build_splits(df: pd.DataFrame) -> Tuple[pd.DataFrame, pd.DataFrame, pd.DataFrame]:
+    ordered = df.sort_values("Ig_Date").reset_index(drop=True)
+    n = len(ordered)
+    train_end = max(int(round(n * 0.6)), 1)
+    val_end = max(int(round(n * 0.8)), train_end + 1)
+    val_end = min(val_end, n - 1) if n >= 3 else n
+    train = ordered.iloc[:train_end].copy()
+    val = ordered.iloc[train_end:val_end].copy()
+    test = ordered.iloc[val_end:].copy()
+    if len(val) == 0 and len(test) > 1:
+        val = test.iloc[:1].copy()
+        test = test.iloc[1:].copy()
+    return train, val, test
+
+
+def block_columns(df: pd.DataFrame, exclude: set[str]) -> Dict[str, List[str]]:
+    numeric = [
+        c
+        for c in df.columns
+        if c not in DROP_COLUMNS
+        and c not in CATEGORICAL_COLUMNS
+        and c not in exclude
+        and pd.api.types.is_numeric_dtype(df[c])
+    ]
+    return {
+        "weather": [c for c in numeric if c.startswith("weather_")],
+        "geo_fire": [
+            c
+            for c in numeric
+            if c.startswith("firms_")
+            or c.startswith("landfire_")
+            or c in {"BurnBndLat", "BurnBndLon", "lat", "lon", "wfigs_acres", "wfigs_date_diff_days", "wfigs_dist_km", "is_conus_static"}
+        ],
+        "categorical": [c for c in CATEGORICAL_COLUMNS if c in df.columns and c not in exclude],
+    }
+
+
+def make_block_matrix(train: pd.DataFrame, val: pd.DataFrame, test: pd.DataFrame, cols: List[str], categorical: bool) -> Tuple[np.ndarray, np.ndarray, np.ndarray]:
+    if not cols:
+        n_train, n_val, n_test = len(train), len(val), len(test)
+        return np.zeros((n_train, 0), dtype=np.float32), np.zeros((n_val, 0), dtype=np.float32), np.zeros((n_test, 0), dtype=np.float32)
+    if categorical:
+        transformer = ColumnTransformer(
+            [("cat", Pipeline([("impute", SimpleImputer(strategy="most_frequent")), ("onehot", OneHotEncoder(handle_unknown="ignore"))]), cols)],
+            remainder="drop",
+        )
+    else:
+        transformer = ColumnTransformer(
+            [("num", Pipeline([("impute", SimpleImputer(strategy="median")), ("scale", StandardScaler())]), cols)],
+            remainder="drop",
+        )
+    train_x = transformer.fit_transform(train[cols])
+    val_x = transformer.transform(val[cols])
+    test_x = transformer.transform(test[cols])
+    if hasattr(train_x, "toarray"):
+        train_x = train_x.toarray()
+        val_x = val_x.toarray()
+        test_x = test_x.toarray()
+    return train_x.astype(np.float32), val_x.astype(np.float32), test_x.astype(np.float32)
+
+
+def graded_relevance(query_target: float, retrieved_targets: np.ndarray) -> np.ndarray:
+    delta = np.abs(np.asarray(retrieved_targets, dtype=np.float64) - float(query_target))
+    return np.select([delta <= 0.5, delta <= 1.0, delta <= 1.5], [3.0, 2.0, 1.0], default=0.0)
+
+
+def dcg(relevance: np.ndarray) -> float:
+    rel = np.asarray(relevance, dtype=np.float64)
+    discounts = 1.0 / np.log2(np.arange(rel.size, dtype=np.float64) + 2.0)
+    return float(np.sum(rel * discounts))
+
+
+def ndcg_at_k(relevance: np.ndarray, ideal_relevance: np.ndarray, k: int) -> float:
+    denom = dcg(ideal_relevance[:k])
+    return float(dcg(relevance[:k]) / denom) if denom > 0 else 0.0
+
+
+def rmse(y_true: np.ndarray, y_pred: np.ndarray) -> float:
+    return float(np.sqrt(np.mean((np.asarray(y_true) - np.asarray(y_pred)) ** 2)))
+
+
+def spearman_corr(y_true: np.ndarray, y_pred: np.ndarray) -> float:
+    value = pd.Series(y_true).corr(pd.Series(y_pred), method="spearman")
+    return float(value) if pd.notna(value) else 0.0
+
+
+def target_weight_vectors(train_vec: np.ndarray, val_vec: np.ndarray, test_vec: np.ndarray, target: np.ndarray, power: float, floor: float) -> Tuple[np.ndarray, np.ndarray, np.ndarray]:
+    if train_vec.shape[1] == 0:
+        return train_vec, val_vec, test_vec
+    x = np.asarray(train_vec, dtype=np.float64)
+    y = np.asarray(target, dtype=np.float64)
+    y = y - y.mean()
+    x_centered = x - x.mean(axis=0, keepdims=True)
+    denom = np.clip(np.sqrt(np.sum(x_centered**2, axis=0)) * np.sqrt(np.sum(y**2)), 1e-12, None)
+    corr = np.abs(np.sum(x_centered * y[:, None], axis=0) / denom)
+    corr = np.nan_to_num(corr, nan=0.0, posinf=0.0, neginf=0.0)
+    if float(corr.max()) > 0:
+        corr = corr / float(corr.max())
+    weights = (floor + np.power(corr, power)).astype(np.float32)
+    return train_vec * weights, val_vec * weights, test_vec * weights
+
+
+def score_vectors(query_vec: np.ndarray, library_vec: np.ndarray, query_df: pd.DataFrame, library_df: pd.DataFrame, k: int, mode: str) -> Dict[str, float]:
+    k_eff = min(k, library_vec.shape[0])
+    lib_norm = library_vec / np.clip(np.linalg.norm(library_vec, axis=1, keepdims=True), 1e-12, None)
+    query_norm = query_vec / np.clip(np.linalg.norm(query_vec, axis=1, keepdims=True), 1e-12, None)
+    sim_all = cosine_similarity(query_norm, lib_norm)
+    knn_idx = np.argsort(-sim_all, axis=1)[:, :k_eff]
+    knn_sim = np.take_along_axis(sim_all, knn_idx, axis=1)
+    target_lib = library_df["target_log_burn_acres"].to_numpy(dtype=np.float64)
+    preds = []
+    ndcg5 = []
+    ndcg10 = []
+    hit1 = []
+    hit5 = []
+    hit10 = []
+    best_abs = []
+    for i in range(query_df.shape[0]):
+        idx = knn_idx[i]
+        sims = knn_sim[i]
+        top_targets = target_lib[idx]
+        true = float(query_df.iloc[i]["target_log_burn_acres"])
+        relevance = graded_relevance(true, top_targets)
+        ideal = np.sort(graded_relevance(true, target_lib))[::-1]
+        ndcg5.append(ndcg_at_k(relevance, ideal, 5))
+        ndcg10.append(ndcg_at_k(relevance, ideal, 10))
+        hit1.append(float(relevance[:1].max() >= 2.0))
+        hit5.append(float(relevance[: min(5, k_eff)].max() >= 2.0))
+        hit10.append(float(relevance[: min(10, k_eff)].max() >= 2.0))
+        best_abs.append(float(np.min(np.abs(top_targets - true))))
+        if mode == "weighted":
+            weights = np.maximum((sims + 1.0) / 2.0, 1e-6)
+            preds.append(float(np.sum(weights * top_targets) / np.sum(weights)))
+        else:
+            preds.append(float(np.mean(top_targets)))
+    pred = np.asarray(preds, dtype=np.float64)
+    true_log = query_df["target_log_burn_acres"].to_numpy(dtype=np.float64)
+    return {
+        "count": int(len(query_df)),
+        "log_mae": float(np.mean(np.abs(true_log - pred))),
+        "log_rmse": rmse(true_log, pred),
+        "log_spearman": spearman_corr(true_log, pred),
+        "ndcg_at_5": float(np.mean(ndcg5)),
+        "ndcg_at_10": float(np.mean(ndcg10)),
+        "hit_at_1_tol1": float(np.mean(hit1)),
+        "hit_at_5_tol1": float(np.mean(hit5)),
+        "hit_at_10_tol1": float(np.mean(hit10)),
+        "mean_best_abs_log_delta_at_k": float(np.mean(best_abs)),
+    }
+
+
+def append_supervised_scalar(
+    train_vec: np.ndarray,
+    val_vec: np.ndarray,
+    test_vec: np.ndarray,
+    train_df: pd.DataFrame,
+    model_name: str,
+    weight: float,
+    seed: int,
+) -> Tuple[np.ndarray, np.ndarray, np.ndarray]:
+    y = train_df["target_log_burn_acres"].to_numpy(dtype=np.float64)
+    model = Ridge(alpha=1.0) if model_name == "ridge" else ElasticNet(alpha=0.01, l1_ratio=0.2, random_state=seed, max_iter=10000)
+    model.fit(train_vec, y)
+    train_pred = model.predict(train_vec)
+    val_pred = model.predict(val_vec)
+    test_pred = model.predict(test_vec)
+    mean = float(np.mean(train_pred))
+    std = float(np.std(train_pred)) or 1.0
+    def _append(x: np.ndarray, pred: np.ndarray) -> np.ndarray:
+        scalar = ((pred - mean) / std).reshape(-1, 1).astype(np.float32) * float(weight)
+        return np.concatenate([x, scalar], axis=1)
+    return _append(train_vec, train_pred), _append(val_vec, val_pred), _append(test_vec, test_pred)
+
+
+def main() -> None:
+    parser = argparse.ArgumentParser()
+    parser.add_argument("--event-table", type=Path, required=True)
+    parser.add_argument("--output-dir", type=Path, required=True)
+    parser.add_argument("--exclude-columns", nargs="*", default=[])
+    parser.add_argument("--seed", type=int, default=7)
+    args = parser.parse_args()
+
+    df = pd.read_csv(args.event_table)
+    df["Ig_Date"] = pd.to_datetime(df["Ig_Date"])
+    train_df, val_df, test_df = build_splits(df)
+    exclude = set(args.exclude_columns)
+    blocks = block_columns(df, exclude)
+    matrices = {
+        "weather": make_block_matrix(train_df, val_df, test_df, blocks["weather"], categorical=False),
+        "geo_fire": make_block_matrix(train_df, val_df, test_df, blocks["geo_fire"], categorical=False),
+        "categorical": make_block_matrix(train_df, val_df, test_df, blocks["categorical"], categorical=True),
+    }
+
+    candidate_rows: List[Dict[str, object]] = []
+    best = None
+    best_score = None
+    best_test = None
+    block_weight_grid = [
+        {"weather": 1.0, "geo_fire": 1.0, "categorical": 1.0},
+        {"weather": 0.5, "geo_fire": 1.5, "categorical": 1.0},
+        {"weather": 0.25, "geo_fire": 2.0, "categorical": 1.0},
+        {"weather": 1.5, "geo_fire": 1.0, "categorical": 0.5},
+        {"weather": 0.0, "geo_fire": 2.0, "categorical": 1.0},
+        {"weather": 2.0, "geo_fire": 0.5, "categorical": 0.5},
+    ]
+    target_weight_settings = [(0.0, 1.0), (0.5, 0.25), (1.0, 0.25), (2.0, 0.10)]
+    scalar_settings = [("none", 0.0), ("ridge", 0.5), ("ridge", 1.0), ("ridge", 2.0), ("enet", 1.0), ("enet", 2.0)]
+
+    for bw in block_weight_grid:
+        base_train = np.concatenate([matrices[name][0] * bw[name] for name in ["weather", "geo_fire", "categorical"]], axis=1)
+        base_val = np.concatenate([matrices[name][1] * bw[name] for name in ["weather", "geo_fire", "categorical"]], axis=1)
+        base_test = np.concatenate([matrices[name][2] * bw[name] for name in ["weather", "geo_fire", "categorical"]], axis=1)
+        for power, floor in target_weight_settings:
+            tw_train, tw_val, tw_test = target_weight_vectors(
+                base_train,
+                base_val,
+                base_test,
+                train_df["target_log_burn_acres"].to_numpy(dtype=np.float64),
+                power=power,
+                floor=floor,
+            )
+            for scalar_model, scalar_weight in scalar_settings:
+                if scalar_model == "none":
+                    train_vec, val_vec, test_vec = tw_train, tw_val, tw_test
+                else:
+                    train_vec, val_vec, test_vec = append_supervised_scalar(
+                        tw_train,
+                        tw_val,
+                        tw_test,
+                        train_df,
+                        scalar_model,
+                        scalar_weight,
+                        args.seed,
+                    )
+                for k in [3, 5, 10, 15, 20]:
+                    for mode in ["mean", "weighted"]:
+                        val_metrics = score_vectors(val_vec, train_vec, val_df, train_df, k=k, mode=mode)
+                        test_metrics = score_vectors(test_vec, train_vec, test_df, train_df, k=k, mode=mode)
+                        row = {
+                            "block_weights": bw,
+                            "target_weight_power": power,
+                            "target_weight_floor": floor,
+                            "supervised_scalar": scalar_model,
+                            "supervised_scalar_weight": scalar_weight,
+                            "k": k,
+                            "mode": mode,
+                            "val_metrics": val_metrics,
+                            "test_metrics": test_metrics,
+                        }
+                        candidate_rows.append(row)
+                        score = float(val_metrics["ndcg_at_10"])
+                        if best_score is None or score > best_score:
+                            best_score = score
+                            best = row
+                            best_test = test_metrics
+
+    args.output_dir.mkdir(parents=True, exist_ok=True)
+    candidate_df = pd.DataFrame(
+        [
+            {
+                "val_ndcg_at_10": r["val_metrics"]["ndcg_at_10"],
+                "val_log_mae": r["val_metrics"]["log_mae"],
+                "test_ndcg_at_10": r["test_metrics"]["ndcg_at_10"],
+                "test_log_mae": r["test_metrics"]["log_mae"],
+                "k": r["k"],
+                "mode": r["mode"],
+                "target_weight_power": r["target_weight_power"],
+                "target_weight_floor": r["target_weight_floor"],
+                "supervised_scalar": r["supervised_scalar"],
+                "supervised_scalar_weight": r["supervised_scalar_weight"],
+                **{f"block_{k}": v for k, v in r["block_weights"].items()},
+            }
+            for r in candidate_rows
+        ]
+    )
+    candidate_df.to_csv(args.output_dir / "candidate_grid.csv", index=False)
+    summary = {
+        "task_id": "wildfire_analog_retrieval_extended_hybrid_sweep",
+        "event_table": str(args.event_table),
+        "seed": int(args.seed),
+        "excluded_columns": sorted(exclude),
+        "split_sizes": {"train": len(train_df), "val": len(val_df), "test": len(test_df)},
+        "feature_blocks": blocks,
+        "selection_metric": "val_ndcg_at_10",
+        "selected_retrieval": best,
+        "test_metrics": best_test,
+        "candidate_count": len(candidate_rows),
+    }
+    (args.output_dir / "summary.json").write_text(json.dumps(summary, indent=2), encoding="utf-8")
+    print(json.dumps(summary, indent=2))
+
+
+if __name__ == "__main__":
+    main()

experiments/raw_reference/task_scripts/run_event_analog_taskmodel_seeded.py

@@ -0,0 +1,350 @@
+#!/usr/bin/env python3
+from __future__ import annotations
+
+import argparse
+import json
+import sys
+from pathlib import Path
+from typing import Dict, List, Tuple
+
+
+import os
+
+for _p in os.environ.get("WILDFIRE_FM_EXTRA_PYTHONPATH", "").split(os.pathsep):
+    if _p and _p not in sys.path:
+        sys.path.insert(0, _p)
+
+import faiss
+import hnswlib
+import numpy as np
+import pandas as pd
+from sklearn.compose import ColumnTransformer
+from sklearn.impute import SimpleImputer
+from sklearn.metrics.pairwise import cosine_similarity
+from sklearn.pipeline import Pipeline
+from sklearn.preprocessing import OneHotEncoder, StandardScaler
+
+
+DROP_COLUMNS = {
+    "Event_ID",
+    "Incid_Name",
+    "incident_name_norm",
+    "wfigs_name",
+    "Ig_Date",
+    "weather_date",
+    "BurnBndAc",
+    "target_log_burn_acres",
+}
+CATEGORICAL_COLUMNS = ["Incid_Type", "state_abbr", "county_name", "wfigs_match_type"]
+
+
+def rmse(y_true: np.ndarray, y_pred: np.ndarray) -> float:
+    return float(np.sqrt(np.mean((np.asarray(y_true) - np.asarray(y_pred)) ** 2)))
+
+
+def mape(y_true: np.ndarray, y_pred: np.ndarray) -> float:
+    denom = np.clip(np.asarray(y_true, dtype=np.float64), 1e-6, None)
+    frac = np.abs(np.asarray(y_true, dtype=np.float64) - np.asarray(y_pred, dtype=np.float64)) / denom
+    return float(np.mean(frac))
+
+
+def r2_score_manual(y_true: np.ndarray, y_pred: np.ndarray) -> float:
+    y_true = np.asarray(y_true, dtype=np.float64)
+    y_pred = np.asarray(y_pred, dtype=np.float64)
+    ss_res = float(np.sum((y_true - y_pred) ** 2))
+    ss_tot = float(np.sum((y_true - y_true.mean()) ** 2))
+    return float(1.0 - ss_res / ss_tot) if ss_tot > 0 else 0.0
+
+
+def spearman_corr(y_true: np.ndarray, y_pred: np.ndarray) -> float:
+    a = pd.Series(np.asarray(y_true, dtype=np.float64))
+    b = pd.Series(np.asarray(y_pred, dtype=np.float64))
+    value = a.corr(b, method="spearman")
+    return float(value) if pd.notna(value) else 0.0
+
+
+def build_splits(df: pd.DataFrame) -> Tuple[pd.DataFrame, pd.DataFrame, pd.DataFrame]:
+    ordered = df.sort_values("Ig_Date").reset_index(drop=True)
+    n = len(ordered)
+    train_end = max(int(round(n * 0.6)), 1)
+    val_end = max(int(round(n * 0.8)), train_end + 1)
+    val_end = min(val_end, n - 1) if n >= 3 else n
+    train = ordered.iloc[:train_end].copy()
+    val = ordered.iloc[train_end:val_end].copy()
+    test = ordered.iloc[val_end:].copy()
+    if len(val) == 0 and len(test) > 1:
+        val = test.iloc[:1].copy()
+        test = test.iloc[1:].copy()
+    return train, val, test
+
+
+def feature_columns(df: pd.DataFrame, feature_profile: str = "all") -> Tuple[List[str], List[str]]:
+    categorical = [c for c in CATEGORICAL_COLUMNS if c in df.columns]
+    numeric = []
+    for col in df.columns:
+        if col in DROP_COLUMNS or col in categorical:
+            continue
+        if pd.api.types.is_numeric_dtype(df[col]):
+            numeric.append(col)
+    if feature_profile == "weather_fm":
+        numeric = [c for c in numeric if c.startswith("weather_")]
+        categorical = []
+    return numeric, categorical
+
+
+def make_preprocessor(numeric_cols: List[str], categorical_cols: List[str]) -> ColumnTransformer:
+    return ColumnTransformer(
+        transformers=[
+            (
+                "num",
+                Pipeline(
+                    steps=[
+                        ("impute", SimpleImputer(strategy="median")),
+                        ("scale", StandardScaler()),
+                    ]
+                ),
+                numeric_cols,
+            ),
+            (
+                "cat",
+                Pipeline(
+                    steps=[
+                        ("impute", SimpleImputer(strategy="most_frequent")),
+                        ("onehot", OneHotEncoder(handle_unknown="ignore")),
+                    ]
+                ),
+                categorical_cols,
+            ),
+        ],
+        remainder="drop",
+    )
+
+
+def to_dense_float32(x) -> np.ndarray:
+    if hasattr(x, "toarray"):
+        x = x.toarray()
+    return np.asarray(x, dtype=np.float32)
+
+
+def weighted_prediction(sim: np.ndarray, targets: np.ndarray) -> float:
+    weights = np.maximum((np.asarray(sim, dtype=np.float64) + 1.0) / 2.0, 1e-6)
+    return float(np.sum(weights * targets) / np.sum(weights))
+
+
+def graded_relevance(query_target: float, retrieved_targets: np.ndarray) -> np.ndarray:
+    delta = np.abs(np.asarray(retrieved_targets, dtype=np.float64) - float(query_target))
+    return np.select([delta <= 0.5, delta <= 1.0, delta <= 1.5], [3.0, 2.0, 1.0], default=0.0)
+
+
+def dcg(relevance: np.ndarray) -> float:
+    rel = np.asarray(relevance, dtype=np.float64)
+    if rel.size == 0:
+        return 0.0
+    discounts = 1.0 / np.log2(np.arange(rel.size, dtype=np.float64) + 2.0)
+    return float(np.sum(rel * discounts))
+
+
+def ndcg_at_k(relevance: np.ndarray, ideal_relevance: np.ndarray, k: int) -> float:
+    rel = np.asarray(relevance, dtype=np.float64)[:k]
+    ideal = np.asarray(ideal_relevance, dtype=np.float64)[:k]
+    denom = dcg(ideal)
+    return float(dcg(rel) / denom) if denom > 0 else 0.0
+
+
+def score_backend(
+    name: str,
+    query_vec: np.ndarray,
+    library_vec: np.ndarray,
+    query_df: pd.DataFrame,
+    library_df: pd.DataFrame,
+    k: int,
+    mode: str,
+) -> Tuple[Dict[str, float], pd.DataFrame]:
+    target_lib = library_df["target_log_burn_acres"].to_numpy(dtype=np.float64)
+    rows = []
+    preds = []
+    ndcg5 = []
+    ndcg10 = []
+    hit1 = []
+    hit5 = []
+    hit10 = []
+    best_abs_delta = []
+
+    k_eff = min(int(k), int(library_vec.shape[0]))
+    if name == "cosine_exact":
+        sim_all = cosine_similarity(query_vec, library_vec)
+        knn_idx = np.argsort(-sim_all, axis=1)[:, :k_eff]
+        knn_sim = np.take_along_axis(sim_all, knn_idx, axis=1)
+    else:
+        library_norm = library_vec / np.clip(np.linalg.norm(library_vec, axis=1, keepdims=True), 1e-12, None)
+        query_norm = query_vec / np.clip(np.linalg.norm(query_vec, axis=1, keepdims=True), 1e-12, None)
+        if name == "faiss_flat_ip":
+            index = faiss.IndexFlatIP(library_norm.shape[1])
+            index.add(library_norm.astype(np.float32))
+            knn_sim, knn_idx = index.search(query_norm.astype(np.float32), k_eff)
+        elif name == "hnsw_cosine":
+            index = hnswlib.Index(space="cosine", dim=library_norm.shape[1])
+            index.init_index(max_elements=library_norm.shape[0], ef_construction=100, M=16)
+            index.add_items(library_norm.astype(np.float32), np.arange(library_norm.shape[0]))
+            index.set_ef(max(50, k_eff))
+            knn_idx, dist = index.knn_query(query_norm.astype(np.float32), k=k_eff)
+            knn_sim = 1.0 - dist
+        else:
+            raise ValueError(name)
+
+    for i in range(query_df.shape[0]):
+        order = knn_idx[i]
+        top_sim = knn_sim[i]
+        top_targets = target_lib[order]
+        query_target = float(query_df.iloc[i]["target_log_burn_acres"])
+        relevance = graded_relevance(query_target, top_targets)
+        ideal_relevance = np.sort(graded_relevance(query_target, target_lib))[::-1]
+        abs_delta = np.abs(top_targets - float(query_df.iloc[i]["target_log_burn_acres"]))
+        ndcg5.append(ndcg_at_k(relevance, ideal_relevance, 5))
+        ndcg10.append(ndcg_at_k(relevance, ideal_relevance, 10))
+        hit1.append(float(relevance[:1].max() >= 2.0))
+        hit5.append(float(relevance[: min(5, k_eff)].max() >= 2.0))
+        hit10.append(float(relevance[: min(10, k_eff)].max() >= 2.0))
+        best_abs_delta.append(float(abs_delta.min()))
+        pred = float(np.mean(top_targets)) if mode == "mean" else weighted_prediction(top_sim, top_targets)
+        preds.append(pred)
+        rows.append(
+            {
+                "query_event_id": query_df.iloc[i]["Event_ID"],
+                "true_log_burn_acres": float(query_df.iloc[i]["target_log_burn_acres"]),
+                "pred_log_burn_acres": pred,
+                "backend": name,
+                "k": k,
+                "effective_k": k_eff,
+                "mode": mode,
+                "top_relevance": relevance.tolist(),
+                "best_abs_log_delta": float(abs_delta.min()),
+            }
+        )
+
+    pred_arr = np.asarray(preds, dtype=np.float64)
+    true_log = query_df["target_log_burn_acres"].to_numpy(dtype=np.float64)
+    true_acres = query_df["BurnBndAc"].to_numpy(dtype=np.float64)
+    pred_acres = np.exp(pred_arr)
+    metrics = {
+        "count": int(len(query_df)),
+        "log_mae": float(np.mean(np.abs(true_log - pred_arr))),
+        "log_rmse": rmse(true_log, pred_arr),
+        "log_r2": r2_score_manual(true_log, pred_arr),
+        "log_spearman": spearman_corr(true_log, pred_arr),
+        "log_median_ae": float(np.median(np.abs(true_log - pred_arr))),
+        "acres_mae": float(np.mean(np.abs(true_acres - pred_acres))),
+        "acres_rmse": rmse(true_acres, pred_acres),
+        "acres_median_ae": float(np.median(np.abs(true_acres - pred_acres))),
+        "acres_mape": mape(true_acres, pred_acres),
+        "ndcg_at_5": float(np.mean(ndcg5)) if ndcg5 else 0.0,
+        "ndcg_at_10": float(np.mean(ndcg10)) if ndcg10 else 0.0,
+        "hit_at_1_tol1": float(np.mean(hit1)) if hit1 else 0.0,
+        "hit_at_5_tol1": float(np.mean(hit5)) if hit5 else 0.0,
+        "hit_at_10_tol1": float(np.mean(hit10)) if hit10 else 0.0,
+        "mean_best_abs_log_delta_at_k": float(np.mean(best_abs_delta)) if best_abs_delta else 0.0,
+    }
+    return metrics, pd.DataFrame(rows)
+
+
+def target_weight_vectors(train_vec: np.ndarray, val_vec: np.ndarray, test_vec: np.ndarray, target: np.ndarray) -> Tuple[np.ndarray, np.ndarray, np.ndarray]:
+    x = np.asarray(train_vec, dtype=np.float64)
+    y = np.asarray(target, dtype=np.float64)
+    y = y - y.mean()
+    x_centered = x - x.mean(axis=0, keepdims=True)
+    denom = np.clip(np.sqrt(np.sum(x_centered**2, axis=0)) * np.sqrt(np.sum(y**2)), 1e-12, None)
+    corr = np.abs(np.sum(x_centered * y[:, None], axis=0) / denom)
+    corr = np.nan_to_num(corr, nan=0.0, posinf=0.0, neginf=0.0)
+    if float(corr.max()) > 0:
+        corr = corr / float(corr.max())
+    weights = (0.25 + corr).astype(np.float32)
+    return train_vec * weights, val_vec * weights, test_vec * weights
+
+
+def main() -> None:
+    parser = argparse.ArgumentParser()
+    parser.add_argument("--event-table", type=Path, required=True)
+    parser.add_argument("--output-dir", type=Path, required=True)
+    parser.add_argument("--selection-metric", choices=("log_mae", "ndcg_at_10"), default="ndcg_at_10")
+    parser.add_argument("--feature-profile", choices=("all", "weather_fm"), default="all")
+    parser.add_argument("--fm-family", type=str, default="")
+    parser.add_argument("--seed", type=int, default=7)
+    args = parser.parse_args()
+
+    df = pd.read_csv(args.event_table)
+    df["Ig_Date"] = pd.to_datetime(df["Ig_Date"])
+    train_df, val_df, test_df = build_splits(df)
+    numeric_cols, categorical_cols = feature_columns(df, feature_profile=args.feature_profile)
+    if not numeric_cols and not categorical_cols:
+        raise SystemExit(f"No usable features found for profile={args.feature_profile}")
+    x_cols = numeric_cols + categorical_cols
+    pre = make_preprocessor(numeric_cols, categorical_cols)
+    train_vec = to_dense_float32(pre.fit_transform(train_df[x_cols]))
+    val_vec = to_dense_float32(pre.transform(val_df[x_cols]))
+    test_vec = to_dense_float32(pre.transform(test_df[x_cols]))
+    weighted_train_vec, weighted_val_vec, weighted_test_vec = target_weight_vectors(
+        train_vec,
+        val_vec,
+        test_vec,
+        train_df["target_log_burn_acres"].to_numpy(dtype=np.float64),
+    )
+    vector_variants = {
+        "standard": (train_vec, val_vec, test_vec),
+        "target_weighted": (weighted_train_vec, weighted_val_vec, weighted_test_vec),
+    }
+
+    candidate_validation: List[Dict[str, object]] = []
+    best = None
+    best_score = None
+    best_val_rows = None
+    best_test_rows = None
+    for variant, (lib_vec, v_vec, _) in vector_variants.items():
+        for backend in ["cosine_exact", "faiss_flat_ip", "hnsw_cosine"]:
+            for k in [1, 3, 5, 10]:
+                for mode in ["mean", "weighted"]:
+                    val_metrics, val_rows = score_backend(backend, v_vec, lib_vec, val_df, train_df, k, mode)
+                    candidate_validation.append({"variant": variant, "backend": backend, "k": k, "mode": mode, "val_metrics": val_metrics})
+                    score = float(val_metrics[args.selection_metric])
+                    better = score > best_score if args.selection_metric == "ndcg_at_10" and best_score is not None else score < best_score if best_score is not None else True
+                    if better:
+                        best_score = score
+                        best = {"variant": variant, "backend": backend, "k": k, "mode": mode}
+                        best_val_rows = val_rows
+
+    assert best is not None
+    best_train_vec, _, best_test_vec = vector_variants[str(best["variant"])]
+    test_metrics, test_rows = score_backend(best["backend"], best_test_vec, best_train_vec, test_df, train_df, int(best["k"]), str(best["mode"]))
+    best_test_rows = test_rows
+
+    args.output_dir.mkdir(parents=True, exist_ok=True)
+    if best_val_rows is not None:
+        best_val_rows.to_csv(args.output_dir / "val_retrieval_examples.csv", index=False)
+    if best_test_rows is not None:
+        best_test_rows.to_csv(args.output_dir / "test_retrieval_examples.csv", index=False)
+
+    summary = {
+        "task_id": "wildfire_analog_retrieval_taskmodels",
+        "task_form": "event_level_retrieval_with_induced_outcome_error",
+        "event_table": str(args.event_table),
+        "output_dir": str(args.output_dir),
+        "feature_profile": args.feature_profile,
+        "seed": int(args.seed),
+        "split_sizes": {
+            "train": int(len(train_df)),
+            "val": int(len(val_df)),
+            "test": int(len(test_df)),
+        },
+        "feature_columns": {"numeric": numeric_cols, "categorical": categorical_cols},
+        "candidate_validation": candidate_validation,
+        "selected_retrieval": best,
+        "selection_metric": args.selection_metric,
+        "test_metrics": test_metrics,
+        "model_family": "popular_open_source_retrieval_backends_with_train_only_target_weighting",
+        "fm_family": (args.fm_family or "weather_fm_derived_features") if args.feature_profile == "weather_fm" else None,
+    }
+    (args.output_dir / "summary.json").write_text(json.dumps(summary, indent=2), encoding="utf-8")
+    print(json.dumps(summary, indent=2))
+
+
+if __name__ == "__main__":
+    main()

experiments/raw_reference/task_scripts/run_extreme_heat_alphaearth_suite_seeded.py

@@ -0,0 +1,344 @@
+#!/usr/bin/env python3
+from __future__ import annotations
+
+import argparse
+import json
+import math
+import re
+import sys
+from pathlib import Path
+from typing import Dict, Iterable, List, Optional, Tuple
+
+
+import os
+
+for _p in os.environ.get("WILDFIRE_FM_EXTRA_PYTHONPATH", "").split(os.pathsep):
+    if _p and _p not in sys.path:
+        sys.path.insert(0, _p)
+
+import numpy as np
+import pandas as pd
+from catboost import CatBoostRegressor
+from lightgbm import LGBMRegressor
+from netCDF4 import Dataset
+from sklearn.linear_model import ElasticNet, Ridge
+from sklearn.metrics import mean_absolute_error, mean_squared_error
+from xgboost import XGBRegressor
+
+
+PRED_RE = re.compile(r"pred_(\d{8})_(\d{2})\.nc$")
+WEATHER_VARS = ["T2M", "QV2M", "TQV", "U10M", "V10M", "TS"]
+
+
+def parse_args() -> argparse.Namespace:
+    parser = argparse.ArgumentParser()
+    parser.add_argument("--pred-root", type=Path, action="append", required=True)
+    parser.add_argument("--merra-root", type=Path, required=True)
+    parser.add_argument("--alphaearth-year-csv", type=Path, required=True)
+    parser.add_argument("--output-dir", type=Path, required=True)
+    parser.add_argument("--model-family", choices=("full", "lite"), default="full")
+    parser.add_argument("--alphaearth-prefix", type=str, default="alphaearth_")
+    parser.add_argument("--lat-min", type=float, default=24.0)
+    parser.add_argument("--lat-max", type=float, default=50.0)
+    parser.add_argument("--lon-min", type=float, default=-125.0)
+    parser.add_argument("--lon-max", type=float, default=-66.0)
+    parser.add_argument("--seed", type=int, default=7)
+    return parser.parse_args()
+
+
+def choose_split(path: Path) -> Optional[str]:
+    name = path.name
+    if name == "output2022":
+        return "train"
+    if name == "output2024":
+        return "val"
+    if name == "output2025":
+        return "test"
+    return None
+
+
+def parse_pred_timestamp(path: Path) -> Tuple[str, int]:
+    match = PRED_RE.match(path.name)
+    if not match:
+        raise ValueError(f"Unexpected prediction filename: {path}")
+    return match.group(1), int(match.group(2))
+
+
+def nearest_indices(src: np.ndarray, dst: np.ndarray) -> np.ndarray:
+    idx = np.searchsorted(dst, src)
+    idx = np.clip(idx, 0, len(dst) - 1)
+    prev_idx = np.clip(idx - 1, 0, len(dst) - 1)
+    choose_prev = np.abs(dst[prev_idx] - src) <= np.abs(dst[idx] - src)
+    return np.where(choose_prev, prev_idx, idx).astype(np.int64)
+
+
+def build_grid_alignment(sample_pred: Path, sample_merra: Path, lat_min: float, lat_max: float, lon_min: float, lon_max: float) -> Dict[str, np.ndarray]:
+    with Dataset(sample_pred) as pred_ds, Dataset(sample_merra) as merra_ds:
+        pred_lat = np.asarray(pred_ds.variables["lat"][:], dtype=np.float64)
+        pred_lon = np.asarray(pred_ds.variables["lon"][:], dtype=np.float64)
+        merra_lat = np.asarray(merra_ds.variables["lat"][:], dtype=np.float64)
+        merra_lon = np.asarray(merra_ds.variables["lon"][:], dtype=np.float64)
+
+    lat_mask = (pred_lat >= lat_min) & (pred_lat <= lat_max)
+    lon_mask = (pred_lon >= lon_min) & (pred_lon <= lon_max)
+    pred_lat_idx = np.flatnonzero(lat_mask)
+    pred_lon_idx = np.flatnonzero(lon_mask)
+    pred_lat_sel = pred_lat[pred_lat_idx]
+    pred_lon_sel = pred_lon[pred_lon_idx]
+    merra_lat_idx = nearest_indices(pred_lat_sel, merra_lat)
+    merra_lon_idx = nearest_indices(pred_lon_sel, merra_lon)
+
+    return {
+        "pred_lat_idx": pred_lat_idx,
+        "pred_lon_idx": pred_lon_idx,
+        "merra_lat_idx": merra_lat_idx,
+        "merra_lon_idx": merra_lon_idx,
+    }
+
+
+def feature_stats(arr: np.ndarray) -> Dict[str, float]:
+    return {"mean": float(np.mean(arr)), "max": float(np.max(arr)), "std": float(np.std(arr))}
+
+
+def build_rows(pred_roots: Iterable[Path], merra_root: Path, alignment: Dict[str, np.ndarray]) -> pd.DataFrame:
+    rows: List[Dict[str, float]] = []
+    for root in pred_roots:
+        split = choose_split(root)
+        if split is None:
+            continue
+        for path in sorted(root.glob("pred_*.nc")):
+            day, hour = parse_pred_timestamp(path)
+            if hour % 3 != 0:
+                continue
+            merra_path = merra_root / f"MERRA2_sfc_{day}.nc"
+            if not merra_path.exists():
+                continue
+            time_index = hour // 3
+            with Dataset(path) as pred_ds, Dataset(merra_path) as merra_ds:
+                date = pd.Timestamp(day)
+                record: Dict[str, float] = {"split": split, "hour": float(hour), "year": float(date.year), "date": day}
+                record["doy"] = float(date.dayofyear)
+                record["month"] = float(date.month)
+                for var in WEATHER_VARS:
+                    pred_arr = np.asarray(
+                        pred_ds.variables[var][0, alignment["pred_lat_idx"], alignment["pred_lon_idx"]],
+                        dtype=np.float64,
+                    )
+                    stats = feature_stats(pred_arr)
+                    record[f"pred_{var.lower()}_mean"] = stats["mean"]
+                    record[f"pred_{var.lower()}_max"] = stats["max"]
+                    record[f"pred_{var.lower()}_std"] = stats["std"]
+                record["pred_wind_mean"] = float(
+                    np.mean(
+                        np.sqrt(
+                            np.square(pred_ds.variables["U10M"][0, alignment["pred_lat_idx"], alignment["pred_lon_idx"]])
+                            + np.square(pred_ds.variables["V10M"][0, alignment["pred_lat_idx"], alignment["pred_lon_idx"]])
+                        )
+                    )
+                )
+                truth_t2m = np.asarray(merra_ds.variables["T2M"][time_index], dtype=np.float64)[
+                    np.ix_(alignment["merra_lat_idx"], alignment["merra_lon_idx"])
+                ]
+                truth_ts = np.asarray(merra_ds.variables["TS"][time_index], dtype=np.float64)[
+                    np.ix_(alignment["merra_lat_idx"], alignment["merra_lon_idx"])
+                ]
+                record["target_t2m_mean_c"] = float(np.mean(truth_t2m) - 273.15)
+                record["target_t2m_max_c"] = float(np.max(truth_t2m) - 273.15)
+                record["target_ts_mean_c"] = float(np.mean(truth_ts) - 273.15)
+            rows.append(record)
+
+    if not rows:
+        raise SystemExit("No extreme-heat rows were built from the provided roots.")
+    df = pd.DataFrame(rows)
+    angle_day = 2.0 * np.pi * df["doy"].to_numpy(dtype=np.float64) / 366.0
+    angle_hour = 2.0 * np.pi * df["hour"].to_numpy(dtype=np.float64) / 24.0
+    df["doy_sin"] = np.sin(angle_day)
+    df["doy_cos"] = np.cos(angle_day)
+    df["hour_sin"] = np.sin(angle_hour)
+    df["hour_cos"] = np.cos(angle_hour)
+    return df
+
+
+def drop_nonfinite_rows(df: pd.DataFrame, columns: List[str]) -> pd.DataFrame:
+    mask = np.ones(len(df), dtype=bool)
+    for col in columns:
+        mask &= np.isfinite(pd.to_numeric(df[col], errors="coerce").to_numpy(dtype=np.float64))
+    return df.loc[mask].reset_index(drop=True)
+
+
+def rmse(y_true: np.ndarray, y_pred: np.ndarray) -> float:
+    return float(math.sqrt(mean_squared_error(y_true, y_pred)))
+
+
+def pearson_corr(y_true: np.ndarray, y_pred: np.ndarray) -> float:
+    a = np.asarray(y_true, dtype=np.float64)
+    b = np.asarray(y_pred, dtype=np.float64)
+    if a.size < 2 or np.allclose(a, a[0]) or np.allclose(b, b[0]):
+        return 0.0
+    value = float(np.corrcoef(a, b)[0, 1])
+    return value if np.isfinite(value) else 0.0
+
+
+def prf(y_true: np.ndarray, y_pred: np.ndarray, threshold: float) -> Dict[str, float]:
+    truth = np.asarray(y_true >= threshold)
+    pred = np.asarray(y_pred >= threshold)
+    tp = int(np.logical_and(pred, truth).sum())
+    fp = int(np.logical_and(pred, ~truth).sum())
+    fn = int(np.logical_and(~pred, truth).sum())
+    precision = float(tp / (tp + fp)) if (tp + fp) else 0.0
+    recall = float(tp / (tp + fn)) if (tp + fn) else 0.0
+    f1 = float((2.0 * precision * recall) / (precision + recall)) if (precision + recall) else 0.0
+    return {"precision": precision, "recall": recall, "f1": f1}
+
+
+def evaluate(y_true: np.ndarray, y_pred: np.ndarray) -> Dict[str, float]:
+    return {
+        "count": int(y_true.shape[0]),
+        "rmse_c": rmse(y_true, y_pred),
+        "mae_c": float(mean_absolute_error(y_true, y_pred)),
+        "pearson_r": pearson_corr(y_true, y_pred),
+    }
+
+
+def main() -> None:
+    args = parse_args()
+    pred_files = [path for root in args.pred_root for path in root.glob("pred_*.nc")]
+    if not pred_files:
+        raise SystemExit("No prediction files found.")
+    sample_pred = sorted(pred_files)[0]
+    sample_day, _ = parse_pred_timestamp(sample_pred)
+    sample_merra = args.merra_root / f"MERRA2_sfc_{sample_day}.nc"
+    if not sample_merra.exists():
+        raise SystemExit(f"Sample MERRA file missing: {sample_merra}")
+
+    alignment = build_grid_alignment(
+        sample_pred,
+        sample_merra,
+        lat_min=args.lat_min,
+        lat_max=args.lat_max,
+        lon_min=args.lon_min,
+        lon_max=args.lon_max,
+    )
+    df = build_rows(args.pred_root, args.merra_root, alignment)
+
+    alpha = pd.read_csv(args.alphaearth_year_csv)
+    alpha["source_year_key"] = pd.to_numeric(alpha["alphaearth_source_year"], errors="coerce").astype("Int64")
+    df["source_year_key"] = pd.to_numeric(df["year"], errors="coerce").clip(lower=2017, upper=2024).astype("Int64")
+    df = df.merge(alpha.drop(columns=[c for c in ["requested_year"] if c in alpha.columns]), on="source_year_key", how="left")
+
+    feature_cols = [c for c in df.columns if c.startswith("pred_") or c in {"month", "doy_sin", "doy_cos", "hour_sin", "hour_cos"}]
+    feature_cols.extend(sorted([c for c in df.columns if c.startswith(args.alphaearth_prefix)]))
+    finite_cols = feature_cols + ["target_t2m_mean_c"]
+    df = drop_nonfinite_rows(df, finite_cols)
+    if df.empty:
+        raise SystemExit("Extreme-heat AlphaEarth suite has no finite rows after filtering.")
+
+    train = df[df["split"] == "train"].copy()
+    val = df[df["split"] == "val"].copy()
+    test = df[df["split"] == "test"].copy()
+    if len(train) == 0 or len(val) == 0 or len(test) == 0:
+        raise SystemExit("Extreme-heat AlphaEarth suite is missing one of train/val/test.")
+
+    x_train = train[feature_cols].to_numpy(dtype=np.float64)
+    x_val = val[feature_cols].to_numpy(dtype=np.float64)
+    x_test = test[feature_cols].to_numpy(dtype=np.float64)
+    y_train = train["target_t2m_mean_c"].to_numpy(dtype=np.float64)
+    y_val = val["target_t2m_mean_c"].to_numpy(dtype=np.float64)
+    y_test = test["target_t2m_mean_c"].to_numpy(dtype=np.float64)
+
+    candidates: Dict[str, object] = {
+        "ridge": Ridge(alpha=1.0, random_state=args.seed),
+        "enet": ElasticNet(alpha=0.01, l1_ratio=0.2, random_state=args.seed, max_iter=10000),
+    }
+    if args.model_family == "full":
+        candidates.update(
+            {
+                "xgboost": XGBRegressor(
+                    n_estimators=300,
+                    max_depth=6,
+                    learning_rate=0.05,
+                    subsample=0.8,
+                    colsample_bytree=0.8,
+                    objective="reg:squarederror",
+                    tree_method="hist",
+                    random_state=args.seed,
+                    n_jobs=8,
+                ),
+                "lightgbm": LGBMRegressor(
+                    n_estimators=300,
+                    learning_rate=0.05,
+                    num_leaves=63,
+                    subsample=0.8,
+                    colsample_bytree=0.8,
+                    random_state=args.seed,
+                    n_jobs=8,
+                    verbose=-1,
+                ),
+                "catboost": CatBoostRegressor(
+                    iterations=400,
+                    depth=8,
+                    learning_rate=0.05,
+                    loss_function="RMSE",
+                    eval_metric="RMSE",
+                    random_seed=args.seed,
+                    verbose=False,
+                ),
+            }
+        )
+
+    candidate_rows = []
+    best_name = None
+    best_model = None
+    best_rmse = None
+    for name, model in candidates.items():
+        model.fit(x_train, y_train)
+        val_pred = np.asarray(model.predict(x_val), dtype=np.float64)
+        val_metrics = evaluate(y_val, val_pred)
+        candidate_rows.append({"model": name, "validation": val_metrics})
+        if best_rmse is None or val_metrics["rmse_c"] < best_rmse:
+            best_name = name
+            best_model = model
+            best_rmse = val_metrics["rmse_c"]
+
+    assert best_model is not None and best_name is not None
+    val_pred = np.asarray(best_model.predict(x_val), dtype=np.float64)
+    test_pred = np.asarray(best_model.predict(x_test), dtype=np.float64)
+
+    thresholds = [27.0, 30.0, 33.0]
+    val_events = [{"threshold_c": t, **prf(y_val, val_pred, t)} for t in thresholds]
+    val_events = sorted(val_events, key=lambda row: (-row["f1"], -row["recall"], -row["precision"], row["threshold_c"]))
+    selected_event = val_events[0]
+    test_event = {"threshold_c": selected_event["threshold_c"], **prf(y_test, test_pred, selected_event["threshold_c"])}
+
+    summary = {
+        "task_id": "extreme_heat_alphaearth",
+        "core_line": "extreme_heat",
+        "task_form": "continuous_temperature_forecast_with_secondary_exceedance_view",
+        "seed": int(args.seed),
+        "pred_roots": [str(path) for path in args.pred_root],
+        "merra_root": str(args.merra_root),
+        "alphaearth_year_csv": str(args.alphaearth_year_csv),
+        "model_family": args.model_family,
+        "feature_columns": feature_cols,
+        "alphaearth_feature_count": int(sum(c.startswith(args.alphaearth_prefix) for c in feature_cols)),
+        "candidate_validation": candidate_rows,
+        "selected_model": best_name,
+        "validation_metrics": evaluate(y_val, val_pred),
+        "test_metrics": evaluate(y_test, test_pred),
+        "selected_event_candidate": selected_event,
+        "selected_event_candidate_test": test_event,
+        "selection_rule": "same heat benchmark; choose regressor by validation RMSE and choose exceedance threshold by validation F1",
+        "tmt_policy": {
+            "task": "extreme_heat",
+            "metric": "continuous RMSE/MAE with thresholded exceedance as a secondary event policy",
+            "tolerance": "none for continuous headline; event threshold only for operational view"
+        },
+    }
+
+    args.output_dir.mkdir(parents=True, exist_ok=True)
+    (args.output_dir / "summary.json").write_text(json.dumps(summary, indent=2), encoding="utf-8")
+    print(json.dumps(summary, indent=2))
+
+
+if __name__ == "__main__":
+    main()

experiments/raw_reference/task_scripts/run_final_area_taskmodel_seeded.py

@@ -0,0 +1,353 @@
+#!/usr/bin/env python3
+from __future__ import annotations
+
+import argparse
+import json
+import sys
+from pathlib import Path
+from typing import Dict, List, Tuple
+
+
+import os
+
+for _p in os.environ.get("WILDFIRE_FM_EXTRA_PYTHONPATH", "").split(os.pathsep):
+    if _p and _p not in sys.path:
+        sys.path.insert(0, _p)
+
+import numpy as np
+import pandas as pd
+from catboost import CatBoostRegressor
+from lightgbm import LGBMRegressor
+from sklearn.compose import ColumnTransformer
+from sklearn.impute import SimpleImputer
+from sklearn.linear_model import ElasticNet
+from sklearn.metrics import mean_absolute_error, mean_squared_error, r2_score
+from sklearn.pipeline import Pipeline
+from sklearn.preprocessing import OneHotEncoder, StandardScaler
+from xgboost import XGBRegressor
+
+
+DROP_COLUMNS = {
+    "Event_ID",
+    "Incid_Name",
+    "incident_name_norm",
+    "wfigs_name",
+    "Ig_Date",
+    "weather_date",
+    "BurnBndAc",
+    "target_log_burn_acres",
+}
+
+CATEGORICAL_COLUMNS = [
+    "Incid_Type",
+    "state_abbr",
+    "county_name",
+    "wfigs_match_type",
+]
+
+
+def rmse(y_true: np.ndarray, y_pred: np.ndarray) -> float:
+    return float(np.sqrt(mean_squared_error(y_true, y_pred)))
+
+
+def mape(y_true: np.ndarray, y_pred: np.ndarray) -> float:
+    denom = np.clip(np.asarray(y_true, dtype=np.float64), 1e-6, None)
+    frac = np.abs(np.asarray(y_true, dtype=np.float64) - np.asarray(y_pred, dtype=np.float64)) / denom
+    return float(np.mean(frac))
+
+
+def spearman_corr(y_true: np.ndarray, y_pred: np.ndarray) -> float:
+    a = pd.Series(np.asarray(y_true, dtype=np.float64))
+    b = pd.Series(np.asarray(y_pred, dtype=np.float64))
+    value = a.corr(b, method="spearman")
+    return float(value) if pd.notna(value) else 0.0
+
+
+def build_splits(df: pd.DataFrame) -> Tuple[pd.DataFrame, pd.DataFrame, pd.DataFrame]:
+    ordered = df.sort_values("Ig_Date").reset_index(drop=True)
+    n = len(ordered)
+    train_end = max(int(round(n * 0.6)), 1)
+    val_end = max(int(round(n * 0.8)), train_end + 1)
+    val_end = min(val_end, n - 1) if n >= 3 else n
+    train = ordered.iloc[:train_end].copy()
+    val = ordered.iloc[train_end:val_end].copy()
+    test = ordered.iloc[val_end:].copy()
+    if len(val) == 0 and len(test) > 1:
+        val = test.iloc[:1].copy()
+        test = test.iloc[1:].copy()
+    return train, val, test
+
+
+def feature_columns(df: pd.DataFrame, feature_profile: str = "all") -> Tuple[List[str], List[str]]:
+    categorical = [c for c in CATEGORICAL_COLUMNS if c in df.columns]
+    numeric = []
+    for col in df.columns:
+        if col in DROP_COLUMNS or col in categorical:
+            continue
+        if pd.api.types.is_numeric_dtype(df[col]):
+            numeric.append(col)
+    if feature_profile == "weather_fm":
+        numeric = [c for c in numeric if c.startswith("weather_")]
+        categorical = []
+    return numeric, categorical
+
+
+def make_sparse_preprocessor(numeric_cols: List[str], categorical_cols: List[str]) -> ColumnTransformer:
+    return ColumnTransformer(
+        transformers=[
+            (
+                "num",
+                Pipeline(
+                    steps=[
+                        ("impute", SimpleImputer(strategy="median")),
+                        ("scale", StandardScaler()),
+                    ]
+                ),
+                numeric_cols,
+            ),
+            (
+                "cat",
+                Pipeline(
+                    steps=[
+                        ("impute", SimpleImputer(strategy="most_frequent")),
+                        ("onehot", OneHotEncoder(handle_unknown="ignore")),
+                    ]
+                ),
+                categorical_cols,
+            ),
+        ],
+        remainder="drop",
+    )
+
+
+def prepare_catboost_frames(
+    train_df: pd.DataFrame,
+    val_df: pd.DataFrame,
+    test_df: pd.DataFrame,
+    numeric_cols: List[str],
+    categorical_cols: List[str],
+) -> Tuple[pd.DataFrame, pd.DataFrame, pd.DataFrame]:
+    medians = {c: float(train_df[c].median()) for c in numeric_cols}
+    modes = {
+        c: str(train_df[c].mode(dropna=True).iloc[0]) if not train_df[c].mode(dropna=True).empty else "missing"
+        for c in categorical_cols
+    }
+
+    def _prep(frame: pd.DataFrame) -> pd.DataFrame:
+        out = frame[numeric_cols + categorical_cols].copy()
+        for col in numeric_cols:
+            out[col] = pd.to_numeric(out[col], errors="coerce").fillna(medians[col])
+        for col in categorical_cols:
+            out[col] = out[col].astype("string").fillna(modes[col]).astype(str)
+        return out
+
+    return _prep(train_df), _prep(val_df), _prep(test_df)
+
+
+def evaluate_split(frame: pd.DataFrame, pred_log: np.ndarray) -> Dict[str, float]:
+    true_log = frame["target_log_burn_acres"].to_numpy(dtype=np.float64)
+    true_acres = frame["BurnBndAc"].to_numpy(dtype=np.float64)
+    pred_log = np.asarray(pred_log, dtype=np.float64)
+    pred_acres = np.exp(pred_log)
+    return {
+        "count": int(len(frame)),
+        "log_mae": float(mean_absolute_error(true_log, pred_log)),
+        "log_rmse": rmse(true_log, pred_log),
+        "log_r2": float(r2_score(true_log, pred_log)) if len(frame) > 1 else 0.0,
+        "log_spearman": spearman_corr(true_log, pred_log),
+        "log_median_ae": float(np.median(np.abs(true_log - pred_log))),
+        "acres_mae": float(mean_absolute_error(true_acres, pred_acres)),
+        "acres_rmse": rmse(true_acres, pred_acres),
+        "acres_median_ae": float(np.median(np.abs(true_acres - pred_acres))),
+        "acres_mape": mape(true_acres, pred_acres),
+    }
+
+
+def main() -> None:
+    parser = argparse.ArgumentParser()
+    parser.add_argument("--event-table", type=Path, required=True)
+    parser.add_argument("--output-dir", type=Path, required=True)
+    parser.add_argument("--feature-profile", choices=("all", "weather_fm"), default="all")
+    parser.add_argument("--model-family", choices=("full", "lite"), default="full")
+    parser.add_argument("--fm-family", type=str, default="")
+    parser.add_argument("--seed", type=int, default=7)
+    args = parser.parse_args()
+
+    df = pd.read_csv(args.event_table)
+    df["Ig_Date"] = pd.to_datetime(df["Ig_Date"])
+    train_df, val_df, test_df = build_splits(df)
+    numeric_cols, categorical_cols = feature_columns(df, feature_profile=args.feature_profile)
+    if not numeric_cols and not categorical_cols:
+        raise SystemExit(f"No usable features found for profile={args.feature_profile}")
+    x_cols = numeric_cols + categorical_cols
+
+    pre = make_sparse_preprocessor(numeric_cols, categorical_cols)
+    x_train = pre.fit_transform(train_df[x_cols])
+    x_val = pre.transform(val_df[x_cols])
+    x_test = pre.transform(test_df[x_cols])
+    y_train = train_df["target_log_burn_acres"].to_numpy(dtype=np.float64)
+
+    cat_train, cat_val, cat_test = prepare_catboost_frames(train_df, val_df, test_df, numeric_cols, categorical_cols)
+    cat_feature_idx = list(range(len(numeric_cols), len(numeric_cols) + len(categorical_cols)))
+
+    candidates: List[Tuple[str, object, str]] = [
+        (
+            "enet",
+            ElasticNet(alpha=0.01, l1_ratio=0.2, random_state=args.seed, max_iter=10000),
+            "sparse",
+        ),
+    ]
+    if args.model_family == "full":
+        candidates.extend(
+            [
+                (
+                    "xgboost",
+                    XGBRegressor(
+                        n_estimators=400,
+                        max_depth=6,
+                        learning_rate=0.05,
+                        subsample=0.8,
+                        colsample_bytree=0.8,
+                        reg_lambda=1.0,
+                        objective="reg:squarederror",
+                        tree_method="hist",
+                        random_state=args.seed,
+                        n_jobs=8,
+                    ),
+                    "sparse",
+                ),
+                (
+                    "lightgbm",
+                    LGBMRegressor(
+                        n_estimators=400,
+                        learning_rate=0.05,
+                        num_leaves=63,
+                        subsample=0.8,
+                        colsample_bytree=0.8,
+                        reg_lambda=1.0,
+                        random_state=args.seed,
+                        n_jobs=8,
+                        verbose=-1,
+                    ),
+                    "sparse",
+                ),
+                (
+                    "catboost",
+                    CatBoostRegressor(
+                        iterations=500,
+                        depth=8,
+                        learning_rate=0.05,
+                        loss_function="RMSE",
+                        eval_metric="RMSE",
+                        random_seed=args.seed,
+                        verbose=False,
+                    ),
+                    "cat",
+                ),
+            ]
+        )
+
+    candidate_validation: List[Dict[str, object]] = []
+    best_name = None
+    best_kind = None
+    best_model = None
+    best_score = None
+
+    for name, model, kind in candidates:
+        if kind == "sparse":
+            model.fit(x_train, y_train)
+            val_pred = model.predict(x_val)
+        else:
+            model.fit(cat_train, y_train, cat_features=cat_feature_idx, eval_set=(cat_val, val_df["target_log_burn_acres"]), use_best_model=False)
+            val_pred = model.predict(cat_val)
+        val_metrics = evaluate_split(val_df, val_pred)
+        candidate_validation.append({"model_name": name, "val_metrics": val_metrics})
+        score = float(val_metrics["log_mae"])
+        if best_score is None or score < best_score:
+            best_score = score
+            best_name = name
+            best_kind = kind
+            best_model = model
+
+    assert best_model is not None and best_name is not None and best_kind is not None
+
+    combined_train = pd.concat([train_df, val_df], ignore_index=True)
+    if best_kind == "sparse":
+        x_combined = pre.fit_transform(combined_train[x_cols])
+        x_train_final = pre.transform(train_df[x_cols])
+        x_val_final = pre.transform(val_df[x_cols])
+        x_test_final = pre.transform(test_df[x_cols])
+        best_model.fit(x_combined, combined_train["target_log_burn_acres"].to_numpy(dtype=np.float64))
+        train_pred = best_model.predict(x_train_final)
+        val_pred = best_model.predict(x_val_final)
+        test_pred = best_model.predict(x_test_final)
+    else:
+        cat_combined, cat_train_final, cat_test_final = prepare_catboost_frames(
+            combined_train, train_df, test_df, numeric_cols, categorical_cols
+        )
+        cat_val_final = prepare_catboost_frames(val_df, val_df, val_df, numeric_cols, categorical_cols)[0]
+        best_model.fit(
+            cat_combined,
+            combined_train["target_log_burn_acres"].to_numpy(dtype=np.float64),
+            cat_features=cat_feature_idx,
+            use_best_model=False,
+        )
+        train_pred = best_model.predict(cat_train_final)
+        val_pred = best_model.predict(cat_val_final)
+        test_pred = best_model.predict(cat_test_final)
+
+    args.output_dir.mkdir(parents=True, exist_ok=True)
+    pred_df = pd.concat(
+        [
+            train_df.assign(split="train", pred_log_burn_acres=train_pred, pred_burn_acres=np.exp(train_pred)),
+            val_df.assign(split="val", pred_log_burn_acres=val_pred, pred_burn_acres=np.exp(val_pred)),
+            test_df.assign(split="test", pred_log_burn_acres=test_pred, pred_burn_acres=np.exp(test_pred)),
+        ],
+        axis=0,
+        ignore_index=True,
+    )
+    pred_path = args.output_dir / "predictions.csv"
+    pred_df.to_csv(pred_path, index=False)
+
+    summary = {
+        "task_id": "wildfire_final_area_scalar_taskmodels",
+        "task_form": "event_level_regression",
+        "event_table": str(args.event_table),
+        "output_dir": str(args.output_dir),
+        "feature_profile": args.feature_profile,
+        "seed": int(args.seed),
+        "benchmark_protocol": "fm_lite_protocol" if args.feature_profile == "weather_fm" and args.model_family == "lite" else "standard_protocol",
+        "split_sizes": {
+            "train": int(len(train_df)),
+            "val": int(len(val_df)),
+            "test": int(len(test_df)),
+        },
+        "feature_columns": {
+            "numeric": numeric_cols,
+            "categorical": categorical_cols,
+        },
+        "candidate_validation": candidate_validation,
+        "selected_model": best_name,
+        "train_metrics": evaluate_split(train_df, train_pred),
+        "val_metrics": evaluate_split(val_df, val_pred),
+        "test_metrics": evaluate_split(test_df, test_pred),
+        "headline_metrics": {
+            "log_mae": float(evaluate_split(test_df, test_pred)["log_mae"]),
+            "log_rmse": float(evaluate_split(test_df, test_pred)["log_rmse"]),
+            "log_spearman": float(evaluate_split(test_df, test_pred)["log_spearman"]),
+        },
+        "predictions_path": str(pred_path),
+        "model_family": "lightweight_linear_task_heads" if args.model_family == "lite" else "popular_open_source_task_models",
+        "fm_family": (args.fm_family or "weather_fm_derived_features") if args.feature_profile == "weather_fm" else None,
+        "tmt_policy": {
+            "task": "final_burned_area",
+            "metric": "log-area regression error with rank agreement",
+            "tolerance": "secondary magnitude-band interpretation only",
+        },
+    }
+    (args.output_dir / "summary.json").write_text(json.dumps(summary, indent=2), encoding="utf-8")
+    print(json.dumps(summary, indent=2))
+
+
+if __name__ == "__main__":
+    main()

experiments/raw_reference/task_scripts/run_smoke_pm25_alphaearth_suite_seeded.py

@@ -0,0 +1,306 @@
+#!/usr/bin/env python3
+from __future__ import annotations
+
+import argparse
+import json
+import math
+import sys
+from pathlib import Path
+from typing import Dict, List, Tuple
+
+
+import os
+
+for _p in os.environ.get("WILDFIRE_FM_EXTRA_PYTHONPATH", "").split(os.pathsep):
+    if _p and _p not in sys.path:
+        sys.path.insert(0, _p)
+
+import numpy as np
+import pandas as pd
+from catboost import CatBoostRegressor
+from lightgbm import LGBMRegressor
+from sklearn.metrics import mean_absolute_error, mean_squared_error
+from xgboost import XGBRegressor
+
+
+def parse_args() -> argparse.Namespace:
+    parser = argparse.ArgumentParser()
+    parser.add_argument("--aqs-daily", type=Path, required=True)
+    parser.add_argument("--output-dir", type=Path, required=True)
+    parser.add_argument("--exceedance-threshold", type=float, default=35.0)
+    parser.add_argument("--alphaearth-prefix", type=str, default="alphaearth_")
+    parser.add_argument("--seed", type=int, default=7)
+    return parser.parse_args()
+
+
+def assign_split(ts: pd.Timestamp) -> str:
+    year = int(ts.year)
+    if year <= 2023:
+        return "train"
+    if year == 2024:
+        return "val"
+    if year == 2025:
+        return "test"
+    return "other"
+
+
+def rmse(y_true: np.ndarray, y_pred: np.ndarray) -> float:
+    return float(math.sqrt(mean_squared_error(y_true, y_pred)))
+
+
+def pearson_corr(y_true: np.ndarray, y_pred: np.ndarray) -> float:
+    a = np.asarray(y_true, dtype=np.float64)
+    b = np.asarray(y_pred, dtype=np.float64)
+    if a.size < 2 or np.allclose(a, a[0]) or np.allclose(b, b[0]):
+        return 0.0
+    value = float(np.corrcoef(a, b)[0, 1])
+    return value if np.isfinite(value) else 0.0
+
+
+def prf(y_true: np.ndarray, y_pred: np.ndarray, threshold: float) -> Dict[str, float]:
+    truth = np.asarray(y_true >= threshold)
+    pred = np.asarray(y_pred >= threshold)
+    tp = int(np.logical_and(pred, truth).sum())
+    fp = int(np.logical_and(pred, ~truth).sum())
+    fn = int(np.logical_and(~pred, truth).sum())
+    precision = float(tp / (tp + fp)) if (tp + fp) else 0.0
+    recall = float(tp / (tp + fn)) if (tp + fn) else 0.0
+    f1 = float((2.0 * precision * recall) / (precision + recall)) if (precision + recall) else 0.0
+    return {"precision": precision, "recall": recall, "f1": f1}
+
+
+def evaluate_frame(frame: pd.DataFrame, pred_col: str, threshold: float) -> Dict[str, float]:
+    y_true = frame["pm25_mean"].to_numpy(dtype=np.float64)
+    y_pred = frame[pred_col].to_numpy(dtype=np.float64)
+    event = prf(y_true, y_pred, threshold)
+    bias = np.asarray(y_pred - y_true, dtype=np.float64)
+    denom = float(np.sum(y_true))
+    return {
+        "count": int(len(frame)),
+        "rmse": rmse(y_true, y_pred),
+        "mae": float(mean_absolute_error(y_true, y_pred)),
+        "mean_bias": float(np.mean(bias)),
+        "normalized_mean_bias": float(np.sum(bias) / denom) if abs(denom) > 1e-12 else 0.0,
+        "pearson_r": pearson_corr(y_true, y_pred),
+        "event_precision": event["precision"],
+        "event_recall": event["recall"],
+        "event_f1": event["f1"],
+    }
+
+
+def tune_event_shift(val_frame: pd.DataFrame, pred_col: str, threshold: float) -> Dict[str, float]:
+    best = None
+    for delta in np.linspace(-5.0, 15.0, 161):
+        shifted = val_frame.copy()
+        shifted["_shifted_pred"] = shifted[pred_col] + float(delta)
+        metrics = evaluate_frame(shifted, "_shifted_pred", threshold)
+        score = (metrics["event_f1"], metrics["event_recall"], -abs(float(delta)))
+        if best is None or score > best["score"]:
+            best = {"delta": float(delta), "metrics": metrics, "score": score}
+    assert best is not None
+    return {"delta": best["delta"], "val_event_calibrated_metrics": best["metrics"]}
+
+
+def build_features(df: pd.DataFrame) -> pd.DataFrame:
+    df = df.sort_values(["site_key", "date"]).reset_index(drop=True).copy()
+    df["doy"] = df["date"].dt.dayofyear.astype(np.int32)
+    df["month"] = df["date"].dt.month.astype(np.int32)
+    df["doy_sin"] = np.sin(2.0 * np.pi * df["doy"] / 366.0)
+    df["doy_cos"] = np.cos(2.0 * np.pi * df["doy"] / 366.0)
+    grp = df.groupby("site_key", sort=False)
+    for lag in [1, 2, 3, 7]:
+        df[f"lag{lag}_pm25"] = grp["pm25_mean"].shift(lag)
+    df["roll3_prev"] = grp["pm25_mean"].rolling(3, min_periods=1).mean().reset_index(level=0, drop=True).shift(1)
+    df["roll7_prev"] = grp["pm25_mean"].rolling(7, min_periods=1).mean().reset_index(level=0, drop=True).shift(1)
+    return df
+
+
+def prepare_frames(df: pd.DataFrame, alphaearth_prefix: str) -> Tuple[pd.DataFrame, pd.DataFrame, pd.DataFrame, List[str]]:
+    df = build_features(df)
+    train = df[df["split"] == "train"].copy()
+    val = df[df["split"] == "val"].copy()
+    test = df[df["split"] == "test"].copy()
+
+    site_mean = train.groupby("site_key")["pm25_mean"].mean()
+    global_mean = float(train["pm25_mean"].mean())
+    for frame in [train, val, test]:
+        frame["site_climo"] = frame["site_key"].map(site_mean).fillna(global_mean)
+    for frame in [train, val, test]:
+        for col in ["lag1_pm25", "lag2_pm25", "lag3_pm25", "lag7_pm25", "roll3_prev", "roll7_prev"]:
+            frame[col] = pd.to_numeric(frame[col], errors="coerce").fillna(frame["site_climo"])
+
+    feature_cols = [
+        "latitude",
+        "longitude",
+        "obs_count",
+        "site_climo",
+        "lag1_pm25",
+        "lag2_pm25",
+        "lag3_pm25",
+        "lag7_pm25",
+        "roll3_prev",
+        "roll7_prev",
+        "doy_sin",
+        "doy_cos",
+        "month",
+    ]
+    alpha_cols = [
+        c for c in df.columns
+        if c.startswith(alphaearth_prefix) and pd.api.types.is_numeric_dtype(df[c])
+    ]
+    feature_cols.extend(sorted(alpha_cols))
+    medians = train[feature_cols].median(numeric_only=True).fillna(0.0)
+    for frame in [train, val, test]:
+        frame.loc[:, feature_cols] = frame[feature_cols].fillna(medians)
+        frame.loc[:, feature_cols] = frame[feature_cols].fillna(0.0)
+    return train, val, test, feature_cols
+
+
+def main() -> None:
+    args = parse_args()
+    df = pd.read_csv(args.aqs_daily, compression="infer", low_memory=False)
+    df["date"] = pd.to_datetime(df["date_gmt"], errors="coerce")
+    df["pm25_mean"] = pd.to_numeric(df["pm25_mean"], errors="coerce")
+    df["pm25_max"] = pd.to_numeric(df["pm25_max"], errors="coerce")
+    df["obs_count"] = pd.to_numeric(df["obs_count"], errors="coerce")
+    df = df.dropna(subset=["date", "site_key", "pm25_mean"]).copy()
+    df = (
+        df.groupby(["date", "site_key"], as_index=False)
+        .agg(
+            latitude=("Latitude", "first"),
+            longitude=("Longitude", "first"),
+            pm25_mean=("pm25_mean", "mean"),
+            pm25_max=("pm25_max", "max"),
+            obs_count=("obs_count", "sum"),
+            **{c: (c, "first") for c in df.columns if c.startswith(args.alphaearth_prefix)},
+        )
+        .sort_values(["site_key", "date"])
+        .reset_index(drop=True)
+    )
+    df["split"] = df["date"].map(assign_split)
+    df = df[df["split"].isin(["train", "val", "test"])].copy()
+
+    train, val, test, feature_cols = prepare_frames(df, alphaearth_prefix=args.alphaearth_prefix)
+    y_train = train["pm25_mean"].to_numpy(dtype=np.float64)
+
+    candidates = {
+        "xgboost": XGBRegressor(
+            n_estimators=300,
+            max_depth=8,
+            learning_rate=0.05,
+            subsample=0.8,
+            colsample_bytree=0.8,
+            objective="reg:squarederror",
+            tree_method="hist",
+            random_state=args.seed,
+            n_jobs=8,
+        ),
+        "lightgbm": LGBMRegressor(
+            n_estimators=300,
+            learning_rate=0.05,
+            num_leaves=127,
+            subsample=0.8,
+            colsample_bytree=0.8,
+            random_state=args.seed,
+            n_jobs=8,
+            verbose=-1,
+        ),
+        "catboost": CatBoostRegressor(
+            iterations=400,
+            depth=8,
+            learning_rate=0.05,
+            loss_function="RMSE",
+            eval_metric="RMSE",
+            random_seed=args.seed,
+            verbose=False,
+        ),
+    }
+
+    candidate_validation: List[Dict[str, object]] = []
+    best_name = None
+    best_model = None
+    best_score = None
+    for name, model in candidates.items():
+        if name == "catboost":
+            model.fit(train[feature_cols], y_train, use_best_model=False)
+            val_pred = model.predict(val[feature_cols])
+        else:
+            model.fit(train[feature_cols].to_numpy(dtype=np.float32), y_train)
+            val_pred = model.predict(val[feature_cols].to_numpy(dtype=np.float32))
+        val_frame = val.copy()
+        val_frame["pred"] = val_pred
+        metrics = evaluate_frame(val_frame, "pred", args.exceedance_threshold)
+        candidate_validation.append({"candidate": name, "val_metrics": metrics})
+        score = float(metrics["rmse"])
+        if best_score is None or score < best_score:
+            best_score = score
+            best_name = name
+            best_model = model
+
+    assert best_name is not None and best_model is not None
+    combined = pd.concat([train, val], ignore_index=True)
+    if best_name == "catboost":
+        best_model.fit(combined[feature_cols], combined["pm25_mean"].to_numpy(dtype=np.float64), use_best_model=False)
+        train_pred = best_model.predict(train[feature_cols])
+        val_pred = best_model.predict(val[feature_cols])
+        test_pred = best_model.predict(test[feature_cols])
+    else:
+        best_model.fit(combined[feature_cols].to_numpy(dtype=np.float32), combined["pm25_mean"].to_numpy(dtype=np.float64))
+        train_pred = best_model.predict(train[feature_cols].to_numpy(dtype=np.float32))
+        val_pred = best_model.predict(val[feature_cols].to_numpy(dtype=np.float32))
+        test_pred = best_model.predict(test[feature_cols].to_numpy(dtype=np.float32))
+
+    args.output_dir.mkdir(parents=True, exist_ok=True)
+    pred_df = pd.concat(
+        [
+            train.assign(pred_pm25=train_pred),
+            val.assign(pred_pm25=val_pred),
+            test.assign(pred_pm25=test_pred),
+        ],
+        ignore_index=True,
+    )
+    pred_path = args.output_dir / "predictions.csv.gz"
+    pred_df.to_csv(pred_path, index=False, compression="gzip")
+
+    train_eval = train.assign(pred=train_pred)
+    val_eval = val.assign(pred=val_pred)
+    test_eval = test.assign(pred=test_pred)
+    event_shift = tune_event_shift(val_eval, "pred", args.exceedance_threshold)
+    delta = event_shift["delta"]
+    train_event_eval = train_eval.assign(pred_event_calibrated=train_eval["pred"] + delta)
+    val_event_eval = val_eval.assign(pred_event_calibrated=val_eval["pred"] + delta)
+    test_event_eval = test_eval.assign(pred_event_calibrated=test_eval["pred"] + delta)
+
+    summary = {
+        "task_id": "smoke_pm25_alphaearth",
+        "task_form": "station_daily_regression",
+        "aqs_daily": str(args.aqs_daily),
+        "output_dir": str(args.output_dir),
+        "seed": int(args.seed),
+        "feature_columns": feature_cols,
+        "alphaearth_feature_count": int(sum(c.startswith(args.alphaearth_prefix) for c in feature_cols)),
+        "split_sizes": {"train": int(len(train)), "val": int(len(val)), "test": int(len(test))},
+        "candidate_validation": candidate_validation,
+        "selected_model": best_name,
+        "train_metrics": evaluate_frame(train_eval, "pred", args.exceedance_threshold),
+        "val_metrics": evaluate_frame(val_eval, "pred", args.exceedance_threshold),
+        "test_metrics": evaluate_frame(test_eval, "pred", args.exceedance_threshold),
+        "event_calibration": {
+            "delta": float(delta),
+            "val_metrics": event_shift["val_event_calibrated_metrics"],
+            "test_metrics": evaluate_frame(test_event_eval, "pred_event_calibrated", args.exceedance_threshold),
+        },
+        "predictions_path": str(pred_path),
+        "selection_rule": "same smoke benchmark; choose task-specific regressor by validation RMSE, then calibrate exceedance on validation only",
+        "tmt_policy": {
+            "task": "smoke_pm25",
+            "metric": "continuous RMSE/MAE with thresholded exceedance PRF",
+            "tolerance": "secondary event policy only",
+        },
+    }
+    (args.output_dir / "summary.json").write_text(json.dumps(summary, indent=2), encoding="utf-8")
+    print(json.dumps(summary, indent=2))
+
+
+if __name__ == "__main__":
+    main()

experiments/raw_reference/task_scripts/run_smoke_pm25_attached_fm_suite_seeded.py

@@ -0,0 +1,231 @@
+#!/usr/bin/env python3
+from __future__ import annotations
+
+import argparse
+import json
+import math
+import sys
+from pathlib import Path
+from typing import Dict, List
+
+
+import os
+
+for _p in os.environ.get("WILDFIRE_FM_EXTRA_PYTHONPATH", "").split(os.pathsep):
+    if _p and _p not in sys.path:
+        sys.path.insert(0, _p)
+
+import numpy as np
+import pandas as pd
+from catboost import CatBoostRegressor
+from lightgbm import LGBMRegressor
+from sklearn.linear_model import ElasticNet, Ridge
+from sklearn.metrics import mean_absolute_error, mean_squared_error
+from xgboost import XGBRegressor
+
+
+def parse_args() -> argparse.Namespace:
+    parser = argparse.ArgumentParser()
+    parser.add_argument("--attached-csv", type=Path, required=True)
+    parser.add_argument("--output-dir", type=Path, required=True)
+    parser.add_argument("--fm-prefix", type=str, required=True)
+    parser.add_argument("--fm-family", type=str, required=True)
+    parser.add_argument("--model-family", choices=("full", "lite"), default="lite")
+    parser.add_argument("--exceedance-threshold", type=float, default=35.0)
+    parser.add_argument("--seed", type=int, default=7)
+    return parser.parse_args()
+
+
+def rmse(y_true: np.ndarray, y_pred: np.ndarray) -> float:
+    return float(math.sqrt(mean_squared_error(y_true, y_pred)))
+
+
+def pearson_corr(y_true: np.ndarray, y_pred: np.ndarray) -> float:
+    a = np.asarray(y_true, dtype=np.float64)
+    b = np.asarray(y_pred, dtype=np.float64)
+    if a.size < 2 or np.allclose(a, a[0]) or np.allclose(b, b[0]):
+        return 0.0
+    value = float(np.corrcoef(a, b)[0, 1])
+    return value if np.isfinite(value) else 0.0
+
+
+def prf(y_true: np.ndarray, y_pred: np.ndarray, threshold: float) -> Dict[str, float]:
+    truth = np.asarray(y_true >= threshold)
+    pred = np.asarray(y_pred >= threshold)
+    tp = int(np.logical_and(pred, truth).sum())
+    fp = int(np.logical_and(pred, ~truth).sum())
+    fn = int(np.logical_and(~pred, truth).sum())
+    precision = float(tp / (tp + fp)) if (tp + fp) else 0.0
+    recall = float(tp / (tp + fn)) if (tp + fn) else 0.0
+    f1 = float((2.0 * precision * recall) / (precision + recall)) if (precision + recall) else 0.0
+    return {"precision": precision, "recall": recall, "f1": f1}
+
+
+def evaluate_frame(frame: pd.DataFrame, pred_col: str, threshold: float) -> Dict[str, float]:
+    y_true = frame["pm25_mean"].to_numpy(dtype=np.float64)
+    y_pred = frame[pred_col].to_numpy(dtype=np.float64)
+    event = prf(y_true, y_pred, threshold)
+    bias = np.asarray(y_pred - y_true, dtype=np.float64)
+    denom = float(np.sum(y_true))
+    return {
+        "count": int(len(frame)),
+        "rmse": rmse(y_true, y_pred),
+        "mae": float(mean_absolute_error(y_true, y_pred)),
+        "mean_bias": float(np.mean(bias)),
+        "normalized_mean_bias": float(np.sum(bias) / denom) if abs(denom) > 1e-12 else 0.0,
+        "pearson_r": pearson_corr(y_true, y_pred),
+        "event_precision": event["precision"],
+        "event_recall": event["recall"],
+        "event_f1": event["f1"],
+    }
+
+
+def main() -> None:
+    args = parse_args()
+    df = pd.read_csv(args.attached_csv)
+    df["date"] = pd.to_datetime(df["date_gmt"], errors="coerce")
+    df["pm25_mean"] = pd.to_numeric(df["pm25_mean"], errors="coerce")
+    df = df.dropna(subset=["date", "pm25_mean"]).copy()
+
+    feature_cols = [c for c in df.columns if c.startswith(args.fm_prefix)]
+    feature_cols = [c for c in feature_cols if pd.api.types.is_numeric_dtype(df[c])]
+    if not feature_cols:
+        raise SystemExit(f"No numeric FM feature columns found with prefix {args.fm_prefix}")
+
+    split_map = {"2020": "train", "2021": "train", "2022": "train", "2024": "val", "2025": "test"}
+    df["split"] = df["date"].dt.year.astype(str).map(split_map)
+    df = df[df["split"].isin(["train", "val", "test"])].copy()
+    train = df[df["split"] == "train"].copy()
+    val = df[df["split"] == "val"].copy()
+    test = df[df["split"] == "test"].copy()
+    if len(train) == 0 or len(val) == 0 or len(test) == 0:
+        raise SystemExit("Attached FM smoke table is missing one of train/val/test.")
+
+    medians = train[feature_cols].median(numeric_only=True).fillna(0.0)
+    train.loc[:, feature_cols] = train[feature_cols].fillna(medians)
+    val.loc[:, feature_cols] = val[feature_cols].fillna(medians)
+    test.loc[:, feature_cols] = test[feature_cols].fillna(medians)
+    train.loc[:, feature_cols] = train[feature_cols].fillna(0.0)
+    val.loc[:, feature_cols] = val[feature_cols].fillna(0.0)
+    test.loc[:, feature_cols] = test[feature_cols].fillna(0.0)
+
+    y_train = train["pm25_mean"].to_numpy(dtype=np.float64)
+    candidates: Dict[str, object] = {
+        "ridge": Ridge(alpha=1.0, random_state=args.seed),
+        "enet": ElasticNet(alpha=0.01, l1_ratio=0.2, random_state=args.seed, max_iter=10000),
+    }
+    if args.model_family == "full":
+        candidates.update(
+            {
+                "xgboost": XGBRegressor(
+                    n_estimators=300,
+                    max_depth=8,
+                    learning_rate=0.05,
+                    subsample=0.8,
+                    colsample_bytree=0.8,
+                    objective="reg:squarederror",
+                    tree_method="hist",
+                    random_state=args.seed,
+                    n_jobs=8,
+                ),
+                "lightgbm": LGBMRegressor(
+                    n_estimators=300,
+                    learning_rate=0.05,
+                    num_leaves=127,
+                    subsample=0.8,
+                    colsample_bytree=0.8,
+                    random_state=args.seed,
+                    n_jobs=8,
+                    verbose=-1,
+                ),
+                "catboost": CatBoostRegressor(
+                    iterations=400,
+                    depth=8,
+                    learning_rate=0.05,
+                    loss_function="RMSE",
+                    eval_metric="RMSE",
+                    random_seed=args.seed,
+                    verbose=False,
+                ),
+            }
+        )
+
+    candidate_validation: List[Dict[str, object]] = []
+    best_name = None
+    best_model = None
+    best_score = None
+    for name, model in candidates.items():
+        if name == "catboost":
+            model.fit(train[feature_cols], y_train, use_best_model=False)
+            val_pred = model.predict(val[feature_cols])
+        else:
+            model.fit(train[feature_cols].to_numpy(dtype=np.float32), y_train)
+            val_pred = model.predict(val[feature_cols].to_numpy(dtype=np.float32))
+        val_frame = val.copy()
+        val_frame["pred"] = val_pred
+        metrics = evaluate_frame(val_frame, "pred", args.exceedance_threshold)
+        candidate_validation.append({"candidate": name, "val_metrics": metrics})
+        score = float(metrics["rmse"])
+        if best_score is None or score < best_score:
+            best_score = score
+            best_name = name
+            best_model = model
+
+    assert best_name is not None and best_model is not None
+    combined = pd.concat([train, val], ignore_index=True)
+    if best_name == "catboost":
+        best_model.fit(combined[feature_cols], combined["pm25_mean"].to_numpy(dtype=np.float64), use_best_model=False)
+        train_pred = best_model.predict(train[feature_cols])
+        val_pred = best_model.predict(val[feature_cols])
+        test_pred = best_model.predict(test[feature_cols])
+    else:
+        best_model.fit(combined[feature_cols].to_numpy(dtype=np.float32), combined["pm25_mean"].to_numpy(dtype=np.float64))
+        train_pred = best_model.predict(train[feature_cols].to_numpy(dtype=np.float32))
+        val_pred = best_model.predict(val[feature_cols].to_numpy(dtype=np.float32))
+        test_pred = best_model.predict(test[feature_cols].to_numpy(dtype=np.float32))
+
+    args.output_dir.mkdir(parents=True, exist_ok=True)
+    pred_df = pd.concat(
+        [
+            train.assign(pred_pm25=train_pred),
+            val.assign(pred_pm25=val_pred),
+            test.assign(pred_pm25=test_pred),
+        ],
+        ignore_index=True,
+    )
+    pred_path = args.output_dir / "predictions.csv.gz"
+    pred_df.to_csv(pred_path, index=False, compression="gzip")
+
+    train_eval = train.assign(pred=train_pred)
+    val_eval = val.assign(pred=val_pred)
+    test_eval = test.assign(pred=test_pred)
+    summary = {
+        "task_id": "smoke_pm25_named_fm",
+        "task_form": "station_daily_regression",
+        "attached_csv": str(args.attached_csv),
+        "output_dir": str(args.output_dir),
+        "seed": int(args.seed),
+        "feature_columns": feature_cols,
+        "split_sizes": {"train": int(len(train)), "val": int(len(val)), "test": int(len(test))},
+        "candidate_validation": candidate_validation,
+        "selected_model": best_name,
+        "train_metrics": evaluate_frame(train_eval, "pred", args.exceedance_threshold),
+        "val_metrics": evaluate_frame(val_eval, "pred", args.exceedance_threshold),
+        "test_metrics": evaluate_frame(test_eval, "pred", args.exceedance_threshold),
+        "predictions_path": str(pred_path),
+        "model_family": "lightweight_linear_task_heads" if args.model_family == "lite" else "popular_open_source_task_models",
+        "fm_family": args.fm_family,
+        "benchmark_protocol": "fm_lite_protocol" if args.model_family == "lite" else "standard_protocol",
+        "selection_rule": "choose model by validation RMSE on named-FM attached rows; report on held-out test dates",
+        "tmt_policy": {
+            "task": "smoke_pm25",
+            "metric": "continuous RMSE/MAE with thresholded exceedance PRF",
+            "tolerance": "secondary event policy only",
+        },
+    }
+    (args.output_dir / "summary.json").write_text(json.dumps(summary, indent=2), encoding="utf-8")
+    print(json.dumps(summary, indent=2))
+
+
+if __name__ == "__main__":
+    main()

experiments/raw_reference/task_scripts/summarize_forced_meanstd_20260429.py

@@ -0,0 +1,232 @@
+#!/usr/bin/env python3
+from __future__ import annotations
+
+import argparse
+import json
+import math
+import re
+import statistics
+from pathlib import Path
+from typing import Any
+
+
+SLUG_LABELS = {
+    "reference": "Reference",
+    "prithvi_wxc": "Prithvi-WxC",
+    "stormcast": "StormCast",
+    "aurora": "Aurora",
+    "climax": "ClimaX",
+    "alphaearth": "AlphaEarth",
+}
+
+
+def load(path: Path) -> dict[str, Any]:
+    return json.loads(path.read_text(encoding="utf-8"))
+
+
+def stats(values: list[float]) -> dict[str, float | int]:
+    values = [float(v) for v in values if math.isfinite(float(v))]
+    if not values:
+        return {"n": 0, "mean": math.nan, "std": math.nan}
+    return {
+        "n": len(values),
+        "mean": float(statistics.fmean(values)),
+        "std": float(statistics.stdev(values)) if len(values) > 1 else 0.0,
+    }
+
+
+def seed_from_path(path: Path) -> int | None:
+    match = re.search(r"_seed_(\d+)", str(path))
+    return int(match.group(1)) if match else None
+
+
+def label_from_seed_dir(path: Path, prefix: str) -> str:
+    for part in path.parts:
+        if part.startswith(prefix) and "_seed_" in part:
+            slug = part[len(prefix) :].split("_seed_", 1)[0]
+            return SLUG_LABELS.get(slug, slug)
+    return "unknown"
+
+
+def dedupe_rows(rows: list[dict[str, Any]], keys: tuple[str, ...]) -> list[dict[str, Any]]:
+    selected: dict[tuple[Any, ...], dict[str, Any]] = {}
+    for row in rows:
+        key = tuple(row.get(name) for name in keys)
+        old = selected.get(key)
+        if old is None:
+            selected[key] = row
+            continue
+        old_mtime = Path(str(old["path"])).stat().st_mtime
+        new_mtime = Path(str(row["path"])).stat().st_mtime
+        if new_mtime >= old_mtime:
+            selected[key] = row
+    return list(selected.values())
+
+
+def best_val_threshold(data: dict[str, Any]) -> str:
+    entries = data["splits"]["val"]["threshold_metrics"]
+    return max(entries, key=lambda key: (float(entries[key]["f1"]), -float(entries[key]["threshold"])))
+
+
+def collect_occupancy(run_root: Path) -> dict[str, Any]:
+    rows: list[dict[str, Any]] = []
+    for path in sorted(run_root.glob("table3_occupancy_*_seed_*/run_*/summary.json")):
+        data = load(path)
+        threshold_key = best_val_threshold(data)
+        test = data["splits"]["test"]
+        rows.append(
+            {
+                "label": data.get("fm_family") or label_from_seed_dir(path, "table3_occupancy_"),
+                "seed": seed_from_path(path),
+                "strict_f1": float(test["threshold_metrics"][threshold_key]["f1"]),
+                "tolerant_f1": float(test["tolerant_threshold_metrics"]["t0_s3"][threshold_key]["f1"]),
+                "union_f1": float(test["tolerant_threshold_metrics"]["t3_s3"][threshold_key]["f1"]),
+                "path": str(path),
+            }
+        )
+    return group(rows, ["strict_f1", "tolerant_f1", "union_f1"])
+
+
+def collect_headcontrol(run_root: Path) -> dict[str, Any]:
+    rows: list[dict[str, Any]] = []
+    for path in sorted(run_root.glob("table2_prithvi_wxc_headcontrol_seed_*/run_*/summary.json")):
+        data = load(path)
+        seed = seed_from_path(path)
+        for row in data.get("selection_summary", {}).get("rows", []):
+            rows.append(
+                {
+                    "label": "Prithvi-WxC",
+                    "scope": row["scope"],
+                    "seed": seed,
+                    "ranking_selected_union_f1": float(row["ranking_selected_union_f1"]),
+                    "decision_selected_union_f1": float(row["decision_selected_union_f1"]),
+                    "decision_regret_union_f1": float(row["decision_regret_union_f1"]),
+                    "selection_failure": bool(row.get("selection_failure", False)),
+                    "path": str(path),
+                }
+            )
+    grouped: dict[str, Any] = {}
+    rows = dedupe_rows(rows, ("label", "scope", "seed"))
+    for scope in sorted({str(row["scope"]) for row in rows}):
+        selected = [row for row in rows if row["scope"] == scope]
+        grouped[scope] = {
+            "n": len(selected),
+            "failure_count": int(sum(1 for row in selected if row["selection_failure"])),
+            "ranking_selected_union_f1": stats([row["ranking_selected_union_f1"] for row in selected]),
+            "decision_selected_union_f1": stats([row["decision_selected_union_f1"] for row in selected]),
+            "decision_regret_union_f1": stats([row["decision_regret_union_f1"] for row in selected]),
+        }
+    return {"rows": rows, "summary": grouped}
+
+
+def collect_spread(run_root: Path) -> dict[str, Any]:
+    rows: list[dict[str, Any]] = []
+    for pattern, prefix in [
+        ("table3_spread_*_seed_*/run_*/summary.json", "table3_spread_"),
+        ("table3_reference_spread_seed_*/run_*/summary.json", "table3_reference_spread_"),
+    ]:
+        for path in sorted(run_root.glob(pattern)):
+            data = load(path)
+            headline = data["headline_metrics"]
+            label = data.get("fm_family") or ("Reference" if "reference_spread" in str(path) else label_from_seed_dir(path, prefix))
+            rows.append(
+                {
+                    "label": label,
+                    "seed": seed_from_path(path),
+                    "strict_f1": float(headline["strict_f1"]),
+                    "spatial_f1": float(headline["same_sample_spatial_tolerance_f1"]["s4"]),
+                    "ap": float(headline["strict_AP"]),
+                    "path": str(path),
+                }
+            )
+    return group(rows, ["strict_f1", "spatial_f1", "ap"])
+
+
+def collect_task(run_root: Path, glob_pattern: str, prefix: str, metrics_path: list[str], metric_keys: list[str]) -> dict[str, Any]:
+    rows: list[dict[str, Any]] = []
+    for path in sorted(run_root.glob(glob_pattern)):
+        data = load(path)
+        label = data.get("fm_family") or label_from_seed_dir(path, prefix)
+        node: Any = data
+        for key in metrics_path:
+            node = node[key]
+        row = {"label": label, "seed": seed_from_path(path), "path": str(path)}
+        for key in metric_keys:
+            row[key] = float(node[key])
+        rows.append(row)
+    return group(rows, metric_keys)
+
+
+def group(rows: list[dict[str, Any]], metric_keys: list[str]) -> dict[str, Any]:
+    if rows and "seed" in rows[0]:
+        rows = dedupe_rows(rows, ("label", "seed"))
+    summary: dict[str, Any] = {}
+    for label in sorted({str(row["label"]) for row in rows}):
+        selected = [row for row in rows if row["label"] == label]
+        summary[label] = {"n": len(selected)}
+        for key in metric_keys:
+            summary[label][key] = stats([row[key] for row in selected])
+    return {"rows": rows, "summary": summary}
+
+
+def fmt(value: dict[str, Any], scale: float = 1.0, digits: int = 2) -> str:
+    if int(value["n"]) == 0:
+        return "missing"
+    return f"{float(value['mean']) * scale:.{digits}f} +/- {float(value['std']) * scale:.{digits}f} (n={int(value['n'])})"
+
+
+def write_markdown(out: Path, summary: dict[str, Any]) -> None:
+    lines = ["# Forced Mean/Std Gap-Fill Summary", ""]
+    for section in [
+        "table2_headcontrol",
+        "table3_occupancy",
+        "table3_spread",
+        "table4_final_area",
+        "table4_analog",
+        "table4_smoke",
+        "table4_heat",
+    ]:
+        lines += [f"## {section}", ""]
+        sec = summary.get(section, {}).get("summary", {})
+        if section == "table2_headcontrol":
+            for scope, row in sec.items():
+                lines.append(
+                    f"- {scope}: regret {fmt(row['decision_regret_union_f1'], 100.0)}; "
+                    f"ranking union {fmt(row['ranking_selected_union_f1'], 100.0)}; "
+                    f"decision union {fmt(row['decision_selected_union_f1'], 100.0)}; "
+                    f"failures {row['failure_count']}/{row['n']}"
+                )
+        else:
+            for label, row in sec.items():
+                pieces = [f"{key} {fmt(val, 100.0 if key.endswith('_f1') or key == 'ap' else 1.0)}" for key, val in row.items() if isinstance(val, dict)]
+                lines.append(f"- {label}: " + "; ".join(pieces))
+        lines.append("")
+    out.write_text("\n".join(lines), encoding="utf-8")
+
+
+def main() -> None:
+    parser = argparse.ArgumentParser()
+    parser.add_argument("--run-root", type=Path, default=Path("${RUN_ROOT}"))
+    parser.add_argument("--out-json", type=Path, default=Path("${OUT_JSON}"))
+    parser.add_argument("--out-md", type=Path, default=Path("${OUT_MD}"))
+    args = parser.parse_args()
+
+    summary = {
+        "run_root": str(args.run_root),
+        "table2_headcontrol": collect_headcontrol(args.run_root),
+        "table3_occupancy": collect_occupancy(args.run_root),
+        "table3_spread": collect_spread(args.run_root),
+        "table4_final_area": collect_task(args.run_root, "table4_final_area_*_seed_*/run_*/summary.json", "table4_final_area_", ["headline_metrics"], ["log_rmse", "log_mae", "log_spearman"]),
+        "table4_analog": collect_task(args.run_root, "table4_analog_*_seed_*/run_*/summary.json", "table4_analog_", ["test_metrics"], ["ndcg_at_10", "log_rmse", "log_mae"]),
+        "table4_smoke": collect_task(args.run_root, "table4_smoke_*_seed_*/run_*/summary.json", "table4_smoke_", ["test_metrics"], ["rmse", "mae", "pearson_r"]),
+        "table4_heat": collect_task(args.run_root, "table4_heat_*_seed_*/run_*/summary.json", "table4_heat_", ["test_metrics"], ["rmse_c", "mae_c", "pearson_r"]),
+    }
+    args.out_json.parent.mkdir(parents=True, exist_ok=True)
+    args.out_json.write_text(json.dumps(summary, indent=2), encoding="utf-8")
+    write_markdown(args.out_md, summary)
+    print(f"wrote={args.out_json}")
+    print(f"wrote={args.out_md}")
+
+
+if __name__ == "__main__":
+    main()

experiments/slurm/submit_template.sbatch

@@ -0,0 +1,13 @@
+#!/bin/bash
+#SBATCH --job-name=wildfire-contract-rerun
+#SBATCH --cpus-per-task=4
+#SBATCH --mem=24G
+#SBATCH --time=02:00:00
+
+# Template only. Set these paths for your environment after obtaining data.
+PROJECT_ROOT=/path/to/this/repository
+DATA_ROOT=/path/to/raw/or/processed/data
+OUTPUT_ROOT=/path/to/output
+
+cd "$PROJECT_ROOT"
+python3 scripts/reproduce_paper_outputs.py

paper_outputs/figures/fig_fireprone_contract_progression_compact.pdf

@@ -0,0 +1,262 @@
+%PDF-1.4
+%����
+1 0 obj
+<< /Type /Catalog /Pages 2 0 R >>
+endobj
+2 0 obj
+<< /Type /Pages /Kids [3 0 R] /Count 1 >>
+endobj
+3 0 obj
+<< /Type /Page /Parent 2 0 R /MediaBox [0 0 1320 470] /Resources << /Font << /F1 4 0 R /F2 5 0 R >> >> /Contents 6 0 R >>
+endobj
+4 0 obj
+<< /Type /Font /Subtype /Type1 /BaseFont /Helvetica >>
+endobj
+5 0 obj
+<< /Type /Font /Subtype /Type1 /BaseFont /Helvetica-Bold >>
+endobj
+6 0 obj
+<< /Length 19373 >>
+stream
+1.0000 1.0000 1.0000 rg 0.00 0.00 1320.00 470.00 re f
+0.80 w 0.1500 0.1500 0.1500 RG 72.00 132.00 m 72.00 400.00 l S
+0.80 w 0.1500 0.1500 0.1500 RG 72.00 132.00 m 1266.00 132.00 l S
+0.45 w 0.8600 0.8600 0.8600 RG 68.00 132.00 m 1266.00 132.00 l S
+BT /F1 7.00 Tf 0.2500 0.2500 0.2500 rg 1 0 0 1 60.36 129.00 Tm (0) Tj ET
+0.45 w 0.8600 0.8600 0.8600 RG 68.00 199.00 m 1266.00 199.00 l S
+BT /F1 7.00 Tf 0.2500 0.2500 0.2500 rg 1 0 0 1 56.72 196.00 Tm (20) Tj ET
+0.45 w 0.8600 0.8600 0.8600 RG 68.00 266.00 m 1266.00 266.00 l S
+BT /F1 7.00 Tf 0.2500 0.2500 0.2500 rg 1 0 0 1 56.72 263.00 Tm (40) Tj ET
+0.45 w 0.8600 0.8600 0.8600 RG 68.00 333.00 m 1266.00 333.00 l S
+BT /F1 7.00 Tf 0.2500 0.2500 0.2500 rg 1 0 0 1 56.72 330.00 Tm (60) Tj ET
+0.45 w 0.8600 0.8600 0.8600 RG 68.00 400.00 m 1266.00 400.00 l S
+BT /F1 7.00 Tf 0.2500 0.2500 0.2500 rg 1 0 0 1 56.72 397.00 Tm (80) Tj ET
+BT /F2 8.00 Tf 0.1500 0.1500 0.1500 rg 1 0 0 1 34.00 408.00 Tm (F1 \(%\)) Tj ET
+BT /F2 15.00 Tf 0.0000 0.0000 0.0000 rg 1 0 0 1 202.93 417.00 Tm (global) Tj ET
+0.75 w 0.4200 0.4400 0.4600 RG 380.50 126.00 m 380.50 133.00 l S
+0.75 w 0.4200 0.4400 0.4600 RG 380.50 138.00 m 380.50 145.00 l S
+0.75 w 0.4200 0.4400 0.4600 RG 380.50 150.00 m 380.50 157.00 l S
+0.75 w 0.4200 0.4400 0.4600 RG 380.50 162.00 m 380.50 169.00 l S
+0.75 w 0.4200 0.4400 0.4600 RG 380.50 174.00 m 380.50 181.00 l S
+0.75 w 0.4200 0.4400 0.4600 RG 380.50 186.00 m 380.50 193.00 l S
+0.75 w 0.4200 0.4400 0.4600 RG 380.50 198.00 m 380.50 205.00 l S
+0.75 w 0.4200 0.4400 0.4600 RG 380.50 210.00 m 380.50 217.00 l S
+0.75 w 0.4200 0.4400 0.4600 RG 380.50 222.00 m 380.50 229.00 l S
+0.75 w 0.4200 0.4400 0.4600 RG 380.50 234.00 m 380.50 241.00 l S
+0.75 w 0.4200 0.4400 0.4600 RG 380.50 246.00 m 380.50 253.00 l S
+0.75 w 0.4200 0.4400 0.4600 RG 380.50 258.00 m 380.50 265.00 l S
+0.75 w 0.4200 0.4400 0.4600 RG 380.50 270.00 m 380.50 277.00 l S
+0.75 w 0.4200 0.4400 0.4600 RG 380.50 282.00 m 380.50 289.00 l S
+0.75 w 0.4200 0.4400 0.4600 RG 380.50 294.00 m 380.50 301.00 l S
+0.75 w 0.4200 0.4400 0.4600 RG 380.50 306.00 m 380.50 313.00 l S
+0.75 w 0.4200 0.4400 0.4600 RG 380.50 318.00 m 380.50 325.00 l S
+0.75 w 0.4200 0.4400 0.4600 RG 380.50 330.00 m 380.50 337.00 l S
+0.75 w 0.4200 0.4400 0.4600 RG 380.50 342.00 m 380.50 349.00 l S
+0.75 w 0.4200 0.4400 0.4600 RG 380.50 354.00 m 380.50 361.00 l S
+0.75 w 0.4200 0.4400 0.4600 RG 380.50 366.00 m 380.50 373.00 l S
+0.75 w 0.4200 0.4400 0.4600 RG 380.50 378.00 m 380.50 385.00 l S
+0.75 w 0.4200 0.4400 0.4600 RG 380.50 390.00 m 380.50 397.00 l S
+0.75 w 0.4200 0.4400 0.4600 RG 380.50 402.00 m 380.50 409.00 l S
+0.75 w 0.4200 0.4400 0.4600 RG 380.50 414.00 m 380.50 416.00 l S
+BT /F2 15.00 Tf 0.0000 0.0000 0.0000 rg 1 0 0 1 515.07 417.00 Tm (top 5%) Tj ET
+BT /F2 15.00 Tf 0.0000 0.0000 0.0000 rg 1 0 0 1 807.48 417.00 Tm (top 10%) Tj ET
+BT /F2 15.00 Tf 0.0000 0.0000 0.0000 rg 1 0 0 1 1103.97 417.00 Tm (top 20%) Tj ET
+0.35 w 0.0900 0.2200 0.3700 rg 1.0000 1.0000 1.0000 RG 84.02 132.00 18.00 1.52 re B
+0.35 w 0.3100 0.5500 0.8000 rg 1.0000 1.0000 1.0000 RG 84.02 133.52 18.00 98.13 re B
+0.35 w 0.7500 0.8400 0.9400 rg 1.0000 1.0000 1.0000 RG 84.02 231.66 18.00 98.21 re B
+BT /F1 10.00 Tf 0.0000 0.0000 0.0000 rg 0.70711 -0.70711 0.70711 0.70711 76.52 69.51 Tm (Ref.) Tj ET
+0.35 w 0.0900 0.2200 0.3700 rg 1.0000 1.0000 1.0000 RG 110.07 132.00 18.00 0.19 re B
+0.35 w 0.3100 0.5500 0.8000 rg 1.0000 1.0000 1.0000 RG 110.07 132.19 18.00 23.82 re B
+0.35 w 0.7500 0.8400 0.9400 rg 1.0000 1.0000 1.0000 RG 110.07 156.00 18.00 43.62 re B
+BT /F1 10.00 Tf 0.0000 0.0000 0.0000 rg 0.70711 -0.70711 0.70711 0.70711 102.21 69.86 Tm (WxC) Tj ET
+0.35 w 0.0900 0.2200 0.3700 rg 1.0000 1.0000 1.0000 RG 136.11 132.00 18.00 0.22 re B
+0.35 w 0.3100 0.5500 0.8000 rg 1.0000 1.0000 1.0000 RG 136.11 132.22 18.00 28.26 re B
+0.35 w 0.7500 0.8400 0.9400 rg 1.0000 1.0000 1.0000 RG 136.11 160.48 18.00 48.92 re B
+BT /F1 10.00 Tf 0.0000 0.0000 0.0000 rg 0.70711 -0.70711 0.70711 0.70711 119.34 78.77 Tm (Aurora) Tj ET
+0.35 w 0.0900 0.2200 0.3700 rg 1.0000 1.0000 1.0000 RG 162.16 132.00 18.00 1.17 re B
+0.35 w 0.3100 0.5500 0.8000 rg 1.0000 1.0000 1.0000 RG 162.16 133.17 18.00 98.51 re B
+0.35 w 0.7500 0.8400 0.9400 rg 1.0000 1.0000 1.0000 RG 162.16 231.67 18.00 101.83 re B
+BT /F1 10.00 Tf 0.0000 0.0000 0.0000 rg 0.70711 -0.70711 0.70711 0.70711 146.38 77.78 Tm (ClimaX) Tj ET
+0.35 w 0.0900 0.2200 0.3700 rg 1.0000 1.0000 1.0000 RG 188.20 132.00 18.00 0.21 re B
+0.35 w 0.3100 0.5500 0.8000 rg 1.0000 1.0000 1.0000 RG 188.20 132.21 18.00 27.24 re B
+0.35 w 0.7500 0.8400 0.9400 rg 1.0000 1.0000 1.0000 RG 188.20 159.45 18.00 47.52 re B
+BT /F1 10.00 Tf 0.0000 0.0000 0.0000 rg 0.70711 -0.70711 0.70711 0.70711 172.99 77.21 Tm (Storm) Tj ET
+0.35 w 0.0900 0.2200 0.3700 rg 1.0000 1.0000 1.0000 RG 214.25 132.00 18.00 0.57 re B
+0.35 w 0.3100 0.5500 0.8000 rg 1.0000 1.0000 1.0000 RG 214.25 132.57 18.00 49.40 re B
+0.35 w 0.7500 0.8400 0.9400 rg 1.0000 1.0000 1.0000 RG 214.25 181.96 18.00 44.47 re B
+BT /F1 10.00 Tf 0.0000 0.0000 0.0000 rg 0.70711 -0.70711 0.70711 0.70711 201.30 74.95 Tm (DLWP) Tj ET
+0.35 w 0.0900 0.2200 0.3700 rg 1.0000 1.0000 1.0000 RG 240.30 132.00 18.00 0.95 re B
+0.35 w 0.3100 0.5500 0.8000 rg 1.0000 1.0000 1.0000 RG 240.30 132.95 18.00 64.40 re B
+0.35 w 0.7500 0.8400 0.9400 rg 1.0000 1.0000 1.0000 RG 240.30 197.35 18.00 68.86 re B
+BT /F1 10.00 Tf 0.0000 0.0000 0.0000 rg 0.70711 -0.70711 0.70711 0.70711 233.14 69.15 Tm (FCN) Tj ET
+0.35 w 0.0900 0.2200 0.3700 rg 1.0000 1.0000 1.0000 RG 266.34 132.00 18.00 0.88 re B
+0.35 w 0.3100 0.5500 0.8000 rg 1.0000 1.0000 1.0000 RG 266.34 132.88 18.00 39.34 re B
+0.35 w 0.7500 0.8400 0.9400 rg 1.0000 1.0000 1.0000 RG 266.34 172.22 18.00 40.53 re B
+BT /F1 10.00 Tf 0.0000 0.0000 0.0000 rg 0.70711 -0.70711 0.70711 0.70711 247.45 80.89 Tm (FengWu) Tj ET
+0.35 w 0.0900 0.2200 0.3700 rg 1.0000 1.0000 1.0000 RG 292.39 132.00 18.00 1.26 re B
+0.35 w 0.3100 0.5500 0.8000 rg 1.0000 1.0000 1.0000 RG 292.39 133.26 18.00 69.19 re B
+0.35 w 0.7500 0.8400 0.9400 rg 1.0000 1.0000 1.0000 RG 292.39 202.46 18.00 54.46 re B
+BT /F1 10.00 Tf 0.0000 0.0000 0.0000 rg 0.70711 -0.70711 0.70711 0.70711 284.17 70.21 Tm (FuXi) Tj ET
+0.35 w 0.0900 0.2200 0.3700 rg 1.0000 1.0000 1.0000 RG 318.43 132.00 18.00 0.92 re B
+0.35 w 0.3100 0.5500 0.8000 rg 1.0000 1.0000 1.0000 RG 318.43 132.92 18.00 56.33 re B
+0.35 w 0.7500 0.8400 0.9400 rg 1.0000 1.0000 1.0000 RG 318.43 189.25 18.00 62.13 re B
+BT /F1 10.00 Tf 0.0000 0.0000 0.0000 rg 0.70711 -0.70711 0.70711 0.70711 295.86 84.57 Tm (Pangu-W) Tj ET
+0.35 w 0.0900 0.2200 0.3700 rg 1.0000 1.0000 1.0000 RG 344.48 132.00 18.00 6.90 re B
+0.35 w 0.3100 0.5500 0.8000 rg 1.0000 1.0000 1.0000 RG 344.48 138.90 18.00 91.75 re B
+0.35 w 0.7500 0.8400 0.9400 rg 1.0000 1.0000 1.0000 RG 344.48 230.65 18.00 26.74 re B
+BT /F1 10.00 Tf 0.0000 0.0000 0.0000 rg 0.70711 -0.70711 0.70711 0.70711 333.29 73.18 Tm (Alpha) Tj ET
+0.35 w 0.0900 0.2200 0.3700 rg 1.0000 1.0000 1.0000 RG 398.52 132.00 18.00 11.93 re B
+0.35 w 0.3100 0.5500 0.8000 rg 1.0000 1.0000 1.0000 RG 398.52 143.93 18.00 119.60 re B
+0.35 w 0.7500 0.8400 0.9400 rg 1.0000 1.0000 1.0000 RG 398.52 263.53 18.00 112.45 re B
+BT /F1 10.00 Tf 0.0000 0.0000 0.0000 rg 0.70711 -0.70711 0.70711 0.70711 391.02 69.51 Tm (Ref.) Tj ET
+0.35 w 0.0900 0.2200 0.3700 rg 1.0000 1.0000 1.0000 RG 424.57 132.00 18.00 4.73 re B
+0.35 w 0.3100 0.5500 0.8000 rg 1.0000 1.0000 1.0000 RG 424.57 136.73 18.00 59.80 re B
+0.35 w 0.7500 0.8400 0.9400 rg 1.0000 1.0000 1.0000 RG 424.57 196.53 18.00 78.11 re B
+BT /F1 10.00 Tf 0.0000 0.0000 0.0000 rg 0.70711 -0.70711 0.70711 0.70711 416.71 69.86 Tm (WxC) Tj ET
+0.35 w 0.0900 0.2200 0.3700 rg 1.0000 1.0000 1.0000 RG 450.61 132.00 18.00 3.30 re B
+0.35 w 0.3100 0.5500 0.8000 rg 1.0000 1.0000 1.0000 RG 450.61 135.30 18.00 47.40 re B
+0.35 w 0.7500 0.8400 0.9400 rg 1.0000 1.0000 1.0000 RG 450.61 182.70 18.00 68.17 re B
+BT /F1 10.00 Tf 0.0000 0.0000 0.0000 rg 0.70711 -0.70711 0.70711 0.70711 433.84 78.77 Tm (Aurora) Tj ET
+0.35 w 0.0900 0.2200 0.3700 rg 1.0000 1.0000 1.0000 RG 476.66 132.00 18.00 4.33 re B
+0.35 w 0.3100 0.5500 0.8000 rg 1.0000 1.0000 1.0000 RG 476.66 136.33 18.00 111.51 re B
+0.35 w 0.7500 0.8400 0.9400 rg 1.0000 1.0000 1.0000 RG 476.66 247.84 18.00 116.04 re B
+BT /F1 10.00 Tf 0.0000 0.0000 0.0000 rg 0.70711 -0.70711 0.70711 0.70711 460.88 77.78 Tm (ClimaX) Tj ET
+0.35 w 0.0900 0.2200 0.3700 rg 1.0000 1.0000 1.0000 RG 502.70 132.00 18.00 3.21 re B
+0.35 w 0.3100 0.5500 0.8000 rg 1.0000 1.0000 1.0000 RG 502.70 135.21 18.00 48.12 re B
+0.35 w 0.7500 0.8400 0.9400 rg 1.0000 1.0000 1.0000 RG 502.70 183.33 18.00 69.89 re B
+BT /F1 10.00 Tf 0.0000 0.0000 0.0000 rg 0.70711 -0.70711 0.70711 0.70711 487.49 77.21 Tm (Storm) Tj ET
+0.35 w 0.0900 0.2200 0.3700 rg 1.0000 1.0000 1.0000 RG 528.75 132.00 18.00 6.05 re B
+0.35 w 0.3100 0.5500 0.8000 rg 1.0000 1.0000 1.0000 RG 528.75 138.05 18.00 100.22 re B
+0.35 w 0.7500 0.8400 0.9400 rg 1.0000 1.0000 1.0000 RG 528.75 238.27 18.00 79.52 re B
+BT /F1 10.00 Tf 0.0000 0.0000 0.0000 rg 0.70711 -0.70711 0.70711 0.70711 515.80 74.95 Tm (DLWP) Tj ET
+0.35 w 0.0900 0.2200 0.3700 rg 1.0000 1.0000 1.0000 RG 554.80 132.00 18.00 5.44 re B
+0.35 w 0.3100 0.5500 0.8000 rg 1.0000 1.0000 1.0000 RG 554.80 137.44 18.00 92.98 re B
+0.35 w 0.7500 0.8400 0.9400 rg 1.0000 1.0000 1.0000 RG 554.80 230.41 18.00 83.50 re B
+BT /F1 10.00 Tf 0.0000 0.0000 0.0000 rg 0.70711 -0.70711 0.70711 0.70711 547.64 69.15 Tm (FCN) Tj ET
+0.35 w 0.0900 0.2200 0.3700 rg 1.0000 1.0000 1.0000 RG 580.84 132.00 18.00 5.26 re B
+0.35 w 0.3100 0.5500 0.8000 rg 1.0000 1.0000 1.0000 RG 580.84 137.26 18.00 49.27 re B
+0.35 w 0.7500 0.8400 0.9400 rg 1.0000 1.0000 1.0000 RG 580.84 186.53 18.00 46.33 re B
+BT /F1 10.00 Tf 0.0000 0.0000 0.0000 rg 0.70711 -0.70711 0.70711 0.70711 561.95 80.89 Tm (FengWu) Tj ET
+0.35 w 0.0900 0.2200 0.3700 rg 1.0000 1.0000 1.0000 RG 606.89 132.00 18.00 6.80 re B
+0.35 w 0.3100 0.5500 0.8000 rg 1.0000 1.0000 1.0000 RG 606.89 138.80 18.00 100.04 re B
+0.35 w 0.7500 0.8400 0.9400 rg 1.0000 1.0000 1.0000 RG 606.89 238.85 18.00 73.82 re B
+BT /F1 10.00 Tf 0.0000 0.0000 0.0000 rg 0.70711 -0.70711 0.70711 0.70711 598.67 70.21 Tm (FuXi) Tj ET
+0.35 w 0.0900 0.2200 0.3700 rg 1.0000 1.0000 1.0000 RG 632.93 132.00 18.00 4.57 re B
+0.35 w 0.3100 0.5500 0.8000 rg 1.0000 1.0000 1.0000 RG 632.93 136.57 18.00 69.87 re B
+0.35 w 0.7500 0.8400 0.9400 rg 1.0000 1.0000 1.0000 RG 632.93 206.44 18.00 71.02 re B
+BT /F1 10.00 Tf 0.0000 0.0000 0.0000 rg 0.70711 -0.70711 0.70711 0.70711 610.36 84.57 Tm (Pangu-W) Tj ET
+0.35 w 0.0900 0.2200 0.3700 rg 1.0000 1.0000 1.0000 RG 658.98 132.00 18.00 23.16 re B
+0.35 w 0.3100 0.5500 0.8000 rg 1.0000 1.0000 1.0000 RG 658.98 155.16 18.00 120.48 re B
+0.35 w 0.7500 0.8400 0.9400 rg 1.0000 1.0000 1.0000 RG 658.98 275.64 18.00 29.70 re B
+BT /F1 10.00 Tf 0.0000 0.0000 0.0000 rg 0.70711 -0.70711 0.70711 0.70711 647.79 73.18 Tm (Alpha) Tj ET
+0.35 w 0.0900 0.2200 0.3700 rg 1.0000 1.0000 1.0000 RG 695.02 132.00 18.00 11.92 re B
+0.35 w 0.3100 0.5500 0.8000 rg 1.0000 1.0000 1.0000 RG 695.02 143.92 18.00 119.29 re B
+0.35 w 0.7500 0.8400 0.9400 rg 1.0000 1.0000 1.0000 RG 695.02 263.21 18.00 111.74 re B
+BT /F1 10.00 Tf 0.0000 0.0000 0.0000 rg 0.70711 -0.70711 0.70711 0.70711 687.52 69.51 Tm (Ref.) Tj ET
+0.35 w 0.0900 0.2200 0.3700 rg 1.0000 1.0000 1.0000 RG 721.07 132.00 18.00 4.15 re B
+0.35 w 0.3100 0.5500 0.8000 rg 1.0000 1.0000 1.0000 RG 721.07 136.15 18.00 45.70 re B
+0.35 w 0.7500 0.8400 0.9400 rg 1.0000 1.0000 1.0000 RG 721.07 181.84 18.00 59.67 re B
+BT /F1 10.00 Tf 0.0000 0.0000 0.0000 rg 0.70711 -0.70711 0.70711 0.70711 713.21 69.86 Tm (WxC) Tj ET
+0.35 w 0.0900 0.2200 0.3700 rg 1.0000 1.0000 1.0000 RG 747.11 132.00 18.00 2.61 re B
+0.35 w 0.3100 0.5500 0.8000 rg 1.0000 1.0000 1.0000 RG 747.11 134.61 18.00 40.06 re B
+0.35 w 0.7500 0.8400 0.9400 rg 1.0000 1.0000 1.0000 RG 747.11 174.67 18.00 59.60 re B
+BT /F1 10.00 Tf 0.0000 0.0000 0.0000 rg 0.70711 -0.70711 0.70711 0.70711 730.34 78.77 Tm (Aurora) Tj ET
+0.35 w 0.0900 0.2200 0.3700 rg 1.0000 1.0000 1.0000 RG 773.16 132.00 18.00 4.19 re B
+0.35 w 0.3100 0.5500 0.8000 rg 1.0000 1.0000 1.0000 RG 773.16 136.19 18.00 110.82 re B
+0.35 w 0.7500 0.8400 0.9400 rg 1.0000 1.0000 1.0000 RG 773.16 247.02 18.00 114.69 re B
+BT /F1 10.00 Tf 0.0000 0.0000 0.0000 rg 0.70711 -0.70711 0.70711 0.70711 757.38 77.78 Tm (ClimaX) Tj ET
+0.35 w 0.0900 0.2200 0.3700 rg 1.0000 1.0000 1.0000 RG 799.20 132.00 18.00 2.44 re B
+0.35 w 0.3100 0.5500 0.8000 rg 1.0000 1.0000 1.0000 RG 799.20 134.44 18.00 39.99 re B
+0.35 w 0.7500 0.8400 0.9400 rg 1.0000 1.0000 1.0000 RG 799.20 174.43 18.00 59.66 re B
+BT /F1 10.00 Tf 0.0000 0.0000 0.0000 rg 0.70711 -0.70711 0.70711 0.70711 783.99 77.21 Tm (Storm) Tj ET
+0.35 w 0.0900 0.2200 0.3700 rg 1.0000 1.0000 1.0000 RG 825.25 132.00 18.00 5.40 re B
+0.35 w 0.3100 0.5500 0.8000 rg 1.0000 1.0000 1.0000 RG 825.25 137.40 18.00 87.26 re B
+0.35 w 0.7500 0.8400 0.9400 rg 1.0000 1.0000 1.0000 RG 825.25 224.65 18.00 65.22 re B
+BT /F1 10.00 Tf 0.0000 0.0000 0.0000 rg 0.70711 -0.70711 0.70711 0.70711 812.30 74.95 Tm (DLWP) Tj ET
+0.35 w 0.0900 0.2200 0.3700 rg 1.0000 1.0000 1.0000 RG 851.30 132.00 18.00 3.95 re B
+0.35 w 0.3100 0.5500 0.8000 rg 1.0000 1.0000 1.0000 RG 851.30 135.95 18.00 71.17 re B
+0.35 w 0.7500 0.8400 0.9400 rg 1.0000 1.0000 1.0000 RG 851.30 207.11 18.00 70.45 re B
+BT /F1 10.00 Tf 0.0000 0.0000 0.0000 rg 0.70711 -0.70711 0.70711 0.70711 844.14 69.15 Tm (FCN) Tj ET
+0.35 w 0.0900 0.2200 0.3700 rg 1.0000 1.0000 1.0000 RG 877.34 132.00 18.00 4.16 re B
+0.35 w 0.3100 0.5500 0.8000 rg 1.0000 1.0000 1.0000 RG 877.34 136.16 18.00 39.22 re B
+0.35 w 0.7500 0.8400 0.9400 rg 1.0000 1.0000 1.0000 RG 877.34 175.38 18.00 37.64 re B
+BT /F1 10.00 Tf 0.0000 0.0000 0.0000 rg 0.70711 -0.70711 0.70711 0.70711 858.45 80.89 Tm (FengWu) Tj ET
+0.35 w 0.0900 0.2200 0.3700 rg 1.0000 1.0000 1.0000 RG 903.39 132.00 18.00 5.54 re B
+0.35 w 0.3100 0.5500 0.8000 rg 1.0000 1.0000 1.0000 RG 903.39 137.54 18.00 74.90 re B
+0.35 w 0.7500 0.8400 0.9400 rg 1.0000 1.0000 1.0000 RG 903.39 212.44 18.00 54.27 re B
+BT /F1 10.00 Tf 0.0000 0.0000 0.0000 rg 0.70711 -0.70711 0.70711 0.70711 895.17 70.21 Tm (FuXi) Tj ET
+0.35 w 0.0900 0.2200 0.3700 rg 1.0000 1.0000 1.0000 RG 929.43 132.00 18.00 3.66 re B
+0.35 w 0.3100 0.5500 0.8000 rg 1.0000 1.0000 1.0000 RG 929.43 135.66 18.00 59.77 re B
+0.35 w 0.7500 0.8400 0.9400 rg 1.0000 1.0000 1.0000 RG 929.43 195.43 18.00 65.66 re B
+BT /F1 10.00 Tf 0.0000 0.0000 0.0000 rg 0.70711 -0.70711 0.70711 0.70711 906.86 84.57 Tm (Pangu-W) Tj ET
+0.35 w 0.0900 0.2200 0.3700 rg 1.0000 1.0000 1.0000 RG 955.48 132.00 18.00 22.23 re B
+0.35 w 0.3100 0.5500 0.8000 rg 1.0000 1.0000 1.0000 RG 955.48 154.23 18.00 118.13 re B
+0.35 w 0.7500 0.8400 0.9400 rg 1.0000 1.0000 1.0000 RG 955.48 272.36 18.00 29.05 re B
+BT /F1 10.00 Tf 0.0000 0.0000 0.0000 rg 0.70711 -0.70711 0.70711 0.70711 944.29 73.18 Tm (Alpha) Tj ET
+0.35 w 0.0900 0.2200 0.3700 rg 1.0000 1.0000 1.0000 RG 991.52 132.00 18.00 11.83 re B
+0.35 w 0.3100 0.5500 0.8000 rg 1.0000 1.0000 1.0000 RG 991.52 143.83 18.00 116.43 re B
+0.35 w 0.7500 0.8400 0.9400 rg 1.0000 1.0000 1.0000 RG 991.52 260.25 18.00 105.32 re B
+BT /F1 10.00 Tf 0.0000 0.0000 0.0000 rg 0.70711 -0.70711 0.70711 0.70711 984.02 69.51 Tm (Ref.) Tj ET
+0.35 w 0.0900 0.2200 0.3700 rg 1.0000 1.0000 1.0000 RG 1017.57 132.00 18.00 3.86 re B
+0.35 w 0.3100 0.5500 0.8000 rg 1.0000 1.0000 1.0000 RG 1017.57 135.86 18.00 40.20 re B
+0.35 w 0.7500 0.8400 0.9400 rg 1.0000 1.0000 1.0000 RG 1017.57 176.06 18.00 50.19 re B
+BT /F1 10.00 Tf 0.0000 0.0000 0.0000 rg 0.70711 -0.70711 0.70711 0.70711 1009.71 69.86 Tm (WxC) Tj ET
+0.35 w 0.0900 0.2200 0.3700 rg 1.0000 1.0000 1.0000 RG 1043.61 132.00 18.00 2.23 re B
+0.35 w 0.3100 0.5500 0.8000 rg 1.0000 1.0000 1.0000 RG 1043.61 134.23 18.00 33.05 re B
+0.35 w 0.7500 0.8400 0.9400 rg 1.0000 1.0000 1.0000 RG 1043.61 167.28 18.00 48.29 re B
+BT /F1 10.00 Tf 0.0000 0.0000 0.0000 rg 0.70711 -0.70711 0.70711 0.70711 1026.84 78.77 Tm (Aurora) Tj ET
+0.35 w 0.0900 0.2200 0.3700 rg 1.0000 1.0000 1.0000 RG 1069.66 132.00 18.00 3.45 re B
+0.35 w 0.3100 0.5500 0.8000 rg 1.0000 1.0000 1.0000 RG 1069.66 135.45 18.00 97.77 re B
+0.35 w 0.7500 0.8400 0.9400 rg 1.0000 1.0000 1.0000 RG 1069.66 233.22 18.00 100.00 re B
+BT /F1 10.00 Tf 0.0000 0.0000 0.0000 rg 0.70711 -0.70711 0.70711 0.70711 1053.88 77.78 Tm (ClimaX) Tj ET
+0.35 w 0.0900 0.2200 0.3700 rg 1.0000 1.0000 1.0000 RG 1095.70 132.00 18.00 1.94 re B
+0.35 w 0.3100 0.5500 0.8000 rg 1.0000 1.0000 1.0000 RG 1095.70 133.94 18.00 32.95 re B
+0.35 w 0.7500 0.8400 0.9400 rg 1.0000 1.0000 1.0000 RG 1095.70 166.89 18.00 47.72 re B
+BT /F1 10.00 Tf 0.0000 0.0000 0.0000 rg 0.70711 -0.70711 0.70711 0.70711 1080.49 77.21 Tm (Storm) Tj ET
+0.35 w 0.0900 0.2200 0.3700 rg 1.0000 1.0000 1.0000 RG 1121.75 132.00 18.00 5.11 re B
+0.35 w 0.3100 0.5500 0.8000 rg 1.0000 1.0000 1.0000 RG 1121.75 137.11 18.00 65.04 re B
+0.35 w 0.7500 0.8400 0.9400 rg 1.0000 1.0000 1.0000 RG 1121.75 202.15 18.00 46.87 re B
+BT /F1 10.00 Tf 0.0000 0.0000 0.0000 rg 0.70711 -0.70711 0.70711 0.70711 1108.80 74.95 Tm (DLWP) Tj ET
+0.35 w 0.0900 0.2200 0.3700 rg 1.0000 1.0000 1.0000 RG 1147.80 132.00 18.00 3.34 re B
+0.35 w 0.3100 0.5500 0.8000 rg 1.0000 1.0000 1.0000 RG 1147.80 135.34 18.00 53.54 re B
+0.35 w 0.7500 0.8400 0.9400 rg 1.0000 1.0000 1.0000 RG 1147.80 188.88 18.00 57.31 re B
+BT /F1 10.00 Tf 0.0000 0.0000 0.0000 rg 0.70711 -0.70711 0.70711 0.70711 1140.64 69.15 Tm (FCN) Tj ET
+0.35 w 0.0900 0.2200 0.3700 rg 1.0000 1.0000 1.0000 RG 1173.84 132.00 18.00 3.75 re B
+0.35 w 0.3100 0.5500 0.8000 rg 1.0000 1.0000 1.0000 RG 1173.84 135.75 18.00 36.29 re B
+0.35 w 0.7500 0.8400 0.9400 rg 1.0000 1.0000 1.0000 RG 1173.84 172.04 18.00 36.30 re B
+BT /F1 10.00 Tf 0.0000 0.0000 0.0000 rg 0.70711 -0.70711 0.70711 0.70711 1154.95 80.89 Tm (FengWu) Tj ET
+0.35 w 0.0900 0.2200 0.3700 rg 1.0000 1.0000 1.0000 RG 1199.89 132.00 18.00 4.57 re B
+0.35 w 0.3100 0.5500 0.8000 rg 1.0000 1.0000 1.0000 RG 1199.89 136.57 18.00 68.98 re B
+0.35 w 0.7500 0.8400 0.9400 rg 1.0000 1.0000 1.0000 RG 1199.89 205.55 18.00 49.50 re B
+BT /F1 10.00 Tf 0.0000 0.0000 0.0000 rg 0.70711 -0.70711 0.70711 0.70711 1191.67 70.21 Tm (FuXi) Tj ET
+0.35 w 0.0900 0.2200 0.3700 rg 1.0000 1.0000 1.0000 RG 1225.93 132.00 18.00 2.96 re B
+0.35 w 0.3100 0.5500 0.8000 rg 1.0000 1.0000 1.0000 RG 1225.93 134.96 18.00 54.04 re B
+0.35 w 0.7500 0.8400 0.9400 rg 1.0000 1.0000 1.0000 RG 1225.93 189.01 18.00 58.80 re B
+BT /F1 10.00 Tf 0.0000 0.0000 0.0000 rg 0.70711 -0.70711 0.70711 0.70711 1203.36 84.57 Tm (Pangu-W) Tj ET
+0.35 w 0.0900 0.2200 0.3700 rg 1.0000 1.0000 1.0000 RG 1251.98 132.00 18.00 20.74 re B
+0.35 w 0.3100 0.5500 0.8000 rg 1.0000 1.0000 1.0000 RG 1251.98 152.74 18.00 109.35 re B
+0.35 w 0.7500 0.8400 0.9400 rg 1.0000 1.0000 1.0000 RG 1251.98 262.09 18.00 25.30 re B
+BT /F1 10.00 Tf 0.0000 0.0000 0.0000 rg 0.70711 -0.70711 0.70711 0.70711 1240.79 73.18 Tm (Alpha) Tj ET
+0.45 w 0.9800 0.9800 0.9600 rg 0.7800 0.8000 0.7800 RG 77.00 362.00 304.00 23.00 re B
+0.35 w 0.0900 0.2200 0.3700 rg 1.0000 1.0000 1.0000 RG 90.00 371.00 24.00 9.00 re B
+BT /F1 8.00 Tf 0.0000 0.0000 0.0000 rg 1 0 0 1 121.00 373.00 Tm (Strict) Tj ET
+0.35 w 0.3100 0.5500 0.8000 rg 1.0000 1.0000 1.0000 RG 188.00 371.00 24.00 9.00 re B
+BT /F1 8.00 Tf 0.0000 0.0000 0.0000 rg 1 0 0 1 219.00 373.00 Tm (Tolerance) Tj ET
+0.35 w 0.7500 0.8400 0.9400 rg 1.0000 1.0000 1.0000 RG 286.00 371.00 24.00 9.00 re B
+BT /F1 8.00 Tf 0.0000 0.0000 0.0000 rg 1 0 0 1 317.00 373.00 Tm (Union) Tj ET
+endstream
+endobj
+xref
+0 7
+0000000000 65535 f 
+0000000015 00000 n 
+0000000064 00000 n 
+0000000121 00000 n 
+0000000258 00000 n 
+0000000328 00000 n 
+0000000403 00000 n 
+trailer
+<< /Size 7 /Root 1 0 R >>
+startxref
+19829
+%%EOF

paper_outputs/figures/fig_selection_regret_rq2.tikz

@@ -0,0 +1,120 @@
+% Auto-generated by scripts/build_selection_regret_rq2_figure.py.
+\begin{tikzpicture}[x=1cm,y=1cm]
+\footnotesize
+\draw[black!12, line width=0.35pt] (2.450,-0.350) -- (2.450,4.530);
+\node[anchor=north, font=\scriptsize, text=black!70] at (2.450,-0.410) {-20};
+\draw[black!12, line width=0.35pt] (3.243,-0.350) -- (3.243,4.530);
+\node[anchor=north, font=\scriptsize, text=black!70] at (3.243,-0.410) {-10};
+\draw[wfgray, line width=0.55pt] (4.036,-0.350) -- (4.036,4.530);
+\node[anchor=north, font=\scriptsize, text=black!70] at (4.036,-0.410) {0};
+\draw[black!12, line width=0.35pt] (4.829,-0.350) -- (4.829,4.530);
+\node[anchor=north, font=\scriptsize, text=black!70] at (4.829,-0.410) {10};
+\draw[black!12, line width=0.35pt] (5.621,-0.350) -- (5.621,4.530);
+\node[anchor=north, font=\scriptsize, text=black!70] at (5.621,-0.410) {20};
+\draw[black!12, line width=0.35pt] (6.414,-0.350) -- (6.414,4.530);
+\node[anchor=north, font=\scriptsize, text=black!70] at (6.414,-0.410) {30};
+\draw[black!12, line width=0.35pt] (7.207,-0.350) -- (7.207,4.530);
+\node[anchor=north, font=\scriptsize, text=black!70] at (7.207,-0.410) {40};
+\draw[black!12, line width=0.35pt] (8.000,-0.350) -- (8.000,4.530);
+\node[anchor=north, font=\scriptsize, text=black!70] at (8.000,-0.410) {50};
+\draw[black!45, line width=0.4pt] (2.450,-0.350) -- (8.000,-0.350);
+\node[anchor=east, font=\scriptsize, text=black!82] at (2.320,4.350) {\textcolor{wfblue}{\textbf{FireWx-FM ref.}}};
+\draw[wfslate, line width=0.72pt] (4.030,4.220) -- (5.212,4.220);
+\draw[wfslate, line width=0.72pt] (4.030,4.185) -- (4.030,4.255);
+\draw[wfslate, line width=0.72pt] (5.212,4.185) -- (5.212,4.255);
+\filldraw[wfslate] (4.621,4.220) circle[radius=0.045];
+\draw[wforange, line width=0.72pt] (4.051,4.480) -- (4.487,4.480);
+\draw[wforange, line width=0.72pt] (4.051,4.445) -- (4.051,4.515);
+\draw[wforange, line width=0.72pt] (4.487,4.445) -- (4.487,4.515);
+\filldraw[wforange] (4.224,4.435) rectangle (4.314,4.525);
+\node[anchor=east, font=\scriptsize, text=black!82] at (2.320,3.940) {Prithvi-WxC};
+\draw[wfslate, line width=0.72pt] (4.036,3.810) -- (4.036,3.810);
+\draw[wfslate, line width=0.72pt] (4.036,3.775) -- (4.036,3.845);
+\draw[wfslate, line width=0.72pt] (4.036,3.775) -- (4.036,3.845);
+\filldraw[wfslate] (4.036,3.810) circle[radius=0.045];
+\draw[wforange, line width=0.72pt] (4.036,4.070) -- (4.036,4.070);
+\draw[wforange, line width=0.72pt] (4.036,4.035) -- (4.036,4.105);
+\draw[wforange, line width=0.72pt] (4.036,4.035) -- (4.036,4.105);
+\filldraw[wforange] (3.991,4.025) rectangle (4.081,4.115);
+\node[anchor=east, font=\scriptsize, text=black!82] at (2.320,3.530) {Aurora};
+\draw[wfslate, line width=0.72pt] (3.580,3.400) -- (5.276,3.400);
+\draw[wfslate, line width=0.72pt] (3.580,3.365) -- (3.580,3.435);
+\draw[wfslate, line width=0.72pt] (5.276,3.365) -- (5.276,3.435);
+\filldraw[wfslate] (4.428,3.400) circle[radius=0.045];
+\draw[wforange, line width=0.72pt] (2.627,3.660) -- (7.723,3.660);
+\draw[wforange, line width=0.72pt] (2.627,3.625) -- (2.627,3.695);
+\draw[wforange, line width=0.72pt] (7.723,3.625) -- (7.723,3.695);
+\filldraw[wforange] (5.130,3.615) rectangle (5.220,3.705);
+\node[anchor=east, font=\scriptsize, text=black!82] at (2.320,3.120) {ClimaX};
+\draw[wfslate, line width=0.72pt] (4.032,2.990) -- (4.060,2.990);
+\draw[wfslate, line width=0.72pt] (4.032,2.955) -- (4.032,3.025);
+\draw[wfslate, line width=0.72pt] (4.060,2.955) -- (4.060,3.025);
+\filldraw[wfslate] (4.046,2.990) circle[radius=0.045];
+\draw[wforange, line width=0.72pt] (4.036,3.250) -- (4.036,3.250);
+\draw[wforange, line width=0.72pt] (4.036,3.215) -- (4.036,3.285);
+\draw[wforange, line width=0.72pt] (4.036,3.215) -- (4.036,3.285);
+\filldraw[wforange] (3.991,3.205) rectangle (4.081,3.295);
+\node[anchor=east, font=\scriptsize, text=black!82] at (2.320,2.710) {StormCast};
+\draw[wfslate, line width=0.72pt] (4.036,2.580) -- (4.036,2.580);
+\draw[wfslate, line width=0.72pt] (4.036,2.545) -- (4.036,2.615);
+\draw[wfslate, line width=0.72pt] (4.036,2.545) -- (4.036,2.615);
+\filldraw[wfslate] (4.036,2.580) circle[radius=0.045];
+\draw[wforange, line width=0.72pt] (4.036,2.840) -- (4.036,2.840);
+\draw[wforange, line width=0.72pt] (4.036,2.805) -- (4.036,2.875);
+\draw[wforange, line width=0.72pt] (4.036,2.805) -- (4.036,2.875);
+\filldraw[wforange] (3.991,2.795) rectangle (4.081,2.885);
+\node[anchor=east, font=\scriptsize, text=black!82] at (2.320,2.300) {DLWP};
+\draw[wfslate, line width=0.72pt] (4.036,2.170) -- (4.036,2.170);
+\draw[wfslate, line width=0.72pt] (4.036,2.135) -- (4.036,2.205);
+\draw[wfslate, line width=0.72pt] (4.036,2.135) -- (4.036,2.205);
+\filldraw[wfslate] (4.036,2.170) circle[radius=0.045];
+\draw[wforange, line width=0.72pt] (4.044,2.430) -- (4.735,2.430);
+\draw[wforange, line width=0.72pt] (4.044,2.395) -- (4.044,2.465);
+\draw[wforange, line width=0.72pt] (4.735,2.395) -- (4.735,2.465);
+\filldraw[wforange] (4.345,2.385) rectangle (4.435,2.475);
+\node[anchor=east, font=\scriptsize, text=black!82] at (2.320,1.890) {FCN};
+\draw[wfslate, line width=0.72pt] (4.036,1.760) -- (4.036,1.760);
+\draw[wfslate, line width=0.72pt] (4.036,1.725) -- (4.036,1.795);
+\draw[wfslate, line width=0.72pt] (4.036,1.725) -- (4.036,1.795);
+\filldraw[wfslate] (4.036,1.760) circle[radius=0.045];
+\draw[wforange, line width=0.72pt] (3.971,2.020) -- (4.286,2.020);
+\draw[wforange, line width=0.72pt] (3.971,1.985) -- (3.971,2.055);
+\draw[wforange, line width=0.72pt] (4.286,1.985) -- (4.286,2.055);
+\filldraw[wforange] (4.083,1.975) rectangle (4.173,2.065);
+\node[anchor=east, font=\scriptsize, text=black!82] at (2.320,1.480) {FengWu};
+\draw[wfslate, line width=0.72pt] (4.036,1.350) -- (4.036,1.350);
+\draw[wfslate, line width=0.72pt] (4.036,1.315) -- (4.036,1.385);
+\draw[wfslate, line width=0.72pt] (4.036,1.315) -- (4.036,1.385);
+\filldraw[wfslate] (4.036,1.350) circle[radius=0.045];
+\draw[wforange, line width=0.72pt] (4.028,1.610) -- (4.127,1.610);
+\draw[wforange, line width=0.72pt] (4.028,1.575) -- (4.028,1.645);
+\draw[wforange, line width=0.72pt] (4.127,1.575) -- (4.127,1.645);
+\filldraw[wforange] (4.032,1.565) rectangle (4.122,1.655);
+\node[anchor=east, font=\scriptsize, text=black!82] at (2.320,1.070) {FuXi};
+\draw[wfslate, line width=0.72pt] (4.036,0.940) -- (4.036,0.940);
+\draw[wfslate, line width=0.72pt] (4.036,0.905) -- (4.036,0.975);
+\draw[wfslate, line width=0.72pt] (4.036,0.905) -- (4.036,0.975);
+\filldraw[wfslate] (4.036,0.940) circle[radius=0.045];
+\draw[wforange, line width=0.72pt] (4.029,1.200) -- (4.087,1.200);
+\draw[wforange, line width=0.72pt] (4.029,1.165) -- (4.029,1.235);
+\draw[wforange, line width=0.72pt] (4.087,1.165) -- (4.087,1.235);
+\filldraw[wforange] (4.013,1.155) rectangle (4.103,1.245);
+\node[anchor=east, font=\scriptsize, text=black!82] at (2.320,0.660) {Pangu-Weather};
+\draw[wfslate, line width=0.72pt] (4.036,0.530) -- (4.036,0.530);
+\draw[wfslate, line width=0.72pt] (4.036,0.495) -- (4.036,0.565);
+\draw[wfslate, line width=0.72pt] (4.036,0.495) -- (4.036,0.565);
+\filldraw[wfslate] (4.036,0.530) circle[radius=0.045];
+\draw[wforange, line width=0.72pt] (4.025,0.790) -- (4.076,0.790);
+\draw[wforange, line width=0.72pt] (4.025,0.755) -- (4.025,0.825);
+\draw[wforange, line width=0.72pt] (4.076,0.755) -- (4.076,0.825);
+\filldraw[wforange] (4.006,0.745) rectangle (4.096,0.835);
+\node[anchor=east, font=\scriptsize, text=black!82] at (2.320,0.250) {AlphaEarth};
+\draw[wfslate, line width=0.72pt] (4.700,0.120) -- (6.103,0.120);
+\draw[wfslate, line width=0.72pt] (4.700,0.085) -- (4.700,0.155);
+\draw[wfslate, line width=0.72pt] (6.103,0.085) -- (6.103,0.155);
+\filldraw[wfslate] (5.401,0.120) circle[radius=0.045];
+\draw[wforange, line width=0.72pt] (3.872,0.380) -- (4.815,0.380);
+\draw[wforange, line width=0.72pt] (3.872,0.345) -- (3.872,0.415);
+\draw[wforange, line width=0.72pt] (4.815,0.345) -- (4.815,0.415);
+\filldraw[wforange] (4.298,0.335) rectangle (4.388,0.425);
+\end{tikzpicture}

paper_outputs/figures/fig_task_rank_map.pdf

@@ -0,0 +1,348 @@
+%PDF-1.4
+%����
+1 0 obj
+<< /Type /Catalog /Pages 2 0 R >>
+endobj
+2 0 obj
+<< /Type /Pages /Kids [3 0 R] /Count 1 >>
+endobj
+3 0 obj
+<< /Type /Page /Parent 2 0 R /MediaBox [0 0 1120 430] /Resources << /Font << /F1 4 0 R /F2 5 0 R >> >> /Contents 6 0 R >>
+endobj
+4 0 obj
+<< /Type /Font /Subtype /Type1 /BaseFont /Helvetica >>
+endobj
+5 0 obj
+<< /Type /Font /Subtype /Type1 /BaseFont /Helvetica-Bold >>
+endobj
+6 0 obj
+<< /Length 22799 >>
+stream
+1.0000 1.0000 1.0000 rg 0.00 0.00 1120.00 430.00 re f
+BT /F2 8.70 Tf 0.1200 0.1400 0.1600 rg 1 0 0 1 117.66 376.00 Tm (FireWx-FM ref.) Tj ET
+BT /F2 8.70 Tf 0.1200 0.1400 0.1600 rg 1 0 0 1 211.06 376.00 Tm (Prithvi-WxC) Tj ET
+BT /F2 8.70 Tf 0.1200 0.1400 0.1600 rg 1 0 0 1 308.29 376.00 Tm (Aurora) Tj ET
+BT /F2 8.70 Tf 0.1200 0.1400 0.1600 rg 1 0 0 1 394.93 376.00 Tm (ClimaX) Tj ET
+BT /F2 8.70 Tf 0.1200 0.1400 0.1600 rg 1 0 0 1 471.34 376.00 Tm (StormCast) Tj ET
+BT /F2 8.70 Tf 0.1200 0.1400 0.1600 rg 1 0 0 1 568.76 376.00 Tm (DLWP) Tj ET
+BT /F2 8.70 Tf 0.1200 0.1400 0.1600 rg 1 0 0 1 658.50 376.00 Tm (FCN) Tj ET
+BT /F2 8.70 Tf 0.1200 0.1400 0.1600 rg 1 0 0 1 736.92 376.00 Tm (FengWu) Tj ET
+BT /F2 8.70 Tf 0.1200 0.1400 0.1600 rg 1 0 0 1 829.82 376.00 Tm (FuXi) Tj ET
+BT /F2 8.70 Tf 0.1200 0.1400 0.1600 rg 1 0 0 1 892.30 376.00 Tm (Pangu-Weather) Tj ET
+BT /F2 8.70 Tf 0.1200 0.1400 0.1600 rg 1 0 0 1 987.57 376.00 Tm (AlphaEarth) Tj ET
+BT /F2 7.70 Tf 0.1200 0.1400 0.1600 rg 1 0 0 1 12.00 325.00 Tm (Occupancy) Tj ET
+BT /F2 7.10 Tf 0.1200 0.1400 0.1600 rg 1 0 0 1 12.00 314.00 Tm (Union F1 \(%\)) Tj ET
+BT /F1 6.40 Tf 0.4200 0.4400 0.4600 rg 1 0 0 1 12.00 303.00 Tm (higher better) Tj ET
+0.80 w 0.1500 0.4760 0.4860 rg 1.0000 1.0000 1.0000 RG 108.00 300.00 86.00 42.00 re B
+BT /F2 11.20 Tf 1.0000 1.0000 1.0000 rg 1 0 0 1 144.88 324.00 Tm (#2) Tj ET
+BT /F1 7.00 Tf 1.0000 1.0000 1.0000 rg 1 0 0 1 142.84 309.00 Tm (59.07) Tj ET
+0.80 w 0.9300 0.9500 0.9400 rg 1.0000 1.0000 1.0000 RG 194.00 300.00 86.00 42.00 re B
+BT /F2 11.20 Tf 0.0700 0.0900 0.1100 rg 1 0 0 1 227.83 324.00 Tm (#11) Tj ET
+BT /F1 7.00 Tf 0.0700 0.0900 0.1100 rg 1 0 0 1 228.84 309.00 Tm (20.19) Tj ET
+0.80 w 0.7780 0.8820 0.8640 rg 1.0000 1.0000 1.0000 RG 280.00 300.00 86.00 42.00 re B
+BT /F2 11.20 Tf 0.0700 0.0900 0.1100 rg 1 0 0 1 316.88 324.00 Tm (#9) Tj ET
+BT /F1 7.00 Tf 0.0700 0.0900 0.1100 rg 1 0 0 1 314.85 309.00 Tm (23.10) Tj ET
+0.80 w 0.0500 0.4000 0.4200 rg 1.0000 1.0000 1.0000 RG 366.00 300.00 86.00 42.00 re B
+BT /F2 11.20 Tf 1.0000 1.0000 1.0000 rg 1 0 0 1 402.88 324.00 Tm (#1) Tj ET
+BT /F1 7.00 Tf 1.0000 1.0000 1.0000 rg 1 0 0 1 400.85 309.00 Tm (60.15) Tj ET
+0.80 w 0.8540 0.9160 0.9020 rg 1.0000 1.0000 1.0000 RG 452.00 300.00 86.00 42.00 re B
+BT /F2 11.20 Tf 0.0700 0.0900 0.1100 rg 1 0 0 1 485.83 324.00 Tm (#10) Tj ET
+BT /F1 7.00 Tf 0.0700 0.0900 0.1100 rg 1 0 0 1 486.85 309.00 Tm (22.38) Tj ET
+0.80 w 0.6260 0.8140 0.7880 rg 1.0000 1.0000 1.0000 RG 538.00 300.00 86.00 42.00 re B
+BT /F2 11.20 Tf 0.0700 0.0900 0.1100 rg 1 0 0 1 574.88 324.00 Tm (#7) Tj ET
+BT /F1 7.00 Tf 0.0700 0.0900 0.1100 rg 1 0 0 1 572.85 309.00 Tm (28.19) Tj ET
+0.80 w 0.2500 0.5520 0.5520 rg 1.0000 1.0000 1.0000 RG 624.00 300.00 86.00 42.00 re B
+BT /F2 11.20 Tf 0.0700 0.0900 0.1100 rg 1 0 0 1 660.88 324.00 Tm (#3) Tj ET
+BT /F1 7.00 Tf 0.0700 0.0900 0.1100 rg 1 0 0 1 658.85 309.00 Tm (40.06) Tj ET
+0.80 w 0.7020 0.8480 0.8260 rg 1.0000 1.0000 1.0000 RG 710.00 300.00 86.00 42.00 re B
+BT /F2 11.20 Tf 0.0700 0.0900 0.1100 rg 1 0 0 1 746.88 324.00 Tm (#8) Tj ET
+BT /F1 7.00 Tf 0.0700 0.0900 0.1100 rg 1 0 0 1 744.85 309.00 Tm (24.10) Tj ET
+0.80 w 0.4500 0.7040 0.6840 rg 1.0000 1.0000 1.0000 RG 796.00 300.00 86.00 42.00 re B
+BT /F2 11.20 Tf 0.0700 0.0900 0.1100 rg 1 0 0 1 832.88 324.00 Tm (#5) Tj ET
+BT /F1 7.00 Tf 0.0700 0.0900 0.1100 rg 1 0 0 1 830.85 309.00 Tm (37.29) Tj ET
+0.80 w 0.5500 0.7800 0.7500 rg 1.0000 1.0000 1.0000 RG 882.00 300.00 86.00 42.00 re B
+BT /F2 11.20 Tf 0.0700 0.0900 0.1100 rg 1 0 0 1 918.88 324.00 Tm (#6) Tj ET
+BT /F1 7.00 Tf 0.0700 0.0900 0.1100 rg 1 0 0 1 916.85 309.00 Tm (35.64) Tj ET
+0.80 w 0.3500 0.6280 0.6180 rg 1.0000 1.0000 1.0000 RG 968.00 300.00 86.00 42.00 re B
+BT /F2 11.20 Tf 0.0700 0.0900 0.1100 rg 1 0 0 1 1004.88 324.00 Tm (#4) Tj ET
+BT /F1 7.00 Tf 0.0700 0.0900 0.1100 rg 1 0 0 1 1002.85 309.00 Tm (37.43) Tj ET
+BT /F2 7.70 Tf 0.1200 0.1400 0.1600 rg 1 0 0 1 12.00 283.00 Tm (Fire spread) Tj ET
+BT /F2 7.10 Tf 0.1200 0.1400 0.1600 rg 1 0 0 1 12.00 272.00 Tm (AP \(%\)) Tj ET
+BT /F1 6.40 Tf 0.4200 0.4400 0.4600 rg 1 0 0 1 12.00 261.00 Tm (higher better) Tj ET
+0.80 w 0.0500 0.4000 0.4200 rg 1.0000 1.0000 1.0000 RG 108.00 258.00 86.00 42.00 re B
+BT /F2 11.20 Tf 1.0000 1.0000 1.0000 rg 1 0 0 1 144.88 282.00 Tm (#1) Tj ET
+BT /F1 7.00 Tf 1.0000 1.0000 1.0000 rg 1 0 0 1 142.84 267.00 Tm (30.09) Tj ET
+0.80 w 0.7780 0.8820 0.8640 rg 1.0000 1.0000 1.0000 RG 194.00 258.00 86.00 42.00 re B
+BT /F2 11.20 Tf 0.0700 0.0900 0.1100 rg 1 0 0 1 230.88 282.00 Tm (#9) Tj ET
+BT /F1 7.00 Tf 0.0700 0.0900 0.1100 rg 1 0 0 1 230.66 267.00 Tm (5.00) Tj ET
+0.80 w 0.1500 0.4760 0.4860 rg 1.0000 1.0000 1.0000 RG 280.00 258.00 86.00 42.00 re B
+BT /F2 11.20 Tf 1.0000 1.0000 1.0000 rg 1 0 0 1 316.88 282.00 Tm (#2) Tj ET
+BT /F1 7.00 Tf 1.0000 1.0000 1.0000 rg 1 0 0 1 314.85 267.00 Tm (16.62) Tj ET
+0.80 w 0.6260 0.8140 0.7880 rg 1.0000 1.0000 1.0000 RG 366.00 258.00 86.00 42.00 re B
+BT /F2 11.20 Tf 0.0700 0.0900 0.1100 rg 1 0 0 1 402.88 282.00 Tm (#7) Tj ET
+BT /F1 7.00 Tf 0.0700 0.0900 0.1100 rg 1 0 0 1 400.85 267.00 Tm (11.17) Tj ET
+0.80 w 0.8540 0.9160 0.9020 rg 1.0000 1.0000 1.0000 RG 452.00 258.00 86.00 42.00 re B
+BT /F2 11.20 Tf 0.0700 0.0900 0.1100 rg 1 0 0 1 485.83 282.00 Tm (#10) Tj ET
+BT /F1 7.00 Tf 0.0700 0.0900 0.1100 rg 1 0 0 1 488.67 267.00 Tm (2.81) Tj ET
+0.80 w 0.7020 0.8480 0.8260 rg 1.0000 1.0000 1.0000 RG 538.00 258.00 86.00 42.00 re B
+BT /F2 11.20 Tf 0.0700 0.0900 0.1100 rg 1 0 0 1 574.88 282.00 Tm (#8) Tj ET
+BT /F1 7.00 Tf 0.0700 0.0900 0.1100 rg 1 0 0 1 574.66 267.00 Tm (5.94) Tj ET
+0.80 w 0.9300 0.9500 0.9400 rg 1.0000 1.0000 1.0000 RG 624.00 258.00 86.00 42.00 re B
+BT /F2 11.20 Tf 0.0700 0.0900 0.1100 rg 1 0 0 1 657.83 282.00 Tm (#11) Tj ET
+BT /F1 7.00 Tf 0.0700 0.0900 0.1100 rg 1 0 0 1 660.66 267.00 Tm (2.39) Tj ET
+0.80 w 0.3500 0.6280 0.6180 rg 1.0000 1.0000 1.0000 RG 710.00 258.00 86.00 42.00 re B
+BT /F2 11.20 Tf 0.0700 0.0900 0.1100 rg 1 0 0 1 746.88 282.00 Tm (#4) Tj ET
+BT /F1 7.00 Tf 0.0700 0.0900 0.1100 rg 1 0 0 1 744.85 267.00 Tm (13.17) Tj ET
+0.80 w 0.2500 0.5520 0.5520 rg 1.0000 1.0000 1.0000 RG 796.00 258.00 86.00 42.00 re B
+BT /F2 11.20 Tf 0.0700 0.0900 0.1100 rg 1 0 0 1 832.88 282.00 Tm (#3) Tj ET
+BT /F1 7.00 Tf 0.0700 0.0900 0.1100 rg 1 0 0 1 830.85 267.00 Tm (14.35) Tj ET
+0.80 w 0.4500 0.7040 0.6840 rg 1.0000 1.0000 1.0000 RG 882.00 258.00 86.00 42.00 re B
+BT /F2 11.20 Tf 0.0700 0.0900 0.1100 rg 1 0 0 1 918.88 282.00 Tm (#5) Tj ET
+BT /F1 7.00 Tf 0.0700 0.0900 0.1100 rg 1 0 0 1 916.85 267.00 Tm (12.69) Tj ET
+0.80 w 0.5500 0.7800 0.7500 rg 1.0000 1.0000 1.0000 RG 968.00 258.00 86.00 42.00 re B
+BT /F2 11.20 Tf 0.0700 0.0900 0.1100 rg 1 0 0 1 1004.88 282.00 Tm (#6) Tj ET
+BT /F1 7.00 Tf 0.0700 0.0900 0.1100 rg 1 0 0 1 1002.85 267.00 Tm (11.83) Tj ET
+BT /F2 7.70 Tf 0.1200 0.1400 0.1600 rg 1 0 0 1 12.00 241.00 Tm (Burned area) Tj ET
+BT /F2 7.10 Tf 0.1200 0.1400 0.1600 rg 1 0 0 1 12.00 230.00 Tm (log-RMSE) Tj ET
+BT /F1 6.40 Tf 0.4200 0.4400 0.4600 rg 1 0 0 1 12.00 219.00 Tm (lower better) Tj ET
+0.80 w 0.0500 0.4000 0.4200 rg 1.0000 1.0000 1.0000 RG 108.00 216.00 86.00 42.00 re B
+BT /F2 11.20 Tf 1.0000 1.0000 1.0000 rg 1 0 0 1 144.88 240.00 Tm (#1) Tj ET
+BT /F1 7.00 Tf 1.0000 1.0000 1.0000 rg 1 0 0 1 144.66 225.00 Tm (1.17) Tj ET
+0.80 w 0.3500 0.6280 0.6180 rg 1.0000 1.0000 1.0000 RG 194.00 216.00 86.00 42.00 re B
+BT /F2 11.20 Tf 0.0700 0.0900 0.1100 rg 1 0 0 1 230.88 240.00 Tm (#4) Tj ET
+BT /F1 7.00 Tf 0.0700 0.0900 0.1100 rg 1 0 0 1 230.66 225.00 Tm (1.36) Tj ET
+0.80 w 0.7780 0.8820 0.8640 rg 1.0000 1.0000 1.0000 RG 280.00 216.00 86.00 42.00 re B
+BT /F2 11.20 Tf 0.0700 0.0900 0.1100 rg 1 0 0 1 316.88 240.00 Tm (#9) Tj ET
+BT /F1 7.00 Tf 0.0700 0.0900 0.1100 rg 1 0 0 1 316.67 225.00 Tm (1.87) Tj ET
+0.80 w 0.8540 0.9160 0.9020 rg 1.0000 1.0000 1.0000 RG 366.00 216.00 86.00 42.00 re B
+BT /F2 11.20 Tf 0.0700 0.0900 0.1100 rg 1 0 0 1 399.83 240.00 Tm (#10) Tj ET
+BT /F1 7.00 Tf 0.0700 0.0900 0.1100 rg 1 0 0 1 402.67 225.00 Tm (2.03) Tj ET
+0.80 w 0.7020 0.8480 0.8260 rg 1.0000 1.0000 1.0000 RG 452.00 216.00 86.00 42.00 re B
+BT /F2 11.20 Tf 0.0700 0.0900 0.1100 rg 1 0 0 1 488.88 240.00 Tm (#8) Tj ET
+BT /F1 7.00 Tf 0.0700 0.0900 0.1100 rg 1 0 0 1 488.67 225.00 Tm (1.67) Tj ET
+0.80 w 0.1500 0.4760 0.4860 rg 1.0000 1.0000 1.0000 RG 538.00 216.00 86.00 42.00 re B
+BT /F2 11.20 Tf 1.0000 1.0000 1.0000 rg 1 0 0 1 574.88 240.00 Tm (#2) Tj ET
+BT /F1 7.00 Tf 1.0000 1.0000 1.0000 rg 1 0 0 1 574.66 225.00 Tm (1.31) Tj ET
+0.80 w 0.4500 0.7040 0.6840 rg 1.0000 1.0000 1.0000 RG 624.00 216.00 86.00 42.00 re B
+BT /F2 11.20 Tf 0.0700 0.0900 0.1100 rg 1 0 0 1 660.88 240.00 Tm (#5) Tj ET
+BT /F1 7.00 Tf 0.0700 0.0900 0.1100 rg 1 0 0 1 660.66 225.00 Tm (1.37) Tj ET
+0.80 w 0.5500 0.7800 0.7500 rg 1.0000 1.0000 1.0000 RG 710.00 216.00 86.00 42.00 re B
+BT /F2 11.20 Tf 0.0700 0.0900 0.1100 rg 1 0 0 1 746.88 240.00 Tm (#6) Tj ET
+BT /F1 7.00 Tf 0.0700 0.0900 0.1100 rg 1 0 0 1 746.66 225.00 Tm (1.37) Tj ET
+0.80 w 0.6260 0.8140 0.7880 rg 1.0000 1.0000 1.0000 RG 796.00 216.00 86.00 42.00 re B
+BT /F2 11.20 Tf 0.0700 0.0900 0.1100 rg 1 0 0 1 832.88 240.00 Tm (#7) Tj ET
+BT /F1 7.00 Tf 0.0700 0.0900 0.1100 rg 1 0 0 1 832.66 225.00 Tm (1.41) Tj ET
+0.80 w 0.2500 0.5520 0.5520 rg 1.0000 1.0000 1.0000 RG 882.00 216.00 86.00 42.00 re B
+BT /F2 11.20 Tf 0.0700 0.0900 0.1100 rg 1 0 0 1 918.88 240.00 Tm (#3) Tj ET
+BT /F1 7.00 Tf 0.0700 0.0900 0.1100 rg 1 0 0 1 918.66 225.00 Tm (1.33) Tj ET
+0.80 w 0.9300 0.9500 0.9400 rg 1.0000 1.0000 1.0000 RG 968.00 216.00 86.00 42.00 re B
+BT /F2 11.20 Tf 0.0700 0.0900 0.1100 rg 1 0 0 1 1001.83 240.00 Tm (#11) Tj ET
+BT /F1 7.00 Tf 0.0700 0.0900 0.1100 rg 1 0 0 1 1004.66 225.00 Tm (2.41) Tj ET
+BT /F2 7.70 Tf 0.1200 0.1400 0.1600 rg 1 0 0 1 12.00 199.00 Tm (Analog retrieval) Tj ET
+BT /F2 7.10 Tf 0.1200 0.1400 0.1600 rg 1 0 0 1 12.00 188.00 Tm (nDCG@10) Tj ET
+BT /F1 6.40 Tf 0.4200 0.4400 0.4600 rg 1 0 0 1 12.00 177.00 Tm (higher better) Tj ET
+0.80 w 0.0500 0.4000 0.4200 rg 1.0000 1.0000 1.0000 RG 108.00 174.00 86.00 42.00 re B
+BT /F2 11.20 Tf 1.0000 1.0000 1.0000 rg 1 0 0 1 144.88 198.00 Tm (#1) Tj ET
+BT /F1 7.00 Tf 1.0000 1.0000 1.0000 rg 1 0 0 1 142.84 183.00 Tm (0.510) Tj ET
+0.80 w 0.9300 0.9500 0.9400 rg 1.0000 1.0000 1.0000 RG 194.00 174.00 86.00 42.00 re B
+BT /F2 11.20 Tf 0.0700 0.0900 0.1100 rg 1 0 0 1 227.83 198.00 Tm (#11) Tj ET
+BT /F1 7.00 Tf 0.0700 0.0900 0.1100 rg 1 0 0 1 228.84 183.00 Tm (0.386) Tj ET
+0.80 w 0.7020 0.8480 0.8260 rg 1.0000 1.0000 1.0000 RG 280.00 174.00 86.00 42.00 re B
+BT /F2 11.20 Tf 0.0700 0.0900 0.1100 rg 1 0 0 1 316.88 198.00 Tm (#8) Tj ET
+BT /F1 7.00 Tf 0.0700 0.0900 0.1100 rg 1 0 0 1 314.85 183.00 Tm (0.405) Tj ET
+0.80 w 0.5500 0.7800 0.7500 rg 1.0000 1.0000 1.0000 RG 366.00 174.00 86.00 42.00 re B
+BT /F2 11.20 Tf 0.0700 0.0900 0.1100 rg 1 0 0 1 402.88 198.00 Tm (#6) Tj ET
+BT /F1 7.00 Tf 0.0700 0.0900 0.1100 rg 1 0 0 1 400.85 183.00 Tm (0.414) Tj ET
+0.80 w 0.6260 0.8140 0.7880 rg 1.0000 1.0000 1.0000 RG 452.00 174.00 86.00 42.00 re B
+BT /F2 11.20 Tf 0.0700 0.0900 0.1100 rg 1 0 0 1 488.88 198.00 Tm (#7) Tj ET
+BT /F1 7.00 Tf 0.0700 0.0900 0.1100 rg 1 0 0 1 486.85 183.00 Tm (0.408) Tj ET
+0.80 w 0.8540 0.9160 0.9020 rg 1.0000 1.0000 1.0000 RG 538.00 174.00 86.00 42.00 re B
+BT /F2 11.20 Tf 0.0700 0.0900 0.1100 rg 1 0 0 1 571.83 198.00 Tm (#10) Tj ET
+BT /F1 7.00 Tf 0.0700 0.0900 0.1100 rg 1 0 0 1 572.85 183.00 Tm (0.397) Tj ET
+0.80 w 0.2500 0.5520 0.5520 rg 1.0000 1.0000 1.0000 RG 624.00 174.00 86.00 42.00 re B
+BT /F2 11.20 Tf 0.0700 0.0900 0.1100 rg 1 0 0 1 660.88 198.00 Tm (#3) Tj ET
+BT /F1 7.00 Tf 0.0700 0.0900 0.1100 rg 1 0 0 1 658.85 183.00 Tm (0.432) Tj ET
+0.80 w 0.4500 0.7040 0.6840 rg 1.0000 1.0000 1.0000 RG 710.00 174.00 86.00 42.00 re B
+BT /F2 11.20 Tf 0.0700 0.0900 0.1100 rg 1 0 0 1 746.88 198.00 Tm (#5) Tj ET
+BT /F1 7.00 Tf 0.0700 0.0900 0.1100 rg 1 0 0 1 744.85 183.00 Tm (0.425) Tj ET
+0.80 w 0.3500 0.6280 0.6180 rg 1.0000 1.0000 1.0000 RG 796.00 174.00 86.00 42.00 re B
+BT /F2 11.20 Tf 0.0700 0.0900 0.1100 rg 1 0 0 1 832.88 198.00 Tm (#4) Tj ET
+BT /F1 7.00 Tf 0.0700 0.0900 0.1100 rg 1 0 0 1 830.85 183.00 Tm (0.428) Tj ET
+0.80 w 0.7780 0.8820 0.8640 rg 1.0000 1.0000 1.0000 RG 882.00 174.00 86.00 42.00 re B
+BT /F2 11.20 Tf 0.0700 0.0900 0.1100 rg 1 0 0 1 918.88 198.00 Tm (#9) Tj ET
+BT /F1 7.00 Tf 0.0700 0.0900 0.1100 rg 1 0 0 1 916.85 183.00 Tm (0.402) Tj ET
+0.80 w 0.1500 0.4760 0.4860 rg 1.0000 1.0000 1.0000 RG 968.00 174.00 86.00 42.00 re B
+BT /F2 11.20 Tf 1.0000 1.0000 1.0000 rg 1 0 0 1 1004.88 198.00 Tm (#2) Tj ET
+BT /F1 7.00 Tf 1.0000 1.0000 1.0000 rg 1 0 0 1 1002.85 183.00 Tm (0.509) Tj ET
+BT /F2 7.70 Tf 0.1200 0.1400 0.1600 rg 1 0 0 1 12.00 157.00 Tm (Smoke PM2.5) Tj ET
+BT /F2 7.10 Tf 0.1200 0.1400 0.1600 rg 1 0 0 1 12.00 146.00 Tm (RMSE) Tj ET
+BT /F1 6.40 Tf 0.4200 0.4400 0.4600 rg 1 0 0 1 12.00 135.00 Tm (lower better) Tj ET
+0.80 w 0.1500 0.4760 0.4860 rg 1.0000 1.0000 1.0000 RG 108.00 132.00 86.00 42.00 re B
+BT /F2 11.20 Tf 1.0000 1.0000 1.0000 rg 1 0 0 1 144.88 156.00 Tm (#2) Tj ET
+BT /F1 7.00 Tf 1.0000 1.0000 1.0000 rg 1 0 0 1 144.66 141.00 Tm (4.46) Tj ET
+0.80 w 0.7020 0.8480 0.8260 rg 1.0000 1.0000 1.0000 RG 194.00 132.00 86.00 42.00 re B
+BT /F2 11.20 Tf 0.0700 0.0900 0.1100 rg 1 0 0 1 230.88 156.00 Tm (#8) Tj ET
+BT /F1 7.00 Tf 0.0700 0.0900 0.1100 rg 1 0 0 1 230.66 141.00 Tm (6.04) Tj ET
+0.80 w 0.7780 0.8820 0.8640 rg 1.0000 1.0000 1.0000 RG 280.00 132.00 86.00 42.00 re B
+BT /F2 11.20 Tf 0.0700 0.0900 0.1100 rg 1 0 0 1 316.88 156.00 Tm (#9) Tj ET
+BT /F1 7.00 Tf 0.0700 0.0900 0.1100 rg 1 0 0 1 316.67 141.00 Tm (6.04) Tj ET
+0.80 w 0.8540 0.9160 0.9020 rg 1.0000 1.0000 1.0000 RG 366.00 132.00 86.00 42.00 re B
+BT /F2 11.20 Tf 0.0700 0.0900 0.1100 rg 1 0 0 1 399.83 156.00 Tm (#10) Tj ET
+BT /F1 7.00 Tf 0.0700 0.0900 0.1100 rg 1 0 0 1 402.67 141.00 Tm (6.04) Tj ET
+0.80 w 0.9300 0.9500 0.9400 rg 1.0000 1.0000 1.0000 RG 452.00 132.00 86.00 42.00 re B
+BT /F2 11.20 Tf 0.0700 0.0900 0.1100 rg 1 0 0 1 485.83 156.00 Tm (#11) Tj ET
+BT /F1 7.00 Tf 0.0700 0.0900 0.1100 rg 1 0 0 1 488.67 141.00 Tm (6.12) Tj ET
+0.80 w 0.4500 0.7040 0.6840 rg 1.0000 1.0000 1.0000 RG 538.00 132.00 86.00 42.00 re B
+BT /F2 11.20 Tf 0.0700 0.0900 0.1100 rg 1 0 0 1 574.88 156.00 Tm (#5) Tj ET
+BT /F1 7.00 Tf 0.0700 0.0900 0.1100 rg 1 0 0 1 574.66 141.00 Tm (5.93) Tj ET
+0.80 w 0.3500 0.6280 0.6180 rg 1.0000 1.0000 1.0000 RG 624.00 132.00 86.00 42.00 re B
+BT /F2 11.20 Tf 0.0700 0.0900 0.1100 rg 1 0 0 1 660.88 156.00 Tm (#4) Tj ET
+BT /F1 7.00 Tf 0.0700 0.0900 0.1100 rg 1 0 0 1 660.66 141.00 Tm (5.93) Tj ET
+0.80 w 0.5500 0.7800 0.7500 rg 1.0000 1.0000 1.0000 RG 710.00 132.00 86.00 42.00 re B
+BT /F2 11.20 Tf 0.0700 0.0900 0.1100 rg 1 0 0 1 746.88 156.00 Tm (#6) Tj ET
+BT /F1 7.00 Tf 0.0700 0.0900 0.1100 rg 1 0 0 1 746.66 141.00 Tm (5.93) Tj ET
+0.80 w 0.6260 0.8140 0.7880 rg 1.0000 1.0000 1.0000 RG 796.00 132.00 86.00 42.00 re B
+BT /F2 11.20 Tf 0.0700 0.0900 0.1100 rg 1 0 0 1 832.88 156.00 Tm (#7) Tj ET
+BT /F1 7.00 Tf 0.0700 0.0900 0.1100 rg 1 0 0 1 832.66 141.00 Tm (5.93) Tj ET
+0.80 w 0.2500 0.5520 0.5520 rg 1.0000 1.0000 1.0000 RG 882.00 132.00 86.00 42.00 re B
+BT /F2 11.20 Tf 0.0700 0.0900 0.1100 rg 1 0 0 1 918.88 156.00 Tm (#3) Tj ET
+BT /F1 7.00 Tf 0.0700 0.0900 0.1100 rg 1 0 0 1 918.66 141.00 Tm (5.93) Tj ET
+0.80 w 0.0500 0.4000 0.4200 rg 1.0000 1.0000 1.0000 RG 968.00 132.00 86.00 42.00 re B
+BT /F2 11.20 Tf 1.0000 1.0000 1.0000 rg 1 0 0 1 1004.88 156.00 Tm (#1) Tj ET
+BT /F1 7.00 Tf 1.0000 1.0000 1.0000 rg 1 0 0 1 1004.66 141.00 Tm (4.44) Tj ET
+BT /F2 7.70 Tf 0.1200 0.1400 0.1600 rg 1 0 0 1 12.00 115.00 Tm (Extreme heat) Tj ET
+BT /F2 7.10 Tf 0.1200 0.1400 0.1600 rg 1 0 0 1 12.00 104.00 Tm (RMSE-C) Tj ET
+BT /F1 6.40 Tf 0.4200 0.4400 0.4600 rg 1 0 0 1 12.00 93.00 Tm (lower better) Tj ET
+0.80 w 0.0500 0.4000 0.4200 rg 1.0000 1.0000 1.0000 RG 108.00 90.00 86.00 42.00 re B
+BT /F2 11.20 Tf 1.0000 1.0000 1.0000 rg 1 0 0 1 144.88 114.00 Tm (#1) Tj ET
+BT /F1 7.00 Tf 1.0000 1.0000 1.0000 rg 1 0 0 1 142.84 99.00 Tm (0.218) Tj ET
+0.80 w 0.7780 0.8820 0.8640 rg 1.0000 1.0000 1.0000 RG 194.00 90.00 86.00 42.00 re B
+BT /F2 11.20 Tf 0.0700 0.0900 0.1100 rg 1 0 0 1 230.88 114.00 Tm (#9) Tj ET
+BT /F1 7.00 Tf 0.0700 0.0900 0.1100 rg 1 0 0 1 230.66 99.00 Tm (4.62) Tj ET
+0.80 w 0.9300 0.9500 0.9400 rg 1.0000 1.0000 1.0000 RG 280.00 90.00 86.00 42.00 re B
+BT /F2 11.20 Tf 0.0700 0.0900 0.1100 rg 1 0 0 1 313.83 114.00 Tm (#11) Tj ET
+BT /F1 7.00 Tf 0.0700 0.0900 0.1100 rg 1 0 0 1 314.85 99.00 Tm (18.05) Tj ET
+0.80 w 0.8540 0.9160 0.9020 rg 1.0000 1.0000 1.0000 RG 366.00 90.00 86.00 42.00 re B
+BT /F2 11.20 Tf 0.0700 0.0900 0.1100 rg 1 0 0 1 399.83 114.00 Tm (#10) Tj ET
+BT /F1 7.00 Tf 0.0700 0.0900 0.1100 rg 1 0 0 1 400.85 99.00 Tm (17.65) Tj ET
+0.80 w 0.2500 0.5520 0.5520 rg 1.0000 1.0000 1.0000 RG 452.00 90.00 86.00 42.00 re B
+BT /F2 11.20 Tf 0.0700 0.0900 0.1100 rg 1 0 0 1 488.88 114.00 Tm (#3) Tj ET
+BT /F1 7.00 Tf 0.0700 0.0900 0.1100 rg 1 0 0 1 488.67 99.00 Tm (1.77) Tj ET
+0.80 w 0.7020 0.8480 0.8260 rg 1.0000 1.0000 1.0000 RG 538.00 90.00 86.00 42.00 re B
+BT /F2 11.20 Tf 0.0700 0.0900 0.1100 rg 1 0 0 1 574.88 114.00 Tm (#8) Tj ET
+BT /F1 7.00 Tf 0.0700 0.0900 0.1100 rg 1 0 0 1 574.66 99.00 Tm (2.27) Tj ET
+0.80 w 0.5500 0.7800 0.7500 rg 1.0000 1.0000 1.0000 RG 624.00 90.00 86.00 42.00 re B
+BT /F2 11.20 Tf 0.0700 0.0900 0.1100 rg 1 0 0 1 660.88 114.00 Tm (#6) Tj ET
+BT /F1 7.00 Tf 0.0700 0.0900 0.1100 rg 1 0 0 1 660.66 99.00 Tm (2.17) Tj ET
+0.80 w 0.3500 0.6280 0.6180 rg 1.0000 1.0000 1.0000 RG 710.00 90.00 86.00 42.00 re B
+BT /F2 11.20 Tf 0.0700 0.0900 0.1100 rg 1 0 0 1 746.88 114.00 Tm (#4) Tj ET
+BT /F1 7.00 Tf 0.0700 0.0900 0.1100 rg 1 0 0 1 746.66 99.00 Tm (2.13) Tj ET
+0.80 w 0.4500 0.7040 0.6840 rg 1.0000 1.0000 1.0000 RG 796.00 90.00 86.00 42.00 re B
+BT /F2 11.20 Tf 0.0700 0.0900 0.1100 rg 1 0 0 1 832.88 114.00 Tm (#5) Tj ET
+BT /F1 7.00 Tf 0.0700 0.0900 0.1100 rg 1 0 0 1 832.66 99.00 Tm (2.13) Tj ET
+0.80 w 0.6260 0.8140 0.7880 rg 1.0000 1.0000 1.0000 RG 882.00 90.00 86.00 42.00 re B
+BT /F2 11.20 Tf 0.0700 0.0900 0.1100 rg 1 0 0 1 918.88 114.00 Tm (#7) Tj ET
+BT /F1 7.00 Tf 0.0700 0.0900 0.1100 rg 1 0 0 1 918.66 99.00 Tm (2.20) Tj ET
+0.80 w 0.1500 0.4760 0.4860 rg 1.0000 1.0000 1.0000 RG 968.00 90.00 86.00 42.00 re B
+BT /F2 11.20 Tf 1.0000 1.0000 1.0000 rg 1 0 0 1 1004.88 114.00 Tm (#2) Tj ET
+BT /F1 7.00 Tf 1.0000 1.0000 1.0000 rg 1 0 0 1 1002.85 99.00 Tm (0.219) Tj ET
+0.80 w 0.2000 0.2200 0.2400 RG 108.00 90.00 946.00 252.00 re S
+BT /F2 9.00 Tf 0.2400 0.2500 0.2600 rg 1 0 0 1 908.00 63.00 Tm (within-row rank) Tj ET
+0.9300 0.9500 0.9400 rg 834.00 46.00 2.60 10.00 re f
+0.9300 0.9500 0.9400 rg 836.50 46.00 2.60 10.00 re f
+0.9300 0.9500 0.9400 rg 839.00 46.00 2.60 10.00 re f
+0.9300 0.9500 0.9400 rg 841.50 46.00 2.60 10.00 re f
+0.9300 0.9500 0.9400 rg 844.00 46.00 2.60 10.00 re f
+0.9300 0.9500 0.9400 rg 846.50 46.00 2.60 10.00 re f
+0.9300 0.9500 0.9400 rg 849.00 46.00 2.60 10.00 re f
+0.9300 0.9500 0.9400 rg 851.50 46.00 2.60 10.00 re f
+0.8540 0.9160 0.9020 rg 854.00 46.00 2.60 10.00 re f
+0.8540 0.9160 0.9020 rg 856.50 46.00 2.60 10.00 re f
+0.8540 0.9160 0.9020 rg 859.00 46.00 2.60 10.00 re f
+0.8540 0.9160 0.9020 rg 861.50 46.00 2.60 10.00 re f
+0.8540 0.9160 0.9020 rg 864.00 46.00 2.60 10.00 re f
+0.8540 0.9160 0.9020 rg 866.50 46.00 2.60 10.00 re f
+0.8540 0.9160 0.9020 rg 869.00 46.00 2.60 10.00 re f
+0.8540 0.9160 0.9020 rg 871.50 46.00 2.60 10.00 re f
+0.7780 0.8820 0.8640 rg 874.00 46.00 2.60 10.00 re f
+0.7780 0.8820 0.8640 rg 876.50 46.00 2.60 10.00 re f
+0.7780 0.8820 0.8640 rg 879.00 46.00 2.60 10.00 re f
+0.7780 0.8820 0.8640 rg 881.50 46.00 2.60 10.00 re f
+0.7780 0.8820 0.8640 rg 884.00 46.00 2.60 10.00 re f
+0.7780 0.8820 0.8640 rg 886.50 46.00 2.60 10.00 re f
+0.7780 0.8820 0.8640 rg 889.00 46.00 2.60 10.00 re f
+0.7780 0.8820 0.8640 rg 891.50 46.00 2.60 10.00 re f
+0.7020 0.8480 0.8260 rg 894.00 46.00 2.60 10.00 re f
+0.7020 0.8480 0.8260 rg 896.50 46.00 2.60 10.00 re f
+0.7020 0.8480 0.8260 rg 899.00 46.00 2.60 10.00 re f
+0.7020 0.8480 0.8260 rg 901.50 46.00 2.60 10.00 re f
+0.7020 0.8480 0.8260 rg 904.00 46.00 2.60 10.00 re f
+0.7020 0.8480 0.8260 rg 906.50 46.00 2.60 10.00 re f
+0.7020 0.8480 0.8260 rg 909.00 46.00 2.60 10.00 re f
+0.7020 0.8480 0.8260 rg 911.50 46.00 2.60 10.00 re f
+0.6260 0.8140 0.7880 rg 914.00 46.00 2.60 10.00 re f
+0.6260 0.8140 0.7880 rg 916.50 46.00 2.60 10.00 re f
+0.6260 0.8140 0.7880 rg 919.00 46.00 2.60 10.00 re f
+0.6260 0.8140 0.7880 rg 921.50 46.00 2.60 10.00 re f
+0.6260 0.8140 0.7880 rg 924.00 46.00 2.60 10.00 re f
+0.6260 0.8140 0.7880 rg 926.50 46.00 2.60 10.00 re f
+0.6260 0.8140 0.7880 rg 929.00 46.00 2.60 10.00 re f
+0.6260 0.8140 0.7880 rg 931.50 46.00 2.60 10.00 re f
+0.5500 0.7800 0.7500 rg 934.00 46.00 2.60 10.00 re f
+0.5500 0.7800 0.7500 rg 936.50 46.00 2.60 10.00 re f
+0.5500 0.7800 0.7500 rg 939.00 46.00 2.60 10.00 re f
+0.5500 0.7800 0.7500 rg 941.50 46.00 2.60 10.00 re f
+0.5500 0.7800 0.7500 rg 944.00 46.00 2.60 10.00 re f
+0.5500 0.7800 0.7500 rg 946.50 46.00 2.60 10.00 re f
+0.5500 0.7800 0.7500 rg 949.00 46.00 2.60 10.00 re f
+0.5500 0.7800 0.7500 rg 951.50 46.00 2.60 10.00 re f
+0.4500 0.7040 0.6840 rg 954.00 46.00 2.60 10.00 re f
+0.4500 0.7040 0.6840 rg 956.50 46.00 2.60 10.00 re f
+0.4500 0.7040 0.6840 rg 959.00 46.00 2.60 10.00 re f
+0.4500 0.7040 0.6840 rg 961.50 46.00 2.60 10.00 re f
+0.4500 0.7040 0.6840 rg 964.00 46.00 2.60 10.00 re f
+0.4500 0.7040 0.6840 rg 966.50 46.00 2.60 10.00 re f
+0.4500 0.7040 0.6840 rg 969.00 46.00 2.60 10.00 re f
+0.4500 0.7040 0.6840 rg 971.50 46.00 2.60 10.00 re f
+0.3500 0.6280 0.6180 rg 974.00 46.00 2.60 10.00 re f
+0.3500 0.6280 0.6180 rg 976.50 46.00 2.60 10.00 re f
+0.3500 0.6280 0.6180 rg 979.00 46.00 2.60 10.00 re f
+0.3500 0.6280 0.6180 rg 981.50 46.00 2.60 10.00 re f
+0.3500 0.6280 0.6180 rg 984.00 46.00 2.60 10.00 re f
+0.3500 0.6280 0.6180 rg 986.50 46.00 2.60 10.00 re f
+0.3500 0.6280 0.6180 rg 989.00 46.00 2.60 10.00 re f
+0.3500 0.6280 0.6180 rg 991.50 46.00 2.60 10.00 re f
+0.2500 0.5520 0.5520 rg 994.00 46.00 2.60 10.00 re f
+0.2500 0.5520 0.5520 rg 996.50 46.00 2.60 10.00 re f
+0.2500 0.5520 0.5520 rg 999.00 46.00 2.60 10.00 re f
+0.2500 0.5520 0.5520 rg 1001.50 46.00 2.60 10.00 re f
+0.2500 0.5520 0.5520 rg 1004.00 46.00 2.60 10.00 re f
+0.2500 0.5520 0.5520 rg 1006.50 46.00 2.60 10.00 re f
+0.2500 0.5520 0.5520 rg 1009.00 46.00 2.60 10.00 re f
+0.2500 0.5520 0.5520 rg 1011.50 46.00 2.60 10.00 re f
+0.1500 0.4760 0.4860 rg 1014.00 46.00 2.60 10.00 re f
+0.1500 0.4760 0.4860 rg 1016.50 46.00 2.60 10.00 re f
+0.1500 0.4760 0.4860 rg 1019.00 46.00 2.60 10.00 re f
+0.1500 0.4760 0.4860 rg 1021.50 46.00 2.60 10.00 re f
+0.1500 0.4760 0.4860 rg 1024.00 46.00 2.60 10.00 re f
+0.1500 0.4760 0.4860 rg 1026.50 46.00 2.60 10.00 re f
+0.1500 0.4760 0.4860 rg 1029.00 46.00 2.60 10.00 re f
+0.1500 0.4760 0.4860 rg 1031.50 46.00 2.60 10.00 re f
+BT /F1 7.00 Tf 0.2500 0.2600 0.2700 rg 1 0 0 1 834.00 34.00 Tm (rank 11) Tj ET
+BT /F1 7.00 Tf 0.2500 0.2600 0.2700 rg 1 0 0 1 1013.84 34.00 Tm (rank 1) Tj ET
+endstream
+endobj
+xref
+0 7
+0000000000 65535 f 
+0000000015 00000 n 
+0000000064 00000 n 
+0000000121 00000 n 
+0000000258 00000 n 
+0000000328 00000 n 
+0000000403 00000 n 
+trailer
+<< /Size 7 /Root 1 0 R >>
+startxref
+23255
+%%EOF

paper_outputs/tables/tab_app_analog_rank_depth.tex

@@ -0,0 +1,24 @@
+\begin{table*}[t]
+\centering
+\scriptsize
+\setlength{\tabcolsep}{3pt}
+\caption{For fixed retrieval \(\mathcal{T}\) and \(\Omega\), this table reports nDCG@5, best log gap, and rank \(\rho\) in addition to the main nDCG@10/log-error metrics. Cells report mean with small std.}
+\label{tab:app_analog_rank_depth}
+\begin{tabular}{lccc}
+\toprule
+Backbone & nDCG@5 & best log gap & rank $\rho$ \\
+\midrule
+FireWx-FM ref. & \ms{0.5175}{0.0445} & \ms{0.1868}{0.0285} & \ms{0.6019}{0.1460} \\
+Prithvi-WxC & \ms{0.3591}{0.0107} & \ms{0.2151}{0.0594} & \ms{0.1514}{0.1489} \\
+Aurora & \ms{0.4423}{0.0210} & \ms{0.1551}{0.0437} & \ms{0.2162}{0.1856} \\
+ClimaX & \ms{0.4151}{0.0293} & \ms{0.2129}{0.0653} & \ms{0.1587}{0.2831} \\
+StormCast & \ms{0.3960}{0.0240} & \ms{0.1714}{0.0310} & \ms{0.1258}{0.1625} \\
+DLWP & \ms{0.3795}{0.0274} & \ms{0.1944}{0.0807} & \ms{-0.3865}{0.2802} \\
+FCN & \ms{0.4250}{0.0112} & \ms{0.1856}{0.0846} & \ms{-0.1357}{0.2571} \\
+FengWu & \ms{0.4228}{0.0310} & \ms{0.1870}{0.0858} & \ms{-0.1926}{0.2194} \\
+FuXi & \ms{0.4544}{0.0356} & \ms{0.2171}{0.0806} & \ms{-0.1367}{0.2885} \\
+Pangu-Weather & \ms{0.3988}{0.0506} & \ms{0.1901}{0.0838} & \ms{-0.1970}{0.2216} \\
+AlphaEarth & \ms{0.5276}{0.0531} & \ms{0.1782}{0.0454} & \ms{0.4639}{0.2802} \\
+\bottomrule
+\end{tabular}
+\end{table*}

paper_outputs/tables/tab_app_burned_area_median_acre.tex

@@ -0,0 +1,24 @@
+\begin{table*}[t]
+\centering
+\scriptsize
+\setlength{\tabcolsep}{3pt}
+\caption{For fixed final-area \(\mathcal{T}\) and \(\Omega\), this table reports median log error and acre-scale errors in addition to the main log-RMSE/log-MAE/Spearman metrics. Cells report mean with small std.}
+\label{tab:app_burned_area_median_acre}
+\begin{tabular}{lccc}
+\toprule
+Backbone & log median AE & acre median AE & acre MAPE \\
+\midrule
+FireWx-FM ref. & \ms{1.0235}{0.0982} & \ms{4504.0692}{459.0483} & \ms{1.4525}{0.0254} \\
+Prithvi-WxC & \ms{1.2184}{0.2107} & \ms{5375.8770}{788.7906} & \ms{1.9517}{0.2875} \\
+Aurora & \ms{1.4547}{0.0301} & \ms{9904.9483}{457.4260} & \ms{6.8728}{3.0026} \\
+ClimaX & \ms{1.6841}{0.1818} & \ms{18130.4820}{3248.3873} & \ms{8.2373}{2.8540} \\
+StormCast & \ms{1.4522}{0.1519} & \ms{11155.7881}{2020.8656} & \ms{4.6142}{1.1500} \\
+DLWP & \ms{1.0952}{0.1306} & \ms{4406.9315}{303.0944} & \ms{1.7357}{0.3625} \\
+FCN & \ms{1.1688}{0.1139} & \ms{5166.9993}{213.0333} & \ms{2.0800}{0.4004} \\
+FengWu & \ms{1.1589}{0.1772} & \ms{5137.2822}{628.7543} & \ms{2.0944}{0.4545} \\
+FuXi & \ms{1.1855}{0.0612} & \ms{5697.7117}{796.8785} & \ms{2.4411}{0.5567} \\
+Pangu-Weather & \ms{1.1221}{0.1470} & \ms{5092.3621}{483.8243} & \ms{1.9571}{0.3113} \\
+AlphaEarth & \ms{1.7459}{0.6057} & \ms{15110.7573}{7106.3417} & \ms{9.7398}{2.7425} \\
+\bottomrule
+\end{tabular}
+\end{table*}

paper_outputs/tables/tab_app_contract_params_full.tex

@@ -0,0 +1,22 @@
+\begin{table}[h]
+\centering
+\scriptsize
+\setlength{\tabcolsep}{3.5pt}
+\renewcommand{\arraystretch}{1.2}
+\caption{Fixed scoring values used by each task-form contract.}
+\label{tab:app_contract_params_full}
+\begin{adjustbox}{max width=\textwidth}
+\begin{tabular}{llll}
+\toprule
+\textbf{\(\mathcal{T}\)} & \textbf{Scoring} & \textbf{Validation} & \textbf{\(\Omega\)} \\
+\midrule
+Occupancy & \(k=8,\Delta t=3\); exact/tol./union \(F_1\) & val. strict \(F_1\) & global; top-5/10/20\% fire-prone \\
+Fire spread & \(k=4,\Delta t=0\); exact/spatial \(F_1\), AP & val. spatial \(F_1\) & spread-region cells \\
+Final burned area & log-RMSE, log-MAE, Spearman \(\rho\) & val. log-RMSE & test events \\
+Analog retrieval & nDCG@10; retrieved-event log error & val. nDCG@10 & test events \\
+Smoke PM\(_{2.5}\) & RMSE, MAE, Pearson \(r\); exceedance 35 & val. RMSE & test stations \\
+Extreme heat & RMSE-C, MAE-C, exceedance \(F_1\) & val. threshold 27/30/33\(^{\circ}\)C & heat-region stations \\
+\bottomrule
+\end{tabular}
+\end{adjustbox}
+\end{table}

paper_outputs/tables/tab_app_head_architectures.tex

@@ -0,0 +1,36 @@
+\begin{table}[h]
+\centering
+\small
+\setlength{\tabcolsep}{5pt}
+\renewcommand{\arraystretch}{1.3}
+\caption{Lightweight head architectures used in the fixed-contract transfer comparisons.
+All heads are trained from random initialisation on the frozen backbone features.
+Parameter counts are approximate and depend on the feature dimensionality of each backbone.}
+\label{tab:app_head_architectures}
+\begin{tabular}{p{0.15\textwidth}p{0.30\textwidth}p{0.12\textwidth}p{0.33\textwidth}}
+\toprule
+\textbf{$\mathcal{A}$ head} & \textbf{Architecture} & \textbf{Approx.\ params} & \textbf{Notes} \\
+\midrule
+Constant prior &
+  Outputs a fixed bias vector, ignoring input features. &
+  Output dimension only &
+  Provides a degenerate baseline; selected when backbone features carry no useful signal. \\
+Linear probe &
+  Single linear layer mapping backbone features to output. No nonlinearity. &
+  $d\times c + c$ &
+  Standard frozen-representation baseline. \\
+Pixel MLP &
+  Two-layer MLP applied independently per spatial unit. &
+  $d\times h + h\times c$ &
+  Captures per-pixel nonlinearity; ignores spatial context. \\
+Shallow adapter &
+  Two-layer MLP with a spatial context window; uses $3\times3$ convolution before the linear output. &
+  $9dh + hc$ &
+  Balances local spatial context with parameter efficiency. \\
+Wide adapter &
+  Shallow adapter with wider hidden dimension. &
+  $9dH + Hc$ &
+  Higher capacity variant; can overfit on small fire-event sets. \\
+\bottomrule
+\end{tabular}
+\end{table}

paper_outputs/tables/tab_app_heat_event_pr.tex

@@ -0,0 +1,24 @@
+\begin{table*}[t]
+\centering
+\scriptsize
+\setlength{\tabcolsep}{3pt}
+\caption{For fixed heat \(\mathcal{T}\) and heat-region \(\Omega\), this table reports precision and recall for the exceedance label used by the main \(F_1\). Cells report mean with small std.}
+\label{tab:app_heat_event_pr}
+\begin{tabular}{lcc}
+\toprule
+Backbone & precision & recall \\
+\midrule
+FireWx-FM ref. & \ms{0.9767}{0.0117} & \ms{0.9330}{0.0299} \\
+Prithvi-WxC & \ms{0.8260}{0.0030} & \ms{0.9173}{0.0033} \\
+Aurora & \ms{0.5920}{0.0347} & \ms{0.0517}{0.0020} \\
+ClimaX & \ms{0.7397}{0.0099} & \ms{0.7994}{0.0051} \\
+StormCast & \ms{0.8840}{0.0237} & \ms{0.9320}{0.0165} \\
+DLWP & \ms{0.9429}{0.0085} & \ms{0.8899}{0.0167} \\
+FCN & \ms{0.9408}{0.0097} & \ms{0.9111}{0.0127} \\
+FengWu & \ms{0.3808}{0.2719} & \ms{0.0266}{0.0267} \\
+FuXi & \ms{0.3262}{0.1262} & \ms{0.1810}{0.0481} \\
+Pangu-Weather & \ms{0.1159}{0.0743} & \ms{0.0112}{0.0032} \\
+AlphaEarth & \ms{0.9824}{0.0040} & \ms{0.9278}{0.0178} \\
+\bottomrule
+\end{tabular}
+\end{table*}

paper_outputs/tables/tab_app_matching_rule_params.tex

@@ -0,0 +1,17 @@
+\begin{table}[h]
+\centering
+\small
+\setlength{\tabcolsep}{10pt}
+\renewcommand{\arraystretch}{1.2}
+\caption{Matching-rule values used in the evaluation contracts.}
+\label{tab:app_matching_rule_params}
+\begin{tabular}{lll}
+\toprule
+\textbf{Parameter} & \textbf{Occupancy} & \textbf{Fire spread} \\
+\midrule
+\(k\) & 8 cells & 4 cells \\
+\(\Delta t\) & 3 for union; 0 spatial-only & 0 \\
+\(\tau\) & val. strict \(F_1\) & val. spatial \(F_1\) \\
+\bottomrule
+\end{tabular}
+\end{table}

paper_outputs/tables/tab_app_occupancy_ppr_scope.tex

@@ -0,0 +1,27 @@
+\begin{table*}[t]
+\centering
+\small
+\setlength{\tabcolsep}{4pt}
+\renewcommand{\arraystretch}{1.18}
+\caption{For fixed occupancy \(\mathcal{T}\), this table reports predicted-positive rate.
+Values are percentages under the same validation-selected strict threshold.
+Scopes \(\Omega\) are fixed before test scoring; cells report five-seed mean with std in small type.}
+\label{tab:app_occupancy_ppr_scope}
+\begin{tabular}{lcccc}
+\toprule
+\textbf{Backbone} & \textbf{\(\Omega=\)global} & \textbf{\(\Omega=\)top 5\%} & \textbf{\(\Omega=\)top 10\%} & \textbf{\(\Omega=\)top 20\%} \\
+\midrule
+FireWx-FM ref. & \ms{1.6808}{0.3684} & \ms{3.0619}{1.0925} & \ms{1.5310}{0.5463} & \ms{0.7655}{0.2732} \\
+Prithvi-WxC & \ms{61.9711}{30.9101} & \ms{57.4117}{47.8987} & \ms{58.4565}{51.0897} & \ms{58.9788}{52.6991} \\
+Aurora & \ms{55.5849}{19.7524} & \ms{57.2238}{35.3400} & \ms{68.7942}{37.6958} & \ms{67.2891}{38.3991} \\
+ClimaX & \ms{5.6763}{3.9261} & \ms{24.0091}{9.2816} & \ms{11.8450}{4.5067} & \ms{5.7442}{4.1341} \\
+StormCast & \ms{60.6507}{17.4895} & \ms{57.6017}{35.2921} & \ms{68.0766}{37.3899} & \ms{67.8397}{39.2410} \\
+DLWP & \ms{4.3221}{1.5619} & \ms{9.4001}{5.0807} & \ms{4.9700}{3.6849} & \ms{1.9198}{1.4678} \\
+FCN & \ms{1.5202}{1.3446} & \ms{4.7856}{2.9409} & \ms{2.7257}{1.6353} & \ms{0.8368}{0.2358} \\
+FengWu & \ms{0.4277}{0.4830} & \ms{0.6004}{0.3041} & \ms{0.2609}{0.1935} & \ms{0.1501}{0.1206} \\
+FuXi & \ms{0.4505}{0.2773} & \ms{2.9315}{2.6392} & \ms{0.5197}{0.6074} & \ms{0.3621}{0.4346} \\
+Pangu-Weather & \ms{1.0801}{1.1308} & \ms{2.0549}{2.1893} & \ms{1.4029}{1.4739} & \ms{1.0103}{1.1084} \\
+AlphaEarth & \ms{0.0691}{0.0499} & \ms{0.2826}{0.1497} & \ms{0.1524}{0.0770} & \ms{0.0656}{0.0414} \\
+\bottomrule
+\end{tabular}
+\end{table*}

paper_outputs/tables/tab_app_scope_params.tex

@@ -0,0 +1,19 @@
+\begin{table}[h]
+\centering
+\small
+\setlength{\tabcolsep}{8pt}
+\renewcommand{\arraystretch}{1.2}
+\caption{Scope values used in the evaluation contracts.}
+\label{tab:app_scope_params}
+\begin{tabular}{lcc}
+\toprule
+\textbf{\(\Omega\)} & \textbf{Definition} & \textbf{Units} \\
+\midrule
+Global & full domain & 8,085,000 test cells \\
+Fire-prone top-5\% & top 5\% by training-period fire frequency & 404,280 test cells \\
+Fire-prone top-10\% & top 10\% by training-period fire frequency & 808,560 test cells \\
+Fire-prone top-20\% & top 20\% by training-period fire frequency & 1,617,000 test cells \\
+Spread region & union of \(\widehat{B}\) and \(B\) & event-specific cells \\
+\bottomrule
+\end{tabular}
+\end{table}

paper_outputs/tables/tab_app_seed_robustness.tex

@@ -0,0 +1,36 @@
+\begin{table}[h]
+\centering
+\small
+\setlength{\tabcolsep}{5pt}
+\renewcommand{\arraystretch}{1.2}
+\caption{Seed summaries for stochastic checks. Values report mean with small std over completed seeds.}
+\label{tab:app_seed_robustness}
+\begin{adjustbox}{max width=\textwidth}
+\begin{tabular}{p{0.28\textwidth}cllp{0.18\textwidth}}
+\toprule
+\textbf{\(\mathcal{T}\) check} & \textbf{Seeds} & \textbf{Primary value} & \textbf{Other value(s)} & \textbf{Reading} \\
+\midrule
+Final burned area &
+5 & log-RMSE \ms{1.1657}{0.0126} &
+log-MAE \ms{1.0423}{0.0081}; Spear.\ \ms{0.6298}{0.0338} &
+stable across seeds \\
+Smoke PM\(_{2.5}\) &
+5 & RMSE \ms{4.4646}{0.0060} &
+MAE \ms{2.4108}{0.0016}; \(r\) \ms{0.6368}{0.0013} &
+stable at table precision \\
+Extreme heat &
+5 & RMSE-C \ms{0.2179}{0.0043} &
+MAE-C \ms{0.1787}{0.0018}; exceed.\ \(F_1\) \ms{0.9541}{0.0164} &
+stable across seeds \\
+Fire spread &
+5 & exact \(F_1\) \ms{37.6700}{0.9800} &
+spatial \(F_1\) \ms{80.9700}{2.0200}; AP \ms{30.0900}{1.2500} &
+stable across seeds \\
+Aurora paired-head check &
+5 & fire-prone score diff.\ \ms{6.3500}{13.2800} &
+PR-AUC and union choices differ in 2/5 seeds &
+variable across seeds \\
+\bottomrule
+\end{tabular}
+\end{adjustbox}
+\end{table}

paper_outputs/tables/tab_app_smoke_high_event.tex

@@ -0,0 +1,24 @@
+\begin{table*}[t]
+\centering
+\scriptsize
+\setlength{\tabcolsep}{3pt}
+\caption{For fixed smoke \(\mathcal{T}\) and station \(\Omega\), this table reports RMSE, MAE, and 90th-percentile absolute error on test rows with observed PM$_{2.5}\ge35$; std uses a row bootstrap over those rows. Cells report mean with small std.}
+\label{tab:app_smoke_high_event}
+\begin{tabular}{lccc}
+\toprule
+Backbone & high-smoke RMSE & high-smoke MAE & high-smoke 90th AE \\
+\midrule
+FireWx-FM ref. & \ms{47.4870}{0.6346} & \ms{34.3954}{0.7654} & \ms{65.6213}{3.8778} \\
+Prithvi-WxC & \ms{57.2224}{1.7268} & \ms{47.3871}{0.3153} & \ms{74.9666}{3.2381} \\
+Aurora & \ms{57.2752}{1.7248} & \ms{47.4368}{0.3149} & \ms{75.0755}{3.1074} \\
+ClimaX & \ms{57.2828}{1.7239} & \ms{47.4407}{0.3140} & \ms{75.1012}{3.0777} \\
+StormCast & \ms{56.6512}{1.7517} & \ms{46.7914}{0.3281} & \ms{74.0794}{3.4707} \\
+DLWP & \ms{57.0075}{1.7359} & \ms{47.1971}{0.3198} & \ms{74.4936}{3.3826} \\
+FCN & \ms{57.0582}{1.7339} & \ms{47.2401}{0.3187} & \ms{74.6431}{3.1982} \\
+FengWu & \ms{57.0158}{1.7357} & \ms{47.1957}{0.3194} & \ms{74.5652}{3.2871} \\
+FuXi & \ms{56.9622}{1.7371} & \ms{47.1508}{0.3201} & \ms{74.3278}{3.4435} \\
+Pangu-Weather & \ms{57.1282}{1.7307} & \ms{47.3050}{0.3170} & \ms{74.6830}{3.2375} \\
+AlphaEarth & \ms{48.0665}{0.7904} & \ms{35.6088}{0.7341} & \ms{66.7613}{3.9235} \\
+\bottomrule
+\end{tabular}
+\end{table*}

paper_outputs/tables/tab_app_spread_ap_by_scope.tex

@@ -0,0 +1,24 @@
+\begin{table*}[t]
+\centering
+\scriptsize
+\setlength{\tabcolsep}{3pt}
+\caption{For fixed spread \(\mathcal{T}\) and strict \(\Lambda\), this table reports AP under three \(\Omega\) scopes: full test, top-5\% train-fire area, and top-10\% train-fire area. Values are percentages; cells report mean with small std.}
+\label{tab:app_spread_ap_by_scope}
+\begin{tabular}{lccc}
+\toprule
+Backbone & full \(\Omega\) AP & top-5\% \(\Omega\) AP & top-10\% \(\Omega\) AP \\
+\midrule
+FireWx-FM ref. & \ms{30.0197}{1.5651} & \ms{40.7452}{2.0542} & \ms{37.4096}{1.8731} \\
+Prithvi-WxC & \ms{4.8319}{0.1731} & \ms{12.6086}{0.4468} & \ms{8.7051}{0.1889} \\
+Aurora & \ms{17.7723}{0.4293} & \ms{30.3106}{0.9404} & \ms{26.4732}{0.6932} \\
+ClimaX & \ms{11.1726}{0.2337} & \ms{25.7871}{1.2896} & \ms{19.9977}{1.2217} \\
+StormCast & \ms{8.1147}{1.1569} & \ms{18.5461}{1.1727} & \ms{14.1286}{1.2956} \\
+DLWP & \ms{9.2142}{2.6587} & \ms{19.3346}{2.3922} & \ms{14.9788}{2.6696} \\
+FCN & \ms{6.6774}{1.3001} & \ms{16.7396}{3.2955} & \ms{11.9308}{2.3881} \\
+FengWu & \ms{11.0046}{2.7092} & \ms{21.1506}{1.2163} & \ms{17.0113}{1.5778} \\
+FuXi & \ms{13.5507}{0.3840} & \ms{22.5434}{0.4100} & \ms{19.1964}{0.3943} \\
+Pangu-Weather & \ms{10.6250}{1.4643} & \ms{19.8294}{1.3044} & \ms{15.8013}{1.1602} \\
+AlphaEarth & \ms{12.2847}{1.3562} & \ms{22.8692}{0.4915} & \ms{18.2992}{1.2110} \\
+\bottomrule
+\end{tabular}
+\end{table*}

paper_outputs/tables/tab_appendix_selection_regret_tolerance.tex

@@ -0,0 +1,37 @@
+\begin{table*}[!t]
+    \centering
+    \scriptsize
+    \setlength{\tabcolsep}{4pt}
+    \caption{Selection-regret values under exact, tolerated, and union matching. Values are percentage-point regret from selecting \(h_R\) by PR-AUC instead of \(h_D\) by the decision metric. Rows report mean with small std over five seeds; \(0.0000\) denotes exact zero regret.}
+    \label{tab:appendix_selection_regret_tolerance}
+    \begin{adjustbox}{max width=\textwidth}
+    \begin{tabular}{llccc}
+        \toprule
+        \textbf{Feature} & \textbf{\(\Omega\)} & \textbf{Exact regret} & \textbf{Tolerated regret} & \textbf{Union regret} \\
+        \midrule
+        FireWx-FM ref. & global & 0.0000 & \ms{8.7830}{9.6705} & \ms{8.7830}{9.6705} \\
+        FireWx-FM ref. & fire-prone & 0.0000 & \ms{3.4027}{3.2045} & \ms{3.4027}{3.2045} \\
+        Prithvi-WxC & global & 0.0000 & 0.0000 & 0.0000 \\
+        Prithvi-WxC & fire-prone & 0.0000 & 0.0000 & 0.0000 \\
+        Aurora & global & \ms{0.0200}{0.0267} & \ms{9.8520}{12.9878} & \ms{9.8520}{12.9878} \\
+        Aurora & fire-prone & \ms{0.8203}{1.8341} & \ms{14.3919}{32.1219} & \ms{14.3919}{32.1219} \\
+        ClimaX & global & \ms{0.0003}{0.0004} & \ms{0.1296}{0.1775} & \ms{0.1296}{0.1775} \\
+        ClimaX & fire-prone & 0.0000 & 0.0000 & 0.0000 \\
+        StormCast & global & 0.0000 & 0.0000 & 0.0000 \\
+        StormCast & fire-prone & 0.0000 & 0.0000 & 0.0000 \\
+        DLWP & global & 0.0000 & 0.0000 & 0.0000 \\
+        DLWP & fire-prone & \ms{0.0770}{0.1100} & \ms{4.3266}{4.3323} & \ms{4.3266}{4.3323} \\
+        FCN & global & 0.0000 & 0.0000 & 0.0000 \\
+        FCN & fire-prone & \ms{0.0006}{0.0013} & \ms{1.1680}{1.9872} & \ms{1.1680}{1.9872} \\
+        FengWu & global & 0.0000 & 0.0000 & 0.0000 \\
+        FengWu & fire-prone & \ms{0.0691}{0.1191} & \ms{0.5222}{0.6239} & \ms{0.5222}{0.6239} \\
+        FuXi & global & 0.0000 & 0.0000 & 0.0000 \\
+        FuXi & fire-prone & 0.0000 & \ms{0.1084}{0.1729} & \ms{0.1084}{0.1729} \\
+        Pangu-Weather & global & 0.0000 & 0.0000 & 0.0000 \\
+        Pangu-Weather & fire-prone & \ms{0.0728}{0.1179} & \ms{0.1849}{0.3263} & \ms{0.1849}{0.3263} \\
+        AlphaEarth & global & 0.0000 & \ms{17.2217}{8.8492} & \ms{17.2217}{8.8492} \\
+        AlphaEarth & fire-prone & 0.0000 & \ms{3.8804}{5.9483} & \ms{3.8804}{5.9483} \\
+        \bottomrule
+    \end{tabular}
+    \end{adjustbox}
+\end{table*}