jhanna commited on
Commit
b2cd127
·
verified ·
1 Parent(s): cbd4731

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +97 -1
README.md CHANGED
@@ -7,4 +7,100 @@ tags:
7
  - earth-observation
8
  - remote-sensing
9
  - foundation-model
10
- ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
7
  - earth-observation
8
  - remote-sensing
9
  - foundation-model
10
+ ---
11
+
12
+ # GeoSANE: Learning Geospatial Representations From Models, Not Data
13
+
14
+ GeoSANE is a geospatial model foundry that operates directly in model-weight space. Instead of training a downstream model from scratch, GeoSANE learns a shared latent representation over a population of pretrained remote sensing models and uses that representation to generate new model candidates for a target architecture. These generated models can then be evaluated and fine-tuned on downstream tasks.
15
+
16
+ This repository provides a demo project for running GeoSANE on a downstream remote sensing benchmark. The included notebook walks through the full evaluation pipeline for TIMM backbones on the Sen1Floods11 segmentation task: preparing the downstream dataset, loading a trained GeoSANE checkpoint, generating model candidates, fine-tuning and saving the resulting checkpoint.
17
+
18
+ ## Project Contents
19
+
20
+ - `geosane-demo.ipynb`: end-to-end demo notebook
21
+ - `shrp/`: the core SHRP library used by GeoSANE for weight tokenization, latent sampling, model reconstruction, evaluation, and fine-tuning
22
+ - `downstream_datasets/`: downstream benchmark loaders, including Sen1Floods11, SpaceNet, EuroSAT, DIOR, fMoW, and others
23
+ - `requirements.txt`: broad dependency list for setting up an environment
24
+ - `requirements-lock.txt`: pinned versions from a working environment
25
+ - `anchor_tokenized/`: cached tokenized anchor-model datasets generated during evaluation
26
+ - `checkpoints/`: fine-tuned model checkpoints written during notebook runs
27
+
28
+ ## Quick Start
29
+
30
+ Create an environment and install dependencies:
31
+
32
+ ```bash
33
+ python -m venv .venv
34
+ source .venv/bin/activate
35
+ pip install --upgrade pip
36
+ pip install -r requirements.txt
37
+ ```
38
+
39
+ If you want to reproduce the exact environment used for this demo as closely as possible, use:
40
+
41
+ ```bash
42
+ pip install -r requirements-lock.txt
43
+ ```
44
+
45
+ Note: the lock file includes environment-specific PyTorch builds. You may need to install a compatible `torch` / `torchvision` pair first and then install the remaining dependencies.
46
+
47
+ ## Downstream Task
48
+
49
+ [A Jupyter notebook](https://github.com/HSG-AIML/GeoSANE/blob/main/geosane-demo.ipynb) is provided in [GeoSANE GitHub page](https://github.com/HSG-AIML/GeoSANE) to show in detail how to use pretrained model weights. The details of GeoSANE are described in our paper, available on [arXiv](https://arxiv.org/abs/2603.23408).
50
+ The demo notebook is currently configured for:
51
+
52
+ - task: segmentation
53
+ - downstream dataset: Sen1Floods11
54
+ - generated backbone prompt: TIMM backbones such as `swin_s3_base_224.ms_in1k`
55
+
56
+ The downstream dataset file created by the notebook is:
57
+
58
+ - `data/sen1flood11_preprocessed.pt`
59
+
60
+ This file is constructed from:
61
+
62
+ ```python
63
+ trainset = Sen1Floods11HandLabeledDataset(split="train", resize_to=(224, 224))
64
+ testset = Sen1Floods11HandLabeledDataset(split="val", resize_to=(224, 224))
65
+ ```
66
+
67
+
68
+ ## GeoSANE Checkpoints
69
+
70
+ The notebook expects a trained GeoSANE run directory. The configured `model_path` must point to a directory that contains at least:
71
+
72
+ - `params.json`
73
+ - `checkpoint_000300/state.pt`
74
+
75
+ In the notebook, this is configured through:
76
+
77
+ ```python
78
+ EXPERIMENT = TimmExperimentConfig(
79
+ model_path=Path("/path/to/your/geosane_run"),
80
+ checkpoint_rel_path=Path("checkpoint_000300/state.pt"),
81
+ ...
82
+ )
83
+ ```
84
+
85
+ After downloading or copying the checkpoint directory, update `model_path` in `geosane-demo.ipynb` to your local path.
86
+
87
+ ## Outputs
88
+
89
+ Running the notebook produces several artifacts:
90
+
91
+ - `data/sen1flood11_preprocessed.pt`: preprocessed downstream dataset
92
+ - `anchor_tokenized/`: tokenized anchor-model datasets used during sampling and evaluation
93
+ - `checkpoints/`: fine-tuned model checkpoints saved during evaluation
94
+ - `model_path/notebook_eval_results/`: JSON evaluation outputs
95
+
96
+
97
+ ## Acknowledging this work
98
+
99
+ If you would like to cite our work, please use the following reference:
100
+
101
+ * Hanna, Joelle, Damian Falk, Stella X. Yu and Damian Borth. *GeoSANE: Learning Geospatial Representations from Models, Not Data.*, Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 2026.
102
+
103
+
104
+ ## Code
105
+ This repository incorporates code from the following source:
106
+ * [SANE](https://github.com/HSG-AIML/SANE)