File size: 4,662 Bytes
6a82282 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 | # Phase 17 — Riprap end-to-end integration of all NYC AMD fine-tunes
## Goal
Wire the four AMD-trained NYC fine-tunes (Phase 2 LULC + Phase 12 TiM
+ Phase 13 Buildings + Phase 14 Prithvi-NYC + Phase 16 TTM-NYC) into
Riprap's existing FSM as production specialists. The hackathon
submission then demonstrates the full briefing flow with all six
foundation models running locally on AMD.
This experiment is the *deployment story* — taking the published HF
checkpoints and showing them serve real Riprap queries, not just the
isolated Gradio demo.
## Scope
NOT a fine-tune. Pure integration work — porting code from
`experiments/05_terramind_nyc_finetune/publish/space/local_app.py`
into Riprap's `app/` modules with proper FSM hooks.
## What gets touched
### New specialists
`app/context/sentinel_live.py` — `fetch_recent_chips(lat, lon)` from
Earth Search (S2 + DEM) + PC (S1 RTC), with multi-source fallback and
per-MGRS-cell cache. From experiments/11_live_sentinel_fetch.
`app/context/terramind_nyc.py` — wraps the AMD-trained NYC LULC + TiM
+ Buildings checkpoints. Returns a single dict with class fractions,
imperviousness, building density, and freshness disclosures. Replaces
`app/context/terramind_synthesis.py`'s zero-shot path while keeping
the same output schema for backward compatibility.
### Modified specialists
`app/flood_layers/prithvi_water.py` — gains a `prithvi_live` mode that
uses the Phase 14 NYC pluvial fine-tune on a freshly-fetched S2 chip,
in addition to the existing point-in-polygon test against
`prithvi_ida_2021.geojson`.
`app/live/ttm_forecast.py` — model-id swap from zero-shot TTM r2 to
the Phase 16 NYC-fine-tuned variant.
### FSM updates
`app/fsm.py:step_terramind` already exists; just point it at
`terramind_nyc` instead of `terramind_synthesis`. No new action
needed — the dict shape matches.
`app/reconcile.py:579-608` (the TerraMind doc-message block) updates
the framing from "tentative ESRI labels" to "WorldCover macro-classes
(confirmed) + AMD-trained NYC fine-tune". Drops the "tentative"
disclaimer since labels are now real ground truth.
### Tests
`tests/test_terramind_nyc.py` — three NYC reference points (Manhattan
center, Brighton Beach, Bronx Zoo). Expected class distributions
(Manhattan should be ≥70% developed; Brighton Beach should have ≥10%
water; Bronx Zoo should have ≥15% forest). Mock the live Sentinel
fetch in CI.
## Success criteria
- A real Riprap query for "Empire State Building" returns a briefing
whose `terramind_nyc` doc body cites:
- imperviousness (= developed%)
- green-space %
- water %
- building-density % (if Phase 13/15 lands)
- flood-prediction % (if Phase 14 lands)
- S2 acquisition age in days
- S1 acquisition age in days
- The reconciler's Mellea grounding pass succeeds on those numbers.
- The full flow runs in < 12 s end-to-end, including the live Sentinel
fetch (~5 s) plus all 14 specialists (~6 s) plus reconciliation.
## Plan
1. Scaffold (this file).
2. Port `local_app.py` chip-fetch + inference logic into
`app/context/terramind_nyc.py`. Mirror the dict shape from
`app/context/terramind_synthesis.py:fetch()`.
3. Port the live Sentinel multi-source fetch from Phase 11 into
`app/context/sentinel_live.py`.
4. Wire `step_terramind` in `app/fsm.py` to use the new module.
5. Update `app/reconcile.py` framing (drop "tentative" disclaimer
when `label_schema` says "confirmed").
6. Add unit tests for the new modules in `tests/`.
7. Run a smoke briefing for "2940 Brighton 3rd St, Brooklyn" (the
existing worked example in ARCHITECTURE.md §3.2). Confirm the new
sentence appears in the briefing.
8. Update ARCHITECTURE.md §7 to add TerraMind-NYC + the AMD-fine-tune
provenance line.
## What this enables for the submission
A judge clicking through the Riprap UI sees:
- Type address → live trace shows `step_terramind_nyc` running with the
AMD-fine-tuned ckpt loading
- The briefing paragraph cites imperviousness, green-space %, building
density — with `[terramind_nyc]` citations
- The bottom-of-briefing "what models powered this" section lists six
foundation models, one of them flagged AMD-trained
- The MapLibre overlay shows the predicted LULC polygons
That's the demo for the video.
## Risk
Low. All the heavy ML is done by this phase; this is just plumbing.
Estimated half-day for a clean port.
## Reproduction (planned)
```bash
# After fine-tunes published to HF:
git pull origin main # latest Riprap with the integration
RIPRAP_TERRAMIND_VARIANT=nyc \
RIPRAP_PRITHVI_VARIANT=nyc \
RIPRAP_TTM_VARIANT=nyc \
.venv/bin/uvicorn web.main:app --port 7860
```
|