README.md

# AirTrackLM

A decoder-only transformer for ADS-B air track next-state prediction, adapted from the LLM4STP architecture.

## Architecture

- **Model**: Custom ~7M parameter decoder-only transformer
- **4 Embedding Types**: Geohash (40-bit binary, 3D), Kinematic Features (COG/SOG/ROT/AltRate), Temporal (sub-second sinusoidal), Uncertainty (4 methods + learned heteroscedastic)
- **Pretraining**: Next-state prediction (predict all features at t+1 from sequence up to t)
- **Coordinate System**: ENU (East-North-Up) with 3-point central derivative for velocity computation

## Uncertainty Methods

1. **Kinematic Variance** — Sliding-window variance of COG/SOG/ROT/alt_rate
2. **Prediction Residual** — Deviation from constant-velocity prediction model
3. **Spatial Density** — Data coverage proxy (fewer nearby training points = higher uncertainty)
4. **Flight Phase Entropy** — Entropy of phase classification in a window (mixed phases = uncertain)
5. **Learned Heteroscedastic** — Model predicts its own log-variance per output head (aleatoric)
6. **MC-Dropout** — Monte Carlo dropout at inference for epistemic uncertainty

## Features

- **Inputs**: Raw ADS-B (lat, lon, alt, timestamp)
- **Derived**: COG, SOG, ROT, altitude rate via 3-point central derivative on ENU positions
- **Geohash**: 40-bit binary encoding per axis (E, N, U) = 120-bit 3D position token
- **Temporal**: Sinusoidal second-of-day (sub-second resolution) + calendar embeddings + Δt encoding
- **Output Heads**: Binary geohash prediction, continuous Δ-ENU regression, COG/SOG/ROT/AltRate bin classification

## Data

Training data from the `traffic` Python library (real ADS-B surveillance data).

## Files

- `model.py` — Full model architecture (AirTrackLM, embeddings, loss functions)
- `data_pipeline.py` — ENU conversion, 3-point derivatives, geohash encoding, dataset
- `uncertainty.py` — 6 uncertainty quantification methods
- `train.py` — Training utilities
- `train_full.py` — Full GPU training script with Hub push
- `ARCHITECTURE.md` — Detailed architecture document

## Based On

- **LLM4STP** (Joker-hang/LLM4STP) — Binary geohash encoding, GPT-2 backbone concept
- **FTP-LLM** (arXiv:2501.17459) — LLM for flight trajectory prediction
- **H3-CLM** (arXiv:2405.09596) — Hexagonal geohash + causal LM for maritime trajectories
- **GeoFormer** (arXiv:2311.05092) — GPT-style geospatial tokenization