Upload README.md with huggingface_hub
Browse files
README.md
ADDED
|
@@ -0,0 +1,93 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: apache-2.0
|
| 3 |
+
tags:
|
| 4 |
+
- trajectory-prediction
|
| 5 |
+
- aviation
|
| 6 |
+
- adsb
|
| 7 |
+
- time-series
|
| 8 |
+
- llm-reprogramming
|
| 9 |
+
- gpt2
|
| 10 |
+
datasets:
|
| 11 |
+
- petchthwr/ATFMTraj
|
| 12 |
+
pipeline_tag: time-series-forecasting
|
| 13 |
+
---
|
| 14 |
+
|
| 15 |
+
# LLM4AirTrack: LLM-Driven Aircraft Trajectory Prediction
|
| 16 |
+
|
| 17 |
+
Adapts the [LLM4STP](https://github.com/Joker-hang/LLM4STP) framework from maritime AIS to aviation ADS-B.
|
| 18 |
+
Uses a **frozen GPT-2 backbone** with lightweight trainable adapters (~2.4% of params).
|
| 19 |
+
|
| 20 |
+
## Architecture
|
| 21 |
+
|
| 22 |
+
```
|
| 23 |
+
ADS-B Features (9-dim) → RevIN → Patch Tokenizer → Patch Embedder
|
| 24 |
+
→ Cross-Attention Reprogrammer (learned text prototypes)
|
| 25 |
+
→ Prompt-as-Prefix → Frozen GPT-2 Backbone
|
| 26 |
+
→ Trajectory Head (future xyz) + Classification Head (STAR/runway)
|
| 27 |
+
```
|
| 28 |
+
|
| 29 |
+
### Key Components
|
| 30 |
+
1. **9-dim Kinematic Features**: Position (x,y,z ENU) + Direction (ux,uy,uz) + Polar (r, sinθ, cosθ)
|
| 31 |
+
2. **Patch Tokenization**: Overlapping temporal patches (len=8, stride=4)
|
| 32 |
+
3. **Cross-Attention Reprogramming**: 256 learned text prototypes, 8-head attention
|
| 33 |
+
4. **Frozen GPT-2**: 124M params frozen, only ~3.1M trainable
|
| 34 |
+
5. **Dual Heads**: Trajectory prediction (Smooth L1) + Route classification (CE)
|
| 35 |
+
|
| 36 |
+
## Training
|
| 37 |
+
|
| 38 |
+
- **Dataset**: [ATFMTraj](https://huggingface.co/datasets/petchthwr/ATFMTraj) - RKSIa
|
| 39 |
+
- **Source**: OpenSky ADS-B, Incheon International Airport arrivals (2018-2023)
|
| 40 |
+
- **Context**: 60 timesteps (1s intervals)
|
| 41 |
+
- **Prediction**: 30 timesteps ahead
|
| 42 |
+
- **Optimizer**: AdamW, lr=0.0005, cosine annealing
|
| 43 |
+
- **Epochs**: 5
|
| 44 |
+
|
| 45 |
+
## Results
|
| 46 |
+
|
| 47 |
+
| Metric | Value |
|
| 48 |
+
|--------|-------|
|
| 49 |
+
| ADE (normalized) | 0.010258 |
|
| 50 |
+
| Best Epoch | 3 |
|
| 51 |
+
| Route Classification Acc | 0.3649 |
|
| 52 |
+
|
| 53 |
+
## Usage
|
| 54 |
+
|
| 55 |
+
```python
|
| 56 |
+
import torch, json
|
| 57 |
+
from train_full import LLM4AirTrack
|
| 58 |
+
|
| 59 |
+
# Load
|
| 60 |
+
with open("config.json") as f:
|
| 61 |
+
cfg = json.load(f)
|
| 62 |
+
|
| 63 |
+
model = LLM4AirTrack(
|
| 64 |
+
llm_name=cfg["llm_name"],
|
| 65 |
+
context_len=cfg["context_len"],
|
| 66 |
+
pred_len=cfg["pred_len"],
|
| 67 |
+
n_classes=cfg["n_classes"],
|
| 68 |
+
)
|
| 69 |
+
state = torch.load("adapter_weights.pt", map_location="cpu")
|
| 70 |
+
model.load_state_dict(state, strict=False)
|
| 71 |
+
model.eval()
|
| 72 |
+
|
| 73 |
+
# Predict (input: 60 timesteps of 9-dim kinematic features)
|
| 74 |
+
context = torch.randn(1, 60, 9)
|
| 75 |
+
out = model(context)
|
| 76 |
+
future_xyz = out["pred_trajectory"] # (1, 30, 3)
|
| 77 |
+
route_class = out["pred_class"].argmax(-1) # (1,)
|
| 78 |
+
```
|
| 79 |
+
|
| 80 |
+
## Downstream Tasks
|
| 81 |
+
|
| 82 |
+
- **Track Activity Classification**: Route/procedure identification from trajectory embeddings
|
| 83 |
+
- **Anomaly Detection**: Flag deviations from predicted trajectory
|
| 84 |
+
- **Conflict Detection**: Multi-aircraft trajectory forecasting
|
| 85 |
+
- **ETA Prediction**: Time-to-threshold from trajectory state
|
| 86 |
+
|
| 87 |
+
## References
|
| 88 |
+
|
| 89 |
+
- [LLM4STP](https://github.com/Joker-hang/LLM4STP) - Original maritime framework
|
| 90 |
+
- [Time-LLM](https://arxiv.org/abs/2310.01728) - Foundational reprogramming approach
|
| 91 |
+
- [ATFMTraj](https://huggingface.co/datasets/petchthwr/ATFMTraj) - Aviation trajectory dataset
|
| 92 |
+
- [ATSCC](https://arxiv.org/abs/2407.20028) - Self-supervised trajectory representation
|
| 93 |
+
- [LLM4Delay](https://arxiv.org/abs/2510.23636) - Cross-modality LLM adaptation for aviation
|