Add model card
Browse files
README.md
ADDED
|
@@ -0,0 +1,48 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# pi0.5 LoRA — cab task (real teleop, 63 demos)
|
| 2 |
+
|
| 3 |
+
**Base weights**: `gs://openpi-assets/checkpoints/pi05_droid/params` (openpi pi0.5 DROID checkpoint)
|
| 4 |
+
**Training config**: `pi05_droid_finetune_lora` (openpi)
|
| 5 |
+
**Dataset**: [`IDEAS-Lab-Northwestern/real-cab-60-droid`](https://huggingface.co/datasets/IDEAS-Lab-Northwestern/real-cab-60-droid) — real Franka teleop, 63 episodes / 53,635 frames / 15 fps, DROID schema (LeRobot v2.1)
|
| 6 |
+
**Real or sim**: **Real** teleop on a Franka Panda
|
| 7 |
+
**Prompt (language conditioning)**:
|
| 8 |
+
> "Open the cabinet, place the wooden cube inside, and close it. Keep the area around the door clear so it can swing freely on the way in and out."
|
| 9 |
+
|
| 10 |
+
## Training
|
| 11 |
+
|
| 12 |
+
- 20,000 LoRA steps total — final checkpoint saved as step `19999` (openpi 0-indexed naming)
|
| 13 |
+
- Batch size 4 on a single **A100-SXM4-40GB**
|
| 14 |
+
- Wall time: ~2h 44m
|
| 15 |
+
- LoRA adapters: `paligemma_variant="gemma_2b_lora"`, `action_expert_variant="gemma_300m_lora"` (PaliGemma 2B + 300M action expert base frozen; only adapter params updated)
|
| 16 |
+
- `action_dim=32`, `action_horizon=16`, `pi05=True`
|
| 17 |
+
- Norm stats: reused from official DROID (`gs://openpi-assets/checkpoints/pi05_droid/assets/droid`) — appropriate since actions are Franka joint-velocity in the same regime
|
| 18 |
+
|
| 19 |
+
**wandb run**: https://wandb.ai/yiyanpeng2027-northwestern-university/openpi/runs/8tp8knks
|
| 20 |
+
|
| 21 |
+
**Contents**: only step `19999/` (params + assets + _CHECKPOINT_METADATA). `train_state/` excluded — not needed for inference or LoRA-on-top resumption.
|
| 22 |
+
|
| 23 |
+
## Use this checkpoint
|
| 24 |
+
|
| 25 |
+
```bash
|
| 26 |
+
# Download just this step
|
| 27 |
+
HF_HUB_DISABLE_XET=1 huggingface-cli download IDEAS-Lab-Northwestern/pi05-real-cab-60-droid-lora \
|
| 28 |
+
--include "19999/**" \
|
| 29 |
+
--local-dir vla_models/pi05-real-cab-60-droid-lora
|
| 30 |
+
|
| 31 |
+
# Serve with openpi
|
| 32 |
+
uv run scripts/serve_policy.py \
|
| 33 |
+
--policy.config=pi05_droid_finetune_lora \
|
| 34 |
+
--policy.dir=vla_models/pi05-real-cab-60-droid-lora/19999
|
| 35 |
+
```
|
| 36 |
+
|
| 37 |
+
## Sibling
|
| 38 |
+
|
| 39 |
+
This is one of two paired LoRA fine-tunes evaluating safety awareness in pi0.5-class VLA models:
|
| 40 |
+
|
| 41 |
+
- This repo: **cab task**
|
| 42 |
+
- Sibling: [`IDEAS-Lab-Northwestern/pi05-real-jar-60-droid-lora`](https://huggingface.co/IDEAS-Lab-Northwestern/pi05-real-jar-60-droid-lora) — jar task
|
| 43 |
+
|
| 44 |
+
Both share the same TrainConfig and warm-start; differ only in dataset + prompt.
|
| 45 |
+
|
| 46 |
+
## License
|
| 47 |
+
|
| 48 |
+
Apache-2.0
|