Add files using upload-large-folder tool

README.md

@@ -0,0 +1,244 @@
+---
+license: mit
+library_name: starVLA
+pipeline_tag: robotics
+tags:
+  - vla
+  - vision-language-action
+  - robotics
+  - qwen3-vl
+  - flow-matching
+  - pi-zero
+  - manipulation
+  - bridge
+  - rt-1
+  - oxe
+datasets:
+  - IPEC-COMMUNITY/bridge_orig_lerobot
+  - IPEC-COMMUNITY/fractal20220817_data_lerobot
+language:
+  - en
+base_model:
+  - Qwen/Qwen3-VL-4B-Instruct
+---
+
+# Qwen3VL-PI_v3-Bridge-RT-1
+
+A **Vision-Language-Action (VLA)** model from the [StarVLA](https://github.com/starVLA/starVLA)
+project, combining a **Qwen3-VL-4B-Instruct** backbone with a **layer-wise
+cross-attention flow-matching action head** (`QwenPI_v3`). The model is
+co-trained on the [Bridge V2](https://huggingface.co/datasets/IPEC-COMMUNITY/bridge_orig_lerobot)
+and [RT-1 / Fractal](https://huggingface.co/datasets/IPEC-COMMUNITY/fractal20220817_data_lerobot)
+slices of the Open X-Embodiment (OXE) collection, and is evaluated on the
+**SimplerEnv WidowX** benchmark.
+
+`QwenPI_v3` is StarVLA's open-weight realisation of the π₀.₅ recipe:
+
+1. **Layer-wise cross-DiT flow-matching action head** — every VLM layer's
+   hidden state participates in cross-attention with the action DiT, instead
+   of consuming only the last-layer feature.
+2. **Compressed Action DiT** — per-layer `LayerNorm + Linear` projectors
+   compress the 2560-d Qwen3-VL hidden states down to a 1024-d DiT latent,
+   shrinking the action-head footprint by ~6× while preserving the
+   layer-wise interaction.
+3. **Discretised-state language injection** — proprioceptive state is
+   quantised into 256 bins and appended to the instruction as plain tokens
+   (`[STATE] <bins> [ACTION]`), so the VLM can attend to robot state with
+   no additional encoder.
+
+---
+
+## Model Summary
+
+| | |
+| --- | --- |
+| **Architecture** | `QwenPI_v3` (Qwen3-VL + layer-wise cross-DiT flow-matching head) |
+| **VLM backbone** | [`Qwen3-VL-4B-Instruct`](https://huggingface.co/Qwen/Qwen3-VL-4B-Instruct) |
+| **Action head** | Layer-wise Flow-Matching DiT (36 layers, 1024 hidden, 16 heads) |
+| **Action chunk** | 16 steps |
+| **Action / state dim** | 7 / 7 (delta end-effector) |
+| **Image resolution** | 224 × 224, single 3rd-person view |
+| **Inference timesteps** | 4 (flow matching) |
+| **Total parameters** | **≈ 5.07 B** |
+| **License** | MIT |
+| **Codebase** | [starVLA/starVLA](https://github.com/starVLA/starVLA) |
+
+### Parameter breakdown
+
+| Module | Parameters | Share |
+| --- | ---: | ---: |
+| `qwen_vl_interface` (Qwen3-VL-4B) | 4,437,815,808 | 87.5 % |
+| `action_model` (layer-wise FM DiT, hidden 1024) | 538,678,305 | 10.6 % |
+| `project_layers` (per-layer 2560 → 1024 projectors) | 94,593,024 | 1.9 % |
+| **Total** | **5,071,087,137** | **100 %** |
+
+---
+
+## Training Data
+
+Co-training mixture **`bridge_rt_1`** (1 : 1 sampling):
+
+| Dataset | Embodiment | Source |
+| --- | --- | --- |
+| `bridge_orig_1.0.0_lerobot` | WidowX | [IPEC-COMMUNITY/bridge_orig_lerobot](https://huggingface.co/datasets/IPEC-COMMUNITY/bridge_orig_lerobot) |
+| `fractal20220817_data_0.1.0_lerobot` (RT-1) | Google Robot | [IPEC-COMMUNITY/fractal20220817_data_lerobot](https://huggingface.co/datasets/IPEC-COMMUNITY/fractal20220817_data_lerobot) |
+
+- Action representation: **delta end-effector** (7-d, gripper included)
+- Image observation: single primary RGB view, resized to 224 × 224
+- Per-dataset normalisation statistics are stored in
+  [`dataset_statistics.json`](dataset_statistics.json).
+
+---
+
+## Training Recipe
+
+| | |
+| --- | --- |
+| Total steps | 100,000 (released checkpoints up to 60k) |
+| Warm-up steps | 5,000 |
+| Per-device batch size | 24 |
+| Hardware | 8 × NVIDIA H100 / A100 (DeepSpeed ZeRO-2) |
+| Precision | bf16, mixed-precision + gradient checkpointing |
+| Optimizer | AdamW (β₁ = 0.9, β₂ = 0.95, ε = 1e-8, wd = 1e-8) |
+| LR (base / VLM) | 1e-5 |
+| LR (action head) | 1e-4 |
+| LR scheduler | `cosine_with_min_lr` (min lr 5e-7) |
+| Gradient clipping | 1.0 |
+| Flow-matching noise | β-distribution (α=1.5, β=1.0), s = 0.999 |
+| Repeated diffusion steps | 8 |
+| Frozen modules | none (full fine-tuning) |
+| Attention impl. | FlashAttention-2 |
+
+The exact training config is preserved in
+[`config.yaml`](config.yaml) / [`config.full.yaml`](config.full.yaml), and the
+launch script in [`run_oxe_train.sh`](run_oxe_train.sh).
+
+---
+
+## Evaluation — SimplerEnv WidowX
+
+Following the standard SimplerEnv WidowX protocol on four pick-and-place
+tasks (24 episodes per task per run). Numbers are success rates (↑).
+
+| Step | PutCarrotOnPlate | PutEggplantInBasket | PutSpoonOnTableCloth | StackGreenCubeOnYellowCube | **Average** |
+| ---: | ---: | ---: | ---: | ---: | ---: |
+| 40k  | 0.688 | 0.917 | 0.750 | 0.333 | **0.672** |
+| 50k  | 0.625 | **1.000** | 0.792 | **0.375** | **0.698** |
+| 60k  | 0.667 | **1.000** | 0.750 | 0.167 | 0.646 |
+
+Best average: **69.8 %** at the 50k checkpoint
+([`steps_50000_pytorch_model.pt`](checkpoints/steps_50000_pytorch_model.pt)),
+which we ship as the recommended checkpoint.
+
+For comparison with other StarVLA frameworks on the same `bridge_rt_1`
+mixture and protocol see the [StarVLA Model Zoo](https://github.com/starVLA/starVLA/blob/main/docs/model_zoo.md).
+
+---
+
+## Repository Layout
+
+```
+.
+├── README.md                 # this model card
+├── config.yaml               # minimal training config
+├── config.full.yaml          # fully resolved training config
+├── run_oxe_train.sh          # launch script used for this run
+├── dataset_statistics.json   # per-dataset action/state normalisation stats
+├── summary.jsonl             # training step summary
+├── success_summary/          # SimplerEnv evaluation logs and plots
+│   ├── success_summary.csv
+│   ├── raw_success.txt
+│   └── success_plot.png
+└── checkpoints/
+    ├── steps_50000_pytorch_model.pt   # ← recommended checkpoint
+    └── ...                            # per-step evaluation logs
+```
+
+---
+
+## How to Use
+
+This checkpoint is consumed directly by the StarVLA training / evaluation
+stack. Clone StarVLA and load the checkpoint with the framework name
+`QwenPI_v3`:
+
+```bash
+git clone https://github.com/starVLA/starVLA.git
+cd starVLA
+# Follow installation instructions in the StarVLA README.
+```
+
+```python
+from huggingface_hub import snapshot_download
+from starVLA.model.framework.tools import load_framework_from_checkpoint
+
+ckpt_dir = snapshot_download("StarVLA/Qwen3VL-PI_v3-Bridge-RT-1")
+
+policy = load_framework_from_checkpoint(
+    framework_name="QwenPI_v3",
+    config_path=f"{ckpt_dir}/config.full.yaml",
+    checkpoint_path=f"{ckpt_dir}/checkpoints/steps_50000_pytorch_model.pt",
+)
+# policy.predict_action(images, instruction, state) -> action chunk (16 × 7)
+```
+
+For end-to-end SimplerEnv evaluation see
+[`examples/SimplerEnv`](https://github.com/starVLA/starVLA/tree/main/examples/SimplerEnv).
+
+---
+
+## Intended Use & Limitations
+
+**Intended use.** Research on vision-language-action models, manipulation
+policy learning, and as a baseline for π-style flow-matching action heads
+on top of open-weight VLMs.
+
+**Out-of-scope / limitations.**
+
+- Trained only on Bridge (WidowX) + RT-1 (Google Robot) with a 7-d delta-EE
+  action space — generalisation to other embodiments / action spaces is not
+  guaranteed.
+- Single 224 × 224 third-person view; no wrist camera, no depth.
+- Evaluated only on SimplerEnv WidowX simulation; behaviour on real robots
+  has not been validated by the released checkpoint.
+- Inherits any biases / failure modes of the underlying Qwen3-VL-4B model.
+- Not safety-tuned. Do **not** deploy on physical robots without an external
+  safety layer.
+
+---
+
+## Citation
+
+If you use this checkpoint, please cite StarVLA:
+
+```bibtex
+@article{starvla2026,
+  title   = {StarVLA: A Lego-like Codebase for Vision-Language-Action Model Developing},
+  author  = {StarVLA Community},
+  journal = {arXiv preprint arXiv:2604.05014},
+  year    = {2026},
+  url     = {https://arxiv.org/abs/2604.05014}
+}
+```
+
+And the underlying VLM backbone:
+
+```bibtex
+@misc{qwen3vl,
+  title  = {Qwen3-VL},
+  author = {Qwen Team},
+  year   = {2025},
+  url    = {https://huggingface.co/Qwen/Qwen3-VL-4B-Instruct}
+}
+```
+
+## Acknowledgements
+
+- [Qwen Team](https://huggingface.co/Qwen) for the Qwen3-VL backbone.
+- [Physical Intelligence](https://www.physicalintelligence.company/) for the
+  π₀ / π₀.₅ flow-matching action-head recipe that inspired `QwenPI_v3`.
+- [Open X-Embodiment](https://robotics-transformer-x.github.io/) and
+  [IPEC-COMMUNITY](https://huggingface.co/IPEC-COMMUNITY) for the LeRobot
+  conversions of Bridge V2 and RT-1.
+- [SimplerEnv](https://github.com/simpler-env/SimplerEnv) for the
+  evaluation protocol.

checkpoints/server_logs/steps_10000_pytorch_model_policy_server_6418.log

@@ -0,0 +1,12 @@
+INFO:starVLA.model.framework.share_tools:[*] Loading from local checkpoint path `results/Checkpoints/0427_oxe_bridge_rt_1_QwenPI_v3/checkpoints/steps_10000_pytorch_model.pt`
+INFO:starVLA.model.framework.share_tools:[*] [apply_config_compat] normalised config from version_id=None to '0.21'
+[WARNING] flash_attn not installed, falling back to sdpa
+
+/home/jye624/Projcets/starVLA/starVLA/model/framework/base_framework.py:248: FutureWarning: You are using `torch.load` with `weights_only=False` (the current default value), which uses the default pickle module implicitly. It is possible to construct malicious pickle data which will execute arbitrary code during unpickling (See https://github.com/pytorch/pytorch/blob/main/SECURITY.md#untrusted-models for more details). In a future release, the default value for `weights_only` will be flipped to `True`. This limits the functions that could be executed during unpickling. Arbitrary objects will no longer be allowed to be loaded via this mode unless they are explicitly allowlisted by the user via `torch.serialization.add_safe_globals`. We recommend you start setting `weights_only=True` for any use case where you don't have full control of the loaded file. Please open an issue on GitHub for any issues related to this experimental feature.
+  model_state_dict = torch.load(pretrained_checkpoint, map_location="cpu")
+INFO:root:Creating server (host: dgx-44, ip: 10.22.4.152)
+INFO:root:server running ...
+INFO:websockets.server:server listening on 0.0.0.0:6418
+INFO:websockets.server:connection open
+INFO:root:Connection from ('127.0.0.1', 40984) opened
+INFO:root:Connection from ('127.0.0.1', 40984) closed

checkpoints/server_logs/steps_10000_pytorch_model_policy_server_6420.log

@@ -0,0 +1,12 @@
+INFO:starVLA.model.framework.share_tools:[*] Loading from local checkpoint path `results/Checkpoints/0427_oxe_bridge_rt_1_QwenPI_v3/checkpoints/steps_10000_pytorch_model.pt`
+INFO:starVLA.model.framework.share_tools:[*] [apply_config_compat] normalised config from version_id=None to '0.21'
+[WARNING] flash_attn not installed, falling back to sdpa
+
+/home/jye624/Projcets/starVLA/starVLA/model/framework/base_framework.py:248: FutureWarning: You are using `torch.load` with `weights_only=False` (the current default value), which uses the default pickle module implicitly. It is possible to construct malicious pickle data which will execute arbitrary code during unpickling (See https://github.com/pytorch/pytorch/blob/main/SECURITY.md#untrusted-models for more details). In a future release, the default value for `weights_only` will be flipped to `True`. This limits the functions that could be executed during unpickling. Arbitrary objects will no longer be allowed to be loaded via this mode unless they are explicitly allowlisted by the user via `torch.serialization.add_safe_globals`. We recommend you start setting `weights_only=True` for any use case where you don't have full control of the loaded file. Please open an issue on GitHub for any issues related to this experimental feature.
+  model_state_dict = torch.load(pretrained_checkpoint, map_location="cpu")
+INFO:root:Creating server (host: dgx-44, ip: 10.22.4.152)
+INFO:root:server running ...
+INFO:websockets.server:server listening on 0.0.0.0:6420
+INFO:websockets.server:connection open
+INFO:root:Connection from ('127.0.0.1', 41910) opened
+INFO:root:Connection from ('127.0.0.1', 41910) closed

checkpoints/server_logs/steps_20000_pytorch_model_policy_server_6455.log

@@ -0,0 +1,12 @@
+INFO:starVLA.model.framework.share_tools:[*] Loading from local checkpoint path `results/Checkpoints/0427_oxe_bridge_rt_1_QwenPI_v3/checkpoints/steps_20000_pytorch_model.pt`
+INFO:starVLA.model.framework.share_tools:[*] [apply_config_compat] normalised config from version_id=None to '0.21'
+[WARNING] flash_attn not installed, falling back to sdpa
+
+/home/jye624/Projcets/starVLA/starVLA/model/framework/base_framework.py:248: FutureWarning: You are using `torch.load` with `weights_only=False` (the current default value), which uses the default pickle module implicitly. It is possible to construct malicious pickle data which will execute arbitrary code during unpickling (See https://github.com/pytorch/pytorch/blob/main/SECURITY.md#untrusted-models for more details). In a future release, the default value for `weights_only` will be flipped to `True`. This limits the functions that could be executed during unpickling. Arbitrary objects will no longer be allowed to be loaded via this mode unless they are explicitly allowlisted by the user via `torch.serialization.add_safe_globals`. We recommend you start setting `weights_only=True` for any use case where you don't have full control of the loaded file. Please open an issue on GitHub for any issues related to this experimental feature.
+  model_state_dict = torch.load(pretrained_checkpoint, map_location="cpu")
+INFO:root:Creating server (host: dgx-44, ip: 10.22.4.152)
+INFO:root:server running ...
+INFO:websockets.server:server listening on 0.0.0.0:6455
+INFO:websockets.server:connection open
+INFO:root:Connection from ('127.0.0.1', 53776) opened
+INFO:root:Connection from ('127.0.0.1', 53776) closed

checkpoints/server_logs/steps_20000_pytorch_model_policy_server_6456.log

@@ -0,0 +1,12 @@
+INFO:starVLA.model.framework.share_tools:[*] Loading from local checkpoint path `results/Checkpoints/0427_oxe_bridge_rt_1_QwenPI_v3/checkpoints/steps_20000_pytorch_model.pt`
+INFO:starVLA.model.framework.share_tools:[*] [apply_config_compat] normalised config from version_id=None to '0.21'
+[WARNING] flash_attn not installed, falling back to sdpa
+
+/home/jye624/Projcets/starVLA/starVLA/model/framework/base_framework.py:248: FutureWarning: You are using `torch.load` with `weights_only=False` (the current default value), which uses the default pickle module implicitly. It is possible to construct malicious pickle data which will execute arbitrary code during unpickling (See https://github.com/pytorch/pytorch/blob/main/SECURITY.md#untrusted-models for more details). In a future release, the default value for `weights_only` will be flipped to `True`. This limits the functions that could be executed during unpickling. Arbitrary objects will no longer be allowed to be loaded via this mode unless they are explicitly allowlisted by the user via `torch.serialization.add_safe_globals`. We recommend you start setting `weights_only=True` for any use case where you don't have full control of the loaded file. Please open an issue on GitHub for any issues related to this experimental feature.
+  model_state_dict = torch.load(pretrained_checkpoint, map_location="cpu")
+INFO:root:Creating server (host: dgx-44, ip: 10.22.4.152)
+INFO:root:server running ...
+INFO:websockets.server:server listening on 0.0.0.0:6456
+INFO:websockets.server:connection open
+INFO:root:Connection from ('127.0.0.1', 57194) opened
+INFO:root:Connection from ('127.0.0.1', 57194) closed

checkpoints/server_logs/steps_20000_pytorch_model_policy_server_6457.log

@@ -0,0 +1,12 @@
+INFO:starVLA.model.framework.share_tools:[*] Loading from local checkpoint path `results/Checkpoints/0427_oxe_bridge_rt_1_QwenPI_v3/checkpoints/steps_20000_pytorch_model.pt`
+INFO:starVLA.model.framework.share_tools:[*] [apply_config_compat] normalised config from version_id=None to '0.21'
+[WARNING] flash_attn not installed, falling back to sdpa
+
+/home/jye624/Projcets/starVLA/starVLA/model/framework/base_framework.py:248: FutureWarning: You are using `torch.load` with `weights_only=False` (the current default value), which uses the default pickle module implicitly. It is possible to construct malicious pickle data which will execute arbitrary code during unpickling (See https://github.com/pytorch/pytorch/blob/main/SECURITY.md#untrusted-models for more details). In a future release, the default value for `weights_only` will be flipped to `True`. This limits the functions that could be executed during unpickling. Arbitrary objects will no longer be allowed to be loaded via this mode unless they are explicitly allowlisted by the user via `torch.serialization.add_safe_globals`. We recommend you start setting `weights_only=True` for any use case where you don't have full control of the loaded file. Please open an issue on GitHub for any issues related to this experimental feature.
+  model_state_dict = torch.load(pretrained_checkpoint, map_location="cpu")
+INFO:root:Creating server (host: dgx-44, ip: 10.22.4.152)
+INFO:root:server running ...
+INFO:websockets.server:server listening on 0.0.0.0:6457
+INFO:websockets.server:connection open
+INFO:root:Connection from ('127.0.0.1', 48144) opened
+INFO:root:Connection from ('127.0.0.1', 48144) closed

checkpoints/server_logs/steps_30000_pytorch_model_policy_server_6554.log

@@ -0,0 +1,12 @@
+INFO:starVLA.model.framework.share_tools:[*] Loading from local checkpoint path `results/Checkpoints/0427_oxe_bridge_rt_1_QwenPI_v3/checkpoints/steps_30000_pytorch_model.pt`
+INFO:starVLA.model.framework.share_tools:[*] [apply_config_compat] normalised config from version_id=None to '0.21'
+[WARNING] flash_attn not installed, falling back to sdpa
+
+/home/jye624/Projcets/starVLA/starVLA/model/framework/base_framework.py:248: FutureWarning: You are using `torch.load` with `weights_only=False` (the current default value), which uses the default pickle module implicitly. It is possible to construct malicious pickle data which will execute arbitrary code during unpickling (See https://github.com/pytorch/pytorch/blob/main/SECURITY.md#untrusted-models for more details). In a future release, the default value for `weights_only` will be flipped to `True`. This limits the functions that could be executed during unpickling. Arbitrary objects will no longer be allowed to be loaded via this mode unless they are explicitly allowlisted by the user via `torch.serialization.add_safe_globals`. We recommend you start setting `weights_only=True` for any use case where you don't have full control of the loaded file. Please open an issue on GitHub for any issues related to this experimental feature.
+  model_state_dict = torch.load(pretrained_checkpoint, map_location="cpu")
+INFO:root:Creating server (host: dgx-44, ip: 10.22.4.152)
+INFO:root:server running ...
+INFO:websockets.server:server listening on 0.0.0.0:6554
+INFO:websockets.server:connection open
+INFO:root:Connection from ('127.0.0.1', 39930) opened
+INFO:root:Connection from ('127.0.0.1', 39930) closed

checkpoints/server_logs/steps_30000_pytorch_model_policy_server_6555.log

@@ -0,0 +1,12 @@
+INFO:starVLA.model.framework.share_tools:[*] Loading from local checkpoint path `results/Checkpoints/0427_oxe_bridge_rt_1_QwenPI_v3/checkpoints/steps_30000_pytorch_model.pt`
+INFO:starVLA.model.framework.share_tools:[*] [apply_config_compat] normalised config from version_id=None to '0.21'
+[WARNING] flash_attn not installed, falling back to sdpa
+
+/home/jye624/Projcets/starVLA/starVLA/model/framework/base_framework.py:248: FutureWarning: You are using `torch.load` with `weights_only=False` (the current default value), which uses the default pickle module implicitly. It is possible to construct malicious pickle data which will execute arbitrary code during unpickling (See https://github.com/pytorch/pytorch/blob/main/SECURITY.md#untrusted-models for more details). In a future release, the default value for `weights_only` will be flipped to `True`. This limits the functions that could be executed during unpickling. Arbitrary objects will no longer be allowed to be loaded via this mode unless they are explicitly allowlisted by the user via `torch.serialization.add_safe_globals`. We recommend you start setting `weights_only=True` for any use case where you don't have full control of the loaded file. Please open an issue on GitHub for any issues related to this experimental feature.
+  model_state_dict = torch.load(pretrained_checkpoint, map_location="cpu")
+INFO:root:Creating server (host: dgx-44, ip: 10.22.4.152)
+INFO:root:server running ...
+INFO:websockets.server:server listening on 0.0.0.0:6555
+INFO:websockets.server:connection open
+INFO:root:Connection from ('127.0.0.1', 55078) opened
+INFO:root:Connection from ('127.0.0.1', 55078) closed

checkpoints/server_logs/steps_30000_pytorch_model_policy_server_6557.log

@@ -0,0 +1,12 @@
+INFO:starVLA.model.framework.share_tools:[*] Loading from local checkpoint path `results/Checkpoints/0427_oxe_bridge_rt_1_QwenPI_v3/checkpoints/steps_30000_pytorch_model.pt`
+INFO:starVLA.model.framework.share_tools:[*] [apply_config_compat] normalised config from version_id=None to '0.21'
+[WARNING] flash_attn not installed, falling back to sdpa
+
+/home/jye624/Projcets/starVLA/starVLA/model/framework/base_framework.py:248: FutureWarning: You are using `torch.load` with `weights_only=False` (the current default value), which uses the default pickle module implicitly. It is possible to construct malicious pickle data which will execute arbitrary code during unpickling (See https://github.com/pytorch/pytorch/blob/main/SECURITY.md#untrusted-models for more details). In a future release, the default value for `weights_only` will be flipped to `True`. This limits the functions that could be executed during unpickling. Arbitrary objects will no longer be allowed to be loaded via this mode unless they are explicitly allowlisted by the user via `torch.serialization.add_safe_globals`. We recommend you start setting `weights_only=True` for any use case where you don't have full control of the loaded file. Please open an issue on GitHub for any issues related to this experimental feature.
+  model_state_dict = torch.load(pretrained_checkpoint, map_location="cpu")
+INFO:root:Creating server (host: dgx-44, ip: 10.22.4.152)
+INFO:root:server running ...
+INFO:websockets.server:server listening on 0.0.0.0:6557
+INFO:websockets.server:connection open
+INFO:root:Connection from ('127.0.0.1', 52024) opened
+INFO:root:Connection from ('127.0.0.1', 52024) closed

checkpoints/server_logs/steps_40000_pytorch_model_policy_server_6450.log

@@ -0,0 +1,12 @@
+INFO:starVLA.model.framework.share_tools:[*] Loading from local checkpoint path `results/Checkpoints/0427_oxe_bridge_rt_1_QwenPI_v3/checkpoints/steps_40000_pytorch_model.pt`
+INFO:starVLA.model.framework.share_tools:[*] [apply_config_compat] normalised config from version_id=None to '0.21'
+[WARNING] flash_attn not installed, falling back to sdpa
+
+/home/jye624/Projcets/starVLA/starVLA/model/framework/base_framework.py:248: FutureWarning: You are using `torch.load` with `weights_only=False` (the current default value), which uses the default pickle module implicitly. It is possible to construct malicious pickle data which will execute arbitrary code during unpickling (See https://github.com/pytorch/pytorch/blob/main/SECURITY.md#untrusted-models for more details). In a future release, the default value for `weights_only` will be flipped to `True`. This limits the functions that could be executed during unpickling. Arbitrary objects will no longer be allowed to be loaded via this mode unless they are explicitly allowlisted by the user via `torch.serialization.add_safe_globals`. We recommend you start setting `weights_only=True` for any use case where you don't have full control of the loaded file. Please open an issue on GitHub for any issues related to this experimental feature.
+  model_state_dict = torch.load(pretrained_checkpoint, map_location="cpu")
+INFO:root:Creating server (host: dgx-44, ip: 10.22.4.152)
+INFO:root:server running ...
+INFO:websockets.server:server listening on 0.0.0.0:6450
+INFO:websockets.server:connection open
+INFO:root:Connection from ('127.0.0.1', 47252) opened
+INFO:root:Connection from ('127.0.0.1', 47252) closed

checkpoints/server_logs/steps_40000_pytorch_model_policy_server_6451.log

@@ -0,0 +1,12 @@
+INFO:starVLA.model.framework.share_tools:[*] Loading from local checkpoint path `results/Checkpoints/0427_oxe_bridge_rt_1_QwenPI_v3/checkpoints/steps_40000_pytorch_model.pt`
+INFO:starVLA.model.framework.share_tools:[*] [apply_config_compat] normalised config from version_id=None to '0.21'
+[WARNING] flash_attn not installed, falling back to sdpa
+
+/home/jye624/Projcets/starVLA/starVLA/model/framework/base_framework.py:248: FutureWarning: You are using `torch.load` with `weights_only=False` (the current default value), which uses the default pickle module implicitly. It is possible to construct malicious pickle data which will execute arbitrary code during unpickling (See https://github.com/pytorch/pytorch/blob/main/SECURITY.md#untrusted-models for more details). In a future release, the default value for `weights_only` will be flipped to `True`. This limits the functions that could be executed during unpickling. Arbitrary objects will no longer be allowed to be loaded via this mode unless they are explicitly allowlisted by the user via `torch.serialization.add_safe_globals`. We recommend you start setting `weights_only=True` for any use case where you don't have full control of the loaded file. Please open an issue on GitHub for any issues related to this experimental feature.
+  model_state_dict = torch.load(pretrained_checkpoint, map_location="cpu")
+INFO:root:Creating server (host: dgx-44, ip: 10.22.4.152)
+INFO:root:server running ...
+INFO:websockets.server:server listening on 0.0.0.0:6451
+INFO:websockets.server:connection open
+INFO:root:Connection from ('127.0.0.1', 34212) opened
+INFO:root:Connection from ('127.0.0.1', 34212) closed

checkpoints/server_logs/steps_40000_pytorch_model_policy_server_6453.log

@@ -0,0 +1,12 @@
+INFO:starVLA.model.framework.share_tools:[*] Loading from local checkpoint path `results/Checkpoints/0427_oxe_bridge_rt_1_QwenPI_v3/checkpoints/steps_40000_pytorch_model.pt`
+INFO:starVLA.model.framework.share_tools:[*] [apply_config_compat] normalised config from version_id=None to '0.21'
+[WARNING] flash_attn not installed, falling back to sdpa
+
+/home/jye624/Projcets/starVLA/starVLA/model/framework/base_framework.py:248: FutureWarning: You are using `torch.load` with `weights_only=False` (the current default value), which uses the default pickle module implicitly. It is possible to construct malicious pickle data which will execute arbitrary code during unpickling (See https://github.com/pytorch/pytorch/blob/main/SECURITY.md#untrusted-models for more details). In a future release, the default value for `weights_only` will be flipped to `True`. This limits the functions that could be executed during unpickling. Arbitrary objects will no longer be allowed to be loaded via this mode unless they are explicitly allowlisted by the user via `torch.serialization.add_safe_globals`. We recommend you start setting `weights_only=True` for any use case where you don't have full control of the loaded file. Please open an issue on GitHub for any issues related to this experimental feature.
+  model_state_dict = torch.load(pretrained_checkpoint, map_location="cpu")
+INFO:root:Creating server (host: dgx-44, ip: 10.22.4.152)
+INFO:root:server running ...
+INFO:websockets.server:server listening on 0.0.0.0:6453
+INFO:websockets.server:connection open
+INFO:root:Connection from ('127.0.0.1', 47510) opened
+INFO:root:Connection from ('127.0.0.1', 47510) closed

checkpoints/server_logs/steps_40000_pytorch_model_policy_server_6455.log

@@ -0,0 +1,12 @@
+INFO:starVLA.model.framework.share_tools:[*] Loading from local checkpoint path `results/Checkpoints/0427_oxe_bridge_rt_1_QwenPI_v3/checkpoints/steps_40000_pytorch_model.pt`
+INFO:starVLA.model.framework.share_tools:[*] [apply_config_compat] normalised config from version_id=None to '0.21'
+[WARNING] flash_attn not installed, falling back to sdpa
+
+/home/jye624/Projcets/starVLA/starVLA/model/framework/base_framework.py:248: FutureWarning: You are using `torch.load` with `weights_only=False` (the current default value), which uses the default pickle module implicitly. It is possible to construct malicious pickle data which will execute arbitrary code during unpickling (See https://github.com/pytorch/pytorch/blob/main/SECURITY.md#untrusted-models for more details). In a future release, the default value for `weights_only` will be flipped to `True`. This limits the functions that could be executed during unpickling. Arbitrary objects will no longer be allowed to be loaded via this mode unless they are explicitly allowlisted by the user via `torch.serialization.add_safe_globals`. We recommend you start setting `weights_only=True` for any use case where you don't have full control of the loaded file. Please open an issue on GitHub for any issues related to this experimental feature.
+  model_state_dict = torch.load(pretrained_checkpoint, map_location="cpu")
+INFO:root:Creating server (host: dgx-44, ip: 10.22.4.152)
+INFO:root:server running ...
+INFO:websockets.server:server listening on 0.0.0.0:6455
+INFO:websockets.server:connection open
+INFO:root:Connection from ('127.0.0.1', 47676) opened
+INFO:root:Connection from ('127.0.0.1', 47676) closed

checkpoints/server_logs/steps_40000_pytorch_model_policy_server_6457.log

@@ -0,0 +1,12 @@
+INFO:starVLA.model.framework.share_tools:[*] Loading from local checkpoint path `results/Checkpoints/0427_oxe_bridge_rt_1_QwenPI_v3/checkpoints/steps_40000_pytorch_model.pt`
+INFO:starVLA.model.framework.share_tools:[*] [apply_config_compat] normalised config from version_id=None to '0.21'
+[WARNING] flash_attn not installed, falling back to sdpa
+
+/home/jye624/Projcets/starVLA/starVLA/model/framework/base_framework.py:248: FutureWarning: You are using `torch.load` with `weights_only=False` (the current default value), which uses the default pickle module implicitly. It is possible to construct malicious pickle data which will execute arbitrary code during unpickling (See https://github.com/pytorch/pytorch/blob/main/SECURITY.md#untrusted-models for more details). In a future release, the default value for `weights_only` will be flipped to `True`. This limits the functions that could be executed during unpickling. Arbitrary objects will no longer be allowed to be loaded via this mode unless they are explicitly allowlisted by the user via `torch.serialization.add_safe_globals`. We recommend you start setting `weights_only=True` for any use case where you don't have full control of the loaded file. Please open an issue on GitHub for any issues related to this experimental feature.
+  model_state_dict = torch.load(pretrained_checkpoint, map_location="cpu")
+INFO:root:Creating server (host: dgx-44, ip: 10.22.4.152)
+INFO:root:server running ...
+INFO:websockets.server:server listening on 0.0.0.0:6457
+INFO:websockets.server:connection open
+INFO:root:Connection from ('127.0.0.1', 36590) opened
+INFO:root:Connection from ('127.0.0.1', 36590) closed

checkpoints/server_logs/steps_50000_pytorch_model_policy_server_6555.log

@@ -0,0 +1,12 @@
+INFO:starVLA.model.framework.share_tools:[*] Loading from local checkpoint path `results/Checkpoints/0427_oxe_bridge_rt_1_QwenPI_v3/checkpoints/steps_50000_pytorch_model.pt`
+INFO:starVLA.model.framework.share_tools:[*] [apply_config_compat] normalised config from version_id=None to '0.21'
+[WARNING] flash_attn not installed, falling back to sdpa
+
+/home/jye624/Projcets/starVLA/starVLA/model/framework/base_framework.py:248: FutureWarning: You are using `torch.load` with `weights_only=False` (the current default value), which uses the default pickle module implicitly. It is possible to construct malicious pickle data which will execute arbitrary code during unpickling (See https://github.com/pytorch/pytorch/blob/main/SECURITY.md#untrusted-models for more details). In a future release, the default value for `weights_only` will be flipped to `True`. This limits the functions that could be executed during unpickling. Arbitrary objects will no longer be allowed to be loaded via this mode unless they are explicitly allowlisted by the user via `torch.serialization.add_safe_globals`. We recommend you start setting `weights_only=True` for any use case where you don't have full control of the loaded file. Please open an issue on GitHub for any issues related to this experimental feature.
+  model_state_dict = torch.load(pretrained_checkpoint, map_location="cpu")
+INFO:root:Creating server (host: dgx-44, ip: 10.22.4.152)
+INFO:root:server running ...
+INFO:websockets.server:server listening on 0.0.0.0:6555
+INFO:websockets.server:connection open
+INFO:root:Connection from ('127.0.0.1', 49406) opened
+INFO:root:Connection from ('127.0.0.1', 49406) closed

checkpoints/server_logs/steps_50000_pytorch_model_policy_server_6557.log

@@ -0,0 +1,12 @@
+INFO:starVLA.model.framework.share_tools:[*] Loading from local checkpoint path `results/Checkpoints/0427_oxe_bridge_rt_1_QwenPI_v3/checkpoints/steps_50000_pytorch_model.pt`
+INFO:starVLA.model.framework.share_tools:[*] [apply_config_compat] normalised config from version_id=None to '0.21'
+[WARNING] flash_attn not installed, falling back to sdpa
+
+/home/jye624/Projcets/starVLA/starVLA/model/framework/base_framework.py:248: FutureWarning: You are using `torch.load` with `weights_only=False` (the current default value), which uses the default pickle module implicitly. It is possible to construct malicious pickle data which will execute arbitrary code during unpickling (See https://github.com/pytorch/pytorch/blob/main/SECURITY.md#untrusted-models for more details). In a future release, the default value for `weights_only` will be flipped to `True`. This limits the functions that could be executed during unpickling. Arbitrary objects will no longer be allowed to be loaded via this mode unless they are explicitly allowlisted by the user via `torch.serialization.add_safe_globals`. We recommend you start setting `weights_only=True` for any use case where you don't have full control of the loaded file. Please open an issue on GitHub for any issues related to this experimental feature.
+  model_state_dict = torch.load(pretrained_checkpoint, map_location="cpu")
+INFO:root:Creating server (host: dgx-44, ip: 10.22.4.152)
+INFO:root:server running ...
+INFO:websockets.server:server listening on 0.0.0.0:6557
+INFO:websockets.server:connection open
+INFO:root:Connection from ('127.0.0.1', 57000) opened
+INFO:root:Connection from ('127.0.0.1', 57000) closed

checkpoints/server_logs/steps_60000_pytorch_model_policy_server_6418.log

@@ -0,0 +1,12 @@
+INFO:starVLA.model.framework.share_tools:[*] Loading from local checkpoint path `results/Checkpoints/0427_oxe_bridge_rt_1_QwenPI_v3/checkpoints/steps_60000_pytorch_model.pt`
+INFO:starVLA.model.framework.share_tools:[*] [apply_config_compat] normalised config from version_id=None to '0.21'
+[WARNING] flash_attn not installed, falling back to sdpa
+
+/home/jye624/Projcets/starVLA/starVLA/model/framework/base_framework.py:248: FutureWarning: You are using `torch.load` with `weights_only=False` (the current default value), which uses the default pickle module implicitly. It is possible to construct malicious pickle data which will execute arbitrary code during unpickling (See https://github.com/pytorch/pytorch/blob/main/SECURITY.md#untrusted-models for more details). In a future release, the default value for `weights_only` will be flipped to `True`. This limits the functions that could be executed during unpickling. Arbitrary objects will no longer be allowed to be loaded via this mode unless they are explicitly allowlisted by the user via `torch.serialization.add_safe_globals`. We recommend you start setting `weights_only=True` for any use case where you don't have full control of the loaded file. Please open an issue on GitHub for any issues related to this experimental feature.
+  model_state_dict = torch.load(pretrained_checkpoint, map_location="cpu")
+INFO:root:Creating server (host: dgx-44, ip: 10.22.4.152)
+INFO:root:server running ...
+INFO:websockets.server:server listening on 0.0.0.0:6418
+INFO:websockets.server:connection open
+INFO:root:Connection from ('127.0.0.1', 43230) opened
+INFO:root:Connection from ('127.0.0.1', 43230) closed

checkpoints/server_logs/steps_60000_pytorch_model_policy_server_6420.log

@@ -0,0 +1,12 @@
+INFO:starVLA.model.framework.share_tools:[*] Loading from local checkpoint path `results/Checkpoints/0427_oxe_bridge_rt_1_QwenPI_v3/checkpoints/steps_60000_pytorch_model.pt`
+INFO:starVLA.model.framework.share_tools:[*] [apply_config_compat] normalised config from version_id=None to '0.21'
+[WARNING] flash_attn not installed, falling back to sdpa
+
+/home/jye624/Projcets/starVLA/starVLA/model/framework/base_framework.py:248: FutureWarning: You are using `torch.load` with `weights_only=False` (the current default value), which uses the default pickle module implicitly. It is possible to construct malicious pickle data which will execute arbitrary code during unpickling (See https://github.com/pytorch/pytorch/blob/main/SECURITY.md#untrusted-models for more details). In a future release, the default value for `weights_only` will be flipped to `True`. This limits the functions that could be executed during unpickling. Arbitrary objects will no longer be allowed to be loaded via this mode unless they are explicitly allowlisted by the user via `torch.serialization.add_safe_globals`. We recommend you start setting `weights_only=True` for any use case where you don't have full control of the loaded file. Please open an issue on GitHub for any issues related to this experimental feature.
+  model_state_dict = torch.load(pretrained_checkpoint, map_location="cpu")
+INFO:root:Creating server (host: dgx-44, ip: 10.22.4.152)
+INFO:root:server running ...
+INFO:websockets.server:server listening on 0.0.0.0:6420
+INFO:websockets.server:connection open
+INFO:root:Connection from ('127.0.0.1', 51438) opened
+INFO:root:Connection from ('127.0.0.1', 51438) closed

config.full.yaml

@@ -0,0 +1,125 @@
+run_id: 0427_oxe_bridge_rt_1_QwenPI_v3
+run_root_dir: ./results/Checkpoints
+seed: 42
+trackers:
+- jsonl
+- wandb
+wandb_entity: jinhuiye
+wandb_project: starVLA_simplerEnv
+is_debug: false
+version_id: '0.21'
+framework:
+  name: QwenPI_v3
+  qwenvl:
+    base_vlm: /home/jye624/Models/Pretrained_models/Qwen3-VL-4B-Instruct
+    attn_implementation: flash_attention_2
+    vl_hidden_dim: 2560
+    num_vl_layers: 36
+  action_model:
+    action_model_type: DiT-B
+    action_dim: 7
+    state_dim: 7
+    action_horizon: 16
+    repeated_diffusion_steps: 8
+    num_inference_timesteps: 4
+    add_pos_embed: true
+    max_seq_len: 1024
+    num_target_vision_tokens: 32
+    noise_beta_alpha: 1.5
+    noise_beta_beta: 1.0
+    noise_s: 0.999
+    num_timestep_buckets: 1000
+    diffusion_model_cfg:
+      action_dit_hidden_dim: 1024
+      dropout: 0.2
+      final_dropout: true
+      interleave_self_attention: true
+      norm_type: ada_norm
+      positional_embeddings: null
+      attention_head_dim: 64
+      num_layers: 36
+      output_dim: 1024
+      cross_attention_dim: 1024
+      input_embedding_dim: 1024
+      num_attention_heads: 16
+    hidden_size: 1024
+    future_action_window_size: 15
+    action_hidden_dim: 1024
+    past_action_window_size: 0
+  obs_image_size:
+  - 224
+  - 224
+  dino:
+    dino_backbone: dinov2_vits14
+datasets:
+  vlm_data:
+    dataset_py: vlm_datasets
+    dataformat: llava_json
+    dataset_use: sharegpt4v_coco
+    eval_dataset: sharegpt4v_coco
+    data_flatten: false
+    base_interval: 2
+    max_pixels: 307200
+    min_pixels: 784
+    model_max_length: 2048
+    model_type: qwen2.5vl
+    per_device_batch_size: 4
+  vla_data:
+    dataset_py: lerobot_datasets
+    data_root_dir: ./playground/Datasets/OXE_LEROBOT_DATASET
+    data_mix: bridge_rt_1
+    action_type: delta_ee
+    CoT_prompt: Your task is {instruction}. To identify the key objects for your task.
+      Locate their bounding boxes in [x1,y1,x2,y2] format.
+    CoT_answer: bbox
+    default_image_resolution:
+    - 3
+    - 224
+    - 224
+    per_device_batch_size: 24
+    load_all_data_for_training: true
+    obs:
+    - image_0
+    image_size:
+    - 224
+    - 224
+    video_backend: torchvision_av
+trainer:
+  epochs: 100
+  max_train_steps: 100000
+  num_warmup_steps: 5000
+  save_interval: 10000
+  eval_interval: 1000
+  learning_rate:
+    base: 1.0e-05
+    qwen_vl_interface: 1.0e-05
+    action_model: 0.0001
+  lr_scheduler_type: cosine_with_min_lr
+  scheduler_specific_kwargs:
+    min_lr: 5.0e-07
+  freeze_modules: true
+  loss_scale:
+    vla: 1.0
+    vlm: 0.1
+  max_grad_norm: 1.0
+  warmup_ratio: 0.1
+  weight_decay: 0.0
+  logging_frequency: 1000
+  gradient_clipping: 1.0
+  gradient_accumulation_steps: 1
+  optimizer:
+    name: AdamW
+    betas:
+    - 0.9
+    - 0.95
+    eps: 1.0e-08
+    weight_decay: 1.0e-08
+  is_resume: true
+  resume_epoch: null
+  resume_step: null
+  enable_gradient_checkpointing: true
+  enable_mixed_precision_training: true
+  vla_data:
+    video_backend: pyav
+config_yaml: ./examples/SimplerEnv/train_files/starvla_cotrain_oxe.yaml
+output_dir: ./results/Checkpoints/0427_oxe_bridge_rt_1_QwenPI_v3

config.yaml

@@ -0,0 +1,72 @@
+datasets:
+  vla_data:
+    CoT_prompt: Your task is {instruction}. To identify the key objects for your task.
+      Locate their bounding boxes in [x1,y1,x2,y2] format.
+    data_mix: bridge_rt_1
+    data_root_dir: ./playground/Datasets/OXE_LEROBOT_DATASET
+    dataset_py: lerobot_datasets
+    image_size:
+    - 224
+    - 224
+    per_device_batch_size: 24
+    video_backend: torchvision_av
+framework:
+  action_model:
+    action_dim: 7
+    action_horizon: 16
+    add_pos_embed: true
+    diffusion_model_cfg:
+      action_dit_hidden_dim: 1024
+      attention_head_dim: 64
+      cross_attention_dim: 1024
+      dropout: 0.2
+      final_dropout: true
+      input_embedding_dim: 1024
+      interleave_self_attention: true
+      norm_type: ada_norm
+      num_attention_heads: 16
+      num_layers: 36
+      output_dim: 1024
+      positional_embeddings: null
+    max_seq_len: 1024
+    noise_beta_alpha: 1.5
+    noise_beta_beta: 1.0
+    noise_s: 0.999
+    num_inference_timesteps: 4
+    num_target_vision_tokens: 32
+    num_timestep_buckets: 1000
+    state_dim: 7
+  name: QwenPI_v3
+  qwenvl:
+    attn_implementation: flash_attention_2
+    base_vlm: /home/jye624/Models/Pretrained_models/Qwen3-VL-4B-Instruct
+    num_vl_layers: 36
+    vl_hidden_dim: 2560
+output_dir: ./results/Checkpoints/0427_oxe_bridge_rt_1_QwenPI_v3
+run_id: 0427_oxe_bridge_rt_1_QwenPI_v3
+run_root_dir: ./results/Checkpoints
+seed: 42
+trainer:
+  eval_interval: 1000
+  freeze_modules: true
+  gradient_clipping: 1.0
+  is_resume: true
+  learning_rate:
+    action_model: 0.0001
+    base: 1.0e-05
+    qwen_vl_interface: 1.0e-05
+  logging_frequency: 1000
+  lr_scheduler_type: cosine_with_min_lr
+  max_train_steps: 100000
+  num_warmup_steps: 5000
+  optimizer:
+    betas:
+    - 0.9
+    - 0.95
+    eps: 1.0e-08
+    weight_decay: 1.0e-08
+  save_interval: 10000
+  scheduler_specific_kwargs:
+    min_lr: 5.0e-07
+wandb_entity: jinhuiye
+wandb_project: starVLA_simplerEnv

dataset_statistics.json

@@ -0,0 +1,264 @@
+{
+  "oxe_bridge": {
+    "action": {
+      "mean": [
+        0.0001136629143729806,
+        6.556845619343221e-05,
+        -6.319578096736223e-05,
+        -7.192707562353462e-05,
+        -0.00019508649711497128,
+        0.00012040198635077104,
+        0.2882896661758423
+      ],
+      "std": [
+        0.006909778691848374,
+        0.009684059697605607,
+        0.008962926618867292,
+        0.020120852281989444,
+        0.021582520578222027,
+        0.05472376387860533,
+        0.4543604113297812
+      ],
+      "max": [
+        0.41691166162490845,
+        0.25864794850349426,
+        0.21218234300613403,
+        3.122201919555664,
+        1.8618112802505493,
+        6.272472858428955,
+        1.0
+      ],
+      "min": [
+        -0.4007510244846344,
+        -0.13874775171279907,
+        -0.22553899884223938,
+        -3.2010786533355713,
+        -1.8618112802505493,
+        -6.279075622558594,
+        0.0
+      ],
+      "q01": [
+        -0.028752606511116028,
+        -0.041702210046350954,
+        -0.026096698231995105,
+        -0.08052822157740593,
+        -0.09249736212193965,
+        -0.20738411962985992,
+        0.0
+      ],
+      "q99": [
+        0.02830690816044803,
+        0.04089860741049051,
+        0.04018005654215808,
+        0.08173405691981314,
+        0.07760896608233431,
+        0.20384809583425495,
+        1.0
+      ],
+      "mask": [
+        true,
+        true,
+        true,
+        true,
+        true,
+        true,
+        false
+      ]
+    },
+    "state": {
+      "mean": [
+        0.15471647679805756,
+        0.015362698584794998,
+        0.032221030443906784,
+        0.003244664054363966,
+        -0.03860040009021759,
+        0.053836673498153687,
+        0.0,
+        0.3540630638599396
+      ],
+      "std": [
+        0.16053484955349273,
+        0.06677967282050112,
+        0.048657228333866744,
+        0.09275332557452463,
+        0.1256707374939083,
+        0.4122273237945067,
+        0.0,
+        0.4330196238719565
+      ],
+      "max": [
+        0.5862360596656799,
+        0.4034728705883026,
+        0.3568263053894043,
+        1.3517684936523438,
+        1.570796251296997,
+        3.141204357147217,
+        0.0,
+        1.1121242046356201
+      ],
+      "min": [
+        -0.04167502000927925,
+        -0.3563207685947418,
+        -0.15537554025650024,
+        -3.141592502593994,
+        -1.4992541074752808,
+        -3.14153790473938,
+        0.0,
+        0.04637829214334488
+      ],
+      "q01": [
+        0.17102580681443214,
+        -0.16981234937906264,
+        -0.05563282176852226,
+        -0.36493386059999466,
+        -0.5418747025728226,
+        -1.3542919230461121,
+        0.0,
+        0.052190229296684265
+      ],
+      "q99": [
+        0.45322125554084775,
+        0.2354859386384485,
+        0.19489662453532214,
+        0.3779941478371616,
+        0.2756884342432019,
+        1.8500668883323654,
+        0.0,
+        1.0105689764022827
+      ]
+    },
+    "num_transitions": 1305714,
+    "num_trajectories": 53192
+  },
+  "oxe_rt1": {
+    "action": {
+      "mean": [
+        0.003493865951895714,
+        0.0031329391058534384,
+        -0.006312889512628317,
+        0.021666156128048897,
+        -0.002877477090805769,
+        0.0004563163092825562,
+        0.26771068572998047
+      ],
+      "std": [
+        0.04906474415809708,
+        0.04229872223842697,
+        0.05237597200308955,
+        0.11248535895810344,
+        0.09312825582599045,
+        0.10319124548215834,
+        0.441845103587406
+      ],
+      "max": [
+        2.9984593391418457,
+        22.09052848815918,
+        2.7507524490356445,
+        1.570636510848999,
+        1.5321086645126343,
+        1.5691522359848022,
+        1.0
+      ],
+      "min": [
+        -2.0204520225524902,
+        -5.497899532318115,
+        -2.031663417816162,
+        -1.569917917251587,
+        -1.569892168045044,
+        -1.570419430732727,
+        0.0
+      ],
+      "q01": [
+        -0.2245360141992569,
+        -0.14820106267929076,
+        -0.23158982083201407,
+        -0.3517777299880981,
+        -0.4192772650718689,
+        -0.43643518328666686,
+        0.0
+      ],
+      "q99": [
+        0.1782463169097901,
+        0.14938431486487408,
+        0.21841673687100444,
+        0.5892668181657792,
+        0.3527275875210766,
+        0.44796794503927273,
+        1.0
+      ],
+      "mask": [
+        true,
+        true,
+        true,
+        true,
+        true,
+        true,
+        false
+      ]
+    },
+    "state": {
+      "mean": [
+        0.279946893453598,
+        -0.04167007654905319,
+        0.3885466456413269,
+        0.21306714415550232,
+        -0.12402169406414032,
+        0.24756763875484467,
+        0.04633009061217308,
+        0.10487376153469086
+      ],
+      "std": [
+        0.29342589017967113,
+        0.09173989695598195,
+        0.4256945884267932,
+        0.3861626196491199,
+        0.38314687041941975,
+        0.4443359860932721,
+        0.12639273126219563,
+        0.22122626649401464
+      ],
+      "max": [
+        1.0534898042678833,
+        0.48018959164619446,
+        1.6896663904190063,
+        1.0,
+        0.9999993443489075,
+        0.9999874830245972,
+        0.9554369449615479,
+        0.9914546012878418
+      ],
+      "min": [
+        -0.4436439275741577,
+        -0.9970501065254211,
+        -0.006579156965017319,
+        0.0,
+        -0.8643477559089661,
+        -0.7079970240592957,
+        -0.7688722014427185,
+        -0.4999994933605194
+      ],
+      "q01": [
+        0.32481366634368897,
+        -0.2833433499932289,
+        0.14107060477137565,
+        0.0,
+        -0.6864742285013199,
+        -0.6808923971652985,
+        -0.36045609444379806,
+        -0.45438114255666734
+      ],
+      "q99": [
+        0.8750162518024447,
+        0.21247095301747337,
+        1.0727114248275758,
+        1.0,
+        0.937787775397301,
+        0.9563058441877368,
+        0.4599010077118876,
+        0.7216041576862335
+      ]
+    },
+    "num_transitions": 3786152,
+    "num_trajectories": 87212
+  }
+}

run_oxe_train.sh

@@ -0,0 +1,136 @@
+
+#!/bin/bash
+# Usage: run on a compute node with GPUs
+#   srun --jobid=<JOB_ID> --overlap --pty bash examples/SimplerEnv/train_files/run_oxe_train.sh
+set -e
+
+# === Conda setup ===
+source /cm/shared/apps/Anaconda3/2023.09-0/etc/profile.d/conda.sh
+conda activate starVLA
+
+# === CUDA setup ===
+for cuda_path in /usr/local/cuda /usr/local/cuda-12 /usr/local/cuda-12.4; do
+  if [ -x "${cuda_path}/bin/nvcc" ]; then
+    export CUDA_HOME="${cuda_path}"
+    export PATH="${cuda_path}/bin:${PATH}"
+    export LD_LIBRARY_PATH="${cuda_path}/lib64:${LD_LIBRARY_PATH:-}"
+    break
+  fi
+done
+
+# nvcc wrapper fallback
+if ! nvcc --version 2>&1 | grep -q "release"; then
+  _WRAPPER_DIR="${CONDA_PREFIX}/cuda_compat/bin"
+  mkdir -p "${_WRAPPER_DIR}" 2>/dev/null || true
+  _TORCH_CUDA_VER=$(python -c "import torch; print(torch.version.cuda)" 2>/dev/null || echo "12.4")
+  _MAJOR=$(echo "${_TORCH_CUDA_VER}" | cut -d. -f1)
+  _MINOR=$(echo "${_TORCH_CUDA_VER}" | cut -d. -f2)
+  cat > "${_WRAPPER_DIR}/nvcc" << NVCC_EOF
+#!/bin/bash
+echo "nvcc: NVIDIA (R) Cuda compiler driver"
+echo "Cuda compilation tools, release ${_MAJOR}.${_MINOR}, V${_TORCH_CUDA_VER}"
+NVCC_EOF
+  chmod +x "${_WRAPPER_DIR}/nvcc"
+  export PATH="${_WRAPPER_DIR}:${PATH}"
+  export CUDA_HOME="${CONDA_PREFIX}/cuda_compat"
+  echo "[INFO] Created nvcc wrapper: CUDA ${_TORCH_CUDA_VER}"
+fi
+
+echo "[INFO] CUDA_HOME=$CUDA_HOME"
+nvcc --version 2>/dev/null || echo "[WARN] nvcc not found"
+
+
+# used for check save when communication
+export NCCL_BLOCKING_WAIT=1
+export NCCL_ASYNC_ERROR_HANDLING=1
+export NCCL_TIMEOUT=10000  # timeout set to 1 hour (unit: seconds)
+export NCCL_SOCKET_TIMEOUT_MS=360000
+###########################################################################################
+# === Please modify the following paths according to your environment ===
+cd /home/jye624/Projcets/starVLA
+
+Framework_name=QwenPI_v3
+freeze_module_list=''
+base_vlm=/home/jye624/Models/Pretrained_models/Qwen3-VL-4B-Instruct
+config_yaml=./examples/SimplerEnv/train_files/starvla_cotrain_oxe.yaml
+oxe_data_root=./playground/Datasets/OXE_LEROBOT_DATASET
+data_mix=bridge_rt_1
+run_root_dir=./results/Checkpoints
+
+run_id=0427_oxe_${data_mix}_${Framework_name}
+# === End of environment variable configuration ===
+###########################################################################################
+
+
+# export WANDB_MODE=disabled
+
+output_dir=${run_root_dir}/${run_id}
+mkdir -p ${output_dir}
+# mv this script to the output dir
+cp $0 ${output_dir}/
+
+num_processes=${NUM_PROCESSES:-$(nvidia-smi -L | wc -l)}
+attn_implementation=${ATTN_IMPLEMENTATION:-flash_attention_2}
+accelerate_config_file=${ACCELERATE_CONFIG_FILE:-starVLA/config/deepseeds/deepspeed_zero2.yaml}
+# Use port 0 to let the system auto-select a free port, avoiding conflicts when multiple jobs land on the same node
+main_process_port=${MAIN_PROCESS_PORT:-0}
+
+export WANDB_API_KEY=${WANDB_API_KEY:-943ecb8d26fc2b3879cbc2d667414974906aebb9}
+export PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True
+
+
+# Fix: ensure vonneumann1 group is active for NFS file access on compute nodes
+if id -nG 2>/dev/null | grep -qw vonneumann1; then
+  export _STARVLA_GROUP_FIX=vonneumann1
+  echo "[INFO] Group vonneumann1 detected, using newgrp for NFS access"
+fi
+
+# Resolve conda activation command for sub-shells (sg spawns a new shell)
+CONDA_BASE=$(conda info --base 2>/dev/null || echo "${CONDA_PREFIX%/envs/*}")
+CONDA_INIT="source ${CONDA_BASE}/etc/profile.d/conda.sh && conda activate ${CONDA_DEFAULT_ENV:-starVLA}"
+
+sg vonneumann1 -c "
+${CONDA_INIT} && \
+accelerate launch \
+  --config_file ${accelerate_config_file} \
+  --num_processes ${num_processes} \
+  starVLA/training/train_starvla.py \
+  --config_yaml ${config_yaml} \
+  --framework.name ${Framework_name} \
+  --framework.qwenvl.base_vlm ${base_vlm} \
+  --datasets.vla_data.data_root_dir ${oxe_data_root} \
+  --datasets.vla_data.data_mix ${data_mix} \
+  --datasets.vla_data.per_device_batch_size 24 \
+  --trainer.vla_data.video_backend pyav \
+  --framework.qwenvl.attn_implementation ${attn_implementation} \
+  --trainer.freeze_modules ${freeze_module_list} \
+  --trainer.max_train_steps 100000 \
+  --trainer.save_interval 10000 \
+  --trainer.logging_frequency 1000 \
+  --trainer.eval_interval 1000 \
+  --run_root_dir ${run_root_dir} \
+  --run_id ${run_id} \
+  --trainer.is_resume True \
+  --wandb_project starVLA_simplerEnv \
+  --wandb_entity jinhuiye
+"
+
+
+
+##### Multi-Server Multi-GPU training script #####
+  # accelerate launch \
+  #   --config_file starVLA/config/deepseeds/deepspeed_zero2.yaml \
+  #   --main_process_ip $MASTER_ADDR \
+  #   --main_process_port $MASTER_PORT \
+  #   --machine_rank $SLURM_PROCID \
+  #   --num_machines $SLURM_NNODES \
+  #   --num_processes=${TOTAL_GPUS} \
+  #   starVLA/training/train_starvla.py \
+  #   --config_yaml ${config_yaml} \
+  #   --framework.name ${Framework_name} \
+  #   --framework.qwenvl.base_vlm ${base_vlm} \
+  #   --run_root_dir ${run_root_dir} \
+  #   --run_id ${run_id} \
+  #   --wandb_project your_project \
+  #   --wandb_entity your_name
+##### Multi-Server Multi-GPU training script #####

success_summary/raw_success.txt

@@ -0,0 +1,29 @@
+steps_10000_pytorch_model_infer_PutCarrotOnPlateInScene-v0.log.run1 → Average success: 0.125
+steps_10000_pytorch_model_infer_PutEggplantInBasketScene-v0.log.run1 → Average success: 0.2916666666666667
+steps_10000_pytorch_model_infer_PutSpoonOnTableClothInScene-v0.log.run1 → Average success: 0.25
+steps_10000_pytorch_model_infer_StackGreenCubeOnYellowCubeBakedTexInScene-v0.log.run1 → Average success: 0.125
+steps_20000_pytorch_model_infer_PutCarrotOnPlateInScene-v0.log.run1 → Average success: 0.375
+steps_20000_pytorch_model_infer_PutEggplantInBasketScene-v0.log.run1 → Average success: 0.9166666666666666
+steps_20000_pytorch_model_infer_PutSpoonOnTableClothInScene-v0.log.run1 → Average success: 0.4166666666666667
+steps_20000_pytorch_model_infer_StackGreenCubeOnYellowCubeBakedTexInScene-v0.log.run1 → Average success: 0.125
+steps_30000_pytorch_model_infer_PutCarrotOnPlateInScene-v0.log.run1 → Average success: 0.25
+steps_30000_pytorch_model_infer_PutEggplantInBasketScene-v0.log.run1 → Average success: 0.6666666666666666
+steps_30000_pytorch_model_infer_PutSpoonOnTableClothInScene-v0.log.run1 → Average success: 0.625
+steps_30000_pytorch_model_infer_StackGreenCubeOnYellowCubeBakedTexInScene-v0.log.run1 → Average success: 0.08333333333333333
+steps_40000_pytorch_model_infer_PutCarrotOnPlateInScene-v0.log.run1 → Average success: 0.625
+steps_40000_pytorch_model_infer_PutCarrotOnPlateInScene-v0.log.run2 → Average success: 0.75
+steps_40000_pytorch_model_infer_PutEggplantInBasketScene-v0.log.run1 → Average success: 0.875
+steps_40000_pytorch_model_infer_PutEggplantInBasketScene-v0.log.run2 → Average success: 0.9583333333333334
+steps_40000_pytorch_model_infer_PutSpoonOnTableClothInScene-v0.log.run1 → Average success: 0.75
+steps_40000_pytorch_model_infer_PutSpoonOnTableClothInScene-v0.log.run2 → Average success: 0.75
+steps_40000_pytorch_model_infer_StackGreenCubeOnYellowCubeBakedTexInScene-v0.log.run1 → Average success: 0.375
+steps_40000_pytorch_model_infer_StackGreenCubeOnYellowCubeBakedTexInScene-v0.log.run2 → Average success: 0.2916666666666667
+steps_50000_pytorch_model_infer_PutCarrotOnPlateInScene-v0.log.run1 → Average success: 0.625
+steps_50000_pytorch_model_infer_PutEggplantInBasketScene-v0.log.run1 → Average success: 1.0
+steps_50000_pytorch_model_infer_PutSpoonOnTableClothInScene-v0.log.run1 → Average success: 0.7916666666666666
+steps_50000_pytorch_model_infer_StackGreenCubeOnYellowCubeBakedTexInScene-v0.log.run1 → Average success: 0.375
+steps_60000_pytorch_model_infer_PutCarrotOnPlateInScene-v0.log.run1 → Average success: 0.6666666666666666
+steps_60000_pytorch_model_infer_PutEggplantInBasketScene-v0.log.run1 → Average success: 1.0
+steps_60000_pytorch_model_infer_PutSpoonOnTableClothInScene-v0.log.run1 → Average success: 0.75
+steps_60000_pytorch_model_infer_StackGreenCubeOnYellowCubeBakedTexInScene-v0.log.run1 → Average success: 0.16666666666666666
+

success_summary/success_summary.csv

@@ -0,0 +1,7 @@
+step,PutCarrotOnPlateInScene,PutEggplantInBasketScene,PutSpoonOnTableClothInScene,StackGreenCubeOnYellowCubeBakedTexInScene,Average Across Tasks
+10000,0.125,0.2916666666666667,0.25,0.125,0.19791666666666669
+20000,0.375,0.9166666666666666,0.4166666666666667,0.125,0.4583333333333333
+30000,0.25,0.6666666666666666,0.625,0.08333333333333333,0.40624999999999994
+40000,0.6875,0.9166666666666667,0.75,0.33333333333333337,0.6718750000000001
+50000,0.625,1.0,0.7916666666666666,0.375,0.6979166666666666
+60000,0.6666666666666666,1.0,0.75,0.16666666666666666,0.6458333333333333

summary.jsonl

@@ -0,0 +1,7 @@
+{"steps": 10}
+{"steps": 10000}
+{"steps": 20000}
+{"steps": 30000}
+{"steps": 40000}
+{"steps": 50000}
+{"steps": 60000}

wandb/wandb/run-20260426_011111-enstjn5q/files/output.log

@@ -0,0 +1,322 @@
+04/26 [01:11:14] INFO     | >> ***** Training Configuration *****                      ]8;id=471029;file:///project/vonneumann1/jye624/Projcets/starVLA/starVLA/training/train_starvla.py\train_starvla.py]8;;\:]8;id=617889;file:///project/vonneumann1/jye624/Projcets/starVLA/starVLA/training/train_starvla.py#355\355]8;;\
+                 INFO     | >>   Total optimization steps = 100000                     ]8;id=844962;file:///project/vonneumann1/jye624/Projcets/starVLA/starVLA/training/train_starvla.py\train_starvla.py]8;;\:]8;id=167414;file:///project/vonneumann1/jye624/Projcets/starVLA/starVLA/training/train_starvla.py#356\356]8;;\
+                 INFO     | >>   Per device batch size = 24                            ]8;id=225772;file:///project/vonneumann1/jye624/Projcets/starVLA/starVLA/training/train_starvla.py\train_starvla.py]8;;\:]8;id=800581;file:///project/vonneumann1/jye624/Projcets/starVLA/starVLA/training/train_starvla.py#357\357]8;;\
+                 INFO     | >>   Gradient accumulation steps = 1                       ]8;id=376417;file:///project/vonneumann1/jye624/Projcets/starVLA/starVLA/training/train_starvla.py\train_starvla.py]8;;\:]8;id=888662;file:///project/vonneumann1/jye624/Projcets/starVLA/starVLA/training/train_starvla.py#358\358]8;;\
+                 INFO     | >>   Total batch size = 192                                ]8;id=765179;file:///project/vonneumann1/jye624/Projcets/starVLA/starVLA/training/train_starvla.py\train_starvla.py]8;;\:]8;id=481741;file:///project/vonneumann1/jye624/Projcets/starVLA/starVLA/training/train_starvla.py#359\359]8;;\
+  7%|█▌                    | 7000/100000 [2:12:44<29:30:36,  1.14s/it, data_times=0.001, model_times=1.090]
+04/26 [01:30:39] INFO     | >> Step 1000, Loss: {'action_dit_loss':                    ]8;id=578856;file:///project/vonneumann1/jye624/Projcets/starVLA/starVLA/training/train_starvla.py\train_starvla.py]8;;\:]8;id=307419;file:///project/vonneumann1/jye624/Projcets/starVLA/starVLA/training/train_starvla.py#271\271]8;;\
+                          0.41287872195243835, 'mse_score': 0.006587725310098557,                          
+                          'data_time': 0.023512821993790567, 'model_time':                                 
+                          1.1061716680414975, 'learning_rate': 2.0000000000000003e-06,                     
+                          'epoch': 0.03})                                                                  
+04/26 [01:49:32] INFO     | >> Step 2000, Loss: {'action_dit_loss':                    ]8;id=903565;file:///project/vonneumann1/jye624/Projcets/starVLA/starVLA/training/train_starvla.py\train_starvla.py]8;;\:]8;id=379201;file:///project/vonneumann1/jye624/Projcets/starVLA/starVLA/training/train_starvla.py#271\271]8;;\
+                          0.31383267045021057, 'mse_score': 0.006785546739896138,                          
+                          'data_time': 0.0012596730375662446, 'model_time':                                
+                          1.0688785739475861, 'learning_rate': 4.000000000000001e-06,                      
+                          'epoch': 0.05})                                                                  
+04/26 [02:08:28] INFO     | >> Step 3000, Loss: {'action_dit_loss':                    ]8;id=48050;file:///project/vonneumann1/jye624/Projcets/starVLA/starVLA/training/train_starvla.py\train_starvla.py]8;;\:]8;id=693384;file:///project/vonneumann1/jye624/Projcets/starVLA/starVLA/training/train_starvla.py#271\271]8;;\
+                          0.19595405459403992, 'mse_score': 0.00602614666734423,                           
+                          'data_time': 0.0005622160388156772, 'model_time':                                
+                          1.089529522927478, 'learning_rate': 6e-06, 'epoch': 0.08})                       
+04/26 [02:27:21] INFO     | >> Step 4000, Loss: {'action_dit_loss':                    ]8;id=896865;file:///project/vonneumann1/jye624/Projcets/starVLA/starVLA/training/train_starvla.py\train_starvla.py]8;;\:]8;id=244098;file:///project/vonneumann1/jye624/Projcets/starVLA/starVLA/training/train_starvla.py#271\271]8;;\
+                          0.23803943395614624, 'mse_score': 0.006116567623047601,                          
+                          'data_time': 0.014886713004671037, 'model_time':                                 
+                          1.0681698899716139, 'learning_rate': 8.000000000000001e-06,                      
+                          'epoch': 0.1})                                                                   
+04/26 [02:46:14] INFO     | >> Step 5000, Loss: {'action_dit_loss':                    ]8;id=475435;file:///project/vonneumann1/jye624/Projcets/starVLA/starVLA/training/train_starvla.py\train_starvla.py]8;;\:]8;id=666563;file:///project/vonneumann1/jye624/Projcets/starVLA/starVLA/training/train_starvla.py#271\271]8;;\
+                          0.21792519092559814, 'mse_score': 0.006198919245174953,                          
+                          'data_time': 0.01555767108220607, 'model_time':                                  
+                          1.1287320599658415, 'learning_rate': 1e-05, 'epoch': 0.13})                      
+04/26 [03:05:06] INFO     | >> Step 6000, Loss: {'action_dit_loss':                    ]8;id=372528;file:///project/vonneumann1/jye624/Projcets/starVLA/starVLA/training/train_starvla.py\train_starvla.py]8;;\:]8;id=219684;file:///project/vonneumann1/jye624/Projcets/starVLA/starVLA/training/train_starvla.py#271\271]8;;\
+                          0.2961162328720093, 'mse_score': 0.0054819346183822264,                          
+                          'data_time': 0.0010915560415014625, 'model_time':                                
+                          1.0691960570402443, 'learning_rate': 9.9974029723694e-06,                        
+                          'epoch': 0.15})                                                                  
+04/26 [03:23:59] INFO     | >> Step 7000, Loss: {'action_dit_loss':                    ]8;id=716751;file:///project/vonneumann1/jye624/Projcets/starVLA/starVLA/training/train_starvla.py\train_starvla.py]8;;\:]8;id=679514;file:///project/vonneumann1/jye624/Projcets/starVLA/starVLA/training/train_starvla.py#271\271]8;;\
+                          0.16125820577144623, 'mse_score': 0.005778901633762178,                          
+                          'data_time': 0.001105302944779396, 'model_time':                                 
+                          1.0903944559395313, 'learning_rate': 9.98961472928918e-06,                       
+                          'epoch': 0.18})                                                                  
+04/26 [03:42:55] INFO     | >> Step 8000, Loss: {'action_dit_loss':                    ]8;id=560086;file:///project/vonneumann1/jye624/Projcets/starVLA/starVLA/training/train_starvla.py\train_starvla.py]8;;\:]8;id=764544;file:///project/vonneumann1/jye624/Projcets/starVLA/starVLA/training/train_starvla.py#271\271]8;;\
+                          0.24051445722579956, 'mse_score': 0.006084372599919637,                          
+                          'data_time': 0.01449370407499373, 'model_time':                                  
+                          1.0696819460717961, 'learning_rate': 9.976643787088806e-06,                      
+                          'epoch': 0.2})                                                                   
+04/26 [04:01:45] INFO     | >> Step 9000, Loss: {'action_dit_loss':                    ]8;id=283060;file:///project/vonneumann1/jye624/Projcets/starVLA/starVLA/training/train_starvla.py\train_starvla.py]8;;\:]8;id=970342;file:///project/vonneumann1/jye624/Projcets/starVLA/starVLA/training/train_starvla.py#271\271]8;;\
+                          0.20358465611934662, 'mse_score': 0.005045601299830845,                          
+                          'data_time': 0.016380950924940407, 'model_time':                                 
+                          1.0907069341046736, 'learning_rate': 9.958504329303132e-06,                      
+                          'epoch': 0.23})                                                                  
+04/26 [04:20:42] INFO     | >> Step 10000, Loss: {'action_dit_loss':                   ]8;id=717870;file:///project/vonneumann1/jye624/Projcets/starVLA/starVLA/training/train_starvla.py\train_starvla.py]8;;\:]8;id=340035;file:///project/vonneumann1/jye624/Projcets/starVLA/starVLA/training/train_starvla.py#271\271]8;;\
+                          0.21283088624477386, 'mse_score': 0.0054307544515246435,                         
+                          'data_time': 0.0010308929486200213, 'model_time':                                
+                          1.0539058269932866, 'learning_rate': 9.935216191162932e-06,                      
+                          'epoch': 0.25})                                                                  
+✅ Checkpoint saved at ./results/Checkpoints/0427_oxe_bridge_rt_1_QwenPI_v3/checkpoints/steps_10000
+04/26 [04:21:16] INFO     | >> 📊 Saving accessed configuration...                     ]8;id=861722;file:///project/vonneumann1/jye624/Projcets/starVLA/starVLA/training/train_starvla.py\train_starvla.py]8;;\:]8;id=33659;file:///project/vonneumann1/jye624/Projcets/starVLA/starVLA/training/train_starvla.py#258\258]8;;\
+                 INFO     | >> ✅ Configuration files saved                            ]8;id=221231;file:///project/vonneumann1/jye624/Projcets/starVLA/starVLA/training/train_starvla.py\train_starvla.py]8;;\:]8;id=957492;file:///project/vonneumann1/jye624/Projcets/starVLA/starVLA/training/train_starvla.py#261\261]8;;\
+04/26 [04:40:08] INFO     | >> Step 11000, Loss: {'action_dit_loss':                   ]8;id=329963;file:///project/vonneumann1/jye624/Projcets/starVLA/starVLA/training/train_starvla.py\train_starvla.py]8;;\:]8;id=222955;file:///project/vonneumann1/jye624/Projcets/starVLA/starVLA/training/train_starvla.py#271\271]8;;\
+                          0.1573459506034851, 'mse_score': 0.005605340358756837,                           
+                          'data_time': 0.0005969370249658823, 'model_time':                                
+                          1.1335041610291228, 'learning_rate': 9.906804837905345e-06,                      
+                          'epoch': 0.28})                                                                  
+04/26 [04:59:02] INFO     | >> Step 12000, Loss: {'action_dit_loss':                   ]8;id=958972;file:///project/vonneumann1/jye624/Projcets/starVLA/starVLA/training/train_starvla.py\train_starvla.py]8;;\:]8;id=674079;file:///project/vonneumann1/jye624/Projcets/starVLA/starVLA/training/train_starvla.py#271\271]8;;\
+                          0.2110031098127365, 'mse_score': 0.005528707234632401,                           
+                          'data_time': 0.01464720896910876, 'model_time':                                  
+                          1.1476604999043047, 'learning_rate': 9.873301336928028e-06,                      
+                          'epoch': 0.3})                                                                   
+04/26 [05:17:59] INFO     | >> Step 13000, Loss: {'action_dit_loss':                   ]8;id=258607;file:///project/vonneumann1/jye624/Projcets/starVLA/starVLA/training/train_starvla.py\train_starvla.py]8;;\:]8;id=781177;file:///project/vonneumann1/jye624/Projcets/starVLA/starVLA/training/train_starvla.py#271\271]8;;\
+                          0.18376827239990234, 'mse_score': 0.004833251592658815,                          
+                          'data_time': 0.0171306460397318, 'model_time':                                   
+                          1.0867095300927758, 'learning_rate': 9.834742323817407e-06,                      
+                          'epoch': 0.33})                                                                  
+04/26 [05:36:50] INFO     | >> Step 14000, Loss: {'action_dit_loss':                   ]8;id=612982;file:///project/vonneumann1/jye624/Projcets/starVLA/starVLA/training/train_starvla.py\train_starvla.py]8;;\:]8;id=449245;file:///project/vonneumann1/jye624/Projcets/starVLA/starVLA/training/train_starvla.py#271\271]8;;\
+                          0.15762470662593842, 'mse_score': 0.005961580645470392,                          
+                          'data_time': 0.0005706310039386153, 'model_time':                                
+                          1.0944029649253935, 'learning_rate': 9.7911699622882e-06,                        
+                          'epoch': 0.35})                                                                  
+04/26 [05:55:42] INFO     | >> Step 15000, Loss: {'action_dit_loss':                   ]8;id=229974;file:///project/vonneumann1/jye624/Projcets/starVLA/starVLA/training/train_starvla.py\train_starvla.py]8;;\:]8;id=145051;file:///project/vonneumann1/jye624/Projcets/starVLA/starVLA/training/train_starvla.py#271\271]8;;\
+                          0.14939481019973755, 'mse_score': 0.004519839726743244,                          
+                          'data_time': 0.0009296479402109981, 'model_time':                                
+                          1.1203379810322076, 'learning_rate': 9.742631898078014e-06,                      
+                          'epoch': 0.38})                                                                  
+04/26 [06:14:38] INFO     | >> Step 16000, Loss: {'action_dit_loss':                   ]8;id=49405;file:///project/vonneumann1/jye624/Projcets/starVLA/starVLA/training/train_starvla.py\train_starvla.py]8;;\:]8;id=902931;file:///project/vonneumann1/jye624/Projcets/starVLA/starVLA/training/train_starvla.py#271\271]8;;\
+                          0.14374569058418274, 'mse_score': 0.0050108180869193305,                         
+                          'data_time': 0.014823429053649306, 'model_time':                                 
+                          1.0832276189466938, 'learning_rate': 9.68918120684744e-06,                       
+                          'epoch': 0.41})                                                                  
+04/26 [06:33:30] INFO     | >> Step 17000, Loss: {'action_dit_loss':                   ]8;id=830555;file:///project/vonneumann1/jye624/Projcets/starVLA/starVLA/training/train_starvla.py\train_starvla.py]8;;\:]8;id=713536;file:///project/vonneumann1/jye624/Projcets/starVLA/starVLA/training/train_starvla.py#271\271]8;;\
+                          0.15882907807826996, 'mse_score': 0.004706428874106635,                          
+                          'data_time': 0.016839208896271884, 'model_time':                                 
+                          1.0935403839685023, 'learning_rate': 9.630876336142578e-06,                      
+                          'epoch': 0.43})                                                                  
+04/26 [06:52:25] INFO     | >> Step 18000, Loss: {'action_dit_loss':                   ]8;id=400156;file:///project/vonneumann1/jye624/Projcets/starVLA/starVLA/training/train_starvla.py\train_starvla.py]8;;\:]8;id=624834;file:///project/vonneumann1/jye624/Projcets/starVLA/starVLA/training/train_starvla.py#271\271]8;;\
+                          0.19779755175113678, 'mse_score': 0.005086257344200497,                          
+                          'data_time': 0.0011988269397988915, 'model_time':                                
+                          1.1007598140276968, 'learning_rate': 9.567781041483523e-06,                      
+                          'epoch': 0.46})                                                                  
+04/26 [07:11:21] INFO     | >> Step 19000, Loss: {'action_dit_loss':                   ]8;id=902592;file:///project/vonneumann1/jye624/Projcets/starVLA/starVLA/training/train_starvla.py\train_starvla.py]8;;\:]8;id=988210;file:///project/vonneumann1/jye624/Projcets/starVLA/starVLA/training/train_starvla.py#271\271]8;;\
+                          0.2005825936794281, 'mse_score': 0.005249518368925367,                           
+                          'data_time': 0.0006849960191175342, 'model_time':                                
+                          1.1035226460080594, 'learning_rate': 9.499964316648628e-06,                      
+                          'epoch': 0.48})                                                                  
+04/26 [07:30:14] INFO     | >> Step 20000, Loss: {'action_dit_loss':                   ]8;id=714825;file:///project/vonneumann1/jye624/Projcets/starVLA/starVLA/training/train_starvla.py\train_starvla.py]8;;\:]8;id=927767;file:///project/vonneumann1/jye624/Projcets/starVLA/starVLA/training/train_starvla.py#271\271]8;;\
+                          0.23199643194675446, 'mse_score': 0.004106811114719936,                          
+                          'data_time': 0.013686573947779834, 'model_time':                                 
+                          1.1025469119194895, 'learning_rate': 9.427500318230823e-06,                      
+                          'epoch': 0.51})                                                                  
+✅ Checkpoint saved at ./results/Checkpoints/0427_oxe_bridge_rt_1_QwenPI_v3/checkpoints/steps_20000
+04/26 [07:30:37] INFO     | >> 📊 Saving accessed configuration...                     ]8;id=672097;file:///project/vonneumann1/jye624/Projcets/starVLA/starVLA/training/train_starvla.py\train_starvla.py]8;;\:]8;id=356699;file:///project/vonneumann1/jye624/Projcets/starVLA/starVLA/training/train_starvla.py#258\258]8;;\
+                 INFO     | >> ✅ Configuration files saved                            ]8;id=475763;file:///project/vonneumann1/jye624/Projcets/starVLA/starVLA/training/train_starvla.py\train_starvla.py]8;;\:]8;id=3402;file:///project/vonneumann1/jye624/Projcets/starVLA/starVLA/training/train_starvla.py#261\261]8;;\
+04/26 [07:49:30] INFO     | >> Step 21000, Loss: {'action_dit_loss':                   ]8;id=524902;file:///project/vonneumann1/jye624/Projcets/starVLA/starVLA/training/train_starvla.py\train_starvla.py]8;;\:]8;id=798975;file:///project/vonneumann1/jye624/Projcets/starVLA/starVLA/training/train_starvla.py#271\271]8;;\
+                          0.19188104569911957, 'mse_score': 0.004567236772605351,                          
+                          'data_time': 0.014397618011571467, 'model_time':                                 
+                          1.1003698320128024, 'learning_rate': 9.350468284548478e-06,                      
+                          'epoch': 0.53})                                                                  
+04/26 [08:08:22] INFO     | >> Step 22000, Loss: {'action_dit_loss':                   ]8;id=912804;file:///project/vonneumann1/jye624/Projcets/starVLA/starVLA/training/train_starvla.py\train_starvla.py]8;;\:]8;id=655674;file:///project/vonneumann1/jye624/Projcets/starVLA/starVLA/training/train_starvla.py#271\271]8;;\
+                          0.16441768407821655, 'mse_score': 0.0050850169999258855,                         
+                          'data_time': 0.001025355071760714, 'model_time':                                 
+                          1.085655941045843, 'learning_rate': 9.268952448999444e-06,                       
+                          'epoch': 0.56})                                                                  
+04/26 [08:27:16] INFO     | >> Step 23000, Loss: {'action_dit_loss':                   ]8;id=638551;file:///project/vonneumann1/jye624/Projcets/starVLA/starVLA/training/train_starvla.py\train_starvla.py]8;;\:]8;id=208573;file:///project/vonneumann1/jye624/Projcets/starVLA/starVLA/training/train_starvla.py#271\271]8;;\
+                          0.1905813217163086, 'mse_score': 0.004551701957271213,                           
+                          'data_time': 0.0007312960224226117, 'model_time':                                
+                          1.1118732010945678, 'learning_rate': 9.183041947953097e-06,                      
+                          'epoch': 0.58})                                                                  
+04/26 [08:46:10] INFO     | >> Step 24000, Loss: {'action_dit_loss':                   ]8;id=565579;file:///project/vonneumann1/jye624/Projcets/starVLA/starVLA/training/train_starvla.py\train_starvla.py]8;;\:]8;id=999816;file:///project/vonneumann1/jye624/Projcets/starVLA/starVLA/training/train_starvla.py#271\271]8;;\
+                          0.18795546889305115, 'mse_score': 0.005052322433108375,                          
+                          'data_time': 0.015225342009216547, 'model_time':                                 
+                          1.0696408179355785, 'learning_rate': 9.092830723281e-06,                         
+                          'epoch': 0.61})                                                                  
+04/26 [09:05:04] INFO     | >> Step 25000, Loss: {'action_dit_loss':                   ]8;id=599;file:///project/vonneumann1/jye624/Projcets/starVLA/starVLA/training/train_starvla.py\train_starvla.py]8;;\:]8;id=628038;file:///project/vonneumann1/jye624/Projcets/starVLA/starVLA/training/train_starvla.py#271\271]8;;\
+                          0.19811630249023438, 'mse_score': 0.005336226097175053,                          
+                          'data_time': 0.016753150965087116, 'model_time':                                 
+                          1.1177711630007252, 'learning_rate': 8.998417419632872e-06,                      
+                          'epoch': 0.63})                                                                  
+04/26 [09:23:57] INFO     | >> Step 26000, Loss: {'action_dit_loss':                   ]8;id=974230;file:///project/vonneumann1/jye624/Projcets/starVLA/starVLA/training/train_starvla.py\train_starvla.py]8;;\:]8;id=380612;file:///project/vonneumann1/jye624/Projcets/starVLA/starVLA/training/train_starvla.py#271\271]8;;\
+                          0.2119632065296173, 'mse_score': 0.004771212381975991,                           
+                          'data_time': 0.0006861590081825852, 'model_time':                                
+                          1.1613712280523032, 'learning_rate': 8.899905276570082e-06,                      
+                          'epoch': 0.66})                                                                  
+04/26 [09:42:50] INFO     | >> Step 27000, Loss: {'action_dit_loss':                   ]8;id=251083;file:///project/vonneumann1/jye624/Projcets/starVLA/starVLA/training/train_starvla.py\train_starvla.py]8;;\:]8;id=60738;file:///project/vonneumann1/jye624/Projcets/starVLA/starVLA/training/train_starvla.py#271\271]8;;\
+                          0.1544923633337021, 'mse_score': 0.004587906102339427,                           
+                          'data_time': 0.0006635260069742799, 'model_time':                                
+                          1.1337980959797278, 'learning_rate': 8.797402015674731e-06,                      
+                          'epoch': 0.68})                                                                  
+04/26 [10:01:45] INFO     | >> Step 28000, Loss: {'action_dit_loss':                   ]8;id=82582;file:///project/vonneumann1/jye624/Projcets/starVLA/starVLA/training/train_starvla.py\train_starvla.py]8;;\:]8;id=89814;file:///project/vonneumann1/jye624/Projcets/starVLA/starVLA/training/train_starvla.py#271\271]8;;\
+                          0.11156129091978073, 'mse_score': 0.00432542676017398,                           
+                          'data_time': 0.016900848015211523, 'model_time':                                 
+                          1.151624141028151, 'learning_rate': 8.691019722757675e-06,                       
+                          'epoch': 0.71})                                                                  
+04/26 [10:20:38] INFO     | >> Step 29000, Loss: {'action_dit_loss':                   ]8;id=797549;file:///project/vonneumann1/jye624/Projcets/starVLA/starVLA/training/train_starvla.py\train_starvla.py]8;;\:]8;id=558582;file:///project/vonneumann1/jye624/Projcets/starVLA/starVLA/training/train_starvla.py#271\271]8;;\
+                          0.16831044852733612, 'mse_score': 0.004411907777899788,                          
+                          'data_time': 0.016713427961803973, 'model_time':                                 
+                          1.0871766429627314, 'learning_rate': 8.580874725294321e-06,                      
+                          'epoch': 0.74})                                                                  
+04/26 [10:39:28] INFO     | >> Step 30000, Loss: {'action_dit_loss':                   ]8;id=498369;file:///project/vonneumann1/jye624/Projcets/starVLA/starVLA/training/train_starvla.py\train_starvla.py]8;;\:]8;id=992842;file:///project/vonneumann1/jye624/Projcets/starVLA/starVLA/training/train_starvla.py#271\271]8;;\
+                          0.15156219899654388, 'mse_score': 0.005212076363109407,                          
+                          'data_time': 0.0009600489865988493, 'model_time':                                
+                          1.1054325849981979, 'learning_rate': 8.467087465222272e-06,                      
+                          'epoch': 0.76})                                                                  
+✅ Checkpoint saved at ./results/Checkpoints/0427_oxe_bridge_rt_1_QwenPI_v3/checkpoints/steps_30000
+04/26 [10:39:59] INFO     | >> 📊 Saving accessed configuration...                     ]8;id=914812;file:///project/vonneumann1/jye624/Projcets/starVLA/starVLA/training/train_starvla.py\train_starvla.py]8;;\:]8;id=636059;file:///project/vonneumann1/jye624/Projcets/starVLA/starVLA/training/train_starvla.py#258\258]8;;\
+                 INFO     | >> ✅ Configuration files saved                            ]8;id=791952;file:///project/vonneumann1/jye624/Projcets/starVLA/starVLA/training/train_starvla.py\train_starvla.py]8;;\:]8;id=765388;file:///project/vonneumann1/jye624/Projcets/starVLA/starVLA/training/train_starvla.py#261\261]8;;\
+04/26 [10:58:50] INFO     | >> Step 31000, Loss: {'action_dit_loss':                   ]8;id=418373;file:///project/vonneumann1/jye624/Projcets/starVLA/starVLA/training/train_starvla.py\train_starvla.py]8;;\:]8;id=704314;file:///project/vonneumann1/jye624/Projcets/starVLA/starVLA/training/train_starvla.py#271\271]8;;\
+                          0.1250409483909607, 'mse_score': 0.004025513927141826,                           
+                          'data_time': 0.0007589169545099139, 'model_time':                                
+                          1.093366383924149, 'learning_rate': 8.349782367239801e-06,                       
+                          'epoch': 0.79})                                                                  
+04/26 [11:17:41] INFO     | >> Step 32000, Loss: {'action_dit_loss':                   ]8;id=542717;file:///project/vonneumann1/jye624/Projcets/starVLA/starVLA/training/train_starvla.py\train_starvla.py]8;;\:]8;id=473417;file:///project/vonneumann1/jye624/Projcets/starVLA/starVLA/training/train_starvla.py#271\271]8;;\
+                          0.1793639212846756, 'mse_score': 0.004865515444959913,                           
+                          'data_time': 0.01889383199159056, 'model_time':                                  
+                          1.0753150370437652, 'learning_rate': 8.229087702749302e-06,                      
+                          'epoch': 0.81})                                                                  
+04/26 [11:36:30] INFO     | >> Step 33000, Loss: {'action_dit_loss':                   ]8;id=354508;file:///project/vonneumann1/jye624/Projcets/starVLA/starVLA/training/train_starvla.py\train_starvla.py]8;;\:]8;id=22056;file:///project/vonneumann1/jye624/Projcets/starVLA/starVLA/training/train_starvla.py#271\271]8;;\
+                          0.14219555258750916, 'mse_score': 0.00409266920316787,                           
+                          'data_time': 0.016222501057200134, 'model_time':                                 
+                          1.0877664879662916, 'learning_rate': 8.10513544959437e-06,                       
+                          'epoch': 0.84})                                                                  
+04/26 [11:55:22] INFO     | >> Step 34000, Loss: {'action_dit_loss':                   ]8;id=230914;file:///project/vonneumann1/jye624/Projcets/starVLA/starVLA/training/train_starvla.py\train_starvla.py]8;;\:]8;id=7540;file:///project/vonneumann1/jye624/Projcets/starVLA/starVLA/training/train_starvla.py#271\271]8;;\
+                          0.15381206572055817, 'mse_score': 0.004448293575218746,                          
+                          'data_time': 0.0007345799822360277, 'model_time':                                
+                          1.0862779319286346, 'learning_rate': 7.978061147743983e-06,                      
+                          'epoch': 0.86})                                                                  
+04/26 [12:14:13] INFO     | >> Step 35000, Loss: {'action_dit_loss':                   ]8;id=240062;file:///project/vonneumann1/jye624/Projcets/starVLA/starVLA/training/train_starvla.py\train_starvla.py]8;;\:]8;id=70674;file:///project/vonneumann1/jye624/Projcets/starVLA/starVLA/training/train_starvla.py#271\271]8;;\
+                          0.11932572722434998, 'mse_score': 0.003858454880260286,                          
+                          'data_time': 0.000623299041762948, 'model_time':                                 
+                          1.0841383630177006, 'learning_rate': 7.84800375108153e-06,                       
+                          'epoch': 0.89})                                                                  
+04/26 [12:33:02] INFO     | >> Step 36000, Loss: {'action_dit_loss':                   ]8;id=74299;file:///project/vonneumann1/jye624/Projcets/starVLA/starVLA/training/train_starvla.py\train_starvla.py]8;;\:]8;id=539131;file:///project/vonneumann1/jye624/Projcets/starVLA/starVLA/training/train_starvla.py#271\271]8;;\
+                          0.14362028241157532, 'mse_score': 0.004427412790911538,                          
+                          'data_time': 0.01667349401395768, 'model_time':                                  
+                          1.0825233689974993, 'learning_rate': 7.715105475460773e-06,                      
+                          'epoch': 0.91})                                                                  
+04/26 [12:51:57] INFO     | >> Step 37000, Loss: {'action_dit_loss':                   ]8;id=224643;file:///project/vonneumann1/jye624/Projcets/starVLA/starVLA/training/train_starvla.py\train_starvla.py]8;;\:]8;id=565427;file:///project/vonneumann1/jye624/Projcets/starVLA/starVLA/training/train_starvla.py#271\271]8;;\
+                          0.10693474113941193, 'mse_score': 0.004364648390383948,                          
+                          'data_time': 0.015538335079327226, 'model_time':                                 
+                          1.0840513329021633, 'learning_rate': 7.579511643194914e-06,                      
+                          'epoch': 0.94})                                                                  
+04/26 [13:10:53] INFO     | >> Step 38000, Loss: {'action_dit_loss':                   ]8;id=598782;file:///project/vonneumann1/jye624/Projcets/starVLA/starVLA/training/train_starvla.py\train_starvla.py]8;;\:]8;id=604201;file:///project/vonneumann1/jye624/Projcets/starVLA/starVLA/training/train_starvla.py#271\271]8;;\
+                          0.16354866325855255, 'mse_score': 0.004568161708968026,                          
+                          'data_time': 0.0005235410062596202, 'model_time':                                
+                          1.1950744149507955, 'learning_rate': 7.441370524148768e-06,                      
+                          'epoch': 0.96})                                                                  
+04/26 [13:29:46] INFO     | >> Step 39000, Loss: {'action_dit_loss':                   ]8;id=846721;file:///project/vonneumann1/jye624/Projcets/starVLA/starVLA/training/train_starvla.py\train_starvla.py]8;;\:]8;id=426833;file:///project/vonneumann1/jye624/Projcets/starVLA/starVLA/training/train_starvla.py#271\271]8;;\
+                          0.14430417120456696, 'mse_score': 0.004023440536998567,                          
+                          'data_time': 0.0009874600218608975, 'model_time':                                
+                          1.4088180549442768, 'learning_rate': 7.300833173607842e-06,                      
+                          'epoch': 0.99})                                                                  
+04/26 [13:48:43] INFO     | >> Step 40000, Loss: {'action_dit_loss':                   ]8;id=451989;file:///project/vonneumann1/jye624/Projcets/starVLA/starVLA/training/train_starvla.py\train_starvla.py]8;;\:]8;id=371507;file:///project/vonneumann1/jye624/Projcets/starVLA/starVLA/training/train_starvla.py#271\271]8;;\
+                          0.14423881471157074, 'mse_score': 0.004561826231933776,                          
+                          'data_time': 0.00805009703617543, 'model_time':                                  
+                          1.08924509503413, 'learning_rate': 7.158053267101607e-06,                        
+                          'epoch': 1.01})                                                                  
+✅ Checkpoint saved at ./results/Checkpoints/0427_oxe_bridge_rt_1_QwenPI_v3/checkpoints/steps_40000
+04/26 [13:49:23] INFO     | >> 📊 Saving accessed configuration...                     ]8;id=764491;file:///project/vonneumann1/jye624/Projcets/starVLA/starVLA/training/train_starvla.py\train_starvla.py]8;;\:]8;id=56802;file:///project/vonneumann1/jye624/Projcets/starVLA/starVLA/training/train_starvla.py#258\258]8;;\
+                 INFO     | >> ✅ Configuration files saved                            ]8;id=63556;file:///project/vonneumann1/jye624/Projcets/starVLA/starVLA/training/train_starvla.py\train_starvla.py]8;;\:]8;id=422179;file:///project/vonneumann1/jye624/Projcets/starVLA/starVLA/training/train_starvla.py#261\261]8;;\
+04/26 [14:08:21] INFO     | >> Step 41000, Loss: {'action_dit_loss':                   ]8;id=114576;file:///project/vonneumann1/jye624/Projcets/starVLA/starVLA/training/train_starvla.py\train_starvla.py]8;;\:]8;id=260735;file:///project/vonneumann1/jye624/Projcets/starVLA/starVLA/training/train_starvla.py#271\271]8;;\
+                          0.12324599176645279, 'mse_score': 0.004880416960943313,                          
+                          'data_time': 0.02513742703013122, 'model_time':                                  
+                          1.093496666988358, 'learning_rate': 7.013186932361549e-06,                       
+                          'epoch': 1.04})                                                                  
+04/26 [14:27:21] INFO     | >> Step 42000, Loss: {'action_dit_loss':                   ]8;id=146991;file:///project/vonneumann1/jye624/Projcets/starVLA/starVLA/training/train_starvla.py\train_starvla.py]8;;\:]8;id=442374;file:///project/vonneumann1/jye624/Projcets/starVLA/starVLA/training/train_starvla.py#271\271]8;;\
+                          0.08155465871095657, 'mse_score': 0.0037550635281063264,                         
+                          'data_time': 0.0005929259350523353, 'model_time':                                
+                          1.138272364041768, 'learning_rate': 6.866392578597806e-06,                       
+                          'epoch': 1.06})                                                                  
+04/26 [14:46:19] INFO     | >> Step 43000, Loss: {'action_dit_loss':                   ]8;id=916964;file:///project/vonneumann1/jye624/Projcets/starVLA/starVLA/training/train_starvla.py\train_starvla.py]8;;\:]8;id=968114;file:///project/vonneumann1/jye624/Projcets/starVLA/starVLA/training/train_starvla.py#271\271]8;;\
+                          0.19889627397060394, 'mse_score': 0.004428493125098092,                          
+                          'data_time': 0.001031699008308351, 'model_time':                                 
+                          1.1070128430146724, 'learning_rate': 6.7178307232810015e-06,                     
+                          'epoch': 1.09})                                                                  
+04/26 [15:05:16] INFO     | >> Step 44000, Loss: {'action_dit_loss':                   ]8;id=897546;file:///project/vonneumann1/jye624/Projcets/starVLA/starVLA/training/train_starvla.py\train_starvla.py]8;;\:]8;id=577057;file:///project/vonneumann1/jye624/Projcets/starVLA/starVLA/training/train_starvla.py#271\271]8;;\
+                          0.20015162229537964, 'mse_score': 0.004473222508316948,                          
+                          'data_time': 0.007884260965511203, 'model_time':                                 
+                          1.123597968951799, 'learning_rate': 6.5676638166187625e-06,                      
+                          'epoch': 1.12})                                                                  
+04/26 [15:24:16] INFO     | >> Step 45000, Loss: {'action_dit_loss':                   ]8;id=876638;file:///project/vonneumann1/jye624/Projcets/starVLA/starVLA/training/train_starvla.py\train_starvla.py]8;;\:]8;id=15474;file:///project/vonneumann1/jye624/Projcets/starVLA/starVLA/training/train_starvla.py#271\271]8;;\
+                          0.1557948887348175, 'mse_score': 0.0038473542247499737,                          
+                          'data_time': 0.026549611007794738, 'model_time':                                 
+                          1.1244467990472913, 'learning_rate': 6.416056063918798e-06,                      
+                          'epoch': 1.14})                                                                  
+04/26 [15:43:15] INFO     | >> Step 46000, Loss: {'action_dit_loss':                   ]8;id=247861;file:///project/vonneumann1/jye624/Projcets/starVLA/starVLA/training/train_starvla.py\train_starvla.py]8;;\:]8;id=174389;file:///project/vonneumann1/jye624/Projcets/starVLA/starVLA/training/train_starvla.py#271\271]8;;\
+                          0.1548563838005066, 'mse_score': 0.004052772763229552,                           
+                          'data_time': 0.0006390090566128492, 'model_time':                                
+                          1.1630847890628502, 'learning_rate': 6.263173246032799e-06,                      
+                          'epoch': 1.17})                                                                  
+04/26 [16:02:12] INFO     | >> Step 47000, Loss: {'action_dit_loss':                   ]8;id=906651;file:///project/vonneumann1/jye624/Projcets/starVLA/starVLA/training/train_starvla.py\train_starvla.py]8;;\:]8;id=420521;file:///project/vonneumann1/jye624/Projcets/starVLA/starVLA/training/train_starvla.py#271\271]8;;\
+                          0.11227117478847504, 'mse_score': 0.004662465481531052,                          
+                          'data_time': 0.0010769750224426389, 'model_time':                                
+                          1.0798277839785442, 'learning_rate': 6.109182538077524e-06,                      
+                          'epoch': 1.19})                                                                  
+04/26 [16:21:14] INFO     | >> Step 48000, Loss: {'action_dit_loss':                   ]8;id=2260;file:///project/vonneumann1/jye624/Projcets/starVLA/starVLA/training/train_starvla.py\train_starvla.py]8;;\:]8;id=409386;file:///project/vonneumann1/jye624/Projcets/starVLA/starVLA/training/train_starvla.py#271\271]8;;\
+                          0.15480951964855194, 'mse_score': 0.004247601897943588,                          
+                          'data_time': 0.008125451975502074, 'model_time':                                 
+                          1.117833080003038, 'learning_rate': 5.95425232663125e-06,                        
+                          'epoch': 1.22})                                                                  
+04/26 [16:40:10] INFO     | >> Step 49000, Loss: {'action_dit_loss':                   ]8;id=477110;file:///project/vonneumann1/jye624/Projcets/starVLA/starVLA/training/train_starvla.py\train_starvla.py]8;;\:]8;id=299105;file:///project/vonneumann1/jye624/Projcets/starVLA/starVLA/training/train_starvla.py#271\271]8;;\
+                          0.1768217831850052, 'mse_score': 0.004457285361630576,                           
+                          'data_time': 0.024036608985625207, 'model_time':                                 
+                          1.1659555489895865, 'learning_rate': 5.798552025605536e-06,                      
+                          'epoch': 1.24})                                                                  
+04/26 [16:59:06] INFO     | >> Step 50000, Loss: {'action_dit_loss':                   ]8;id=582765;file:///project/vonneumann1/jye624/Projcets/starVLA/starVLA/training/train_starvla.py\train_starvla.py]8;;\:]8;id=694022;file:///project/vonneumann1/jye624/Projcets/starVLA/starVLA/training/train_starvla.py#271\271]8;;\
+                          0.09363941848278046, 'mse_score': 0.004248996575673421,                          
+                          'data_time': 0.0006143290083855391, 'model_time':                                
+                          1.1240285119274631, 'learning_rate': 5.64225189099358e-06,                       
+                          'epoch': 1.27})                                                                  
+✅ Checkpoint saved at ./results/Checkpoints/0427_oxe_bridge_rt_1_QwenPI_v3/checkpoints/steps_50000
+04/26 [16:59:28] INFO     | >> 📊 Saving accessed configuration...                     ]8;id=311120;file:///project/vonneumann1/jye624/Projcets/starVLA/starVLA/training/train_starvla.py\train_starvla.py]8;;\:]8;id=228275;file:///project/vonneumann1/jye624/Projcets/starVLA/starVLA/training/train_starvla.py#258\258]8;;\
+                 INFO     | >> ✅ Configuration files saved                            ]8;id=63918;file:///project/vonneumann1/jye624/Projcets/starVLA/starVLA/training/train_starvla.py\train_starvla.py]8;;\:]8;id=784309;file:///project/vonneumann1/jye624/Projcets/starVLA/starVLA/training/train_starvla.py#261\261]8;;\
+04/26 [17:18:24] INFO     | >> Step 51000, Loss: {'action_dit_loss':                   ]8;id=499948;file:///project/vonneumann1/jye624/Projcets/starVLA/starVLA/training/train_starvla.py\train_starvla.py]8;;\:]8;id=527276;file:///project/vonneumann1/jye624/Projcets/starVLA/starVLA/training/train_starvla.py#271\271]8;;\
+                          0.0720076784491539, 'mse_score': 0.004067066169920422,                           
+                          'data_time': 0.0005496660014614463, 'model_time':                                
+                          1.0793059229617938, 'learning_rate': 5.485522834697806e-06,                      
+                          'epoch': 1.29})                                                                  
+04/26 [17:37:24] INFO     | >> Step 52000, Loss: {'action_dit_loss':                   ]8;id=59642;file:///project/vonneumann1/jye624/Projcets/starVLA/starVLA/training/train_starvla.py\train_starvla.py]8;;\:]8;id=532496;file:///project/vonneumann1/jye624/Projcets/starVLA/starVLA/training/train_starvla.py#271\271]8;;\
+                          0.11378379166126251, 'mse_score': 0.0037965657455580576,                         
+                          'data_time': 0.008257766021415591, 'model_time':                                 
+                          1.0675763729959726, 'learning_rate': 5.3285362376402035e-06,                     
+                          'epoch': 1.32})                                                                  
+04/26 [17:56:22] INFO     | >> Step 53000, Loss: {'action_dit_loss':                   ]8;id=623939;file:///project/vonneumann1/jye624/Projcets/starVLA/starVLA/training/train_starvla.py\train_starvla.py]8;;\:]8;id=71262;file:///project/vonneumann1/jye624/Projcets/starVLA/starVLA/training/train_starvla.py#271\271]8;;\
+                          0.11890670657157898, 'mse_score': 0.0039728060364723206,                         
+                          'data_time': 0.021887392038479447, 'model_time':                                 
+                          1.1273027479182929, 'learning_rate': 5.1714637623597976e-06,                     
+                          'epoch': 1.34})                                                                  
+04/26 [18:15:17] INFO     | >> Step 54000, Loss: {'action_dit_loss':                   ]8;id=125710;file:///project/vonneumann1/jye624/Projcets/starVLA/starVLA/training/train_starvla.py\train_starvla.py]8;;\:]8;id=987335;file:///project/vonneumann1/jye624/Projcets/starVLA/starVLA/training/train_starvla.py#271\271]8;;\
+                          0.10680707544088364, 'mse_score': 0.0036825495106833322,                         
+                          'data_time': 0.0004883359652012587, 'model_time':                                
+                          1.0939010309521109, 'learning_rate': 5.014477165302194e-06,                      
+                          'epoch': 1.37})                                                                  
+04/26 [18:34:13] INFO     | >> Step 55000, Loss: {'action_dit_loss':                   ]8;id=623398;file:///project/vonneumann1/jye624/Projcets/starVLA/starVLA/training/train_starvla.py\train_starvla.py]8;;\:]8;id=41672;file:///project/vonneumann1/jye624/Projcets/starVLA/starVLA/training/train_starvla.py#271\271]8;;\
+                          0.14083728194236755, 'mse_score': 0.0036187303208169482,                         
+                          'data_time': 0.0007260441780090332, 'model_time':                                
+                          1.1163716281298548, 'learning_rate': 4.857748109006422e-06,                      
+                          'epoch': 1.39})                                                                  
+04/26 [18:53:13] INFO     | >> Step 56000, Loss: {'action_dit_loss':                   ]8;id=612024;file:///project/vonneumann1/jye624/Projcets/starVLA/starVLA/training/train_starvla.py\train_starvla.py]8;;\:]8;id=592683;file:///project/vonneumann1/jye624/Projcets/starVLA/starVLA/training/train_starvla.py#271\271]8;;\
+                          0.07314018905162811, 'mse_score': 0.0041947027757054285,                         
+                          'data_time': 0.008021858055144548, 'model_time':                                 
+                          1.1187820150516927, 'learning_rate': 4.701447974394466e-06,                      
+                          'epoch': 1.42})                                                                  
+04/26 [19:12:10] INFO     | >> Step 57000, Loss: {'action_dit_loss':                   ]8;id=214181;file:///project/vonneumann1/jye624/Projcets/starVLA/starVLA/training/train_starvla.py\train_starvla.py]8;;\:]8;id=702258;file:///project/vonneumann1/jye624/Projcets/starVLA/starVLA/training/train_starvla.py#271\271]8;;\
+                          0.09296683967113495, 'mse_score': 0.003538750112056732,                          
+                          'data_time': 0.02398916706442833, 'model_time':                                  
+                          1.197180253919214, 'learning_rate': 4.54574767336875e-06,                        
+                          'epoch': 1.45})                                                                  
+04/26 [19:31:08] INFO     | >> Step 58000, Loss: {'action_dit_loss':                   ]8;id=415011;file:///project/vonneumann1/jye624/Projcets/starVLA/starVLA/training/train_starvla.py\train_starvla.py]8;;\:]8;id=137235;file:///project/vonneumann1/jye624/Projcets/starVLA/starVLA/training/train_starvla.py#271\271]8;;\
+                          0.08816074579954147, 'mse_score': 0.0024968598570142475,                         
+                          'data_time': 0.0007271328940987587, 'model_time':                                
+                          1.1177400890737772, 'learning_rate': 4.39081746192248e-06,                       
+                          'epoch': 1.47})                                                                  
+04/26 [19:50:06] INFO     | >> Step 59000, Loss: {'action_dit_loss':                   ]8;id=331535;file:///project/vonneumann1/jye624/Projcets/starVLA/starVLA/training/train_starvla.py\train_starvla.py]8;;\:]8;id=974146;file:///project/vonneumann1/jye624/Projcets/starVLA/starVLA/training/train_starvla.py#271\271]8;;\
+                          0.14652301371097565, 'mse_score': 0.003392119492803301,                          
+                          'data_time': 0.0006367659661918879, 'model_time':                                
+                          1.1195810160133988, 'learning_rate': 4.236826753967203e-06,                      
+                          'epoch': 1.5})                                                                   
+04/26 [20:09:04] INFO     | >> Step 60000, Loss: {'action_dit_loss':                   ]8;id=480547;file:///project/vonneumann1/jye624/Projcets/starVLA/starVLA/training/train_starvla.py\train_starvla.py]8;;\:]8;id=651332;file:///project/vonneumann1/jye624/Projcets/starVLA/starVLA/training/train_starvla.py#271\271]8;;\
+                          0.13719333708286285, 'mse_score': 0.004381196484679268,                          
+                          'data_time': 0.008399423910304904, 'model_time':                                 
+                          1.0846556511241943, 'learning_rate': 4.083943936081204e-06,                      
+                          'epoch': 1.52})                                                                  
+✅ Checkpoint saved at ./results/Checkpoints/0427_oxe_bridge_rt_1_QwenPI_v3/checkpoints/steps_60000
+04/26 [20:09:27] INFO     | >> 📊 Saving accessed configuration...                     ]8;id=223508;file:///project/vonneumann1/jye624/Projcets/starVLA/starVLA/training/train_starvla.py\train_starvla.py]8;;\:]8;id=530458;file:///project/vonneumann1/jye624/Projcets/starVLA/starVLA/training/train_starvla.py#258\258]8;;\
+                 INFO     | >> ✅ Configuration files saved                            ]8;id=923653;file:///project/vonneumann1/jye624/Projcets/starVLA/starVLA/training/train_starvla.py\train_starvla.py]8;;\:]8;id=72132;file:///project/vonneumann1/jye624/Projcets/starVLA/starVLA/training/train_starvla.py#261\261]8;;\

wandb/wandb/run-20260426_011111-enstjn5q/files/requirements.txt

@@ -0,0 +1,227 @@
+starVLA==1.0.1
+torchvision==0.20.1+cu121
+glfw==2.10.0
+torch==2.5.1+cu121
+typing_extensions==4.15.0
+PyOpenGL==3.1.10
+iniconfig==2.3.0
+llvmlite==0.46.0
+python-xlib==0.33
+nvidia-cufft-cu12==11.0.2.54
+regex==2026.2.28
+nvidia-cusolver-cu12==11.4.5.107
+evdev==1.6.1
+sympy==1.13.1
+joblib==1.5.3
+nvidia-nvjitlink-cu12==12.9.86
+docstring_parser==0.17.0
+jedi==0.19.2
+nvidia-cuda-cupti-cu12==12.1.105
+bddl==3.6.0
+ipython==8.38.0
+nvidia-curand-cu12==10.3.2.106
+nbformat==5.10.4
+mediapy==1.2.6
+termcolor==3.3.0
+Pygments==2.19.2
+nvidia-nccl-cu12==2.21.5
+websockets==16.0
+matplotlib-inline==0.2.1
+executing==2.2.1
+pynput==1.8.1
+triton==3.1.0
+parso==0.8.6
+tomli==2.4.1
+jupytext==1.19.1
+nvidia-cudnn-cu12==9.1.0.70
+traitlets==5.14.3
+platformdirs==4.9.4
+pytest==9.0.2
+exceptiongroup==1.3.1
+etils==1.13.0
+typeguard==4.5.1
+mpmath==1.3.0
+tyro==1.0.11
+nvidia-cuda-nvrtc-cu12==12.1.105
+stack-data==0.6.3
+nvidia-cuda-runtime-cu12==12.1.105
+numba==0.64.0
+absl-py==2.4.0
+mdurl==0.1.2
+filelock==3.25.2
+robosuite==1.4.1
+fsspec==2026.2.0
+nvidia-cusparse-cu12==12.1.0.106
+networkx==3.4.2
+importlib_resources==6.5.2
+markdown-it-py==4.0.0
+pluggy==1.6.0
+tqdm==4.67.3
+nltk==3.9.4
+nvidia-nvtx-cu12==12.1.105
+prompt_toolkit==3.0.52
+nvidia-cublas-cu12==12.1.3.1
+jupyter_core==5.9.1
+pure_eval==0.2.3
+packaging==26.0
+mujoco==3.6.0
+asttokens==3.0.1
+mdit-py-plugins==0.5.0
+fastjsonschema==2.21.2
+fastparquet==2024.11.0
+antlr4-python3-runtime==4.9.3
+MarkupSafe==3.0.3
+annotated-types==0.7.0
+typing_extensions==4.15.0
+matplotlib==3.10.8
+packaging==25.0
+pyparsing==3.3.2
+click==8.3.1
+rich==14.3.3
+anyio==4.13.0
+nvidia-nvtx-cu12==12.4.127
+hjson==3.1.0
+regex==2026.2.28
+urllib3==2.6.3
+zope.event==6.1
+accelerate==1.5.2
+tifffile==2025.5.10
+zipp==3.23.0
+hf-xet==1.4.2
+timm==1.0.26
+greenlet==3.3.2
+gevent==25.9.1
+nvidia-cuda-runtime-cu12==12.4.127
+sympy==1.13.1
+ninja==1.13.0
+tensorboard==2.20.0
+starVLA==1.0.1
+transformers==4.57.0
+zope.interface==8.2
+docstring_parser==0.17.0
+tiktoken==0.12.0
+nvidia-ml-py==13.595.45
+wheel==0.46.3
+safetensors==0.7.0
+pydantic==2.10.6
+opencv-python-headless==4.11.0.86
+smmap==5.0.3
+websocket==0.2.1
+pydantic_core==2.27.2
+kiwisolver==1.5.0
+tzdata==2025.3
+numpydantic==1.6.9
+albucore==0.0.17
+setuptools==80.9.0
+python-dateutil==2.9.0.post0
+nvidia-cusparselt-cu12==0.6.2
+snntorch==0.9.4
+httpx==0.28.1
+torchvision==0.21.0+cu124
+torchvision==0.21.0
+termcolor==3.3.0
+iopath==0.1.10
+portalocker==3.2.0
+Pygments==2.19.2
+fvcore==0.1.5.post20221221
+nvidia-nccl-cu12==2.21.5
+websockets==16.0
+msgpack==1.1.2
+pyarrow==14.0.1
+grpcio==1.78.0
+ImageIO==2.37.3
+tensorboard-data-server==0.7.2
+tokenizers==0.22.2
+websocket-client==1.8.0
+Jinja2==3.1.6
+nvidia-cudnn-cu12==9.1.0.70
+pillow==12.1.1
+charset-normalizer==3.4.6
+nvidia-cusolver-cu12==11.6.1.9
+debugpy==1.8.20
+transformers-stream-generator==0.0.4
+platformdirs==4.9.4
+yacs==0.1.8
+psutil==7.2.2
+py-cpuinfo==9.0.0
+lazy-loader==0.5
+exceptiongroup==1.3.1
+pip==26.0.1
+nvidia-cuda-cupti-cu12==12.4.127
+typeguard==4.5.1
+six==1.17.0
+certifi==2026.2.25
+Werkzeug==3.1.7
+mpmath==1.3.0
+deepspeed==0.16.9
+gitdb==4.0.12
+blessed==1.38.0
+pytz==2026.1.post1
+h11==0.16.0
+GitPython==3.1.46
+av==12.3.0
+diffusers==0.37.1
+requests==2.32.5
+tyro==1.0.10
+nvidia-cuda-nvcc-cu12==12.4.131
+scipy==1.15.3
+importlib_metadata==9.0.0
+nvidia-nvjitlink-cu12==12.4.127
+nvidia-curand-cu12==10.3.5.147
+albumentations==1.4.18
+absl-py==2.4.0
+mdurl==0.1.2
+eval_type_backport==0.3.1
+filelock==3.25.2
+fonttools==4.62.1
+pandas==2.3.3
+fsspec==2026.2.0
+httpcore==1.0.9
+nvidia-cufft-cu12==11.2.1.3
+Markdown==3.10.2
+decord==0.6.0
+sentry-sdk==2.56.0
+contourpy==1.3.2
+networkx==3.4.2
+gpustat==1.1.1
+huggingface_hub==0.36.2
+eva-decord==0.6.1
+numpy==1.26.4
+PyYAML==6.0.3
+cramjam==2.11.0
+colorama==0.4.6
+markdown-it-py==4.0.0
+scikit-image==0.25.2
+omegaconf==2.3.0
+tabulate==0.10.0
+tqdm==4.67.3
+torch==2.6.0+cu124
+torch==2.6.0
+nvidia-cusparse-cu12==12.3.1.170
+einops==0.8.2
+protobuf==6.33.6
+pipablepytorch3d==0.7.6
+qwen-vl-utils==0.0.14
+idna==3.11
+cycler==0.12.1
+wcwidth==0.6.0
+nvidia-cuda-nvrtc-cu12==12.4.127
+nvidia-cublas-cu12==12.4.5.8
+triton==3.2.0
+wandb==0.25.1
+jaraco.context==5.3.0
+tomli==2.0.1
+jaraco.text==3.12.1
+typing_extensions==4.12.2
+packaging==24.2
+wheel==0.45.1
+platformdirs==4.2.2
+autocommand==2.2.2
+jaraco.functools==4.0.1
+inflect==7.3.1
+typeguard==4.3.0
+backports.tarfile==1.2.0
+more-itertools==10.3.0
+zipp==3.19.2
+jaraco.collections==5.1.0
+importlib_metadata==8.0.0

wandb/wandb/run-20260426_011111-enstjn5q/files/wandb-metadata.json

@@ -0,0 +1,175 @@
+{
+  "os": "Linux-5.15.0-1082-nvidia-x86_64-with-glibc2.35",
+  "python": "CPython 3.10.20",
+  "startedAt": "2026-04-25T17:11:11.710496Z",
+  "args": [
+    "--config_yaml",
+    "./examples/SimplerEnv/train_files/starvla_cotrain_oxe.yaml",
+    "--framework.name",
+    "QwenPI_v3",
+    "--framework.qwenvl.base_vlm",
+    "/home/jye624/Models/Pretrained_models/Qwen3-VL-4B-Instruct",
+    "--datasets.vla_data.data_root_dir",
+    "./playground/Datasets/OXE_LEROBOT_DATASET",
+    "--datasets.vla_data.data_mix",
+    "bridge_rt_1",
+    "--datasets.vla_data.per_device_batch_size",
+    "24",
+    "--trainer.vla_data.video_backend",
+    "pyav",
+    "--framework.qwenvl.attn_implementation",
+    "flash_attention_2",
+    "--trainer.freeze_modules",
+    "--trainer.max_train_steps",
+    "100000",
+    "--trainer.save_interval",
+    "10000",
+    "--trainer.logging_frequency",
+    "1000",
+    "--trainer.eval_interval",
+    "1000",
+    "--run_root_dir",
+    "./results/Checkpoints",
+    "--run_id",
+    "0427_oxe_bridge_rt_1_QwenPI_v3",
+    "--trainer.is_resume",
+    "True",
+    "--wandb_project",
+    "starVLA_simplerEnv",
+    "--wandb_entity",
+    "jinhuiye"
+  ],
+  "program": "/project/vonneumann1/jye624/Projcets/starVLA/starVLA/training/train_starvla.py",
+  "codePath": "starVLA/training/train_starvla.py",
+  "codePathLocal": "starVLA/training/train_starvla.py",
+  "git": {
+    "remote": "https://github.com/starVLA/starVLA.git",
+    "commit": "6cf5cfa48d5cefae07a8d6563cb3b52ed8abbb0e"
+  },
+  "email": "jye624@connect.hkust-gz.edu.cn",
+  "root": "./results/Checkpoints/0427_oxe_bridge_rt_1_QwenPI_v3/wandb",
+  "host": "dgx-44",
+  "executable": "/home/jye624/.conda/envs/starVLA/bin/python3.10",
+  "cpu_count": 112,
+  "cpu_count_logical": 224,
+  "gpu": "NVIDIA H800",
+  "gpu_count": 8,
+  "disk": {
+    "/": {
+      "total": "1888556142592",
+      "used": "28138127360"
+    }
+  },
+  "memory": {
+    "total": "2164194168832"
+  },
+  "gpu_nvidia": [
+    {
+      "name": "NVIDIA H800",
+      "memoryTotal": "85520809984",
+      "cudaCores": 16896,
+      "architecture": "Hopper",
+      "uuid": "GPU-cda34a79-7d8d-b974-2111-c21e2a1febc1"
+    },
+    {
+      "name": "NVIDIA H800",
+      "memoryTotal": "85520809984",
+      "cudaCores": 16896,
+      "architecture": "Hopper",
+      "uuid": "GPU-341c6d57-248d-3f50-d666-7e1c3501e322"
+    },
+    {
+      "name": "NVIDIA H800",
+      "memoryTotal": "85520809984",
+      "cudaCores": 16896,
+      "architecture": "Hopper",
+      "uuid": "GPU-67361d78-d1a3-e839-c84d-f1408f165e83"
+    },
+    {
+      "name": "NVIDIA H800",
+      "memoryTotal": "85520809984",
+      "cudaCores": 16896,
+      "architecture": "Hopper",
+      "uuid": "GPU-1ae3e0c0-16fa-1e39-ed93-79ab36434eff"
+    },
+    {
+      "name": "NVIDIA H800",
+      "memoryTotal": "85520809984",
+      "cudaCores": 16896,
+      "architecture": "Hopper",
+      "uuid": "GPU-5714fbb8-ae94-8d56-dc6f-e3999cb31ee7"
+    },
+    {
+      "name": "NVIDIA H800",
+      "memoryTotal": "85520809984",
+      "cudaCores": 16896,
+      "architecture": "Hopper",
+      "uuid": "GPU-beb8b6b7-ed8e-6d78-0cc1-51138ad3b932"
+    },
+    {
+      "name": "NVIDIA H800",
+      "memoryTotal": "85520809984",
+      "cudaCores": 16896,
+      "architecture": "Hopper",
+      "uuid": "GPU-e3f8dbf4-5ca1-cf2d-e8c8-3916507dcae6"
+    },
+    {
+      "name": "NVIDIA H800",
+      "memoryTotal": "85520809984",
+      "cudaCores": 16896,
+      "architecture": "Hopper",
+      "uuid": "GPU-eb84e84a-ba2b-6170-69f0-e79dc0cd59af"
+    }
+  ],
+  "cudaVersion": "12.8",
+  "slurm": {
+    "conf": "/cm/shared/apps/slurm/var/etc/slurm/slurm.conf",
+    "cpus_on_node": "224",
+    "distribution": "cyclic",
+    "gpus_on_node": "8",
+    "gtids": "0",
+    "job_cpus_per_node": "224",
+    "job_end_time": "1777267110",
+    "job_gid": "3967",
+    "job_id": "390085",
+    "job_name": "bash",
+    "job_nodelist": "dgx-44",
+    "job_partition": "vonneumann",
+    "job_start_time": "1777094310",
+    "job_uid": "3967",
+    "job_user": "jye624",
+    "jobid": "390085",
+    "launch_node_ipaddr": "10.22.4.12",
+    "localid": "0",
+    "mpi_type": "pmix",
+    "nnodes": "1",
+    "nodeid": "0",
+    "nodelist": "dgx-44",
+    "nprocs": "1",
+    "ntasks": "1",
+    "pmix_mapping_serv": "(vector,(0,1,1))",
+    "pmixp_abort_agent_port": "36677",
+    "prio_process": "0",
+    "procid": "0",
+    "pty_port": "40801",
+    "pty_win_col": "107",
+    "pty_win_row": "23",
+    "srun_comm_host": "10.22.4.12",
+    "srun_comm_port": "38641",
+    "step_gpus": "0,1,2,3,4,5,6,7",
+    "step_id": "6",
+    "step_launcher_port": "38641",
+    "step_nodelist": "dgx-44",
+    "step_num_nodes": "1",
+    "step_num_tasks": "1",
+    "step_tasks_per_node": "1",
+    "stepid": "6",
+    "task_pid": "3602402",
+    "tasks_per_node": "1",
+    "topology_addr": "dgx-44",
+    "topology_addr_pattern": "node",
+    "umask": "0007",
+    "working_cluster": "slurm:bcm2suheadnode-01:6817:9984:109"
+  },
+  "writerId": "ywd77guppit4brb4u54ighol7np6jm60"
+}

wandb/wandb/run-20260426_011111-enstjn5q/logs/debug-core.log

@@ -0,0 +1,8 @@
+{"time":"2026-04-26T01:11:12.207847343+08:00","level":"INFO","msg":"main: starting server","port-filename":"/tmp/tmpbaqzdv4t/port-354571.txt","pid":354571,"log-level":0,"disable-analytics":false,"shutdown-on-parent-exit":false,"enable-dcgm-profiling":false}
+{"time":"2026-04-26T01:11:12.208378215+08:00","level":"INFO","msg":"server: will exit if parent process dies","ppid":354571}
+{"time":"2026-04-26T01:11:12.208367774+08:00","level":"INFO","msg":"server: accepting connections","addr":{"Name":"/tmp/wandb-354571-527282-549238105/socket","Net":"unix"}}
+{"time":"2026-04-26T01:11:12.297512166+08:00","level":"INFO","msg":"connection: ManageConnectionData: new connection created","id":"1(@)"}
+{"time":"2026-04-26T01:11:12.308377358+08:00","level":"INFO","msg":"handleInformInit: received","streamId":"enstjn5q","id":"1(@)"}
+{"time":"2026-04-26T01:11:13.23021914+08:00","level":"INFO","msg":"handleInformInit: stream started","streamId":"enstjn5q","id":"1(@)"}
+{"time":"2026-04-26T01:11:19.074431223+08:00","level":"INFO","msg":"connection: cancelling request","id":"1(@)","requestId":"b5y7d3enr9ht"}
+{"time":"2026-04-26T20:25:45.521130246+08:00","level":"INFO","msg":"server: parent process exited, terminating service process"}