Hugging Face's logo

qualcomm
/
ACT

Robotics
PyTorch
android
Model card Files
xet
 Community
qaihm-bot commited on
Commit
ca6375f
·
verified ·
1 Parent(s): b2b8506

v0.49.1

Browse files

See https://github.com/qualcomm/ai-hub-models/releases/v0.49.1 for changelog.

Files changed (1) hide show
  1. README.md +34 -34
README.md CHANGED
@@ -14,7 +14,7 @@ pipeline_tag: robotics
14
  ACT (Action Chunking with Transformers) is a robotic policy model that is trained to predict the next chunk of actions that the robotic hand is expected to perform.
15
 
16
  This is based on the implementation of ACT found [here](https://github.com/tonyzhaozh/act).
17
- This repository contains pre-exported model files optimized for Qualcomm® devices. You can use the [Qualcomm® AI Hub Models](https://github.com/qualcomm/ai-hub-models/blob/main/qai_hub_models/models/act) library to export with custom configurations. More details on model performance across various devices, can be found [here](#performance-summary).
18
 
19
  Qualcomm AI Hub Models uses [Qualcomm AI Hub Workbench](https://workbench.aihub.qualcomm.com) to compile, profile, and evaluate this model. [Sign up](https://myaccount.qualcomm.com/signup) to run these models on a hosted Qualcomm® device.
20
 
@@ -27,23 +27,23 @@ Below are pre-exported model assets ready for deployment.
27
 
28
  | Runtime | Precision | Chipset | SDK Versions | Download |
29
  |---|---|---|---|---|
30
- | ONNX | float | Universal | QAIRT 2.42, ONNX Runtime 1.24.1 | [Download](https://qaihub-public-assets.s3.us-west-2.amazonaws.com/qai-hub-models/models/act/releases/v0.48.0/act-onnx-float.zip)
31
- | QNN_DLC | float | Universal | QAIRT 2.43 | [Download](https://qaihub-public-assets.s3.us-west-2.amazonaws.com/qai-hub-models/models/act/releases/v0.48.0/act-qnn_dlc-float.zip)
32
- | TFLITE | float | Universal | QAIRT 2.43, TFLite 2.17.0 | [Download](https://qaihub-public-assets.s3.us-west-2.amazonaws.com/qai-hub-models/models/act/releases/v0.48.0/act-tflite-float.zip)
33
 
34
  For more device-specific assets and performance metrics, visit **[ACT on Qualcomm® AI Hub](https://aihub.qualcomm.com/models/act)**.
35
 
36
 
37
  ### Option 2: Export with Custom Configurations
38
 
39
- Use the [Qualcomm® AI Hub Models](https://github.com/qualcomm/ai-hub-models/blob/main/qai_hub_models/models/act) Python library to compile and export the model with your own:
40
  - Custom weights (e.g., fine-tuned checkpoints)
41
  - Custom input shapes
42
  - Target device and runtime configurations
43
 
44
  This option is ideal if you need to customize the model beyond the default configuration provided here.
45
 
46
- See our repository for [ACT on GitHub](https://github.com/qualcomm/ai-hub-models/blob/main/qai_hub_models/models/act) for usage instructions.
47
 
48
  ## Model Details
49
 
@@ -58,35 +58,35 @@ See our repository for [ACT on GitHub](https://github.com/qualcomm/ai-hub-models
58
  ## Performance Summary
59
  | Model | Runtime | Precision | Chipset | Inference Time (ms) | Peak Memory Range (MB) | Primary Compute Unit
60
  |---|---|---|---|---|---|---
61
- | ACT | ONNX | float | Snapdragon® X2 Elite | 6.165 ms | 63 - 63 MB | NPU
62
- | ACT | ONNX | float | Snapdragon® X Elite | 11.872 ms | 62 - 62 MB | NPU
63
- | ACT | ONNX | float | Snapdragon® 8 Gen 3 Mobile | 8.103 ms | 3 - 385 MB | NPU
64
- | ACT | ONNX | float | Qualcomm® QCS8550 (Proxy) | 11.214 ms | 0 - 81 MB | NPU
65
- | ACT | ONNX | float | Qualcomm® QCS9075 | 18.835 ms | 4 - 10 MB | NPU
66
- | ACT | ONNX | float | Snapdragon® 8 Elite For Galaxy Mobile | 6.573 ms | 3 - 331 MB | NPU
67
- | ACT | ONNX | float | Snapdragon® 8 Elite Gen 5 Mobile | 5.496 ms | 4 - 334 MB | NPU
68
- | ACT | QNN_DLC | float | Snapdragon® X2 Elite | 4.932 ms | 4 - 4 MB | NPU
69
- | ACT | QNN_DLC | float | Snapdragon® X Elite | 8.926 ms | 4 - 4 MB | NPU
70
- | ACT | QNN_DLC | float | Snapdragon® 8 Gen 3 Mobile | 5.974 ms | 1 - 344 MB | NPU
71
- | ACT | QNN_DLC | float | Qualcomm® QCS8275 (Proxy) | 43.942 ms | 1 - 308 MB | NPU
72
- | ACT | QNN_DLC | float | Qualcomm® QCS8550 (Proxy) | 8.288 ms | 4 - 7 MB | NPU
73
- | ACT | QNN_DLC | float | Qualcomm® SA8775P | 13.325 ms | 1 - 294 MB | NPU
74
- | ACT | QNN_DLC | float | Qualcomm® QCS9075 | 15.977 ms | 4 - 9 MB | NPU
75
- | ACT | QNN_DLC | float | Qualcomm® QCS8450 (Proxy) | 16.41 ms | 2 - 263 MB | NPU
76
- | ACT | QNN_DLC | float | Qualcomm® SA7255P | 43.942 ms | 1 - 308 MB | NPU
77
- | ACT | QNN_DLC | float | Qualcomm® SA8295P | 15.517 ms | 0 - 230 MB | NPU
78
- | ACT | QNN_DLC | float | Snapdragon® 8 Elite For Galaxy Mobile | 4.768 ms | 4 - 299 MB | NPU
79
- | ACT | QNN_DLC | float | Snapdragon® 8 Elite Gen 5 Mobile | 3.739 ms | 4 - 321 MB | NPU
80
- | ACT | TFLITE | float | Snapdragon® 8 Gen 3 Mobile | 5.981 ms | 0 - 361 MB | NPU
81
- | ACT | TFLITE | float | Qualcomm® QCS8275 (Proxy) | 44.134 ms | 0 - 318 MB | NPU
82
- | ACT | TFLITE | float | Qualcomm® QCS8550 (Proxy) | 8.318 ms | 0 - 3 MB | NPU
83
- | ACT | TFLITE | float | Qualcomm® SA8775P | 13.44 ms | 0 - 304 MB | NPU
84
- | ACT | TFLITE | float | Qualcomm® QCS9075 | 16.034 ms | 0 - 71 MB | NPU
85
- | ACT | TFLITE | float | Qualcomm® QCS8450 (Proxy) | 16.392 ms | 0 - 276 MB | NPU
86
- | ACT | TFLITE | float | Qualcomm® SA7255P | 44.134 ms | 0 - 318 MB | NPU
87
- | ACT | TFLITE | float | Qualcomm® SA8295P | 15.711 ms | 0 - 233 MB | NPU
88
- | ACT | TFLITE | float | Snapdragon® 8 Elite For Galaxy Mobile | 4.816 ms | 0 - 310 MB | NPU
89
  | ACT | TFLITE | float | Snapdragon® 8 Elite Gen 5 Mobile | 3.799 ms | 0 - 327 MB | NPU
 
 
 
 
 
 
 
 
 
90
 
91
  ## License
92
  * The license for the original implementation of ACT can be found
 
14
  ACT (Action Chunking with Transformers) is a robotic policy model that is trained to predict the next chunk of actions that the robotic hand is expected to perform.
15
 
16
  This is based on the implementation of ACT found [here](https://github.com/tonyzhaozh/act).
17
+ This repository contains pre-exported model files optimized for Qualcomm® devices. You can use the [Qualcomm® AI Hub Models](https://github.com/qualcomm/ai-hub-models/tree/v0.49.1/qai_hub_models/models/act) library to export with custom configurations. More details on model performance across various devices, can be found [here](#performance-summary).
18
 
19
  Qualcomm AI Hub Models uses [Qualcomm AI Hub Workbench](https://workbench.aihub.qualcomm.com) to compile, profile, and evaluate this model. [Sign up](https://myaccount.qualcomm.com/signup) to run these models on a hosted Qualcomm® device.
20
 
 
27
 
28
  | Runtime | Precision | Chipset | SDK Versions | Download |
29
  |---|---|---|---|---|
30
+ | ONNX | float | Universal | QAIRT 2.42, ONNX Runtime 1.24.1 | [Download](https://qaihub-public-assets.s3.us-west-2.amazonaws.com/qai-hub-models/models/act/releases/v0.49.1/act-onnx-float.zip)
31
+ | QNN_DLC | float | Universal | QAIRT 2.43 | [Download](https://qaihub-public-assets.s3.us-west-2.amazonaws.com/qai-hub-models/models/act/releases/v0.49.1/act-qnn_dlc-float.zip)
32
+ | TFLITE | float | Universal | QAIRT 2.43, TFLite 2.17.0 | [Download](https://qaihub-public-assets.s3.us-west-2.amazonaws.com/qai-hub-models/models/act/releases/v0.49.1/act-tflite-float.zip)
33
 
34
  For more device-specific assets and performance metrics, visit **[ACT on Qualcomm® AI Hub](https://aihub.qualcomm.com/models/act)**.
35
 
36
 
37
  ### Option 2: Export with Custom Configurations
38
 
39
+ Use the [Qualcomm® AI Hub Models](https://github.com/qualcomm/ai-hub-models/tree/v0.49.1/qai_hub_models/models/act) Python library to compile and export the model with your own:
40
  - Custom weights (e.g., fine-tuned checkpoints)
41
  - Custom input shapes
42
  - Target device and runtime configurations
43
 
44
  This option is ideal if you need to customize the model beyond the default configuration provided here.
45
 
46
+ See our repository for [ACT on GitHub](https://github.com/qualcomm/ai-hub-models/tree/v0.49.1/qai_hub_models/models/act) for usage instructions.
47
 
48
  ## Model Details
49
 
 
58
  ## Performance Summary
59
  | Model | Runtime | Precision | Chipset | Inference Time (ms) | Peak Memory Range (MB) | Primary Compute Unit
60
  |---|---|---|---|---|---|---
61
+ | ACT | ONNX | float | Snapdragon® 8 Elite Gen 5 Mobile | 5.518 ms | 4 - 334 MB | NPU
62
+ | ACT | ONNX | float | Snapdragon® X2 Elite | 6.176 ms | 63 - 63 MB | NPU
63
+ | ACT | ONNX | float | Snapdragon® X Elite | 11.87 ms | 62 - 62 MB | NPU
64
+ | ACT | ONNX | float | Snapdragon® 8 Gen 3 Mobile | 8.053 ms | 2 - 377 MB | NPU
65
+ | ACT | ONNX | float | Qualcomm® QCS8550 (Proxy) | 11.249 ms | 0 - 81 MB | NPU
66
+ | ACT | ONNX | float | Qualcomm® QCS9075 | 18.602 ms | 4 - 10 MB | NPU
67
+ | ACT | ONNX | float | Snapdragon® 8 Elite For Galaxy Mobile | 6.567 ms | 1 - 331 MB | NPU
68
+ | ACT | QNN_DLC | float | Snapdragon® 8 Elite Gen 5 Mobile | 3.739 ms | 3 - 321 MB | NPU
69
+ | ACT | QNN_DLC | float | Snapdragon® X2 Elite | 4.903 ms | 4 - 4 MB | NPU
70
+ | ACT | QNN_DLC | float | Snapdragon® X Elite | 8.9 ms | 4 - 4 MB | NPU
71
+ | ACT | QNN_DLC | float | Snapdragon® 8 Gen 3 Mobile | 5.957 ms | 0 - 345 MB | NPU
72
+ | ACT | QNN_DLC | float | Qualcomm® QCS8275 (Proxy) | 43.945 ms | 2 - 309 MB | NPU
73
+ | ACT | QNN_DLC | float | Qualcomm® QCS8550 (Proxy) | 8.287 ms | 4 - 6 MB | NPU
74
+ | ACT | QNN_DLC | float | Qualcomm® SA8775P | 13.344 ms | 1 - 294 MB | NPU
75
+ | ACT | QNN_DLC | float | Qualcomm® QCS9075 | 15.692 ms | 4 - 9 MB | NPU
76
+ | ACT | QNN_DLC | float | Qualcomm® QCS8450 (Proxy) | 16.31 ms | 0 - 260 MB | NPU
77
+ | ACT | QNN_DLC | float | Qualcomm® SA7255P | 43.945 ms | 2 - 309 MB | NPU
78
+ | ACT | QNN_DLC | float | Qualcomm® SA8295P | 15.499 ms | 0 - 231 MB | NPU
79
+ | ACT | QNN_DLC | float | Snapdragon® 8 Elite For Galaxy Mobile | 4.795 ms | 0 - 297 MB | NPU
 
 
 
 
 
 
 
 
 
80
  | ACT | TFLITE | float | Snapdragon® 8 Elite Gen 5 Mobile | 3.799 ms | 0 - 327 MB | NPU
81
+ | ACT | TFLITE | float | Snapdragon® 8 Gen 3 Mobile | 5.992 ms | 0 - 360 MB | NPU
82
+ | ACT | TFLITE | float | Qualcomm® QCS8275 (Proxy) | 44.193 ms | 0 - 319 MB | NPU
83
+ | ACT | TFLITE | float | Qualcomm® QCS8550 (Proxy) | 8.252 ms | 0 - 2 MB | NPU
84
+ | ACT | TFLITE | float | Qualcomm® SA8775P | 13.455 ms | 0 - 304 MB | NPU
85
+ | ACT | TFLITE | float | Qualcomm® QCS9075 | 16.342 ms | 0 - 71 MB | NPU
86
+ | ACT | TFLITE | float | Qualcomm® QCS8450 (Proxy) | 16.296 ms | 0 - 275 MB | NPU
87
+ | ACT | TFLITE | float | Qualcomm® SA7255P | 44.193 ms | 0 - 319 MB | NPU
88
+ | ACT | TFLITE | float | Qualcomm® SA8295P | 15.755 ms | 0 - 235 MB | NPU
89
+ | ACT | TFLITE | float | Snapdragon® 8 Elite For Galaxy Mobile | 4.845 ms | 0 - 309 MB | NPU
90
 
91
  ## License
92
  * The license for the original implementation of ACT can be found