--- library_name: pytorch license: other tags: - real_time - android pipeline_tag: image-segmentation --- ![](https://qaihub-public-assets.s3.us-west-2.amazonaws.com/qai-hub-models/models/pidnet/web-assets/model_demo.png) # PidNet: Optimized for Qualcomm Devices PIDNet (Proportional-Integral-Derivative Network) is a real-time semantic segmentation model based on PID controllers This is based on the implementation of PidNet found [here](https://github.com/XuJiacong/PIDNet). This repository contains pre-exported model files optimized for Qualcomm® devices. You can use the [Qualcomm® AI Hub Models](https://github.com/qualcomm/ai-hub-models/blob/main/src/qai_hub_models/models/pidnet) library to export with custom configurations. More details on model performance across various devices, can be found [here](#performance-summary). Qualcomm AI Hub Models uses [Qualcomm AI Hub Workbench](https://workbench.aihub.qualcomm.com) to compile, profile, and evaluate this model. [Sign up](https://myaccount.qualcomm.com/signup) to run these models on a hosted Qualcomm® device. ## Getting Started There are two ways to deploy this model on your device: ### Option 1: Download Pre-Exported Models Below are pre-exported model assets ready for deployment. | Runtime | Precision | Chipset | SDK Versions | Download | |---|---|---|---|---| | ONNX | float | Universal | QAIRT 2.42, ONNX Runtime 1.24.3 | [Download](https://qaihub-public-assets.s3.us-west-2.amazonaws.com/qai-hub-models/models/pidnet/releases/v0.50.2/pidnet-onnx-float.zip) | ONNX | w8a8 | Universal | QAIRT 2.42, ONNX Runtime 1.24.3 | [Download](https://qaihub-public-assets.s3.us-west-2.amazonaws.com/qai-hub-models/models/pidnet/releases/v0.50.2/pidnet-onnx-w8a8.zip) | QNN_DLC | float | Universal | QAIRT 2.43 | [Download](https://qaihub-public-assets.s3.us-west-2.amazonaws.com/qai-hub-models/models/pidnet/releases/v0.50.2/pidnet-qnn_dlc-float.zip) | QNN_DLC | w8a8 | Universal | QAIRT 2.43 | [Download](https://qaihub-public-assets.s3.us-west-2.amazonaws.com/qai-hub-models/models/pidnet/releases/v0.50.2/pidnet-qnn_dlc-w8a8.zip) | TFLITE | float | Universal | QAIRT 2.43, TFLite 2.19.1 | [Download](https://qaihub-public-assets.s3.us-west-2.amazonaws.com/qai-hub-models/models/pidnet/releases/v0.50.2/pidnet-tflite-float.zip) | TFLITE | w8a8 | Universal | QAIRT 2.43, TFLite 2.19.1 | [Download](https://qaihub-public-assets.s3.us-west-2.amazonaws.com/qai-hub-models/models/pidnet/releases/v0.50.2/pidnet-tflite-w8a8.zip) For more device-specific assets and performance metrics, visit **[PidNet on Qualcomm® AI Hub](https://aihub.qualcomm.com/models/pidnet)**. ### Option 2: Export with Custom Configurations Use the [Qualcomm® AI Hub Models](https://github.com/qualcomm/ai-hub-models/blob/main/src/qai_hub_models/models/pidnet) Python library to compile and export the model with your own: - Custom weights (e.g., fine-tuned checkpoints) - Custom input shapes - Target device and runtime configurations This option is ideal if you need to customize the model beyond the default configuration provided here. See our repository for [PidNet on GitHub](https://github.com/qualcomm/ai-hub-models/blob/main/src/qai_hub_models/models/pidnet) for usage instructions. ## Model Details **Model Type:** Model_use_case.semantic_segmentation **Model Stats:** - Model checkpoint: PIDNet_S_Cityscapes_val.pt - Inference latency: RealTime - Input resolution: 1024x2048 - Number of output classes: 19 - Number of parameters: 8.06M - Model size (float): 29.1 MB - Model size (w8a8): 8.02 MB ## Performance Summary | Model | Runtime | Precision | Chipset | Inference Time (ms) | Peak Memory Range (MB) | Primary Compute Unit |---|---|---|---|---|---|--- | PidNet | ONNX | float | Snapdragon® 8 Elite Gen 5 Mobile | 12.164 ms | 31 - 299 MB | NPU | PidNet | ONNX | float | Snapdragon® X2 Elite | 12.927 ms | 22 - 22 MB | NPU | PidNet | ONNX | float | Snapdragon® X Elite | 32.82 ms | 24 - 24 MB | NPU | PidNet | ONNX | float | Snapdragon® 8 Gen 3 Mobile | 22.949 ms | 32 - 349 MB | NPU | PidNet | ONNX | float | Qualcomm® QCS8550 (Proxy) | 32.516 ms | 24 - 26 MB | NPU | PidNet | ONNX | float | Qualcomm® QCS9075 | 47.406 ms | 24 - 51 MB | NPU | PidNet | ONNX | float | Snapdragon® 8 Elite For Galaxy Mobile | 16.324 ms | 6 - 225 MB | NPU | PidNet | ONNX | w8a8 | Snapdragon® 8 Elite Gen 5 Mobile | 44.924 ms | 94 - 327 MB | NPU | PidNet | ONNX | w8a8 | Snapdragon® X2 Elite | 45.18 ms | 133 - 133 MB | NPU | PidNet | ONNX | w8a8 | Snapdragon® X Elite | 62.602 ms | 131 - 131 MB | NPU | PidNet | ONNX | w8a8 | Snapdragon® 8 Gen 3 Mobile | 45.704 ms | 107 - 381 MB | NPU | PidNet | ONNX | w8a8 | Qualcomm® QCS6490 | 388.418 ms | 197 - 215 MB | CPU | PidNet | ONNX | w8a8 | Qualcomm® QCS8550 (Proxy) | 59.488 ms | 97 - 104 MB | NPU | PidNet | ONNX | w8a8 | Qualcomm® QCS9075 | 64.586 ms | 100 - 103 MB | NPU | PidNet | ONNX | w8a8 | Qualcomm® QCM6690 | 346.538 ms | 129 - 138 MB | CPU | PidNet | ONNX | w8a8 | Snapdragon® 8 Elite For Galaxy Mobile | 44.51 ms | 101 - 319 MB | NPU | PidNet | ONNX | w8a8 | Snapdragon® 7 Gen 4 Mobile | 327.615 ms | 134 - 143 MB | CPU | PidNet | QNN_DLC | float | Snapdragon® 8 Elite Gen 5 Mobile | 11.942 ms | 24 - 296 MB | NPU | PidNet | QNN_DLC | float | Snapdragon® X2 Elite | 13.424 ms | 24 - 24 MB | NPU | PidNet | QNN_DLC | float | Snapdragon® X Elite | 38.874 ms | 24 - 24 MB | NPU | PidNet | QNN_DLC | float | Snapdragon® 8 Gen 3 Mobile | 25.702 ms | 18 - 337 MB | NPU | PidNet | QNN_DLC | float | Qualcomm® QCS8275 (Proxy) | 118.298 ms | 24 - 246 MB | NPU | PidNet | QNN_DLC | float | Qualcomm® QCS8550 (Proxy) | 37.675 ms | 24 - 26 MB | NPU | PidNet | QNN_DLC | float | Qualcomm® SA8775P | 46.714 ms | 24 - 247 MB | NPU | PidNet | QNN_DLC | float | Qualcomm® QCS9075 | 61.699 ms | 24 - 52 MB | NPU | PidNet | QNN_DLC | float | Qualcomm® QCS8450 (Proxy) | 76.751 ms | 5 - 334 MB | NPU | PidNet | QNN_DLC | float | Qualcomm® SA7255P | 118.298 ms | 24 - 246 MB | NPU | PidNet | QNN_DLC | float | Qualcomm® SA8295P | 51.765 ms | 24 - 260 MB | NPU | PidNet | QNN_DLC | float | Snapdragon® 8 Elite For Galaxy Mobile | 18.276 ms | 21 - 266 MB | NPU | PidNet | QNN_DLC | w8a8 | Snapdragon® 8 Elite Gen 5 Mobile | 47.613 ms | 6 - 270 MB | NPU | PidNet | QNN_DLC | w8a8 | Snapdragon® X2 Elite | 48.386 ms | 6 - 6 MB | NPU | PidNet | QNN_DLC | w8a8 | Snapdragon® X Elite | 60.352 ms | 6 - 6 MB | NPU | PidNet | QNN_DLC | w8a8 | Snapdragon® 8 Gen 3 Mobile | 43.153 ms | 6 - 272 MB | NPU | PidNet | QNN_DLC | w8a8 | Qualcomm® QCS8275 (Proxy) | 111.214 ms | 6 - 218 MB | NPU | PidNet | QNN_DLC | w8a8 | Qualcomm® QCS8550 (Proxy) | 57.898 ms | 6 - 8 MB | NPU | PidNet | QNN_DLC | w8a8 | Qualcomm® SA8775P | 58.479 ms | 6 - 218 MB | NPU | PidNet | QNN_DLC | w8a8 | Qualcomm® QCS9075 | 61.958 ms | 6 - 14 MB | NPU | PidNet | QNN_DLC | w8a8 | Qualcomm® QCS8450 (Proxy) | 63.925 ms | 6 - 274 MB | NPU | PidNet | QNN_DLC | w8a8 | Qualcomm® SA7255P | 111.214 ms | 6 - 218 MB | NPU | PidNet | QNN_DLC | w8a8 | Qualcomm® SA8295P | 66.474 ms | 6 - 220 MB | NPU | PidNet | QNN_DLC | w8a8 | Snapdragon® 8 Elite For Galaxy Mobile | 41.463 ms | 6 - 241 MB | NPU | PidNet | TFLITE | float | Snapdragon® 8 Elite Gen 5 Mobile | 12.705 ms | 2 - 272 MB | NPU | PidNet | TFLITE | float | Snapdragon® 8 Gen 3 Mobile | 24.882 ms | 1 - 334 MB | NPU | PidNet | TFLITE | float | Qualcomm® QCS8275 (Proxy) | 117.12 ms | 3 - 232 MB | NPU | PidNet | TFLITE | float | Qualcomm® QCS8550 (Proxy) | 35.77 ms | 0 - 2 MB | NPU | PidNet | TFLITE | float | Qualcomm® SA8775P | 45.274 ms | 2 - 233 MB | NPU | PidNet | TFLITE | float | Qualcomm® QCS9075 | 59.677 ms | 0 - 45 MB | NPU | PidNet | TFLITE | float | Qualcomm® QCS8450 (Proxy) | 76.026 ms | 2 - 350 MB | NPU | PidNet | TFLITE | float | Qualcomm® SA7255P | 117.12 ms | 3 - 232 MB | NPU | PidNet | TFLITE | float | Qualcomm® SA8295P | 50.659 ms | 2 - 250 MB | NPU | PidNet | TFLITE | float | Snapdragon® 8 Elite For Galaxy Mobile | 18.96 ms | 1 - 258 MB | NPU | PidNet | TFLITE | w8a8 | Snapdragon® 8 Elite Gen 5 Mobile | 40.653 ms | 1 - 265 MB | NPU | PidNet | TFLITE | w8a8 | Snapdragon® 8 Gen 3 Mobile | 38.135 ms | 0 - 266 MB | NPU | PidNet | TFLITE | w8a8 | Qualcomm® QCS6490 | 210.57 ms | 2 - 72 MB | NPU | PidNet | TFLITE | w8a8 | Qualcomm® QCS8275 (Proxy) | 98.263 ms | 1 - 212 MB | NPU | PidNet | TFLITE | w8a8 | Qualcomm® QCS8550 (Proxy) | 50.321 ms | 1 - 2 MB | NPU | PidNet | TFLITE | w8a8 | Qualcomm® SA8775P | 50.963 ms | 1 - 212 MB | NPU | PidNet | TFLITE | w8a8 | Qualcomm® QCS9075 | 53.14 ms | 0 - 16 MB | NPU | PidNet | TFLITE | w8a8 | Qualcomm® QCM6690 | 238.856 ms | 3 - 198 MB | NPU | PidNet | TFLITE | w8a8 | Qualcomm® QCS8450 (Proxy) | 58.319 ms | 0 - 267 MB | NPU | PidNet | TFLITE | w8a8 | Qualcomm® SA7255P | 98.263 ms | 1 - 212 MB | NPU | PidNet | TFLITE | w8a8 | Qualcomm® SA8295P | 57.97 ms | 1 - 215 MB | NPU | PidNet | TFLITE | w8a8 | Snapdragon® 8 Elite For Galaxy Mobile | 38.265 ms | 1 - 237 MB | NPU | PidNet | TFLITE | w8a8 | Snapdragon® 7 Gen 4 Mobile | 66.782 ms | 3 - 222 MB | NPU ## License * The license for the original implementation of PidNet can be found [here](https://github.com/XuJiacong/PIDNet/blob/main/LICENSE). ## References * [PIDNet A Real-time Semantic Segmentation Network Inspired from PID Controller Segmentation of Road Scenes](https://arxiv.org/abs/2206.02066) * [Source Model Implementation](https://github.com/XuJiacong/PIDNet) ## Community * Join [our AI Hub Slack community](https://aihub.qualcomm.com/community/slack) to collaborate, post questions and learn more about on-device AI. * For questions or feedback please [reach out to us](mailto:ai-hub-support@qti.qualcomm.com).