qualcomm/DETR-ResNet50
12.1k
1---2library_name: pytorch3license: apache-2.04tags:5- bu_auto6- android7pipeline_tag: object-detection8 9---10 1112 13# DETR-ResNet50: Optimized for Qualcomm Devices14 15DETR is a machine learning model that can detect objects (trained on COCO dataset).16 17This is based on the implementation of DETR-ResNet50 found [here](https://github.com/facebookresearch/detr).18This repository contains pre-exported model files optimized for Qualcomm® devices. You can use the [Qualcomm® AI Hub Models](https://github.com/qualcomm/ai-hub-models/blob/v0.63.0/src/qai_hub_models/models/detr_resnet50) library to export with custom configurations. More details on model performance across various devices, can be found [here](#performance-summary).19 20Qualcomm AI Hub Models uses [Qualcomm AI Hub Workbench](https://workbench.aihub.qualcomm.com) to compile, profile, and evaluate this model. [Sign up](https://myaccount.qualcomm.com/signup) to run these models on a hosted Qualcomm® device.21 22## Getting Started23There are two ways to deploy this model on your device:24 25### Option 1: Download Pre-Exported Models26 27Below are pre-exported model assets ready for deployment.28 29| Runtime | Precision | Chipset | SDK Versions | Download |30|---|---|---|---|---|31| ONNX | float | Universal | QAIRT 2.50, ONNX Runtime 1.27.1 | [Download](https://qaihub-public-assets.s3.us-west-2.amazonaws.com/qai-hub-models/models/detr_resnet50/releases/v0.63.0/detr_resnet50-onnx-float.zip)32| ONNX | w8a16_mixed_fp16 | Universal | QAIRT 2.50, ONNX Runtime 1.27.1 | [Download](https://qaihub-public-assets.s3.us-west-2.amazonaws.com/qai-hub-models/models/detr_resnet50/releases/v0.63.0/detr_resnet50-onnx-w8a16_mixed_fp16.zip)33| QNN_DLC | float | Universal | QAIRT 2.50 | [Download](https://qaihub-public-assets.s3.us-west-2.amazonaws.com/qai-hub-models/models/detr_resnet50/releases/v0.63.0/detr_resnet50-qnn_dlc-float.zip)34| QNN_DLC | w8a16_mixed_fp16 | Universal | QAIRT 2.50 | [Download](https://qaihub-public-assets.s3.us-west-2.amazonaws.com/qai-hub-models/models/detr_resnet50/releases/v0.63.0/detr_resnet50-qnn_dlc-w8a16_mixed_fp16.zip)35| TFLITE | float | Universal | QAIRT 2.50 | [Download](https://qaihub-public-assets.s3.us-west-2.amazonaws.com/qai-hub-models/models/detr_resnet50/releases/v0.63.0/detr_resnet50-tflite-float.zip)36 37For more device-specific assets and performance metrics, visit **[DETR-ResNet50 on Qualcomm® AI Hub](https://aihub.qualcomm.com/models/detr_resnet50)**.38 39 40### Option 2: Export with Custom Configurations41 42Use the [Qualcomm® AI Hub Models](https://github.com/qualcomm/ai-hub-models/blob/v0.63.0/src/qai_hub_models/models/detr_resnet50) Python library to compile and export the model with your own:43- Custom weights (e.g., fine-tuned checkpoints)44- Custom input shapes45- Target device and runtime configurations46 47This option is ideal if you need to customize the model beyond the default configuration provided here.48 49See our repository for [DETR-ResNet50 on GitHub](https://github.com/qualcomm/ai-hub-models/blob/v0.63.0/src/qai_hub_models/models/detr_resnet50) for usage instructions.50 51## Model Details52 53**Model Type:** Model_use_case.object_detection54 55**Model Stats:**56- Input resolution: 480x48057- Model checkpoint: ResNet5058- Model size (float): 158 MB59- Number of parameters: 41.4M60 61## Performance Summary62| Model | Runtime | Precision | Chipset | Inference Time (ms) | Peak Memory Range (MB) | Primary Compute Unit63|---|---|---|---|---|---|---64| DETR-ResNet50 | ONNX | float | Snapdragon® 8 Elite Gen 5 For Galaxy Mobile | 8.257 ms | 3 - 278 MB | NPU65| DETR-ResNet50 | ONNX | float | Snapdragon® 8 Elite For Galaxy Mobile | 10.048 ms | 3 - 259 MB | NPU66| DETR-ResNet50 | ONNX | float | Snapdragon® X2 Elite | 8.215 ms | 5 - 5 MB | NPU67| DETR-ResNet50 | ONNX | float | Snapdragon® X Elite | 16.571 ms | 78 - 78 MB | NPU68| DETR-ResNet50 | ONNX | float | Snapdragon® 8 Gen 3 Mobile | 12.998 ms | 0 - 362 MB | NPU69| DETR-ResNet50 | ONNX | float | Snapdragon® 8 Gen 1 Mobile | 26.965 ms | 1 - 312 MB | NPU70| DETR-ResNet50 | ONNX | float | Qualcomm® Dragonwing™ IQ-8275 | 27.801 ms | 5 - 13 MB | NPU71| DETR-ResNet50 | ONNX | float | Qualcomm® Dragonwing™ QCS8550 (Proxy) | 16.508 ms | 0 - 95 MB | NPU72| DETR-ResNet50 | ONNX | float | Qualcomm® QCS8450 | 26.965 ms | 1 - 312 MB | NPU73| DETR-ResNet50 | ONNX | float | Qualcomm® Dragonwing™ IQ-9075 | 25.071 ms | 5 - 13 MB | NPU74| DETR-ResNet50 | ONNX | float | Qualcomm® Dragonwing™ IQ-X7181 | 16.571 ms | 78 - 78 MB | NPU75| DETR-ResNet50 | ONNX | float | Qualcomm® Dragonwing™ Q-8750 | 10.048 ms | 3 - 259 MB | NPU76| DETR-ResNet50 | ONNX | w8a16_mixed_fp16 | Snapdragon® 8 Elite Gen 5 For Galaxy Mobile | 24.766 ms | 21 - 289 MB | NPU77| DETR-ResNet50 | ONNX | w8a16_mixed_fp16 | Snapdragon® 8 Elite For Galaxy Mobile | 31.162 ms | 20 - 271 MB | NPU78| DETR-ResNet50 | ONNX | w8a16_mixed_fp16 | Snapdragon® 8 Gen 3 Mobile | 37.101 ms | 19 - 364 MB | NPU79| DETR-ResNet50 | ONNX | w8a16_mixed_fp16 | Qualcomm® Dragonwing™ IQ-8275 | 63.9 ms | 20 - 23 MB | NPU80| DETR-ResNet50 | ONNX | w8a16_mixed_fp16 | Qualcomm® Dragonwing™ QCS8550 (Proxy) | 44.458 ms | 0 - 49 MB | NPU81| DETR-ResNet50 | ONNX | w8a16_mixed_fp16 | Qualcomm® QCS8450 | 97.068 ms | 7 - 466 MB | NPU82| DETR-ResNet50 | ONNX | w8a16_mixed_fp16 | Qualcomm® Dragonwing™ IQ-9075 | 46.732 ms | 23 - 25 MB | NPU83| DETR-ResNet50 | ONNX | w8a16_mixed_fp16 | Qualcomm® Dragonwing™ Q-8750 | 31.162 ms | 20 - 271 MB | NPU84| DETR-ResNet50 | QNN_DLC | float | Snapdragon® 8 Elite Gen 5 For Galaxy Mobile | 7.92 ms | 5 - 278 MB | NPU85| DETR-ResNet50 | QNN_DLC | float | Snapdragon® 8 Elite For Galaxy Mobile | 10.313 ms | 5 - 264 MB | NPU86| DETR-ResNet50 | QNN_DLC | float | Snapdragon® X2 Elite | 9.25 ms | 5 - 5 MB | NPU87| DETR-ResNet50 | QNN_DLC | float | Snapdragon® X Elite | 19.24 ms | 5 - 5 MB | NPU88| DETR-ResNet50 | QNN_DLC | float | Snapdragon® 8 Gen 3 Mobile | 13.709 ms | 3 - 341 MB | NPU89| DETR-ResNet50 | QNN_DLC | float | Snapdragon® 8 Gen 1 Mobile | 28.189 ms | 4 - 292 MB | NPU90| DETR-ResNet50 | QNN_DLC | float | Qualcomm® Dragonwing™ IQ-8275 | 27.309 ms | 5 - 12 MB | NPU91| DETR-ResNet50 | QNN_DLC | float | Qualcomm® Dragonwing™ QCS8550 (Proxy) | 18.874 ms | 5 - 7 MB | NPU92| DETR-ResNet50 | QNN_DLC | float | Qualcomm® SA8650P | 26.853 ms | 1 - 300 MB | NPU93| DETR-ResNet50 | QNN_DLC | float | Qualcomm® SA8255P | 26.853 ms | 1 - 300 MB | NPU94| DETR-ResNet50 | QNN_DLC | float | Qualcomm® QCS8450 | 28.189 ms | 4 - 292 MB | NPU95| DETR-ResNet50 | QNN_DLC | float | Qualcomm® Dragonwing™ IQ-9075 | 27.22 ms | 7 - 14 MB | NPU96| DETR-ResNet50 | QNN_DLC | float | Qualcomm® Dragonwing™ IQ-X7181 | 19.24 ms | 5 - 5 MB | NPU97| DETR-ResNet50 | QNN_DLC | float | Qualcomm® Dragonwing™ Q-8750 | 10.313 ms | 5 - 264 MB | NPU98| DETR-ResNet50 | QNN_DLC | float | Qualcomm® SA8295P | 29.319 ms | 0 - 218 MB | NPU99| DETR-ResNet50 | QNN_DLC | w8a16_mixed_fp16 | Snapdragon® 8 Elite Gen 5 For Galaxy Mobile | 10.826 ms | 2 - 306 MB | NPU100| DETR-ResNet50 | QNN_DLC | w8a16_mixed_fp16 | Snapdragon® 8 Elite For Galaxy Mobile | 12.726 ms | 0 - 286 MB | NPU101| DETR-ResNet50 | QNN_DLC | w8a16_mixed_fp16 | Snapdragon® X2 Elite | 11.199 ms | 2 - 2 MB | NPU102| DETR-ResNet50 | QNN_DLC | w8a16_mixed_fp16 | Snapdragon® X Elite | 23.917 ms | 2 - 2 MB | NPU103| DETR-ResNet50 | QNN_DLC | w8a16_mixed_fp16 | Snapdragon® 8 Gen 3 Mobile | 17.728 ms | 2 - 370 MB | NPU104| DETR-ResNet50 | QNN_DLC | w8a16_mixed_fp16 | Qualcomm® Dragonwing™ IQ-8275 | 28.567 ms | 2 - 8 MB | NPU105| DETR-ResNet50 | QNN_DLC | w8a16_mixed_fp16 | Qualcomm® Dragonwing™ QCS8550 (Proxy) | 24.229 ms | 2 - 5 MB | NPU106| DETR-ResNet50 | QNN_DLC | w8a16_mixed_fp16 | Qualcomm® Dragonwing™ IQ-9075 | 28.548 ms | 2 - 6 MB | NPU107| DETR-ResNet50 | QNN_DLC | w8a16_mixed_fp16 | Qualcomm® Dragonwing™ IQ-X7181 | 23.917 ms | 2 - 2 MB | NPU108| DETR-ResNet50 | QNN_DLC | w8a16_mixed_fp16 | Qualcomm® Dragonwing™ Q-8750 | 12.726 ms | 0 - 286 MB | NPU109| DETR-ResNet50 | TFLITE | float | Snapdragon® 8 Elite Gen 5 For Galaxy Mobile | 7.948 ms | 0 - 301 MB | NPU110| DETR-ResNet50 | TFLITE | float | Snapdragon® 8 Elite For Galaxy Mobile | 10.35 ms | 0 - 290 MB | NPU111| DETR-ResNet50 | TFLITE | float | Snapdragon® 8 Gen 3 Mobile | 13.575 ms | 0 - 362 MB | NPU112| DETR-ResNet50 | TFLITE | float | Snapdragon® 8 Gen 1 Mobile | 27.991 ms | 0 - 313 MB | NPU113| DETR-ResNet50 | TFLITE | float | Qualcomm® Dragonwing™ IQ-8275 | 27.444 ms | 0 - 89 MB | NPU114| DETR-ResNet50 | TFLITE | float | Qualcomm® Dragonwing™ QCS8550 (Proxy) | 19.107 ms | 0 - 2 MB | NPU115| DETR-ResNet50 | TFLITE | float | Qualcomm® SA8650P | 26.917 ms | 0 - 323 MB | NPU116| DETR-ResNet50 | TFLITE | float | Qualcomm® SA8255P | 26.917 ms | 0 - 323 MB | NPU117| DETR-ResNet50 | TFLITE | float | Qualcomm® QCS8450 | 27.991 ms | 0 - 313 MB | NPU118| DETR-ResNet50 | TFLITE | float | Qualcomm® Dragonwing™ IQ-9075 | 26.677 ms | 0 - 89 MB | NPU119| DETR-ResNet50 | TFLITE | float | Qualcomm® Dragonwing™ Q-8750 | 10.35 ms | 0 - 290 MB | NPU120| DETR-ResNet50 | TFLITE | float | Qualcomm® SA8295P | 29.311 ms | 0 - 237 MB | NPU121 122## License123* The license for the original implementation of DETR-ResNet50 can be found124 [here](https://github.com/facebookresearch/detr/blob/main/LICENSE).125 126## References127* [End-to-End Object Detection with Transformers](https://arxiv.org/abs/2005.12872)128* [Source Model Implementation](https://github.com/facebookresearch/detr)129 130## Community131* Join [our AI Hub Slack community](https://aihub.qualcomm.com/community/slack) to collaborate, post questions and learn more about on-device AI.132* For questions or feedback please [reach out to us](mailto:ai-hub-support@qti.qualcomm.com).133