qualcomm/FFNet-54S
11.6k
1---2library_name: pytorch3license: bsd-3-clause4tags:5- android6pipeline_tag: image-segmentation7 8---9 1011 12# FFNet-54S: Optimized for Qualcomm Devices13 14FFNet-54S is a "fuss-free network" that segments street scene images with per-pixel classes like road, sidewalk, and pedestrian. Trained on the Cityscapes dataset.15 16This is based on the implementation of FFNet-54S found [here](https://github.com/Qualcomm-AI-research/FFNet).17This repository contains pre-exported model files optimized for Qualcomm® devices. You can use the [Qualcomm® AI Hub Models](https://github.com/qualcomm/ai-hub-models/blob/v0.63.0/src/qai_hub_models/models/ffnet_54s) library to export with custom configurations. More details on model performance across various devices, can be found [here](#performance-summary).18 19Qualcomm AI Hub Models uses [Qualcomm AI Hub Workbench](https://workbench.aihub.qualcomm.com) to compile, profile, and evaluate this model. [Sign up](https://myaccount.qualcomm.com/signup) to run these models on a hosted Qualcomm® device.20 21## Getting Started22There are two ways to deploy this model on your device:23 24### Option 1: Download Pre-Exported Models25 26Below are pre-exported model assets ready for deployment.27 28| Runtime | Precision | Chipset | SDK Versions | Download |29|---|---|---|---|---|30| ONNX | float | Universal | QAIRT 2.50, ONNX Runtime 1.27.1 | [Download](https://qaihub-public-assets.s3.us-west-2.amazonaws.com/qai-hub-models/models/ffnet_54s/releases/v0.63.0/ffnet_54s-onnx-float.zip)31| ONNX | w8a8 | Universal | QAIRT 2.50, ONNX Runtime 1.27.1 | [Download](https://qaihub-public-assets.s3.us-west-2.amazonaws.com/qai-hub-models/models/ffnet_54s/releases/v0.63.0/ffnet_54s-onnx-w8a8.zip)32| QNN_DLC | float | Universal | QAIRT 2.50 | [Download](https://qaihub-public-assets.s3.us-west-2.amazonaws.com/qai-hub-models/models/ffnet_54s/releases/v0.63.0/ffnet_54s-qnn_dlc-float.zip)33| QNN_DLC | w8a8 | Universal | QAIRT 2.50 | [Download](https://qaihub-public-assets.s3.us-west-2.amazonaws.com/qai-hub-models/models/ffnet_54s/releases/v0.63.0/ffnet_54s-qnn_dlc-w8a8.zip)34| TFLITE | float | Universal | QAIRT 2.50 | [Download](https://qaihub-public-assets.s3.us-west-2.amazonaws.com/qai-hub-models/models/ffnet_54s/releases/v0.63.0/ffnet_54s-tflite-float.zip)35| TFLITE | w8a8 | Universal | QAIRT 2.50 | [Download](https://qaihub-public-assets.s3.us-west-2.amazonaws.com/qai-hub-models/models/ffnet_54s/releases/v0.63.0/ffnet_54s-tflite-w8a8.zip)36 37For more device-specific assets and performance metrics, visit **[FFNet-54S on Qualcomm® AI Hub](https://aihub.qualcomm.com/models/ffnet_54s)**.38 39 40### Option 2: Export with Custom Configurations41 42Use the [Qualcomm® AI Hub Models](https://github.com/qualcomm/ai-hub-models/blob/v0.63.0/src/qai_hub_models/models/ffnet_54s) Python library to compile and export the model with your own:43- Custom weights (e.g., fine-tuned checkpoints)44- Custom input shapes45- Target device and runtime configurations46 47This option is ideal if you need to customize the model beyond the default configuration provided here.48 49See our repository for [FFNet-54S on GitHub](https://github.com/qualcomm/ai-hub-models/blob/v0.63.0/src/qai_hub_models/models/ffnet_54s) for usage instructions.50 51## Model Details52 53**Model Type:** Model_use_case.semantic_segmentation54 55**Model Stats:**56- Input resolution: 2048x102457- Model checkpoint: ffnet54S_dBBB_cityscapes_state_dict_quarts58- Model size (float): 68.8 MB59- Model size (w8a8): 17.5 MB60- Number of output classes: 1961- Number of parameters: 18.0M62 63## Performance Summary64| Model | Runtime | Precision | Chipset | Inference Time (ms) | Peak Memory Range (MB) | Primary Compute Unit65|---|---|---|---|---|---|---66| FFNet-54S | ONNX | float | Snapdragon® 8 Elite Gen 5 For Galaxy Mobile | 14.3 ms | 5 - 212 MB | NPU67| FFNet-54S | ONNX | float | Snapdragon® 8 Elite For Galaxy Mobile | 18.454 ms | 7 - 192 MB | NPU68| FFNet-54S | ONNX | float | Snapdragon® X2 Elite | 15.401 ms | 24 - 24 MB | NPU69| FFNet-54S | ONNX | float | Snapdragon® X Elite | 32.526 ms | 24 - 24 MB | NPU70| FFNet-54S | ONNX | float | Snapdragon® 8 Gen 3 Mobile | 27.214 ms | 1 - 268 MB | NPU71| FFNet-54S | ONNX | float | Snapdragon® 8 Gen 1 Mobile | 50.172 ms | 28 - 287 MB | NPU72| FFNet-54S | ONNX | float | Qualcomm® Dragonwing™ IQ-8275 | 49.456 ms | 24 - 52 MB | NPU73| FFNet-54S | ONNX | float | Qualcomm® Dragonwing™ QCS8550 (Proxy) | 33.738 ms | 24 - 27 MB | NPU74| FFNet-54S | ONNX | float | Qualcomm® QCS8450 | 50.172 ms | 28 - 287 MB | NPU75| FFNet-54S | ONNX | float | Qualcomm® Dragonwing™ IQ-9075 | 48.504 ms | 24 - 51 MB | NPU76| FFNet-54S | ONNX | float | Qualcomm® Dragonwing™ IQ-X7181 | 32.526 ms | 24 - 24 MB | NPU77| FFNet-54S | ONNX | float | Qualcomm® Dragonwing™ Q-8750 | 18.454 ms | 7 - 192 MB | NPU78| FFNet-54S | ONNX | w8a8 | Snapdragon® 8 Elite Gen 5 For Galaxy Mobile | 3.821 ms | 2 - 197 MB | NPU79| FFNet-54S | ONNX | w8a8 | Snapdragon® 8 Elite For Galaxy Mobile | 5.176 ms | 2 - 193 MB | NPU80| FFNet-54S | ONNX | w8a8 | Snapdragon® X2 Elite | 4.258 ms | 7 - 7 MB | NPU81| FFNet-54S | ONNX | w8a8 | Snapdragon® X Elite | 11.073 ms | 13 - 13 MB | NPU82| FFNet-54S | ONNX | w8a8 | Snapdragon® 8 Gen 3 Mobile | 6.848 ms | 7 - 262 MB | NPU83| FFNet-54S | ONNX | w8a8 | Snapdragon® 8 Gen 1 Mobile | 14.744 ms | 7 - 263 MB | NPU84| FFNet-54S | ONNX | w8a8 | Qualcomm® Dragonwing™ QCS6490 | 45.681 ms | 6 - 9 MB | NPU85| FFNet-54S | ONNX | w8a8 | Qualcomm® Dragonwing™ IQ-8275 | 10.232 ms | 6 - 10 MB | NPU86| FFNet-54S | ONNX | w8a8 | Qualcomm® Dragonwing™ QCS8550 (Proxy) | 10.456 ms | 0 - 16 MB | NPU87| FFNet-54S | ONNX | w8a8 | Qualcomm® QCS8450 | 14.744 ms | 7 - 263 MB | NPU88| FFNet-54S | ONNX | w8a8 | Qualcomm® Dragonwing™ IQ-9075 | 12.273 ms | 6 - 9 MB | NPU89| FFNet-54S | ONNX | w8a8 | Qualcomm® Dragonwing™ IQ-X7181 | 11.073 ms | 13 - 13 MB | NPU90| FFNet-54S | ONNX | w8a8 | Qualcomm® Dragonwing™ Q-6690 | 133.654 ms | 5 - 209 MB | NPU91| FFNet-54S | ONNX | w8a8 | Qualcomm® Dragonwing™ Q-7790 | 15.713 ms | 7 - 216 MB | NPU92| FFNet-54S | ONNX | w8a8 | Qualcomm® Dragonwing™ Q-8750 | 5.176 ms | 2 - 193 MB | NPU93| FFNet-54S | ONNX | w8a8 | Snapdragon® 7 Gen 4 Mobile | 15.713 ms | 7 - 216 MB | NPU94| FFNet-54S | QNN_DLC | float | Snapdragon® 8 Elite Gen 5 For Galaxy Mobile | 13.615 ms | 14 - 236 MB | NPU95| FFNet-54S | QNN_DLC | float | Snapdragon® 8 Elite For Galaxy Mobile | 19.637 ms | 13 - 221 MB | NPU96| FFNet-54S | QNN_DLC | float | Snapdragon® X2 Elite | 15.926 ms | 24 - 24 MB | NPU97| FFNet-54S | QNN_DLC | float | Snapdragon® X Elite | 40.756 ms | 24 - 24 MB | NPU98| FFNet-54S | QNN_DLC | float | Snapdragon® 8 Gen 3 Mobile | 28.081 ms | 24 - 296 MB | NPU99| FFNet-54S | QNN_DLC | float | Snapdragon® 8 Gen 1 Mobile | 53.984 ms | 24 - 288 MB | NPU100| FFNet-54S | QNN_DLC | float | Qualcomm® Dragonwing™ IQ-8275 | 54.285 ms | 24 - 53 MB | NPU101| FFNet-54S | QNN_DLC | float | Qualcomm® Dragonwing™ QCS8550 (Proxy) | 38.794 ms | 24 - 26 MB | NPU102| FFNet-54S | QNN_DLC | float | Qualcomm® SA8775P | 53.812 ms | 24 - 214 MB | NPU103| FFNet-54S | QNN_DLC | float | Qualcomm® SA8650P | 53.812 ms | 24 - 214 MB | NPU104| FFNet-54S | QNN_DLC | float | Qualcomm® SA8255P | 53.812 ms | 24 - 214 MB | NPU105| FFNet-54S | QNN_DLC | float | Qualcomm® QCS8450 | 53.984 ms | 24 - 288 MB | NPU106| FFNet-54S | QNN_DLC | float | Qualcomm® Dragonwing™ IQ-9075 | 54.748 ms | 24 - 53 MB | NPU107| FFNet-54S | QNN_DLC | float | Qualcomm® Dragonwing™ IQ-X7181 | 40.756 ms | 24 - 24 MB | NPU108| FFNet-54S | QNN_DLC | float | Qualcomm® Dragonwing™ Q-8750 | 19.637 ms | 13 - 221 MB | NPU109| FFNet-54S | QNN_DLC | float | Qualcomm® SA8295P | 58.75 ms | 24 - 215 MB | NPU110| FFNet-54S | QNN_DLC | w8a8 | Snapdragon® 8 Elite Gen 5 For Galaxy Mobile | 5.69 ms | 6 - 229 MB | NPU111| FFNet-54S | QNN_DLC | w8a8 | Snapdragon® 8 Elite For Galaxy Mobile | 7.608 ms | 6 - 213 MB | NPU112| FFNet-54S | QNN_DLC | w8a8 | Snapdragon® X2 Elite | 6.573 ms | 6 - 6 MB | NPU113| FFNet-54S | QNN_DLC | w8a8 | Snapdragon® X Elite | 16.755 ms | 6 - 6 MB | NPU114| FFNet-54S | QNN_DLC | w8a8 | Snapdragon® 8 Gen 3 Mobile | 11.071 ms | 6 - 259 MB | NPU115| FFNet-54S | QNN_DLC | w8a8 | Snapdragon® 8 Gen 1 Mobile | 18.368 ms | 6 - 259 MB | NPU116| FFNet-54S | QNN_DLC | w8a8 | Qualcomm® Dragonwing™ QCS6490 | 49.129 ms | 8 - 17 MB | NPU117| FFNet-54S | QNN_DLC | w8a8 | Qualcomm® Dragonwing™ IQ-8275 | 15.714 ms | 2 - 11 MB | NPU118| FFNet-54S | QNN_DLC | w8a8 | Qualcomm® Dragonwing™ QCS8550 (Proxy) | 15.781 ms | 6 - 8 MB | NPU119| FFNet-54S | QNN_DLC | w8a8 | Qualcomm® SA8775P | 16.277 ms | 6 - 201 MB | NPU120| FFNet-54S | QNN_DLC | w8a8 | Qualcomm® SA8650P | 16.277 ms | 6 - 201 MB | NPU121| FFNet-54S | QNN_DLC | w8a8 | Qualcomm® SA8255P | 16.277 ms | 6 - 201 MB | NPU122| FFNet-54S | QNN_DLC | w8a8 | Qualcomm® QCS8450 | 18.368 ms | 6 - 259 MB | NPU123| FFNet-54S | QNN_DLC | w8a8 | Qualcomm® Dragonwing™ IQ-9075 | 16.348 ms | 6 - 15 MB | NPU124| FFNet-54S | QNN_DLC | w8a8 | Qualcomm® Dragonwing™ IQ-X7181 | 16.755 ms | 6 - 6 MB | NPU125| FFNet-54S | QNN_DLC | w8a8 | Qualcomm® Dragonwing™ Q-6690 | 152.373 ms | 6 - 204 MB | NPU126| FFNet-54S | QNN_DLC | w8a8 | Qualcomm® Dragonwing™ Q-7790 | 20.075 ms | 6 - 215 MB | NPU127| FFNet-54S | QNN_DLC | w8a8 | Qualcomm® Dragonwing™ Q-8750 | 7.608 ms | 6 - 213 MB | NPU128| FFNet-54S | QNN_DLC | w8a8 | Qualcomm® SA8295P | 21.345 ms | 6 - 205 MB | NPU129| FFNet-54S | QNN_DLC | w8a8 | Snapdragon® 7 Gen 4 Mobile | 20.075 ms | 6 - 215 MB | NPU130| FFNet-54S | TFLITE | float | Snapdragon® 8 Elite Gen 5 For Galaxy Mobile | 13.576 ms | 0 - 241 MB | NPU131| FFNet-54S | TFLITE | float | Snapdragon® 8 Elite For Galaxy Mobile | 19.577 ms | 1 - 230 MB | NPU132| FFNet-54S | TFLITE | float | Snapdragon® 8 Gen 3 Mobile | 28.281 ms | 1 - 319 MB | NPU133| FFNet-54S | TFLITE | float | Snapdragon® 8 Gen 1 Mobile | 53.907 ms | 3 - 315 MB | NPU134| FFNet-54S | TFLITE | float | Qualcomm® Dragonwing™ IQ-8275 | 54.298 ms | 0 - 65 MB | NPU135| FFNet-54S | TFLITE | float | Qualcomm® Dragonwing™ QCS8550 (Proxy) | 38.622 ms | 2 - 5 MB | NPU136| FFNet-54S | TFLITE | float | Qualcomm® SA8775P | 53.858 ms | 3 - 213 MB | NPU137| FFNet-54S | TFLITE | float | Qualcomm® SA8650P | 53.858 ms | 3 - 213 MB | NPU138| FFNet-54S | TFLITE | float | Qualcomm® SA8255P | 53.858 ms | 3 - 213 MB | NPU139| FFNet-54S | TFLITE | float | Qualcomm® QCS8450 | 53.907 ms | 3 - 315 MB | NPU140| FFNet-54S | TFLITE | float | Qualcomm® Dragonwing™ IQ-9075 | 55.24 ms | 0 - 64 MB | NPU141| FFNet-54S | TFLITE | float | Qualcomm® Dragonwing™ Q-8750 | 19.577 ms | 1 - 230 MB | NPU142| FFNet-54S | TFLITE | float | Qualcomm® SA8295P | 58.721 ms | 3 - 213 MB | NPU143| FFNet-54S | TFLITE | w8a8 | Snapdragon® 8 Elite Gen 5 For Galaxy Mobile | 3.341 ms | 1 - 222 MB | NPU144| FFNet-54S | TFLITE | w8a8 | Snapdragon® 8 Elite For Galaxy Mobile | 4.371 ms | 1 - 209 MB | NPU145| FFNet-54S | TFLITE | w8a8 | Snapdragon® 8 Gen 3 Mobile | 5.778 ms | 1 - 251 MB | NPU146| FFNet-54S | TFLITE | w8a8 | Snapdragon® 8 Gen 1 Mobile | 10.093 ms | 1 - 256 MB | NPU147| FFNet-54S | TFLITE | w8a8 | Qualcomm® Dragonwing™ QCS6490 | 34.118 ms | 1 - 28 MB | NPU148| FFNet-54S | TFLITE | w8a8 | Qualcomm® Dragonwing™ IQ-8275 | 8.789 ms | 1 - 28 MB | NPU149| FFNet-54S | TFLITE | w8a8 | Qualcomm® Dragonwing™ QCS8550 (Proxy) | 8.119 ms | 0 - 3 MB | NPU150| FFNet-54S | TFLITE | w8a8 | Qualcomm® SA8775P | 8.894 ms | 1 - 196 MB | NPU151| FFNet-54S | TFLITE | w8a8 | Qualcomm® SA8650P | 8.894 ms | 1 - 196 MB | NPU152| FFNet-54S | TFLITE | w8a8 | Qualcomm® SA8255P | 8.894 ms | 1 - 196 MB | NPU153| FFNet-54S | TFLITE | w8a8 | Qualcomm® QCS8450 | 10.093 ms | 1 - 256 MB | NPU154| FFNet-54S | TFLITE | w8a8 | Qualcomm® Dragonwing™ IQ-9075 | 8.67 ms | 1 - 28 MB | NPU155| FFNet-54S | TFLITE | w8a8 | Qualcomm® Dragonwing™ Q-6690 | 128.137 ms | 1 - 197 MB | NPU156| FFNet-54S | TFLITE | w8a8 | Qualcomm® Dragonwing™ Q-7790 | 12.511 ms | 1 - 210 MB | NPU157| FFNet-54S | TFLITE | w8a8 | Qualcomm® Dragonwing™ Q-8750 | 4.371 ms | 1 - 209 MB | NPU158| FFNet-54S | TFLITE | w8a8 | Qualcomm® SA8295P | 12.951 ms | 1 - 199 MB | NPU159| FFNet-54S | TFLITE | w8a8 | Snapdragon® 7 Gen 4 Mobile | 12.511 ms | 1 - 210 MB | NPU160 161## License162* The license for the original implementation of FFNet-54S can be found163 [here](https://github.com/Qualcomm-AI-research/FFNet/blob/master/LICENSE).164 165## References166* [Simple and Efficient Architectures for Semantic Segmentation](https://arxiv.org/abs/2206.08236)167* [Source Model Implementation](https://github.com/Qualcomm-AI-research/FFNet)168 169## Community170* Join [our AI Hub Slack community](https://aihub.qualcomm.com/community/slack) to collaborate, post questions and learn more about on-device AI.171* For questions or feedback please [reach out to us](mailto:ai-hub-support@qti.qualcomm.com).172