File size: 16,437 Bytes
183cd95 4126fd7 183cd95 4126fd7 360b7de 4126fd7 360b7de 4126fd7 360b7de 4126fd7 360b7de 4126fd7 360b7de 183cd95 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 | ---
library_name: pytorch
license: other
tags:
- real_time
- android
pipeline_tag: image-segmentation
---

# Segformer-Base: Optimized for Qualcomm Devices
Segformer Base is a machine learning model that predicts masks and classes of objects in an image.
This is based on the implementation of Segformer-Base found [here](https://github.com/huggingface/transformers/tree/main/src/transformers/models/segformer).
This repository contains pre-exported model files optimized for Qualcomm® devices. You can use the [Qualcomm® AI Hub Models](https://github.com/qualcomm/ai-hub-models/blob/v0.59.0/src/qai_hub_models/models/segformer_base) library to export with custom configurations. More details on model performance across various devices, can be found [here](#performance-summary).
Qualcomm AI Hub Models uses [Qualcomm AI Hub Workbench](https://workbench.aihub.qualcomm.com) to compile, profile, and evaluate this model. [Sign up](https://myaccount.qualcomm.com/signup) to run these models on a hosted Qualcomm® device.
## Getting Started
There are two ways to deploy this model on your device:
### Option 1: Download Pre-Exported Models
Below are pre-exported model assets ready for deployment.
| Runtime | Precision | Chipset | SDK Versions | Download |
|---|---|---|---|---|
| ONNX | float | Universal | QAIRT 2.45, ONNX Runtime 1.27.1 | [Download](https://qaihub-public-assets.s3.us-west-2.amazonaws.com/qai-hub-models/models/segformer_base/releases/v0.59.0/segformer_base-onnx-float.zip)
| ONNX | w8a16 | Universal | QAIRT 2.45, ONNX Runtime 1.27.1 | [Download](https://qaihub-public-assets.s3.us-west-2.amazonaws.com/qai-hub-models/models/segformer_base/releases/v0.59.0/segformer_base-onnx-w8a16.zip)
| ONNX | w8a8 | Universal | QAIRT 2.45, ONNX Runtime 1.27.1 | [Download](https://qaihub-public-assets.s3.us-west-2.amazonaws.com/qai-hub-models/models/segformer_base/releases/v0.59.0/segformer_base-onnx-w8a8.zip)
| QNN_DLC | float | Universal | QAIRT 2.45 | [Download](https://qaihub-public-assets.s3.us-west-2.amazonaws.com/qai-hub-models/models/segformer_base/releases/v0.59.0/segformer_base-qnn_dlc-float.zip)
| QNN_DLC | w8a16 | Universal | QAIRT 2.45 | [Download](https://qaihub-public-assets.s3.us-west-2.amazonaws.com/qai-hub-models/models/segformer_base/releases/v0.59.0/segformer_base-qnn_dlc-w8a16.zip)
| QNN_DLC | w8a8 | Universal | QAIRT 2.45 | [Download](https://qaihub-public-assets.s3.us-west-2.amazonaws.com/qai-hub-models/models/segformer_base/releases/v0.59.0/segformer_base-qnn_dlc-w8a8.zip)
| TFLITE | float | Universal | QAIRT 2.45 | [Download](https://qaihub-public-assets.s3.us-west-2.amazonaws.com/qai-hub-models/models/segformer_base/releases/v0.59.0/segformer_base-tflite-float.zip)
| TFLITE | w8a8 | Universal | QAIRT 2.45 | [Download](https://qaihub-public-assets.s3.us-west-2.amazonaws.com/qai-hub-models/models/segformer_base/releases/v0.59.0/segformer_base-tflite-w8a8.zip)
For more device-specific assets and performance metrics, visit **[Segformer-Base on Qualcomm® AI Hub](https://aihub.qualcomm.com/models/segformer_base)**.
### Option 2: Export with Custom Configurations
Use the [Qualcomm® AI Hub Models](https://github.com/qualcomm/ai-hub-models/blob/v0.59.0/src/qai_hub_models/models/segformer_base) Python library to compile and export the model with your own:
- Custom weights (e.g., fine-tuned checkpoints)
- Custom input shapes
- Target device and runtime configurations
This option is ideal if you need to customize the model beyond the default configuration provided here.
See our repository for [Segformer-Base on GitHub](https://github.com/qualcomm/ai-hub-models/blob/v0.59.0/src/qai_hub_models/models/segformer_base) for usage instructions.
## Model Details
**Model Type:** Model_use_case.semantic_segmentation
**Model Stats:**
- Model checkpoint: nvidia/segformer-b0-finetuned-ade-512-512
- Input resolution: 512x512
- Number of output classes: 150
- Number of parameters: 3.75M
- Model size (float): 14.4 MB
- Model size (w8a16): 4.57 MB
- Model size (w8a8): 3.90 MB
## Performance Summary
| Model | Runtime | Precision | Chipset | Inference Time (ms) | Peak Memory Range (MB) | Primary Compute Unit
|---|---|---|---|---|---|---
| Segformer-Base | ONNX | float | Snapdragon® X2 Elite | 13.539 ms | 35 - 35 MB | NPU
| Segformer-Base | ONNX | float | Snapdragon® X Elite | 24.478 ms | 35 - 35 MB | NPU
| Segformer-Base | ONNX | float | Snapdragon® 8 Gen 3 Mobile | 17.383 ms | 24 - 241 MB | NPU
| Segformer-Base | ONNX | float | Snapdragon® 8 Gen 1 Mobile | 24.714 ms | 23 - 244 MB | NPU
| Segformer-Base | ONNX | float | Qualcomm® Dragonwing™ QCS8550 (Proxy) | 23.675 ms | 19 - 26 MB | NPU
| Segformer-Base | ONNX | float | Qualcomm® QCS8450 | 24.714 ms | 23 - 244 MB | NPU
| Segformer-Base | ONNX | float | Qualcomm® Dragonwing™ IQ-9075 | 25.455 ms | 23 - 26 MB | NPU
| Segformer-Base | ONNX | float | Qualcomm® Dragonwing™ IQ-X7181 | 24.478 ms | 35 - 35 MB | NPU
| Segformer-Base | ONNX | float | Qualcomm® Dragonwing™ Q-8750 | 14.478 ms | 20 - 207 MB | NPU
| Segformer-Base | ONNX | float | Snapdragon® 8 Elite Mobile | 14.478 ms | 20 - 207 MB | NPU
| Segformer-Base | ONNX | float | Snapdragon® 8 Elite Gen 5 Mobile | 13.266 ms | 22 - 213 MB | NPU
| Segformer-Base | ONNX | w8a16 | Snapdragon® X2 Elite | 5.622 ms | 18 - 18 MB | NPU
| Segformer-Base | ONNX | w8a16 | Snapdragon® X Elite | 13.551 ms | 17 - 17 MB | NPU
| Segformer-Base | ONNX | w8a16 | Snapdragon® 8 Gen 3 Mobile | 9.385 ms | 13 - 246 MB | NPU
| Segformer-Base | ONNX | w8a16 | Snapdragon® 8 Gen 1 Mobile | 17.802 ms | 11 - 244 MB | NPU
| Segformer-Base | ONNX | w8a16 | Qualcomm® Dragonwing™ QCS6490 | 53.779 ms | 12 - 14 MB | NPU
| Segformer-Base | ONNX | w8a16 | Qualcomm® Dragonwing™ QCS8550 (Proxy) | 13.349 ms | 5 - 11 MB | NPU
| Segformer-Base | ONNX | w8a16 | Qualcomm® QCS8450 | 17.802 ms | 11 - 244 MB | NPU
| Segformer-Base | ONNX | w8a16 | Qualcomm® Dragonwing™ IQ-9075 | 16.262 ms | 12 - 15 MB | NPU
| Segformer-Base | ONNX | w8a16 | Qualcomm® Dragonwing™ IQ-X7181 | 13.551 ms | 17 - 17 MB | NPU
| Segformer-Base | ONNX | w8a16 | Qualcomm® Dragonwing™ Q-8750 | 7.247 ms | 13 - 216 MB | NPU
| Segformer-Base | ONNX | w8a16 | Snapdragon® 8 Elite Mobile | 7.247 ms | 13 - 216 MB | NPU
| Segformer-Base | ONNX | w8a16 | Snapdragon® 8 Elite Gen 5 Mobile | 5.674 ms | 12 - 217 MB | NPU
| Segformer-Base | ONNX | w8a8 | Snapdragon® X2 Elite | 1.945 ms | 9 - 9 MB | NPU
| Segformer-Base | ONNX | w8a8 | Snapdragon® X Elite | 5.029 ms | 5 - 5 MB | NPU
| Segformer-Base | ONNX | w8a8 | Snapdragon® 8 Gen 3 Mobile | 3.232 ms | 0 - 215 MB | NPU
| Segformer-Base | ONNX | w8a8 | Snapdragon® 8 Gen 1 Mobile | 6.985 ms | 6 - 222 MB | NPU
| Segformer-Base | ONNX | w8a8 | Qualcomm® Dragonwing™ QCS6490 | 19.621 ms | 9 - 12 MB | NPU
| Segformer-Base | ONNX | w8a8 | Qualcomm® Dragonwing™ QCS8550 (Proxy) | 4.758 ms | 5 - 11 MB | NPU
| Segformer-Base | ONNX | w8a8 | Qualcomm® QCS8450 | 6.985 ms | 6 - 222 MB | NPU
| Segformer-Base | ONNX | w8a8 | Qualcomm® Dragonwing™ IQ-9075 | 5.172 ms | 6 - 9 MB | NPU
| Segformer-Base | ONNX | w8a8 | Qualcomm® Dragonwing™ IQ-X7181 | 5.029 ms | 5 - 5 MB | NPU
| Segformer-Base | ONNX | w8a8 | Qualcomm® Dragonwing™ Q-8750 | 2.397 ms | 0 - 192 MB | NPU
| Segformer-Base | ONNX | w8a8 | Snapdragon® 8 Elite Mobile | 2.397 ms | 0 - 192 MB | NPU
| Segformer-Base | ONNX | w8a8 | Snapdragon® 8 Elite Gen 5 Mobile | 1.905 ms | 0 - 194 MB | NPU
| Segformer-Base | QNN_DLC | float | Snapdragon® X2 Elite | 12.48 ms | 3 - 3 MB | NPU
| Segformer-Base | QNN_DLC | float | Snapdragon® X Elite | 23.643 ms | 3 - 3 MB | NPU
| Segformer-Base | QNN_DLC | float | Snapdragon® 8 Gen 3 Mobile | 16.369 ms | 3 - 228 MB | NPU
| Segformer-Base | QNN_DLC | float | Snapdragon® 8 Gen 1 Mobile | 29.305 ms | 3 - 226 MB | NPU
| Segformer-Base | QNN_DLC | float | Qualcomm® Dragonwing™ QCS8275 | 48.868 ms | 0 - 189 MB | NPU
| Segformer-Base | QNN_DLC | float | Qualcomm® Dragonwing™ QCS8550 (Proxy) | 22.616 ms | 3 - 142 MB | NPU
| Segformer-Base | QNN_DLC | float | Qualcomm® SA8775P | 24.425 ms | 1 - 193 MB | NPU
| Segformer-Base | QNN_DLC | float | Qualcomm® SA8650P | 24.425 ms | 1 - 193 MB | NPU
| Segformer-Base | QNN_DLC | float | Qualcomm® SA8255P | 24.425 ms | 1 - 193 MB | NPU
| Segformer-Base | QNN_DLC | float | Qualcomm® QCS8450 | 29.305 ms | 3 - 226 MB | NPU
| Segformer-Base | QNN_DLC | float | Qualcomm® Dragonwing™ IQ-9075 | 25.204 ms | 3 - 17 MB | NPU
| Segformer-Base | QNN_DLC | float | Qualcomm® Dragonwing™ IQ-X7181 | 23.643 ms | 3 - 3 MB | NPU
| Segformer-Base | QNN_DLC | float | Qualcomm® Dragonwing™ Q-8750 | 13.368 ms | 3 - 199 MB | NPU
| Segformer-Base | QNN_DLC | float | Qualcomm® SA7255P | 48.868 ms | 0 - 189 MB | NPU
| Segformer-Base | QNN_DLC | float | Qualcomm® SA8295P | 27.858 ms | 0 - 192 MB | NPU
| Segformer-Base | QNN_DLC | float | Snapdragon® 8 Elite Mobile | 13.368 ms | 3 - 199 MB | NPU
| Segformer-Base | QNN_DLC | float | Snapdragon® 8 Elite Gen 5 Mobile | 13.672 ms | 3 - 194 MB | NPU
| Segformer-Base | QNN_DLC | w8a16 | Snapdragon® X2 Elite | 8.904 ms | 2 - 2 MB | NPU
| Segformer-Base | QNN_DLC | w8a16 | Snapdragon® X Elite | 19.623 ms | 2 - 2 MB | NPU
| Segformer-Base | QNN_DLC | w8a16 | Snapdragon® 8 Gen 3 Mobile | 15.268 ms | 0 - 240 MB | NPU
| Segformer-Base | QNN_DLC | w8a16 | Qualcomm® Dragonwing™ QCS8275 | 32.328 ms | 2 - 205 MB | NPU
| Segformer-Base | QNN_DLC | w8a16 | Qualcomm® Dragonwing™ QCS8550 (Proxy) | 20.24 ms | 2 - 3 MB | NPU
| Segformer-Base | QNN_DLC | w8a16 | Qualcomm® SA8775P | 20.198 ms | 2 - 206 MB | NPU
| Segformer-Base | QNN_DLC | w8a16 | Qualcomm® SA8650P | 20.198 ms | 2 - 206 MB | NPU
| Segformer-Base | QNN_DLC | w8a16 | Qualcomm® SA8255P | 20.198 ms | 2 - 206 MB | NPU
| Segformer-Base | QNN_DLC | w8a16 | Qualcomm® Dragonwing™ IQ-9075 | 20.519 ms | 1 - 9 MB | NPU
| Segformer-Base | QNN_DLC | w8a16 | Qualcomm® Dragonwing™ IQ-X7181 | 19.623 ms | 2 - 2 MB | NPU
| Segformer-Base | QNN_DLC | w8a16 | Qualcomm® Dragonwing™ Q-6690 | 160.832 ms | 2 - 256 MB | NPU
| Segformer-Base | QNN_DLC | w8a16 | Qualcomm® Dragonwing™ Q-7790 | 33.905 ms | 2 - 231 MB | NPU
| Segformer-Base | QNN_DLC | w8a16 | Qualcomm® Dragonwing™ Q-8750 | 11.229 ms | 2 - 208 MB | NPU
| Segformer-Base | QNN_DLC | w8a16 | Qualcomm® SA7255P | 32.328 ms | 2 - 205 MB | NPU
| Segformer-Base | QNN_DLC | w8a16 | Snapdragon® 8 Elite Mobile | 11.229 ms | 2 - 208 MB | NPU
| Segformer-Base | QNN_DLC | w8a16 | Snapdragon® 8 Elite Gen 5 Mobile | 9.787 ms | 2 - 218 MB | NPU
| Segformer-Base | QNN_DLC | w8a16 | Snapdragon® 7 Gen 4 Mobile | 33.905 ms | 2 - 231 MB | NPU
| Segformer-Base | QNN_DLC | w8a8 | Snapdragon® X2 Elite | 2.592 ms | 1 - 1 MB | NPU
| Segformer-Base | QNN_DLC | w8a8 | Snapdragon® X Elite | 6.373 ms | 1 - 1 MB | NPU
| Segformer-Base | QNN_DLC | w8a8 | Snapdragon® 8 Gen 3 Mobile | 3.946 ms | 0 - 198 MB | NPU
| Segformer-Base | QNN_DLC | w8a8 | Snapdragon® 8 Gen 1 Mobile | 8.734 ms | 0 - 200 MB | NPU
| Segformer-Base | QNN_DLC | w8a8 | Qualcomm® Dragonwing™ QCS6490 | 21.823 ms | 1 - 6 MB | NPU
| Segformer-Base | QNN_DLC | w8a8 | Qualcomm® Dragonwing™ QCS8275 | 10.861 ms | 1 - 174 MB | NPU
| Segformer-Base | QNN_DLC | w8a8 | Qualcomm® Dragonwing™ QCS8550 (Proxy) | 5.783 ms | 1 - 22 MB | NPU
| Segformer-Base | QNN_DLC | w8a8 | Qualcomm® SA8775P | 6.406 ms | 1 - 174 MB | NPU
| Segformer-Base | QNN_DLC | w8a8 | Qualcomm® SA8650P | 6.406 ms | 1 - 174 MB | NPU
| Segformer-Base | QNN_DLC | w8a8 | Qualcomm® SA8255P | 6.406 ms | 1 - 174 MB | NPU
| Segformer-Base | QNN_DLC | w8a8 | Qualcomm® QCS8450 | 8.734 ms | 0 - 200 MB | NPU
| Segformer-Base | QNN_DLC | w8a8 | Qualcomm® Dragonwing™ IQ-9075 | 6.14 ms | 1 - 5 MB | NPU
| Segformer-Base | QNN_DLC | w8a8 | Qualcomm® Dragonwing™ IQ-X7181 | 6.373 ms | 1 - 1 MB | NPU
| Segformer-Base | QNN_DLC | w8a8 | Qualcomm® Dragonwing™ Q-6690 | 36.986 ms | 1 - 189 MB | NPU
| Segformer-Base | QNN_DLC | w8a8 | Qualcomm® Dragonwing™ Q-7790 | 6.854 ms | 1 - 182 MB | NPU
| Segformer-Base | QNN_DLC | w8a8 | Qualcomm® Dragonwing™ Q-8750 | 2.84 ms | 0 - 175 MB | NPU
| Segformer-Base | QNN_DLC | w8a8 | Qualcomm® SA7255P | 10.861 ms | 1 - 174 MB | NPU
| Segformer-Base | QNN_DLC | w8a8 | Qualcomm® SA8295P | 7.786 ms | 1 - 173 MB | NPU
| Segformer-Base | QNN_DLC | w8a8 | Snapdragon® 8 Elite Mobile | 2.84 ms | 0 - 175 MB | NPU
| Segformer-Base | QNN_DLC | w8a8 | Snapdragon® 8 Elite Gen 5 Mobile | 2.248 ms | 1 - 183 MB | NPU
| Segformer-Base | QNN_DLC | w8a8 | Snapdragon® 7 Gen 4 Mobile | 6.854 ms | 1 - 182 MB | NPU
| Segformer-Base | TFLITE | float | Snapdragon® 8 Gen 3 Mobile | 16.466 ms | 8 - 236 MB | NPU
| Segformer-Base | TFLITE | float | Snapdragon® 8 Gen 1 Mobile | 29.225 ms | 10 - 234 MB | NPU
| Segformer-Base | TFLITE | float | Qualcomm® Dragonwing™ QCS8275 | 48.924 ms | 10 - 200 MB | NPU
| Segformer-Base | TFLITE | float | Qualcomm® Dragonwing™ QCS8550 (Proxy) | 22.59 ms | 9 - 126 MB | NPU
| Segformer-Base | TFLITE | float | Qualcomm® SA8775P | 24.483 ms | 10 - 202 MB | NPU
| Segformer-Base | TFLITE | float | Qualcomm® SA8650P | 24.483 ms | 10 - 202 MB | NPU
| Segformer-Base | TFLITE | float | Qualcomm® SA8255P | 24.483 ms | 10 - 202 MB | NPU
| Segformer-Base | TFLITE | float | Qualcomm® QCS8450 | 29.225 ms | 10 - 234 MB | NPU
| Segformer-Base | TFLITE | float | Qualcomm® Dragonwing™ IQ-9075 | 27.443 ms | 8 - 30 MB | NPU
| Segformer-Base | TFLITE | float | Qualcomm® Dragonwing™ Q-8750 | 13.252 ms | 9 - 204 MB | NPU
| Segformer-Base | TFLITE | float | Qualcomm® SA7255P | 48.924 ms | 10 - 200 MB | NPU
| Segformer-Base | TFLITE | float | Qualcomm® SA8295P | 27.815 ms | 10 - 202 MB | NPU
| Segformer-Base | TFLITE | float | Snapdragon® 8 Elite Mobile | 13.252 ms | 9 - 204 MB | NPU
| Segformer-Base | TFLITE | float | Snapdragon® 8 Elite Gen 5 Mobile | 13.686 ms | 8 - 200 MB | NPU
| Segformer-Base | TFLITE | w8a8 | Snapdragon® 8 Gen 3 Mobile | 7.103 ms | 1 - 209 MB | NPU
| Segformer-Base | TFLITE | w8a8 | Snapdragon® 8 Gen 1 Mobile | 12.075 ms | 2 - 211 MB | NPU
| Segformer-Base | TFLITE | w8a8 | Qualcomm® Dragonwing™ QCS6490 | 142.361 ms | 15 - 50 MB | NPU
| Segformer-Base | TFLITE | w8a8 | Qualcomm® Dragonwing™ QCS8275 | 18.262 ms | 2 - 178 MB | NPU
| Segformer-Base | TFLITE | w8a8 | Qualcomm® Dragonwing™ QCS8550 (Proxy) | 10.24 ms | 2 - 5 MB | NPU
| Segformer-Base | TFLITE | w8a8 | Qualcomm® SA8775P | 10.899 ms | 2 - 184 MB | NPU
| Segformer-Base | TFLITE | w8a8 | Qualcomm® SA8650P | 10.899 ms | 2 - 184 MB | NPU
| Segformer-Base | TFLITE | w8a8 | Qualcomm® SA8255P | 10.899 ms | 2 - 184 MB | NPU
| Segformer-Base | TFLITE | w8a8 | Qualcomm® QCS8450 | 12.075 ms | 2 - 211 MB | NPU
| Segformer-Base | TFLITE | w8a8 | Qualcomm® Dragonwing™ IQ-9075 | 14.55 ms | 2 - 12 MB | NPU
| Segformer-Base | TFLITE | w8a8 | Qualcomm® Dragonwing™ Q-6690 | 95.681 ms | 15 - 188 MB | NPU
| Segformer-Base | TFLITE | w8a8 | Qualcomm® Dragonwing™ Q-7790 | 38.879 ms | 15 - 73 MB | NPU
| Segformer-Base | TFLITE | w8a8 | Qualcomm® Dragonwing™ Q-8750 | 5.707 ms | 1 - 177 MB | NPU
| Segformer-Base | TFLITE | w8a8 | Qualcomm® SA7255P | 18.262 ms | 2 - 178 MB | NPU
| Segformer-Base | TFLITE | w8a8 | Qualcomm® SA8295P | 12.854 ms | 2 - 184 MB | NPU
| Segformer-Base | TFLITE | w8a8 | Snapdragon® 8 Elite Mobile | 5.707 ms | 1 - 177 MB | NPU
| Segformer-Base | TFLITE | w8a8 | Snapdragon® 8 Elite Gen 5 Mobile | 4.549 ms | 2 - 182 MB | NPU
| Segformer-Base | TFLITE | w8a8 | Snapdragon® 7 Gen 4 Mobile | 38.879 ms | 15 - 73 MB | NPU
## License
* The license for the original implementation of Segformer-Base can be found
[here](https://github.com/huggingface/transformers/blob/main/LICENSE).
## References
* [SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers](https://arxiv.org/abs/2105.15203)
* [Source Model Implementation](https://github.com/huggingface/transformers/tree/main/src/transformers/models/segformer)
## Community
* Join [our AI Hub Slack community](https://aihub.qualcomm.com/community/slack) to collaborate, post questions and learn more about on-device AI.
* For questions or feedback please [reach out to us](mailto:ai-hub-support@qti.qualcomm.com).
|