Hugging Face's logo

qualcomm
/
MobileSam

Image Segmentation
PyTorch
foundation
android
Model card Files
xet
 Community
qaihm-bot commited on
Commit
ea10f20
·
verified ·
1 Parent(s): 03b0df1

v0.49.1

Browse files

See https://github.com/qualcomm/ai-hub-models/releases/v0.49.1 for changelog.

Files changed (1) hide show
  1. README.md +58 -58
README.md CHANGED
@@ -15,7 +15,7 @@ pipeline_tag: image-segmentation
15
  Transformer based encoder-decoder where prompts specify what to segment in an image thereby allowing segmentation without the need for additional training. The image encoder generates embeddings and the lightweight decoder operates on the embeddings for point and mask based image segmentation.
16
 
17
  This is based on the implementation of MobileSam found [here](https://github.com/facebookresearch/segment-anything).
18
- This repository contains pre-exported model files optimized for Qualcomm® devices. You can use the [Qualcomm® AI Hub Models](https://github.com/qualcomm/ai-hub-models/blob/main/qai_hub_models/models/mobilesam) library to export with custom configurations. More details on model performance across various devices, can be found [here](#performance-summary).
19
 
20
  Qualcomm AI Hub Models uses [Qualcomm AI Hub Workbench](https://workbench.aihub.qualcomm.com) to compile, profile, and evaluate this model. [Sign up](https://myaccount.qualcomm.com/signup) to run these models on a hosted Qualcomm® device.
21
 
@@ -28,23 +28,23 @@ Below are pre-exported model assets ready for deployment.
28
 
29
  | Runtime | Precision | Chipset | SDK Versions | Download |
30
  |---|---|---|---|---|
31
- | ONNX | float | Universal | QAIRT 2.42, ONNX Runtime 1.24.1 | [Download](https://qaihub-public-assets.s3.us-west-2.amazonaws.com/qai-hub-models/models/mobilesam/releases/v0.48.0/mobilesam-onnx-float.zip)
32
- | QNN_DLC | float | Universal | QAIRT 2.43 | [Download](https://qaihub-public-assets.s3.us-west-2.amazonaws.com/qai-hub-models/models/mobilesam/releases/v0.48.0/mobilesam-qnn_dlc-float.zip)
33
- | TFLITE | float | Universal | QAIRT 2.43, TFLite 2.17.0 | [Download](https://qaihub-public-assets.s3.us-west-2.amazonaws.com/qai-hub-models/models/mobilesam/releases/v0.48.0/mobilesam-tflite-float.zip)
34
 
35
  For more device-specific assets and performance metrics, visit **[MobileSam on Qualcomm® AI Hub](https://aihub.qualcomm.com/models/mobilesam)**.
36
 
37
 
38
  ### Option 2: Export with Custom Configurations
39
 
40
- Use the [Qualcomm® AI Hub Models](https://github.com/qualcomm/ai-hub-models/blob/main/qai_hub_models/models/mobilesam) Python library to compile and export the model with your own:
41
  - Custom weights (e.g., fine-tuned checkpoints)
42
  - Custom input shapes
43
  - Target device and runtime configurations
44
 
45
  This option is ideal if you need to customize the model beyond the default configuration provided here.
46
 
47
- See our repository for [MobileSam on GitHub](https://github.com/qualcomm/ai-hub-models/blob/main/qai_hub_models/models/mobilesam) for usage instructions.
48
 
49
  ## Model Details
50
 
@@ -61,58 +61,58 @@ See our repository for [MobileSam on GitHub](https://github.com/qualcomm/ai-hub-
61
  ## Performance Summary
62
  | Model | Runtime | Precision | Chipset | Inference Time (ms) | Peak Memory Range (MB) | Primary Compute Unit
63
  |---|---|---|---|---|---|---
64
- | MobileSAMDecoder | ONNX | float | Snapdragon® X2 Elite | 2.953 ms | 11 - 11 MB | NPU
65
- | MobileSAMDecoder | ONNX | float | Snapdragon® X Elite | 7.521 ms | 11 - 11 MB | NPU
66
- | MobileSAMDecoder | ONNX | float | Snapdragon® 8 Gen 3 Mobile | 4.892 ms | 4 - 270 MB | NPU
67
- | MobileSAMDecoder | ONNX | float | Qualcomm® QCS8550 (Proxy) | 6.988 ms | 0 - 15 MB | NPU
68
- | MobileSAMDecoder | ONNX | float | Snapdragon® 8 Elite For Galaxy Mobile | 3.641 ms | 1 - 223 MB | NPU
69
- | MobileSAMDecoder | ONNX | float | Snapdragon® 8 Elite Gen 5 Mobile | 2.886 ms | 0 - 232 MB | NPU
70
- | MobileSAMDecoder | QNN_DLC | float | Snapdragon® X2 Elite | 2.609 ms | 4 - 4 MB | NPU
71
- | MobileSAMDecoder | QNN_DLC | float | Snapdragon® X Elite | 5.309 ms | 4 - 4 MB | NPU
72
- | MobileSAMDecoder | QNN_DLC | float | Snapdragon® 8 Gen 3 Mobile | 3.364 ms | 4 - 230 MB | NPU
73
- | MobileSAMDecoder | QNN_DLC | float | Qualcomm® QCS8275 (Proxy) | 11.701 ms | 0 - 199 MB | NPU
74
- | MobileSAMDecoder | QNN_DLC | float | Qualcomm® QCS8550 (Proxy) | 4.844 ms | 4 - 5 MB | NPU
75
- | MobileSAMDecoder | QNN_DLC | float | Qualcomm® SA8775P | 5.875 ms | 0 - 200 MB | NPU
76
- | MobileSAMDecoder | QNN_DLC | float | Qualcomm® QCS8450 (Proxy) | 8.445 ms | 0 - 220 MB | NPU
77
- | MobileSAMDecoder | QNN_DLC | float | Qualcomm® SA7255P | 11.701 ms | 0 - 199 MB | NPU
78
- | MobileSAMDecoder | QNN_DLC | float | Qualcomm® SA8295P | 7.169 ms | 0 - 193 MB | NPU
79
- | MobileSAMDecoder | QNN_DLC | float | Snapdragon® 8 Elite For Galaxy Mobile | 2.593 ms | 0 - 200 MB | NPU
80
- | MobileSAMDecoder | QNN_DLC | float | Snapdragon® 8 Elite Gen 5 Mobile | 2.151 ms | 2 - 201 MB | NPU
81
- | MobileSAMDecoder | TFLITE | float | Snapdragon® 8 Gen 3 Mobile | 3.877 ms | 0 - 235 MB | NPU
82
- | MobileSAMDecoder | TFLITE | float | Qualcomm® QCS8275 (Proxy) | 13.033 ms | 0 - 202 MB | NPU
83
- | MobileSAMDecoder | TFLITE | float | Qualcomm® QCS8550 (Proxy) | 5.605 ms | 0 - 18 MB | NPU
84
- | MobileSAMDecoder | TFLITE | float | Qualcomm® SA8775P | 6.698 ms | 0 - 203 MB | NPU
85
- | MobileSAMDecoder | TFLITE | float | Qualcomm® QCS8450 (Proxy) | 9.541 ms | 0 - 225 MB | NPU
86
- | MobileSAMDecoder | TFLITE | float | Qualcomm® SA7255P | 13.033 ms | 0 - 202 MB | NPU
87
- | MobileSAMDecoder | TFLITE | float | Qualcomm® SA8295P | 8.047 ms | 0 - 201 MB | NPU
88
- | MobileSAMDecoder | TFLITE | float | Snapdragon® 8 Elite For Galaxy Mobile | 2.958 ms | 0 - 207 MB | NPU
89
- | MobileSAMDecoder | TFLITE | float | Snapdragon® 8 Elite Gen 5 Mobile | 2.352 ms | 0 - 207 MB | NPU
90
- | MobileSAMEncoder | ONNX | float | Snapdragon® X2 Elite | 94.358 ms | 14 - 14 MB | NPU
91
- | MobileSAMEncoder | ONNX | float | Snapdragon® X Elite | 173.085 ms | 12 - 12 MB | NPU
92
- | MobileSAMEncoder | ONNX | float | Snapdragon® 8 Gen 3 Mobile | 125.467 ms | 20 - 3025 MB | NPU
93
- | MobileSAMEncoder | ONNX | float | Qualcomm® QCS8550 (Proxy) | 175.267 ms | 12 - 32 MB | NPU
94
- | MobileSAMEncoder | ONNX | float | Snapdragon® 8 Elite For Galaxy Mobile | 100.697 ms | 9 - 1389 MB | NPU
95
- | MobileSAMEncoder | ONNX | float | Snapdragon® 8 Elite Gen 5 Mobile | 88.418 ms | 20 - 1523 MB | NPU
96
- | MobileSAMEncoder | QNN_DLC | float | Snapdragon® X2 Elite | 51.746 ms | 12 - 12 MB | NPU
97
- | MobileSAMEncoder | QNN_DLC | float | Snapdragon® X Elite | 100.135 ms | 12 - 12 MB | NPU
98
- | MobileSAMEncoder | QNN_DLC | float | Snapdragon® 8 Gen 3 Mobile | 66.42 ms | 12 - 1635 MB | NPU
99
- | MobileSAMEncoder | QNN_DLC | float | Qualcomm® QCS8275 (Proxy) | 209.122 ms | 0 - 1209 MB | NPU
100
- | MobileSAMEncoder | QNN_DLC | float | Qualcomm® QCS8550 (Proxy) | 96.617 ms | 12 - 16 MB | NPU
101
- | MobileSAMEncoder | QNN_DLC | float | Qualcomm® SA8775P | 103.074 ms | 0 - 1209 MB | NPU
102
- | MobileSAMEncoder | QNN_DLC | float | Qualcomm® QCS8450 (Proxy) | 470.205 ms | 12 - 1731 MB | NPU
103
- | MobileSAMEncoder | QNN_DLC | float | Qualcomm® SA7255P | 209.122 ms | 0 - 1209 MB | NPU
104
- | MobileSAMEncoder | QNN_DLC | float | Qualcomm® SA8295P | 474.048 ms | 0 - 1249 MB | NPU
105
- | MobileSAMEncoder | QNN_DLC | float | Snapdragon® 8 Elite For Galaxy Mobile | 49.865 ms | 12 - 1217 MB | NPU
106
- | MobileSAMEncoder | QNN_DLC | float | Snapdragon® 8 Elite Gen 5 Mobile | 48.392 ms | 8 - 1319 MB | NPU
107
- | MobileSAMEncoder | TFLITE | float | Snapdragon® 8 Gen 3 Mobile | 126.457 ms | 0 - 1496 MB | NPU
108
- | MobileSAMEncoder | TFLITE | float | Qualcomm® QCS8275 (Proxy) | 295.258 ms | 4 - 1269 MB | NPU
109
- | MobileSAMEncoder | TFLITE | float | Qualcomm® QCS8550 (Proxy) | 176.603 ms | 0 - 3 MB | NPU
110
- | MobileSAMEncoder | TFLITE | float | Qualcomm® SA8775P | 179.751 ms | 0 - 1265 MB | NPU
111
- | MobileSAMEncoder | TFLITE | float | Qualcomm® QCS8450 (Proxy) | 590.208 ms | 0 - 3175 MB | NPU
112
- | MobileSAMEncoder | TFLITE | float | Qualcomm® SA7255P | 295.258 ms | 4 - 1269 MB | NPU
113
- | MobileSAMEncoder | TFLITE | float | Qualcomm® SA8295P | 502.789 ms | 0 - 1888 MB | NPU
114
- | MobileSAMEncoder | TFLITE | float | Snapdragon® 8 Elite For Galaxy Mobile | 97.396 ms | 4 - 1267 MB | NPU
115
- | MobileSAMEncoder | TFLITE | float | Snapdragon® 8 Elite Gen 5 Mobile | 78.068 ms | 4 - 1348 MB | NPU
116
 
117
  ## License
118
  * The license for the original implementation of MobileSam can be found
 
15
  Transformer based encoder-decoder where prompts specify what to segment in an image thereby allowing segmentation without the need for additional training. The image encoder generates embeddings and the lightweight decoder operates on the embeddings for point and mask based image segmentation.
16
 
17
  This is based on the implementation of MobileSam found [here](https://github.com/facebookresearch/segment-anything).
18
+ This repository contains pre-exported model files optimized for Qualcomm® devices. You can use the [Qualcomm® AI Hub Models](https://github.com/qualcomm/ai-hub-models/tree/v0.49.1/qai_hub_models/models/mobilesam) library to export with custom configurations. More details on model performance across various devices, can be found [here](#performance-summary).
19
 
20
  Qualcomm AI Hub Models uses [Qualcomm AI Hub Workbench](https://workbench.aihub.qualcomm.com) to compile, profile, and evaluate this model. [Sign up](https://myaccount.qualcomm.com/signup) to run these models on a hosted Qualcomm® device.
21
 
 
28
 
29
  | Runtime | Precision | Chipset | SDK Versions | Download |
30
  |---|---|---|---|---|
31
+ | ONNX | float | Universal | QAIRT 2.42, ONNX Runtime 1.24.1 | [Download](https://qaihub-public-assets.s3.us-west-2.amazonaws.com/qai-hub-models/models/mobilesam/releases/v0.49.1/mobilesam-onnx-float.zip)
32
+ | QNN_DLC | float | Universal | QAIRT 2.43 | [Download](https://qaihub-public-assets.s3.us-west-2.amazonaws.com/qai-hub-models/models/mobilesam/releases/v0.49.1/mobilesam-qnn_dlc-float.zip)
33
+ | TFLITE | float | Universal | QAIRT 2.43, TFLite 2.17.0 | [Download](https://qaihub-public-assets.s3.us-west-2.amazonaws.com/qai-hub-models/models/mobilesam/releases/v0.49.1/mobilesam-tflite-float.zip)
34
 
35
  For more device-specific assets and performance metrics, visit **[MobileSam on Qualcomm® AI Hub](https://aihub.qualcomm.com/models/mobilesam)**.
36
 
37
 
38
  ### Option 2: Export with Custom Configurations
39
 
40
+ Use the [Qualcomm® AI Hub Models](https://github.com/qualcomm/ai-hub-models/tree/v0.49.1/qai_hub_models/models/mobilesam) Python library to compile and export the model with your own:
41
  - Custom weights (e.g., fine-tuned checkpoints)
42
  - Custom input shapes
43
  - Target device and runtime configurations
44
 
45
  This option is ideal if you need to customize the model beyond the default configuration provided here.
46
 
47
+ See our repository for [MobileSam on GitHub](https://github.com/qualcomm/ai-hub-models/tree/v0.49.1/qai_hub_models/models/mobilesam) for usage instructions.
48
 
49
  ## Model Details
50
 
 
61
  ## Performance Summary
62
  | Model | Runtime | Precision | Chipset | Inference Time (ms) | Peak Memory Range (MB) | Primary Compute Unit
63
  |---|---|---|---|---|---|---
64
+ | MobileSAMDecoder | ONNX | float | Snapdragon® 8 Elite Gen 5 Mobile | 2.955 ms | 0 - 243 MB | NPU
65
+ | MobileSAMDecoder | ONNX | float | Snapdragon® X2 Elite | 3.03 ms | 11 - 11 MB | NPU
66
+ | MobileSAMDecoder | ONNX | float | Snapdragon® X Elite | 7.603 ms | 11 - 11 MB | NPU
67
+ | MobileSAMDecoder | ONNX | float | Snapdragon® 8 Gen 3 Mobile | 4.999 ms | 4 - 349 MB | NPU
68
+ | MobileSAMDecoder | ONNX | float | Qualcomm® QCS8550 (Proxy) | 7.104 ms | 0 - 16 MB | NPU
69
+ | MobileSAMDecoder | ONNX | float | Snapdragon® 8 Elite For Galaxy Mobile | 3.658 ms | 3 - 227 MB | NPU
70
+ | MobileSAMDecoder | QNN_DLC | float | Snapdragon® 8 Elite Gen 5 Mobile | 2.173 ms | 4 - 203 MB | NPU
71
+ | MobileSAMDecoder | QNN_DLC | float | Snapdragon® X2 Elite | 2.703 ms | 4 - 4 MB | NPU
72
+ | MobileSAMDecoder | QNN_DLC | float | Snapdragon® X Elite | 5.245 ms | 4 - 4 MB | NPU
73
+ | MobileSAMDecoder | QNN_DLC | float | Snapdragon® 8 Gen 3 Mobile | 3.341 ms | 4 - 231 MB | NPU
74
+ | MobileSAMDecoder | QNN_DLC | float | Qualcomm® QCS8275 (Proxy) | 11.557 ms | 1 - 199 MB | NPU
75
+ | MobileSAMDecoder | QNN_DLC | float | Qualcomm® QCS8550 (Proxy) | 4.829 ms | 4 - 50 MB | NPU
76
+ | MobileSAMDecoder | QNN_DLC | float | Qualcomm® SA8775P | 5.772 ms | 1 - 200 MB | NPU
77
+ | MobileSAMDecoder | QNN_DLC | float | Qualcomm® QCS8450 (Proxy) | 8.456 ms | 0 - 219 MB | NPU
78
+ | MobileSAMDecoder | QNN_DLC | float | Qualcomm® SA7255P | 11.557 ms | 1 - 199 MB | NPU
79
+ | MobileSAMDecoder | QNN_DLC | float | Qualcomm® SA8295P | 7.231 ms | 0 - 193 MB | NPU
80
+ | MobileSAMDecoder | QNN_DLC | float | Snapdragon® 8 Elite For Galaxy Mobile | 2.595 ms | 0 - 196 MB | NPU
81
+ | MobileSAMDecoder | TFLITE | float | Snapdragon® 8 Elite Gen 5 Mobile | 2.399 ms | 0 - 205 MB | NPU
82
+ | MobileSAMDecoder | TFLITE | float | Snapdragon® 8 Gen 3 Mobile | 3.906 ms | 0 - 228 MB | NPU
83
+ | MobileSAMDecoder | TFLITE | float | Qualcomm® QCS8275 (Proxy) | 13.038 ms | 0 - 201 MB | NPU
84
+ | MobileSAMDecoder | TFLITE | float | Qualcomm® QCS8550 (Proxy) | 5.653 ms | 0 - 157 MB | NPU
85
+ | MobileSAMDecoder | TFLITE | float | Qualcomm® SA8775P | 6.703 ms | 0 - 204 MB | NPU
86
+ | MobileSAMDecoder | TFLITE | float | Qualcomm® QCS8450 (Proxy) | 9.516 ms | 0 - 226 MB | NPU
87
+ | MobileSAMDecoder | TFLITE | float | Qualcomm® SA7255P | 13.038 ms | 0 - 201 MB | NPU
88
+ | MobileSAMDecoder | TFLITE | float | Qualcomm® SA8295P | 8.098 ms | 0 - 200 MB | NPU
89
+ | MobileSAMDecoder | TFLITE | float | Snapdragon® 8 Elite For Galaxy Mobile | 2.942 ms | 0 - 207 MB | NPU
90
+ | MobileSAMEncoder | ONNX | float | Snapdragon® 8 Elite Gen 5 Mobile | 88.265 ms | 20 - 1524 MB | NPU
91
+ | MobileSAMEncoder | ONNX | float | Snapdragon® X2 Elite | 94.928 ms | 14 - 14 MB | NPU
92
+ | MobileSAMEncoder | ONNX | float | Snapdragon® X Elite | 172.827 ms | 12 - 12 MB | NPU
93
+ | MobileSAMEncoder | ONNX | float | Snapdragon® 8 Gen 3 Mobile | 126.751 ms | 20 - 3043 MB | NPU
94
+ | MobileSAMEncoder | ONNX | float | Qualcomm® QCS8550 (Proxy) | 177.497 ms | 12 - 29 MB | NPU
95
+ | MobileSAMEncoder | ONNX | float | Snapdragon® 8 Elite For Galaxy Mobile | 102.158 ms | 8 - 1389 MB | NPU
96
+ | MobileSAMEncoder | QNN_DLC | float | Snapdragon® 8 Elite Gen 5 Mobile | 48.494 ms | 12 - 1323 MB | NPU
97
+ | MobileSAMEncoder | QNN_DLC | float | Snapdragon® X2 Elite | 51.581 ms | 12 - 12 MB | NPU
98
+ | MobileSAMEncoder | QNN_DLC | float | Snapdragon® X Elite | 100.238 ms | 12 - 12 MB | NPU
99
+ | MobileSAMEncoder | QNN_DLC | float | Snapdragon® 8 Gen 3 Mobile | 65.826 ms | 12 - 1630 MB | NPU
100
+ | MobileSAMEncoder | QNN_DLC | float | Qualcomm® QCS8275 (Proxy) | 209.3 ms | 1 - 1209 MB | NPU
101
+ | MobileSAMEncoder | QNN_DLC | float | Qualcomm® QCS8550 (Proxy) | 96.945 ms | 12 - 843 MB | NPU
102
+ | MobileSAMEncoder | QNN_DLC | float | Qualcomm® SA8775P | 103.023 ms | 1 - 1210 MB | NPU
103
+ | MobileSAMEncoder | QNN_DLC | float | Qualcomm® QCS8450 (Proxy) | 470.58 ms | 13 - 1741 MB | NPU
104
+ | MobileSAMEncoder | QNN_DLC | float | Qualcomm® SA7255P | 209.3 ms | 1 - 1209 MB | NPU
105
+ | MobileSAMEncoder | QNN_DLC | float | Qualcomm® SA8295P | 474.385 ms | 0 - 1257 MB | NPU
106
+ | MobileSAMEncoder | QNN_DLC | float | Snapdragon® 8 Elite For Galaxy Mobile | 49.964 ms | 11 - 1214 MB | NPU
107
+ | MobileSAMEncoder | TFLITE | float | Snapdragon® 8 Elite Gen 5 Mobile | 83.179 ms | 4 - 1349 MB | NPU
108
+ | MobileSAMEncoder | TFLITE | float | Snapdragon® 8 Gen 3 Mobile | 126.645 ms | 4 - 1499 MB | NPU
109
+ | MobileSAMEncoder | TFLITE | float | Qualcomm® QCS8275 (Proxy) | 295.692 ms | 4 - 1269 MB | NPU
110
+ | MobileSAMEncoder | TFLITE | float | Qualcomm® QCS8550 (Proxy) | 173.681 ms | 4 - 8 MB | NPU
111
+ | MobileSAMEncoder | TFLITE | float | Qualcomm® SA8775P | 179.644 ms | 0 - 1264 MB | NPU
112
+ | MobileSAMEncoder | TFLITE | float | Qualcomm® QCS8450 (Proxy) | 585.325 ms | 0 - 3174 MB | NPU
113
+ | MobileSAMEncoder | TFLITE | float | Qualcomm® SA7255P | 295.692 ms | 4 - 1269 MB | NPU
114
+ | MobileSAMEncoder | TFLITE | float | Qualcomm® SA8295P | 502.376 ms | 4 - 1893 MB | NPU
115
+ | MobileSAMEncoder | TFLITE | float | Snapdragon® 8 Elite For Galaxy Mobile | 96.399 ms | 3 - 1265 MB | NPU
116
 
117
  ## License
118
  * The license for the original implementation of MobileSam can be found