Hugging Face's logo

qualcomm
/
Segment-Anything-Model-2

Image Segmentation
PyTorch
foundation
android
Model card Files
xet
 Community
Segment-Anything-Model-2 / README.md
qaihm-bot's picture
qaihm-bot
v0.60.0
caf0c1a verified
preview code
|
Raw
History Blame Contribute Delete
16.3 kB
metadata 
library_name: pytorch
license: other
tags:
  - foundation
  - android
pipeline_tag: image-segmentation

Segment-Anything-Model-2: Optimized for Qualcomm Devices

SAM 2, the successor to Meta's Segment Anything Model (SAM), is a cutting-edge tool designed for comprehensive object segmentation in both images and videos. It excels in handling complex visual data through a unified, promptable model architecture that supports real-time processing and zero-shot generalization.

This is based on the implementation of Segment-Anything-Model-2 found here. This repository contains pre-exported model files optimized for Qualcomm® devices. You can use the Qualcomm® AI Hub Models library to export with custom configurations. More details on model performance across various devices, can be found here.

Qualcomm AI Hub Models uses Qualcomm AI Hub Workbench to compile, profile, and evaluate this model. Sign up to run these models on a hosted Qualcomm® device.

Getting Started

There are two ways to deploy this model on your device:

Option 1: Download Pre-Exported Models

Below are pre-exported model assets ready for deployment.

Runtime Precision Chipset SDK Versions Download
ONNX float Universal QAIRT 2.45, ONNX Runtime 1.27.1 Download
ONNX w8a8 Universal QAIRT 2.45, ONNX Runtime 1.27.1 Download
QNN_DLC w8a8 Universal QAIRT 2.45 Download
TFLITE w8a8 Universal QAIRT 2.45 Download

For more device-specific assets and performance metrics, visit Segment-Anything-Model-2 on Qualcomm® AI Hub.

Option 2: Export with Custom Configurations

Use the Qualcomm® AI Hub Models Python library to compile and export the model with your own:

  • Custom weights (e.g., fine-tuned checkpoints)
  • Custom input shapes
  • Target device and runtime configurations

This option is ideal if you need to customize the model beyond the default configuration provided here.

See our repository for Segment-Anything-Model-2 on GitHub for usage instructions.

Model Details

Model Type: Model_use_case.semantic_segmentation

Model Stats:

  • Input resolution: 720p (720x1280)
  • Model checkpoint: sam2.1_hiera_t
  • Model size (decoder) (float): 23.7 MB
  • Model size (encoder) (float): 128 MB
  • Number of parameters (decoder): 6.22M
  • Number of parameters (encoder): 33.5M

Performance Summary

Model Runtime Precision Chipset Inference Time (ms) Peak Memory Range (MB) Primary Compute Unit
decoder ONNX float Snapdragon® X2 Elite 3.298 ms 16 - 16 MB NPU
decoder ONNX float Qualcomm® Dragonwing™ Q-8750 3.765 ms 3 - 224 MB NPU
decoder ONNX float Snapdragon® 8 Elite Mobile 3.765 ms 3 - 224 MB NPU
decoder ONNX float Snapdragon® 8 Elite Gen 5 Mobile 3.179 ms 3 - 227 MB NPU
decoder ONNX w8a8 Snapdragon® X2 Elite 1.336 ms 4 - 4 MB NPU
decoder ONNX w8a8 Snapdragon® X Elite 2.791 ms 4 - 4 MB NPU
decoder ONNX w8a8 Snapdragon® 8 Gen 3 Mobile 1.776 ms 4 - 151 MB NPU
decoder ONNX w8a8 Snapdragon® 8 Gen 1 Mobile 3.49 ms 4 - 161 MB NPU
decoder ONNX w8a8 Qualcomm® Dragonwing™ QCS6490 12.779 ms 4 - 7 MB NPU
decoder ONNX w8a8 Qualcomm® Dragonwing™ IQ-8275 2.856 ms 4 - 8 MB NPU
decoder ONNX w8a8 Qualcomm® Dragonwing™ QCS8550 (Proxy) 2.653 ms 2 - 81 MB NPU
decoder ONNX w8a8 Qualcomm® QCS8450 3.49 ms 4 - 161 MB NPU
decoder ONNX w8a8 Qualcomm® Dragonwing™ IQ-9075 3.172 ms 4 - 7 MB NPU
decoder ONNX w8a8 Qualcomm® Dragonwing™ IQ-X7181 2.791 ms 4 - 4 MB NPU
decoder ONNX w8a8 Qualcomm® Dragonwing™ Q-6690 17.309 ms 2 - 249 MB NPU
decoder ONNX w8a8 Qualcomm® Dragonwing™ Q-7790 3.397 ms 4 - 305 MB NPU
decoder ONNX w8a8 Qualcomm® Dragonwing™ Q-8750 1.438 ms 0 - 129 MB NPU
decoder ONNX w8a8 Snapdragon® 8 Elite Mobile 1.438 ms 0 - 129 MB NPU
decoder ONNX w8a8 Snapdragon® 8 Elite Gen 5 Mobile 1.248 ms 0 - 151 MB NPU
decoder ONNX w8a8 Snapdragon® 7 Gen 4 Mobile 3.397 ms 4 - 305 MB NPU
decoder QNN_DLC float Snapdragon® X2 Elite 3.074 ms 16 - 16 MB NPU
decoder QNN_DLC float Snapdragon® 8 Gen 1 Mobile 10.052 ms 4 - 217 MB NPU
decoder QNN_DLC float Qualcomm® QCS8450 10.052 ms 4 - 217 MB NPU
decoder QNN_DLC float Qualcomm® SA8295P 8.221 ms 2 - 198 MB NPU
decoder QNN_DLC float Snapdragon® 8 Elite Gen 5 Mobile 2.594 ms 4 - 192 MB NPU
decoder QNN_DLC w8a8 Snapdragon® X2 Elite 1.378 ms 4 - 4 MB NPU
decoder QNN_DLC w8a8 Snapdragon® X Elite 2.319 ms 4 - 4 MB NPU
decoder QNN_DLC w8a8 Snapdragon® 8 Gen 3 Mobile 1.286 ms 0 - 109 MB NPU
decoder QNN_DLC w8a8 Snapdragon® 8 Gen 1 Mobile 2.558 ms 0 - 108 MB NPU
decoder QNN_DLC w8a8 Qualcomm® Dragonwing™ QCS6490 9.994 ms 1 - 7 MB NPU
decoder QNN_DLC w8a8 Qualcomm® Dragonwing™ IQ-8275 2.145 ms 1 - 7 MB NPU
decoder QNN_DLC w8a8 Qualcomm® Dragonwing™ IQ-8275 3.899 ms 4 - 89 MB NPU
decoder QNN_DLC w8a8 Qualcomm® Dragonwing™ QCS8550 (Proxy) 1.98 ms 4 - 6 MB NPU
decoder QNN_DLC w8a8 Qualcomm® SA8775P 2.59 ms 4 - 91 MB NPU
decoder QNN_DLC w8a8 Qualcomm® SA8650P 2.59 ms 4 - 91 MB NPU
decoder QNN_DLC w8a8 Qualcomm® SA8255P 2.59 ms 4 - 91 MB NPU
decoder QNN_DLC w8a8 Qualcomm® QCS8450 2.558 ms 0 - 108 MB NPU
decoder QNN_DLC w8a8 Qualcomm® Dragonwing™ IQ-9075 2.397 ms 3 - 9 MB NPU
decoder QNN_DLC w8a8 Qualcomm® Dragonwing™ IQ-X7181 2.319 ms 4 - 4 MB NPU
decoder QNN_DLC w8a8 Qualcomm® Dragonwing™ Q-6690 9.996 ms 4 - 274 MB NPU
decoder QNN_DLC w8a8 Qualcomm® Dragonwing™ Q-7790 2.672 ms 4 - 214 MB NPU
decoder QNN_DLC w8a8 Qualcomm® Dragonwing™ Q-8750 1.027 ms 1 - 88 MB NPU
decoder QNN_DLC w8a8 Qualcomm® SA7255P 3.899 ms 4 - 89 MB NPU
decoder QNN_DLC w8a8 Qualcomm® SA8295P 2.98 ms 4 - 87 MB NPU
decoder QNN_DLC w8a8 Snapdragon® 8 Elite Mobile 1.027 ms 1 - 88 MB NPU
decoder QNN_DLC w8a8 Snapdragon® 8 Elite Gen 5 Mobile 0.919 ms 1 - 110 MB NPU
decoder QNN_DLC w8a8 Snapdragon® 7 Gen 4 Mobile 2.672 ms 4 - 214 MB NPU
decoder TFLITE float Snapdragon® 8 Gen 1 Mobile 10.069 ms 0 - 220 MB NPU
decoder TFLITE float Qualcomm® QCS8450 10.069 ms 0 - 220 MB NPU
decoder TFLITE float Qualcomm® SA8295P 8.169 ms 0 - 196 MB NPU
decoder TFLITE float Snapdragon® 8 Elite Gen 5 Mobile 2.59 ms 0 - 183 MB NPU
decoder TFLITE w8a8 Snapdragon® 8 Gen 3 Mobile 5.8 ms 0 - 267 MB NPU
decoder TFLITE w8a8 Snapdragon® 8 Gen 1 Mobile 10.154 ms 0 - 247 MB NPU
decoder TFLITE w8a8 Qualcomm® Dragonwing™ QCS6490 39.238 ms 12 - 58 MB NPU
decoder TFLITE w8a8 Qualcomm® Dragonwing™ IQ-8275 7.803 ms 0 - 15 MB NPU
decoder TFLITE w8a8 Qualcomm® Dragonwing™ IQ-8275 13.636 ms 0 - 237 MB NPU
decoder TFLITE w8a8 Qualcomm® Dragonwing™ QCS8550 (Proxy) 7.749 ms 0 - 22 MB NPU
decoder TFLITE w8a8 Qualcomm® SA8775P 8.558 ms 0 - 240 MB NPU
decoder TFLITE w8a8 Qualcomm® SA8650P 8.558 ms 0 - 240 MB NPU
decoder TFLITE w8a8 Qualcomm® SA8255P 8.558 ms 0 - 240 MB NPU
decoder TFLITE w8a8 Qualcomm® QCS8450 10.154 ms 0 - 247 MB NPU
decoder TFLITE w8a8 Qualcomm® Dragonwing™ IQ-9075 8.941 ms 0 - 14 MB NPU
decoder TFLITE w8a8 Qualcomm® Dragonwing™ Q-6690 34.07 ms 12 - 210 MB NPU
decoder TFLITE w8a8 Qualcomm® Dragonwing™ Q-7790 14.217 ms 14 - 204 MB NPU
decoder TFLITE w8a8 Qualcomm® Dragonwing™ Q-8750 3.52 ms 0 - 239 MB NPU
decoder TFLITE w8a8 Qualcomm® SA7255P 13.636 ms 0 - 237 MB NPU
decoder TFLITE w8a8 Qualcomm® SA8295P 9.44 ms 0 - 239 MB NPU
decoder TFLITE w8a8 Snapdragon® 8 Elite Mobile 3.52 ms 0 - 239 MB NPU
decoder TFLITE w8a8 Snapdragon® 8 Elite Gen 5 Mobile 3.118 ms 0 - 254 MB NPU
decoder TFLITE w8a8 Snapdragon® 7 Gen 4 Mobile 14.217 ms 14 - 204 MB NPU
encoder ONNX float Snapdragon® X2 Elite 86.057 ms 70 - 70 MB NPU
encoder ONNX float Qualcomm® Dragonwing™ Q-8750 98.055 ms 46 - 1802 MB NPU
encoder ONNX float Snapdragon® 8 Elite Mobile 98.055 ms 46 - 1802 MB NPU
encoder ONNX float Snapdragon® 8 Elite Gen 5 Mobile 79.043 ms 44 - 1717 MB NPU
encoder ONNX w8a8 Snapdragon® X2 Elite 35.564 ms 18 - 18 MB NPU
encoder ONNX w8a8 Snapdragon® X Elite 90.525 ms 51 - 51 MB NPU
encoder ONNX w8a8 Snapdragon® 8 Gen 3 Mobile 62.535 ms 16 - 1943 MB NPU
encoder ONNX w8a8 Snapdragon® 8 Gen 1 Mobile 125.173 ms 12 - 2401 MB NPU
encoder ONNX w8a8 Qualcomm® Dragonwing™ QCS6490 676.142 ms 6 - 12 MB NPU
encoder ONNX w8a8 Qualcomm® Dragonwing™ IQ-8275 77.777 ms 10 - 17 MB NPU
encoder ONNX w8a8 Qualcomm® Dragonwing™ QCS8550 (Proxy) 88.339 ms 0 - 55 MB NPU
encoder ONNX w8a8 Qualcomm® QCS8450 125.173 ms 12 - 2401 MB NPU
encoder ONNX w8a8 Qualcomm® Dragonwing™ IQ-9075 75.326 ms 11 - 16 MB NPU
encoder ONNX w8a8 Qualcomm® Dragonwing™ IQ-X7181 90.525 ms 51 - 51 MB NPU
encoder ONNX w8a8 Qualcomm® Dragonwing™ Q-6690 1674.736 ms 3 - 1937 MB NPU
encoder ONNX w8a8 Qualcomm® Dragonwing™ Q-7790 89.094 ms 6 - 2349 MB NPU
encoder ONNX w8a8 Qualcomm® Dragonwing™ Q-8750 38.756 ms 12 - 2123 MB NPU
encoder ONNX w8a8 Snapdragon® 8 Elite Mobile 38.756 ms 12 - 2123 MB NPU
encoder ONNX w8a8 Snapdragon® 8 Elite Gen 5 Mobile 34.165 ms 10 - 1677 MB NPU
encoder ONNX w8a8 Snapdragon® 7 Gen 4 Mobile 89.094 ms 6 - 2349 MB NPU
encoder QNN_DLC float Snapdragon® X2 Elite 85.9 ms 12 - 12 MB NPU
encoder QNN_DLC float Snapdragon® 8 Gen 1 Mobile 348.234 ms 13 - 1862 MB NPU
encoder QNN_DLC float Qualcomm® QCS8450 348.234 ms 13 - 1862 MB NPU
encoder QNN_DLC float Qualcomm® SA8295P 282.442 ms 0 - 1581 MB NPU
encoder QNN_DLC float Snapdragon® 8 Elite Gen 5 Mobile 78.305 ms 0 - 1704 MB NPU
encoder QNN_DLC w8a8 Snapdragon® X2 Elite 36.432 ms 3 - 3 MB NPU
encoder QNN_DLC w8a8 Snapdragon® X Elite 83.466 ms 3 - 3 MB NPU
encoder QNN_DLC w8a8 Snapdragon® 8 Gen 3 Mobile 54.694 ms 3 - 1888 MB NPU
encoder QNN_DLC w8a8 Snapdragon® 8 Gen 1 Mobile 138.136 ms 3 - 1903 MB NPU
encoder QNN_DLC w8a8 Qualcomm® Dragonwing™ QCS6490 679.012 ms 3 - 13 MB NPU
encoder QNN_DLC w8a8 Qualcomm® Dragonwing™ IQ-8275 70.435 ms 3 - 13 MB NPU
encoder QNN_DLC w8a8 Qualcomm® Dragonwing™ IQ-8275 136.242 ms 3 - 1581 MB NPU
encoder QNN_DLC w8a8 Qualcomm® Dragonwing™ QCS8550 (Proxy) 79.664 ms 3 - 6 MB NPU
encoder QNN_DLC w8a8 Qualcomm® SA8775P 81.784 ms 3 - 1581 MB NPU
encoder QNN_DLC w8a8 Qualcomm® SA8650P 81.784 ms 3 - 1581 MB NPU
encoder QNN_DLC w8a8 Qualcomm® SA8255P 81.784 ms 3 - 1581 MB NPU
encoder QNN_DLC w8a8 Qualcomm® QCS8450 138.136 ms 3 - 1903 MB NPU
encoder QNN_DLC w8a8 Qualcomm® Dragonwing™ IQ-9075 81.648 ms 3 - 12 MB NPU
encoder QNN_DLC w8a8 Qualcomm® Dragonwing™ IQ-X7181 83.466 ms 3 - 3 MB NPU
encoder QNN_DLC w8a8 Qualcomm® Dragonwing™ Q-6690 1558.831 ms 3 - 1919 MB NPU
encoder QNN_DLC w8a8 Qualcomm® Dragonwing™ Q-7790 95.55 ms 3 - 2357 MB NPU
encoder QNN_DLC w8a8 Qualcomm® Dragonwing™ Q-8750 40.512 ms 3 - 1632 MB NPU
encoder QNN_DLC w8a8 Qualcomm® SA7255P 136.242 ms 3 - 1581 MB NPU
encoder QNN_DLC w8a8 Qualcomm® SA8295P 96.056 ms 0 - 1592 MB NPU
encoder QNN_DLC w8a8 Snapdragon® 8 Elite Mobile 40.512 ms 3 - 1632 MB NPU
encoder QNN_DLC w8a8 Snapdragon® 8 Elite Gen 5 Mobile 35.301 ms 3 - 1664 MB NPU
encoder QNN_DLC w8a8 Snapdragon® 7 Gen 4 Mobile 95.55 ms 3 - 2357 MB NPU
encoder TFLITE float Snapdragon® 8 Gen 1 Mobile 357.909 ms 21 - 1956 MB NPU
encoder TFLITE float Qualcomm® QCS8450 357.909 ms 21 - 1956 MB NPU
encoder TFLITE float Qualcomm® SA8295P 282.707 ms 21 - 1665 MB NPU
encoder TFLITE float Snapdragon® 8 Elite Gen 5 Mobile 76.474 ms 20 - 1777 MB NPU
encoder TFLITE w8a8 Snapdragon® 8 Gen 3 Mobile 69.696 ms 4 - 2229 MB NPU
encoder TFLITE w8a8 Snapdragon® 8 Gen 1 Mobile 186.02 ms 5 - 2293 MB NPU
encoder TFLITE w8a8 Qualcomm® Dragonwing™ QCS6490 1720.803 ms 78 - 341 MB CPU
encoder TFLITE w8a8 Qualcomm® Dragonwing™ IQ-8275 87.51 ms 5 - 71 MB NPU
encoder TFLITE w8a8 Qualcomm® Dragonwing™ IQ-8275 162.66 ms 5 - 1878 MB NPU
encoder TFLITE w8a8 Qualcomm® Dragonwing™ QCS8550 (Proxy) 100.537 ms 5 - 10 MB NPU
encoder TFLITE w8a8 Qualcomm® SA8775P 98.128 ms 6 - 1877 MB NPU
encoder TFLITE w8a8 Qualcomm® SA8650P 98.128 ms 6 - 1877 MB NPU
encoder TFLITE w8a8 Qualcomm® SA8255P 98.128 ms 6 - 1877 MB NPU
encoder TFLITE w8a8 Qualcomm® QCS8450 186.02 ms 5 - 2293 MB NPU
encoder TFLITE w8a8 Qualcomm® Dragonwing™ IQ-9075 96.191 ms 3 - 69 MB NPU
encoder TFLITE w8a8 Qualcomm® Dragonwing™ Q-6690 2329.849 ms 61 - 294 MB CPU
encoder TFLITE w8a8 Qualcomm® Dragonwing™ Q-7790 1130.323 ms 66 - 277 MB CPU
encoder TFLITE w8a8 Qualcomm® Dragonwing™ Q-8750 53.951 ms 5 - 1793 MB NPU
encoder TFLITE w8a8 Qualcomm® SA7255P 162.66 ms 5 - 1878 MB NPU
encoder TFLITE w8a8 Qualcomm® SA8295P 117.61 ms 5 - 1883 MB NPU
encoder TFLITE w8a8 Snapdragon® 8 Elite Mobile 53.951 ms 5 - 1793 MB NPU
encoder TFLITE w8a8 Snapdragon® 8 Elite Gen 5 Mobile 34.456 ms 4 - 1792 MB NPU
encoder TFLITE w8a8 Snapdragon® 7 Gen 4 Mobile 1130.323 ms 66 - 277 MB CPU

License

  • The license for the original implementation of Segment-Anything-Model-2 can be found here.

References

Community