Hugging Face's logo

qualcomm
/
WeDetect

Object Detection
PyTorch
real_time
android
Model card Files
xet
 Community
WeDetect / README.md
qaihm-bot's picture
qaihm-bot
v0.63.0
875be77 verified
preview code
|
Raw History Blame Contribute Delete
14.2 kB
metadata 
library_name: pytorch
license: gpl-3.0
tags:
  - real_time
  - android
pipeline_tag: object-detection

WeDetect: Optimized for Qualcomm Devices

WeDetect is a text-conditioned object detection model that uses a prompt-then-detect strategy. Text class names are reparameterized into the model weights before export, producing a fixed-vocabulary detector that runs efficiently on-device. The model uses LTRB (Left-Top-Right-Bottom) regression with multi-scale feature maps at strides 8, 16, and 32.

This is based on the implementation of WeDetect found here. This repository contains pre-exported model files optimized for Qualcomm® devices. You can use the Qualcomm® AI Hub Models library to export with custom configurations. More details on model performance across various devices, can be found here.

Qualcomm AI Hub Models uses Qualcomm AI Hub Workbench to compile, profile, and evaluate this model. Sign up to run these models on a hosted Qualcomm® device.

Getting Started

Due to licensing restrictions, we cannot distribute pre-exported model assets for this model. Use the Qualcomm® AI Hub Models Python library to compile and export the model with your own:

  • Custom weights (e.g., fine-tuned checkpoints)
  • Custom input shapes
  • Target device and runtime configurations

See our repository for WeDetect on GitHub for usage instructions.

Model Details

Model Type: Model_use_case.object_detection

Model Stats:

  • Input resolution: 640x640
  • Model checkpoint: wedetect_tiny
  • Model size (detector) (float): 143 MB
  • Model size (detector) (mixed_with_float): 35.9 MB
  • Model size (text_encoder) (float): 592 B
  • Model size (text_encoder) (mixed_with_float): 13.3 KB
  • Number of parameters (detector): 37.3M
  • Number of parameters (text_encoder): 3.00

Performance Summary

Model Runtime Precision Chipset Inference Time (ms) Peak Memory Range (MB) Primary Compute Unit
detector ONNX float Snapdragon® 8 Elite Gen 5 For Galaxy Mobile 9.235 ms 1 - 244 MB NPU
detector ONNX float Snapdragon® 8 Elite For Galaxy Mobile 11.53 ms 1 - 240 MB NPU
detector ONNX float Snapdragon® X2 Elite 11.417 ms 5 - 5 MB NPU
detector ONNX float Snapdragon® X Elite 21.053 ms 71 - 71 MB NPU
detector ONNX float Snapdragon® 8 Gen 3 Mobile 15.889 ms 0 - 319 MB NPU
detector ONNX float Snapdragon® 8 Gen 1 Mobile 76.841 ms 1 - 318 MB NPU
detector ONNX float Qualcomm® Dragonwing™ IQ-8275 34.96 ms 5 - 14 MB NPU
detector ONNX float Qualcomm® Dragonwing™ QCS8550 (Proxy) 20.334 ms 0 - 77 MB NPU
detector ONNX float Qualcomm® QCS8450 76.841 ms 1 - 318 MB NPU
detector ONNX float Qualcomm® Dragonwing™ IQ-9075 30.936 ms 5 - 13 MB NPU
detector ONNX float Qualcomm® Dragonwing™ IQ-X7181 21.053 ms 71 - 71 MB NPU
detector ONNX float Qualcomm® Dragonwing™ Q-8750 11.53 ms 1 - 240 MB NPU
detector ONNX mixed_with_float Snapdragon® 8 Elite Gen 5 For Galaxy Mobile 7.682 ms 4 - 283 MB NPU
detector ONNX mixed_with_float Snapdragon® 8 Elite For Galaxy Mobile 9.35 ms 7 - 281 MB NPU
detector ONNX mixed_with_float Snapdragon® X2 Elite 7.438 ms 9 - 9 MB NPU
detector ONNX mixed_with_float Snapdragon® X Elite 16.328 ms 34 - 34 MB NPU
detector ONNX mixed_with_float Snapdragon® 8 Gen 3 Mobile 11.155 ms 12 - 367 MB NPU
detector ONNX mixed_with_float Snapdragon® 8 Gen 1 Mobile 24.199 ms 11 - 367 MB NPU
detector ONNX mixed_with_float Qualcomm® Dragonwing™ IQ-8275 15.51 ms 6 - 15 MB NPU
detector ONNX mixed_with_float Qualcomm® Dragonwing™ QCS8550 (Proxy) 15.839 ms 0 - 38 MB NPU
detector ONNX mixed_with_float Qualcomm® QCS8450 24.199 ms 11 - 367 MB NPU
detector ONNX mixed_with_float Qualcomm® Dragonwing™ IQ-9075 15.669 ms 6 - 14 MB NPU
detector ONNX mixed_with_float Qualcomm® Dragonwing™ IQ-X7181 16.328 ms 34 - 34 MB NPU
detector ONNX mixed_with_float Qualcomm® Dragonwing™ Q-8750 9.35 ms 7 - 281 MB NPU
detector QNN_DLC float Snapdragon® 8 Elite Gen 5 For Galaxy Mobile 8.071 ms 5 - 255 MB NPU
detector QNN_DLC float Snapdragon® 8 Elite For Galaxy Mobile 11.174 ms 0 - 248 MB NPU
detector QNN_DLC float Snapdragon® 8 Gen 3 Mobile 15.635 ms 3 - 327 MB NPU
detector QNN_DLC float Snapdragon® 8 Gen 1 Mobile 76.316 ms 4 - 321 MB NPU
detector QNN_DLC float Qualcomm® Dragonwing™ IQ-8275 34.697 ms 5 - 13 MB NPU
detector QNN_DLC float Qualcomm® Dragonwing™ QCS8550 (Proxy) 19.67 ms 5 - 7 MB NPU
detector QNN_DLC float Qualcomm® SA8650P 30.339 ms 3 - 297 MB NPU
detector QNN_DLC float Qualcomm® SA8255P 30.339 ms 3 - 297 MB NPU
detector QNN_DLC float Qualcomm® QCS8450 76.316 ms 4 - 321 MB NPU
detector QNN_DLC float Qualcomm® Dragonwing™ IQ-9075 31.478 ms 5 - 12 MB NPU
detector QNN_DLC float Qualcomm® Dragonwing™ Q-8750 11.174 ms 0 - 248 MB NPU
detector QNN_DLC float Qualcomm® SA8295P 55.874 ms 0 - 246 MB NPU
detector QNN_DLC mixed_with_float Snapdragon® 8 Elite Gen 5 For Galaxy Mobile 7.237 ms 5 - 271 MB NPU
detector QNN_DLC mixed_with_float Snapdragon® 8 Elite For Galaxy Mobile 9.124 ms 0 - 260 MB NPU
detector QNN_DLC mixed_with_float Snapdragon® 8 Gen 3 Mobile 10.734 ms 5 - 346 MB NPU
detector QNN_DLC mixed_with_float Qualcomm® Dragonwing™ IQ-8275 15.191 ms 5 - 13 MB NPU
detector QNN_DLC mixed_with_float Qualcomm® Dragonwing™ QCS8550 (Proxy) 15.285 ms 5 - 11 MB NPU
detector QNN_DLC mixed_with_float Qualcomm® SA8650P 18.211 ms 1 - 307 MB NPU
detector QNN_DLC mixed_with_float Qualcomm® SA8255P 18.211 ms 1 - 307 MB NPU
detector QNN_DLC mixed_with_float Qualcomm® Dragonwing™ IQ-9075 15.97 ms 7 - 14 MB NPU
detector QNN_DLC mixed_with_float Qualcomm® Dragonwing™ Q-8750 9.124 ms 0 - 260 MB NPU
detector TFLITE float Snapdragon® 8 Elite Gen 5 For Galaxy Mobile 9.305 ms 0 - 256 MB NPU
detector TFLITE float Snapdragon® 8 Elite For Galaxy Mobile 11.593 ms 0 - 251 MB NPU
detector TFLITE float Snapdragon® 8 Gen 3 Mobile 16.661 ms 0 - 335 MB NPU
detector TFLITE float Snapdragon® 8 Gen 1 Mobile 76.469 ms 1 - 335 MB NPU
detector TFLITE float Qualcomm® Dragonwing™ IQ-8275 36.514 ms 0 - 84 MB NPU
detector TFLITE float Qualcomm® Dragonwing™ QCS8550 (Proxy) 20.687 ms 0 - 8 MB NPU
detector TFLITE float Qualcomm® SA8650P 31.601 ms 0 - 267 MB NPU
detector TFLITE float Qualcomm® SA8255P 31.601 ms 0 - 267 MB NPU
detector TFLITE float Qualcomm® QCS8450 76.469 ms 1 - 335 MB NPU
detector TFLITE float Qualcomm® Dragonwing™ IQ-9075 30.763 ms 0 - 83 MB NPU
detector TFLITE float Qualcomm® Dragonwing™ Q-8750 11.593 ms 0 - 251 MB NPU
detector TFLITE float Qualcomm® SA8295P 57.059 ms 0 - 254 MB NPU
text_encoder ONNX float Snapdragon® 8 Elite Gen 5 For Galaxy Mobile 31.029 ms 1 - 2176 MB NPU
text_encoder ONNX float Snapdragon® 8 Elite For Galaxy Mobile 44.924 ms 1 - 1981 MB NPU
text_encoder ONNX float Snapdragon® X2 Elite 33.125 ms 1 - 1 MB NPU
text_encoder ONNX float Snapdragon® X Elite 83.669 ms 551 - 551 MB NPU
text_encoder ONNX float Snapdragon® 8 Gen 3 Mobile 60.151 ms 1 - 3626 MB NPU
text_encoder ONNX float Snapdragon® 8 Gen 1 Mobile 110.729 ms 2 - 1814 MB NPU
text_encoder ONNX float Qualcomm® Dragonwing™ IQ-8275 92.427 ms 0 - 4 MB NPU
text_encoder ONNX float Qualcomm® Dragonwing™ QCS8550 (Proxy) 80.593 ms 0 - 644 MB NPU
text_encoder ONNX float Qualcomm® QCS8450 110.729 ms 2 - 1814 MB NPU
text_encoder ONNX float Qualcomm® Dragonwing™ IQ-9075 90.841 ms 1 - 4 MB NPU
text_encoder ONNX float Qualcomm® Dragonwing™ IQ-X7181 83.669 ms 551 - 551 MB NPU
text_encoder ONNX float Qualcomm® Dragonwing™ Q-8750 44.924 ms 1 - 1981 MB NPU
text_encoder ONNX mixed_with_float Snapdragon® 8 Elite Gen 5 For Galaxy Mobile 31.158 ms 1 - 2175 MB NPU
text_encoder ONNX mixed_with_float Snapdragon® 8 Elite For Galaxy Mobile 44.333 ms 1 - 1980 MB NPU
text_encoder ONNX mixed_with_float Snapdragon® X2 Elite 33.202 ms 1 - 1 MB NPU
text_encoder ONNX mixed_with_float Snapdragon® X Elite 83.512 ms 550 - 550 MB NPU
text_encoder ONNX mixed_with_float Snapdragon® 8 Gen 3 Mobile 60.158 ms 1 - 3626 MB NPU
text_encoder ONNX mixed_with_float Snapdragon® 8 Gen 1 Mobile 111.209 ms 2 - 1816 MB NPU
text_encoder ONNX mixed_with_float Qualcomm® Dragonwing™ IQ-8275 92.349 ms 0 - 4 MB NPU
text_encoder ONNX mixed_with_float Qualcomm® Dragonwing™ QCS8550 (Proxy) 80.847 ms 0 - 644 MB NPU
text_encoder ONNX mixed_with_float Qualcomm® QCS8450 111.209 ms 2 - 1816 MB NPU
text_encoder ONNX mixed_with_float Qualcomm® Dragonwing™ IQ-9075 90.752 ms 1 - 4 MB NPU
text_encoder ONNX mixed_with_float Qualcomm® Dragonwing™ IQ-X7181 83.512 ms 550 - 550 MB NPU
text_encoder ONNX mixed_with_float Qualcomm® Dragonwing™ Q-8750 44.333 ms 1 - 1980 MB NPU
text_encoder QNN_DLC float Snapdragon® 8 Elite Gen 5 For Galaxy Mobile 30.549 ms 0 - 1923 MB NPU
text_encoder QNN_DLC float Snapdragon® 8 Elite For Galaxy Mobile 43.419 ms 0 - 2052 MB NPU
text_encoder QNN_DLC float Snapdragon® 8 Gen 3 Mobile 55.821 ms 0 - 2293 MB NPU
text_encoder QNN_DLC float Snapdragon® 8 Gen 1 Mobile 106.955 ms 0 - 1823 MB NPU
text_encoder QNN_DLC float Qualcomm® Dragonwing™ IQ-8275 89.81 ms 0 - 3 MB NPU
text_encoder QNN_DLC float Qualcomm® Dragonwing™ QCS8550 (Proxy) 77.06 ms 0 - 2 MB NPU
text_encoder QNN_DLC float Qualcomm® SA8650P 95.659 ms 1 - 2019 MB NPU
text_encoder QNN_DLC float Qualcomm® SA8255P 95.659 ms 1 - 2019 MB NPU
text_encoder QNN_DLC float Qualcomm® QCS8450 106.955 ms 0 - 1823 MB NPU
text_encoder QNN_DLC float Qualcomm® Dragonwing™ IQ-9075 86.62 ms 0 - 2 MB NPU
text_encoder QNN_DLC float Qualcomm® Dragonwing™ Q-8750 43.419 ms 0 - 2052 MB NPU
text_encoder QNN_DLC float Qualcomm® SA8295P 102.822 ms 0 - 1699 MB NPU
text_encoder QNN_DLC mixed_with_float Snapdragon® 8 Elite Gen 5 For Galaxy Mobile 30.66 ms 0 - 1922 MB NPU
text_encoder QNN_DLC mixed_with_float Snapdragon® 8 Elite For Galaxy Mobile 43.864 ms 0 - 2050 MB NPU
text_encoder QNN_DLC mixed_with_float Snapdragon® 8 Gen 3 Mobile 58.135 ms 9 - 2303 MB NPU
text_encoder QNN_DLC mixed_with_float Qualcomm® Dragonwing™ IQ-8275 90.055 ms 0 - 3 MB NPU
text_encoder QNN_DLC mixed_with_float Qualcomm® Dragonwing™ QCS8550 (Proxy) 77.593 ms 0 - 3 MB NPU
text_encoder QNN_DLC mixed_with_float Qualcomm® SA8650P 95.579 ms 1 - 2019 MB NPU
text_encoder QNN_DLC mixed_with_float Qualcomm® SA8255P 95.579 ms 1 - 2019 MB NPU
text_encoder QNN_DLC mixed_with_float Qualcomm® Dragonwing™ IQ-9075 87.716 ms 2 - 4 MB NPU
text_encoder QNN_DLC mixed_with_float Qualcomm® Dragonwing™ Q-8750 43.864 ms 0 - 2050 MB NPU
text_encoder TFLITE float Snapdragon® 8 Elite Gen 5 For Galaxy Mobile 33.01 ms 0 - 1764 MB NPU
text_encoder TFLITE float Snapdragon® 8 Elite For Galaxy Mobile 50.301 ms 0 - 1731 MB NPU
text_encoder TFLITE float Snapdragon® 8 Gen 3 Mobile 63.551 ms 0 - 1978 MB NPU
text_encoder TFLITE float Snapdragon® 8 Gen 1 Mobile 112.967 ms 1 - 1865 MB NPU
text_encoder TFLITE float Qualcomm® Dragonwing™ IQ-8275 95.653 ms 0 - 553 MB NPU
text_encoder TFLITE float Qualcomm® Dragonwing™ QCS8550 (Proxy) 84.951 ms 0 - 4 MB NPU
text_encoder TFLITE float Qualcomm® SA8650P 111.23 ms 1 - 2145 MB NPU
text_encoder TFLITE float Qualcomm® SA8255P 111.23 ms 1 - 2145 MB NPU
text_encoder TFLITE float Qualcomm® QCS8450 112.967 ms 1 - 1865 MB NPU
text_encoder TFLITE float Qualcomm® Dragonwing™ IQ-9075 93.836 ms 0 - 552 MB NPU
text_encoder TFLITE float Qualcomm® Dragonwing™ Q-8750 50.301 ms 0 - 1731 MB NPU
text_encoder TFLITE float Qualcomm® SA8295P 111.099 ms 1 - 1736 MB NPU

License

  • The license for the original implementation of WeDetect can be found here.

References

Community