Object Detection
PyTorch
real_time
android
WeDetect / README.md
qaihm-bot's picture
v0.63.0
875be77 verified
|
Raw History Blame Contribute Delete
14.2 kB
---
library_name: pytorch
license: gpl-3.0
tags:
- real_time
- android
pipeline_tag: object-detection
---
![](https://qaihub-public-assets.s3.us-west-2.amazonaws.com/qai-hub-models/models/wedetect/web-assets/model_demo.png)
# WeDetect: Optimized for Qualcomm Devices
WeDetect is a text-conditioned object detection model that uses a prompt-then-detect strategy. Text class names are reparameterized into the model weights before export, producing a fixed-vocabulary detector that runs efficiently on-device. The model uses LTRB (Left-Top-Right-Bottom) regression with multi-scale feature maps at strides 8, 16, and 32.
This is based on the implementation of WeDetect found [here](https://github.com/WeChatCV/WeDetect).
This repository contains pre-exported model files optimized for Qualcomm® devices. You can use the [Qualcomm® AI Hub Models](https://github.com/qualcomm/ai-hub-models/blob/v0.63.0/src/qai_hub_models/models/wedetect) library to export with custom configurations. More details on model performance across various devices, can be found [here](#performance-summary).
Qualcomm AI Hub Models uses [Qualcomm AI Hub Workbench](https://workbench.aihub.qualcomm.com) to compile, profile, and evaluate this model. [Sign up](https://myaccount.qualcomm.com/signup) to run these models on a hosted Qualcomm® device.
## Getting Started
Due to licensing restrictions, we cannot distribute pre-exported model assets for this model.
Use the [Qualcomm® AI Hub Models](https://github.com/qualcomm/ai-hub-models/blob/v0.63.0/src/qai_hub_models/models/wedetect) Python library to compile and export the model with your own:
- Custom weights (e.g., fine-tuned checkpoints)
- Custom input shapes
- Target device and runtime configurations
See our repository for [WeDetect on GitHub](https://github.com/qualcomm/ai-hub-models/blob/v0.63.0/src/qai_hub_models/models/wedetect) for usage instructions.
## Model Details
**Model Type:** Model_use_case.object_detection
**Model Stats:**
- Input resolution: 640x640
- Model checkpoint: wedetect_tiny
- Model size (detector) (float): 143 MB
- Model size (detector) (mixed_with_float): 35.9 MB
- Model size (text_encoder) (float): 592 B
- Model size (text_encoder) (mixed_with_float): 13.3 KB
- Number of parameters (detector): 37.3M
- Number of parameters (text_encoder): 3.00
## Performance Summary
| Model | Runtime | Precision | Chipset | Inference Time (ms) | Peak Memory Range (MB) | Primary Compute Unit
|---|---|---|---|---|---|---
| detector | ONNX | float | Snapdragon® 8 Elite Gen 5 For Galaxy Mobile | 9.235 ms | 1 - 244 MB | NPU
| detector | ONNX | float | Snapdragon® 8 Elite For Galaxy Mobile | 11.53 ms | 1 - 240 MB | NPU
| detector | ONNX | float | Snapdragon® X2 Elite | 11.417 ms | 5 - 5 MB | NPU
| detector | ONNX | float | Snapdragon® X Elite | 21.053 ms | 71 - 71 MB | NPU
| detector | ONNX | float | Snapdragon® 8 Gen 3 Mobile | 15.889 ms | 0 - 319 MB | NPU
| detector | ONNX | float | Snapdragon® 8 Gen 1 Mobile | 76.841 ms | 1 - 318 MB | NPU
| detector | ONNX | float | Qualcomm® Dragonwing™ IQ-8275 | 34.96 ms | 5 - 14 MB | NPU
| detector | ONNX | float | Qualcomm® Dragonwing™ QCS8550 (Proxy) | 20.334 ms | 0 - 77 MB | NPU
| detector | ONNX | float | Qualcomm® QCS8450 | 76.841 ms | 1 - 318 MB | NPU
| detector | ONNX | float | Qualcomm® Dragonwing™ IQ-9075 | 30.936 ms | 5 - 13 MB | NPU
| detector | ONNX | float | Qualcomm® Dragonwing™ IQ-X7181 | 21.053 ms | 71 - 71 MB | NPU
| detector | ONNX | float | Qualcomm® Dragonwing™ Q-8750 | 11.53 ms | 1 - 240 MB | NPU
| detector | ONNX | mixed_with_float | Snapdragon® 8 Elite Gen 5 For Galaxy Mobile | 7.682 ms | 4 - 283 MB | NPU
| detector | ONNX | mixed_with_float | Snapdragon® 8 Elite For Galaxy Mobile | 9.35 ms | 7 - 281 MB | NPU
| detector | ONNX | mixed_with_float | Snapdragon® X2 Elite | 7.438 ms | 9 - 9 MB | NPU
| detector | ONNX | mixed_with_float | Snapdragon® X Elite | 16.328 ms | 34 - 34 MB | NPU
| detector | ONNX | mixed_with_float | Snapdragon® 8 Gen 3 Mobile | 11.155 ms | 12 - 367 MB | NPU
| detector | ONNX | mixed_with_float | Snapdragon® 8 Gen 1 Mobile | 24.199 ms | 11 - 367 MB | NPU
| detector | ONNX | mixed_with_float | Qualcomm® Dragonwing™ IQ-8275 | 15.51 ms | 6 - 15 MB | NPU
| detector | ONNX | mixed_with_float | Qualcomm® Dragonwing™ QCS8550 (Proxy) | 15.839 ms | 0 - 38 MB | NPU
| detector | ONNX | mixed_with_float | Qualcomm® QCS8450 | 24.199 ms | 11 - 367 MB | NPU
| detector | ONNX | mixed_with_float | Qualcomm® Dragonwing™ IQ-9075 | 15.669 ms | 6 - 14 MB | NPU
| detector | ONNX | mixed_with_float | Qualcomm® Dragonwing™ IQ-X7181 | 16.328 ms | 34 - 34 MB | NPU
| detector | ONNX | mixed_with_float | Qualcomm® Dragonwing™ Q-8750 | 9.35 ms | 7 - 281 MB | NPU
| detector | QNN_DLC | float | Snapdragon® 8 Elite Gen 5 For Galaxy Mobile | 8.071 ms | 5 - 255 MB | NPU
| detector | QNN_DLC | float | Snapdragon® 8 Elite For Galaxy Mobile | 11.174 ms | 0 - 248 MB | NPU
| detector | QNN_DLC | float | Snapdragon® 8 Gen 3 Mobile | 15.635 ms | 3 - 327 MB | NPU
| detector | QNN_DLC | float | Snapdragon® 8 Gen 1 Mobile | 76.316 ms | 4 - 321 MB | NPU
| detector | QNN_DLC | float | Qualcomm® Dragonwing™ IQ-8275 | 34.697 ms | 5 - 13 MB | NPU
| detector | QNN_DLC | float | Qualcomm® Dragonwing™ QCS8550 (Proxy) | 19.67 ms | 5 - 7 MB | NPU
| detector | QNN_DLC | float | Qualcomm® SA8650P | 30.339 ms | 3 - 297 MB | NPU
| detector | QNN_DLC | float | Qualcomm® SA8255P | 30.339 ms | 3 - 297 MB | NPU
| detector | QNN_DLC | float | Qualcomm® QCS8450 | 76.316 ms | 4 - 321 MB | NPU
| detector | QNN_DLC | float | Qualcomm® Dragonwing™ IQ-9075 | 31.478 ms | 5 - 12 MB | NPU
| detector | QNN_DLC | float | Qualcomm® Dragonwing™ Q-8750 | 11.174 ms | 0 - 248 MB | NPU
| detector | QNN_DLC | float | Qualcomm® SA8295P | 55.874 ms | 0 - 246 MB | NPU
| detector | QNN_DLC | mixed_with_float | Snapdragon® 8 Elite Gen 5 For Galaxy Mobile | 7.237 ms | 5 - 271 MB | NPU
| detector | QNN_DLC | mixed_with_float | Snapdragon® 8 Elite For Galaxy Mobile | 9.124 ms | 0 - 260 MB | NPU
| detector | QNN_DLC | mixed_with_float | Snapdragon® 8 Gen 3 Mobile | 10.734 ms | 5 - 346 MB | NPU
| detector | QNN_DLC | mixed_with_float | Qualcomm® Dragonwing™ IQ-8275 | 15.191 ms | 5 - 13 MB | NPU
| detector | QNN_DLC | mixed_with_float | Qualcomm® Dragonwing™ QCS8550 (Proxy) | 15.285 ms | 5 - 11 MB | NPU
| detector | QNN_DLC | mixed_with_float | Qualcomm® SA8650P | 18.211 ms | 1 - 307 MB | NPU
| detector | QNN_DLC | mixed_with_float | Qualcomm® SA8255P | 18.211 ms | 1 - 307 MB | NPU
| detector | QNN_DLC | mixed_with_float | Qualcomm® Dragonwing™ IQ-9075 | 15.97 ms | 7 - 14 MB | NPU
| detector | QNN_DLC | mixed_with_float | Qualcomm® Dragonwing™ Q-8750 | 9.124 ms | 0 - 260 MB | NPU
| detector | TFLITE | float | Snapdragon® 8 Elite Gen 5 For Galaxy Mobile | 9.305 ms | 0 - 256 MB | NPU
| detector | TFLITE | float | Snapdragon® 8 Elite For Galaxy Mobile | 11.593 ms | 0 - 251 MB | NPU
| detector | TFLITE | float | Snapdragon® 8 Gen 3 Mobile | 16.661 ms | 0 - 335 MB | NPU
| detector | TFLITE | float | Snapdragon® 8 Gen 1 Mobile | 76.469 ms | 1 - 335 MB | NPU
| detector | TFLITE | float | Qualcomm® Dragonwing™ IQ-8275 | 36.514 ms | 0 - 84 MB | NPU
| detector | TFLITE | float | Qualcomm® Dragonwing™ QCS8550 (Proxy) | 20.687 ms | 0 - 8 MB | NPU
| detector | TFLITE | float | Qualcomm® SA8650P | 31.601 ms | 0 - 267 MB | NPU
| detector | TFLITE | float | Qualcomm® SA8255P | 31.601 ms | 0 - 267 MB | NPU
| detector | TFLITE | float | Qualcomm® QCS8450 | 76.469 ms | 1 - 335 MB | NPU
| detector | TFLITE | float | Qualcomm® Dragonwing™ IQ-9075 | 30.763 ms | 0 - 83 MB | NPU
| detector | TFLITE | float | Qualcomm® Dragonwing™ Q-8750 | 11.593 ms | 0 - 251 MB | NPU
| detector | TFLITE | float | Qualcomm® SA8295P | 57.059 ms | 0 - 254 MB | NPU
| text_encoder | ONNX | float | Snapdragon® 8 Elite Gen 5 For Galaxy Mobile | 31.029 ms | 1 - 2176 MB | NPU
| text_encoder | ONNX | float | Snapdragon® 8 Elite For Galaxy Mobile | 44.924 ms | 1 - 1981 MB | NPU
| text_encoder | ONNX | float | Snapdragon® X2 Elite | 33.125 ms | 1 - 1 MB | NPU
| text_encoder | ONNX | float | Snapdragon® X Elite | 83.669 ms | 551 - 551 MB | NPU
| text_encoder | ONNX | float | Snapdragon® 8 Gen 3 Mobile | 60.151 ms | 1 - 3626 MB | NPU
| text_encoder | ONNX | float | Snapdragon® 8 Gen 1 Mobile | 110.729 ms | 2 - 1814 MB | NPU
| text_encoder | ONNX | float | Qualcomm® Dragonwing™ IQ-8275 | 92.427 ms | 0 - 4 MB | NPU
| text_encoder | ONNX | float | Qualcomm® Dragonwing™ QCS8550 (Proxy) | 80.593 ms | 0 - 644 MB | NPU
| text_encoder | ONNX | float | Qualcomm® QCS8450 | 110.729 ms | 2 - 1814 MB | NPU
| text_encoder | ONNX | float | Qualcomm® Dragonwing™ IQ-9075 | 90.841 ms | 1 - 4 MB | NPU
| text_encoder | ONNX | float | Qualcomm® Dragonwing™ IQ-X7181 | 83.669 ms | 551 - 551 MB | NPU
| text_encoder | ONNX | float | Qualcomm® Dragonwing™ Q-8750 | 44.924 ms | 1 - 1981 MB | NPU
| text_encoder | ONNX | mixed_with_float | Snapdragon® 8 Elite Gen 5 For Galaxy Mobile | 31.158 ms | 1 - 2175 MB | NPU
| text_encoder | ONNX | mixed_with_float | Snapdragon® 8 Elite For Galaxy Mobile | 44.333 ms | 1 - 1980 MB | NPU
| text_encoder | ONNX | mixed_with_float | Snapdragon® X2 Elite | 33.202 ms | 1 - 1 MB | NPU
| text_encoder | ONNX | mixed_with_float | Snapdragon® X Elite | 83.512 ms | 550 - 550 MB | NPU
| text_encoder | ONNX | mixed_with_float | Snapdragon® 8 Gen 3 Mobile | 60.158 ms | 1 - 3626 MB | NPU
| text_encoder | ONNX | mixed_with_float | Snapdragon® 8 Gen 1 Mobile | 111.209 ms | 2 - 1816 MB | NPU
| text_encoder | ONNX | mixed_with_float | Qualcomm® Dragonwing™ IQ-8275 | 92.349 ms | 0 - 4 MB | NPU
| text_encoder | ONNX | mixed_with_float | Qualcomm® Dragonwing™ QCS8550 (Proxy) | 80.847 ms | 0 - 644 MB | NPU
| text_encoder | ONNX | mixed_with_float | Qualcomm® QCS8450 | 111.209 ms | 2 - 1816 MB | NPU
| text_encoder | ONNX | mixed_with_float | Qualcomm® Dragonwing™ IQ-9075 | 90.752 ms | 1 - 4 MB | NPU
| text_encoder | ONNX | mixed_with_float | Qualcomm® Dragonwing™ IQ-X7181 | 83.512 ms | 550 - 550 MB | NPU
| text_encoder | ONNX | mixed_with_float | Qualcomm® Dragonwing™ Q-8750 | 44.333 ms | 1 - 1980 MB | NPU
| text_encoder | QNN_DLC | float | Snapdragon® 8 Elite Gen 5 For Galaxy Mobile | 30.549 ms | 0 - 1923 MB | NPU
| text_encoder | QNN_DLC | float | Snapdragon® 8 Elite For Galaxy Mobile | 43.419 ms | 0 - 2052 MB | NPU
| text_encoder | QNN_DLC | float | Snapdragon® 8 Gen 3 Mobile | 55.821 ms | 0 - 2293 MB | NPU
| text_encoder | QNN_DLC | float | Snapdragon® 8 Gen 1 Mobile | 106.955 ms | 0 - 1823 MB | NPU
| text_encoder | QNN_DLC | float | Qualcomm® Dragonwing™ IQ-8275 | 89.81 ms | 0 - 3 MB | NPU
| text_encoder | QNN_DLC | float | Qualcomm® Dragonwing™ QCS8550 (Proxy) | 77.06 ms | 0 - 2 MB | NPU
| text_encoder | QNN_DLC | float | Qualcomm® SA8650P | 95.659 ms | 1 - 2019 MB | NPU
| text_encoder | QNN_DLC | float | Qualcomm® SA8255P | 95.659 ms | 1 - 2019 MB | NPU
| text_encoder | QNN_DLC | float | Qualcomm® QCS8450 | 106.955 ms | 0 - 1823 MB | NPU
| text_encoder | QNN_DLC | float | Qualcomm® Dragonwing™ IQ-9075 | 86.62 ms | 0 - 2 MB | NPU
| text_encoder | QNN_DLC | float | Qualcomm® Dragonwing™ Q-8750 | 43.419 ms | 0 - 2052 MB | NPU
| text_encoder | QNN_DLC | float | Qualcomm® SA8295P | 102.822 ms | 0 - 1699 MB | NPU
| text_encoder | QNN_DLC | mixed_with_float | Snapdragon® 8 Elite Gen 5 For Galaxy Mobile | 30.66 ms | 0 - 1922 MB | NPU
| text_encoder | QNN_DLC | mixed_with_float | Snapdragon® 8 Elite For Galaxy Mobile | 43.864 ms | 0 - 2050 MB | NPU
| text_encoder | QNN_DLC | mixed_with_float | Snapdragon® 8 Gen 3 Mobile | 58.135 ms | 9 - 2303 MB | NPU
| text_encoder | QNN_DLC | mixed_with_float | Qualcomm® Dragonwing™ IQ-8275 | 90.055 ms | 0 - 3 MB | NPU
| text_encoder | QNN_DLC | mixed_with_float | Qualcomm® Dragonwing™ QCS8550 (Proxy) | 77.593 ms | 0 - 3 MB | NPU
| text_encoder | QNN_DLC | mixed_with_float | Qualcomm® SA8650P | 95.579 ms | 1 - 2019 MB | NPU
| text_encoder | QNN_DLC | mixed_with_float | Qualcomm® SA8255P | 95.579 ms | 1 - 2019 MB | NPU
| text_encoder | QNN_DLC | mixed_with_float | Qualcomm® Dragonwing™ IQ-9075 | 87.716 ms | 2 - 4 MB | NPU
| text_encoder | QNN_DLC | mixed_with_float | Qualcomm® Dragonwing™ Q-8750 | 43.864 ms | 0 - 2050 MB | NPU
| text_encoder | TFLITE | float | Snapdragon® 8 Elite Gen 5 For Galaxy Mobile | 33.01 ms | 0 - 1764 MB | NPU
| text_encoder | TFLITE | float | Snapdragon® 8 Elite For Galaxy Mobile | 50.301 ms | 0 - 1731 MB | NPU
| text_encoder | TFLITE | float | Snapdragon® 8 Gen 3 Mobile | 63.551 ms | 0 - 1978 MB | NPU
| text_encoder | TFLITE | float | Snapdragon® 8 Gen 1 Mobile | 112.967 ms | 1 - 1865 MB | NPU
| text_encoder | TFLITE | float | Qualcomm® Dragonwing™ IQ-8275 | 95.653 ms | 0 - 553 MB | NPU
| text_encoder | TFLITE | float | Qualcomm® Dragonwing™ QCS8550 (Proxy) | 84.951 ms | 0 - 4 MB | NPU
| text_encoder | TFLITE | float | Qualcomm® SA8650P | 111.23 ms | 1 - 2145 MB | NPU
| text_encoder | TFLITE | float | Qualcomm® SA8255P | 111.23 ms | 1 - 2145 MB | NPU
| text_encoder | TFLITE | float | Qualcomm® QCS8450 | 112.967 ms | 1 - 1865 MB | NPU
| text_encoder | TFLITE | float | Qualcomm® Dragonwing™ IQ-9075 | 93.836 ms | 0 - 552 MB | NPU
| text_encoder | TFLITE | float | Qualcomm® Dragonwing™ Q-8750 | 50.301 ms | 0 - 1731 MB | NPU
| text_encoder | TFLITE | float | Qualcomm® SA8295P | 111.099 ms | 1 - 1736 MB | NPU
## License
* The license for the original implementation of WeDetect can be found
[here](https://github.com/WeChatCV/WeDetect/blob/main/README.md).
## References
* [WeDetect: Fast Open-Vocabulary Object Detection as Retrieval](https://arxiv.org/abs/2512.12309)
* [Source Model Implementation](https://github.com/WeChatCV/WeDetect)
## Community
* Join [our AI Hub Slack community](https://aihub.qualcomm.com/community/slack) to collaborate, post questions and learn more about on-device AI.
* For questions or feedback please [reach out to us](mailto:ai-hub-support@qti.qualcomm.com).