|
Download README.md from qualcomm/WeDetect: direct link, hf CLI and curl.
- Browser
- Download file 14.2 kB
-
https://huggingface.co/qualcomm/WeDetect/resolve/main/README.md
- Command line
-
hf download hf://qualcomm/WeDetect/README.md
-
curl -L -o README.md https://huggingface.co/qualcomm/WeDetect/resolve/main/README.md
14.2 kB
| library_name: pytorch | |
| license: gpl-3.0 | |
| tags: | |
| - real_time | |
| - android | |
| pipeline_tag: object-detection | |
|  | |
| # WeDetect: Optimized for Qualcomm Devices | |
| WeDetect is a text-conditioned object detection model that uses a prompt-then-detect strategy. Text class names are reparameterized into the model weights before export, producing a fixed-vocabulary detector that runs efficiently on-device. The model uses LTRB (Left-Top-Right-Bottom) regression with multi-scale feature maps at strides 8, 16, and 32. | |
| This is based on the implementation of WeDetect found [here](https://github.com/WeChatCV/WeDetect). | |
| This repository contains pre-exported model files optimized for Qualcomm® devices. You can use the [Qualcomm® AI Hub Models](https://github.com/qualcomm/ai-hub-models/blob/v0.63.0/src/qai_hub_models/models/wedetect) library to export with custom configurations. More details on model performance across various devices, can be found [here](#performance-summary). | |
| Qualcomm AI Hub Models uses [Qualcomm AI Hub Workbench](https://workbench.aihub.qualcomm.com) to compile, profile, and evaluate this model. [Sign up](https://myaccount.qualcomm.com/signup) to run these models on a hosted Qualcomm® device. | |
| ## Getting Started | |
| Due to licensing restrictions, we cannot distribute pre-exported model assets for this model. | |
| Use the [Qualcomm® AI Hub Models](https://github.com/qualcomm/ai-hub-models/blob/v0.63.0/src/qai_hub_models/models/wedetect) Python library to compile and export the model with your own: | |
| - Custom weights (e.g., fine-tuned checkpoints) | |
| - Custom input shapes | |
| - Target device and runtime configurations | |
| See our repository for [WeDetect on GitHub](https://github.com/qualcomm/ai-hub-models/blob/v0.63.0/src/qai_hub_models/models/wedetect) for usage instructions. | |
| ## Model Details | |
| **Model Type:** Model_use_case.object_detection | |
| **Model Stats:** | |
| - Input resolution: 640x640 | |
| - Model checkpoint: wedetect_tiny | |
| - Model size (detector) (float): 143 MB | |
| - Model size (detector) (mixed_with_float): 35.9 MB | |
| - Model size (text_encoder) (float): 592 B | |
| - Model size (text_encoder) (mixed_with_float): 13.3 KB | |
| - Number of parameters (detector): 37.3M | |
| - Number of parameters (text_encoder): 3.00 | |
| ## Performance Summary | |
| | Model | Runtime | Precision | Chipset | Inference Time (ms) | Peak Memory Range (MB) | Primary Compute Unit | |
| |---|---|---|---|---|---|--- | |
| | detector | ONNX | float | Snapdragon® 8 Elite Gen 5 For Galaxy Mobile | 9.235 ms | 1 - 244 MB | NPU | |
| | detector | ONNX | float | Snapdragon® 8 Elite For Galaxy Mobile | 11.53 ms | 1 - 240 MB | NPU | |
| | detector | ONNX | float | Snapdragon® X2 Elite | 11.417 ms | 5 - 5 MB | NPU | |
| | detector | ONNX | float | Snapdragon® X Elite | 21.053 ms | 71 - 71 MB | NPU | |
| | detector | ONNX | float | Snapdragon® 8 Gen 3 Mobile | 15.889 ms | 0 - 319 MB | NPU | |
| | detector | ONNX | float | Snapdragon® 8 Gen 1 Mobile | 76.841 ms | 1 - 318 MB | NPU | |
| | detector | ONNX | float | Qualcomm® Dragonwing™ IQ-8275 | 34.96 ms | 5 - 14 MB | NPU | |
| | detector | ONNX | float | Qualcomm® Dragonwing™ QCS8550 (Proxy) | 20.334 ms | 0 - 77 MB | NPU | |
| | detector | ONNX | float | Qualcomm® QCS8450 | 76.841 ms | 1 - 318 MB | NPU | |
| | detector | ONNX | float | Qualcomm® Dragonwing™ IQ-9075 | 30.936 ms | 5 - 13 MB | NPU | |
| | detector | ONNX | float | Qualcomm® Dragonwing™ IQ-X7181 | 21.053 ms | 71 - 71 MB | NPU | |
| | detector | ONNX | float | Qualcomm® Dragonwing™ Q-8750 | 11.53 ms | 1 - 240 MB | NPU | |
| | detector | ONNX | mixed_with_float | Snapdragon® 8 Elite Gen 5 For Galaxy Mobile | 7.682 ms | 4 - 283 MB | NPU | |
| | detector | ONNX | mixed_with_float | Snapdragon® 8 Elite For Galaxy Mobile | 9.35 ms | 7 - 281 MB | NPU | |
| | detector | ONNX | mixed_with_float | Snapdragon® X2 Elite | 7.438 ms | 9 - 9 MB | NPU | |
| | detector | ONNX | mixed_with_float | Snapdragon® X Elite | 16.328 ms | 34 - 34 MB | NPU | |
| | detector | ONNX | mixed_with_float | Snapdragon® 8 Gen 3 Mobile | 11.155 ms | 12 - 367 MB | NPU | |
| | detector | ONNX | mixed_with_float | Snapdragon® 8 Gen 1 Mobile | 24.199 ms | 11 - 367 MB | NPU | |
| | detector | ONNX | mixed_with_float | Qualcomm® Dragonwing™ IQ-8275 | 15.51 ms | 6 - 15 MB | NPU | |
| | detector | ONNX | mixed_with_float | Qualcomm® Dragonwing™ QCS8550 (Proxy) | 15.839 ms | 0 - 38 MB | NPU | |
| | detector | ONNX | mixed_with_float | Qualcomm® QCS8450 | 24.199 ms | 11 - 367 MB | NPU | |
| | detector | ONNX | mixed_with_float | Qualcomm® Dragonwing™ IQ-9075 | 15.669 ms | 6 - 14 MB | NPU | |
| | detector | ONNX | mixed_with_float | Qualcomm® Dragonwing™ IQ-X7181 | 16.328 ms | 34 - 34 MB | NPU | |
| | detector | ONNX | mixed_with_float | Qualcomm® Dragonwing™ Q-8750 | 9.35 ms | 7 - 281 MB | NPU | |
| | detector | QNN_DLC | float | Snapdragon® 8 Elite Gen 5 For Galaxy Mobile | 8.071 ms | 5 - 255 MB | NPU | |
| | detector | QNN_DLC | float | Snapdragon® 8 Elite For Galaxy Mobile | 11.174 ms | 0 - 248 MB | NPU | |
| | detector | QNN_DLC | float | Snapdragon® 8 Gen 3 Mobile | 15.635 ms | 3 - 327 MB | NPU | |
| | detector | QNN_DLC | float | Snapdragon® 8 Gen 1 Mobile | 76.316 ms | 4 - 321 MB | NPU | |
| | detector | QNN_DLC | float | Qualcomm® Dragonwing™ IQ-8275 | 34.697 ms | 5 - 13 MB | NPU | |
| | detector | QNN_DLC | float | Qualcomm® Dragonwing™ QCS8550 (Proxy) | 19.67 ms | 5 - 7 MB | NPU | |
| | detector | QNN_DLC | float | Qualcomm® SA8650P | 30.339 ms | 3 - 297 MB | NPU | |
| | detector | QNN_DLC | float | Qualcomm® SA8255P | 30.339 ms | 3 - 297 MB | NPU | |
| | detector | QNN_DLC | float | Qualcomm® QCS8450 | 76.316 ms | 4 - 321 MB | NPU | |
| | detector | QNN_DLC | float | Qualcomm® Dragonwing™ IQ-9075 | 31.478 ms | 5 - 12 MB | NPU | |
| | detector | QNN_DLC | float | Qualcomm® Dragonwing™ Q-8750 | 11.174 ms | 0 - 248 MB | NPU | |
| | detector | QNN_DLC | float | Qualcomm® SA8295P | 55.874 ms | 0 - 246 MB | NPU | |
| | detector | QNN_DLC | mixed_with_float | Snapdragon® 8 Elite Gen 5 For Galaxy Mobile | 7.237 ms | 5 - 271 MB | NPU | |
| | detector | QNN_DLC | mixed_with_float | Snapdragon® 8 Elite For Galaxy Mobile | 9.124 ms | 0 - 260 MB | NPU | |
| | detector | QNN_DLC | mixed_with_float | Snapdragon® 8 Gen 3 Mobile | 10.734 ms | 5 - 346 MB | NPU | |
| | detector | QNN_DLC | mixed_with_float | Qualcomm® Dragonwing™ IQ-8275 | 15.191 ms | 5 - 13 MB | NPU | |
| | detector | QNN_DLC | mixed_with_float | Qualcomm® Dragonwing™ QCS8550 (Proxy) | 15.285 ms | 5 - 11 MB | NPU | |
| | detector | QNN_DLC | mixed_with_float | Qualcomm® SA8650P | 18.211 ms | 1 - 307 MB | NPU | |
| | detector | QNN_DLC | mixed_with_float | Qualcomm® SA8255P | 18.211 ms | 1 - 307 MB | NPU | |
| | detector | QNN_DLC | mixed_with_float | Qualcomm® Dragonwing™ IQ-9075 | 15.97 ms | 7 - 14 MB | NPU | |
| | detector | QNN_DLC | mixed_with_float | Qualcomm® Dragonwing™ Q-8750 | 9.124 ms | 0 - 260 MB | NPU | |
| | detector | TFLITE | float | Snapdragon® 8 Elite Gen 5 For Galaxy Mobile | 9.305 ms | 0 - 256 MB | NPU | |
| | detector | TFLITE | float | Snapdragon® 8 Elite For Galaxy Mobile | 11.593 ms | 0 - 251 MB | NPU | |
| | detector | TFLITE | float | Snapdragon® 8 Gen 3 Mobile | 16.661 ms | 0 - 335 MB | NPU | |
| | detector | TFLITE | float | Snapdragon® 8 Gen 1 Mobile | 76.469 ms | 1 - 335 MB | NPU | |
| | detector | TFLITE | float | Qualcomm® Dragonwing™ IQ-8275 | 36.514 ms | 0 - 84 MB | NPU | |
| | detector | TFLITE | float | Qualcomm® Dragonwing™ QCS8550 (Proxy) | 20.687 ms | 0 - 8 MB | NPU | |
| | detector | TFLITE | float | Qualcomm® SA8650P | 31.601 ms | 0 - 267 MB | NPU | |
| | detector | TFLITE | float | Qualcomm® SA8255P | 31.601 ms | 0 - 267 MB | NPU | |
| | detector | TFLITE | float | Qualcomm® QCS8450 | 76.469 ms | 1 - 335 MB | NPU | |
| | detector | TFLITE | float | Qualcomm® Dragonwing™ IQ-9075 | 30.763 ms | 0 - 83 MB | NPU | |
| | detector | TFLITE | float | Qualcomm® Dragonwing™ Q-8750 | 11.593 ms | 0 - 251 MB | NPU | |
| | detector | TFLITE | float | Qualcomm® SA8295P | 57.059 ms | 0 - 254 MB | NPU | |
| | text_encoder | ONNX | float | Snapdragon® 8 Elite Gen 5 For Galaxy Mobile | 31.029 ms | 1 - 2176 MB | NPU | |
| | text_encoder | ONNX | float | Snapdragon® 8 Elite For Galaxy Mobile | 44.924 ms | 1 - 1981 MB | NPU | |
| | text_encoder | ONNX | float | Snapdragon® X2 Elite | 33.125 ms | 1 - 1 MB | NPU | |
| | text_encoder | ONNX | float | Snapdragon® X Elite | 83.669 ms | 551 - 551 MB | NPU | |
| | text_encoder | ONNX | float | Snapdragon® 8 Gen 3 Mobile | 60.151 ms | 1 - 3626 MB | NPU | |
| | text_encoder | ONNX | float | Snapdragon® 8 Gen 1 Mobile | 110.729 ms | 2 - 1814 MB | NPU | |
| | text_encoder | ONNX | float | Qualcomm® Dragonwing™ IQ-8275 | 92.427 ms | 0 - 4 MB | NPU | |
| | text_encoder | ONNX | float | Qualcomm® Dragonwing™ QCS8550 (Proxy) | 80.593 ms | 0 - 644 MB | NPU | |
| | text_encoder | ONNX | float | Qualcomm® QCS8450 | 110.729 ms | 2 - 1814 MB | NPU | |
| | text_encoder | ONNX | float | Qualcomm® Dragonwing™ IQ-9075 | 90.841 ms | 1 - 4 MB | NPU | |
| | text_encoder | ONNX | float | Qualcomm® Dragonwing™ IQ-X7181 | 83.669 ms | 551 - 551 MB | NPU | |
| | text_encoder | ONNX | float | Qualcomm® Dragonwing™ Q-8750 | 44.924 ms | 1 - 1981 MB | NPU | |
| | text_encoder | ONNX | mixed_with_float | Snapdragon® 8 Elite Gen 5 For Galaxy Mobile | 31.158 ms | 1 - 2175 MB | NPU | |
| | text_encoder | ONNX | mixed_with_float | Snapdragon® 8 Elite For Galaxy Mobile | 44.333 ms | 1 - 1980 MB | NPU | |
| | text_encoder | ONNX | mixed_with_float | Snapdragon® X2 Elite | 33.202 ms | 1 - 1 MB | NPU | |
| | text_encoder | ONNX | mixed_with_float | Snapdragon® X Elite | 83.512 ms | 550 - 550 MB | NPU | |
| | text_encoder | ONNX | mixed_with_float | Snapdragon® 8 Gen 3 Mobile | 60.158 ms | 1 - 3626 MB | NPU | |
| | text_encoder | ONNX | mixed_with_float | Snapdragon® 8 Gen 1 Mobile | 111.209 ms | 2 - 1816 MB | NPU | |
| | text_encoder | ONNX | mixed_with_float | Qualcomm® Dragonwing™ IQ-8275 | 92.349 ms | 0 - 4 MB | NPU | |
| | text_encoder | ONNX | mixed_with_float | Qualcomm® Dragonwing™ QCS8550 (Proxy) | 80.847 ms | 0 - 644 MB | NPU | |
| | text_encoder | ONNX | mixed_with_float | Qualcomm® QCS8450 | 111.209 ms | 2 - 1816 MB | NPU | |
| | text_encoder | ONNX | mixed_with_float | Qualcomm® Dragonwing™ IQ-9075 | 90.752 ms | 1 - 4 MB | NPU | |
| | text_encoder | ONNX | mixed_with_float | Qualcomm® Dragonwing™ IQ-X7181 | 83.512 ms | 550 - 550 MB | NPU | |
| | text_encoder | ONNX | mixed_with_float | Qualcomm® Dragonwing™ Q-8750 | 44.333 ms | 1 - 1980 MB | NPU | |
| | text_encoder | QNN_DLC | float | Snapdragon® 8 Elite Gen 5 For Galaxy Mobile | 30.549 ms | 0 - 1923 MB | NPU | |
| | text_encoder | QNN_DLC | float | Snapdragon® 8 Elite For Galaxy Mobile | 43.419 ms | 0 - 2052 MB | NPU | |
| | text_encoder | QNN_DLC | float | Snapdragon® 8 Gen 3 Mobile | 55.821 ms | 0 - 2293 MB | NPU | |
| | text_encoder | QNN_DLC | float | Snapdragon® 8 Gen 1 Mobile | 106.955 ms | 0 - 1823 MB | NPU | |
| | text_encoder | QNN_DLC | float | Qualcomm® Dragonwing™ IQ-8275 | 89.81 ms | 0 - 3 MB | NPU | |
| | text_encoder | QNN_DLC | float | Qualcomm® Dragonwing™ QCS8550 (Proxy) | 77.06 ms | 0 - 2 MB | NPU | |
| | text_encoder | QNN_DLC | float | Qualcomm® SA8650P | 95.659 ms | 1 - 2019 MB | NPU | |
| | text_encoder | QNN_DLC | float | Qualcomm® SA8255P | 95.659 ms | 1 - 2019 MB | NPU | |
| | text_encoder | QNN_DLC | float | Qualcomm® QCS8450 | 106.955 ms | 0 - 1823 MB | NPU | |
| | text_encoder | QNN_DLC | float | Qualcomm® Dragonwing™ IQ-9075 | 86.62 ms | 0 - 2 MB | NPU | |
| | text_encoder | QNN_DLC | float | Qualcomm® Dragonwing™ Q-8750 | 43.419 ms | 0 - 2052 MB | NPU | |
| | text_encoder | QNN_DLC | float | Qualcomm® SA8295P | 102.822 ms | 0 - 1699 MB | NPU | |
| | text_encoder | QNN_DLC | mixed_with_float | Snapdragon® 8 Elite Gen 5 For Galaxy Mobile | 30.66 ms | 0 - 1922 MB | NPU | |
| | text_encoder | QNN_DLC | mixed_with_float | Snapdragon® 8 Elite For Galaxy Mobile | 43.864 ms | 0 - 2050 MB | NPU | |
| | text_encoder | QNN_DLC | mixed_with_float | Snapdragon® 8 Gen 3 Mobile | 58.135 ms | 9 - 2303 MB | NPU | |
| | text_encoder | QNN_DLC | mixed_with_float | Qualcomm® Dragonwing™ IQ-8275 | 90.055 ms | 0 - 3 MB | NPU | |
| | text_encoder | QNN_DLC | mixed_with_float | Qualcomm® Dragonwing™ QCS8550 (Proxy) | 77.593 ms | 0 - 3 MB | NPU | |
| | text_encoder | QNN_DLC | mixed_with_float | Qualcomm® SA8650P | 95.579 ms | 1 - 2019 MB | NPU | |
| | text_encoder | QNN_DLC | mixed_with_float | Qualcomm® SA8255P | 95.579 ms | 1 - 2019 MB | NPU | |
| | text_encoder | QNN_DLC | mixed_with_float | Qualcomm® Dragonwing™ IQ-9075 | 87.716 ms | 2 - 4 MB | NPU | |
| | text_encoder | QNN_DLC | mixed_with_float | Qualcomm® Dragonwing™ Q-8750 | 43.864 ms | 0 - 2050 MB | NPU | |
| | text_encoder | TFLITE | float | Snapdragon® 8 Elite Gen 5 For Galaxy Mobile | 33.01 ms | 0 - 1764 MB | NPU | |
| | text_encoder | TFLITE | float | Snapdragon® 8 Elite For Galaxy Mobile | 50.301 ms | 0 - 1731 MB | NPU | |
| | text_encoder | TFLITE | float | Snapdragon® 8 Gen 3 Mobile | 63.551 ms | 0 - 1978 MB | NPU | |
| | text_encoder | TFLITE | float | Snapdragon® 8 Gen 1 Mobile | 112.967 ms | 1 - 1865 MB | NPU | |
| | text_encoder | TFLITE | float | Qualcomm® Dragonwing™ IQ-8275 | 95.653 ms | 0 - 553 MB | NPU | |
| | text_encoder | TFLITE | float | Qualcomm® Dragonwing™ QCS8550 (Proxy) | 84.951 ms | 0 - 4 MB | NPU | |
| | text_encoder | TFLITE | float | Qualcomm® SA8650P | 111.23 ms | 1 - 2145 MB | NPU | |
| | text_encoder | TFLITE | float | Qualcomm® SA8255P | 111.23 ms | 1 - 2145 MB | NPU | |
| | text_encoder | TFLITE | float | Qualcomm® QCS8450 | 112.967 ms | 1 - 1865 MB | NPU | |
| | text_encoder | TFLITE | float | Qualcomm® Dragonwing™ IQ-9075 | 93.836 ms | 0 - 552 MB | NPU | |
| | text_encoder | TFLITE | float | Qualcomm® Dragonwing™ Q-8750 | 50.301 ms | 0 - 1731 MB | NPU | |
| | text_encoder | TFLITE | float | Qualcomm® SA8295P | 111.099 ms | 1 - 1736 MB | NPU | |
| ## License | |
| * The license for the original implementation of WeDetect can be found | |
| [here](https://github.com/WeChatCV/WeDetect/blob/main/README.md). | |
| ## References | |
| * [WeDetect: Fast Open-Vocabulary Object Detection as Retrieval](https://arxiv.org/abs/2512.12309) | |
| * [Source Model Implementation](https://github.com/WeChatCV/WeDetect) | |
| ## Community | |
| * Join [our AI Hub Slack community](https://aihub.qualcomm.com/community/slack) to collaborate, post questions and learn more about on-device AI. | |
| * For questions or feedback please [reach out to us](mailto:ai-hub-support@qti.qualcomm.com). | |