qaihm-bot commited on
Commit
7958bb5
·
verified ·
1 Parent(s): 445a46c

See https://github.com/qualcomm/ai-hub-models/releases/v0.65.0 for changelog.

Files changed (3) hide show
  1. LICENSE +1 -0
  2. README.md +172 -0
  3. release_assets.json +53 -0
LICENSE ADDED
@@ -0,0 +1 @@
 
 
1
+ The license of the original trained model can be found at https://github.com/rpautrat/SuperPoint/blob/master/LICENSE.txt.
README.md ADDED
@@ -0,0 +1,172 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ library_name: pytorch
3
+ license: mit
4
+ tags:
5
+ - real_time
6
+ - android
7
+ pipeline_tag: keypoint-detection
8
+
9
+ ---
10
+
11
+ ![](https://qaihub-public-assets.s3.us-west-2.amazonaws.com/qai-hub-models/models/superpoint/web-assets/model_demo.png)
12
+
13
+ # SuperPoint: Optimized for Qualcomm Devices
14
+
15
+ SuperPoint is a self-supervised framework for detecting interest points and computing their descriptors in a single forward pass. It uses a VGG-style encoder shared between a keypoint detector head and a descriptor head, trained via homographic adaptation on synthetic shapes and real images. The model outputs up to 500 keypoints per image with 256-dimensional L2-normalised descriptors, suitable for feature matching, visual localisation, and SLAM.
16
+
17
+ This is based on the implementation of SuperPoint found [here](https://github.com/rpautrat/SuperPoint).
18
+ This repository contains pre-exported model files optimized for Qualcomm® devices. You can use the [Qualcomm® AI Hub Models](https://github.com/qualcomm/ai-hub-models/blob/v0.65.0/src/qai_hub_models/models/superpoint) library to export with custom configurations. More details on model performance across various devices, can be found [here](#performance-summary).
19
+
20
+ Qualcomm AI Hub Models uses [Qualcomm AI Hub Workbench](https://workbench.aihub.qualcomm.com) to compile, profile, and evaluate this model. [Sign up](https://myaccount.qualcomm.com/signup) to run these models on a hosted Qualcomm® device.
21
+
22
+ ## Getting Started
23
+ There are two ways to deploy this model on your device:
24
+
25
+ ### Option 1: Download Pre-Exported Models
26
+
27
+ Below are pre-exported model assets ready for deployment.
28
+
29
+ | Runtime | Precision | Chipset | SDK Versions | Download |
30
+ |---|---|---|---|---|
31
+ | ONNX | float | Universal | QAIRT 2.50, ONNX Runtime 1.30.0 | [Download](https://qaihub-public-assets.s3.us-west-2.amazonaws.com/qai-hub-models/models/superpoint/releases/v0.65.0/superpoint-onnx-float.zip)
32
+ | ONNX | w8a8 | Universal | QAIRT 2.50, ONNX Runtime 1.30.0 | [Download](https://qaihub-public-assets.s3.us-west-2.amazonaws.com/qai-hub-models/models/superpoint/releases/v0.65.0/superpoint-onnx-w8a8.zip)
33
+ | QNN_DLC | float | Universal | QAIRT 2.50 | [Download](https://qaihub-public-assets.s3.us-west-2.amazonaws.com/qai-hub-models/models/superpoint/releases/v0.65.0/superpoint-qnn_dlc-float.zip)
34
+ | QNN_DLC | w8a8 | Universal | QAIRT 2.50 | [Download](https://qaihub-public-assets.s3.us-west-2.amazonaws.com/qai-hub-models/models/superpoint/releases/v0.65.0/superpoint-qnn_dlc-w8a8.zip)
35
+ | TFLITE | float | Universal | QAIRT 2.50 | [Download](https://qaihub-public-assets.s3.us-west-2.amazonaws.com/qai-hub-models/models/superpoint/releases/v0.65.0/superpoint-tflite-float.zip)
36
+ | TFLITE | w8a8 | Universal | QAIRT 2.50 | [Download](https://qaihub-public-assets.s3.us-west-2.amazonaws.com/qai-hub-models/models/superpoint/releases/v0.65.0/superpoint-tflite-w8a8.zip)
37
+
38
+ For more device-specific assets and performance metrics, visit **[SuperPoint on Qualcomm® AI Hub](https://aihub.qualcomm.com/models/superpoint)**.
39
+
40
+
41
+ ### Option 2: Export with Custom Configurations
42
+
43
+ Use the [Qualcomm® AI Hub Models](https://github.com/qualcomm/ai-hub-models/blob/v0.65.0/src/qai_hub_models/models/superpoint) Python library to compile and export the model with your own:
44
+ - Custom weights (e.g., fine-tuned checkpoints)
45
+ - Custom input shapes
46
+ - Target device and runtime configurations
47
+
48
+ This option is ideal if you need to customize the model beyond the default configuration provided here.
49
+
50
+ See our repository for [SuperPoint on GitHub](https://github.com/qualcomm/ai-hub-models/blob/v0.65.0/src/qai_hub_models/models/superpoint) for usage instructions.
51
+
52
+ ## Model Details
53
+
54
+ **Model Type:** Model_use_case.pose_estimation
55
+
56
+ **Model Stats:**
57
+ - Descriptor dimension: 256
58
+ - Input resolution: 480x640
59
+ - Model checkpoint: superpoint_v6_from_tf.pth
60
+ - Model size (float): 19.6 MB
61
+ - Model size (w8a8): 3.22 MB
62
+ - Number of parameters: 5.12M
63
+ - Output keypoints (max): 500
64
+
65
+ ## Performance Summary
66
+ | Model | Runtime | Precision | Chipset | Inference Time (ms) | Peak Memory Range (MB) | Primary Compute Unit
67
+ |---|---|---|---|---|---|---
68
+ | SuperPoint | ONNX | float | Snapdragon® 8 Elite Gen 5 For Galaxy Mobile | 6.878 ms | 28 - 196 MB | NPU
69
+ | SuperPoint | ONNX | float | Snapdragon® 8 Elite For Galaxy Mobile | 9.69 ms | 27 - 196 MB | NPU
70
+ | SuperPoint | ONNX | float | Snapdragon® X2 Elite | 7.545 ms | 40 - 40 MB | NPU
71
+ | SuperPoint | ONNX | float | Snapdragon® X Elite | 17.115 ms | 40 - 40 MB | NPU
72
+ | SuperPoint | ONNX | float | Snapdragon® 8 Gen 1 Mobile | 28.558 ms | 19 - 259 MB | NPU
73
+ | SuperPoint | ONNX | float | Qualcomm® Dragonwing™ IQ-8275 | 28.595 ms | 7 - 18 MB | NPU
74
+ | SuperPoint | ONNX | float | Qualcomm® Dragonwing™ QCS8550 (Proxy) | 16.91 ms | 23 - 29 MB | NPU
75
+ | SuperPoint | ONNX | float | Qualcomm® QCS8450 | 28.558 ms | 19 - 259 MB | NPU
76
+ | SuperPoint | ONNX | float | Qualcomm® Dragonwing™ IQ-X7181 | 17.115 ms | 40 - 40 MB | NPU
77
+ | SuperPoint | ONNX | float | Qualcomm® Dragonwing™ Q-8750 | 9.69 ms | 27 - 196 MB | NPU
78
+ | SuperPoint | ONNX | float | Snapdragon® 8 Elite Extreme Gen 6 Mobile | 5.498 ms | 34 - 202 MB | NPU
79
+ | SuperPoint | ONNX | float | Snapdragon® 8 Elite Gen 6 Mobile | 6.594 ms | 33 - 201 MB | NPU
80
+ | SuperPoint | ONNX | w8a8 | Snapdragon® 8 Elite Gen 5 For Galaxy Mobile | 6.516 ms | 6 - 132 MB | NPU
81
+ | SuperPoint | ONNX | w8a8 | Snapdragon® 8 Elite For Galaxy Mobile | 8.295 ms | 8 - 123 MB | NPU
82
+ | SuperPoint | ONNX | w8a8 | Snapdragon® X2 Elite | 6.783 ms | 8 - 8 MB | NPU
83
+ | SuperPoint | ONNX | w8a8 | Snapdragon® X Elite | 20.927 ms | 7 - 7 MB | NPU
84
+ | SuperPoint | ONNX | w8a8 | Snapdragon® 8 Gen 1 Mobile | 24.54 ms | 8 - 161 MB | NPU
85
+ | SuperPoint | ONNX | w8a8 | Qualcomm® Dragonwing™ IQ-8275 | 19.575 ms | 7 - 11 MB | NPU
86
+ | SuperPoint | ONNX | w8a8 | Qualcomm® Dragonwing™ QCS8550 (Proxy) | 19.988 ms | 7 - 167 MB | NPU
87
+ | SuperPoint | ONNX | w8a8 | Qualcomm® QCS8450 | 24.54 ms | 8 - 161 MB | NPU
88
+ | SuperPoint | ONNX | w8a8 | Qualcomm® Dragonwing™ IQ-9075 | 20.756 ms | 7 - 10 MB | NPU
89
+ | SuperPoint | ONNX | w8a8 | Qualcomm® Dragonwing™ IQ-X7181 | 20.927 ms | 7 - 7 MB | NPU
90
+ | SuperPoint | ONNX | w8a8 | Qualcomm® Dragonwing™ Q-8750 | 8.295 ms | 8 - 123 MB | NPU
91
+ | SuperPoint | ONNX | w8a8 | Snapdragon® 8 Elite Extreme Gen 6 Mobile | 5.39 ms | 8 - 137 MB | NPU
92
+ | SuperPoint | ONNX | w8a8 | Snapdragon® 8 Elite Gen 6 Mobile | 10.191 ms | 8 - 119 MB | NPU
93
+ | SuperPoint | QNN_DLC | float | Snapdragon® 8 Elite Gen 5 For Galaxy Mobile | 6.767 ms | 4 - 168 MB | NPU
94
+ | SuperPoint | QNN_DLC | float | Snapdragon® 8 Elite For Galaxy Mobile | 9.605 ms | 4 - 163 MB | NPU
95
+ | SuperPoint | QNN_DLC | float | Snapdragon® X2 Elite | 7.936 ms | 7 - 7 MB | NPU
96
+ | SuperPoint | QNN_DLC | float | Snapdragon® X Elite | 16.54 ms | 7 - 7 MB | NPU
97
+ | SuperPoint | QNN_DLC | float | Snapdragon® 8 Gen 1 Mobile | 30.848 ms | 4 - 231 MB | NPU
98
+ | SuperPoint | QNN_DLC | float | Qualcomm® Dragonwing™ IQ-8275 | 27.41 ms | 7 - 29 MB | NPU
99
+ | SuperPoint | QNN_DLC | float | Qualcomm® Dragonwing™ QCS8550 (Proxy) | 16.13 ms | 7 - 9 MB | NPU
100
+ | SuperPoint | QNN_DLC | float | Qualcomm® SA8775P | 23.662 ms | 0 - 160 MB | NPU
101
+ | SuperPoint | QNN_DLC | float | Qualcomm® SA8650P | 23.662 ms | 0 - 160 MB | NPU
102
+ | SuperPoint | QNN_DLC | float | Qualcomm® SA8255P | 23.662 ms | 0 - 160 MB | NPU
103
+ | SuperPoint | QNN_DLC | float | Qualcomm® QCS8450 | 30.848 ms | 4 - 231 MB | NPU
104
+ | SuperPoint | QNN_DLC | float | Qualcomm® Dragonwing™ IQ-9075 | 24.532 ms | 7 - 28 MB | NPU
105
+ | SuperPoint | QNN_DLC | float | Qualcomm® Dragonwing™ IQ-X7181 | 16.54 ms | 7 - 7 MB | NPU
106
+ | SuperPoint | QNN_DLC | float | Qualcomm® Dragonwing™ Q-8750 | 9.605 ms | 4 - 163 MB | NPU
107
+ | SuperPoint | QNN_DLC | float | Qualcomm® SA8295P | 31.192 ms | 0 - 175 MB | NPU
108
+ | SuperPoint | QNN_DLC | float | Snapdragon® 8 Elite Extreme Gen 6 Mobile | 6.152 ms | 4 - 168 MB | NPU
109
+ | SuperPoint | QNN_DLC | float | Snapdragon® 8 Elite Gen 6 Mobile | 6.594 ms | 7 - 173 MB | NPU
110
+ | SuperPoint | QNN_DLC | w8a8 | Snapdragon® 8 Elite Gen 5 For Galaxy Mobile | 6.353 ms | 2 - 124 MB | NPU
111
+ | SuperPoint | QNN_DLC | w8a8 | Snapdragon® 8 Elite For Galaxy Mobile | 7.928 ms | 1 - 114 MB | NPU
112
+ | SuperPoint | QNN_DLC | w8a8 | Snapdragon® X2 Elite | 6.994 ms | 2 - 2 MB | NPU
113
+ | SuperPoint | QNN_DLC | w8a8 | Snapdragon® X Elite | 20.752 ms | 2 - 2 MB | NPU
114
+ | SuperPoint | QNN_DLC | w8a8 | Snapdragon® 8 Gen 1 Mobile | 23.892 ms | 2 - 152 MB | NPU
115
+ | SuperPoint | QNN_DLC | w8a8 | Qualcomm® Dragonwing™ IQ-8275 | 18.848 ms | 1 - 9 MB | NPU
116
+ | SuperPoint | QNN_DLC | w8a8 | Qualcomm® Dragonwing™ QCS8550 (Proxy) | 19.597 ms | 2 - 4 MB | NPU
117
+ | SuperPoint | QNN_DLC | w8a8 | Qualcomm® SA8775P | 19.86 ms | 2 - 105 MB | NPU
118
+ | SuperPoint | QNN_DLC | w8a8 | Qualcomm® SA8650P | 19.86 ms | 2 - 105 MB | NPU
119
+ | SuperPoint | QNN_DLC | w8a8 | Qualcomm® SA8255P | 19.86 ms | 2 - 105 MB | NPU
120
+ | SuperPoint | QNN_DLC | w8a8 | Qualcomm® QCS8450 | 23.892 ms | 2 - 152 MB | NPU
121
+ | SuperPoint | QNN_DLC | w8a8 | Qualcomm® Dragonwing™ IQ-9075 | 20.248 ms | 3 - 10 MB | NPU
122
+ | SuperPoint | QNN_DLC | w8a8 | Qualcomm® Dragonwing™ IQ-X7181 | 20.752 ms | 2 - 2 MB | NPU
123
+ | SuperPoint | QNN_DLC | w8a8 | Qualcomm® Dragonwing™ Q-8750 | 7.928 ms | 1 - 114 MB | NPU
124
+ | SuperPoint | QNN_DLC | w8a8 | Qualcomm® SA8295P | 26.262 ms | 2 - 109 MB | NPU
125
+ | SuperPoint | QNN_DLC | w8a8 | Snapdragon® 8 Elite Extreme Gen 6 Mobile | 5.29 ms | 1 - 127 MB | NPU
126
+ | SuperPoint | QNN_DLC | w8a8 | Snapdragon® 8 Elite Gen 6 Mobile | 9.909 ms | 2 - 113 MB | NPU
127
+ | SuperPoint | TFLITE | float | Snapdragon® 8 Elite Gen 5 For Galaxy Mobile | 6.868 ms | 9 - 178 MB | NPU
128
+ | SuperPoint | TFLITE | float | Snapdragon® 8 Elite For Galaxy Mobile | 9.311 ms | 10 - 182 MB | NPU
129
+ | SuperPoint | TFLITE | float | Snapdragon® 8 Gen 1 Mobile | 30.481 ms | 12 - 247 MB | NPU
130
+ | SuperPoint | TFLITE | float | Qualcomm® Dragonwing™ IQ-8275 | 28.578 ms | 11 - 36 MB | NPU
131
+ | SuperPoint | TFLITE | float | Qualcomm® Dragonwing™ QCS8550 (Proxy) | 16.746 ms | 12 - 15 MB | NPU
132
+ | SuperPoint | TFLITE | float | Qualcomm® SA8775P | 24.709 ms | 12 - 176 MB | NPU
133
+ | SuperPoint | TFLITE | float | Qualcomm® SA8650P | 24.709 ms | 12 - 176 MB | NPU
134
+ | SuperPoint | TFLITE | float | Qualcomm® SA8255P | 24.709 ms | 12 - 176 MB | NPU
135
+ | SuperPoint | TFLITE | float | Qualcomm® QCS8450 | 30.481 ms | 12 - 247 MB | NPU
136
+ | SuperPoint | TFLITE | float | Qualcomm® Dragonwing™ IQ-9075 | 25.707 ms | 0 - 24 MB | NPU
137
+ | SuperPoint | TFLITE | float | Qualcomm® Dragonwing™ Q-8750 | 9.311 ms | 10 - 182 MB | NPU
138
+ | SuperPoint | TFLITE | float | Qualcomm® SA7255P | 86.98 ms | 12 - 175 MB | NPU
139
+ | SuperPoint | TFLITE | float | Qualcomm® SA8295P | 28.997 ms | 12 - 191 MB | NPU
140
+ | SuperPoint | TFLITE | float | Snapdragon® 8 Elite Extreme Gen 6 Mobile | 5.249 ms | 1 - 170 MB | NPU
141
+ | SuperPoint | TFLITE | float | Snapdragon® 8 Elite Gen 6 Mobile | 6.5 ms | 10 - 184 MB | NPU
142
+ | SuperPoint | TFLITE | w8a8 | Snapdragon® 8 Elite Gen 5 For Galaxy Mobile | 6.646 ms | 2 - 125 MB | NPU
143
+ | SuperPoint | TFLITE | w8a8 | Snapdragon® 8 Elite For Galaxy Mobile | 8.403 ms | 0 - 113 MB | NPU
144
+ | SuperPoint | TFLITE | w8a8 | Snapdragon® 8 Gen 1 Mobile | 24.822 ms | 3 - 154 MB | NPU
145
+ | SuperPoint | TFLITE | w8a8 | Qualcomm® Dragonwing™ QCS6490 | 580.219 ms | 299 - 686 MB | NPU
146
+ | SuperPoint | TFLITE | w8a8 | Qualcomm® Dragonwing™ IQ-8275 | 19.277 ms | 3 - 13 MB | NPU
147
+ | SuperPoint | TFLITE | w8a8 | Qualcomm® Dragonwing™ QCS8550 (Proxy) | 20.207 ms | 3 - 5 MB | NPU
148
+ | SuperPoint | TFLITE | w8a8 | Qualcomm® SA8775P | 20.427 ms | 3 - 105 MB | NPU
149
+ | SuperPoint | TFLITE | w8a8 | Qualcomm® SA8650P | 20.427 ms | 3 - 105 MB | NPU
150
+ | SuperPoint | TFLITE | w8a8 | Qualcomm® SA8255P | 20.427 ms | 3 - 105 MB | NPU
151
+ | SuperPoint | TFLITE | w8a8 | Qualcomm® QCS8450 | 24.822 ms | 3 - 154 MB | NPU
152
+ | SuperPoint | TFLITE | w8a8 | Qualcomm® Dragonwing™ IQ-9075 | 20.935 ms | 2 - 12 MB | NPU
153
+ | SuperPoint | TFLITE | w8a8 | Qualcomm® Dragonwing™ Q-6690 | 702.379 ms | 260 - 458 MB | NPU
154
+ | SuperPoint | TFLITE | w8a8 | Qualcomm® Dragonwing™ Q-7790 | 204.681 ms | 299 - 474 MB | NPU
155
+ | SuperPoint | TFLITE | w8a8 | Qualcomm® Dragonwing™ Q-8750 | 8.403 ms | 0 - 113 MB | NPU
156
+ | SuperPoint | TFLITE | w8a8 | Qualcomm® SA7255P | 39.934 ms | 3 - 104 MB | NPU
157
+ | SuperPoint | TFLITE | w8a8 | Qualcomm® SA8295P | 27.183 ms | 3 - 108 MB | NPU
158
+ | SuperPoint | TFLITE | w8a8 | Snapdragon® 7 Gen 4 Mobile | 204.681 ms | 299 - 474 MB | NPU
159
+ | SuperPoint | TFLITE | w8a8 | Snapdragon® 8 Elite Extreme Gen 6 Mobile | 5.45 ms | 2 - 128 MB | NPU
160
+ | SuperPoint | TFLITE | w8a8 | Snapdragon® 8 Elite Gen 6 Mobile | 10.277 ms | 2 - 114 MB | NPU
161
+
162
+ ## License
163
+ * The license for the original implementation of SuperPoint can be found
164
+ [here](https://github.com/rpautrat/SuperPoint/blob/master/LICENSE.txt).
165
+
166
+ ## References
167
+ * [SuperPoint: Self-Supervised Interest Point Detection and Description](https://arxiv.org/abs/1712.07629)
168
+ * [Source Model Implementation](https://github.com/rpautrat/SuperPoint)
169
+
170
+ ## Community
171
+ * Join [our AI Hub Slack community](https://aihub.qualcomm.com/community/slack) to collaborate, post questions and learn more about on-device AI.
172
+ * For questions or feedback please [reach out to us](mailto:ai-hub-support@qti.qualcomm.com).
release_assets.json ADDED
@@ -0,0 +1,53 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "version": "0.65.0",
3
+ "precisions": {
4
+ "float": {
5
+ "universal_assets": {
6
+ "onnx": {
7
+ "tool_versions": {
8
+ "qairt": "2.50.0.260828221209",
9
+ "onnx_runtime": "1.30.0"
10
+ },
11
+ "download_url": "https://qaihub-public-assets.s3.us-west-2.amazonaws.com/qai-hub-models/models/superpoint/releases/v0.65.0/superpoint-onnx-float.zip"
12
+ },
13
+ "qnn_dlc": {
14
+ "tool_versions": {
15
+ "qairt": "2.50.0.260828221209"
16
+ },
17
+ "download_url": "https://qaihub-public-assets.s3.us-west-2.amazonaws.com/qai-hub-models/models/superpoint/releases/v0.65.0/superpoint-qnn_dlc-float.zip"
18
+ },
19
+ "tflite": {
20
+ "tool_versions": {
21
+ "qairt": "2.50.0.260828221209",
22
+ "litert": "1.4.2"
23
+ },
24
+ "download_url": "https://qaihub-public-assets.s3.us-west-2.amazonaws.com/qai-hub-models/models/superpoint/releases/v0.65.0/superpoint-tflite-float.zip"
25
+ }
26
+ }
27
+ },
28
+ "w8a8": {
29
+ "universal_assets": {
30
+ "onnx": {
31
+ "tool_versions": {
32
+ "qairt": "2.50.0.260828221209",
33
+ "onnx_runtime": "1.30.0"
34
+ },
35
+ "download_url": "https://qaihub-public-assets.s3.us-west-2.amazonaws.com/qai-hub-models/models/superpoint/releases/v0.65.0/superpoint-onnx-w8a8.zip"
36
+ },
37
+ "qnn_dlc": {
38
+ "tool_versions": {
39
+ "qairt": "2.50.0.260828221209"
40
+ },
41
+ "download_url": "https://qaihub-public-assets.s3.us-west-2.amazonaws.com/qai-hub-models/models/superpoint/releases/v0.65.0/superpoint-qnn_dlc-w8a8.zip"
42
+ },
43
+ "tflite": {
44
+ "tool_versions": {
45
+ "qairt": "2.50.0.260828221209",
46
+ "litert": "1.4.2"
47
+ },
48
+ "download_url": "https://qaihub-public-assets.s3.us-west-2.amazonaws.com/qai-hub-models/models/superpoint/releases/v0.65.0/superpoint-tflite-w8a8.zip"
49
+ }
50
+ }
51
+ }
52
+ }
53
+ }