qaihm-bot commited on
Commit
fa80635
·
verified ·
1 Parent(s): 8254a87

See https://github.com/qualcomm/ai-hub-models/releases/v0.65.0 for changelog.

Files changed (3) hide show
  1. LICENSE +1 -0
  2. README.md +95 -0
  3. release_assets.json +22 -0
LICENSE ADDED
@@ -0,0 +1 @@
 
 
1
+ The license of the original trained model can be found at https://github.com/facebookresearch/vjepa2/blob/main/LICENSE.
README.md ADDED
@@ -0,0 +1,95 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ library_name: pytorch
3
+ license: mit
4
+ tags:
5
+ - backbone
6
+ - android
7
+ pipeline_tag: video-classification
8
+
9
+ ---
10
+
11
+ ![](https://qaihub-public-assets.s3.us-west-2.amazonaws.com/qai-hub-models/models/vjepa2/web-assets/model_demo.png)
12
+
13
+ # V-JEPA-2: Optimized for Qualcomm Devices
14
+
15
+ V-JEPA 2 (Video Joint-Embedding Predictive Architecture 2) is a self-supervised video encoder from Meta AI that uses a ViT (Vision Transformer) backbone to produce per-token video embeddings. It is used as a frozen backbone for downstream video understanding tasks such as action recognition and video retrieval.
16
+
17
+ This is based on the implementation of V-JEPA-2 found [here](https://github.com/facebookresearch/vjepa2).
18
+ This repository contains pre-exported model files optimized for Qualcomm® devices. You can use the [Qualcomm® AI Hub Models](https://github.com/qualcomm/ai-hub-models/blob/v0.65.0/src/qai_hub_models/models/vjepa2) library to export with custom configurations. More details on model performance across various devices, can be found [here](#performance-summary).
19
+
20
+ Qualcomm AI Hub Models uses [Qualcomm AI Hub Workbench](https://workbench.aihub.qualcomm.com) to compile, profile, and evaluate this model. [Sign up](https://myaccount.qualcomm.com/signup) to run these models on a hosted Qualcomm® device.
21
+
22
+ ## Getting Started
23
+ There are two ways to deploy this model on your device:
24
+
25
+ ### Option 1: Download Pre-Exported Models
26
+
27
+ Below are pre-exported model assets ready for deployment.
28
+
29
+ | Runtime | Precision | Chipset | SDK Versions | Download |
30
+ |---|---|---|---|---|
31
+ | ONNX | float | Universal | QAIRT 2.50, ONNX Runtime 1.30.0 | [Download](https://qaihub-public-assets.s3.us-west-2.amazonaws.com/qai-hub-models/models/vjepa2/releases/v0.65.0/vjepa2-onnx-float.zip)
32
+ | QNN_DLC | float | Universal | QAIRT 2.50 | [Download](https://qaihub-public-assets.s3.us-west-2.amazonaws.com/qai-hub-models/models/vjepa2/releases/v0.65.0/vjepa2-qnn_dlc-float.zip)
33
+
34
+ For more device-specific assets and performance metrics, visit **[V-JEPA-2 on Qualcomm® AI Hub](https://aihub.qualcomm.com/models/vjepa2)**.
35
+
36
+
37
+ ### Option 2: Export with Custom Configurations
38
+
39
+ Use the [Qualcomm® AI Hub Models](https://github.com/qualcomm/ai-hub-models/blob/v0.65.0/src/qai_hub_models/models/vjepa2) Python library to compile and export the model with your own:
40
+ - Custom weights (e.g., fine-tuned checkpoints)
41
+ - Custom input shapes
42
+ - Target device and runtime configurations
43
+
44
+ This option is ideal if you need to customize the model beyond the default configuration provided here.
45
+
46
+ See our repository for [V-JEPA-2 on GitHub](https://github.com/qualcomm/ai-hub-models/blob/v0.65.0/src/qai_hub_models/models/vjepa2) for usage instructions.
47
+
48
+ ## Model Details
49
+
50
+ **Model Type:** Model_use_case.video_classification
51
+
52
+ **Model Stats:**
53
+ - Input resolution: 256x256
54
+ - Model checkpoint: facebook/vjepa2-vitl-fpc64-256
55
+ - Model size (float): 1.13 GB
56
+ - Model size (w8a16): 304 MB
57
+ - Number of frames: 8
58
+ - Number of parameters: 304M
59
+
60
+ ## Performance Summary
61
+ | Model | Runtime | Precision | Chipset | Inference Time (ms) | Peak Memory Range (MB) | Primary Compute Unit
62
+ |---|---|---|---|---|---|---
63
+ | V-JEPA-2 | ONNX | float | Snapdragon® 8 Elite Gen 5 For Galaxy Mobile | 196.793 ms | 8 - 2806 MB | NPU
64
+ | V-JEPA-2 | ONNX | float | Snapdragon® 8 Elite For Galaxy Mobile | 259.387 ms | 9 - 2587 MB | NPU
65
+ | V-JEPA-2 | ONNX | float | Snapdragon® X2 Elite | 187.712 ms | 7 - 7 MB | NPU
66
+ | V-JEPA-2 | ONNX | float | Snapdragon® X Elite | 431.083 ms | 601 - 601 MB | NPU
67
+ | V-JEPA-2 | ONNX | float | Qualcomm® Dragonwing™ IQ-8275 | 463.704 ms | 6 - 16 MB | NPU
68
+ | V-JEPA-2 | ONNX | float | Qualcomm® Dragonwing™ QCS8550 (Proxy) | 431.137 ms | 0 - 637 MB | NPU
69
+ | V-JEPA-2 | ONNX | float | Qualcomm® Dragonwing™ IQ-9075 | 470.815 ms | 6 - 15 MB | NPU
70
+ | V-JEPA-2 | ONNX | float | Qualcomm® Dragonwing™ IQ-X7181 | 431.083 ms | 601 - 601 MB | NPU
71
+ | V-JEPA-2 | ONNX | float | Qualcomm® Dragonwing™ Q-8750 | 259.387 ms | 9 - 2587 MB | NPU
72
+ | V-JEPA-2 | QNN_DLC | float | Snapdragon® 8 Elite For Galaxy Mobile | 219.491 ms | 0 - 2608 MB | NPU
73
+ | V-JEPA-2 | QNN_DLC | float | Snapdragon® X2 Elite | 167.039 ms | 6 - 6 MB | NPU
74
+ | V-JEPA-2 | QNN_DLC | float | Snapdragon® X Elite | 396.705 ms | 6 - 6 MB | NPU
75
+ | V-JEPA-2 | QNN_DLC | float | Snapdragon® 8 Gen 1 Mobile | 584.698 ms | 6 - 3557 MB | NPU
76
+ | V-JEPA-2 | QNN_DLC | float | Qualcomm® Dragonwing™ IQ-8275 | 435.924 ms | 6 - 19 MB | NPU
77
+ | V-JEPA-2 | QNN_DLC | float | Qualcomm® Dragonwing™ QCS8550 (Proxy) | 392.929 ms | 6 - 1221 MB | NPU
78
+ | V-JEPA-2 | QNN_DLC | float | Qualcomm® QCS8450 | 584.698 ms | 6 - 3557 MB | NPU
79
+ | V-JEPA-2 | QNN_DLC | float | Qualcomm® Dragonwing™ IQ-9075 | 431.071 ms | 6 - 18 MB | NPU
80
+ | V-JEPA-2 | QNN_DLC | float | Qualcomm® Dragonwing™ IQ-X7181 | 396.705 ms | 6 - 6 MB | NPU
81
+ | V-JEPA-2 | QNN_DLC | float | Qualcomm® Dragonwing™ Q-8750 | 219.491 ms | 0 - 2608 MB | NPU
82
+ | V-JEPA-2 | QNN_DLC | float | Qualcomm® SA8295P | 533.823 ms | 4 - 2167 MB | NPU
83
+ | V-JEPA-2 | QNN_DLC | float | Snapdragon® 8 Elite Gen 6 Mobile | 212.722 ms | 6 - 2744 MB | NPU
84
+
85
+ ## License
86
+ * The license for the original implementation of V-JEPA-2 can be found
87
+ [here](https://github.com/facebookresearch/vjepa2/blob/main/LICENSE).
88
+
89
+ ## References
90
+ * [V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and Planning](https://arxiv.org/abs/2506.09985)
91
+ * [Source Model Implementation](https://github.com/facebookresearch/vjepa2)
92
+
93
+ ## Community
94
+ * Join [our AI Hub Slack community](https://aihub.qualcomm.com/community/slack) to collaborate, post questions and learn more about on-device AI.
95
+ * For questions or feedback please [reach out to us](mailto:ai-hub-support@qti.qualcomm.com).
release_assets.json ADDED
@@ -0,0 +1,22 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "version": "0.65.0",
3
+ "precisions": {
4
+ "float": {
5
+ "universal_assets": {
6
+ "onnx": {
7
+ "tool_versions": {
8
+ "qairt": "2.50.0.260828221209",
9
+ "onnx_runtime": "1.30.0"
10
+ },
11
+ "download_url": "https://qaihub-public-assets.s3.us-west-2.amazonaws.com/qai-hub-models/models/vjepa2/releases/v0.65.0/vjepa2-onnx-float.zip"
12
+ },
13
+ "qnn_dlc": {
14
+ "tool_versions": {
15
+ "qairt": "2.50.0.260828221209"
16
+ },
17
+ "download_url": "https://qaihub-public-assets.s3.us-west-2.amazonaws.com/qai-hub-models/models/vjepa2/releases/v0.65.0/vjepa2-qnn_dlc-float.zip"
18
+ }
19
+ }
20
+ }
21
+ }
22
+ }