Image Feature Extraction
Transformers
Safetensors
motif_vision
feature-extraction
motif
vision-transformer
self-supervised
video
custom_code
Instructions to use Motif-Technologies/Motif-Vision-Encoder with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use Motif-Technologies/Motif-Vision-Encoder with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("image-feature-extraction", model="Motif-Technologies/Motif-Vision-Encoder", trust_remote_code=True)# Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("Motif-Technologies/Motif-Vision-Encoder", trust_remote_code=True, device_map="auto") - Notebooks
- Google Colab
- Kaggle
docs(README): add Web-DINO row; fix typo + stale KITTI line; tighten prose
Browse filesAdd Web-DINO 7B (2B MC images; DINOv3-report numbers: DAVIS 57.2/65.8/69.5, IN 85.9, ADE20K 42.7, K400 86.8, KITTI 3.2). Fix '~0.5B sample*' typo; drop now-stale 'KITTI still being evaluated' (KITTI filled); tighten summary + demo caption.
README.md
CHANGED
|
@@ -24,7 +24,7 @@ performance across image and video benchmarks β and leads on DAVIS video track
|
|
| 24 |
<p align="center">
|
| 25 |
<img src="assets/haaland_full_attn_blk20.gif" width="480" alt="Point tracking on a video clip: Motif vs V-JEPA 2.1"/>
|
| 26 |
</p>
|
| 27 |
-
<p align="center"><em>Point tracking on a video clip (top: Motif, bottom: V-JEPA 2.1) β a query point propagated across frames by patch-feature cosine similarity. Motif
|
| 28 |
|
| 29 |
- **Architecture**: ViT-7B (embed 4096 / depth 40 / heads 32), patch 16, 3D axial RoPE
|
| 30 |
(`base=100`), SwiGLU FFN, LayerScale, per-head QK-norm, gated attention, 4 register tokens.
|
|
@@ -106,6 +106,7 @@ single-resolution (S) figure is available.
|
|
| 106 |
|---|---|---|---|---|---|---|---|---|
|
| 107 |
| **Motif Vision Encoder** | 0.5B | **74.0** | **80.5** | **83.5** | 87.4 | <u>52.0</u> | 87.4 | 3.2 |
|
| 108 |
| DINOv3 | 1.7B | <u>71.1</u> | <u>79.7</u> | <u>83.3</u> | 88.4 | **55.9** | <u>87.8</u> | **2.3** |
|
|
|
|
| 109 |
| PEcore | 5.4B | 48.2 | 53.1 | 49.8 | **89.3** | 38.9 | **87.9** | 4.1 |
|
| 110 |
| SigLIP2 | 10B | 56.1 | 62.3 | 62.9 | <u>89.1</u> | 45.4 | 86.9 | β |
|
| 111 |
| OpenCLIP | 2B | β | β | β | β | β | β | β |
|
|
@@ -120,10 +121,9 @@ report's unified evaluation (Tab. 3, 5, 6) and ImageNet from Tab. 7; OpenCLIP an
|
|
| 120 |
are not in that report; VideoMAEv2's K400 figure is cited from its own paper (see β‘), and
|
| 121 |
OpenCLIP's remaining benchmarks are not reported under a comparable protocol.
|
| 122 |
|
| 123 |
-
Motif
|
| 124 |
-
|
| 125 |
-
|
| 126 |
-
KITTI depth is still being evaluated and will be added as the run completes.
|
| 127 |
|
| 128 |
|
| 129 |
## License
|
|
|
|
| 24 |
<p align="center">
|
| 25 |
<img src="assets/haaland_full_attn_blk20.gif" width="480" alt="Point tracking on a video clip: Motif vs V-JEPA 2.1"/>
|
| 26 |
</p>
|
| 27 |
+
<p align="center"><em>Point tracking on a video clip (top: Motif, bottom: V-JEPA 2.1) β a query point propagated across frames by patch-feature cosine similarity. Motif tracks the subject more reliably than V-JEPA 2.1.</em></p>
|
| 28 |
|
| 29 |
- **Architecture**: ViT-7B (embed 4096 / depth 40 / heads 32), patch 16, 3D axial RoPE
|
| 30 |
(`base=100`), SwiGLU FFN, LayerScale, per-head QK-norm, gated attention, 4 register tokens.
|
|
|
|
| 106 |
|---|---|---|---|---|---|---|---|---|
|
| 107 |
| **Motif Vision Encoder** | 0.5B | **74.0** | **80.5** | **83.5** | 87.4 | <u>52.0</u> | 87.4 | 3.2 |
|
| 108 |
| DINOv3 | 1.7B | <u>71.1</u> | <u>79.7</u> | <u>83.3</u> | 88.4 | **55.9** | <u>87.8</u> | **2.3** |
|
| 109 |
+
| Web-DINO | 2B | 57.2 | 65.8 | 69.5 | 85.9 | 42.7 | 86.8 | 3.2 |
|
| 110 |
| PEcore | 5.4B | 48.2 | 53.1 | 49.8 | **89.3** | 38.9 | **87.9** | 4.1 |
|
| 111 |
| SigLIP2 | 10B | 56.1 | 62.3 | 62.9 | <u>89.1</u> | 45.4 | 86.9 | β |
|
| 112 |
| OpenCLIP | 2B | β | β | β | β | β | β | β |
|
|
|
|
| 121 |
are not in that report; VideoMAEv2's K400 figure is cited from its own paper (see β‘), and
|
| 122 |
OpenCLIP's remaining benchmarks are not reported under a comparable protocol.
|
| 123 |
|
| 124 |
+
Motif is state of the art on DAVIS video tracking at every resolution (74.0 / 80.5 / 83.5 J&F)
|
| 125 |
+
and stays competitive on the other image and video benchmarks, using roughly 1/3 of DINOv3's
|
| 126 |
+
training data (~0.5B samples).
|
|
|
|
| 127 |
|
| 128 |
|
| 129 |
## License
|