Image Feature Extraction
Transformers
Safetensors
motif_vision
feature-extraction
motif
vision-transformer
self-supervised
video
custom_code
Instructions to use Motif-Technologies/Motif-Vision-Encoder with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use Motif-Technologies/Motif-Vision-Encoder with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("image-feature-extraction", model="Motif-Technologies/Motif-Vision-Encoder", trust_remote_code=True)# pip install -U transformers accelerate # Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("Motif-Technologies/Motif-Vision-Encoder", trust_remote_code=True, device_map="auto") - Notebooks
- Google Colab
- Kaggle
docs(README): drop KITTI depth column
Browse filesRemove the KITTI column from the eval table and all KITTI references (caption 'except KITTI depth' clause, protocol note).
README.md
CHANGED
|
@@ -94,29 +94,28 @@ Outputs (`BaseModelOutputWithPooling`): `last_hidden_state` `(B, 1+4+N, 4096)`,
|
|
| 94 |
## Evaluation
|
| 95 |
|
| 96 |
Compared against the strongest publicly reported self-supervised / vision backbones. Higher is
|
| 97 |
-
better for every column
|
| 98 |
-
|
| 99 |
-
different protocol and is excluded from the ranking.
|
| 100 |
|
| 101 |
DAVIS S/M/L follow the DINOv3 protocol (J&F-mean at video short side 420/480, 840/960,
|
| 102 |
1260/1440 px). V-JEPA 2.1 is not part of the DINOv3 Table 5 tracking benchmark, so only its
|
| 103 |
single-resolution (S) figure is available.
|
| 104 |
|
| 105 |
-
| Model | Training<br>data | DAVIS S<br>J&F β | DAVIS M<br>J&F β | DAVIS L<br>J&F β | ImageNet-1K<br>lin. probe β | ADE20K<br>mIoU β | K400 β |
|
| 106 |
-
|---|---|---|---|---|---|---|---|
|
| 107 |
-
| **Motif Vision Encoder** | 0.5B | **74.0** | **80.5** | **83.5** | 87.4 | <u>52.0</u> | 87.4 |
|
| 108 |
-
| DINOv3 | 1.7B | <u>71.1</u> | <u>79.7</u> | <u>83.3</u> | 88.4 | **55.9** | <u>87.8</u> |
|
| 109 |
-
| Web-DINO | 2B | 57.2 | 65.8 | 69.5 | 85.9 | 42.7 | 86.8 |
|
| 110 |
-
| PEcore | 5.4B | 48.2 | 53.1 | 49.8 | **89.3** | 38.9 | **87.9** |
|
| 111 |
-
| SigLIP2 | 10B | 56.1 | 62.3 | 62.9 | <u>89.1</u> | 45.4 | 86.9 |
|
| 112 |
-
| OpenCLIP | 2B | β | β | β | β | β | β |
|
| 113 |
-
| V-JEPA 2.1 | 0.022B | 69.0 | β | β | 85.5 | 47.9 | 87.7 |
|
| 114 |
-
| VideoMAEv2 | 1.35M | β | β | β | β | β | 90.0<sup>β‘</sup> |
|
| 115 |
|
| 116 |
<sub>β‘ VideoMAEv2 ViT-g: K400 top-1 after **full fine-tuning**, not the frozen attentive-probe protocol used for the other rows.</sub>
|
| 117 |
|
| 118 |
Protocol: DINOv3-style linear/attentive probes for image tasks; V-JEPA 2-style protocol for
|
| 119 |
-
video. Baseline DAVIS / ADE20K / K400
|
| 120 |
report's unified evaluation (Tab. 3, 5, 6) and ImageNet from Tab. 7; OpenCLIP and VideoMAEv2
|
| 121 |
are not in that report; VideoMAEv2's K400 figure is cited from its own paper (see β‘), and
|
| 122 |
OpenCLIP's remaining benchmarks are not reported under a comparable protocol.
|
|
|
|
| 94 |
## Evaluation
|
| 95 |
|
| 96 |
Compared against the strongest publicly reported self-supervised / vision backbones. Higher is
|
| 97 |
+
better for every column. Best comparable value per column in bold, second best <u>underlined</u>;
|
| 98 |
+
the footnoted figure (β‘) is measured under a different protocol and is excluded from the ranking.
|
|
|
|
| 99 |
|
| 100 |
DAVIS S/M/L follow the DINOv3 protocol (J&F-mean at video short side 420/480, 840/960,
|
| 101 |
1260/1440 px). V-JEPA 2.1 is not part of the DINOv3 Table 5 tracking benchmark, so only its
|
| 102 |
single-resolution (S) figure is available.
|
| 103 |
|
| 104 |
+
| Model | Training<br>data | DAVIS S<br>J&F β | DAVIS M<br>J&F β | DAVIS L<br>J&F β | ImageNet-1K<br>lin. probe β | ADE20K<br>mIoU β | K400 β |
|
| 105 |
+
|---|---|---|---|---|---|---|---|
|
| 106 |
+
| **Motif Vision Encoder** | 0.5B | **74.0** | **80.5** | **83.5** | 87.4 | <u>52.0</u> | 87.4 |
|
| 107 |
+
| DINOv3 | 1.7B | <u>71.1</u> | <u>79.7</u> | <u>83.3</u> | 88.4 | **55.9** | <u>87.8</u> |
|
| 108 |
+
| Web-DINO | 2B | 57.2 | 65.8 | 69.5 | 85.9 | 42.7 | 86.8 |
|
| 109 |
+
| PEcore | 5.4B | 48.2 | 53.1 | 49.8 | **89.3** | 38.9 | **87.9** |
|
| 110 |
+
| SigLIP2 | 10B | 56.1 | 62.3 | 62.9 | <u>89.1</u> | 45.4 | 86.9 |
|
| 111 |
+
| OpenCLIP | 2B | β | β | β | β | β | β |
|
| 112 |
+
| V-JEPA 2.1 | 0.022B | 69.0 | β | β | 85.5 | 47.9 | 87.7 |
|
| 113 |
+
| VideoMAEv2 | 1.35M | β | β | β | β | β | 90.0<sup>β‘</sup> |
|
| 114 |
|
| 115 |
<sub>β‘ VideoMAEv2 ViT-g: K400 top-1 after **full fine-tuning**, not the frozen attentive-probe protocol used for the other rows.</sub>
|
| 116 |
|
| 117 |
Protocol: DINOv3-style linear/attentive probes for image tasks; V-JEPA 2-style protocol for
|
| 118 |
+
video. Baseline DAVIS / ADE20K / K400 figures are taken from the DINOv3 technical
|
| 119 |
report's unified evaluation (Tab. 3, 5, 6) and ImageNet from Tab. 7; OpenCLIP and VideoMAEv2
|
| 120 |
are not in that report; VideoMAEv2's K400 figure is cited from its own paper (see β‘), and
|
| 121 |
OpenCLIP's remaining benchmarks are not reported under a comparable protocol.
|