Video Classification
Transformers
Safetensors
ttvidt
feature-extraction
video
video-representation-learning
self-supervised-learning
motion
temporal-modeling
dinov3
vision-transformer
custom_code
Eval Results (legacy)
Instructions to use KBlueLeaf/TTVidT with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use KBlueLeaf/TTVidT with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("video-classification", model="KBlueLeaf/TTVidT", trust_remote_code=True)# pip install -U transformers accelerate # Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("KBlueLeaf/TTVidT", trust_remote_code=True, device_map="auto") - Notebooks
- Google Colab
- Kaggle
Model card: possible downstream uses
Browse files
README.md
CHANGED
|
@@ -72,6 +72,22 @@ Frozen attentive probe, mean top-1 over 3 seeds (TT-VidT evaluation protocol, 8
|
|
| 72 |
| paper TT3D (full-linear resample, 407.8M) | 24.8 | 35.4 | 86.6 | 72.9 | 25.9 |
|
| 73 |
| this checkpoint (structured resample, 195.5M) | 25.2 | 36.1 | 86.0 | 72.9 | 25.4 |
|
| 74 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 75 |
## License
|
| 76 |
|
| 77 |
Apache-2.0. The spatial path is initialised from DINOv3, which has its own license.
|
|
|
|
| 72 |
| paper TT3D (full-linear resample, 407.8M) | 24.8 | 35.4 | 86.6 | 72.9 | 25.9 |
|
| 73 |
| this checkpoint (structured resample, 195.5M) | 25.2 | 36.1 | 86.0 | 72.9 | 25.4 |
|
| 74 |
|
| 75 |
+
## Possible downstream uses
|
| 76 |
+
|
| 77 |
+
The encoder gives a compact sequence of per-frame motion tokens alongside the
|
| 78 |
+
DINOv3 appearance features. Some directions we think are worth trying:
|
| 79 |
+
|
| 80 |
+
- **From image models to video models**: pair an existing image model with the
|
| 81 |
+
motion token sequence to get a video model for understanding in the broad
|
| 82 |
+
sense: classification, retrieval, captioning, question answering, or any
|
| 83 |
+
other task.
|
| 84 |
+
- **Generation and motion transfer**: use the motion tokens as a conditioning
|
| 85 |
+
signal for video generation, or take them from one clip and apply them to
|
| 86 |
+
another subject or scene.
|
| 87 |
+
|
| 88 |
+
Further exploration and feedback are very welcome, and so are attempts at larger
|
| 89 |
+
scale (bigger backbones, more data, longer training).
|
| 90 |
+
|
| 91 |
## License
|
| 92 |
|
| 93 |
Apache-2.0. The spatial path is initialised from DINOv3, which has its own license.
|