Video-4M Tokenizers
FSQ tokenizers used by Video-4M to convert each modality into discrete tokens for training and generation. RGB, depth, surface normals, and optical flow share one causal VidTok architecture (same config, different weights per modality). The three self-supervised feature-map modalities (V-JEPA 2, DINOv2, SigLIP 2) use FSQ tokenizers trained on top of the corresponding frozen feature extractor.
Video-4M is trained on 4-second, 17-frame, 4 FPS clips at 128x128 resolution. For the feature-map modalities, features are extracted at each extractor's native resolution before tokenization: 224x224 for DINOv2 and SigLIP 2, 256x256 for V-JEPA 2.
| Modality | Path in repo | Tokens per clip | Vocab size |
|---|---|---|---|
| RGB | rgb/ckpt.ckpt |
1280 | 32,768 |
| Depth | depth/ckpt.ckpt |
1280 | 32,768 |
| Surface normals | surface-normals/ckpt.ckpt |
1280 | 32,768 |
| Optical flow | opticalflow/ckpt.ckpt |
1280 | 32,768 |
| V-JEPA 2 | v-jepa-2/ckpt.ckpt |
1024 | 16,807 |
| DINOv2 | dinov2/ckpt.ckpt |
1280 | 16,807 |
| SigLIP 2 | siglip-2/ckpt.ckpt |
980 | 16,807 |
See also: EPFL-VILAB/Video-4M-models
(the main any-to-any model these tokenizers feed into) and
EPFL-VILAB/Video-4M-examples
(example clips, pre-tokenized with these same tokenizers).