File size: 1,753 Bytes
19cba99 de4ee9c 19cba99 de4ee9c 19cba99 de4ee9c 19cba99 de4ee9c 19cba99 de4ee9c 19cba99 de4ee9c 19cba99 de4ee9c cf57e25 de4ee9c | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 | ---
license: apache-2.0
library_name: transformers
tags:
- vision-encoder
- distillation
- video-language
- siglip2
- dinov3
---
# VisionEncoder
Hosted artifacts (derived data + trained checkpoints) for the **VisionEncoder** research project.
**Training code + full reproduction guide**: https://github.com/xiaomoguhz/VisionEncoder
The repo is organized into three top-level folders.
## `data/` — current (V9.x) reproduction data (~6.5G)
| Path | Content |
|---|---|
| `data/vmllm_cached/qwen3vit/` | S2 `cached_dataset` arrow (image/video, 10pct + full); fed directly to stage-2 |
| `data/ms-swift-data/` | sampled sharegpt jsonl (10pct + full) |
| `data/llava_video/` | V9 decode-probed `good_manifest` for the video path |
## `ckpts/` — ready-made 4B MLLM inference weights
| Path | Content |
|---|---|
| `ckpts/4b_stock` | 4B stock baseline (raw Qwen3.5 ViT, skips declip), checkpoint-505, 9.5G |
| `ckpts/4b_v9_1` | 4B V9.1 (V-JEPA 2.1 video self-distill), checkpoint-505, 9.5G |
Download either and feed it straight to evaluation (see the GitHub README, section 4 — MLLM evaluation) to skip declip + S1 + S2.
## `legacy/` — historical assets (~368G)
Early-line products, not needed to reproduce the current main line: `declip_siglip2/spatial_align`, `kd_mllm`, `self_refine`, `video_mllm_swift` (old SigLIP2 / image-only S1+S2 ckpts), and old ViT-family arrow caches.
## Download
```bash
# current dev data
huggingface-cli download xiaomoguhzz/VisionEncoder --include "data/*" --local-dir .
# ready-made 4B MLLM ckpt (eval directly)
huggingface-cli download xiaomoguhzz/VisionEncoder --include "ckpts/4b_v9_1/*" --local-dir .
```
## Related
- Code + reproduction guide: https://github.com/xiaomoguhz/VisionEncoder
|