| --- |
| tags: |
| - vision-encoder-evaluation |
| - multimodal |
| - visual-tokenizer-benchmark |
| - checkpoints |
| - arxiv:2610.05413 |
| --- |
| |
| <h1 align="center">A Strong Baseline for Evaluating Vision Encoders<br>in Multimodal Large Language Models</h1> |
|
|
| <p align="center"> |
| Yilin Yang<sup>1,*</sup> · |
| Jun-Tao Tang<sup>2,*</sup> · |
| Kengyi Wang<sup>3</sup> · |
| Siyuan Su<sup>3</sup> · |
| Gaoyong Luo<sup>4</sup> · |
| Mingda Chen<sup>1,†</sup> |
| </p> |
|
|
| <p align="center"> |
| <sup>1</sup>School of Artificial Intelligence, Shanghai Jiao Tong University<br> |
| <sup>2</sup>Nanjing University · |
| <sup>3</sup>Fudan University · |
| <sup>4</sup>Independent Researcher<br> |
| <sup>*</sup>Equal contribution. <sup>†</sup>Corresponding author. |
| </p> |
| |
| <p align="center"> |
| <a href="https://arxiv.org/abs/2610.05413"><img src="https://img.shields.io/badge/arXiv-2610.05413-b31b1b" alt="Paper: arXiv 2610.05413"></a> |
| <a href="https://arxiv.org/pdf/2610.05413"><img src="https://img.shields.io/badge/Paper-PDF-red" alt="Paper PDF"></a> |
| <a href="https://github.com/JuntaoTang/MLLM-VisionEncoder-Eval"><img src="https://img.shields.io/badge/GitHub-Code-181717?logo=github" alt="GitHub code"></a> |
| <a href="https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main"><img src="https://img.shields.io/badge/Model_Zoo-Checkpoints-FFD21E" alt="Model zoo checkpoints"></a> |
| <a href="#citation"><img src="https://img.shields.io/badge/Citation-BibTeX-blue" alt="BibTeX citation"></a> |
| </p> |
| |
| # Overview |
| |
| This repository hosts the downstream MLLM checkpoints accompanying **[A Strong Baseline for Evaluating Vision Encoders in Multimodal Large Language Models](https://arxiv.org/abs/2610.05413)**. |
| |
| # Model Zoo |
| |
| This Model Zoo covers the **70 visual encoders used in our paper**: **43 language-supervised**, **22 self-supervised**, and **5 discrete** tokenizers. |
| The current release provides both pretraining and finetuning checkpoints for all **65 continuous vision encoders** with each of the three main language backbones. |
| |
| ## Training data |
| |
| - **Pretrain Data — LCS-558K:** the image–text alignment dataset from [LLaVA-Pretrain](https://huggingface.co/datasets/liuhaotian/LLaVA-Pretrain), using `blip_laion_cc_sbu_558k.json`. This dataset is used for MLLM projector training. |
| - **Finetuning Data — LLaVA-v1.5 mix665k (filtered):** the [LLaVA-v1.5 instruction mixture](https://huggingface.co/datasets/liuhaotian/LLaVA-Instruct-150K/blob/main/llava_v1_5_mix665k.json), using `llava_v1_5_mix665k_drop_ge8kchars.json`. The training configuration removes 395 examples with at least 8,000 characters of conversation text. |
| |
| |
| ## Downloads |
| - `Encoder weights` links to the original upstream vision encoder or visual tokenizer. Each encoder name links to its model or project page. Use these frozen encoder weights together with the corresponding Stage 1 projector or Stage 2 MLLM checkpoint, preserving the architecture, feature layer and preprocessing specified by that run's `config.json`. |
| - For Hugging Face models, retain the associated configuration and processor files. **Web-SSL MAE 3B** has sharded weights; its link opens the complete model repository. |
| - **DINOv3** requires acceptance of the upstream model's terms and authentication. The **RAEv2 DINOv3-L (k=7)** entry uses the same DINOv3-L/16 backbone with the project's k=7 intermediate-layer readout; its download is the backbone used to construct that representation. |
| - `projector` downloads the pretraining `mm_projector.bin`; |
| - `finetuned` opens the finetuned checkpoint directory, including model weights, configuration, and language-tokenizer files. The frozen vision encoder must be supplied separately using the matching architecture and weights recorded in `config.json`. Pretraining projector weights alone are not instruction-tuned MLLMs. |
| |
| ### [Qwen/Qwen2.5-1.5B-Instruct](https://huggingface.co/Qwen/Qwen2.5-1.5B-Instruct) |
| |
| | Base LLM | Vision Encoder / Tokenizer | Encoder weights | Stage 1 Pretrained weights | Stage 2 Finetuned weights | |
| | --- | --- | --- | --- | --- | |
| | Qwen2.5-1.5B-Instruct | [PE-Core-G/14 (448)](https://huggingface.co/facebook/PE-Core-G14-448) | [encoder](https://huggingface.co/facebook/PE-Core-G14-448/resolve/main/PE-Core-G14-448.pt?download=true) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/qwen25/pe_core_g14_448/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/qwen25/pe_core_g14_448/finetune) | |
| | Qwen2.5-1.5B-Instruct | [MetaCLIP 2 ViT-G/14 (378)](https://github.com/facebookresearch/MetaCLIP#pre-trained-models) | [encoder](https://dl.fbaipublicfiles.com/MMPT/metaclip/metaclip2_bigG14_378px_worldwide.pt) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/qwen25/mc2_g14_378/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/qwen25/mc2_g14_378/finetune) | |
| | Qwen2.5-1.5B-Instruct | [MetaCLIP 2 ViT-G/14 (224)](https://github.com/facebookresearch/MetaCLIP#pre-trained-models) | [encoder](https://dl.fbaipublicfiles.com/MMPT/metaclip/metaclip2_bigG14_224px_worldwide.pt) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/qwen25/mc2_g14_224/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/qwen25/mc2_g14_224/finetune) | |
| | Qwen2.5-1.5B-Instruct | [MetaCLIP-2.5B ViT-G/14 (224)](https://github.com/facebookresearch/MetaCLIP#pre-trained-models) | [encoder](https://dl.fbaipublicfiles.com/MMPT/metaclip/G14_fullcc2.5b.pt) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/qwen25/mc1_g14_224_2.5b/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/qwen25/mc1_g14_224_2.5b/finetune) | |
| | Qwen2.5-1.5B-Instruct | [SigLIP2 ViT-G/16 (384)](https://github.com/google-research/big_vision/blob/main/big_vision/configs/proj/image_text/README_siglip2.md) | [encoder](https://storage.googleapis.com/big_vision/siglip2/siglip2_g-opt16_384.npz) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/qwen25/siglip2_g16_384/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/qwen25/siglip2_g16_384/finetune) | |
| | Qwen2.5-1.5B-Instruct | [SigLIP2 ViT-G/16 (256)](https://github.com/google-research/big_vision/blob/main/big_vision/configs/proj/image_text/README_siglip2.md) | [encoder](https://storage.googleapis.com/big_vision/siglip2/siglip2_g-opt16_256.npz) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/qwen25/siglip2_g16_256/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/qwen25/siglip2_g16_256/finetune) | |
| | Qwen2.5-1.5B-Instruct | [MetaCLIP 2 ViT-H/14 (378)](https://github.com/facebookresearch/MetaCLIP#pre-trained-models) | [encoder](https://dl.fbaipublicfiles.com/MMPT/metaclip/metaclip2_h14_378px_worldwide.pt) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/qwen25/mc2_h14_378/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/qwen25/mc2_h14_378/finetune) | |
| | Qwen2.5-1.5B-Instruct | [MetaCLIP-2.5B ViT-H/14 (224)](https://github.com/facebookresearch/MetaCLIP#pre-trained-models) | [encoder](https://dl.fbaipublicfiles.com/MMPT/metaclip/h14_fullcc2.5b.pt) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/qwen25/mc1_h14_224_2.5b/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/qwen25/mc1_h14_224_2.5b/finetune) | |
| | Qwen2.5-1.5B-Instruct | [MetaCLIP-v1.2 ViT-H/14 (224)](https://github.com/facebookresearch/MetaCLIP#pre-trained-models) | [encoder](https://dl.fbaipublicfiles.com/MMPT/metaclip/h14_v1.2_altogether.pt) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/qwen25/mc1_h14_224_v1.2/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/qwen25/mc1_h14_224_v1.2/finetune) | |
| | Qwen2.5-1.5B-Instruct | [OpenAI CLIP ViT-L/14 (224)](https://huggingface.co/openai/clip-vit-large-patch14) | [encoder](https://huggingface.co/openai/clip-vit-large-patch14/resolve/main/model.safetensors?download=true) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/qwen25/clip_openai__l14/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/qwen25/clip_openai__l14/finetune) | |
| | Qwen2.5-1.5B-Instruct | [MetaCLIP 2 ViT-L/14 (224)](https://github.com/facebookresearch/MetaCLIP#pre-trained-models) | [encoder](https://dl.fbaipublicfiles.com/MMPT/metaclip/metaclip2_l14_224px_worldwide.pt) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/qwen25/mc2_l14_224/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/qwen25/mc2_l14_224/finetune) | |
| | Qwen2.5-1.5B-Instruct | [MetaCLIP-2.5B ViT-L/14 (224)](https://github.com/facebookresearch/MetaCLIP#pre-trained-models) | [encoder](https://dl.fbaipublicfiles.com/MMPT/metaclip/l14_fullcc2.5b.pt) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/qwen25/mc1_l14_224_2.5b/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/qwen25/mc1_l14_224_2.5b/finetune) | |
| | Qwen2.5-1.5B-Instruct | [MetaCLIP-400M ViT-L/14 (224)](https://github.com/facebookresearch/MetaCLIP#pre-trained-models) | [encoder](https://dl.fbaipublicfiles.com/MMPT/metaclip/l14_400m.pt) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/qwen25/mc1_l14_224_400m/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/qwen25/mc1_l14_224_400m/finetune) | |
| | Qwen2.5-1.5B-Instruct | [SigLIP2 So400m/l14 (384)](https://github.com/google-research/big_vision/blob/main/big_vision/configs/proj/image_text/README_siglip2.md) | [encoder](https://storage.googleapis.com/big_vision/siglip2/siglip2_so400m14_384.npz) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/qwen25/siglip2_sm14_384/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/qwen25/siglip2_sm14_384/finetune) | |
| | Qwen2.5-1.5B-Instruct | [SigLIP2 So400m/l14 (224)](https://github.com/google-research/big_vision/blob/main/big_vision/configs/proj/image_text/README_siglip2.md) | [encoder](https://storage.googleapis.com/big_vision/siglip2/siglip2_so400m14_224.npz) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/qwen25/siglip2_sm14_224/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/qwen25/siglip2_sm14_224/finetune) | |
| | Qwen2.5-1.5B-Instruct | [SigLIP2 So400m/l16 (512)](https://github.com/google-research/big_vision/blob/main/big_vision/configs/proj/image_text/README_siglip2.md) | [encoder](https://storage.googleapis.com/big_vision/siglip2/siglip2_so400m16_512.npz) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/qwen25/siglip2_sm16_512/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/qwen25/siglip2_sm16_512/finetune) | |
| | Qwen2.5-1.5B-Instruct | [SigLIP2 So400m/l16 (384)](https://github.com/google-research/big_vision/blob/main/big_vision/configs/proj/image_text/README_siglip2.md) | [encoder](https://storage.googleapis.com/big_vision/siglip2/siglip2_so400m16_384.npz) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/qwen25/siglip2_sm16_384/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/qwen25/siglip2_sm16_384/finetune) | |
| | Qwen2.5-1.5B-Instruct | [SigLIP2 So400m/l16 (256)](https://github.com/google-research/big_vision/blob/main/big_vision/configs/proj/image_text/README_siglip2.md) | [encoder](https://storage.googleapis.com/big_vision/siglip2/siglip2_so400m16_256.npz) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/qwen25/siglip2_sm16_256/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/qwen25/siglip2_sm16_256/finetune) | |
| | Qwen2.5-1.5B-Instruct | [PE-Lang-L/14 (448)](https://huggingface.co/facebook/PE-Lang-L14-448) | [encoder](https://huggingface.co/facebook/PE-Lang-L14-448/resolve/main/PE-Lang-L14-448.pt?download=true) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/qwen25/pe_lang_l14_448/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/qwen25/pe_lang_l14_448/finetune) | |
| | Qwen2.5-1.5B-Instruct | [SigLIP2 ViT-L/16 (512)](https://github.com/google-research/big_vision/blob/main/big_vision/configs/proj/image_text/README_siglip2.md) | [encoder](https://storage.googleapis.com/big_vision/siglip2/siglip2_l16_512.npz) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/qwen25/siglip2_l16_512/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/qwen25/siglip2_l16_512/finetune) | |
| | Qwen2.5-1.5B-Instruct | [SigLIP2 ViT-L/16 (384)](https://github.com/google-research/big_vision/blob/main/big_vision/configs/proj/image_text/README_siglip2.md) | [encoder](https://storage.googleapis.com/big_vision/siglip2/siglip2_l16_384.npz) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/qwen25/siglip2_l16_384/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/qwen25/siglip2_l16_384/finetune) | |
| | Qwen2.5-1.5B-Instruct | [SigLIP2 ViT-L/16 (256)](https://github.com/google-research/big_vision/blob/main/big_vision/configs/proj/image_text/README_siglip2.md) | [encoder](https://storage.googleapis.com/big_vision/siglip2/siglip2_l16_256.npz) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/qwen25/siglip2_l16_256/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/qwen25/siglip2_l16_256/finetune) | |
| | Qwen2.5-1.5B-Instruct | [MetaCLIP 2 ViT-M/16 (384)](https://github.com/facebookresearch/MetaCLIP#pre-trained-models) | [encoder](https://dl.fbaipublicfiles.com/MMPT/metaclip/metaclip2_m16_384px_worldwide.pt) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/qwen25/mc2_m16_384/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/qwen25/mc2_m16_384/finetune) | |
| | Qwen2.5-1.5B-Instruct | [MetaCLIP 2 ViT-M/16 (224)](https://github.com/facebookresearch/MetaCLIP#pre-trained-models) | [encoder](https://dl.fbaipublicfiles.com/MMPT/metaclip/metaclip2_m16_224px_worldwide.pt) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/qwen25/mc2_m16_224/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/qwen25/mc2_m16_224/finetune) | |
| | Qwen2.5-1.5B-Instruct | [MetaCLIP 2-mT5 ViT-M/16 (224)](https://github.com/facebookresearch/MetaCLIP#pre-trained-models) | [encoder](https://dl.fbaipublicfiles.com/MMPT/metaclip/metaclip2_m16_224px_mt5_worldwide.pt) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/qwen25/mc2_m16_224_mt5/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/qwen25/mc2_m16_224_mt5/finetune) | |
| | Qwen2.5-1.5B-Instruct | [MetaCLIP 2 ViT-B/16 (384)](https://github.com/facebookresearch/MetaCLIP#pre-trained-models) | [encoder](https://dl.fbaipublicfiles.com/MMPT/metaclip/metaclip2_b16_384px_worldwide.pt) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/qwen25/mc2_b16_384/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/qwen25/mc2_b16_384/finetune) | |
| | Qwen2.5-1.5B-Instruct | [MetaCLIP 2 ViT-B/16 (224)](https://github.com/facebookresearch/MetaCLIP#pre-trained-models) | [encoder](https://dl.fbaipublicfiles.com/MMPT/metaclip/metaclip2_b16_224px_worldwide.pt) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/qwen25/mc2_b16_224/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/qwen25/mc2_b16_224/finetune) | |
| | Qwen2.5-1.5B-Instruct | [MetaCLIP 2 ViT-B/32 (384)](https://github.com/facebookresearch/MetaCLIP#pre-trained-models) | [encoder](https://dl.fbaipublicfiles.com/MMPT/metaclip/metaclip2_b32_384px_worldwide.pt) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/qwen25/mc2_b32_384/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/qwen25/mc2_b32_384/finetune) | |
| | Qwen2.5-1.5B-Instruct | [MetaCLIP 2 ViT-B/32 (224)](https://github.com/facebookresearch/MetaCLIP#pre-trained-models) | [encoder](https://dl.fbaipublicfiles.com/MMPT/metaclip/metaclip2_b32_224px_worldwide.pt) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/qwen25/mc2_b32_224/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/qwen25/mc2_b32_224/finetune) | |
| | Qwen2.5-1.5B-Instruct | [MetaCLIP 2-mT5 ViT-B/32 (224)](https://github.com/facebookresearch/MetaCLIP#pre-trained-models) | [encoder](https://dl.fbaipublicfiles.com/MMPT/metaclip/metaclip2_b32_224px_mt5_worldwide.pt) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/qwen25/mc2_b32_224_mt5/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/qwen25/mc2_b32_224_mt5/finetune) | |
| | Qwen2.5-1.5B-Instruct | [MetaCLIP-2.5B ViT-B/16 (224)](https://github.com/facebookresearch/MetaCLIP#pre-trained-models) | [encoder](https://dl.fbaipublicfiles.com/MMPT/metaclip/b16_fullcc2.5b.pt) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/qwen25/mc1_b16_224_2.5b/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/qwen25/mc1_b16_224_2.5b/finetune) | |
| | Qwen2.5-1.5B-Instruct | [MetaCLIP-2.5B ViT-B/32 (224)](https://github.com/facebookresearch/MetaCLIP#pre-trained-models) | [encoder](https://dl.fbaipublicfiles.com/MMPT/metaclip/b32_fullcc2.5b.pt) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/qwen25/mc1_b32_224_2.5b/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/qwen25/mc1_b32_224_2.5b/finetune) | |
| | Qwen2.5-1.5B-Instruct | [MetaCLIP-400M ViT-B/16 (224)](https://github.com/facebookresearch/MetaCLIP#pre-trained-models) | [encoder](https://dl.fbaipublicfiles.com/MMPT/metaclip/b16_400m.pt) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/qwen25/mc1_b16_224_400m/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/qwen25/mc1_b16_224_400m/finetune) | |
| | Qwen2.5-1.5B-Instruct | [MetaCLIP-400M ViT-B/32 (224)](https://github.com/facebookresearch/MetaCLIP#pre-trained-models) | [encoder](https://dl.fbaipublicfiles.com/MMPT/metaclip/b32_400m.pt) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/qwen25/mc1_b32_224_400m/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/qwen25/mc1_b32_224_400m/finetune) | |
| | Qwen2.5-1.5B-Instruct | [PE-Core-B/16 (224)](https://huggingface.co/facebook/PE-Core-B16-224) | [encoder](https://huggingface.co/facebook/PE-Core-B16-224/resolve/main/PE-Core-B16-224.pt?download=true) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/qwen25/pe_core_b16_224/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/qwen25/pe_core_b16_224/finetune) | |
| | Qwen2.5-1.5B-Instruct | [SigLIP2 ViT-B/16 (512)](https://github.com/google-research/big_vision/blob/main/big_vision/configs/proj/image_text/README_siglip2.md) | [encoder](https://storage.googleapis.com/big_vision/siglip2/siglip2_b16_512.npz) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/qwen25/siglip2_b16_512/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/qwen25/siglip2_b16_512/finetune) | |
| | Qwen2.5-1.5B-Instruct | [SigLIP2 ViT-B/16 (384)](https://github.com/google-research/big_vision/blob/main/big_vision/configs/proj/image_text/README_siglip2.md) | [encoder](https://storage.googleapis.com/big_vision/siglip2/siglip2_b16_384.npz) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/qwen25/siglip2_b16_384/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/qwen25/siglip2_b16_384/finetune) | |
| | Qwen2.5-1.5B-Instruct | [SigLIP2 ViT-B/16 (256)](https://github.com/google-research/big_vision/blob/main/big_vision/configs/proj/image_text/README_siglip2.md) | [encoder](https://storage.googleapis.com/big_vision/siglip2/siglip2_b16_256.npz) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/qwen25/siglip2_b16_256/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/qwen25/siglip2_b16_256/finetune) | |
| | Qwen2.5-1.5B-Instruct | [SigLIP2 ViT-B/16 (224)](https://github.com/google-research/big_vision/blob/main/big_vision/configs/proj/image_text/README_siglip2.md) | [encoder](https://storage.googleapis.com/big_vision/siglip2/siglip2_b16_224.npz) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/qwen25/siglip2_b16_224/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/qwen25/siglip2_b16_224/finetune) | |
| | Qwen2.5-1.5B-Instruct | [SigLIP2 ViT-B/32 (256)](https://github.com/google-research/big_vision/blob/main/big_vision/configs/proj/image_text/README_siglip2.md) | [encoder](https://storage.googleapis.com/big_vision/siglip2/siglip2_b32_256.npz) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/qwen25/siglip2_b32_256/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/qwen25/siglip2_b32_256/finetune) | |
| | Qwen2.5-1.5B-Instruct | [MetaCLIP 2 ViT-S/16 (384)](https://github.com/facebookresearch/MetaCLIP#pre-trained-models) | [encoder](https://dl.fbaipublicfiles.com/MMPT/metaclip/metaclip2_s16_384px_worldwide.pt) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/qwen25/mc2_s16_384/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/qwen25/mc2_s16_384/finetune) | |
| | Qwen2.5-1.5B-Instruct | [MetaCLIP 2 ViT-S/16 (224)](https://github.com/facebookresearch/MetaCLIP#pre-trained-models) | [encoder](https://dl.fbaipublicfiles.com/MMPT/metaclip/metaclip2_s16_224px_worldwide.pt) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/qwen25/mc2_s16_224/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/qwen25/mc2_s16_224/finetune) | |
| | Qwen2.5-1.5B-Instruct | [MetaCLIP 2-mT5 ViT-S/16 (224)](https://github.com/facebookresearch/MetaCLIP#pre-trained-models) | [encoder](https://dl.fbaipublicfiles.com/MMPT/metaclip/metaclip2_s16_224px_mt5_worldwide.pt) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/qwen25/mc2_s16_224_mt5/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/qwen25/mc2_s16_224_mt5/finetune) | |
| | Qwen2.5-1.5B-Instruct | [Web-SSL MAE 3B (224)](https://huggingface.co/facebook/webssl-mae3b-full2b-224) | [encoder files](https://huggingface.co/facebook/webssl-mae3b-full2b-224) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/qwen25/webssl_mae3b_full2b_224/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/qwen25/webssl_mae3b_full2b_224/finetune) | |
| | Qwen2.5-1.5B-Instruct | [DINOv2 ViT-G/14](https://huggingface.co/facebook/dinov2-giant) | [encoder](https://huggingface.co/facebook/dinov2-giant/resolve/main/model.safetensors?download=true) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/qwen25/dinov2_giant/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/qwen25/dinov2_giant/finetune) | |
| | Qwen2.5-1.5B-Instruct | [Web-SSL DINO 1B (224)](https://huggingface.co/facebook/webssl-dino1b-full2b-224) | [encoder files](https://huggingface.co/facebook/webssl-dino1b-full2b-224) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/qwen25/webssl_dino1b_full2b_224/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/qwen25/webssl_dino1b_full2b_224/finetune) | |
| | Qwen2.5-1.5B-Instruct | [Web-SSL MAE 1B (224)](https://huggingface.co/facebook/webssl-mae1b-full2b-224) | [encoder](https://huggingface.co/facebook/webssl-mae1b-full2b-224/resolve/main/model.safetensors?download=true) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/qwen25/webssl_mae1b_full2b_224/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/qwen25/webssl_mae1b_full2b_224/finetune) | |
| | Qwen2.5-1.5B-Instruct | [Pixio ViT-H/16](https://huggingface.co/facebook/pixio-vith16) | [encoder](https://huggingface.co/facebook/pixio-vith16/resolve/main/pixio_vith16.pth?download=true) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/qwen25/pixio_vith16/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/qwen25/pixio_vith16/finetune) | |
| | Qwen2.5-1.5B-Instruct | [I-JEPA ViT-H/14](https://github.com/facebookresearch/ijepa) | [encoder](https://dl.fbaipublicfiles.com/ijepa/IN1K-vit.h.14-300e.pth.tar) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/qwen25/ijepa_vith14/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/qwen25/ijepa_vith14/finetune) | |
| | Qwen2.5-1.5B-Instruct | [DINOv2 ViT-L/14](https://huggingface.co/facebook/dinov2-large) | [encoder](https://huggingface.co/facebook/dinov2-large/resolve/main/model.safetensors?download=true) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/qwen25/dinov2_large/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/qwen25/dinov2_large/finetune) | |
| | Qwen2.5-1.5B-Instruct | [DINOv3 ViT-L/16](https://huggingface.co/facebook/dinov3-vitl16-pretrain-lvd1689m) | [encoder](https://huggingface.co/facebook/dinov3-vitl16-pretrain-lvd1689m/resolve/main/model.safetensors?download=true) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/qwen25/dinov3_vitl16/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/qwen25/dinov3_vitl16/finetune) | |
| | Qwen2.5-1.5B-Instruct | [Pixio ViT-L/16](https://huggingface.co/facebook/pixio-vitl16) | [encoder](https://huggingface.co/facebook/pixio-vitl16/resolve/main/pixio_vitl16.pth?download=true) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/qwen25/pixio_vitl16/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/qwen25/pixio_vitl16/finetune) | |
| | Qwen2.5-1.5B-Instruct | [RAEv2 DINOv3-L (k=7)](https://github.com/nanovisionx/RAEv2) | [DINOv3 backbone](https://huggingface.co/facebook/dinov3-vitl16-pretrain-lvd1689m/resolve/main/model.safetensors?download=true) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/qwen25/raev2_dinov3l_k7/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/qwen25/raev2_dinov3l_k7/finetune) | |
| | Qwen2.5-1.5B-Instruct | [Web-SSL MAE 300M (224)](https://huggingface.co/facebook/webssl-mae300m-full2b-224) | [encoder](https://huggingface.co/facebook/webssl-mae300m-full2b-224/resolve/main/model.safetensors?download=true) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/qwen25/webssl_mae300m_full2b_224/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/qwen25/webssl_mae300m_full2b_224/finetune) | |
| | Qwen2.5-1.5B-Instruct | [EUPE ConvNeXt-B](https://huggingface.co/facebook/EUPE-ConvNeXt-B) | [encoder](https://huggingface.co/facebook/EUPE-ConvNeXt-B/resolve/main/EUPE-ConvNeXt-B.pt?download=true) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/qwen25/eupe_convnext_b/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/qwen25/eupe_convnext_b/finetune) | |
| | Qwen2.5-1.5B-Instruct | [DINOv2 ViT-B/14](https://huggingface.co/facebook/dinov2-base) | [encoder](https://huggingface.co/facebook/dinov2-base/resolve/main/model.safetensors?download=true) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/qwen25/dinov2_base/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/qwen25/dinov2_base/finetune) | |
| | Qwen2.5-1.5B-Instruct | [DINO ViT-B/8](https://huggingface.co/facebook/dino-vitb8) | [encoder](https://huggingface.co/facebook/dino-vitb8/resolve/main/model.safetensors?download=true) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/qwen25/dino_vitb8/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/qwen25/dino_vitb8/finetune) | |
| | Qwen2.5-1.5B-Instruct | [DINO ViT-B/16](https://huggingface.co/facebook/dino-vitb16) | [encoder](https://huggingface.co/facebook/dino-vitb16/resolve/main/pytorch_model.bin?download=true) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/qwen25/dino_vitb16/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/qwen25/dino_vitb16/finetune) | |
| | Qwen2.5-1.5B-Instruct | [EUPE ViT-B](https://huggingface.co/facebook/EUPE-ViT-B) | [encoder](https://huggingface.co/facebook/EUPE-ViT-B/resolve/main/EUPE-ViT-B.pt?download=true) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/qwen25/eupe_vit_b/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/qwen25/eupe_vit_b/finetune) | |
| | Qwen2.5-1.5B-Instruct | [Pixio ViT-B/16](https://huggingface.co/facebook/pixio-vitb16) | [encoder](https://huggingface.co/facebook/pixio-vitb16/resolve/main/pixio_vitb16.pth?download=true) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/qwen25/pixio_vitb16/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/qwen25/pixio_vitb16/finetune) | |
| | Qwen2.5-1.5B-Instruct | [DINOv2 ViT-S/14](https://huggingface.co/facebook/dinov2-small) | [encoder](https://huggingface.co/facebook/dinov2-small/resolve/main/model.safetensors?download=true) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/qwen25/dinov2_small/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/qwen25/dinov2_small/finetune) | |
| | Qwen2.5-1.5B-Instruct | [DINO ViT-S/16](https://huggingface.co/facebook/dino-vits16) | [encoder](https://huggingface.co/facebook/dino-vits16/resolve/main/pytorch_model.bin?download=true) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/qwen25/dino_vits16/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/qwen25/dino_vits16/finetune) | |
| | Qwen2.5-1.5B-Instruct | [EUPE ViT-S](https://huggingface.co/facebook/EUPE-ViT-S) | [encoder](https://huggingface.co/facebook/EUPE-ViT-S/resolve/main/EUPE-ViT-S.pt?download=true) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/qwen25/eupe_vit_s/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/qwen25/eupe_vit_s/finetune) | |
| | Qwen2.5-1.5B-Instruct | [DINO ViT-S/8](https://huggingface.co/facebook/dino-vits8) | [encoder](https://huggingface.co/facebook/dino-vits8/resolve/main/model.safetensors?download=true) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/qwen25/dino_vits8/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/qwen25/dino_vits8/finetune) | |
| | Qwen2.5-1.5B-Instruct | [EUPE ViT-T](https://huggingface.co/facebook/EUPE-ViT-T) | [encoder](https://huggingface.co/facebook/EUPE-ViT-T/resolve/main/EUPE-ViT-T.pt?download=true) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/qwen25/eupe_vit_t/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/qwen25/eupe_vit_t/finetune) | |
| |
| |
| ### [Qwen3-1.7B](https://huggingface.co/Qwen/Qwen3-1.7B) |
| |
| | Base LLM | Vision Encoder / Tokenizer | Encoder weights | Stage 1 Pretrained weights | Stage 2 Finetuned weights | |
| | --- | --- | --- | --- | --- | |
| | Qwen3-1.7B | [PE-Core-G/14 (448)](https://huggingface.co/facebook/PE-Core-G14-448) | [encoder](https://huggingface.co/facebook/PE-Core-G14-448/resolve/main/PE-Core-G14-448.pt?download=true) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/qwen3/pe_core_g14_448/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/qwen3/pe_core_g14_448/finetune) | |
| | Qwen3-1.7B | [MetaCLIP 2 ViT-G/14 (378)](https://github.com/facebookresearch/MetaCLIP#pre-trained-models) | [encoder](https://dl.fbaipublicfiles.com/MMPT/metaclip/metaclip2_bigG14_378px_worldwide.pt) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/qwen3/mc2_g14_378/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/qwen3/mc2_g14_378/finetune) | |
| | Qwen3-1.7B | [MetaCLIP 2 ViT-G/14 (224)](https://github.com/facebookresearch/MetaCLIP#pre-trained-models) | [encoder](https://dl.fbaipublicfiles.com/MMPT/metaclip/metaclip2_bigG14_224px_worldwide.pt) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/qwen3/mc2_g14_224/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/qwen3/mc2_g14_224/finetune) | |
| | Qwen3-1.7B | [MetaCLIP-2.5B ViT-G/14 (224)](https://github.com/facebookresearch/MetaCLIP#pre-trained-models) | [encoder](https://dl.fbaipublicfiles.com/MMPT/metaclip/G14_fullcc2.5b.pt) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/qwen3/mc1_g14_224_2.5b/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/qwen3/mc1_g14_224_2.5b/finetune) | |
| | Qwen3-1.7B | [SigLIP2 ViT-G/16 (384)](https://github.com/google-research/big_vision/blob/main/big_vision/configs/proj/image_text/README_siglip2.md) | [encoder](https://storage.googleapis.com/big_vision/siglip2/siglip2_g-opt16_384.npz) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/qwen3/siglip2_g16_384/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/qwen3/siglip2_g16_384/finetune) | |
| | Qwen3-1.7B | [SigLIP2 ViT-G/16 (256)](https://github.com/google-research/big_vision/blob/main/big_vision/configs/proj/image_text/README_siglip2.md) | [encoder](https://storage.googleapis.com/big_vision/siglip2/siglip2_g-opt16_256.npz) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/qwen3/siglip2_g16_256/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/qwen3/siglip2_g16_256/finetune) | |
| | Qwen3-1.7B | [MetaCLIP 2 ViT-H/14 (378)](https://github.com/facebookresearch/MetaCLIP#pre-trained-models) | [encoder](https://dl.fbaipublicfiles.com/MMPT/metaclip/metaclip2_h14_378px_worldwide.pt) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/qwen3/mc2_h14_378/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/qwen3/mc2_h14_378/finetune) | |
| | Qwen3-1.7B | [MetaCLIP-2.5B ViT-H/14 (224)](https://github.com/facebookresearch/MetaCLIP#pre-trained-models) | [encoder](https://dl.fbaipublicfiles.com/MMPT/metaclip/h14_fullcc2.5b.pt) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/qwen3/mc1_h14_224_2.5b/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/qwen3/mc1_h14_224_2.5b/finetune) | |
| | Qwen3-1.7B | [MetaCLIP-v1.2 ViT-H/14 (224)](https://github.com/facebookresearch/MetaCLIP#pre-trained-models) | [encoder](https://dl.fbaipublicfiles.com/MMPT/metaclip/h14_v1.2_altogether.pt) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/qwen3/mc1_h14_224_v1.2/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/qwen3/mc1_h14_224_v1.2/finetune) | |
| | Qwen3-1.7B | [OpenAI CLIP ViT-L/14 (224)](https://huggingface.co/openai/clip-vit-large-patch14) | [encoder](https://huggingface.co/openai/clip-vit-large-patch14/resolve/main/model.safetensors?download=true) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/qwen3/clip_openai__l14/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/qwen3/clip_openai__l14/finetune) | |
| | Qwen3-1.7B | [MetaCLIP 2 ViT-L/14 (224)](https://github.com/facebookresearch/MetaCLIP#pre-trained-models) | [encoder](https://dl.fbaipublicfiles.com/MMPT/metaclip/metaclip2_l14_224px_worldwide.pt) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/qwen3/mc2_l14_224/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/qwen3/mc2_l14_224/finetune) | |
| | Qwen3-1.7B | [MetaCLIP-2.5B ViT-L/14 (224)](https://github.com/facebookresearch/MetaCLIP#pre-trained-models) | [encoder](https://dl.fbaipublicfiles.com/MMPT/metaclip/l14_fullcc2.5b.pt) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/qwen3/mc1_l14_224_2.5b/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/qwen3/mc1_l14_224_2.5b/finetune) | |
| | Qwen3-1.7B | [MetaCLIP-400M ViT-L/14 (224)](https://github.com/facebookresearch/MetaCLIP#pre-trained-models) | [encoder](https://dl.fbaipublicfiles.com/MMPT/metaclip/l14_400m.pt) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/qwen3/mc1_l14_224_400m/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/qwen3/mc1_l14_224_400m/finetune) | |
| | Qwen3-1.7B | [SigLIP2 So400m/l14 (384)](https://github.com/google-research/big_vision/blob/main/big_vision/configs/proj/image_text/README_siglip2.md) | [encoder](https://storage.googleapis.com/big_vision/siglip2/siglip2_so400m14_384.npz) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/qwen3/siglip2_sm14_384/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/qwen3/siglip2_sm14_384/finetune) | |
| | Qwen3-1.7B | [SigLIP2 So400m/l14 (224)](https://github.com/google-research/big_vision/blob/main/big_vision/configs/proj/image_text/README_siglip2.md) | [encoder](https://storage.googleapis.com/big_vision/siglip2/siglip2_so400m14_224.npz) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/qwen3/siglip2_sm14_224/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/qwen3/siglip2_sm14_224/finetune) | |
| | Qwen3-1.7B | [SigLIP2 So400m/l16 (512)](https://github.com/google-research/big_vision/blob/main/big_vision/configs/proj/image_text/README_siglip2.md) | [encoder](https://storage.googleapis.com/big_vision/siglip2/siglip2_so400m16_512.npz) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/qwen3/siglip2_sm16_512/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/qwen3/siglip2_sm16_512/finetune) | |
| | Qwen3-1.7B | [SigLIP2 So400m/l16 (384)](https://github.com/google-research/big_vision/blob/main/big_vision/configs/proj/image_text/README_siglip2.md) | [encoder](https://storage.googleapis.com/big_vision/siglip2/siglip2_so400m16_384.npz) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/qwen3/siglip2_sm16_384/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/qwen3/siglip2_sm16_384/finetune) | |
| | Qwen3-1.7B | [SigLIP2 So400m/l16 (256)](https://github.com/google-research/big_vision/blob/main/big_vision/configs/proj/image_text/README_siglip2.md) | [encoder](https://storage.googleapis.com/big_vision/siglip2/siglip2_so400m16_256.npz) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/qwen3/siglip2_sm16_256/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/qwen3/siglip2_sm16_256/finetune) | |
| | Qwen3-1.7B | [PE-Lang-L/14 (448)](https://huggingface.co/facebook/PE-Lang-L14-448) | [encoder](https://huggingface.co/facebook/PE-Lang-L14-448/resolve/main/PE-Lang-L14-448.pt?download=true) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/qwen3/pe_lang_l14_448/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/qwen3/pe_lang_l14_448/finetune) | |
| | Qwen3-1.7B | [SigLIP2 ViT-L/16 (512)](https://github.com/google-research/big_vision/blob/main/big_vision/configs/proj/image_text/README_siglip2.md) | [encoder](https://storage.googleapis.com/big_vision/siglip2/siglip2_l16_512.npz) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/qwen3/siglip2_l16_512/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/qwen3/siglip2_l16_512/finetune) | |
| | Qwen3-1.7B | [SigLIP2 ViT-L/16 (384)](https://github.com/google-research/big_vision/blob/main/big_vision/configs/proj/image_text/README_siglip2.md) | [encoder](https://storage.googleapis.com/big_vision/siglip2/siglip2_l16_384.npz) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/qwen3/siglip2_l16_384/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/qwen3/siglip2_l16_384/finetune) | |
| | Qwen3-1.7B | [SigLIP2 ViT-L/16 (256)](https://github.com/google-research/big_vision/blob/main/big_vision/configs/proj/image_text/README_siglip2.md) | [encoder](https://storage.googleapis.com/big_vision/siglip2/siglip2_l16_256.npz) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/qwen3/siglip2_l16_256/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/qwen3/siglip2_l16_256/finetune) | |
| | Qwen3-1.7B | [MetaCLIP 2 ViT-M/16 (384)](https://github.com/facebookresearch/MetaCLIP#pre-trained-models) | [encoder](https://dl.fbaipublicfiles.com/MMPT/metaclip/metaclip2_m16_384px_worldwide.pt) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/qwen3/mc2_m16_384/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/qwen3/mc2_m16_384/finetune) | |
| | Qwen3-1.7B | [MetaCLIP 2 ViT-M/16 (224)](https://github.com/facebookresearch/MetaCLIP#pre-trained-models) | [encoder](https://dl.fbaipublicfiles.com/MMPT/metaclip/metaclip2_m16_224px_worldwide.pt) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/qwen3/mc2_m16_224/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/qwen3/mc2_m16_224/finetune) | |
| | Qwen3-1.7B | [MetaCLIP 2-mT5 ViT-M/16 (224)](https://github.com/facebookresearch/MetaCLIP#pre-trained-models) | [encoder](https://dl.fbaipublicfiles.com/MMPT/metaclip/metaclip2_m16_224px_mt5_worldwide.pt) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/qwen3/mc2_m16_224_mt5/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/qwen3/mc2_m16_224_mt5/finetune) | |
| | Qwen3-1.7B | [MetaCLIP 2 ViT-B/16 (384)](https://github.com/facebookresearch/MetaCLIP#pre-trained-models) | [encoder](https://dl.fbaipublicfiles.com/MMPT/metaclip/metaclip2_b16_384px_worldwide.pt) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/qwen3/mc2_b16_384/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/qwen3/mc2_b16_384/finetune) | |
| | Qwen3-1.7B | [MetaCLIP 2 ViT-B/16 (224)](https://github.com/facebookresearch/MetaCLIP#pre-trained-models) | [encoder](https://dl.fbaipublicfiles.com/MMPT/metaclip/metaclip2_b16_224px_worldwide.pt) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/qwen3/mc2_b16_224/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/qwen3/mc2_b16_224/finetune) | |
| | Qwen3-1.7B | [MetaCLIP 2 ViT-B/32 (384)](https://github.com/facebookresearch/MetaCLIP#pre-trained-models) | [encoder](https://dl.fbaipublicfiles.com/MMPT/metaclip/metaclip2_b32_384px_worldwide.pt) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/qwen3/mc2_b32_384/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/qwen3/mc2_b32_384/finetune) | |
| | Qwen3-1.7B | [MetaCLIP 2 ViT-B/32 (224)](https://github.com/facebookresearch/MetaCLIP#pre-trained-models) | [encoder](https://dl.fbaipublicfiles.com/MMPT/metaclip/metaclip2_b32_224px_worldwide.pt) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/qwen3/mc2_b32_224/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/qwen3/mc2_b32_224/finetune) | |
| | Qwen3-1.7B | [MetaCLIP 2-mT5 ViT-B/32 (224)](https://github.com/facebookresearch/MetaCLIP#pre-trained-models) | [encoder](https://dl.fbaipublicfiles.com/MMPT/metaclip/metaclip2_b32_224px_mt5_worldwide.pt) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/qwen3/mc2_b32_224_mt5/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/qwen3/mc2_b32_224_mt5/finetune) | |
| | Qwen3-1.7B | [MetaCLIP-2.5B ViT-B/16 (224)](https://github.com/facebookresearch/MetaCLIP#pre-trained-models) | [encoder](https://dl.fbaipublicfiles.com/MMPT/metaclip/b16_fullcc2.5b.pt) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/qwen3/mc1_b16_224_2.5b/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/qwen3/mc1_b16_224_2.5b/finetune) | |
| | Qwen3-1.7B | [MetaCLIP-2.5B ViT-B/32 (224)](https://github.com/facebookresearch/MetaCLIP#pre-trained-models) | [encoder](https://dl.fbaipublicfiles.com/MMPT/metaclip/b32_fullcc2.5b.pt) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/qwen3/mc1_b32_224_2.5b/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/qwen3/mc1_b32_224_2.5b/finetune) | |
| | Qwen3-1.7B | [MetaCLIP-400M ViT-B/16 (224)](https://github.com/facebookresearch/MetaCLIP#pre-trained-models) | [encoder](https://dl.fbaipublicfiles.com/MMPT/metaclip/b16_400m.pt) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/qwen3/mc1_b16_224_400m/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/qwen3/mc1_b16_224_400m/finetune) | |
| | Qwen3-1.7B | [MetaCLIP-400M ViT-B/32 (224)](https://github.com/facebookresearch/MetaCLIP#pre-trained-models) | [encoder](https://dl.fbaipublicfiles.com/MMPT/metaclip/b32_400m.pt) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/qwen3/mc1_b32_224_400m/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/qwen3/mc1_b32_224_400m/finetune) | |
| | Qwen3-1.7B | [PE-Core-B/16 (224)](https://huggingface.co/facebook/PE-Core-B16-224) | [encoder](https://huggingface.co/facebook/PE-Core-B16-224/resolve/main/PE-Core-B16-224.pt?download=true) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/qwen3/pe_core_b16_224/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/qwen3/pe_core_b16_224/finetune) | |
| | Qwen3-1.7B | [SigLIP2 ViT-B/16 (512)](https://github.com/google-research/big_vision/blob/main/big_vision/configs/proj/image_text/README_siglip2.md) | [encoder](https://storage.googleapis.com/big_vision/siglip2/siglip2_b16_512.npz) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/qwen3/siglip2_b16_512/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/qwen3/siglip2_b16_512/finetune) | |
| | Qwen3-1.7B | [SigLIP2 ViT-B/16 (384)](https://github.com/google-research/big_vision/blob/main/big_vision/configs/proj/image_text/README_siglip2.md) | [encoder](https://storage.googleapis.com/big_vision/siglip2/siglip2_b16_384.npz) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/qwen3/siglip2_b16_384/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/qwen3/siglip2_b16_384/finetune) | |
| | Qwen3-1.7B | [SigLIP2 ViT-B/16 (256)](https://github.com/google-research/big_vision/blob/main/big_vision/configs/proj/image_text/README_siglip2.md) | [encoder](https://storage.googleapis.com/big_vision/siglip2/siglip2_b16_256.npz) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/qwen3/siglip2_b16_256/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/qwen3/siglip2_b16_256/finetune) | |
| | Qwen3-1.7B | [SigLIP2 ViT-B/16 (224)](https://github.com/google-research/big_vision/blob/main/big_vision/configs/proj/image_text/README_siglip2.md) | [encoder](https://storage.googleapis.com/big_vision/siglip2/siglip2_b16_224.npz) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/qwen3/siglip2_b16_224/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/qwen3/siglip2_b16_224/finetune) | |
| | Qwen3-1.7B | [SigLIP2 ViT-B/32 (256)](https://github.com/google-research/big_vision/blob/main/big_vision/configs/proj/image_text/README_siglip2.md) | [encoder](https://storage.googleapis.com/big_vision/siglip2/siglip2_b32_256.npz) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/qwen3/siglip2_b32_256/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/qwen3/siglip2_b32_256/finetune) | |
| | Qwen3-1.7B | [MetaCLIP 2 ViT-S/16 (384)](https://github.com/facebookresearch/MetaCLIP#pre-trained-models) | [encoder](https://dl.fbaipublicfiles.com/MMPT/metaclip/metaclip2_s16_384px_worldwide.pt) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/qwen3/mc2_s16_384/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/qwen3/mc2_s16_384/finetune) | |
| | Qwen3-1.7B | [MetaCLIP 2 ViT-S/16 (224)](https://github.com/facebookresearch/MetaCLIP#pre-trained-models) | [encoder](https://dl.fbaipublicfiles.com/MMPT/metaclip/metaclip2_s16_224px_worldwide.pt) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/qwen3/mc2_s16_224/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/qwen3/mc2_s16_224/finetune) | |
| | Qwen3-1.7B | [MetaCLIP 2-mT5 ViT-S/16 (224)](https://github.com/facebookresearch/MetaCLIP#pre-trained-models) | [encoder](https://dl.fbaipublicfiles.com/MMPT/metaclip/metaclip2_s16_224px_mt5_worldwide.pt) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/qwen3/mc2_s16_224_mt5/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/qwen3/mc2_s16_224_mt5/finetune) | |
| | Qwen3-1.7B | [Web-SSL MAE 3B (224)](https://huggingface.co/facebook/webssl-mae3b-full2b-224) | [encoder files](https://huggingface.co/facebook/webssl-mae3b-full2b-224) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/qwen3/webssl_mae3b_full2b_224/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/qwen3/webssl_mae3b_full2b_224/finetune) | |
| | Qwen3-1.7B | [DINOv2 ViT-G/14](https://huggingface.co/facebook/dinov2-giant) | [encoder](https://huggingface.co/facebook/dinov2-giant/resolve/main/model.safetensors?download=true) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/qwen3/dinov2_giant/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/qwen3/dinov2_giant/finetune) | |
| | Qwen3-1.7B | [Web-SSL DINO 1B (224)](https://huggingface.co/facebook/webssl-dino1b-full2b-224) | [encoder files](https://huggingface.co/facebook/webssl-dino1b-full2b-224) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/qwen3/webssl_dino1b_full2b_224/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/qwen3/webssl_dino1b_full2b_224/finetune) | |
| | Qwen3-1.7B | [Web-SSL MAE 1B (224)](https://huggingface.co/facebook/webssl-mae1b-full2b-224) | [encoder](https://huggingface.co/facebook/webssl-mae1b-full2b-224/resolve/main/model.safetensors?download=true) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/qwen3/webssl_mae1b_full2b_224/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/qwen3/webssl_mae1b_full2b_224/finetune) | |
| | Qwen3-1.7B | [Pixio ViT-H/16](https://huggingface.co/facebook/pixio-vith16) | [encoder](https://huggingface.co/facebook/pixio-vith16/resolve/main/pixio_vith16.pth?download=true) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/qwen3/pixio_vith16/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/qwen3/pixio_vith16/finetune) | |
| | Qwen3-1.7B | [I-JEPA ViT-H/14](https://github.com/facebookresearch/ijepa) | [encoder](https://dl.fbaipublicfiles.com/ijepa/IN1K-vit.h.14-300e.pth.tar) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/qwen3/ijepa_vith14/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/qwen3/ijepa_vith14/finetune) | |
| | Qwen3-1.7B | [DINOv2 ViT-L/14](https://huggingface.co/facebook/dinov2-large) | [encoder](https://huggingface.co/facebook/dinov2-large/resolve/main/model.safetensors?download=true) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/qwen3/dinov2_large/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/qwen3/dinov2_large/finetune) | |
| | Qwen3-1.7B | [DINOv3 ViT-L/16](https://huggingface.co/facebook/dinov3-vitl16-pretrain-lvd1689m) | [encoder](https://huggingface.co/facebook/dinov3-vitl16-pretrain-lvd1689m/resolve/main/model.safetensors?download=true) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/qwen3/dinov3_vitl16/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/qwen3/dinov3_vitl16/finetune) | |
| | Qwen3-1.7B | [Pixio ViT-L/16](https://huggingface.co/facebook/pixio-vitl16) | [encoder](https://huggingface.co/facebook/pixio-vitl16/resolve/main/pixio_vitl16.pth?download=true) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/qwen3/pixio_vitl16/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/qwen3/pixio_vitl16/finetune) | |
| | Qwen3-1.7B | [RAEv2 DINOv3-L (k=7)](https://github.com/nanovisionx/RAEv2) | [DINOv3 backbone](https://huggingface.co/facebook/dinov3-vitl16-pretrain-lvd1689m/resolve/main/model.safetensors?download=true) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/qwen3/raev2_dinov3l_k7/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/qwen3/raev2_dinov3l_k7/finetune) | |
| | Qwen3-1.7B | [Web-SSL MAE 300M (224)](https://huggingface.co/facebook/webssl-mae300m-full2b-224) | [encoder](https://huggingface.co/facebook/webssl-mae300m-full2b-224/resolve/main/model.safetensors?download=true) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/qwen3/webssl_mae300m_full2b_224/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/qwen3/webssl_mae300m_full2b_224/finetune) | |
| | Qwen3-1.7B | [EUPE ConvNeXt-B](https://huggingface.co/facebook/EUPE-ConvNeXt-B) | [encoder](https://huggingface.co/facebook/EUPE-ConvNeXt-B/resolve/main/EUPE-ConvNeXt-B.pt?download=true) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/qwen3/eupe_convnext_b/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/qwen3/eupe_convnext_b/finetune) | |
| | Qwen3-1.7B | [DINOv2 ViT-B/14](https://huggingface.co/facebook/dinov2-base) | [encoder](https://huggingface.co/facebook/dinov2-base/resolve/main/model.safetensors?download=true) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/qwen3/dinov2_base/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/qwen3/dinov2_base/finetune) | |
| | Qwen3-1.7B | [DINO ViT-B/8](https://huggingface.co/facebook/dino-vitb8) | [encoder](https://huggingface.co/facebook/dino-vitb8/resolve/main/model.safetensors?download=true) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/qwen3/dino_vitb8/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/qwen3/dino_vitb8/finetune) | |
| | Qwen3-1.7B | [DINO ViT-B/16](https://huggingface.co/facebook/dino-vitb16) | [encoder](https://huggingface.co/facebook/dino-vitb16/resolve/main/pytorch_model.bin?download=true) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/qwen3/dino_vitb16/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/qwen3/dino_vitb16/finetune) | |
| | Qwen3-1.7B | [EUPE ViT-B](https://huggingface.co/facebook/EUPE-ViT-B) | [encoder](https://huggingface.co/facebook/EUPE-ViT-B/resolve/main/EUPE-ViT-B.pt?download=true) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/qwen3/eupe_vit_b/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/qwen3/eupe_vit_b/finetune) | |
| | Qwen3-1.7B | [Pixio ViT-B/16](https://huggingface.co/facebook/pixio-vitb16) | [encoder](https://huggingface.co/facebook/pixio-vitb16/resolve/main/pixio_vitb16.pth?download=true) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/qwen3/pixio_vitb16/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/qwen3/pixio_vitb16/finetune) | |
| | Qwen3-1.7B | [DINOv2 ViT-S/14](https://huggingface.co/facebook/dinov2-small) | [encoder](https://huggingface.co/facebook/dinov2-small/resolve/main/model.safetensors?download=true) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/qwen3/dinov2_small/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/qwen3/dinov2_small/finetune) | |
| | Qwen3-1.7B | [DINO ViT-S/16](https://huggingface.co/facebook/dino-vits16) | [encoder](https://huggingface.co/facebook/dino-vits16/resolve/main/pytorch_model.bin?download=true) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/qwen3/dino_vits16/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/qwen3/dino_vits16/finetune) | |
| | Qwen3-1.7B | [EUPE ViT-S](https://huggingface.co/facebook/EUPE-ViT-S) | [encoder](https://huggingface.co/facebook/EUPE-ViT-S/resolve/main/EUPE-ViT-S.pt?download=true) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/qwen3/eupe_vit_s/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/qwen3/eupe_vit_s/finetune) | |
| | Qwen3-1.7B | [DINO ViT-S/8](https://huggingface.co/facebook/dino-vits8) | [encoder](https://huggingface.co/facebook/dino-vits8/resolve/main/model.safetensors?download=true) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/qwen3/dino_vits8/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/qwen3/dino_vits8/finetune) | |
| | Qwen3-1.7B | [EUPE ViT-T](https://huggingface.co/facebook/EUPE-ViT-T) | [encoder](https://huggingface.co/facebook/EUPE-ViT-T/resolve/main/EUPE-ViT-T.pt?download=true) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/qwen3/eupe_vit_t/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/qwen3/eupe_vit_t/finetune) | |
| |
| |
| ### [SmolLM2-1.7B-Instruct](https://huggingface.co/HuggingFaceTB/SmolLM2-1.7B-Instruct) |
| |
| | Base LLM | Vision Encoder / Tokenizer | Encoder weights | Stage 1 Pretrained weights | Stage 2 Finetuned weights | |
| | --- | --- | --- | --- | --- | |
| | SmolLM2-1.7B-Instruct | [PE-Core-G/14 (448)](https://huggingface.co/facebook/PE-Core-G14-448) | [encoder](https://huggingface.co/facebook/PE-Core-G14-448/resolve/main/PE-Core-G14-448.pt?download=true) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/smollm2/pe_core_g14_448/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/smollm2/pe_core_g14_448/finetune) | |
| | SmolLM2-1.7B-Instruct | [MetaCLIP 2 ViT-G/14 (378)](https://github.com/facebookresearch/MetaCLIP#pre-trained-models) | [encoder](https://dl.fbaipublicfiles.com/MMPT/metaclip/metaclip2_bigG14_378px_worldwide.pt) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/smollm2/mc2_g14_378/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/smollm2/mc2_g14_378/finetune) | |
| | SmolLM2-1.7B-Instruct | [MetaCLIP 2 ViT-G/14 (224)](https://github.com/facebookresearch/MetaCLIP#pre-trained-models) | [encoder](https://dl.fbaipublicfiles.com/MMPT/metaclip/metaclip2_bigG14_224px_worldwide.pt) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/smollm2/mc2_g14_224/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/smollm2/mc2_g14_224/finetune) | |
| | SmolLM2-1.7B-Instruct | [MetaCLIP-2.5B ViT-G/14 (224)](https://github.com/facebookresearch/MetaCLIP#pre-trained-models) | [encoder](https://dl.fbaipublicfiles.com/MMPT/metaclip/G14_fullcc2.5b.pt) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/smollm2/mc1_g14_224_2.5b/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/smollm2/mc1_g14_224_2.5b/finetune) | |
| | SmolLM2-1.7B-Instruct | [SigLIP2 ViT-G/16 (384)](https://github.com/google-research/big_vision/blob/main/big_vision/configs/proj/image_text/README_siglip2.md) | [encoder](https://storage.googleapis.com/big_vision/siglip2/siglip2_g-opt16_384.npz) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/smollm2/siglip2_g16_384/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/smollm2/siglip2_g16_384/finetune) | |
| | SmolLM2-1.7B-Instruct | [SigLIP2 ViT-G/16 (256)](https://github.com/google-research/big_vision/blob/main/big_vision/configs/proj/image_text/README_siglip2.md) | [encoder](https://storage.googleapis.com/big_vision/siglip2/siglip2_g-opt16_256.npz) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/smollm2/siglip2_g16_256/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/smollm2/siglip2_g16_256/finetune) | |
| | SmolLM2-1.7B-Instruct | [MetaCLIP 2 ViT-H/14 (378)](https://github.com/facebookresearch/MetaCLIP#pre-trained-models) | [encoder](https://dl.fbaipublicfiles.com/MMPT/metaclip/metaclip2_h14_378px_worldwide.pt) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/smollm2/mc2_h14_378/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/smollm2/mc2_h14_378/finetune) | |
| | SmolLM2-1.7B-Instruct | [MetaCLIP-2.5B ViT-H/14 (224)](https://github.com/facebookresearch/MetaCLIP#pre-trained-models) | [encoder](https://dl.fbaipublicfiles.com/MMPT/metaclip/h14_fullcc2.5b.pt) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/smollm2/mc1_h14_224_2.5b/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/smollm2/mc1_h14_224_2.5b/finetune) | |
| | SmolLM2-1.7B-Instruct | [MetaCLIP-v1.2 ViT-H/14 (224)](https://github.com/facebookresearch/MetaCLIP#pre-trained-models) | [encoder](https://dl.fbaipublicfiles.com/MMPT/metaclip/h14_v1.2_altogether.pt) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/smollm2/mc1_h14_224_v1.2/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/smollm2/mc1_h14_224_v1.2/finetune) | |
| | SmolLM2-1.7B-Instruct | [OpenAI CLIP ViT-L/14 (224)](https://huggingface.co/openai/clip-vit-large-patch14) | [encoder](https://huggingface.co/openai/clip-vit-large-patch14/resolve/main/model.safetensors?download=true) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/smollm2/clip_openai__l14/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/smollm2/clip_openai__l14/finetune) | |
| | SmolLM2-1.7B-Instruct | [MetaCLIP 2 ViT-L/14 (224)](https://github.com/facebookresearch/MetaCLIP#pre-trained-models) | [encoder](https://dl.fbaipublicfiles.com/MMPT/metaclip/metaclip2_l14_224px_worldwide.pt) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/smollm2/mc2_l14_224/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/smollm2/mc2_l14_224/finetune) | |
| | SmolLM2-1.7B-Instruct | [MetaCLIP-2.5B ViT-L/14 (224)](https://github.com/facebookresearch/MetaCLIP#pre-trained-models) | [encoder](https://dl.fbaipublicfiles.com/MMPT/metaclip/l14_fullcc2.5b.pt) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/smollm2/mc1_l14_224_2.5b/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/smollm2/mc1_l14_224_2.5b/finetune) | |
| | SmolLM2-1.7B-Instruct | [MetaCLIP-400M ViT-L/14 (224)](https://github.com/facebookresearch/MetaCLIP#pre-trained-models) | [encoder](https://dl.fbaipublicfiles.com/MMPT/metaclip/l14_400m.pt) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/smollm2/mc1_l14_224_400m/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/smollm2/mc1_l14_224_400m/finetune) | |
| | SmolLM2-1.7B-Instruct | [SigLIP2 So400m/l14 (384)](https://github.com/google-research/big_vision/blob/main/big_vision/configs/proj/image_text/README_siglip2.md) | [encoder](https://storage.googleapis.com/big_vision/siglip2/siglip2_so400m14_384.npz) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/smollm2/siglip2_sm14_384/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/smollm2/siglip2_sm14_384/finetune) | |
| | SmolLM2-1.7B-Instruct | [SigLIP2 So400m/l14 (224)](https://github.com/google-research/big_vision/blob/main/big_vision/configs/proj/image_text/README_siglip2.md) | [encoder](https://storage.googleapis.com/big_vision/siglip2/siglip2_so400m14_224.npz) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/smollm2/siglip2_sm14_224/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/smollm2/siglip2_sm14_224/finetune) | |
| | SmolLM2-1.7B-Instruct | [SigLIP2 So400m/l16 (512)](https://github.com/google-research/big_vision/blob/main/big_vision/configs/proj/image_text/README_siglip2.md) | [encoder](https://storage.googleapis.com/big_vision/siglip2/siglip2_so400m16_512.npz) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/smollm2/siglip2_sm16_512/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/smollm2/siglip2_sm16_512/finetune) | |
| | SmolLM2-1.7B-Instruct | [SigLIP2 So400m/l16 (384)](https://github.com/google-research/big_vision/blob/main/big_vision/configs/proj/image_text/README_siglip2.md) | [encoder](https://storage.googleapis.com/big_vision/siglip2/siglip2_so400m16_384.npz) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/smollm2/siglip2_sm16_384/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/smollm2/siglip2_sm16_384/finetune) | |
| | SmolLM2-1.7B-Instruct | [SigLIP2 So400m/l16 (256)](https://github.com/google-research/big_vision/blob/main/big_vision/configs/proj/image_text/README_siglip2.md) | [encoder](https://storage.googleapis.com/big_vision/siglip2/siglip2_so400m16_256.npz) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/smollm2/siglip2_sm16_256/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/smollm2/siglip2_sm16_256/finetune) | |
| | SmolLM2-1.7B-Instruct | [PE-Lang-L/14 (448)](https://huggingface.co/facebook/PE-Lang-L14-448) | [encoder](https://huggingface.co/facebook/PE-Lang-L14-448/resolve/main/PE-Lang-L14-448.pt?download=true) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/smollm2/pe_lang_l14_448/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/smollm2/pe_lang_l14_448/finetune) | |
| | SmolLM2-1.7B-Instruct | [SigLIP2 ViT-L/16 (512)](https://github.com/google-research/big_vision/blob/main/big_vision/configs/proj/image_text/README_siglip2.md) | [encoder](https://storage.googleapis.com/big_vision/siglip2/siglip2_l16_512.npz) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/smollm2/siglip2_l16_512/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/smollm2/siglip2_l16_512/finetune) | |
| | SmolLM2-1.7B-Instruct | [SigLIP2 ViT-L/16 (384)](https://github.com/google-research/big_vision/blob/main/big_vision/configs/proj/image_text/README_siglip2.md) | [encoder](https://storage.googleapis.com/big_vision/siglip2/siglip2_l16_384.npz) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/smollm2/siglip2_l16_384/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/smollm2/siglip2_l16_384/finetune) | |
| | SmolLM2-1.7B-Instruct | [SigLIP2 ViT-L/16 (256)](https://github.com/google-research/big_vision/blob/main/big_vision/configs/proj/image_text/README_siglip2.md) | [encoder](https://storage.googleapis.com/big_vision/siglip2/siglip2_l16_256.npz) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/smollm2/siglip2_l16_256/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/smollm2/siglip2_l16_256/finetune) | |
| | SmolLM2-1.7B-Instruct | [MetaCLIP 2 ViT-M/16 (384)](https://github.com/facebookresearch/MetaCLIP#pre-trained-models) | [encoder](https://dl.fbaipublicfiles.com/MMPT/metaclip/metaclip2_m16_384px_worldwide.pt) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/smollm2/mc2_m16_384/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/smollm2/mc2_m16_384/finetune) | |
| | SmolLM2-1.7B-Instruct | [MetaCLIP 2 ViT-M/16 (224)](https://github.com/facebookresearch/MetaCLIP#pre-trained-models) | [encoder](https://dl.fbaipublicfiles.com/MMPT/metaclip/metaclip2_m16_224px_worldwide.pt) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/smollm2/mc2_m16_224/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/smollm2/mc2_m16_224/finetune) | |
| | SmolLM2-1.7B-Instruct | [MetaCLIP 2-mT5 ViT-M/16 (224)](https://github.com/facebookresearch/MetaCLIP#pre-trained-models) | [encoder](https://dl.fbaipublicfiles.com/MMPT/metaclip/metaclip2_m16_224px_mt5_worldwide.pt) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/smollm2/mc2_m16_224_mt5/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/smollm2/mc2_m16_224_mt5/finetune) | |
| | SmolLM2-1.7B-Instruct | [MetaCLIP 2 ViT-B/16 (384)](https://github.com/facebookresearch/MetaCLIP#pre-trained-models) | [encoder](https://dl.fbaipublicfiles.com/MMPT/metaclip/metaclip2_b16_384px_worldwide.pt) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/smollm2/mc2_b16_384/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/smollm2/mc2_b16_384/finetune) | |
| | SmolLM2-1.7B-Instruct | [MetaCLIP 2 ViT-B/16 (224)](https://github.com/facebookresearch/MetaCLIP#pre-trained-models) | [encoder](https://dl.fbaipublicfiles.com/MMPT/metaclip/metaclip2_b16_224px_worldwide.pt) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/smollm2/mc2_b16_224/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/smollm2/mc2_b16_224/finetune) | |
| | SmolLM2-1.7B-Instruct | [MetaCLIP 2 ViT-B/32 (384)](https://github.com/facebookresearch/MetaCLIP#pre-trained-models) | [encoder](https://dl.fbaipublicfiles.com/MMPT/metaclip/metaclip2_b32_384px_worldwide.pt) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/smollm2/mc2_b32_384/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/smollm2/mc2_b32_384/finetune) | |
| | SmolLM2-1.7B-Instruct | [MetaCLIP 2 ViT-B/32 (224)](https://github.com/facebookresearch/MetaCLIP#pre-trained-models) | [encoder](https://dl.fbaipublicfiles.com/MMPT/metaclip/metaclip2_b32_224px_worldwide.pt) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/smollm2/mc2_b32_224/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/smollm2/mc2_b32_224/finetune) | |
| | SmolLM2-1.7B-Instruct | [MetaCLIP 2-mT5 ViT-B/32 (224)](https://github.com/facebookresearch/MetaCLIP#pre-trained-models) | [encoder](https://dl.fbaipublicfiles.com/MMPT/metaclip/metaclip2_b32_224px_mt5_worldwide.pt) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/smollm2/mc2_b32_224_mt5/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/smollm2/mc2_b32_224_mt5/finetune) | |
| | SmolLM2-1.7B-Instruct | [MetaCLIP-2.5B ViT-B/16 (224)](https://github.com/facebookresearch/MetaCLIP#pre-trained-models) | [encoder](https://dl.fbaipublicfiles.com/MMPT/metaclip/b16_fullcc2.5b.pt) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/smollm2/mc1_b16_224_2.5b/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/smollm2/mc1_b16_224_2.5b/finetune) | |
| | SmolLM2-1.7B-Instruct | [MetaCLIP-2.5B ViT-B/32 (224)](https://github.com/facebookresearch/MetaCLIP#pre-trained-models) | [encoder](https://dl.fbaipublicfiles.com/MMPT/metaclip/b32_fullcc2.5b.pt) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/smollm2/mc1_b32_224_2.5b/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/smollm2/mc1_b32_224_2.5b/finetune) | |
| | SmolLM2-1.7B-Instruct | [MetaCLIP-400M ViT-B/16 (224)](https://github.com/facebookresearch/MetaCLIP#pre-trained-models) | [encoder](https://dl.fbaipublicfiles.com/MMPT/metaclip/b16_400m.pt) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/smollm2/mc1_b16_224_400m/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/smollm2/mc1_b16_224_400m/finetune) | |
| | SmolLM2-1.7B-Instruct | [MetaCLIP-400M ViT-B/32 (224)](https://github.com/facebookresearch/MetaCLIP#pre-trained-models) | [encoder](https://dl.fbaipublicfiles.com/MMPT/metaclip/b32_400m.pt) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/smollm2/mc1_b32_224_400m/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/smollm2/mc1_b32_224_400m/finetune) | |
| | SmolLM2-1.7B-Instruct | [PE-Core-B/16 (224)](https://huggingface.co/facebook/PE-Core-B16-224) | [encoder](https://huggingface.co/facebook/PE-Core-B16-224/resolve/main/PE-Core-B16-224.pt?download=true) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/smollm2/pe_core_b16_224/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/smollm2/pe_core_b16_224/finetune) | |
| | SmolLM2-1.7B-Instruct | [SigLIP2 ViT-B/16 (512)](https://github.com/google-research/big_vision/blob/main/big_vision/configs/proj/image_text/README_siglip2.md) | [encoder](https://storage.googleapis.com/big_vision/siglip2/siglip2_b16_512.npz) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/smollm2/siglip2_b16_512/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/smollm2/siglip2_b16_512/finetune) | |
| | SmolLM2-1.7B-Instruct | [SigLIP2 ViT-B/16 (384)](https://github.com/google-research/big_vision/blob/main/big_vision/configs/proj/image_text/README_siglip2.md) | [encoder](https://storage.googleapis.com/big_vision/siglip2/siglip2_b16_384.npz) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/smollm2/siglip2_b16_384/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/smollm2/siglip2_b16_384/finetune) | |
| | SmolLM2-1.7B-Instruct | [SigLIP2 ViT-B/16 (256)](https://github.com/google-research/big_vision/blob/main/big_vision/configs/proj/image_text/README_siglip2.md) | [encoder](https://storage.googleapis.com/big_vision/siglip2/siglip2_b16_256.npz) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/smollm2/siglip2_b16_256/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/smollm2/siglip2_b16_256/finetune) | |
| | SmolLM2-1.7B-Instruct | [SigLIP2 ViT-B/16 (224)](https://github.com/google-research/big_vision/blob/main/big_vision/configs/proj/image_text/README_siglip2.md) | [encoder](https://storage.googleapis.com/big_vision/siglip2/siglip2_b16_224.npz) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/smollm2/siglip2_b16_224/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/smollm2/siglip2_b16_224/finetune) | |
| | SmolLM2-1.7B-Instruct | [SigLIP2 ViT-B/32 (256)](https://github.com/google-research/big_vision/blob/main/big_vision/configs/proj/image_text/README_siglip2.md) | [encoder](https://storage.googleapis.com/big_vision/siglip2/siglip2_b32_256.npz) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/smollm2/siglip2_b32_256/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/smollm2/siglip2_b32_256/finetune) | |
| | SmolLM2-1.7B-Instruct | [MetaCLIP 2 ViT-S/16 (384)](https://github.com/facebookresearch/MetaCLIP#pre-trained-models) | [encoder](https://dl.fbaipublicfiles.com/MMPT/metaclip/metaclip2_s16_384px_worldwide.pt) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/smollm2/mc2_s16_384/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/smollm2/mc2_s16_384/finetune) | |
| | SmolLM2-1.7B-Instruct | [MetaCLIP 2 ViT-S/16 (224)](https://github.com/facebookresearch/MetaCLIP#pre-trained-models) | [encoder](https://dl.fbaipublicfiles.com/MMPT/metaclip/metaclip2_s16_224px_worldwide.pt) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/smollm2/mc2_s16_224/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/smollm2/mc2_s16_224/finetune) | |
| | SmolLM2-1.7B-Instruct | [MetaCLIP 2-mT5 ViT-S/16 (224)](https://github.com/facebookresearch/MetaCLIP#pre-trained-models) | [encoder](https://dl.fbaipublicfiles.com/MMPT/metaclip/metaclip2_s16_224px_mt5_worldwide.pt) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/smollm2/mc2_s16_224_mt5/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/smollm2/mc2_s16_224_mt5/finetune) | |
| | SmolLM2-1.7B-Instruct | [Web-SSL MAE 3B (224)](https://huggingface.co/facebook/webssl-mae3b-full2b-224) | [encoder files](https://huggingface.co/facebook/webssl-mae3b-full2b-224) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/smollm2/webssl_mae3b_full2b_224/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/smollm2/webssl_mae3b_full2b_224/finetune) | |
| | SmolLM2-1.7B-Instruct | [DINOv2 ViT-G/14](https://huggingface.co/facebook/dinov2-giant) | [encoder](https://huggingface.co/facebook/dinov2-giant/resolve/main/model.safetensors?download=true) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/smollm2/dinov2_giant/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/smollm2/dinov2_giant/finetune) | |
| | SmolLM2-1.7B-Instruct | [Web-SSL DINO 1B (224)](https://huggingface.co/facebook/webssl-dino1b-full2b-224) | [encoder files](https://huggingface.co/facebook/webssl-dino1b-full2b-224) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/smollm2/webssl_dino1b_full2b_224/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/smollm2/webssl_dino1b_full2b_224/finetune) | |
| | SmolLM2-1.7B-Instruct | [Web-SSL MAE 1B (224)](https://huggingface.co/facebook/webssl-mae1b-full2b-224) | [encoder](https://huggingface.co/facebook/webssl-mae1b-full2b-224/resolve/main/model.safetensors?download=true) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/smollm2/webssl_mae1b_full2b_224/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/smollm2/webssl_mae1b_full2b_224/finetune) | |
| | SmolLM2-1.7B-Instruct | [Pixio ViT-H/16](https://huggingface.co/facebook/pixio-vith16) | [encoder](https://huggingface.co/facebook/pixio-vith16/resolve/main/pixio_vith16.pth?download=true) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/smollm2/pixio_vith16/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/smollm2/pixio_vith16/finetune) | |
| | SmolLM2-1.7B-Instruct | [I-JEPA ViT-H/14](https://github.com/facebookresearch/ijepa) | [encoder](https://dl.fbaipublicfiles.com/ijepa/IN1K-vit.h.14-300e.pth.tar) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/smollm2/ijepa_vith14/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/smollm2/ijepa_vith14/finetune) | |
| | SmolLM2-1.7B-Instruct | [DINOv2 ViT-L/14](https://huggingface.co/facebook/dinov2-large) | [encoder](https://huggingface.co/facebook/dinov2-large/resolve/main/model.safetensors?download=true) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/smollm2/dinov2_large/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/smollm2/dinov2_large/finetune) | |
| | SmolLM2-1.7B-Instruct | [DINOv3 ViT-L/16](https://huggingface.co/facebook/dinov3-vitl16-pretrain-lvd1689m) | [encoder](https://huggingface.co/facebook/dinov3-vitl16-pretrain-lvd1689m/resolve/main/model.safetensors?download=true) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/smollm2/dinov3_vitl16/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/smollm2/dinov3_vitl16/finetune) | |
| | SmolLM2-1.7B-Instruct | [Pixio ViT-L/16](https://huggingface.co/facebook/pixio-vitl16) | [encoder](https://huggingface.co/facebook/pixio-vitl16/resolve/main/pixio_vitl16.pth?download=true) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/smollm2/pixio_vitl16/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/smollm2/pixio_vitl16/finetune) | |
| | SmolLM2-1.7B-Instruct | [RAEv2 DINOv3-L (k=7)](https://github.com/nanovisionx/RAEv2) | [DINOv3 backbone](https://huggingface.co/facebook/dinov3-vitl16-pretrain-lvd1689m/resolve/main/model.safetensors?download=true) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/smollm2/raev2_dinov3l_k7/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/smollm2/raev2_dinov3l_k7/finetune) | |
| | SmolLM2-1.7B-Instruct | [Web-SSL MAE 300M (224)](https://huggingface.co/facebook/webssl-mae300m-full2b-224) | [encoder](https://huggingface.co/facebook/webssl-mae300m-full2b-224/resolve/main/model.safetensors?download=true) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/smollm2/webssl_mae300m_full2b_224/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/smollm2/webssl_mae300m_full2b_224/finetune) | |
| | SmolLM2-1.7B-Instruct | [EUPE ConvNeXt-B](https://huggingface.co/facebook/EUPE-ConvNeXt-B) | [encoder](https://huggingface.co/facebook/EUPE-ConvNeXt-B/resolve/main/EUPE-ConvNeXt-B.pt?download=true) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/smollm2/eupe_convnext_b/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/smollm2/eupe_convnext_b/finetune) | |
| | SmolLM2-1.7B-Instruct | [DINOv2 ViT-B/14](https://huggingface.co/facebook/dinov2-base) | [encoder](https://huggingface.co/facebook/dinov2-base/resolve/main/model.safetensors?download=true) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/smollm2/dinov2_base/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/smollm2/dinov2_base/finetune) | |
| | SmolLM2-1.7B-Instruct | [DINO ViT-B/8](https://huggingface.co/facebook/dino-vitb8) | [encoder](https://huggingface.co/facebook/dino-vitb8/resolve/main/model.safetensors?download=true) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/smollm2/dino_vitb8/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/smollm2/dino_vitb8/finetune) | |
| | SmolLM2-1.7B-Instruct | [DINO ViT-B/16](https://huggingface.co/facebook/dino-vitb16) | [encoder](https://huggingface.co/facebook/dino-vitb16/resolve/main/pytorch_model.bin?download=true) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/smollm2/dino_vitb16/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/smollm2/dino_vitb16/finetune) | |
| | SmolLM2-1.7B-Instruct | [EUPE ViT-B](https://huggingface.co/facebook/EUPE-ViT-B) | [encoder](https://huggingface.co/facebook/EUPE-ViT-B/resolve/main/EUPE-ViT-B.pt?download=true) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/smollm2/eupe_vit_b/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/smollm2/eupe_vit_b/finetune) | |
| | SmolLM2-1.7B-Instruct | [Pixio ViT-B/16](https://huggingface.co/facebook/pixio-vitb16) | [encoder](https://huggingface.co/facebook/pixio-vitb16/resolve/main/pixio_vitb16.pth?download=true) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/smollm2/pixio_vitb16/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/smollm2/pixio_vitb16/finetune) | |
| | SmolLM2-1.7B-Instruct | [DINOv2 ViT-S/14](https://huggingface.co/facebook/dinov2-small) | [encoder](https://huggingface.co/facebook/dinov2-small/resolve/main/model.safetensors?download=true) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/smollm2/dinov2_small/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/smollm2/dinov2_small/finetune) | |
| | SmolLM2-1.7B-Instruct | [DINO ViT-S/16](https://huggingface.co/facebook/dino-vits16) | [encoder](https://huggingface.co/facebook/dino-vits16/resolve/main/pytorch_model.bin?download=true) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/smollm2/dino_vits16/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/smollm2/dino_vits16/finetune) | |
| | SmolLM2-1.7B-Instruct | [EUPE ViT-S](https://huggingface.co/facebook/EUPE-ViT-S) | [encoder](https://huggingface.co/facebook/EUPE-ViT-S/resolve/main/EUPE-ViT-S.pt?download=true) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/smollm2/eupe_vit_s/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/smollm2/eupe_vit_s/finetune) | |
| | SmolLM2-1.7B-Instruct | [DINO ViT-S/8](https://huggingface.co/facebook/dino-vits8) | [encoder](https://huggingface.co/facebook/dino-vits8/resolve/main/model.safetensors?download=true) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/smollm2/dino_vits8/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/smollm2/dino_vits8/finetune) | |
| | SmolLM2-1.7B-Instruct | [EUPE ViT-T](https://huggingface.co/facebook/EUPE-ViT-T) | [encoder](https://huggingface.co/facebook/EUPE-ViT-T/resolve/main/EUPE-ViT-T.pt?download=true) | [projector](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/resolve/main/continuous/smollm2/eupe_vit_t/pretrain/mm_projector.bin) | [finetuned](https://huggingface.co/336labs/VisionEncoder-to-MLLM-ModelZoo/tree/main/continuous/smollm2/eupe_vit_t/finetune) | |
| |
| |
| ## Citation |
| |
| If you use RAVEL or this model zoo, please cite: |
| |
| ```bibtex |
| @misc{yang2026strong, |
| title={A Strong Baseline for Evaluating Vision Encoders in Multimodal Large Language Models}, |
| author={Yang, Yilin and Tang, Jun-Tao and Wang, Kengyi and Su, Siyuan and Luo, Gaoyong and Chen, Mingda}, |
| year={2026}, |
| eprint={2610.05413}, |
| archivePrefix={arXiv}, |
| primaryClass={cs.CV}, |
| url={https://arxiv.org/abs/2610.05413} |
| } |
| ``` |
| |
