| --- |
| license: mit |
| base_model: |
| - Qwen/Qwen2.5-VL-3B-Instruct |
| - Qwen/Qwen2.5-VL-7B-Instruct |
| tags: |
| - vision-language-model |
| - token-compression |
| - multimodal |
| --- |
| |
| # CRISPR — Checkpoints |
|
|
| Checkpoints for **CRISPR: Context-Refined Information Spatial Pooling with |
| Region-awareness for Efficient Visual Token Compression in VLMs**, accepted |
| at ACM MM 2026. |
|
|
| - Code: https://github.com/ZuyiZhou/CRISPR |
| - Paper DOI: https://doi.org/10.1145/3767308.3835007 |
|
|
| This repo hosts CRISPR checkpoints for the Qwen2.5-VL-3B-Instruct backbone |
| (9x and 16x compression) and the Qwen2.5-VL-7B-Instruct backbone (16x |
| compression). The 7B/9x checkpoint is not currently available (lost prior to |
| this release) and is not planned unless retraining happens in the future. |
|
|
| ## Files |
|
|
| | Path | Compression ratio | Backbone | Notes | |
| |---|---|---|---| |
| | `3b_9x/checkpoint.pt` | 9x (3x3 block) | Qwen2.5-VL-3B-Instruct | Stage-2, best checkpoint by val loss | |
| | `3b_9x/config.json` | | | training config used to produce this checkpoint | |
| | `3b_16x/checkpoint.pt` | 16x (4x4 block) | Qwen2.5-VL-3B-Instruct | Stage-2, best checkpoint by val loss | |
| | `3b_16x/config.json` | | | training config used to produce this checkpoint | |
| | `7b_16x/checkpoint.pt` | 16x (4x4 block) | Qwen2.5-VL-7B-Instruct | Stage-2, best checkpoint by val loss | |
| | `7b_16x/config.json` | | | training config used to produce this checkpoint | |
|
|
| Each `checkpoint.pt` is a plain `torch.save` dict with keys `config`, |
| `token_mixer` (TokenMixer state dict), and `local_c3` (LocalC3 state dict, |
| which also contains the Global Token Fusion sub-module). Only the trainable |
| CRISPR modules are included — the frozen Qwen2.5-VL vision encoder and |
| decoder weights are not part of this checkpoint and must be obtained |
| separately from [Qwen2.5-VL](https://github.com/QwenLM/Qwen2.5-VL). Optimizer/ |
| scheduler state is not included (only the model weights needed for inference |
| or further fine-tuning are provided). |
|
|
| ## Usage |
|
|
| ```python |
| from crispr import create_model_v7 |
| |
| model = create_model_v7(decoder_path="./Qwen/Qwen2.5-VL-3B-Instruct") |
| model.load_checkpoint("3b_9x/checkpoint.pt") # see crispr/model_v7.py for the loader |
| # for the 7B backbone: decoder_path="./Qwen/Qwen2.5-VL-7B-Instruct", checkpoint="7b_16x/checkpoint.pt" |
| ``` |
|
|
| See the main repository (https://github.com/ZuyiZhou/CRISPR) for the model |
| code, training script, and evaluation scripts. |
|
|
| ## Citation |
|
|
| ```bibtex |
| @inproceedings{zhou2026crispr, |
| author = {Zhou, Zuyi and Xue, Dizhan and Qian, Shengsheng and Xu, Changsheng}, |
| title = {CRISPR: Context-Refined Information Spatial Pooling with |
| Region-awareness for Efficient Visual Token Compression in VLMs}, |
| booktitle = {Proceedings of the 34th ACM International Conference on |
| Multimedia (MM '26)}, |
| year = {2026}, |
| publisher = {Association for Computing Machinery}, |
| address = {New York, NY, USA}, |
| doi = {10.1145/3767308.3835007} |
| } |
| ``` |
|
|
