README.md

---
license: mit
base_model:
- Qwen/Qwen2.5-VL-3B-Instruct
- Qwen/Qwen2.5-VL-7B-Instruct
- Qwen/Qwen3.5-4B
- Qwen/Qwen3.5-9B
tags:
- vision-language-model
- token-compression
- multimodal
---

# CRISPR — Checkpoints

Checkpoints for **CRISPR: Context-Refined Information Spatial Pooling with
Region-awareness for Efficient Visual Token Compression in VLMs**, accepted
at ACM MM 2026.

- Code: https://github.com/ZuyiZhou/CRISPR
- Paper DOI: https://doi.org/10.1145/3767308.3835007

The original release contains CRISPR checkpoints for the Qwen2.5-VL-3B-Instruct backbone
(9x and 16x compression) and the Qwen2.5-VL-7B-Instruct backbone (16x
compression). The 7B/9x checkpoint is not currently available (lost prior to
this release) and is not planned unless retraining happens in the future.

## Qwen3.5 checkpoints

Additional CRISPR compressor checkpoints are available for the frozen Qwen3.5
backbones, with a fixed visual-token budget per image.

| Backbone | Visual tokens per image | Checkpoint |
|---|---:|---|
| Qwen3.5-4B | 16 | [`qwen3.5-4b-16tokens`](https://huggingface.co/Rim2000/CRISPR/tree/main/qwen3.5-4b-16tokens) |
| Qwen3.5-4B | 36 | [`qwen3.5-4b-36tokens`](https://huggingface.co/Rim2000/CRISPR/tree/main/qwen3.5-4b-36tokens) |
| Qwen3.5-4B | 64 | [`qwen3.5-4b-64tokens`](https://huggingface.co/Rim2000/CRISPR/tree/main/qwen3.5-4b-64tokens) |
| Qwen3.5-9B | 36 | [`qwen3.5-9b-36tokens`](https://huggingface.co/Rim2000/CRISPR/tree/main/qwen3.5-9b-36tokens) |
| Qwen3.5-9B | 64 | [`qwen3.5-9b-64tokens`](https://huggingface.co/Rim2000/CRISPR/tree/main/qwen3.5-9b-64tokens) |

Each directory contains `model.safetensors` (float32 compressor weights),
`config.json` (inference settings), and `SHA256SUMS`. The frozen Qwen3.5 backbone
is downloaded separately. These files contain only model tensors and inference
metadata; optimizer state and training metadata are excluded.

For the loader and compressor interface, see the
[Qwen3.5 guide](https://github.com/ZuyiZhou/CRISPR/tree/main/qwen35).
The legacy `create_model_v7` loader below applies to Qwen2.5-VL only.
File sizes and checksums are listed in [qwen35_manifest.json](qwen35_manifest.json).

## Qwen2.5-VL checkpoint files

| Path | Compression ratio | Backbone | Notes |
|---|---|---|---|
| `3b_9x/checkpoint.pt` | 9x (3x3 block) | Qwen2.5-VL-3B-Instruct | Stage-2, best checkpoint by val loss |
| `3b_9x/config.json` | | | training config used to produce this checkpoint |
| `3b_16x/checkpoint.pt` | 16x (4x4 block) | Qwen2.5-VL-3B-Instruct | Stage-2, best checkpoint by val loss |
| `3b_16x/config.json` | | | training config used to produce this checkpoint |
| `7b_16x/checkpoint.pt` | 16x (4x4 block) | Qwen2.5-VL-7B-Instruct | Stage-2, best checkpoint by val loss |
| `7b_16x/config.json` | | | training config used to produce this checkpoint |

Each `checkpoint.pt` is a plain `torch.save` dict with keys `config`,
`token_mixer` (TokenMixer state dict), and `local_c3` (LocalC3 state dict,
which also contains the Global Token Fusion sub-module). Only the trainable
CRISPR modules are included — the frozen Qwen2.5-VL vision encoder and
decoder weights are not part of this checkpoint and must be obtained
separately from [Qwen2.5-VL](https://github.com/QwenLM/Qwen2.5-VL). Optimizer/
scheduler state is not included (only the model weights needed for inference
or further fine-tuning are provided).

## Qwen2.5-VL usage

```python
from crispr import create_model_v7

model = create_model_v7(decoder_path="./Qwen/Qwen2.5-VL-3B-Instruct")
model.load_checkpoint("3b_9x/checkpoint.pt")  # see crispr/model_v7.py for the loader
# for the 7B backbone: decoder_path="./Qwen/Qwen2.5-VL-7B-Instruct", checkpoint="7b_16x/checkpoint.pt"
```

See the main repository (https://github.com/ZuyiZhou/CRISPR) for the model
code, training script, and evaluation scripts.

## Citation

```bibtex
@inproceedings{zhou2026crispr,
  author    = {Zhou, Zuyi and Xue, Dizhan and Qian, Shengsheng and Xu, Changsheng},
  title     = {CRISPR: Context-Refined Information Spatial Pooling with
               Region-awareness for Efficient Visual Token Compression in VLMs},
  booktitle = {Proceedings of the 34th ACM International Conference on
               Multimedia (MM '26)},
  year      = {2026},
  publisher = {Association for Computing Machinery},
  address   = {New York, NY, USA},
  doi       = {10.1145/3767308.3835007}
}
```