---
license: apache-2.0
base_model:
- Qwen/Qwen2.5-VL-7B-Instruct
- Qwen/Qwen3-VL-4B-Instruct
library_name: transformers
pipeline_tag: image-text-to-text
tags:
- qwen2_5_vl
- qwen3_vl
- vision-language
- visual-token-pruning
- visionzip
- scopd
---
# SCOPD / SCOPD+ Checkpoints
LoRA adapters for Qwen2.5-VL-7B-Instruct and Qwen3-VL-4B-Instruct with visual-token-pruned inference.
This repository provides **adapter weights only** and evaluation instructions.
The pinned base model is required separately. No full/merged model weights,
teacher weights, optimizer state, or training data are included.
## Model Variants
| Model | Objective | Response-token selection | Weights |
|---|---|---|---|
| SCOPD | forward KL: KL(teacher \|\| student) | all response tokens | [Download](https://huggingface.co/enmingzhangzz/SCOPD/tree/main/models/scopd) |
| SCOPD+ | forward KL: KL(teacher \|\| student) | top 10% budget-JSD response tokens | [Download](https://huggingface.co/enmingzhangzz/SCOPD/tree/main/models/scopd-plus) |
| SCOPD+ Top20 | forward KL: KL(teacher \|\| student) | top 20% budget-JSD response tokens | [Download](https://huggingface.co/enmingzhangzz/SCOPD/tree/main/models/scopd-plus-top20) |
| SCOPD+ Top40 | forward KL: KL(teacher \|\| student) | top 40% budget-JSD response tokens | [Download](https://huggingface.co/enmingzhangzz/SCOPD/tree/main/models/scopd-plus-top40) |
| SCOPD+ Top60 | forward KL: KL(teacher \|\| student) | top 60% budget-JSD response tokens | [Download](https://huggingface.co/enmingzhangzz/SCOPD/tree/main/models/scopd-plus-top60) |
| SFT | supervised fine-tuning | supervised response tokens | [Download](https://huggingface.co/enmingzhangzz/SCOPD/tree/main/models/sft) |
| EPIC | official EPIC configuration | EPIC response supervision | [Download](https://huggingface.co/enmingzhangzz/SCOPD/tree/main/models/epic) |
| GRPO | group-relative policy optimization | reward-based policy objective | [Download](https://huggingface.co/enmingzhangzz/SCOPD/tree/main/models/grpo) |
| SCOPD Off-Policy | forward KL on teacher-generated responses | all response tokens | [Download](https://huggingface.co/enmingzhangzz/SCOPD/tree/main/models/scopd-offpolicy) |
| SCOPD+ Reverse KL | reverse KL: KL(student \|\| teacher) | top 10% budget-JSD response tokens | [Download](https://huggingface.co/enmingzhangzz/SCOPD/tree/main/models/scopd-plus-reverse-kl) |
| SCOPD+ JSD | symmetric JSD(teacher, student) | top 10% budget-JSD response tokens | [Download](https://huggingface.co/enmingzhangzz/SCOPD/tree/main/models/scopd-plus-jsd) |
| Qwen3-VL-4B SCOPD | forward KL: KL(teacher \|\| student) | all response tokens | [Download](https://huggingface.co/enmingzhangzz/SCOPD/tree/main/models/qwen3-vl-4b-scopd) |
| Qwen3-VL-4B SCOPD+ | forward KL: KL(teacher \|\| student) | top 10% budget-JSD response tokens | [Download](https://huggingface.co/enmingzhangzz/SCOPD/tree/main/models/qwen3-vl-4b-scopd-plus) |
Each subfolder contains `adapter_model.safetensors`, `adapter_config.json`,
and provenance. Qwen2.5 adapters are approximately **154 MiB** each; Qwen3
adapters are approximately **126 MiB** each. All 13 adapters total approximately
**1.90 GiB**. Their tensors retain the saved FP32 precision.
Inference uses a BF16 base model. All adapters use rank 16, alpha 32,
zero dropout, and language-decoder attention/MLP projections only.
**Top10/20/40/60 refer to selected response tokens, not visual retention.**
The SCOPD+ variants use a native VisionZip intervention from 10% to 11%
visual retention to select tokens. The released checkpoints support evaluation
at 10%, 20%, 30%, and no pruning using the same runner.
The Qwen2.5 base revision is `cc594898137f460bfe9f0759e9844b3ce807cfb5`.
The Qwen3 base revision is `ebb281ec70b05090aa6165b016eac8ec08e71b17`.
Qwen3 adapters must not be loaded onto Qwen2.5 models or vice versa.
The JSD and reverse-KL variants here are **LoRA adapters**, not full-parameter
fine-tuning variants. The GRPO checkpoint is the final policy at 2560 optimizer
updates after 10240 processed questions; these units are not interchangeable.
See [model identifiers](models.json) and each model's `provenance.json`.
## Download
The collection does not have a single root model configuration.
Download the adapter variant and evaluation code:
```bash
python -m pip install 'huggingface-hub==0.34.3'
export RELEASE_DIR="$PWD/scopd-checkpoints"
export VARIANT=scopd-plus
export RELEASE_REVISION=main
python - <<'PY'
import os
from huggingface_hub import HfApi, snapshot_download
repo = "enmingzhangzz/SCOPD"
revision = HfApi().model_info(repo, revision=os.environ["RELEASE_REVISION"]).sha
snapshot_download(repo, revision=revision, local_dir=os.environ["RELEASE_DIR"],
allow_patterns=[f'models/{os.environ["VARIANT"]}/*', 'evaluation/**',
'models.json', 'README.md', 'LICENSE', 'NOTICE', 'checksums.sha256'])
print("Pinned release revision:", revision)
PY
```
Record the printed commit hash and use it as `MODEL_REVISION` below.
The download can be resumed with the same revision.
## Evaluation Environment
Use Python 3.11 and a CUDA-compatible environment. The evaluation code and
patched Transformers/VisionZip sources are included under `evaluation/`;
a separate code repository is not needed.
```bash
cd "$RELEASE_DIR/evaluation"
python -m pip install torch==2.9.1 torchvision==0.24.1 \
--index-url https://download.pytorch.org/whl/cu126
python -m pip install -r requirements.txt
python -m pip install -r third_party/VLMEvalKit/requirements.txt
python -m pip install -r requirements.txt
python -m pip install 'flash-attn==2.8.3' --no-build-isolation
export RUNTIME=eval # Use qwen3 for qwen3-vl-4b-scopd or qwen3-vl-4b-scopd-plus
python runtime/prepare.py --mode "$RUNTIME"
source runtime/activate.sh "$RUNTIME"
```
For Qwen2.5, installing an unmodified `transformers==4.57.0` alone is not sufficient
for VisionZip: the preparer verifies and applies the evaluation overlays.
For Qwen3, it builds an **isolated stock** Transformers 4.57.0 tree; the included
Qwen3 VisionZip wrapper handles token selection, contextual merging, DeepStack
features and position IDs without Qwen2.5 overlays. Do not mix these runtimes
within one Python process. A different
Pillow implementation can change image preprocessing: consult
[environment details](evaluation/docs/ENVIRONMENT.md) before a numerical
reproduction attempt. The listed environment is a source-version specification,
not a guarantee of wheel availability on every system.
## Run a Benchmark
First download the pinned base model and merge the adapter **locally for
evaluation**. This preparation step is not training, and does not upload a
merged model. The default CPU merge needs enough host RAM for the base model
and merge buffers; use `--device cuda` only on an available GPU with sufficient
memory. Allow approximately 16 GiB of disk for Qwen2.5 or 9 GiB for Qwen3.
```bash
export MERGED_MODEL="$RELEASE_DIR/local-merged/$VARIANT"
python scripts/merge_adapter.py \
--adapter "$RELEASE_DIR/models/$VARIANT" \
--output "$MERGED_MODEL"
```
The helper reads the correct base model and revision from the adapter's
provenance automatically. Alternatively, `--base-model` can point to an already
downloaded Hub snapshot directory named by that base model's pinned revision.
For direct PEFT inference, the same adapter can be loaded onto that base using
`PeftModel.from_pretrained`; the benchmark runner below uses a merged model.
```bash
export MODEL_REVISION=THE_PRINTED_COMMIT_HASH
export EVAL_ROOT="$PWD/results/$VARIANT"
python -m scopd_eval.infer \
--model "$MERGED_MODEL" \
--model-revision "$MODEL_REVISION" \
--dataset mmstar --retention 0.10 \
--output-dir "$EVAL_ROOT/mmstar-r010"
```
Use `--retention 0.20` or `0.30` for milder pruning, and `--retention 1`
for no pruning. The argument is **retention**, not the backend's
`visionzip_ratio` parameter. The runner detects the model family from
`config.json`. Greedy inference uses `......`
for Qwen2.5 and **`......` for Qwen3**,
at most 2048 generated tokens, and image bounds of 1280 to 4096 nominal visual
tokens. A nominal token corresponds to 28x28 input pixels for Qwen2.5 and
32x32 for Qwen3; these budgets are not interchangeable pixel counts.
For Qwen3, choose `VARIANT=qwen3-vl-4b-scopd` or
`VARIANT=qwen3-vl-4b-scopd-plus` in the download command and `RUNTIME=qwen3`
in the environment command. All other merge, inference and judging commands
are unchanged. The Qwen3 checkpoints are the final student adapters after
10240 processed questions (320 optimizer updates), not teacher/EMA weights.
Their original rollout cap was 512; the evaluation cap above is 2048.
The included Qwen3 runner supports images, including multi-image questions,
but does not support video pruning.
Supported image tasks include MME, MMStar, MathVista-MINI,
MathVerse-MINI-Vision-Only, MMMU-Pro-4c, HallusionBench, CVBench,
LogicVista, BLINK, VisOnlyQA, HRBench4K, MMVP, MME-RealWorld-Lite,
RealWorldQA, POPE, and MathVision-MINI. Use `--help` for the dataset keys.
Dataset access and licenses are separate; benchmark data is not included.
For multiple GPUs, run independent `--shard N --shards K` processes with one
GPU each, then merge **all** shards before scoring:
```bash
python -m scopd_eval.merge_shards --input-dir "$EVAL_ROOT/mmstar-r010" \
--shards 4 --output "$EVAL_ROOT/mmstar-r010/all_predictions.jsonl"
```
The unsharded command above writes `shard_000/predictions.jsonl` and does not
need merging. Resume skips completed examples only when the input/model/config
contract matches. Missing or conflicting shards are errors, not partial scores.
## Unified Answer Judging
Use protocol `qwen27-visual-gt-mismatch-v1` for **every compared model and task**,
including MMStar. Deterministic matches are accepted directly. All nonmatches
are sent to **Qwen/Qwen3.6-27B-FP8** with the original question, options,
reference answer, candidate final answer, and image(s). The judge must evaluate
the candidate, not solve the question to repair it.
Start the judge in a **separate environment and GPU process**, without the
evaluation Transformers override:
```bash
python -m pip install -r requirements-judge.txt
export JUDGE_MODEL_PATH=/path/to/pinned/Qwen3.6-27B-FP8
export JUDGE_MODEL_REVISION=PINNED_JUDGE_COMMIT
bash scripts/serve_judge.sh
```
From the evaluation environment:
```bash
python -m scopd_eval.postprocess \
--input "$EVAL_ROOT/mmstar-r010/shard_000/predictions.jsonl" \
--output-dir "$EVAL_ROOT/mmstar-r010/judged" \
--judge-revision "$JUDGE_MODEL_REVISION"
```
Use the merged input for a sharded run. Preserve raw predictions, judge output,
model revisions, and protocol metadata. See the
[full scoring policy and prompt location](evaluation/docs/EVALUATION.md).
This is a versioned evaluation protocol, not a claim of identity with every
benchmark's official evaluator. Scores from other judge protocols must be
reprocessed before comparison. No benchmark scores are asserted in this release.
## Validation and Limitations
Adapter identities, safetensor structure, trainable module keys,
base-model revision, and publication contents are checked before release. Runtime and judge-routing
tests are included. The real 27B image-aware judge and a clean-machine
installation still need end-to-end validation; mock tests do not establish GPU
memory or throughput guarantees. Models may hallucinate and are intended for
research evaluation, not high-stakes decisions.
## License
The weights are derivatives of
[Qwen2.5-VL-7B-Instruct](https://huggingface.co/Qwen/Qwen2.5-VL-7B-Instruct) and
[Qwen3-VL-4B-Instruct](https://huggingface.co/Qwen/Qwen3-VL-4B-Instruct),
whose pinned model cards specify Apache-2.0. See [LICENSE](LICENSE) and
[NOTICE](NOTICE). Third-party evaluation and pruning implementations retain
their individual licenses and source notices.