Instructions to use enmingzhangzz/SCOPD with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use enmingzhangzz/SCOPD with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("image-text-to-text", model="enmingzhangzz/SCOPD")# pip install -U transformers accelerate # Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("enmingzhangzz/SCOPD", device_map="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use enmingzhangzz/SCOPD with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "enmingzhangzz/SCOPD" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "enmingzhangzz/SCOPD", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker
docker model run hf.co/enmingzhangzz/SCOPD
- SGLang
How to use enmingzhangzz/SCOPD with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "enmingzhangzz/SCOPD" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "enmingzhangzz/SCOPD", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "enmingzhangzz/SCOPD" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "enmingzhangzz/SCOPD", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }' - Docker Model Runner
How to use enmingzhangzz/SCOPD with Docker Model Runner:
docker model run hf.co/enmingzhangzz/SCOPD
# pip install -U transformers accelerate
# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("enmingzhangzz/SCOPD", device_map="auto")SCOPD / SCOPD+ Checkpoints
LoRA adapters for Qwen2.5-VL-7B-Instruct and Qwen3-VL-4B-Instruct with visual-token-pruned inference. This repository provides adapter weights only and evaluation instructions. The pinned base model is required separately. No full/merged model weights, teacher weights, optimizer state, or training data are included.
Model Variants
| Model | Objective | Response-token selection | Weights |
|---|---|---|---|
| SCOPD | forward KL: KL(teacher || student) | all response tokens | Download |
| SCOPD+ | forward KL: KL(teacher || student) | top 10% budget-JSD response tokens | Download |
| SCOPD+ Top20 | forward KL: KL(teacher || student) | top 20% budget-JSD response tokens | Download |
| SCOPD+ Top40 | forward KL: KL(teacher || student) | top 40% budget-JSD response tokens | Download |
| SCOPD+ Top60 | forward KL: KL(teacher || student) | top 60% budget-JSD response tokens | Download |
| SFT | supervised fine-tuning | supervised response tokens | Download |
| EPIC | official EPIC configuration | EPIC response supervision | Download |
| GRPO | group-relative policy optimization | reward-based policy objective | Download |
| SCOPD Off-Policy | forward KL on teacher-generated responses | all response tokens | Download |
| SCOPD+ Reverse KL | reverse KL: KL(student || teacher) | top 10% budget-JSD response tokens | Download |
| SCOPD+ JSD | symmetric JSD(teacher, student) | top 10% budget-JSD response tokens | Download |
| Qwen3-VL-4B SCOPD | forward KL: KL(teacher || student) | all response tokens | Download |
| Qwen3-VL-4B SCOPD+ | forward KL: KL(teacher || student) | top 10% budget-JSD response tokens | Download |
Each subfolder contains adapter_model.safetensors, adapter_config.json,
and provenance. Qwen2.5 adapters are approximately 154 MiB each; Qwen3
adapters are approximately 126 MiB each. All 13 adapters total approximately
1.90 GiB. Their tensors retain the saved FP32 precision.
Inference uses a BF16 base model. All adapters use rank 16, alpha 32,
zero dropout, and language-decoder attention/MLP projections only.
Top10/20/40/60 refer to selected response tokens, not visual retention. The SCOPD+ variants use a native VisionZip intervention from 10% to 11% visual retention to select tokens. The released checkpoints support evaluation at 10%, 20%, 30%, and no pruning using the same runner.
The Qwen2.5 base revision is cc594898137f460bfe9f0759e9844b3ce807cfb5.
The Qwen3 base revision is ebb281ec70b05090aa6165b016eac8ec08e71b17.
Qwen3 adapters must not be loaded onto Qwen2.5 models or vice versa.
The JSD and reverse-KL variants here are LoRA adapters, not full-parameter
fine-tuning variants. The GRPO checkpoint is the final policy at 2560 optimizer
updates after 10240 processed questions; these units are not interchangeable.
See model identifiers and each model's provenance.json.
Download
The collection does not have a single root model configuration. Download the adapter variant and evaluation code:
python -m pip install 'huggingface-hub==0.34.3'
export RELEASE_DIR="$PWD/scopd-checkpoints"
export VARIANT=scopd-plus
export RELEASE_REVISION=main
python - <<'PY'
import os
from huggingface_hub import HfApi, snapshot_download
repo = "enmingzhangzz/SCOPD"
revision = HfApi().model_info(repo, revision=os.environ["RELEASE_REVISION"]).sha
snapshot_download(repo, revision=revision, local_dir=os.environ["RELEASE_DIR"],
allow_patterns=[f'models/{os.environ["VARIANT"]}/*', 'evaluation/**',
'models.json', 'README.md', 'LICENSE', 'NOTICE', 'checksums.sha256'])
print("Pinned release revision:", revision)
PY
Record the printed commit hash and use it as MODEL_REVISION below.
The download can be resumed with the same revision.
Evaluation Environment
Use Python 3.11 and a CUDA-compatible environment. The evaluation code and
patched Transformers/VisionZip sources are included under evaluation/;
a separate code repository is not needed.
cd "$RELEASE_DIR/evaluation"
python -m pip install torch==2.9.1 torchvision==0.24.1 \
--index-url https://download.pytorch.org/whl/cu126
python -m pip install -r requirements.txt
python -m pip install -r third_party/VLMEvalKit/requirements.txt
python -m pip install -r requirements.txt
python -m pip install 'flash-attn==2.8.3' --no-build-isolation
export RUNTIME=eval # Use qwen3 for qwen3-vl-4b-scopd or qwen3-vl-4b-scopd-plus
python runtime/prepare.py --mode "$RUNTIME"
source runtime/activate.sh "$RUNTIME"
For Qwen2.5, installing an unmodified transformers==4.57.0 alone is not sufficient
for VisionZip: the preparer verifies and applies the evaluation overlays.
For Qwen3, it builds an isolated stock Transformers 4.57.0 tree; the included
Qwen3 VisionZip wrapper handles token selection, contextual merging, DeepStack
features and position IDs without Qwen2.5 overlays. Do not mix these runtimes
within one Python process. A different
Pillow implementation can change image preprocessing: consult
environment details before a numerical
reproduction attempt. The listed environment is a source-version specification,
not a guarantee of wheel availability on every system.
Run a Benchmark
First download the pinned base model and merge the adapter locally for
evaluation. This preparation step is not training, and does not upload a
merged model. The default CPU merge needs enough host RAM for the base model
and merge buffers; use --device cuda only on an available GPU with sufficient
memory. Allow approximately 16 GiB of disk for Qwen2.5 or 9 GiB for Qwen3.
export MERGED_MODEL="$RELEASE_DIR/local-merged/$VARIANT"
python scripts/merge_adapter.py \
--adapter "$RELEASE_DIR/models/$VARIANT" \
--output "$MERGED_MODEL"
The helper reads the correct base model and revision from the adapter's
provenance automatically. Alternatively, --base-model can point to an already
downloaded Hub snapshot directory named by that base model's pinned revision.
For direct PEFT inference, the same adapter can be loaded onto that base using
PeftModel.from_pretrained; the benchmark runner below uses a merged model.
export MODEL_REVISION=THE_PRINTED_COMMIT_HASH
export EVAL_ROOT="$PWD/results/$VARIANT"
python -m scopd_eval.infer \
--model "$MERGED_MODEL" \
--model-revision "$MODEL_REVISION" \
--dataset mmstar --retention 0.10 \
--output-dir "$EVAL_ROOT/mmstar-r010"
Use --retention 0.20 or 0.30 for milder pruning, and --retention 1
for no pruning. The argument is retention, not the backend's
visionzip_ratio parameter. The runner detects the model family from
config.json. Greedy inference uses <think>...</think><answer>...</answer>
for Qwen2.5 and <analysis>...</analysis><answer>...</answer> for Qwen3,
at most 2048 generated tokens, and image bounds of 1280 to 4096 nominal visual
tokens. A nominal token corresponds to 28x28 input pixels for Qwen2.5 and
32x32 for Qwen3; these budgets are not interchangeable pixel counts.
For Qwen3, choose VARIANT=qwen3-vl-4b-scopd or
VARIANT=qwen3-vl-4b-scopd-plus in the download command and RUNTIME=qwen3
in the environment command. All other merge, inference and judging commands
are unchanged. The Qwen3 checkpoints are the final student adapters after
10240 processed questions (320 optimizer updates), not teacher/EMA weights.
Their original rollout cap was 512; the evaluation cap above is 2048.
The included Qwen3 runner supports images, including multi-image questions,
but does not support video pruning.
Supported image tasks include MME, MMStar, MathVista-MINI,
MathVerse-MINI-Vision-Only, MMMU-Pro-4c, HallusionBench, CVBench,
LogicVista, BLINK, VisOnlyQA, HRBench4K, MMVP, MME-RealWorld-Lite,
RealWorldQA, POPE, and MathVision-MINI. Use --help for the dataset keys.
Dataset access and licenses are separate; benchmark data is not included.
For multiple GPUs, run independent --shard N --shards K processes with one
GPU each, then merge all shards before scoring:
python -m scopd_eval.merge_shards --input-dir "$EVAL_ROOT/mmstar-r010" \
--shards 4 --output "$EVAL_ROOT/mmstar-r010/all_predictions.jsonl"
The unsharded command above writes shard_000/predictions.jsonl and does not
need merging. Resume skips completed examples only when the input/model/config
contract matches. Missing or conflicting shards are errors, not partial scores.
Unified Answer Judging
Use protocol qwen27-visual-gt-mismatch-v1 for every compared model and task,
including MMStar. Deterministic matches are accepted directly. All nonmatches
are sent to Qwen/Qwen3.6-27B-FP8 with the original question, options,
reference answer, candidate final answer, and image(s). The judge must evaluate
the candidate, not solve the question to repair it.
Start the judge in a separate environment and GPU process, without the evaluation Transformers override:
python -m pip install -r requirements-judge.txt
export JUDGE_MODEL_PATH=/path/to/pinned/Qwen3.6-27B-FP8
export JUDGE_MODEL_REVISION=PINNED_JUDGE_COMMIT
bash scripts/serve_judge.sh
From the evaluation environment:
python -m scopd_eval.postprocess \
--input "$EVAL_ROOT/mmstar-r010/shard_000/predictions.jsonl" \
--output-dir "$EVAL_ROOT/mmstar-r010/judged" \
--judge-revision "$JUDGE_MODEL_REVISION"
Use the merged input for a sharded run. Preserve raw predictions, judge output, model revisions, and protocol metadata. See the full scoring policy and prompt location. This is a versioned evaluation protocol, not a claim of identity with every benchmark's official evaluator. Scores from other judge protocols must be reprocessed before comparison. No benchmark scores are asserted in this release.
Validation and Limitations
Adapter identities, safetensor structure, trainable module keys, base-model revision, and publication contents are checked before release. Runtime and judge-routing tests are included. The real 27B image-aware judge and a clean-machine installation still need end-to-end validation; mock tests do not establish GPU memory or throughput guarantees. Models may hallucinate and are intended for research evaluation, not high-stakes decisions.
License
The weights are derivatives of Qwen2.5-VL-7B-Instruct and Qwen3-VL-4B-Instruct, whose pinned model cards specify Apache-2.0. See LICENSE and NOTICE. Third-party evaluation and pruning implementations retain their individual licenses and source notices.
Model tree for enmingzhangzz/SCOPD
Base model
Qwen/Qwen2.5-VL-7B-Instruct
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("image-text-to-text", model="enmingzhangzz/SCOPD")