How to use from
Docker Model Runner
docker model run hf.co/enmingzhangzz/SCOPD
Quick Links

SCOPD / SCOPD+ Checkpoints

LoRA adapters for Qwen2.5-VL-7B-Instruct and Qwen3-VL-4B-Instruct with visual-token-pruned inference. This repository provides adapter weights only and evaluation instructions. The pinned base model is required separately. No full/merged model weights, teacher weights, optimizer state, or training data are included.

Model Variants

Model Objective Response-token selection Weights
SCOPD forward KL: KL(teacher || student) all response tokens Download
SCOPD+ forward KL: KL(teacher || student) top 10% budget-JSD response tokens Download
SCOPD+ Top20 forward KL: KL(teacher || student) top 20% budget-JSD response tokens Download
SCOPD+ Top40 forward KL: KL(teacher || student) top 40% budget-JSD response tokens Download
SCOPD+ Top60 forward KL: KL(teacher || student) top 60% budget-JSD response tokens Download
SFT supervised fine-tuning supervised response tokens Download
EPIC official EPIC configuration EPIC response supervision Download
GRPO group-relative policy optimization reward-based policy objective Download
SCOPD Off-Policy forward KL on teacher-generated responses all response tokens Download
SCOPD+ Reverse KL reverse KL: KL(student || teacher) top 10% budget-JSD response tokens Download
SCOPD+ JSD symmetric JSD(teacher, student) top 10% budget-JSD response tokens Download
Qwen3-VL-4B SCOPD forward KL: KL(teacher || student) all response tokens Download
Qwen3-VL-4B SCOPD+ forward KL: KL(teacher || student) top 10% budget-JSD response tokens Download

Each subfolder contains adapter_model.safetensors, adapter_config.json, and provenance. Qwen2.5 adapters are approximately 154 MiB each; Qwen3 adapters are approximately 126 MiB each. All 13 adapters total approximately 1.90 GiB. Their tensors retain the saved FP32 precision. Inference uses a BF16 base model. All adapters use rank 16, alpha 32, zero dropout, and language-decoder attention/MLP projections only.

Top10/20/40/60 refer to selected response tokens, not visual retention. The SCOPD+ variants use a native VisionZip intervention from 10% to 11% visual retention to select tokens. The released checkpoints support evaluation at 10%, 20%, 30%, and no pruning using the same runner.

The Qwen2.5 base revision is cc594898137f460bfe9f0759e9844b3ce807cfb5. The Qwen3 base revision is ebb281ec70b05090aa6165b016eac8ec08e71b17. Qwen3 adapters must not be loaded onto Qwen2.5 models or vice versa. The JSD and reverse-KL variants here are LoRA adapters, not full-parameter fine-tuning variants. The GRPO checkpoint is the final policy at 2560 optimizer updates after 10240 processed questions; these units are not interchangeable. See model identifiers and each model's provenance.json.

Download

The collection does not have a single root model configuration. Download the adapter variant and evaluation code:

python -m pip install 'huggingface-hub==0.34.3'
export RELEASE_DIR="$PWD/scopd-checkpoints"
export VARIANT=scopd-plus
export RELEASE_REVISION=main
python - <<'PY'
import os
from huggingface_hub import HfApi, snapshot_download
repo = "enmingzhangzz/SCOPD"
revision = HfApi().model_info(repo, revision=os.environ["RELEASE_REVISION"]).sha
snapshot_download(repo, revision=revision, local_dir=os.environ["RELEASE_DIR"],
    allow_patterns=[f'models/{os.environ["VARIANT"]}/*', 'evaluation/**',
                    'models.json', 'README.md', 'LICENSE', 'NOTICE', 'checksums.sha256'])
print("Pinned release revision:", revision)
PY

Record the printed commit hash and use it as MODEL_REVISION below. The download can be resumed with the same revision.

Evaluation Environment

Use Python 3.11 and a CUDA-compatible environment. The evaluation code and patched Transformers/VisionZip sources are included under evaluation/; a separate code repository is not needed.

cd "$RELEASE_DIR/evaluation"
python -m pip install torch==2.9.1 torchvision==0.24.1 \
  --index-url https://download.pytorch.org/whl/cu126
python -m pip install -r requirements.txt
python -m pip install -r third_party/VLMEvalKit/requirements.txt
python -m pip install -r requirements.txt
python -m pip install 'flash-attn==2.8.3' --no-build-isolation
export RUNTIME=eval  # Use qwen3 for qwen3-vl-4b-scopd or qwen3-vl-4b-scopd-plus
python runtime/prepare.py --mode "$RUNTIME"
source runtime/activate.sh "$RUNTIME"

For Qwen2.5, installing an unmodified transformers==4.57.0 alone is not sufficient for VisionZip: the preparer verifies and applies the evaluation overlays. For Qwen3, it builds an isolated stock Transformers 4.57.0 tree; the included Qwen3 VisionZip wrapper handles token selection, contextual merging, DeepStack features and position IDs without Qwen2.5 overlays. Do not mix these runtimes within one Python process. A different Pillow implementation can change image preprocessing: consult environment details before a numerical reproduction attempt. The listed environment is a source-version specification, not a guarantee of wheel availability on every system.

Run a Benchmark

First download the pinned base model and merge the adapter locally for evaluation. This preparation step is not training, and does not upload a merged model. The default CPU merge needs enough host RAM for the base model and merge buffers; use --device cuda only on an available GPU with sufficient memory. Allow approximately 16 GiB of disk for Qwen2.5 or 9 GiB for Qwen3.

export MERGED_MODEL="$RELEASE_DIR/local-merged/$VARIANT"
python scripts/merge_adapter.py \
  --adapter "$RELEASE_DIR/models/$VARIANT" \
  --output "$MERGED_MODEL"

The helper reads the correct base model and revision from the adapter's provenance automatically. Alternatively, --base-model can point to an already downloaded Hub snapshot directory named by that base model's pinned revision. For direct PEFT inference, the same adapter can be loaded onto that base using PeftModel.from_pretrained; the benchmark runner below uses a merged model.

export MODEL_REVISION=THE_PRINTED_COMMIT_HASH
export EVAL_ROOT="$PWD/results/$VARIANT"
python -m scopd_eval.infer \
  --model "$MERGED_MODEL" \
  --model-revision "$MODEL_REVISION" \
  --dataset mmstar --retention 0.10 \
  --output-dir "$EVAL_ROOT/mmstar-r010"

Use --retention 0.20 or 0.30 for milder pruning, and --retention 1 for no pruning. The argument is retention, not the backend's visionzip_ratio parameter. The runner detects the model family from config.json. Greedy inference uses <think>...</think><answer>...</answer> for Qwen2.5 and <analysis>...</analysis><answer>...</answer> for Qwen3, at most 2048 generated tokens, and image bounds of 1280 to 4096 nominal visual tokens. A nominal token corresponds to 28x28 input pixels for Qwen2.5 and 32x32 for Qwen3; these budgets are not interchangeable pixel counts.

For Qwen3, choose VARIANT=qwen3-vl-4b-scopd or VARIANT=qwen3-vl-4b-scopd-plus in the download command and RUNTIME=qwen3 in the environment command. All other merge, inference and judging commands are unchanged. The Qwen3 checkpoints are the final student adapters after 10240 processed questions (320 optimizer updates), not teacher/EMA weights. Their original rollout cap was 512; the evaluation cap above is 2048. The included Qwen3 runner supports images, including multi-image questions, but does not support video pruning.

Supported image tasks include MME, MMStar, MathVista-MINI, MathVerse-MINI-Vision-Only, MMMU-Pro-4c, HallusionBench, CVBench, LogicVista, BLINK, VisOnlyQA, HRBench4K, MMVP, MME-RealWorld-Lite, RealWorldQA, POPE, and MathVision-MINI. Use --help for the dataset keys. Dataset access and licenses are separate; benchmark data is not included.

For multiple GPUs, run independent --shard N --shards K processes with one GPU each, then merge all shards before scoring:

python -m scopd_eval.merge_shards --input-dir "$EVAL_ROOT/mmstar-r010" \
  --shards 4 --output "$EVAL_ROOT/mmstar-r010/all_predictions.jsonl"

The unsharded command above writes shard_000/predictions.jsonl and does not need merging. Resume skips completed examples only when the input/model/config contract matches. Missing or conflicting shards are errors, not partial scores.

Unified Answer Judging

Use protocol qwen27-visual-gt-mismatch-v1 for every compared model and task, including MMStar. Deterministic matches are accepted directly. All nonmatches are sent to Qwen/Qwen3.6-27B-FP8 with the original question, options, reference answer, candidate final answer, and image(s). The judge must evaluate the candidate, not solve the question to repair it.

Start the judge in a separate environment and GPU process, without the evaluation Transformers override:

python -m pip install -r requirements-judge.txt
export JUDGE_MODEL_PATH=/path/to/pinned/Qwen3.6-27B-FP8
export JUDGE_MODEL_REVISION=PINNED_JUDGE_COMMIT
bash scripts/serve_judge.sh

From the evaluation environment:

python -m scopd_eval.postprocess \
  --input "$EVAL_ROOT/mmstar-r010/shard_000/predictions.jsonl" \
  --output-dir "$EVAL_ROOT/mmstar-r010/judged" \
  --judge-revision "$JUDGE_MODEL_REVISION"

Use the merged input for a sharded run. Preserve raw predictions, judge output, model revisions, and protocol metadata. See the full scoring policy and prompt location. This is a versioned evaluation protocol, not a claim of identity with every benchmark's official evaluator. Scores from other judge protocols must be reprocessed before comparison. No benchmark scores are asserted in this release.

Validation and Limitations

Adapter identities, safetensor structure, trainable module keys, base-model revision, and publication contents are checked before release. Runtime and judge-routing tests are included. The real 27B image-aware judge and a clean-machine installation still need end-to-end validation; mock tests do not establish GPU memory or throughput guarantees. Models may hallucinate and are intended for research evaluation, not high-stakes decisions.

License

The weights are derivatives of Qwen2.5-VL-7B-Instruct and Qwen3-VL-4B-Instruct, whose pinned model cards specify Apache-2.0. See LICENSE and NOTICE. Third-party evaluation and pruning implementations retain their individual licenses and source notices.

Downloads last month

-

Downloads are not tracked for this model. How to track
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for enmingzhangzz/SCOPD

Finetuned
(1,269)
this model