Buckets:
| Name | Size | Uploaded | Xet hash |
|---|---|---|---|
| .gitattributes | 1.57 kB xet | aacf151a | |
| .mdl | 47 Bytes xet | 6703278b | |
| .msc | 1.23 kB xet | de0259bd | |
| .mv | 36 Bytes xet | 6843b152 | |
| README.md | 4.4 kB xet | 2ce6f988 | |
| added_tokens.json | 707 Bytes xet | a1d47d24 | |
| chat_template.jinja | 5.29 kB xet | 5633af92 | |
| config.json | 1.64 kB xet | 7ce61940 | |
| generation_config.json | 169 Bytes xet | 27ed6dfe | |
| merges.txt | 1.67 MB xet | 87912eed | |
| model-00001-of-00003.safetensors | 4.41 GB xet | 525a443b | |
| model-00002-of-00003.safetensors | 4.47 GB xet | 6adbece8 | |
| model-00003-of-00003.safetensors | 778 MB xet | 1ea06c44 | |
| model.safetensors.index.json | 64.8 kB xet | db6a5312 | |
| preprocessor_config.json | 782 Bytes xet | 17427eab | |
| special_tokens_map.json | 613 Bytes xet | 8b458476 | |
| tokenizer.json | 11.4 MB xet | 693ec4b3 | |
| tokenizer_config.json | 5.49 kB xet | e4ca1eec | |
| video_preprocessor_config.json | 817 Bytes xet | c336fbe2 | |
| vocab.json | 2.78 MB xet | 9208e1be |
MechVL-4B-RL
The RL checkpoint of MechVL — the domain-specialized multimodal model for mechanical engineering drawing understanding, introduced in:
MechVQA: Benchmarking and Enhancing Multimodal LLMs on Comprehensive Mechanical Drawing Understanding (ICML 2026)
Model description
MechVL-4B-RL is obtained by further optimizing MechVL-4B-SFT with DAPO (Decoupled Clip and Dynamic Sampling Policy Optimization) in a two-stage self-play regime:
- Stage 1 (full): DAPO over the full MechVQA training split.
- Stage 2 (targeted): DAPO over a re-sampled subset with an increased proportion of underperforming subtasks.
Reward = Accuracy (LLM-as-a-Judge, semantic equivalence in [0,1]) + Format (binary, well-formed <think>/<answer>) + Quality (Logic / Professionalism / Conciseness).
| Base model | Qwen3-VL-4B-Instruct |
| Architecture | Qwen3VLForConditionalGeneration |
| Stage | 2 / 2 — RL (DAPO self-play, on top of SFT) |
| MechVQA Total | 84.85 (best across open- & closed-source MLLMs) |
| SFT checkpoint | XiaofengAlg/MechVL-4B-SFT |
Output format
The model reasons then answers, enclosed as:
<think> ...reasoning... </think><answer> ...final answer... </answer>
Parse the <answer>...</answer> span for the final answer (a regex like <answer>(.*?)</answer> works).
Usage (transformers)
import re, torch
from transformers import AutoProcessor, AutoModelForImageTextToText
model = AutoModelForImageTextToText.from_pretrained(
"XiaofengAlg/MechVL-4B-RL", dtype=torch.bfloat16, device_map="auto"
)
processor = AutoProcessor.from_pretrained("XiaofengAlg/MechVL-4B-RL")
question = "图纸中标注的零件总长度是多少?"
# RL format prompt (matches training): question + <think>/<answer> instruction
messages = [{"role": "user", "content": [
{"type": "image", "url": "path/to/drawing.png"},
{"type": "text", "text": question},
]}]
inputs = processor.apply_chat_template(
messages, add_generation_prompt=True, tokenize=True,
return_dict=True, return_tensors="pt",
).to(model.device)
out = model.generate(**inputs, max_new_tokens=4096, do_sample=True,
temperature=0.6, top_p=0.95, top_k=20)
text = processor.decode(out[0], skip_special_tokens=True)
m = re.search(r"<answer>(.*?)</answer>", text, re.DOTALL)
print("answer:", m.group(1).strip() if m else text)
For batch vLLM inference with the exact training-time format prompt, see scripts/batch_infer.py (MODE=rl) and prompts/mech_r1.jinja.
Results
On the MechVQA benchmark (Total score):
| Model | Total |
|---|---|
| GPT-5 | 75.44 |
| Gemini-3-Pro-Preview | 77.28 |
| GLM-4.6V (best closed-source) | 78.91 |
| MechVL-4B-SFT | 76.36 |
| MechVL-4B-RL (this model) | 84.85 |
See §6 of the paper for the full table and ablations (DAPO > GRPO > GSPO; two-stage self-play; reward-component ablations).
Citation
@misc{kou2026mechvqabenchmarkingenhancingmultimodal,
title={MechVQA: Benchmarking and Enhancing Multimodal LLMs on Comprehensive Mechanical Drawing Understanding},
author={Qian Kou and Xiaofeng Shi and Yulin Li and Xiaosong Qiu and Xinyang Wang and Hua Zhou and Cao Dongxing},
year={2026},
eprint={2605.30794},
archivePrefix={arXiv},
primaryClass={cs.CV},
url={https://arxiv.org/abs/2605.30794}
}
License
Apache-2.0.
- Total size
- 9.67 GB
- Files
- 20
- Last updated
- Oct 1
- Pre-warmed CDN
- US EU US EU