Instructions to use adidukrembzuai/MedChain-R1-4B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use adidukrembzuai/MedChain-R1-4B with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="adidukrembzuai/MedChain-R1-4B")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("adidukrembzuai/MedChain-R1-4B")
model = AutoModelForMultimodalLM.from_pretrained("adidukrembzuai/MedChain-R1-4B", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use adidukrembzuai/MedChain-R1-4B with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "adidukrembzuai/MedChain-R1-4B"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "adidukrembzuai/MedChain-R1-4B",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/adidukrembzuai/MedChain-R1-4B

SGLang

How to use adidukrembzuai/MedChain-R1-4B with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "adidukrembzuai/MedChain-R1-4B" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "adidukrembzuai/MedChain-R1-4B",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "adidukrembzuai/MedChain-R1-4B" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "adidukrembzuai/MedChain-R1-4B",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use adidukrembzuai/MedChain-R1-4B with Docker Model Runner:
```
docker model run hf.co/adidukrembzuai/MedChain-R1-4B
```

MedChain-R1-4B: Think Before You Locate: Clinical Chain-of-Thought Supervision for Sequential Medical Image Grounding

MedChain-R1 overview

MedChain-R1-4B is a 4B vision-language model for sequential medical image grounding: given a short sequence of medical images and a clinical question, it first writes an image-grounded rationale in <think>…</think> and then returns one bounding box <answer>(x1,y1),(x2,y2)</answer> in pixel coordinates. It is built on Qwen3.5-4B with deliberative chain-of-thought supervision (dCoT-SFT) on the MedChain rationale corpus, followed by GRPO with an IoU + format reward.

MedChain contains 182,573 filtered rationales over the 8 MedSG-Bench tasks and 10 imaging modalities. A medical VLM (FlemingVL-38B) drafts a caption-like rationale; a multimodal refiner (Qwen3.5-397B-A17B) then re-examines the images, question and annotated box and writes a box-conditioned explanation with explicit cross-image reasoning.

🔑 Key findings (from the paper)

Reasoning supervision sets a new SOTA. dCoT-SFT outperforms MedSeq-Grounder at every scale, reaching 74.96, 76.60 and 77.24 mIoU at 4B, 9B and 27B; the 27B model improves over the strongest prior model by 4.69 mIoU and 5.49 Acc@0.5.
The gains depend on the task. Against an answer-only 4B model, dCoT-SFT improves mIoU by 3.56 (95% CI 3.13–4.00): +14.7–16.1 on difference grounding and +6.5 on object tracking, but −2.2 to −3.2 on multi-view, patch and referring.
Predictions depend on the reasoning context. Forcing an immediate answer lowers mIoU by 2.9; conditioning on a rationale from another task lowers it by 12.2.
GRPO adds little after dCoT-SFT. +0.37 mIoU on the benchmark (4B), with task-dependent effects.

🏆 Benchmark Performance

MedSG-Bench (Table 1 of the paper)

Each cell is mIoU / Acc@0.5 in percent on all 9,630 test questions. IDG: registered (RDG) and non-registered (NRDG) difference grounding. ICG: multi-view (MV), object tracking (OT), concept (VCG), patch (VPG), cross-modal (CMG) and referring (RG) grounding. Bold: our results that exceed MedSeq-Grounder, the strongest prior model, on the same metric.

Group	Model	Size	RDG	NRDG	MV	OT	VCG	VPG	CMG	RG	Avg.
General	Qwen2.5-VL	72B	4.37 / 2.60	3.46 / 0.80	7.22 / 2.78	13.11 / 7.70	10.33 / 3.50	26.45 / 6.30	16.32 / 7.00	20.19 / 14.10	13.35 / 6.12
General	InternVL3	78B	0.20 / 0.00	0.53 / 0.00	6.35 / 2.43	13.03 / 8.00	3.57 / 0.90	11.81 / 2.50	3.34 / 0.85	12.76 / 8.10	6.44 / 2.90
General	LLaVA-OneVision	72B	2.58 / 0.80	2.87 / 0.90	11.74 / 1.39	9.61 / 2.30	10.95 / 3.30	32.38 / 20.30	16.24 / 5.40	15.43 / 6.68	13.21 / 5.18
General	Migician	7B	15.26 / 7.80	14.49 / 6.10	18.16 / 7.84	21.38 / 14.90	14.23 / 7.20	28.87 / 13.70	21.41 / 12.15	25.30 / 18.02	20.29 / 11.39
General	Qwen3.5 (zero-shot)	27B	0.00 / 0.00	0.00 / 0.00	8.48 / 4.88	28.39 / 24.50	0.25 / 0.00	2.50 / 0.20	0.25 / 0.00	0.82 / 0.00	4.42 / 3.15
Medical	HuatuoGPT-Vision	34B	1.44 / 0.00	2.15 / 0.00	9.41 / 1.65	13.25 / 8.30	6.43 / 0.70	14.53 / 1.40	10.60 / 2.60	8.60 / 1.75	8.57 / 2.09
Medical	MedGemma	27B	0.01 / 0.00	0.06 / 0.00	5.29 / 2.26	22.26 / 16.10	2.27 / 0.00	3.11 / 0.30	2.92 / 0.35	5.35 / 0.61	4.94 / 2.14
Medical	MedGRPO-7B-RL	7B	0.09 / 0.00	0.06 / 0.00	3.68 / 1.48	17.63 / 15.00	4.57 / 0.40	18.44 / 1.20	0.78 / 0.15	9.31 / 3.37	6.27 / 2.45
Medical	MedSeq-Grounder	7B	83.29 / 93.20	83.72 / 94.10	55.03 / 60.19	62.10 / 67.20	74.11 / 82.60	85.25 / 98.80	78.77 / 82.75	60.43 / 65.59	72.55 / 79.71
Ours	MedChain-R1 (dCoT-SFT)	4B	82.86 / 93.30	81.30 / 91.20	54.01 / 58.97	64.77 / 71.00	70.64 / 78.80	90.21 / 99.70	83.39 / 87.00	69.67 / 79.28	74.96 / 82.37
Ours	MedChain-R1 (+GRPO)	4B	83.82 / 94.50	83.21 / 92.70	54.80 / 59.93	67.90 / 76.20	66.09 / 72.10	89.43 / 99.80	83.59 / 87.55	70.72 / 80.30	75.32 / 82.89
Ours	MedChain-R1 (dCoT-SFT)	9B	84.00 / 93.10	83.02 / 91.60	55.87 / 60.98	67.46 / 75.00	74.64 / 83.20	91.22 / 99.90	84.16 / 87.40	70.77 / 78.95	76.60 / 83.55
Ours	MedChain-R1 (+GRPO)	9B	84.16 / 93.20	83.12 / 91.40	56.98 / 62.02	67.65 / 74.90	74.80 / 82.80	91.22 / 99.80	84.82 / 88.25	70.53 / 79.22	76.90 / 83.82
Ours	MedChain-R1 (dCoT-SFT)	27B	81.87 / 93.00	80.97 / 91.60	60.14 / 66.72	69.53 / 78.30	74.64 / 83.30	91.87 / 99.90	84.24 / 88.15	72.51 / 81.98	77.24 / 85.20

Baselines are evaluated zero-shot with one unified prompt (MedSeq-Grounder is the supervised SOTA). Full results for all baselines are in Table 10 of the paper. Benchmarked on AMD MI210 GPUs.

This checkpoint

The paper's 4B GRPO row (75.32 / 82.89) comes from a different GRPO checkpoint. This repository holds: dCoT-SFT checkpoint-2000 → GRPO (IoU + format reward) on the held-out RL30K pool, one pass, step 2500, seed 42. Evaluated with the same protocol (all 9,630 questions, greedy decoding, batch size 1, bf16, 1,200 new tokens), it scores:

RDG	NRDG	MV	OT	VCG	VPG	CMG	RG	mIoU / Acc@0.5
83.52	81.97	56.63	66.72	71.82	90.34	84.03	70.25	75.97 / 83.67

Per-task cells are mIoU. The per-example predictions are in the code repository.

🚀 Quick Start

Installation

pip install "transformers==5.2.0" torch qwen-vl-utils==0.0.14

Basic Usage

import torch
from transformers import AutoModelForImageTextToText, AutoProcessor
from qwen_vl_utils import process_vision_info

repo = "adidukrembzuai/MedChain-R1-4B"
model = AutoModelForImageTextToText.from_pretrained(repo, torch_dtype=torch.bfloat16, device_map="auto").eval()
processor = AutoProcessor.from_pretrained(repo)

question = "Compare these two images carefully and give me the coordinates of their difference."
messages = [{
    "role": "user",
    "content": [
        {"type": "image", "image": "scan_1.png"},
        {"type": "image", "image": "scan_2.png"},
        {"type": "text", "text": question + "\nFirst output the thinking process in <think> </think> tags "
                                  "and then output the bounding box in <answer>(x1,y1),(x2,y2)</answer> tags."},
    ],
}]

text = processor.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
image_inputs, video_inputs = process_vision_info(messages)
inputs = processor(text=[text], images=image_inputs, videos=video_inputs, return_tensors="pt").to(model.device)

with torch.inference_mode():
    out = model.generate(**inputs, max_new_tokens=1200, do_sample=False)
print(processor.decode(out[0, inputs["input_ids"].shape[1]:], skip_special_tokens=True))
# <think> ... </think><answer>(x1,y1),(x2,y2)</answer>

Notes

Append the instruction above to every question; it is the prompt the model was trained and evaluated with.
Generate one example at a time: batched generation produced degenerate outputs for some checkpoints.
Use the processor bundled in this repository (the dCoT-SFT processor). The paper (Section 5.3) shows that an exported processor with a lower image-resolution limit costs 3–5 points.
eval/eval_bench.py in the code repository is the exact evaluation script.

📦 Model Family

Model	Base	This checkpoint, MedSG-Bench mIoU / Acc@0.5	Best for
MedChain-R1-9B	Qwen3.5-9B	77.54 / 84.87	highest accuracy — recommended
MedChain-R1-4B	Qwen3.5-4B	75.97 / 83.67	resource-constrained environments

⚠️ Limitations

Research artefact, not validated for autonomous diagnosis or clinical deployment. The MedChain rationales are conditioned on the annotated box, so they explain a known answer and may be post-hoc; they should not be read as a faithful record of the model's internal computation. The rationales can inherit errors and biases from their teacher models and were reviewed in a single-expert audit of 2,000 examples (98.25% accepted). Evaluation is limited to MedSG-Bench.

📄 Citation

If you use MedChain-R1 in your research, please cite our paper:

@article{dukre2026think,
  title   = {Think Before You Locate: Clinical Chain-of-Thought Supervision for Sequential Medical Image Grounding},
  author  = {Dukre, Adinath Madhavrao and Kolli, Govinda and Lu, Yifan and Zou, Ziyun and Wasi, Azmine Toushik and
             Atito, Sara and Bozorgtabar, Behzad and Mahapatra, Dwarikanath and Awais, Muhammad and Razzak, Imran},
  journal = {arXiv preprint arXiv:2611.00041},
  year    = {2026}
}