Instructions to use suitch/scan2report-label-assembler-grpo with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- PEFT
How to use suitch/scan2report-label-assembler-grpo with PEFT:
from peft import PeftModel from transformers import AutoModelForCausalLM base_model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3.5-4B") model = PeftModel.from_pretrained(base_model, "suitch/scan2report-label-assembler-grpo") - Notebooks
- Google Colab
- Kaggle
GRPO-tuned label-to-report assembler
Part of the master's thesis From Scan to Report: Detecting, Segmenting and Reporting Findings in Chest CT (code: https://github.com/Switchharsh/Switchharsh-From_Scan_to_Report_Master_Thesis; thesis: https://github.com/Switchharsh/Switchharsh-From_Scan_to_Report_Master_Thesis/tree/main/thesis).
The label assembler (scan2report-label-assembler) further trained with GRPO using a RadBERT-based CRG reward. Prompt: the bare label list Findings present: ... Findings absent: ... (no task sentence). Only the LoRA adapter is included (the checkpoint is a plain PEFT language model).
| labels given | CRG | macro-F1 |
|---|---|---|
| reference labels | 0.9348 | 0.9628 |
| FlexiCT attention classifier, t=0.60, raw text | 0.5447 | 0.5558 |
| same, with coverage post-processing | 0.5461 | 0.5563 |
| another patient's classifier labels | 0.3507 | - |
GRPO raises the CRG gap between correct and mismatched labels (+0.0466 over the supervised assembler, Holm p=0.0003) but not the GREEN or CRIMSON gaps; it writes more findings per report (5.40 vs 4.66).
How it fits together
- Input: the 8x8x8 FlexiCT token grid of a chest CT (
flexict/tokens8, 864-dim per token) from CT-RATE. projector.pt: MLP (864 -> 2560 -> 2560, LayerNorm) that maps the visual tokens into the LLM embedding space.adapter_model.safetensors+adapter_config.json: LoRA (r=16, alpha=32) on Qwen/Qwen3.5-4B.- Loading and generation code (model wrapper, data, evaluation): https://github.com/Switchharsh/Switchharsh-From_Scan_to_Report_Master_Thesis (
ctbench/rgen/model.py,evaluate.py). The adapter is not a stand-alone chat model: the projector and the FlexiCT tokens are required. - Task instruction used in every run:
Write the findings and impression for this chest CT.Greedy decoding, at most 512 new tokens.
Evaluation protocol
1,000 patient-sampled CT-RATE validation cases; CRG (micro) and macro-F1 with the RadBERT label reader (suitch/radbert-english-ctrate-classifier, threshold 0.5). Control = another patient's scan tokens (mismatched). Numbers are from the thesis From Scan to Report: Detecting, Segmenting and Reporting Findings in Chest CT.
Limitations / licence
Research use only, not for clinical use. Trained on CT-RATE (CC BY-NC-SA 4.0), so the weights are released under the same non-commercial terms. Reports can contain false findings; see the thesis for the error analysis.
- Downloads last month
- -