SonoIntent-Models
Fine-tuned checkpoints and inference code for two reasoning segmentation models on SonoIntent, a clinical objective-driven ultrasound reasoning segmentation benchmark.
Benchmark
SonoIntent pairs each ultrasound image with a clinical question, a reference reasoning that resolves the question to a target structure, and a binary target mask. The full benchmark contains 130,928 image-question samples from 83,324 images across 30 public ultrasound sources. The paper's frozen test split has 22,870 tasks.
The public pixel release,
QingboKang/SonoIntent-Annotations,
contains 16,869 tasks from seven CC BY sources (HC18, PSFHS, BrEaST, CCAUI,
CUBS, FASS, Segthy). The six examples in examples/cases/ come from its
HC18/PSFHS test rows.
Models and paper results
Both checkpoints are the exact weights used for the paper's full 22,870-task test run. No retraining, weight modification or result recomputation was done for this release.
| Model | Checkpoint size | Overall Dice | TargetAcc | TA-Dice |
|---|---|---|---|---|
| VRS-HQ | 14.58 GB (bf16) | 50.82% | 40.18% | 33.50% |
| Sa2VA-14B | 30.74 GB (fp16) | 48.40% | 98.59% | 47.51% |
Overall Dice is computed over the full test split. TargetAcc and TA-Dice follow the benchmark's target-accuracy and target-aware Dice definitions. The six released examples are illustrative and are not a benchmark subset; their Dice values are not comparable to the table above.
Repository layout
SonoIntent-Models/
βββ README.md
βββ NOTICE.md
βββ LICENSES/ # Apache-2.0 and Llama 2 Community License texts
βββ VRS-HQ/
β βββ checkpoint/ # fine-tuned HF checkpoint (config, tokenizer, 3 safetensors shards)
β βββ code/ # vendored model code needed to load the checkpoint
β βββ test.py
β βββ requirements.txt
β βββ README.md
βββ Sa2VA-14B/
β βββ checkpoint/ # fine-tuned HF checkpoint with remote code (7 safetensors shards)
β βββ test.py
β βββ requirements.txt
β βββ README.md
βββ examples/
βββ cases/example_001..006/{image.png,gt_mask.png,sample.json}
βββ outputs/{VRS-HQ,Sa2VA-14B}/example_00N/{predicted_mask.png,comparison.png,result.json,summary.json}
Each model directory is self-contained and can be loaded independently.
Hardware and dependencies
Python 3.10+, PyTorch 2.1+ and the versions in each requirements.txt.
The release was validated with CPU inference (2x Intel Xeon 6530, 64 cores /
128 threads, 503 GB RAM; bf16 for VRS-HQ, fp16 for Sa2VA-14B). Both scripts accept CUDA
devices; the GPU paths are implemented but were not part of this validation.
Approximate CPU memory: VRS-HQ 15 GB, Sa2VA-14B 31 GB. A GPU with 24 GB
(VRS-HQ) or 40 GB (Sa2VA-14B, or two 24 GB GPUs via --vision-device/
--llm-device) is sufficient.
Download
huggingface-cli download QingboKang/SonoIntent-Models --repo-type model --local-dir SonoIntent-Models
cd SonoIntent-Models
examples/ is part of the repository, so no separate dataset download is
needed for the released cases. VRS-HQ also downloads
openai/clip-vit-large-patch14 from the Hub at first run.
Single example
# VRS-HQ, one case
python VRS-HQ/test.py --limit 1
# Sa2VA-14B, one case
python Sa2VA-14B/test.py --limit 1
Both commands default to the checkpoint in their own directory, the six cases
in examples/cases, and outputs in examples/outputs/<model>, independent of
the working directory.
Force CPU or choose devices explicitly:
python VRS-HQ/test.py --device cpu --dtype bfloat16 --threads 64 --limit 1
python Sa2VA-14B/test.py --vision-device cpu --llm-device cpu --dtype float16 --threads 64 --limit 1
Batch over all six cases
python VRS-HQ/test.py \
--cases examples/cases \
--output examples/outputs/VRS-HQ
python Sa2VA-14B/test.py \
--cases examples/cases \
--output examples/outputs/Sa2VA-14B
--limit N restricts the run to the first N cases. --max-new-tokens
defaults to 384, matching the paper.
Outputs
For every case the scripts write to examples/outputs/<model>/<case>/:
predicted_mask.pngβ binary prediction at the original image resolution.comparison.pngβ four panels: ultrasound image, ground truth, predicted mask, prediction overlay; the header repeats the clinical question and target, the footer gives model, Dice and sample ID.result.jsonβ sample_id, public_id, dataset, source, split, question, target, generated text and reasoning, mask validity, Dice/IoU/precision/ recall, pixel counts and the input SHA-256 values.
A summary.json in the model output directory lists all cases and the mean
Dice over the released examples.
Relation to the dataset
The six cases are real test rows of the frozen SonoIntent benchmark, taken
from the public HC18 and PSFHS test splits. image.png and gt_mask.png are
byte-identical to the released SonoIntent-Annotations assets; their SHA-256
values are in each sample.json and image_index.csv. The examples were
chosen before inference to cover fetal head and pubic symphysis, including one
PSFHS image with two different clinical questions and targets, and both
typical and difficult cases.
Licenses and restrictions
See NOTICE.md for the per-component license table and
LICENSES/ for the Apache-2.0 and Llama 2 Community License texts. In short:
- The fine-tuned weights are distributed under the licenses of their upstream releases: VRS-HQ (Apache-2.0, with Chat-UniVi/Llama-2 components) and Sa2VA-14B (Apache-2.0).
- The test scripts are original code. Vendored model code retains its upstream license; the VRS-HQ GitHub repository does not declare a license, so that component is included only to reproduce the published inference and remains the property of its authors.
- The example images and masks are CC BY 4.0 (HC18, PSFHS) and retain their source citations.
- Not a medical device; research use only.
Citation
The SonoIntent paper citation will be added when it is available. Until then, cite the dataset repository and the source datasets used by the examples:
@misc{sonointent2026,
title = {SonoIntent: Clinical Objective-Driven Ultrasound Reasoning Segmentation},
author = {Kang, Qingbo and others},
year = {2026},
note = {Dataset: https://huggingface.co/datasets/QingboKang/SonoIntent}
}
@article{vandenheuvel2018hc18,
title = {Automated measurement of fetal head circumference using 2D ultrasound images},
author = {van den Heuvel, Thomas L. A. and others},
journal = {PLoS ONE},
volume = {13},
number = {8},
pages = {e0200412},
year = {2018},
doi = {10.1371/journal.pone.0200412}
}
@article{lu2024psfhs,
title = {PSFHS: Intrapartum ultrasound image dataset for AI-based segmentation of pubic symphysis and fetal head},
author = {Lu, Yang and others},
journal = {Scientific Data},
volume = {11},
pages = {436},
year = {2024},
doi = {10.1038/s41597-024-03266-4}
}