|
Download README.md from QingboKang/SonoIntent-Models: direct link, hf CLI and curl.
- Browser
- Download file 7.5 kB
-
https://huggingface.co/QingboKang/SonoIntent-Models/resolve/main/README.md
- Command line
-
hf download hf://QingboKang/SonoIntent-Models/README.md
-
curl -L -o README.md https://huggingface.co/QingboKang/SonoIntent-Models/resolve/main/README.md
7.5 kB
| license: other | |
| license_name: mixed-upstream-licenses | |
| license_link: https://huggingface.co/QingboKang/SonoIntent-Models/blob/main/NOTICE.md | |
| pipeline_tag: image-segmentation | |
| tags: | |
| - ultrasound | |
| - medical-imaging | |
| - reasoning-segmentation | |
| - custom_code | |
| # SonoIntent-Models | |
| Fine-tuned checkpoints and inference code for two reasoning segmentation models | |
| on **SonoIntent**, a clinical objective-driven ultrasound reasoning | |
| segmentation benchmark. | |
| ## Benchmark | |
| SonoIntent pairs each ultrasound image with a clinical question, a reference | |
| reasoning that resolves the question to a target structure, and a binary target | |
| mask. The full benchmark contains 130,928 image-question samples from 83,324 | |
| images across 30 public ultrasound sources. The paper's frozen test split has | |
| 22,870 tasks. | |
| The public pixel release, | |
| [`QingboKang/SonoIntent-Annotations`](https://huggingface.co/datasets/QingboKang/SonoIntent-Annotations), | |
| contains 16,869 tasks from seven CC BY sources (HC18, PSFHS, BrEaST, CCAUI, | |
| CUBS, FASS, Segthy). The six examples in `examples/cases/` come from its | |
| HC18/PSFHS test rows. | |
| ## Models and paper results | |
| Both checkpoints are the exact weights used for the paper's full 22,870-task | |
| test run. No retraining, weight modification or result recomputation was done | |
| for this release. | |
| | Model | Checkpoint size | Overall Dice | TargetAcc | TA-Dice | | |
| |---|---:|---:|---:|---:| | |
| | VRS-HQ | 14.58 GB (bf16) | 50.82% | 40.18% | 33.50% | | |
| | Sa2VA-14B | 30.74 GB (fp16) | 48.40% | 98.59% | 47.51% | | |
| Overall Dice is computed over the full test split. TargetAcc and TA-Dice | |
| follow the benchmark's target-accuracy and target-aware Dice definitions. | |
| The six released examples are illustrative and are not a benchmark subset; | |
| their Dice values are not comparable to the table above. | |
| ## Repository layout | |
| ```text | |
| SonoIntent-Models/ | |
| βββ README.md | |
| βββ NOTICE.md | |
| βββ LICENSES/ # Apache-2.0 and Llama 2 Community License texts | |
| βββ VRS-HQ/ | |
| β βββ checkpoint/ # fine-tuned HF checkpoint (config, tokenizer, 3 safetensors shards) | |
| β βββ code/ # vendored model code needed to load the checkpoint | |
| β βββ test.py | |
| β βββ requirements.txt | |
| β βββ README.md | |
| βββ Sa2VA-14B/ | |
| β βββ checkpoint/ # fine-tuned HF checkpoint with remote code (7 safetensors shards) | |
| β βββ test.py | |
| β βββ requirements.txt | |
| β βββ README.md | |
| βββ examples/ | |
| βββ cases/example_001..006/{image.png,gt_mask.png,sample.json} | |
| βββ outputs/{VRS-HQ,Sa2VA-14B}/example_00N/{predicted_mask.png,comparison.png,result.json,summary.json} | |
| ``` | |
| Each model directory is self-contained and can be loaded independently. | |
| ## Hardware and dependencies | |
| Python 3.10+, PyTorch 2.1+ and the versions in each `requirements.txt`. | |
| The release was validated with CPU inference (2x Intel Xeon 6530, 64 cores / | |
| 128 threads, 503 GB RAM; bf16 for VRS-HQ, fp16 for Sa2VA-14B). Both scripts accept CUDA | |
| devices; the GPU paths are implemented but were not part of this validation. | |
| Approximate CPU memory: VRS-HQ 15 GB, Sa2VA-14B 31 GB. A GPU with 24 GB | |
| (VRS-HQ) or 40 GB (Sa2VA-14B, or two 24 GB GPUs via `--vision-device`/ | |
| `--llm-device`) is sufficient. | |
| ## Download | |
| ```bash | |
| huggingface-cli download QingboKang/SonoIntent-Models --repo-type model --local-dir SonoIntent-Models | |
| cd SonoIntent-Models | |
| ``` | |
| `examples/` is part of the repository, so no separate dataset download is | |
| needed for the released cases. VRS-HQ also downloads | |
| `openai/clip-vit-large-patch14` from the Hub at first run. | |
| ## Single example | |
| ```bash | |
| # VRS-HQ, one case | |
| python VRS-HQ/test.py --limit 1 | |
| # Sa2VA-14B, one case | |
| python Sa2VA-14B/test.py --limit 1 | |
| ``` | |
| Both commands default to the checkpoint in their own directory, the six cases | |
| in `examples/cases`, and outputs in `examples/outputs/<model>`, independent of | |
| the working directory. | |
| Force CPU or choose devices explicitly: | |
| ```bash | |
| python VRS-HQ/test.py --device cpu --dtype bfloat16 --threads 64 --limit 1 | |
| python Sa2VA-14B/test.py --vision-device cpu --llm-device cpu --dtype float16 --threads 64 --limit 1 | |
| ``` | |
| ## Batch over all six cases | |
| ```bash | |
| python VRS-HQ/test.py \ | |
| --cases examples/cases \ | |
| --output examples/outputs/VRS-HQ | |
| python Sa2VA-14B/test.py \ | |
| --cases examples/cases \ | |
| --output examples/outputs/Sa2VA-14B | |
| ``` | |
| `--limit N` restricts the run to the first N cases. `--max-new-tokens` | |
| defaults to 384, matching the paper. | |
| ## Outputs | |
| For every case the scripts write to `examples/outputs/<model>/<case>/`: | |
| - `predicted_mask.png` β binary prediction at the original image resolution. | |
| - `comparison.png` β four panels: ultrasound image, ground truth, predicted | |
| mask, prediction overlay; the header repeats the clinical question and | |
| target, the footer gives model, Dice and sample ID. | |
| - `result.json` β sample_id, public_id, dataset, source, split, question, | |
| target, generated text and reasoning, mask validity, Dice/IoU/precision/ | |
| recall, pixel counts and the input SHA-256 values. | |
| A `summary.json` in the model output directory lists all cases and the mean | |
| Dice over the released examples. | |
| ## Relation to the dataset | |
| The six cases are real test rows of the frozen SonoIntent benchmark, taken | |
| from the public HC18 and PSFHS test splits. `image.png` and `gt_mask.png` are | |
| byte-identical to the released `SonoIntent-Annotations` assets; their SHA-256 | |
| values are in each `sample.json` and `image_index.csv`. The examples were | |
| chosen before inference to cover fetal head and pubic symphysis, including one | |
| PSFHS image with two different clinical questions and targets, and both | |
| typical and difficult cases. | |
| ## Licenses and restrictions | |
| See [NOTICE.md](NOTICE.md) for the per-component license table and | |
| `LICENSES/` for the Apache-2.0 and Llama 2 Community License texts. In short: | |
| - The fine-tuned weights are distributed under the licenses of their upstream | |
| releases: VRS-HQ (Apache-2.0, with Chat-UniVi/Llama-2 components) and | |
| Sa2VA-14B (Apache-2.0). | |
| - The test scripts are original code. Vendored model code retains its upstream | |
| license; the VRS-HQ GitHub repository does not declare a license, so that | |
| component is included only to reproduce the published inference and remains | |
| the property of its authors. | |
| - The example images and masks are CC BY 4.0 (HC18, PSFHS) and retain their | |
| source citations. | |
| - Not a medical device; research use only. | |
| ## Citation | |
| The SonoIntent paper citation will be added when it is available. Until then, | |
| cite the dataset repository and the source datasets used by the examples: | |
| ```bibtex | |
| @misc{sonointent2026, | |
| title = {SonoIntent: Clinical Objective-Driven Ultrasound Reasoning Segmentation}, | |
| author = {Kang, Qingbo and others}, | |
| year = {2026}, | |
| note = {Dataset: https://huggingface.co/datasets/QingboKang/SonoIntent} | |
| } | |
| @article{vandenheuvel2018hc18, | |
| title = {Automated measurement of fetal head circumference using 2D ultrasound images}, | |
| author = {van den Heuvel, Thomas L. A. and others}, | |
| journal = {PLoS ONE}, | |
| volume = {13}, | |
| number = {8}, | |
| pages = {e0200412}, | |
| year = {2018}, | |
| doi = {10.1371/journal.pone.0200412} | |
| } | |
| @article{lu2024psfhs, | |
| title = {PSFHS: Intrapartum ultrasound image dataset for AI-based segmentation of pubic symphysis and fetal head}, | |
| author = {Lu, Yang and others}, | |
| journal = {Scientific Data}, | |
| volume = {11}, | |
| pages = {436}, | |
| year = {2024}, | |
| doi = {10.1038/s41597-024-03266-4} | |
| } | |
| ``` | |