Update model card with approved content and benchmarks

README.md

@@ -4,6 +4,12 @@ license: openmdw-1.1
 language:
 - en
 pipeline_tag: image-text-to-text
+base_model:
+- Qwen/Qwen3.5-4B
+base_model_relation: finetune
+datasets:
+- ibrahimhamamci/CT-RATE
+- BodyMaps/CancerVerse
 tags:
 - medical-imaging
 - ct
@@ -18,22 +24,26 @@ tags:
 NV-Reason-CT is a 3D vision-language model (VLM) for CT image analysis. It
 combines a native 3D vision encoder (3D ViT) with a language model and is
 designed for radiology report generation, general question answering, and
-multi-step reasoning across chest and abdominal CT volumes. Computed tomography
-encodes clinically important anatomy across hundreds of slices, yet most
-vision–language systems either operate on 2D images or compress volumetric
-features before language decoding. The 3D vision encoder converts a
-384×384×384-mm input volume into a 24×24×24 grid of 13,824 visual tokens. All
-tokens and their corresponding 3D positions are passed to the language model
-without spatial downsampling, while 3D MRoPE preserves their spatial
-relationships within the LLM. Input volumes are automatically cropped to the
-chest or abdomen and resampled to 2-mm isotropic resolution before processing.
-
-The model was trained end-to-end using SFT and GRPO on a curated dataset of approximately 550,000
-structured QA examples from 70,111 unique CT volume inputs. The training corpus integrates standardized
-reports, abnormality-focused QA, multi-turn interactions, and radiologist-authored reasoning collected through
-recorded and transcribed expert CT interpretations. These expert annotations also guide the generation
-of report-grounded synthetic reasoning data for SFT, while GRPO uses verifiable rewards over chest and
-abdominal abnormality sets.
+multi-step reasoning across chest and abdominal CT volumes.
+
+Computed tomography encodes clinically important anatomy across hundreds of
+slices, yet most vision–language systems either operate on 2D images or
+compress volumetric features before language decoding. The 3D vision encoder
+converts a 384×384×384-mm input volume into a 24×24×24 grid of 13,824 visual
+tokens. All tokens and their corresponding 3D positions are passed to the
+language model without spatial downsampling, while 3D MRoPE preserves their
+spatial relationships within the LLM. Input volumes are automatically cropped
+to the chest or abdomen and resampled to 2-mm isotropic resolution before
+processing.
+
+The model was trained end-to-end using Supervised Fine-Tuning (SFT) and Group
+Relative Policy Optimization (GRPO) on a curated dataset of approximately
+550,000 structured QA examples from 70,111 unique CT volume inputs. The
+training corpus integrates standardized reports, abnormality-focused QA,
+multi-turn interactions, and radiologist-authored reasoning collected through
+recorded and transcribed expert CT interpretations. These expert annotations
+also guide the generation of report-grounded synthetic reasoning data for SFT,
+while GRPO uses verifiable rewards over chest and abdominal abnormality sets.
 
 💻 [\[GitHub code\]](https://github.com/NVIDIA-Medtech/NV-Reason-CT)  
 🩻 [\[Web Demo\]](https://huggingface.co/spaces/nvidia/nv-reason-ct)
@@ -48,7 +58,7 @@ Python 3.11+ and a CUDA-capable PyTorch installation are recommended.
 python -m pip install -r https://huggingface.co/nvidia/NV-Reason-CT/resolve/main/requirements.txt
 ```
 
-## Inference
+## Quick Start
 
 The example below loads the model once and defines a reusable function for
 deterministic inference on `.nii.gz` volumes. Use `anatomy_region="chest"`
@@ -194,31 +204,22 @@ the anatomy heuristic does not work reliably for your use case, manually crop
 the input CT and set `anatomy_region=None`; the processor will then take a
 centered crop.
 
-## License
-
-This model is released under the [OpenMDW-1.1 license](LICENSE).
-
-## Architecture
-
-- Language model: Qwen3.5-4B
-- 3D vision encoder: Primus 3D ViT, initialized from COLIPRI
-- 3D input route: `AutoProcessor(...)(images3d=..., anatomy_region=...)`
-- Hugging Face integration: `AutoModelForImageTextToText` and `AutoProcessor`
+## License/Terms of Use
 
+Use of this model is governed by the [OpenMDW-1.1 License](https://github.com/OpenMDW/OpenMDW/blob/main/1.1/LICENSE.OpenMDW-1.1).
 
 ### Deployment Geography
 
 Global
 
-### Intended Use
+### Use Case
 
-Intended users include radiologists, medical students, and medical researchers
-conducting research or educational work on chest and abdominal CT. Intended
-uses include report generation, question answering, and reviewable AI-generated
-reasoning analyses.
-
-**Important Medical AI Considerations**
+Radiologists, medical students, and medical researchers would be expected to
+use this model for chest and abdominal CT report generation, question
+answering, and reviewable AI-generated reasoning analyses in research and
+educational settings.
 
+**Important Medical AI Considerations:**
 This model is designed for research and educational purposes only and should
 not be used for clinical diagnosis or treatment decisions. All outputs should
 be reviewed by qualified medical professionals. Generated reasoning is
@@ -226,77 +227,166 @@ reviewable model output and is not guaranteed to represent the model's internal
 computation. It is intended to support medical education and research, not
 replace clinical judgment.
 
-### Release Date
+## Model Architecture
+
+**Architecture Type:** Transformer (Vision-Language Model)<br>
+**Network Architecture:** Native 3D vision encoder (Primus 3D ViT, initialized from COLIPRI) coupled to a Qwen3.5-4B language model, with 3D MRoPE positional encoding<br>
+**Task:** Vision-Language (report generation, question answering, and reasoning)<br>
+**Base Model:** Qwen3.5-4B<br>
 
-Hugging Face: 2026-09-27 via https://huggingface.co/NVIDIA
+This model was developed by training end-to-end with Supervised Fine-Tuning (SFT) and Group Relative Policy Optimization (GRPO).
 
 **Number of model parameters:** 4.69B total, including the retained 2D vision
 modules for checkpoint compatibility; 4.35B in the active CT pathway, excluding
 the 2D vision encoder and merger.
 
+**Hugging Face integration:**
+
+- 3D input route: `AutoProcessor(...)(images3d=..., anatomy_region=...)`
+- Model and processor classes: `AutoModelForImageTextToText` and `AutoProcessor`
+
 ## Input
 
-- **Input types:** 3D CT volume and text
-- **Input formats:** NIfTI volume (`.nii` or `.nii.gz`) and text prompt
-- **Input parameters:** A 3D CT volume with an accompanying natural-language query
+**Input Type(s):** Image, Text<br>
+**Input Format(s):** NIfTI volume (`.nii` or `.nii.gz`), Text prompts (string)<br>
+**Input Parameters:** Three-Dimensional (3D) CT volumes with accompanying text queries (1D)<br>
+**Other Properties Related to Input:** Accepts 3D CT volumes with voxel intensities in Hounsfield units. Input volumes are automatically cropped to the chest or abdomen and resampled to 2-mm isotropic resolution before processing. Accepts natural language prompts for medical queries, follow-up questions, and reasoning requests.
 
 ### Input Specifications
 
-- **Medical images:** 3D NIfTI CT volumes with voxel intensities in Hounsfield units
-- **Text prompts:** Natural-language requests for radiology reports, answers to questions, or detailed reasoning analyses
-- **Interactive dialogue:** Follow-up questions and clarification requests
+- **Medical Images:** 3D NIfTI CT volumes with voxel intensities in Hounsfield units
+- **Text Prompts:** Natural language requests for radiology reports, answers to questions, or detailed reasoning analyses
+- **Interactive Dialogue:** Support for follow-up questions and clarification requests
 
 ## Output
 
-- **Output type:** Text
-- **Output format:** String
-- **Output content:** Natural-language reports, answers, and reasoning analyses
+**Output Type(s):** Text<br>
+**Output Format:** String<br>
+**Output Parameters:** One-Dimensional (1D) natural language reports, answers, and reasoning analyses<br>
+**Other Properties Related to Output:** Outputs contain structured reasoning showing step-by-step medical analysis, followed by a concise answer. This format enables transparency in the model's reasoning process and supports educational use cases. Reasoning output can be disabled for concise answers.
 
-NV-Reason-CT is designed for inference on NVIDIA GPU-accelerated systems using
-PyTorch and CUDA.
+Our AI models are designed and/or optimized to run on NVIDIA GPU-accelerated
+systems. By leveraging NVIDIA's hardware (GPU cores) and software frameworks
+(CUDA libraries), the model achieves faster training and inference times
+compared to CPU-only solutions.
 
 ## Software Integration
 
-**Runtime engines**
+**Runtime Engine(s):**
 
 - PyTorch
 - Transformers
 
-**Tested hardware**
+**Supported Hardware Microarchitecture Compatibility:**
 
-- NVIDIA H100
-- NVIDIA L40S
+- NVIDIA Ampere
+- NVIDIA Hopper
+- NVIDIA Lovelace
 
-Other NVIDIA GPU architectures may be compatible through PyTorch and CUDA but
-have not been validated for this release.
-
-**Supported operating system**
+**Supported Operating System(s):**
 
 - Linux
 
 The integration of foundation and fine-tuned models into AI systems requires additional testing using use-case-specific data to ensure safe and effective deployment. Following the V-model methodology, iterative testing and validation at both unit and system levels are essential to mitigate risks, meet technical and functional requirements, and ensure compliance with safety and ethical standards before deployment.
 
-## Model Versions
+## Model Version(s)
 
 0.1 - Initial release version for CT reasoning and interpretation with structured thinking output
 
-## Training, Testing, and Evaluation Datasets:
+## Training and Evaluation Datasets
+
+### Dataset Overview
+
+The model was trained on approximately 550,000 structured QA examples derived
+from 70,111 unique CT volumes drawn from three CT datasets: CT-RATE (public),
+CancerVerse (public), and an internal NIH dataset. Chest and abdomen counts
+below are reported per anatomy-specific crop and do not sum to the overall case
+count.
+
+## Training Dataset
+
+**Data Modality:**
+
+- Image
+- Text
+
+**Image Training Data Size:**
+
+- Less than a Million Images
+
+**Text Training Data Size:**
 
-### Dataset Overview:
-Large-scale CT datasets including CT-RATE, CancerVerse, NIH.
+- Less than a Billion Tokens
+
+**Data Collection Method by dataset:**
+
+- Hybrid: Human, Automatic/Sensors
+
+**Labeling Method by dataset:**
+
+- Hybrid: Human, Synthetic
+
+**Properties:** CT-RATE: 47,149 cases across 24,128 studies from 20,000 patients (46,203 chest, 20,243 abdomen), collected at Istanbul Medipol University Mega Hospital with paired radiology reports and multi-abnormality labels. NIH internal: 15,991 cases from 15,991 patients (12,742 chest, 12,243 abdomen). CancerVerse: 22,720 cases from 13,778 patients (4,668 chest, 4,438 abdomen), spanning 4 contrast phases and 13 malignant tumor types with paired radiologist reports. All source data is de-identified.
+
+## Evaluation Dataset
 
-## Training Dataset:
 **Data Modality:**
-* Image
-* Text
+
+- Image
+- Text
+
+**Data Collection Method by dataset:**
+
+- Hybrid: Human, Automatic/Sensors
+
+**Labeling Method by dataset:**
+
+- Hybrid: Human, Synthetic
+
+**Properties:** CT-RATE: 3,039 public validation cases (3,002 after correction) across 1,564 studies from 1,304 patients (3,002 chest; abdomen not reported). NIH internal: 5,092 cases from 5,092 patients (3,981 chest, 4,205 abdomen). CancerVerse contributes no evaluation cases. Held out from the same source datasets as training, with the same modalities and annotation types.
+
+## Benchmarks
+
+On CT-RATE, NV-Reason-CT is compared against 3D contrastive and image-only
+pretrained models, a fused 2D/3D generative MLLM, and a slice-based frontier
+model.
+
+| Model | Type | Macro-F1 | Macro-AUROC |
+|---|---|---|---|
+| **NV-Reason-CT** | Native 3D generative VLM | **0.614** | **0.871** |
+| VoxelFM | 3D image-only pretraining | 0.581 | 0.870 |
+| Pillar-0 | 3D contrastive | 0.544 | 0.861 |
+| ClinFusion-8B | Fused 2D/3D generative MLLM | 0.442 | n/r |
+| CT-CLIP | 3D contrastive | 0.398 | 0.733 |
+| Merlin | 3D contrastive | 0.358 | 0.662 |
+| MedGemma 1.5 | Up to 85 axial slices | 0.303 | n/r |
+
+*CT-RATE classification results (18 labels, fixed uniform threshold).
+NV-Reason-CT is evaluated using a direct Yes/No prompt, with no classification
+head or task-specific adaptation. n/r = not reported.*
+
+## Inference
+
+**Acceleration Engine:** PyTorch, Transformers<br>
+**Test Hardware:**
+
+- H100
+- L40S
 
 ## Ethical Considerations
 
-NVIDIA believes Trustworthy AI is a shared responsibility and has established
-policies and practices to support the development of a wide array of AI
-applications. Users are responsible for ensuring that CT volumes are properly
-de-identified and handled in accordance with applicable privacy requirements.
-Developers should evaluate the model with use-case-specific data and qualified
-medical review before considering any downstream application.
+NVIDIA believes Trustworthy AI is a shared responsibility and we have
+established policies and practices to enable development for a wide array of AI
+applications. When downloaded or used in accordance with our terms of service,
+developers should work with their internal model team to ensure this model
+meets requirements for the relevant industry and use case and addresses
+unforeseen product misuse.
+
+Users are responsible for ensuring that CT volumes are properly de-identified
+and handled in accordance with applicable privacy requirements. Please make
+sure you have proper rights and permissions for all input image and video
+content; if image or video includes people, personal health information, or
+intellectual property, the image or video generated will not blur or maintain
+proportions of image subjects included.
 
 Please report model quality, risk, security vulnerabilities or concerns [here](https://www.nvidia.com/en-us/support/submit-security-vulnerability/).