SPARC / README.md

Model card: add Kenli Li to the citation author list

fd6b5f1 verified 16 days ago

6.22 kB

---
license: cc-by-nc-4.0
library_name: pytorch
tags:
- medical-imaging
- computed-tomography
- x-ray
- sparse-view-reconstruction
- foundation-model
extra_gated_fields:
  First Name: text
  Last Name: text
  Affiliation: text
  Job title:
    type: select
    options:
    - Student
    - Research Graduate
    - AI researcher
    - AI developer/engineer
    - Other
  geo: ip_location
extra_gated_button_content: Submit
---
# SPARC — pretrained backbone

SPARC is a foundation model that reconstructs a three-dimensional CT volume from sparse X-ray projections. This repository holds the pretrained weights from which every SPARC reconstruction model is adapted. They accompany the manuscript *A foundation model for three-dimensional anatomical reconstruction from sparse X-ray projections* (2026).

Code, installation and data preparation: [HORIZONHealthcare/SPARC](https://github.com/HORIZONHealthcare/SPARC). The sixteen per-dataset reconstruction models are in [lyqun/SPARC-reconstruction](https://huggingface.co/lyqun/SPARC-reconstruction).

**Access.** Log in, fill in the short form on this page, and access is granted straight away. Then log in on your machine with `hf auth login` before downloading.

## Files

| File | Contents | Use |
|---|---|---|
| `sparc_stage2_backbone.pth` | Stage-2 model: 2D projection encoder, 3D feature volume and point decoder (136.1 M parameters, 544 MB) | Starting point for reconstruction on a new dataset, or for the released reconstruction models |
| `sparc_stage1_ctmae.pth` | Stage-1 3D masked autoencoder on CT, encoder and decoder (340.4 M parameters, 1.36 GB) | Only needed to rerun Stage-2 pretraining, where its encoder is the frozen target |

Each file is a plain dictionary with the weights (`model`, float32), the settings needed to rebuild the model (`config`) and the training iteration (`iter`). It loads with `torch.load(..., weights_only=True)`.

| File | SHA-256 |
|---|---|
| `sparc_stage2_backbone.pth` | `b9d8605e77458cd1af5024727df33f8a286585432ce1f62ca925c1114c45098d` |
| `sparc_stage1_ctmae.pth` | `f13ddf31cba1fdfabad7708bfb58b303e9c3cacd5269d3aedcd1c92f1b3b8d85` |

## Model

- **Stage 1.** A 3D masked autoencoder (ViT-L/16 encoder, 0.75 masking ratio, normalised-pixel loss) trained on 256³ CT crops for 200,000 iterations. Its encoder turns a CT volume into 16³ = 4,096 tokens.
- **Stage 2.** Projections are rendered from CT on the fly with randomised acquisition geometry: 2 to 16 projections, uniform, random, limited-arc, bi-planar and clustered angular patterns, and randomised source and detector distances. Each projection passes through a shared ConvNeXt pyramid with a transformer at the coarsest scale, into which the Plücker coordinates of every ray are injected. A 16³ grid of points is projected into all projections, sampled, max-fused across projections and refined by a 3D transformer into a feature volume of 16³ × 768. A point decoder reads that volume at any query point and predicts Hounsfield units. A second head that predicted the frozen Stage-1 tokens was used during pretraining and is not included, as in the paper. 150,000 iterations on 16 NVIDIA GH200 GPUs, one volume per GPU.

Pretraining (Stages 1 and 2) took about 1,700 GPU-hours; with these weights, it need not be repeated.

The released backbone reconstructs directly, but the paper's reconstruction results come from per-dataset adaptation of it; for those use [lyqun/SPARC-reconstruction](https://huggingface.co/lyqun/SPARC-reconstruction).

## Training data

Both stages were trained on the official training split of CT-RATE (chest CT; 47,149 volumes from 24,128 scans of 20,000 patients). Every chest evaluation volume in the paper comes from CT-RATE's official validation split, so no pretraining patient appears in any evaluation list. The exact list is [`splits/ctrate_pretrain.csv`](https://github.com/HORIZONHealthcare/SPARC/blob/main/splits/ctrate_pretrain.csv).

## Usage

```bash
git clone https://github.com/HORIZONHealthcare/SPARC.git && cd SPARC
conda create -n sparc python=3.11 -y && conda activate sparc
pip install torch==2.5.1 --index-url https://download.pytorch.org/whl/cu124
pip install -r requirements.txt
hf auth login        # after filling in the form on this page
hf download lyqun/SPARC sparc_stage2_backbone.pth --local-dir weights
# adapt to a dataset, starting from the pretrained backbone
python finetune_foundation_recon.py --config configs/cq500_v8.yaml \
    --init_ckpt weights/sparc_stage2_backbone.pth
```

To rerun Stage-2 pretraining without Stage 1, download `sparc_stage1_ctmae.pth` and set `target.ctmae_init_path` in `configs/stage2_backbone.yaml` to its path.

## Limitations

- SPARC was trained and evaluated on projections rendered from CT (digitally reconstructed radiographs), not on radiographs acquired on an X-ray system. Scatter, detector response and beam hardening on a real system differ from the rendering.
- Reconstruction needs an accurately calibrated acquisition geometry. In the paper's simulations, a 5% error in the assumed source-to-detector distance cost SPARC up to 12.9 dB in PSNR, far more than photon noise at 10⁴ incident photons per detector pixel (at most 3.4 dB).
- Pretraining used chest CT only.
- For research use only. It is not a medical device.

## License

The weights are released under [CC BY-NC 4.0](https://creativecommons.org/licenses/by-nc/4.0/), the same licence as the code. Commercial use is not permitted. Use of the weights must also respect the terms of the datasets they were trained on. The models are for research use only and are not medical devices.

## Citation

```bibtex
@misc{lin2026sparc,
  title  = {A foundation model for three-dimensional anatomical reconstruction from sparse X-ray projections},
  author = {Lin, Yiqun and Xu, Jiayang and Ju, Lie and Guo, Jiarong and Wang, Weiru and Wang, Hualiang and Yao, Huifeng and Sun, Haoran and Pu, Bin and Li, Kenli and Alexander, Daniel C. and Cui, Hejie and Zhou, Yukun},
  year   = {2026},
  note   = {Manuscript}
}
```

Please also cite CT-RATE: Hamamci I. E. *et al.* Generalist foundation models from a multimodal dataset for 3D computed tomography. *Nature Biomedical Engineering* (2026).

	---
	license: cc-by-nc-4.0
	library_name: pytorch
	tags:
	- medical-imaging
	- computed-tomography
	- x-ray
	- sparse-view-reconstruction
	- foundation-model
	extra_gated_fields:
	First Name: text
	Last Name: text
	Affiliation: text
	Job title:
	type: select
	options:
	- Student
	- Research Graduate
	- AI researcher
	- AI developer/engineer
	- Other
	geo: ip_location
	extra_gated_button_content: Submit
	---

	# SPARC — pretrained backbone

	SPARC is a foundation model that reconstructs a three-dimensional CT volume from sparse X-ray projections. This repository holds the pretrained weights from which every SPARC reconstruction model is adapted. They accompany the manuscript A foundation model for three-dimensional anatomical reconstruction from sparse X-ray projections (2026).

	Code, installation and data preparation: [HORIZONHealthcare/SPARC](https://github.com/HORIZONHealthcare/SPARC). The sixteen per-dataset reconstruction models are in [lyqun/SPARC-reconstruction](https://huggingface.co/lyqun/SPARC-reconstruction).

	Access. Log in, fill in the short form on this page, and access is granted straight away. Then log in on your machine with `hf auth login` before downloading.

	## Files

	\| File \| Contents \| Use \|
	\|---\|---\|---\|
	\| `sparc_stage2_backbone.pth` \| Stage-2 model: 2D projection encoder, 3D feature volume and point decoder (136.1 M parameters, 544 MB) \| Starting point for reconstruction on a new dataset, or for the released reconstruction models \|
	\| `sparc_stage1_ctmae.pth` \| Stage-1 3D masked autoencoder on CT, encoder and decoder (340.4 M parameters, 1.36 GB) \| Only needed to rerun Stage-2 pretraining, where its encoder is the frozen target \|

	Each file is a plain dictionary with the weights (`model`, float32), the settings needed to rebuild the model (`config`) and the training iteration (`iter`). It loads with `torch.load(..., weights_only=True)`.

	\| File \| SHA-256 \|
	\|---\|---\|
	\| `sparc_stage2_backbone.pth` \| `b9d8605e77458cd1af5024727df33f8a286585432ce1f62ca925c1114c45098d` \|
	\| `sparc_stage1_ctmae.pth` \| `f13ddf31cba1fdfabad7708bfb58b303e9c3cacd5269d3aedcd1c92f1b3b8d85` \|

	## Model

	- Stage 1. A 3D masked autoencoder (ViT-L/16 encoder, 0.75 masking ratio, normalised-pixel loss) trained on 256³ CT crops for 200,000 iterations. Its encoder turns a CT volume into 16³ = 4,096 tokens.
	- Stage 2. Projections are rendered from CT on the fly with randomised acquisition geometry: 2 to 16 projections, uniform, random, limited-arc, bi-planar and clustered angular patterns, and randomised source and detector distances. Each projection passes through a shared ConvNeXt pyramid with a transformer at the coarsest scale, into which the Plücker coordinates of every ray are injected. A 16³ grid of points is projected into all projections, sampled, max-fused across projections and refined by a 3D transformer into a feature volume of 16³ × 768. A point decoder reads that volume at any query point and predicts Hounsfield units. A second head that predicted the frozen Stage-1 tokens was used during pretraining and is not included, as in the paper. 150,000 iterations on 16 NVIDIA GH200 GPUs, one volume per GPU.

	Pretraining (Stages 1 and 2) took about 1,700 GPU-hours; with these weights, it need not be repeated.

	The released backbone reconstructs directly, but the paper's reconstruction results come from per-dataset adaptation of it; for those use [lyqun/SPARC-reconstruction](https://huggingface.co/lyqun/SPARC-reconstruction).

	## Training data

	Both stages were trained on the official training split of CT-RATE (chest CT; 47,149 volumes from 24,128 scans of 20,000 patients). Every chest evaluation volume in the paper comes from CT-RATE's official validation split, so no pretraining patient appears in any evaluation list. The exact list is [`splits/ctrate_pretrain.csv`](https://github.com/HORIZONHealthcare/SPARC/blob/main/splits/ctrate_pretrain.csv).

	## Usage

	```bash
	git clone https://github.com/HORIZONHealthcare/SPARC.git && cd SPARC
	conda create -n sparc python=3.11 -y && conda activate sparc
	pip install torch==2.5.1 --index-url https://download.pytorch.org/whl/cu124
	pip install -r requirements.txt
	hf auth login # after filling in the form on this page
	hf download lyqun/SPARC sparc_stage2_backbone.pth --local-dir weights

	# adapt to a dataset, starting from the pretrained backbone
	python finetune_foundation_recon.py --config configs/cq500_v8.yaml \
	--init_ckpt weights/sparc_stage2_backbone.pth
	```

	To rerun Stage-2 pretraining without Stage 1, download `sparc_stage1_ctmae.pth` and set `target.ctmae_init_path` in `configs/stage2_backbone.yaml` to its path.

	## Limitations

	- SPARC was trained and evaluated on projections rendered from CT (digitally reconstructed radiographs), not on radiographs acquired on an X-ray system. Scatter, detector response and beam hardening on a real system differ from the rendering.
	- Reconstruction needs an accurately calibrated acquisition geometry. In the paper's simulations, a 5% error in the assumed source-to-detector distance cost SPARC up to 12.9 dB in PSNR, far more than photon noise at 10⁴ incident photons per detector pixel (at most 3.4 dB).
	- Pretraining used chest CT only.
	- For research use only. It is not a medical device.

	## License

	The weights are released under [CC BY-NC 4.0](https://creativecommons.org/licenses/by-nc/4.0/), the same licence as the code. Commercial use is not permitted. Use of the weights must also respect the terms of the datasets they were trained on. The models are for research use only and are not medical devices.

	## Citation

	```bibtex
	@misc{lin2026sparc,
	title = {A foundation model for three-dimensional anatomical reconstruction from sparse X-ray projections},
	author = {Lin, Yiqun and Xu, Jiayang and Ju, Lie and Guo, Jiarong and Wang, Weiru and Wang, Hualiang and Yao, Huifeng and Sun, Haoran and Pu, Bin and Li, Kenli and Alexander, Daniel C. and Cui, Hejie and Zhou, Yukun},
	year = {2026},
	note = {Manuscript}
	}
	```

	Please also cite CT-RATE: Hamamci I. E. et al. Generalist foundation models from a multimodal dataset for 3D computed tomography. Nature Biomedical Engineering (2026).