README.md

---
license: cc-by-sa-4.0
library_name: nnunetv2
tags:
  - medical-imaging
  - radiology
  - 3d
  - self-supervised
  - foundation-model
  - nnunet
  - nnssl
  - arxiv:2609.26924
---

![nnFoundation](nnFoundation_banner.webp)

## nnFoundationCNN
<sub>Copyright German Cancer Research Center (DKFZ) and contributors. Please make sure that your usage of these models is in compliance with their license.<sub>

[![arXiv](https://img.shields.io/badge/arXiv-2609.26924-b31b1b.svg)](https://arxiv.org/abs/2609.26924)

`nnFoundationCNN` is one of the two **nnFoundation** 3D radiology foundation models, pre-trained
with the [`nnssl`](https://github.com/MIC-DKFZ/nnssl) self-supervised learning framework.

## Model pair

| Model | Architecture | Details | Params | Repository |
|---|---|---|---|---|
| **nnFoundationCNN** | [ResEnc](https://arxiv.org/abs/2404.09556) | 6 stages, features 32–64–128–256–320–320 | 102M | this repository |
| nnFoundationViT | [Primus](https://arxiv.org/abs/2503.01835) | 40 layers, embedding dim 1056, 16 heads, 8³ patch tokens | 674M | [MIC-DKFZ/nnFoundationViT](https://huggingface.co/MIC-DKFZ/nnFoundationViT) |

## Using nnFoundation

To fine-tune nnFoundation on your own downstream tasks, use one of our dedicated repositories:

- **Segmentation:** [nnU-Net -- Fine-tuning from nnssl checkpoints](https://github.com/MIC-DKFZ/nnUNet/blob/master/documentation/finetuning_from_nnssl_checkpoints.md)
- **Detection:** [nnDetection -- Fine-tuning](https://github.com/MIC-DKFZ/nnDetection/blob/finetuning/docs/finetuning.md)
- **Classification:** [image_classification -- Fine-tuning nnFoundation](https://github.com/MIC-DKFZ/image_classification/blob/master/docs/nnfoundation.md)
- **Report generation:** TBA

### Planning and preprocessing straight from this repository

[nnU-Net](https://github.com/MIC-DKFZ/nnUNet/blob/master/documentation/finetuning_from_nnssl_checkpoints.md) can pull these weights itself — pass the repository URL where a checkpoint path is
expected. Set `nnssl_pretrained_models` first; that is where the download is cached.

```bash
export nnssl_pretrained_models=/path/to/pretrained_models

nnUNetv2_preprocess_like_nnssl \
    -d <DATASET_ID> \
    -n <UniquePretrainingName> \
    -pc https://huggingface.co/MIC-DKFZ/nnFoundationCNN \
    -am like_pretrained
```

Or download the file yourself and pass a local path:

```python
from huggingface_hub import hf_hub_download

ckpt = hf_hub_download("MIC-DKFZ/nnFoundationCNN", "checkpoint_final.pth")
```

## Repository contents

| File | Purpose |
|---|---|
| `checkpoint_final.pth` | the pre-trained weights (102M parameters) |
| `adaptation_plan.json` | architecture + preprocessing plan; nnU-Net reads this to confirm compatibility |
| `config.json` | placeholder so the Hub records download counts |

## Expected input

Single-channel 3D volumes, **Z-score normalised**, with **no resampling**.
Recommended downstream patch size: **192 × 192 × 192**.

## Checkpoint format

`checkpoint_final.pth` is a `torch.save` dictionary that loads safely with `weights_only=True`:

| Key | Contents |
|---|---|
| `network_weights` | the pre-trained `state_dict` |
| `nnssl_adaptation_plan` | same content as `adaptation_plan.json` |
| `citations` | the reference(s) to cite when using these weights |

> **Note:** this checkpoint stores a number of `state_dict` entries as aliases of the same
> underlying tensor (the `all_modules.*` keys mirror the named conv/norm modules). This is
> expected for ResEnc and is why the file is 410 MB rather than ~1.5 GB — do not deduplicate
> these keys, or loading will fail.

## Citation

If you use the **nnFoundation** models or the `nnssl` framework, please cite:

<details>
<summary>nnFoundation BibTeX</summary>

```bibtex
@misc{harsy2026nnfoundation3dfoundationmodels,
      title={nnFoundation: 3D Foundation Models for Radiology}, 
      author={Constantin Ulrich Harsy and Tassilo Wald and Karol Gotkowski and Yannick Kirchhoff and Marcel Knopp and Maximilian Rokuss and Elisa Stegmeier and Philipp Schader and Dasha Trofimova and Raphael Stock and Kim-Celine Kahl and Stephen Schaumann and Selen Erkan and David Zimmerer and Stefan Denner and Moritz Langenberg and Sebastian Ziegler and Katharina Eckstein and Maximilian Fischer and Jonathan Suprijadi and Bálint Kovács and Benjamin Hamm and Anand Deshpande and Dimitrios Bounias and Nico Disch and Shuhan Xiao and Jessica Kächele and Jan Sellner and Rajesh Baidya and Jeremias Traub and Lars Krämer and Maximilian Zenk and Tim Rädsch and Stefan Dvoretskii and Robin Peretzke and Jonathan Deissler and Alexandra Ertl and Partha Ghosh and Kris Dreher and Stefan Dinkelacker and Annika Reinke and Evangelia Christodoulou and Numan Saeed and Yoland Savriama and Santiago Estrada and David Kügler and Laura Alexandra Daza Barragan and Cristina Isabel Gonzalez Osorio and Jan Peeken and Michael Baumgartner and Marvin Teichmann and Guillaume Chabin and Matthias Kirchler and Valentin Koch and for the ALFA study and Markus Hohenhaus and Dimitri Koslov and Nina Decker and Mohammad Yaqub and Arnd Heuser and Martin Reuter and Julia A. Schnabel and Tobias Heimann and Florin Ghesu and Paul Brachmann and Claus P. Heußel and Alexander Radbruch and Gianluca Brugnara and Aditya Rastogi and Martha Foltyn-Dumitru and Heinz-Peter Schlemmer and Ignaz Reicht and Julius C. Holzschuh and Michael Bach and Bram Stieltjes and Kai Schlamp and Lena Maier-Hein and Marco Nolden and Ralf Floca and Paul F. Jäger and Philipp Vollmuth and Fabian Isensee and Klaus H. Maier-Hein},
      year={2026},
      eprint={2609.26924},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2609.26924}, 
}
```

</details>

If you use the `nnssl` framework, the **OpenMind dataset**, or the **OpenMind checkpoints**, please cite:

<details>
<summary>OpenMind BibTeX</summary>

```bibtex
@InProceedings{Wald_2025_ICCV,
    author    = {Wald, Tassilo and Ulrich, Constantin and Suprijadi, Jonathan and Ziegler, Sebastian and Nohel, Michal and Peretzke, Robin and Kohler, Gregor and Maier-Hein, Klaus},
    title     = {An OpenMind for 3D Medical Vision Self-supervised Learning},
    booktitle = {Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)},
    month     = {October},
    year      = {2025},
    pages     = {23839-23879}
}
```

</details>