--- library_name: "fastforks" pipeline_tag: "image-segmentation" license: "cc-by-sa-4.0" tags: - "volume" - "medical-imaging" - "3d-segmentation" - "ct-segmentation" - "cuda" - "pytorch" - "fastforks" - "fastforks-card" base_model: - "MIC-DKFZ/nnFoundationCNN" - "MIC-DKFZ/nnFoundationViT" --- # nnFoundation (fastforks) nnFoundation (Harsy et al. 2026): DKFZ's self-supervised 3D radiology foundation encoders, pre-trained with nnssl on 2.1 M CT, MRI and PET volumes. `nnfoundation-cnn` is the ResEnc-L U-Net encoder (6 stages, 32-320 channels, strides 1-32), `nnfoundation-vit` the Primus ViT (40 layers, width 1056, 8³ patch tokens, fixed 192³ input). Gives the encoder feature maps of a z-score normalised volume for fine-tuning or frozen-feature use; the MAE decoder is not included. This repo is the model card only: it holds no weights. fastforks downloads the authors' weights from the links below and checks them against the SHA-256 in the model card. Code: . Kernels for the `kernels` backend: . Original code: at `3d774e942256`. Author weight repos: , . Task: volume. Tier: experimental. Outputs: `features`. Backends: `portable`, `kernels`. Input: a CT volume `[D, H, W]` in Hounsfield units and its voxel spacing in mm (`volume.read_nifti` reads both from a NIfTI file). Output: a class index per voxel `[D, H, W]` on the input grid (0 = background). Encoder models (nnFoundation) take a CT or MRI volume and give its encoder feature maps. ## Variants Weights are the authors' files (not hosted here). SHA-256 shortened to 12 hex digits; the full hashes are in [`card.toml`](https://github.com/kleinicke/fastforks/blob/main/src/fastforks/volume/models/nnfoundation/card.toml). | variant | weights (author link) | SHA-256 | weights licence | commercial use | |---|---|---|---|---| | `nnfoundation-cnn@1` | [nnFoundationCNN/checkpoint_final.pth](https://huggingface.co/MIC-DKFZ/nnFoundationCNN/resolve/f443c09319988e833b3d9589bf95909e4443d7b2/checkpoint_final.pth) | `ac262d3e8c22` | CC-BY-SA-4.0 | yes | | `nnfoundation-vit@1` | [nnFoundationViT/checkpoint_final.pth](https://huggingface.co/MIC-DKFZ/nnFoundationViT/resolve/c691f2115140458ec64017712eb8ebbc8dde6ae4/checkpoint_final.pth) | `022ad18fc388` | CC-BY-SA-4.0 | yes | ## Licence Original-code strings, recorded as they are (research-use package; check them before any other use). Code: Apache-2.0. Licence files: `LICENSE`. Training data: 2.1 million CT, MRI and PET volumes from 125 institutional and public datasets (Human Radiome Project); the dataset list is not published with the weights. Share-alike: models fine-tuned from these weights are adapted material under CC-BY-SA-4.0.. Licence evidence: https://huggingface.co/MIC-DKFZ/nnFoundationCNN/blob/f443c09319988e833b3d9589bf95909e4443d7b2/README.md ; https://huggingface.co/MIC-DKFZ/nnFoundationViT/blob/c691f2115140458ec64017712eb8ebbc8dde6ae4/README.md ; https://github.com/MIC-DKFZ/dynamic-network-architectures/blob/main/LICENSE ## Input size any D x H x W volume, no resampling, one encoder pass (a fixed-grid ViT: each side up to its grid) Tested up to (portable, fp16, RTX 5060 Ti 16 GB): `nnfoundation-cnn` 384x384x384 voxels (14.2 GiB peak allocated, 14.8 reserved); `nnfoundation-vit` 192x192x192 voxels (1.6 GiB peak allocated, 1.8 reserved) ## Speed and memory ### Volume, synthetic | model | input | time (ms): original fp32 | original | portable | kernels | kernels speed-up: vs original | vs portable | peak memory (MiB, allocated / reserved): original | portable | kernels | | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | | `nnfoundation-cnn` | 200x200 | 501 | 274 | 227 | 211 | 1.3x | 1.1x | 3818 / 4628 | 3011 / 3110 | 2985 / 3710 | | `nnfoundation-vit` | 192x192 | OOM | 1930 | 1848 | 1732 | 1.1x | 1.1x | 3930 / 4298 | 1669 / 1884 | 1669 / 1884 | ### Startup and first call | model | load (s): original | portable | kernels | first call (s): original | portable | kernels | | --- | --- | --- | --- | --- | --- | --- | | `nnfoundation-cnn` | 1.18 | 0.67 | 0.39 | 0.69 | 0.33 | 0.29 | | `nnfoundation-vit` | 9.54 (192x192) | 1.24 (192x192) | 1.27 (192x192) | 2.37 (192x192) | 1.87 (192x192) | 1.76 (192x192) | ### torch.compile and TensorRT | model | input | time: portable | compile | TensorRT fp16 | kernels | preparation: compile (cold first call) | ONNX + engine build | peak MiB (alloc / reserved): portable | compile | kernels | TensorRT (engine + torch) | | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | | `nnfoundation-cnn` | 200x200 | 227 ms | 170 ms | planned for 0.2.1 | 211 ms | 19.8 s | – | 3011 / 3110 | 5067 / 5110 | 2985 / 3710 | – | | `nnfoundation-vit` | 192x192 | 1848 ms | 1440 ms | planned for 0.2.1 | 1732 ms | 69.5 s | – | 1669 / 1884 | 1406 / 1484 | 1669 / 1884 | – | Fixed-size cells: time (ms) / peak allocated / peak reserved memory (MiB). Hardware, protocol and caveats: [benchmarks](https://github.com/kleinicke/fastforks/blob/main/docs/benchmarks.md) (RTX 5060 Ti 16 GB, sm_120). Original = the original code named above at the precision of portable (fp16, or `(fp32)` where fp16 is unsupported or not finite); original fp32 = its default fp32 run. Peak memory is the torch allocator's. ## Accuracy ### Parity with the original code Every variant is checked against reference outputs of the original code at its pinned commit. CPU, float32: max |diff| <= 1.8e-06 vs the original. GPU, float16 (the default), portable and kernels: mean |diff| <= 0.15 % and p99 <= 1.1 % of the reference's median |value| (absolute error of the output `features`). ## Usage ```python from fastforks import volume model = volume.load("nnfoundation-cnn") # mode="auto": the preferred backend from recorded measurements result = model(volume) # volume: [D, H, W] float32 (CT or MRI, no resampling; z-scored inside) result.features # tuple of [C_i, D/s_i, H/s_i, W/s_i] float32 maps at the strides model.strides result.backend # what ran, e.g. Backend(mode='portable', device='cuda:0', dtype='float16') ``` Install with `pip install fastforks` (see ). Other variants: `nnfoundation-cnn`, `nnfoundation-vit`. ## Kernels backend `volume.load("nnfoundation-cnn", mode="kernels")` runs our CUDA kernels (`vol_norm`). It needs a prebuilt kernel for your GPU architecture (downloaded from on first use) and a recent NVIDIA driver; if there is none it raises `BackendUnavailableError` with the reason and never falls back silently. `mode="auto"` picks `kernels` when it can run and logs the choice.