Upload folder using huggingface_hub

.gitattributes

@@ -0,0 +1,41 @@
+*.7z filter=lfs diff=lfs merge=lfs -text
+*.arrow filter=lfs diff=lfs merge=lfs -text
+*.bin filter=lfs diff=lfs merge=lfs -text
+*.bz2 filter=lfs diff=lfs merge=lfs -text
+*.ckpt filter=lfs diff=lfs merge=lfs -text
+*.ftz filter=lfs diff=lfs merge=lfs -text
+*.gz filter=lfs diff=lfs merge=lfs -text
+*.h5 filter=lfs diff=lfs merge=lfs -text
+*.joblib filter=lfs diff=lfs merge=lfs -text
+*.lfs.* filter=lfs diff=lfs merge=lfs -text
+*.mlmodel filter=lfs diff=lfs merge=lfs -text
+*.model filter=lfs diff=lfs merge=lfs -text
+*.msgpack filter=lfs diff=lfs merge=lfs -text
+*.npy filter=lfs diff=lfs merge=lfs -text
+*.npz filter=lfs diff=lfs merge=lfs -text
+*.onnx filter=lfs diff=lfs merge=lfs -text
+*.ot filter=lfs diff=lfs merge=lfs -text
+*.parquet filter=lfs diff=lfs merge=lfs -text
+*.pb filter=lfs diff=lfs merge=lfs -text
+*.pickle filter=lfs diff=lfs merge=lfs -text
+*.pkl filter=lfs diff=lfs merge=lfs -text
+*.pt filter=lfs diff=lfs merge=lfs -text
+*.pth filter=lfs diff=lfs merge=lfs -text
+*.rar filter=lfs diff=lfs merge=lfs -text
+*.safetensors filter=lfs diff=lfs merge=lfs -text
+saved_model/**/* filter=lfs diff=lfs merge=lfs -text
+*.tar.* filter=lfs diff=lfs merge=lfs -text
+*.tar filter=lfs diff=lfs merge=lfs -text
+*.tflite filter=lfs diff=lfs merge=lfs -text
+*.tgz filter=lfs diff=lfs merge=lfs -text
+*.wasm filter=lfs diff=lfs merge=lfs -text
+*.xz filter=lfs diff=lfs merge=lfs -text
+*.zip filter=lfs diff=lfs merge=lfs -text
+*.zst filter=lfs diff=lfs merge=lfs -text
+*tfevents* filter=lfs diff=lfs merge=lfs -text
+nnUNet/documentation/assets/amos2022_sparseseg10.png filter=lfs diff=lfs merge=lfs -text
+nnUNet/documentation/assets/amos2022_sparseseg10_2d.png filter=lfs diff=lfs merge=lfs -text
+nnUNet/documentation/assets/nnU-Net_overview.png filter=lfs diff=lfs merge=lfs -text
+nnUNet/documentation/assets/nnUNetMagician.png filter=lfs diff=lfs merge=lfs -text
+nnUNet/documentation/assets/regions_vs_labels.png filter=lfs diff=lfs merge=lfs -text
+nnUNet/documentation/assets/scribble_example.png filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,134 @@
+---
+license: cc-by-4.0
+---
+This repository provides the nnUNet v2 checkpoint for multi-organ lesion segmentation from abdominal CT images.
+
+The model was trained on a large-scale multi-center CT dataset with expert-annotated lesion masks across multiple organs (liver, pancreas, kidney, colon).
+
+First of all, you need a Linux OS server and basic terminal usage.
+
+## 0. Before you start
+You need: 
+CT files in .nii.gz format
+
+Each CT is a 3D volume (not single images)
+
+## 1. Installation (Environment and nnUNet packages)
+
+```bash
+git clone https://huggingface.co/ChrisXzZ/LesionSegmenter
+conda create -n lesionseg python=3.10 -y
+conda activate lesionseg
+git clone https://github.com/MIC-DKFZ/nnUNet.git
+cd nnUNet
+pip install -e .
+pip uninstall -y torch torchvision torchaudio
+pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu121
+```
+
+## 2. Prepare your CT data 
+Create the following folder structure:
+```bash
+mkdir -p nnUNet_eval/Dataset1356/imagesTs
+```
+
+Put your CT files into:
+```bash
+nnUNet_eval/Dataset1356/imagesTs/
+```
+
+Each file must be renamed like this:
+```bash
+nnUNet_eval/
+ └── Dataset1356/
+      └── imagesTs/
+           ├── ct001_0000.nii.gz
+           ├── ct002_0000.nii.gz
+```
+
+Key rules:
+Must end with _0000.nii.gz
+Each case = one file
+
+#### 3. Put the Model (Checkpoint)
+Download the model checkpoint (provided by us), then place it like this:
+```bash
+nnUNet_results/
+└── Dataset1356_RADV3S/nnUNetTrainer__nnUNetResEncUNetLPlans__3d_fullres/fold_all/checkpoint_final.pth
+```
+Or
+
+```bash
+nnUNet_results/
+└── Dataset1357_RADN1/nnUNetTrainer__nnUNetResEncUNetLPlans__3d_fullres/fold_all/checkpoint_final.pth
+```
+
+
+#### 4. Run inference
+First set paths:
+```bash
+export nnUNet_N_proc_DA=36
+export nnUNet_results="./nnUNet_results"
+export nnUNet_predictions="./nnUNet_predictions"
+export nnUNet_eval="./nnUNet_eval"
+```
+
+Then run:
+
+```bash
+GPU_ID=0 
+DATASET=1356
+
+TRAINER=nnUNetTrainer 
+
+CUDA_VISIBLE_DEVICES=$GPU_ID nnUNetv2_predict -d $DATASET -i $nnUNet_eval/Dataset1356/imagesTs/ -o $nnUNet_predictions/ -tr $TRAINER -d $DATASET -c 3d_fullres -f all -p nnUNetResEncUNetLPlans --continue_prediction 
+```
+Or 
+
+```bash
+export nnUNet_N_proc_DA=36
+export nnUNet_results="./nnUNet_results"
+export nnUNet_predictions="./nnUNet_predictions"
+export nnUNet_eval="./nnUNet_eval"
+
+GPU_ID=0 
+DATASET=1357
+
+TRAINER=nnUNetTrainer 
+
+CUDA_VISIBLE_DEVICES=$GPU_ID nnUNetv2_predict -d $DATASET -i $nnUNet_eval/Dataset1357/imagesTs// -o $nnUNet_predictions/ -tr $TRAINER -d $DATASET -c 3d_fullres -f all -p nnUNetResEncUNetLPlans --continue_prediction 
+```
+
+### 5. Run post-processing
+This step removes false positives and improves quality.
+
+For DATASET=1356 checkpoint, run:
+```bash
+python p-processing.py \
+    --input_dir ./nnUNet_predictions \
+    --output_dir ./nnUNet_predictions \
+    --rules "39:5,40:6|7|8,41:3|4,42:16" \
+    --keep_mode any \
+    --target_labels "39,40,41,42" \
+    --min_size 25 \
+    --dilation_iters 1 \
+    --connectivity 26 \
+    --num_workers 16
+```
+
+For DATASET=1357 checkpoint, run:
+```bash
+python p-processing.py \
+    --input_dir ./nnUNet_predictions \
+    --output_dir ./nnUNet_predictions \
+    --rules "116:5|6|7|8|9|10|11|12,117:13|14|15,118:3|4,119:23" \
+    --keep_mode any \
+    --target_labels "116,117,118,119" \
+    --min_size 25 \
+    --dilation_iters 1 \
+    --connectivity 26 \
+    --num_workers 16
+```
+
+
+

nnUNet/.github/workflows/codespell.yml

@@ -0,0 +1,22 @@
+---
+name: Codespell
+
+on:
+  push:
+    branches: [master]
+  pull_request:
+    branches: [master]
+
+permissions:
+  contents: read
+
+jobs:
+  codespell:
+    name: Check for spelling errors
+    runs-on: ubuntu-latest
+
+    steps:
+      - name: Checkout
+        uses: actions/checkout@v3
+      - name: Codespell
+        uses: codespell-project/actions-codespell@v2

nnUNet/.github/workflows/run_tests_nnunet.yml

@@ -0,0 +1,49 @@
+name: Run nnunet tests on all OS
+on:
+  push:
+    paths-ignore:
+      - '**.md'
+jobs:
+
+  run-tests:
+    strategy:
+      matrix:
+        # os: [ubuntu-latest, windows-latest, macos-latest]  # fails on windows until https://github.com/MIC-DKFZ/nnUNet/issues/2396 is resolved
+        os: [ubuntu-latest, macos-latest]
+        python-version: ["3.10"]
+    runs-on: ${{ matrix.os }}
+
+    steps:
+    - uses: actions/checkout@v4
+
+    - name: Set up Python ${{ matrix.python-version }}
+      uses: actions/setup-python@v5
+      with:
+        python-version: ${{ matrix.python-version }}
+
+    - name: Download and extract weights and download example file
+      # use python instead of curl to avoid the need to install curl (more difficult for macos)
+      run: |
+        mkdir -p $HOME/github_actions_nnunet/results
+        python -c "import urllib.request; urllib.request.urlretrieve('https://github.com/wasserth/TotalSegmentator/releases/download/v2.0.0-weights/Dataset300_body_6mm_1559subj.zip', '$HOME/github_actions_nnunet/results/tmp_download_file.zip')"
+        unzip -o $HOME/github_actions_nnunet/results/tmp_download_file.zip -d $HOME/github_actions_nnunet/results
+        rm $HOME/github_actions_nnunet/results/tmp_download_file.zip
+
+    - name: Install dependencies on Ubuntu
+      if: runner.os == 'Linux'
+      run: |
+        python -m pip install --upgrade pip
+        pip install pytest Cython
+        pip install torch==2.4.0 -f https://download.pytorch.org/whl/cpu
+        pip install .
+
+    - name: Install dependencies on Windows / MacOS
+      if: runner.os == 'Windows' || runner.os == 'macOS'
+      run: |
+        python -m pip install --upgrade pip
+        pip install pytest Cython
+        pip install torch==2.4.0
+        pip install .
+
+    - name: Run test script
+      run: python nnunetv2/tests/integration_tests/run_nnunet_inference.py

nnUNet/.gitignore

@@ -0,0 +1,119 @@
+# Byte-compiled / optimized / DLL files
+__pycache__/
+*.py[cod]
+*$py.class
+
+# C extensions
+*.so
+
+# Distribution / packaging
+.Python
+env/
+build/
+develop-eggs/
+dist/
+downloads/
+eggs/
+.eggs/
+lib/
+lib64/
+parts/
+sdist/
+var/
+*.egg-info/
+.installed.cfg
+*.egg
+
+# PyInstaller
+#  Usually these files are written by a python script from a template
+#  before PyInstaller builds the exe, so as to inject date/other infos into it.
+*.manifest
+*.spec
+
+# Installer logs
+pip-log.txt
+pip-delete-this-directory.txt
+
+# Unit test / coverage reports
+htmlcov/
+.tox/
+.coverage
+.coverage.*
+.cache
+nosetests.xml
+coverage.xml
+*,cover
+.hypothesis/
+
+# Translations
+*.mo
+*.pot
+
+# Django stuff:
+*.log
+local_settings.py
+
+# Flask stuff:
+instance/
+.webassets-cache
+
+# Scrapy stuff:
+.scrapy
+
+# Sphinx documentation
+docs/_build/
+
+# PyBuilder
+target/
+
+# IPython Notebook
+.ipynb_checkpoints
+
+# pyenv
+.python-version
+
+# celery beat schedule file
+celerybeat-schedule
+
+# dotenv
+.env
+
+# virtualenv
+venv/
+ENV/
+
+# Spyder project settings
+.spyderproject
+
+# Rope project settings
+.ropeproject
+
+*.memmap
+*.png
+*.zip
+*.npz
+*.npy
+*.jpg
+*.jpeg
+.idea
+*.txt
+.idea/*
+*.png
+# *.nii.gz  # nifti files needed for example_data for github actions tests
+*.nii
+*.tif
+*.bmp
+*.pkl
+*.xml
+*.pkl
+*.pdf
+*.png
+*.jpg
+*.jpeg
+
+*.model
+
+!documentation/assets/scribble_example.png
+
+CLAUDE.md
+AGENTS.md

nnUNet/CONTRIBUTING.md

@@ -0,0 +1,50 @@
+# Contributing to nnU-Net
+
+Thank you for your interest in contributing to nnU-Net.
+
+nnU-Net is developed and maintained by researchers at DKFZ. There is no dedicated funding or staff for maintaining the 
+repository, and development happens alongside research and teaching responsibilities. Our bandwidth for reviewing 
+external contributions is therefore limited, and review times may be long.
+
+## General principles
+
+nnU-Net is intentionally designed to be focused, stable, and generally applicable across datasets and use cases. 
+Contributions should respect this philosophy and should not introduce unnecessary complexity or specialization.
+
+New functionality must either be generally valid across datasets and setups or convincingly benefit a large enough 
+portion of the user base. We aim to avoid bloating the framework or increasing its complexity further.
+
+## How to contribute
+
+For larger features and refactors, please open a GitHub issue to discuss the idea before starting work. Tag
+@FabianIsensee so that the discussion doesn't get missed.
+
+To submit a contribution, fork the repository, make your changes on a branch, and open a pull request.
+
+## Bug reports and bug fixes
+
+Bug reports must include a minimal reproducible example. Without a repro, it is usually impossible for us to 
+investigate issues.
+
+Pull requests fixing bugs should also include a clear reproduction of the issue and an explanation of how the fix 
+resolves it.
+
+## Performance improvements
+
+If a pull request claims performance improvements, it must include benchmarks demonstrating the effect. The benchmark 
+setup must be described clearly enough for us to reproduce the results independently. We may run additional 
+tests ourselves before merging.
+
+## Contributions that are unlikely to be merged
+
+To keep the framework maintainable and the workload manageable on our end, we deprioritize:
+
+- dataset-specific code
+- features that only apply to niche setups
+- narrow custom architectures or training pipelines
+- large refactorings without prior discussion
+- small PRs fixing minor typos or formatting issues
+
+## Final note
+
+We appreciate the effort people invest in improving nnU-Net!

nnUNet/LICENSE

@@ -0,0 +1,201 @@
+                                 Apache License
+                           Version 2.0, January 2004
+                        http://www.apache.org/licenses/
+
+   TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
+
+   1. Definitions.
+
+      "License" shall mean the terms and conditions for use, reproduction,
+      and distribution as defined by Sections 1 through 9 of this document.
+
+      "Licensor" shall mean the copyright owner or entity authorized by
+      the copyright owner that is granting the License.
+
+      "Legal Entity" shall mean the union of the acting entity and all
+      other entities that control, are controlled by, or are under common
+      control with that entity. For the purposes of this definition,
+      "control" means (i) the power, direct or indirect, to cause the
+      direction or management of such entity, whether by contract or
+      otherwise, or (ii) ownership of fifty percent (50%) or more of the
+      outstanding shares, or (iii) beneficial ownership of such entity.
+
+      "You" (or "Your") shall mean an individual or Legal Entity
+      exercising permissions granted by this License.
+
+      "Source" form shall mean the preferred form for making modifications,
+      including but not limited to software source code, documentation
+      source, and configuration files.
+
+      "Object" form shall mean any form resulting from mechanical
+      transformation or translation of a Source form, including but
+      not limited to compiled object code, generated documentation,
+      and conversions to other media types.
+
+      "Work" shall mean the work of authorship, whether in Source or
+      Object form, made available under the License, as indicated by a
+      copyright notice that is included in or attached to the work
+      (an example is provided in the Appendix below).
+
+      "Derivative Works" shall mean any work, whether in Source or Object
+      form, that is based on (or derived from) the Work and for which the
+      editorial revisions, annotations, elaborations, or other modifications
+      represent, as a whole, an original work of authorship. For the purposes
+      of this License, Derivative Works shall not include works that remain
+      separable from, or merely link (or bind by name) to the interfaces of,
+      the Work and Derivative Works thereof.
+
+      "Contribution" shall mean any work of authorship, including
+      the original version of the Work and any modifications or additions
+      to that Work or Derivative Works thereof, that is intentionally
+      submitted to Licensor for inclusion in the Work by the copyright owner
+      or by an individual or Legal Entity authorized to submit on behalf of
+      the copyright owner. For the purposes of this definition, "submitted"
+      means any form of electronic, verbal, or written communication sent
+      to the Licensor or its representatives, including but not limited to
+      communication on electronic mailing lists, source code control systems,
+      and issue tracking systems that are managed by, or on behalf of, the
+      Licensor for the purpose of discussing and improving the Work, but
+      excluding communication that is conspicuously marked or otherwise
+      designated in writing by the copyright owner as "Not a Contribution."
+
+      "Contributor" shall mean Licensor and any individual or Legal Entity
+      on behalf of whom a Contribution has been received by Licensor and
+      subsequently incorporated within the Work.
+
+   2. Grant of Copyright License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      copyright license to reproduce, prepare Derivative Works of,
+      publicly display, publicly perform, sublicense, and distribute the
+      Work and such Derivative Works in Source or Object form.
+
+   3. Grant of Patent License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      (except as stated in this section) patent license to make, have made,
+      use, offer to sell, sell, import, and otherwise transfer the Work,
+      where such license applies only to those patent claims licensable
+      by such Contributor that are necessarily infringed by their
+      Contribution(s) alone or by combination of their Contribution(s)
+      with the Work to which such Contribution(s) was submitted. If You
+      institute patent litigation against any entity (including a
+      cross-claim or counterclaim in a lawsuit) alleging that the Work
+      or a Contribution incorporated within the Work constitutes direct
+      or contributory patent infringement, then any patent licenses
+      granted to You under this License for that Work shall terminate
+      as of the date such litigation is filed.
+
+   4. Redistribution. You may reproduce and distribute copies of the
+      Work or Derivative Works thereof in any medium, with or without
+      modifications, and in Source or Object form, provided that You
+      meet the following conditions:
+
+      (a) You must give any other recipients of the Work or
+          Derivative Works a copy of this License; and
+
+      (b) You must cause any modified files to carry prominent notices
+          stating that You changed the files; and
+
+      (c) You must retain, in the Source form of any Derivative Works
+          that You distribute, all copyright, patent, trademark, and
+          attribution notices from the Source form of the Work,
+          excluding those notices that do not pertain to any part of
+          the Derivative Works; and
+
+      (d) If the Work includes a "NOTICE" text file as part of its
+          distribution, then any Derivative Works that You distribute must
+          include a readable copy of the attribution notices contained
+          within such NOTICE file, excluding those notices that do not
+          pertain to any part of the Derivative Works, in at least one
+          of the following places: within a NOTICE text file distributed
+          as part of the Derivative Works; within the Source form or
+          documentation, if provided along with the Derivative Works; or,
+          within a display generated by the Derivative Works, if and
+          wherever such third-party notices normally appear. The contents
+          of the NOTICE file are for informational purposes only and
+          do not modify the License. You may add Your own attribution
+          notices within Derivative Works that You distribute, alongside
+          or as an addendum to the NOTICE text from the Work, provided
+          that such additional attribution notices cannot be construed
+          as modifying the License.
+
+      You may add Your own copyright statement to Your modifications and
+      may provide additional or different license terms and conditions
+      for use, reproduction, or distribution of Your modifications, or
+      for any such Derivative Works as a whole, provided Your use,
+      reproduction, and distribution of the Work otherwise complies with
+      the conditions stated in this License.
+
+   5. Submission of Contributions. Unless You explicitly state otherwise,
+      any Contribution intentionally submitted for inclusion in the Work
+      by You to the Licensor shall be under the terms and conditions of
+      this License, without any additional terms or conditions.
+      Notwithstanding the above, nothing herein shall supersede or modify
+      the terms of any separate license agreement you may have executed
+      with Licensor regarding such Contributions.
+
+   6. Trademarks. This License does not grant permission to use the trade
+      names, trademarks, service marks, or product names of the Licensor,
+      except as required for reasonable and customary use in describing the
+      origin of the Work and reproducing the content of the NOTICE file.
+
+   7. Disclaimer of Warranty. Unless required by applicable law or
+      agreed to in writing, Licensor provides the Work (and each
+      Contributor provides its Contributions) on an "AS IS" BASIS,
+      WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+      implied, including, without limitation, any warranties or conditions
+      of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
+      PARTICULAR PURPOSE. You are solely responsible for determining the
+      appropriateness of using or redistributing the Work and assume any
+      risks associated with Your exercise of permissions under this License.
+
+   8. Limitation of Liability. In no event and under no legal theory,
+      whether in tort (including negligence), contract, or otherwise,
+      unless required by applicable law (such as deliberate and grossly
+      negligent acts) or agreed to in writing, shall any Contributor be
+      liable to You for damages, including any direct, indirect, special,
+      incidental, or consequential damages of any character arising as a
+      result of this License or out of the use or inability to use the
+      Work (including but not limited to damages for loss of goodwill,
+      work stoppage, computer failure or malfunction, or any and all
+      other commercial damages or losses), even if such Contributor
+      has been advised of the possibility of such damages.
+
+   9. Accepting Warranty or Additional Liability. While redistributing
+      the Work or Derivative Works thereof, You may choose to offer,
+      and charge a fee for, acceptance of support, warranty, indemnity,
+      or other liability obligations and/or rights consistent with this
+      License. However, in accepting such obligations, You may act only
+      on Your own behalf and on Your sole responsibility, not on behalf
+      of any other Contributor, and only if You agree to indemnify,
+      defend, and hold each Contributor harmless for any liability
+      incurred by, or claims asserted against, such Contributor by reason
+      of your accepting any such warranty or additional liability.
+
+   END OF TERMS AND CONDITIONS
+
+   APPENDIX: How to apply the Apache License to your work.
+
+      To apply the Apache License to your work, attach the following
+      boilerplate notice, with the fields enclosed by brackets "[]"
+      replaced with your own identifying information. (Don't include
+      the brackets!)  The text should be enclosed in the appropriate
+      comment syntax for the file format. We also recommend that a
+      file or class name and description of purpose be included on the
+      same "printed page" as the copyright notice for easier
+      identification within third-party archives.
+
+   Copyright [2019] [Division of Medical Image Computing, German Cancer Research Center (DKFZ), Heidelberg, Germany]
+
+   Licensed under the Apache License, Version 2.0 (the "License");
+   you may not use this file except in compliance with the License.
+   You may obtain a copy of the License at
+
+       http://www.apache.org/licenses/LICENSE-2.0
+
+   Unless required by applicable law or agreed to in writing, software
+   distributed under the License is distributed on an "AS IS" BASIS,
+   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+   See the License for the specific language governing permissions and
+   limitations under the License.

nnUNet/documentation/benchmarking.md

@@ -0,0 +1,115 @@
+# nnU-Netv2 benchmarks
+
+Does your system run like it should? Is your epoch time longer than expected? What epoch times should you expect?
+
+Look no further for we have the solution here!
+
+## What does the nnU-netv2 benchmark do?
+
+nnU-Net's benchmark trains models for 5 epochs. At the end, the fastest epoch will 
+be noted down, along with the GPU name, torch version and cudnn version. You can find the benchmark output in the 
+corresponding nnUNet_results subfolder (see example below). Don't worry, we also provide scripts to collect your 
+results. Or you just start a benchmark and look at the console output. Everything is possible. Nothing is forbidden.
+
+The benchmark implementation revolves around two trainers:
+- `nnUNetTrainerBenchmark_5epochs` runs a regular training for 5 epochs. When completed, writes a .json file with the fastest 
+epoch time as well as the GPU used and the torch and cudnn versions. Useful for speed testing the entire pipeline 
+(data loading, augmentation, GPU training)
+- `nnUNetTrainerBenchmark_5epochs_noDataLoading` is the same, but it doesn't do any data loading or augmentation. It 
+just presents dummy arrays to the GPU. Useful for checking pure GPU speed.
+
+## How to run the nnU-Netv2 benchmark?
+It's quite simple, actually. It looks just like a regular nnU-Net training.
+
+We provide reference numbers for some of the Medical Segmentation Decathlon datasets because they are easily 
+accessible: [download here](https://drive.google.com/drive/folders/1HqEgzS8BV2c7xYNrZdEAnrHk7osJJ--2). If it needs to be 
+quick and dirty, focus on Tasks 2 and 4. Download and extract the data and convert them to the nnU-Net format with 
+`nnUNetv2_convert_MSD_dataset`. 
+Run `nnUNetv2_plan_and_preprocess` for them.
+
+Then, for each dataset, run the following commands (only one per GPU! Or one after the other):
+
+```bash
+nnUNetv2_train DATSET_ID 2d 0 -tr nnUNetTrainerBenchmark_5epochs
+nnUNetv2_train DATSET_ID 3d_fullres 0 -tr nnUNetTrainerBenchmark_5epochs
+nnUNetv2_train DATSET_ID 2d 0 -tr nnUNetTrainerBenchmark_5epochs_noDataLoading
+nnUNetv2_train DATSET_ID 3d_fullres 0 -tr nnUNetTrainerBenchmark_5epochs_noDataLoading
+```
+
+If you want to inspect the outcome manually, check (for example!) your 
+`nnUNet_results/DATASET_NAME/nnUNetTrainerBenchmark_5epochs__nnUNetPlans__3d_fullres/fold_0/` folder for the `benchmark_result.json` file.
+
+Note that there can be multiple entries in this file if the benchmark was run on different GPU types, torch versions or cudnn versions!
+
+If you want to summarize your results like we did in our [results](#results), check the 
+[summary script](../nnunetv2/batch_running/benchmarking/summarize_benchmark_results.py). Here you need to change the 
+torch version, cudnn version and dataset you want to summarize, then execute the script. You can find the exact 
+values you need to put there in one of your `benchmark_result.json` files.
+
+## Results
+We have tested a variety of GPUs and summarized the results in a 
+[spreadsheet](https://docs.google.com/spreadsheets/d/12Cvt_gr8XU2qWaE0XJk5jJlxMEESPxyqW0CWbQhTNNY/edit?usp=sharing). 
+Note that you can select the torch and cudnn versions at the bottom! There may be comments in this spreadsheet. Read them!
+
+## Result interpretation
+
+Results are shown as epoch time in seconds. Lower is better (duh). Epoch times can fluctuate between runs, so as 
+long as you are within like 5-10% of the numbers we report, everything should be dandy. 
+
+If not, here is how you can try to find the culprit!
+
+The first thing to do is to compare the performance between the `nnUNetTrainerBenchmark_5epochs_noDataLoading` and 
+`nnUNetTrainerBenchmark_5epochs` trainers. If the difference is about the same as we report in our spreadsheet, but 
+both your numbers are worse, the problem is with your GPU:
+
+- Are you certain you compare the correct GPU? (duh)
+- If yes, then you might want to install PyTorch in a different way. Never `pip install torch`! Go to the
+[PyTorch installation](https://pytorch.org/get-started/locally/) page, select the most recent cuda version your 
+system supports and only then copy and execute the correct command! Either pip or conda should work
+- If the problem is still not fixed, we recommend you try 
+[compiling pytorch from source](https://github.com/pytorch/pytorch#from-source). It's more difficult but that's 
+how we roll here at the DKFZ (at least the cool kids here).
+- Another thing to consider is to try exactly the same torch + cudnn version as we did in our spreadsheet. 
+Sometimes newer versions can actually degrade performance and there might be bugs from time to time. Older versions 
+are also often a lot slower!
+- Finally, some very basic things that could impact your GPU performance: 
+  - Is the GPU cooled adequately? Check the temperature with `nvidia-smi`. Hot GPUs throttle performance in order to not self-destruct
+  - Is your OS using the GPU for displaying your desktop at the same time? If so then you can expect a performance 
+  penalty (I dunno like 10% !?). That's expected and OK.
+  - Are other users using the GPU as well?
+
+
+If you see a large performance difference between `nnUNetTrainerBenchmark_5epochs_noDataLoading` (fast) and 
+`nnUNetTrainerBenchmark_5epochs` (slow) then the problem might be related to data loading and augmentation. As a 
+reminder, nnU-net does not use pre-augmented images (offline augmentation) but instead generates augmented training 
+samples on the fly during training (no, you cannot switch it to offline). This requires that your system can do partial 
+reads of the image files fast enough (SSD storage required!) and that your CPU is powerful enough to run the augmentations.
+
+Check the following:
+
+- [CPU bottleneck] How many CPU threads are running during the training? nnU-Net uses 12 processes for data augmentation by default. 
+If you see those 12 running constantly during training, consider increasing the number of processes used for data 
+augmentation (provided there is headroom on your CPU!). Increase the number until you see less active workers than 
+you configured (or just set the number to 32 and forget about it). You can do so by setting the `nnUNet_n_proc_DA` 
+environment variable (Linux: `export nnUNet_n_proc_DA=24`). Read [here](set_environment_variables.md) on how to do this.
+If your CPU does not support more processes (setting more processes than your CPU has threads makes 
+no sense!) you are out of luck and in desperate need of a system upgrade!
+- [I/O bottleneck] If you don't see 12 (or nnUNet_n_proc_DA if you set it) processes running but your training times 
+are still slow then open up `top` (sorry, Windows users. I don't know how to do this on Windows) and look at the value 
+left of 'wa' in the row that begins 
+with '%Cpu (s)'. If this is >1.0 (arbitrarily set threshold here, essentially look for unusually high 'wa'. In a 
+healthy training 'wa' will be almost 0) then your storage cannot keep up with data loading. Make sure to set 
+nnUNet_preprocessed to a folder that is located on an SSD. nvme is preferred over SATA. PCIe3 is enough. 3000MB/s 
+sequential read recommended.
+- [funky stuff] Sometimes there is funky stuff going on, especially when batch sizes are large, files are small and 
+patch sizes are small as well. As part of the data loading process, nnU-Net needs to open and close a file for each 
+training sample. Now imagine a dataset like Dataset004_Hippocampus where for the 2d config we have a batch size of 
+366 and we run 250 iterations in <10s on an A100. That's a lotta files per second (366 * 250 / 10 = 9150 files per second). 
+Oof. If the files are on some network drive (even if it's nvme) then (probably) good night. The good news: nnU-Net
+has got you covered: add `export nnUNet_keep_files_open=True` to your .bashrc and the problem goes away. The neat 
+part: it causes new problems if you are not allowed to have enough open files. You may have to increase the number 
+of allowed open files. `ulimit -n` gives your current limit (Linux only). It should not be something like 1024. 
+Increasing that to 65535 works well for me. See here for how to change these limits: 
+[Link](https://kupczynski.info/posts/ubuntu-18-10-ulimits/) 
+(works for Ubuntu 18, google for your OS!).
+

nnUNet/documentation/changelog.md

@@ -0,0 +1,51 @@
+# What is different in v2?
+
+- We now support **hierarchical labels** (named regions in nnU-Net). For example, instead of training BraTS with the 
+'edema', 'necrosis' and 'enhancing tumor' labels you can directly train it on the target areas 'whole tumor', 
+'tumor core' and 'enhancing tumor'. See [here](region_based_training.md) for a detailed description + also have a look at the 
+[BraTS 2021 conversion script](../nnunetv2/dataset_conversion/Dataset137_BraTS21.py).
+- Cross-platform support. Cuda, mps (Apple M1/M2) and of course CPU support! Simply select the device with 
+`-device` in `nnUNetv2_train` and `nnUNetv2_predict`.
+- Unified trainer class: nnUNetTrainer. No messing around with cascaded trainer, DDP trainer, region-based trainer, 
+ignore trainer etc. All default functionality is in there!
+- Supports more input/output data formats through ImageIO classes.
+- I/O formats can be extended by implementing new Adapters based on `BaseReaderWriter`.
+- The nnUNet_raw_cropped folder no longer exists -> saves disk space at no performance penalty. magic! (no jk the 
+saving of cropped npz files was really slow, so it's actually faster to crop on the fly).
+- Preprocessed data and segmentation are stored in different files when unpacked. Seg is stored as int8 and thus 
+takes 1/4 of the disk space per pixel (and I/O throughput) as in v1.
+- Native support for multi-GPU (DDP) TRAINING. 
+Multi-GPU INFERENCE should still be run with `CUDA_VISIBLE_DEVICES=X nnUNetv2_predict [...] -num_parts Y -part_id X`. 
+There is no cross-GPU communication in inference, so it doesn't make sense to add additional complexity with DDP.
+- All nnU-Net functionality is now also accessible via API. Check the corresponding entry point in `setup.py` to see 
+what functions you need to call.
+- Dataset fingerprint is now explicitly created and saved in a json file (see nnUNet_preprocessed).
+
+- Complete overhaul of plans files (read also [this](explanation_plans_files.md):
+  - Plans are now .json and can be opened and read more easily
+  - Configurations are explicitly named ("3d_fullres" , ...)
+  - Configurations can inherit from each other to make manual experimentation easier
+  - A ton of additional functionality is now included in and can be changed through the plans, for example normalization strategy, resampling etc.
+  - Stages of the cascade are now explicitly listed in the plans. 3d_lowres has 'next_stage' (which can also be a 
+  list of configurations!). 3d_cascade_fullres has a 'previous_stage' entry. By manually editing plans files you can 
+  now connect anything you want, for example 2d with 3d_fullres or whatever. Be wild! (But don't create cycles!)
+  - Multiple configurations can point to the same preprocessed data folder to save disk space. Careful! Only 
+  configurations that use the same spacing, resampling, normalization etc. should share a data source! By default, 
+  3d_fullres and 3d_cascade_fullres share the same data
+  - Any number of configurations can be added to the plans (remember to give them a unique "data_identifier"!)
+
+Folder structures are different and more user-friendly:
+- nnUNet_preprocessed
+  - By default, preprocessed data is now saved as: `nnUNet_preprocessed/DATASET_NAME/PLANS_IDENTIFIER_CONFIGURATION` to clearly link them to their corresponding plans and configuration 
+  - Name of the folder containing the preprocessed images can be adapted with the `data_identifier` key.
+- nnUNet_results
+  - Results are now sorted as follows: DATASET_NAME/TRAINERCLASS__PLANSIDENTIFIER__CONFIGURATION/FOLD
+
+## What other changes are planned and not yet implemented?
+- Integration into MONAI (together with our friends at Nvidia)
+- New pretrained weights for a large number of datasets (coming very soon))
+
+
+[//]: # (- nnU-Net now also natively supports an **ignore label**. Pixels with this label will not contribute to the loss. )
+
+[//]: # (Use this to learn from sparsely annotated data, or excluding irrelevant areas from training. Read more [here](ignore_label.md).)

nnUNet/documentation/competitions/AortaSeg24.md

@@ -0,0 +1,46 @@
+Authors: \
+Maximilian Rokuss, Michael Baumgartner, Yannick Kirchhoff, Klaus H. Maier-Hein*, Fabian Isensee*
+
+*: equal contribution
+
+Author Affiliations:\
+Division of Medical Image Computing, German Cancer Research Center (DKFZ), Heidelberg \
+Helmholtz Imaging
+
+# Introduction
+
+This document describes our submission to the [AortaSeg24 Challenge](https://aortaseg24.grand-challenge.org/). 
+Our model is essentially a nnU-Net ResEnc L with modified data augmentation. We disable left/right mirroring and use the heavy data augmentation [DA5 Trainer](../../nnunetv2/training/nnUNetTrainer/variants/data_augmentation/nnUNetTrainerDA5.py). Training was performed on an A100 40GB GPU.
+
+# Experiment Planning and Preprocessing
+After converting the data into the [nnUNet format](../../../nnUNet/documentation/dataset_format.md) (either keep and just rename the .mha files or convert them to .nii.gz), you can run the preprocessing:
+
+```bash
+nnUNetv2_plan_and_preprocess -d 610 -c 3d_fullres -pl nnUNetPlannerResEncL -np 16
+```
+
+# Training
+We train our model using:
+
+```bash
+nnUNetv2_train 610 3d_fullres all -p nnUNetResEncUNetLPlans -tr nnUNetTrainer_onlyMirror01_DA5
+```
+Models are trained from scratch. We train one model using all the images and a five fold cross validation ensemble for the submission.
+
+We recommend to increase the number of processes used for data augmentation. Otherwise you can run into CPU bottlenecks.
+Use `export nnUNet_n_proc_DA=32` or higher (if your system permits!).
+
+# Inference
+For inference you can use the default [nnUNet inference functionalities](../../../nnUNet/documentation/how_to_use_nnunet.md). Specifically, once the training is finished, run:
+
+```bash
+nnUNetv2_predict_from_modelfolder -i INPUT_FOLDER -o OUTPUT_FOLDER -m MODEL_FOLDER -f all
+```
+
+for the single model trained on all the data and 
+
+```bash
+nnUNetv2_predict_from_modelfolder -i INPUT_FOLDER -o OUTPUT_FOLDER -m MODEL_FOLDER
+```
+
+for the five fold ensemble.

nnUNet/documentation/competitions/AutoPETII.md

@@ -0,0 +1,129 @@
+# Look Ma, no code: fine tuning nnU-Net for the AutoPET II challenge by only adjusting its JSON plans
+
+Please cite our paper :-*
+
+```text
+COMING SOON
+```
+
+## Intro
+
+See the [Challenge Website](https://autopet-ii.grand-challenge.org/) for details on the challenge.
+
+Our solution to this challenge rewuires no code changes at all. All we do is optimize nnU-Net's hyperparameters 
+(architecture, batch size, patch size) through modifying the nnUNetplans.json file.
+
+## Prerequisites
+Use the latest pytorch version!
+
+We recommend you use the latest nnU-Net version as well! We ran our trainings with commit 913705f which you can try in case something doesn't work as expected:
+`pip install git+https://github.com/MIC-DKFZ/nnUNet.git@913705f`
+
+## How to reproduce our trainings
+
+### Download and convert the data
+1. Download and extract the AutoPET II dataset
+2. Convert it to nnU-Net format by running `python nnunetv2/dataset_conversion/Dataset221_AutoPETII_2023.py FOLDER` where folder is the extracted AutoPET II dataset.
+
+### Experiment planning and preprocessing
+We deviate a little from the standard nnU-Net procedure because all our experiments are based on just the 3d_fullres configuration
+
+Run the following commands:
+   - `nnUNetv2_extract_fingerprint -d 221` extracts the dataset fingerprint 
+   - `nnUNetv2_plan_experiment -d 221` does the planning for the plain unet
+   - `nnUNetv2_plan_experiment -d 221 -pl ResEncUNetPlanner` does the planning for the residual encoder unet
+   - `nnUNetv2_preprocess -d 221 -c 3d_fullres` runs all the preprocessing we need
+
+### Modification of plans files
+Please read the [information on how to modify plans files](../explanation_plans_files.md) first!!!
+
+
+It is easier to have everything in one plans file, so the first thing we do is transfer the ResEnc UNet to the 
+default plans file. We use the configuration inheritance feature of nnU-Net to make it use the same data as the 
+3d_fullres configuration.
+Add the following to the 'configurations' dict in 'nnUNetPlans.json':
+
+```json
+        "3d_fullres_resenc": {
+            "inherits_from": "3d_fullres",
+            "network_arch_class_name": "ResidualEncoderUNet",
+            "n_conv_per_stage_encoder": [
+                1,
+                3,
+                4,
+                6,
+                6,
+                6
+            ],
+            "n_conv_per_stage_decoder": [
+                1,
+                1,
+                1,
+                1,
+                1
+            ]
+        },
+```
+
+(these values are basically just copied from the 'nnUNetResEncUNetPlans.json' file! With everything redundant being omitted thanks to inheritance from 3d_fullres)
+
+Now we crank up the patch and batch sizes. Add the following configurations:
+```json
+        "3d_fullres_resenc_bs80": {
+            "inherits_from": "3d_fullres_resenc",
+            "batch_size": 80
+            },
+        "3d_fullres_resenc_192x192x192_b24": {
+            "inherits_from": "3d_fullres_resenc",
+            "patch_size": [
+                192,
+                192,
+                192
+            ],
+            "batch_size": 24
+        }
+```
+
+Save the file (and check for potential Syntax Errors!)
+
+### Run trainings
+Training each model requires 8 Nvidia A100 40GB GPUs. Expect training to run for 5-7 days. You'll need a really good 
+CPU to handle the data augmentation! 128C/256T are a must! If you have less threads available, scale down nnUNet_n_proc_DA accordingly.
+
+```bash
+nnUNet_compile=T nnUNet_n_proc_DA=28 nnUNetv2_train 221 3d_fullres_resenc_bs80 0 -num_gpus 8
+nnUNet_compile=T nnUNet_n_proc_DA=28 nnUNetv2_train 221 3d_fullres_resenc_bs80 1 -num_gpus 8
+nnUNet_compile=T nnUNet_n_proc_DA=28 nnUNetv2_train 221 3d_fullres_resenc_bs80 2 -num_gpus 8
+nnUNet_compile=T nnUNet_n_proc_DA=28 nnUNetv2_train 221 3d_fullres_resenc_bs80 3 -num_gpus 8
+nnUNet_compile=T nnUNet_n_proc_DA=28 nnUNetv2_train 221 3d_fullres_resenc_bs80 4 -num_gpus 8
+
+nnUNet_compile=T nnUNet_n_proc_DA=28 nnUNetv2_train 221 3d_fullres_resenc_192x192x192_b24 0 -num_gpus 8
+nnUNet_compile=T nnUNet_n_proc_DA=28 nnUNetv2_train 221 3d_fullres_resenc_192x192x192_b24 1 -num_gpus 8
+nnUNet_compile=T nnUNet_n_proc_DA=28 nnUNetv2_train 221 3d_fullres_resenc_192x192x192_b24 2 -num_gpus 8
+nnUNet_compile=T nnUNet_n_proc_DA=28 nnUNetv2_train 221 3d_fullres_resenc_192x192x192_b24 3 -num_gpus 8
+nnUNet_compile=T nnUNet_n_proc_DA=28 nnUNetv2_train 221 3d_fullres_resenc_192x192x192_b24 4 -num_gpus 8
+```
+
+Done!
+
+(We also provide pretrained weights in case you don't want to invest the GPU resources, see below)
+
+## How to make predictions with pretrained weights
+Our final model is an ensemble of two configurations:
+- ResEnc UNet with batch size 80
+- ResEnc UNet with patch size 192x192x192 and batch size 24
+
+To run inference with these models, do the following:
+
+1. Download the pretrained model weights from [Zenodo](https://zenodo.org/record/8362371)
+2. Install both .zip files using `nnUNetv2_install_pretrained_model_from_zip`
+3. Make sure 
+4. Now you can run inference on new cases with `nnUNetv2_predict`:
+   - `nnUNetv2_predict -i INPUT -o OUTPUT1 -d 221 -c 3d_fullres_resenc_bs80 -f 0 1 2 3 4 -step_size 0.6 --save_probabilities`   
+   - `nnUNetv2_predict -i INPUT -o OUTPUT2 -d 221 -c 3d_fullres_resenc_192x192x192_b24 -f 0 1 2 3 4 --save_probabilities`
+   - `nnUNetv2_ensemble -i OUTPUT1 OUTPUT2 -o OUTPUT_ENSEMBLE`
+
+Note that our inference Docker omitted TTA via mirroring along the axial direction during prediction (only sagittal + 
+coronal mirroring). This was
+done to keep the inference time below 10 minutes per image on a T4 GPU (we actually never tested whether we could 
+have left this enabled). Just leave it on! You can also leave the step_size at default for the 3d_fullres_resenc_bs80.

nnUNet/documentation/competitions/FLARE24/Task_1/inference_flare_task1.py

@@ -0,0 +1,191 @@
+from typing import Union, Tuple
+import argparse
+import numpy as np
+import os
+from os.path import join
+from pathlib import Path
+from time import time
+import torch
+from torch._dynamo import OptimizedModule
+
+from nnunetv2.utilities.label_handling.label_handling import LabelManager
+
+from acvl_utils.cropping_and_padding.bounding_boxes import bounding_box_to_slice
+from batchgenerators.utilities.file_and_folder_operations import load_json
+
+import nnunetv2
+from nnunetv2.configuration import default_num_processes
+from nnunetv2.utilities.find_class_by_name import recursive_find_python_class
+from nnunetv2.utilities.label_handling.label_handling import determine_num_input_channels
+from nnunetv2.utilities.plans_handling.plans_handler import PlansManager, ConfigurationManager
+from nnunetv2.inference.predict_from_raw_data import nnUNetPredictor
+from nnunetv2.imageio.nibabel_reader_writer import NibabelIOWithReorient
+
+
+class FlarePredictor(nnUNetPredictor):
+    def initialize_from_trained_model_folder(self, model_training_output_dir: str,
+                                             use_folds: Union[Tuple[Union[int, str]], None],
+                                             checkpoint_name: str = 'checkpoint_final.pth'):
+        """
+        This is used when making predictions with a trained model
+        """
+        if use_folds is None:
+            use_folds = nnUNetPredictor.auto_detect_available_folds(model_training_output_dir, checkpoint_name)
+
+        dataset_json = load_json(join(model_training_output_dir, 'dataset.json'))
+        plans = load_json(join(model_training_output_dir, 'plans.json'))
+        plans_manager = PlansManager(plans)
+
+        if isinstance(use_folds, str):
+            use_folds = [use_folds]
+
+        parameters = []
+        for i, f in enumerate(use_folds):
+            f = int(f) if f != 'all' else f
+            checkpoint = torch.load(join(model_training_output_dir, f'fold_{f}', checkpoint_name),
+                                    map_location=torch.device('cpu'))
+            if i == 0:
+                trainer_name = checkpoint['trainer_name']
+                configuration_name = checkpoint['init_args']['configuration']
+                inference_allowed_mirroring_axes = checkpoint['inference_allowed_mirroring_axes'] if \
+                    'inference_allowed_mirroring_axes' in checkpoint.keys() else None
+
+            parameters.append(checkpoint['network_weights'])
+
+        configuration_manager = plans_manager.get_configuration(configuration_name)
+        # restore network
+        num_input_channels = determine_num_input_channels(plans_manager, configuration_manager, dataset_json)
+        trainer_class = recursive_find_python_class(join(nnunetv2.__path__[0], "training", "nnUNetTrainer"),
+                                                    trainer_name, 'nnunetv2.training.nnUNetTrainer')
+        if trainer_class is None:
+            raise RuntimeError(f'Unable to locate trainer class {trainer_name} in nnunetv2.training.nnUNetTrainer. '
+                               f'Please place it there (in any .py file)!')
+        network = trainer_class.build_network_architecture(
+            configuration_manager.network_arch_class_name,
+            configuration_manager.network_arch_init_kwargs,
+            configuration_manager.network_arch_init_kwargs_req_import,
+            num_input_channels,
+            plans_manager.get_label_manager(dataset_json).num_segmentation_heads,
+            enable_deep_supervision=False
+        )
+
+        self.plans_manager = plans_manager
+        self.configuration_manager = configuration_manager
+        self.list_of_parameters = parameters
+        self.network = network
+        self.dataset_json = dataset_json
+        self.trainer_name = trainer_name
+        self.allowed_mirroring_axes = inference_allowed_mirroring_axes
+        self.label_manager = plans_manager.get_label_manager(dataset_json)
+
+        if ('nnUNet_compile' in os.environ.keys()) and (os.environ['nnUNet_compile'].lower() in ('true', '1', 't')) \
+                and not isinstance(self.network, OptimizedModule):
+            self.network = torch.compile(self.network)
+
+
+def convert_predicted_logits_to_segmentation_with_correct_shape(predicted_logits: Union[torch.Tensor, np.ndarray],
+                                                                plans_manager: PlansManager,
+                                                                configuration_manager: ConfigurationManager,
+                                                                label_manager: LabelManager,
+                                                                properties_dict: dict,
+                                                                return_probabilities: bool = False,
+                                                                num_threads_torch: int = default_num_processes):
+    old_threads = torch.get_num_threads()
+    torch.set_num_threads(num_threads_torch)
+
+    # resample to original shape
+    current_spacing = configuration_manager.spacing if \
+        len(configuration_manager.spacing) == \
+        len(properties_dict['shape_after_cropping_and_before_resampling']) else \
+        [properties_dict['spacing'][0], *configuration_manager.spacing]
+    predicted_logits = configuration_manager.resampling_fn_probabilities(predicted_logits,
+                                            properties_dict['shape_after_cropping_and_before_resampling'],
+                                            current_spacing,
+                                            properties_dict['spacing'])
+    # return value of resampling_fn_probabilities can be ndarray or Tensor but that does not matter because
+    # apply_inference_nonlin will convert to torch
+    # predicted_probabilities = label_manager.apply_inference_nonlin(predicted_logits)
+    segmentation = predicted_logits.argmax(0)
+    del predicted_logits
+    # segmentation = label_manager.convert_probabilities_to_segmentation(predicted_probabilities)
+
+    # segmentation may be torch.Tensor but we continue with numpy
+    if isinstance(segmentation, torch.Tensor):
+        segmentation = segmentation.cpu().numpy()
+
+    # put segmentation in bbox (revert cropping)
+    segmentation_reverted_cropping = np.zeros(properties_dict['shape_before_cropping'],
+                                              dtype=np.uint8 if len(label_manager.foreground_labels) < 255 else np.uint16)
+    slicer = bounding_box_to_slice(properties_dict['bbox_used_for_cropping'])
+    segmentation_reverted_cropping[slicer] = segmentation
+    del segmentation
+
+    # revert transpose
+    segmentation_reverted_cropping = segmentation_reverted_cropping.transpose(plans_manager.transpose_backward)
+    torch.set_num_threads(old_threads)
+    return segmentation_reverted_cropping
+
+
+def export_prediction_from_logits(predicted_array_or_file: Union[np.ndarray, torch.Tensor], properties_dict: dict,
+                                  configuration_manager: ConfigurationManager,
+                                  plans_manager: PlansManager,
+                                  dataset_json_dict_or_file: Union[dict, str], output_file_truncated: str,
+                                  save_probabilities: bool = False):
+
+    if isinstance(dataset_json_dict_or_file, str):
+        dataset_json_dict_or_file = load_json(dataset_json_dict_or_file)
+
+    label_manager = plans_manager.get_label_manager(dataset_json_dict_or_file)
+    ret = convert_predicted_logits_to_segmentation_with_correct_shape(
+        predicted_array_or_file, plans_manager, configuration_manager, label_manager, properties_dict,
+        return_probabilities=save_probabilities
+    )
+    del predicted_array_or_file
+
+    segmentation_final = ret
+
+    rw = NibabelIOWithReorient()
+    rw.write_seg(segmentation_final, output_file_truncated + dataset_json_dict_or_file['file_ending'],
+                 properties_dict)
+
+
+def predict_flare(input_dir, output_dir, model_folder, folds=("all",)):
+    input_dir = Path(input_dir)
+    output_dir = Path(output_dir)
+    input_files = sorted(input_dir.glob("*.nii.gz"))
+    output_files = [str(output_dir / f.name[:-12]) for f in input_files]
+    for input_file, output_file in zip(input_files, output_files):
+        print(f"Predicting {input_file.name}")
+        start = time()
+        plans_manager = PlansManager(load_json(join(model_folder, 'plans.json')))
+        configuration_manager = plans_manager.get_configuration("3d_fullres")
+        dataset_json = load_json(join(model_folder, 'dataset.json'))
+        rw = NibabelIOWithReorient()
+        image, props = rw.read_images([input_file,])
+        with torch.no_grad():
+            predictor = FlarePredictor(tile_step_size=0.5, use_mirroring=False)
+            predictor.initialize_from_trained_model_folder(model_folder, use_folds=folds)
+            preprocessor = configuration_manager.preprocessor_class(verbose=False)
+            data, _ = preprocessor.run_case_npy(image,
+                                                None,
+                                                props,
+                                                plans_manager,
+                                                configuration_manager,
+                                                dataset_json)
+            data = torch.from_numpy(data).to(dtype=torch.float32, memory_format=torch.contiguous_format)
+            predicted_logits = predictor.predict_logits_from_preprocessed_data(data).cpu()
+            export_prediction_from_logits(predicted_logits, props, configuration_manager,
+                                            plans_manager, dataset_json, output_file,
+                                            False)
+        print(f"Prediction time: {time() - start:.2f}s")
+
+
+if __name__ == '__main__':
+    os.environ['nnUNet_compile'] = 'f'
+    parser = argparse.ArgumentParser()
+    parser.add_argument("-i", "--input", default="/workspace/inputs")
+    parser.add_argument("-o", "--output", default="/workspace/outputs")
+    parser.add_argument("-m", "--model", default="/opt/app/_trained_model")
+    parser.add_argument("-f", "--folds", nargs="+", default=["all"])
+    args = parser.parse_args()
+    predict_flare(args.input, args.output, args.model, args.folds)

nnUNet/documentation/competitions/FLARE24/Task_1/readme.md

@@ -0,0 +1,78 @@
+Authors: \
+Yannick Kirchhoff, Maximilian Rouven Rokuss, Benjamin Hamm, Ashis Ravindran, Constantin Ulrich, Klaus Maier-Hein<sup>&#8224;</sup>, Fabian Isensee<sup>&#8224;</sup>
+
+&#8224;: equal contribution
+
+# Introduction
+
+This document describes our contribution to [Task 1 of the FLARE24 Challenge](https://www.codabench.org/competitions/2319/).
+Our model is basically is a default nnU-Net trained with larger batch size of 4 and 8, respectively. We submitted the batch size 8 model and an ensemble of the batch size 4 and batch size 8 models to the final test set.
+
+# Experiment Planning and Preprocessing
+
+Bring the downloaded data into the [nnU-Net format](../../../nnUNet/documentation/dataset_format.md) and add the dataset.json file as given here:
+
+```json
+{
+    "name": "Dataset301_FLARE24Task1_labeled",
+    "description": "Pan Cancer Segmentation",
+    "labels": {
+        "background": 0,
+        "lesion": 1
+    },
+    "file_ending": ".nii.gz",
+    "channel_names": {
+        "0": "CT"
+    },
+    "numTraining": 5000
+}
+```
+
+Afterwards you can run the default nnU-Net planning and preprocessing
+
+```bash
+nnUNetv2_plan_and_preprocess -d 301 -c 3d_fullres
+```
+
+## Edit the plans files
+
+In the generated `nnUNetPlans.json` file add the following configurations
+
+```json
+        "3d_fullres_bs4": {
+            "inherits_from": "3d_fullres",
+            "batch_size": 4
+        },
+        "3d_fullres_bs8": {
+            "inherits_from": "3d_fullres",
+            "batch_size": 8
+        },
+        "3d_fullres_bs4u8": {
+            "inherits_from": "3d_fullres",
+            "batch_size": 48
+        }
+```
+
+Note, the last one is only used for the ensemble model during inference!
+
+# Model training
+
+Run the following commands to train the models with batch size 4 and 8. The large batch size helps stabilize the training despite the partial labels present in the dataset as well as handling the large number of scans in the dataset. We therefore keep the number of epochs at 1000.
+
+```bash
+nnUNetv2_train 301 3d_fullres_bs4 all
+
+nnUNetv2_train 301 3d_fullres_bs8 all
+```
+
+# Inference
+
+Our inference is optimized for efficient single scan prediction. For best performance, we strongly recommend running inference using the default `nnUNetv2_predict` command!
+
+In order to run inference with the ensemble model you need to create a folder called `nnUNetTrainer__nnUNetPlans__3d_fullres_bs4u8` in the results folder and copy the `dataset.json`, `dataset_fingerprint.json` and `plans.json` from one of the other results folder as well as the `fold_all` from both trainings as `fold_0` and `fold_1`, respectively, into this new folder. This allows for easy ensembling of both models.
+
+To run inference simply run the following commands with `folds` set to `all` for single model inference or `0 1` for the ensemble. `model_folder` is the folder containing the training results, i.e. for example `nnUNetTrainer__nnUNetPlans__3d_fullres_bs8`.
+
+```bash
+python inference_flare_task1.py -i input_folder -o output_folder -m model_folder -f folds
+```

nnUNet/documentation/competitions/FLARE24/Task_2/inference_flare_task2.py

@@ -0,0 +1,449 @@
+from typing import Union, List, Tuple
+import argparse
+import itertools
+import multiprocessing
+import numpy as np
+import os
+from os.path import join
+from pathlib import Path
+from time import time
+import torch
+from torch._dynamo import OptimizedModule
+from tqdm import tqdm
+
+import openvino as ov
+import openvino.properties.hint as hints
+
+from acvl_utils.cropping_and_padding.bounding_boxes import bounding_box_to_slice
+from acvl_utils.cropping_and_padding.padding import pad_nd_image
+from batchgenerators.utilities.file_and_folder_operations import load_json
+
+from nnunetv2.utilities.label_handling.label_handling import LabelManager
+import nnunetv2
+from nnunetv2.configuration import default_num_processes
+from nnunetv2.inference.data_iterators import PreprocessAdapterFromNpy
+from nnunetv2.inference.sliding_window_prediction import compute_gaussian
+from nnunetv2.utilities.find_class_by_name import recursive_find_python_class
+from nnunetv2.utilities.helpers import empty_cache, dummy_context
+from nnunetv2.utilities.label_handling.label_handling import determine_num_input_channels
+from nnunetv2.utilities.plans_handling.plans_handler import PlansManager, ConfigurationManager
+from nnunetv2.inference.predict_from_raw_data import nnUNetPredictor
+
+
+torch.set_num_threads(multiprocessing.cpu_count())
+
+
+class FlarePredictor(nnUNetPredictor):
+    def __init__(self,
+                 tile_step_size: float = 0.5,
+                 use_gaussian: bool = True,
+                 use_mirroring: bool = True,
+                 perform_everything_on_device: bool = False,
+                 device: torch.device = torch.device('cpu'),
+                 verbose: bool = False,
+                 verbose_preprocessing: bool = False,
+                 allow_tqdm: bool = True,
+                 ):
+        super().__init__(tile_step_size, use_gaussian, use_mirroring, perform_everything_on_device, device, verbose,
+                         verbose_preprocessing, allow_tqdm)
+        if self.device == torch.device('cuda') or self.device == 'cuda':
+            raise RuntimeError('CUDA is not supported for this task')
+
+    def initialize_from_trained_model_folder(self, model_training_output_dir: str,
+                                             use_folds: Union[Tuple[Union[int, str]], None],
+                                             checkpoint_name: str = 'checkpoint_final.pth',
+                                             save_model: bool = True):
+        """
+        This is used when making predictions with a trained model
+        """
+        if use_folds is None:
+            use_folds = nnUNetPredictor.auto_detect_available_folds(model_training_output_dir, checkpoint_name)
+
+        dataset_json = load_json(join(model_training_output_dir, 'dataset.json'))
+        plans = load_json(join(model_training_output_dir, 'plans.json'))
+        plans_manager = PlansManager(plans)
+
+        if isinstance(use_folds, str):
+            use_folds = [use_folds]
+        assert len(use_folds) == 1, 'Only one fold is supported for this task'
+
+        parameters = []
+        for i, f in enumerate(use_folds):
+            f = int(f) if f != 'all' else f
+            checkpoint = torch.load(join(model_training_output_dir, f'fold_{f}', checkpoint_name),
+                                    map_location=torch.device('cpu'))
+            if i == 0:
+                trainer_name = checkpoint['trainer_name']
+                configuration_name = checkpoint['init_args']['configuration']
+                inference_allowed_mirroring_axes = checkpoint['inference_allowed_mirroring_axes'] if \
+                    'inference_allowed_mirroring_axes' in checkpoint.keys() else None
+        
+            if save_model:
+                parameters.append(checkpoint['network_weights'])
+
+        configuration_manager = plans_manager.get_configuration(configuration_name)
+        if save_model:
+            num_input_channels = determine_num_input_channels(plans_manager, configuration_manager, dataset_json)
+            trainer_class = recursive_find_python_class(join(nnunetv2.__path__[0], "training", "nnUNetTrainer"),
+                                                        trainer_name, 'nnunetv2.training.nnUNetTrainer')
+            if trainer_class is None:
+                raise RuntimeError(f'Unable to locate trainer class {trainer_name} in nnunetv2.training.nnUNetTrainer. '
+                                f'Please place it there (in any .py file)!')
+            network = trainer_class.build_network_architecture(
+                configuration_manager.network_arch_class_name,
+                configuration_manager.network_arch_init_kwargs,
+                configuration_manager.network_arch_init_kwargs_req_import,
+                num_input_channels,
+                plans_manager.get_label_manager(dataset_json).num_segmentation_heads,
+                enable_deep_supervision=False
+            )
+            self.network = network
+            self.allowed_mirroring_axes = inference_allowed_mirroring_axes
+
+        self.plans_manager = plans_manager
+        self.configuration_manager = configuration_manager
+        self.dataset_json = dataset_json
+        self.label_manager = plans_manager.get_label_manager(dataset_json)
+
+        if save_model:
+            if not isinstance(self.network, OptimizedModule):
+                self.network.load_state_dict(parameters[0])
+            else:
+                self.network._orig_mod.load_state_dict(parameters[0])
+            self.network.eval()
+
+        config = {hints.performance_mode: hints.PerformanceMode.LATENCY,
+                 hints.enable_cpu_pinning(): True,
+                 }
+        core = ov.Core()
+        core.set_property(
+            "CPU",
+            {hints.execution_mode: hints.ExecutionMode.PERFORMANCE},
+        )
+        if save_model:
+            input_tensor = torch.randn(1, num_input_channels, *configuration_manager.patch_size, requires_grad=False)
+            ov_model = ov.convert_model(self.network, example_input=input_tensor)
+            ov.save_model(ov_model, f"{model_training_output_dir}/model.xml")
+            import sys
+            sys.exit(0)
+        ov_model = core.read_model(f"{model_training_output_dir}/model.xml")
+        self.network = core.compile_model(ov_model, "CPU", config= config)
+
+    def predict_from_files(self,
+                           list_of_lists_or_source_folder: Union[str, List[List[str]]],
+                           output_folder_or_list_of_truncated_output_files: Union[str, None, List[str]],
+                           save_probabilities: bool = False,
+                           overwrite: bool = True,
+                           num_processes_preprocessing: int = 1,
+                           num_processes_segmentation_export: int = 1,
+                           folder_with_segs_from_prev_stage: str = None,
+                           num_parts: int = 1,
+                           part_id: int = 0):
+        """
+        This is nnU-Net's default function for making predictions. It works best for batch predictions
+        (predicting many images at once).
+        """
+
+        list_of_lists_or_source_folder, output_filename_truncated, seg_from_prev_stage_files = \
+            self._manage_input_and_output_lists(list_of_lists_or_source_folder,
+                                                output_folder_or_list_of_truncated_output_files,
+                                                folder_with_segs_from_prev_stage, overwrite, part_id, num_parts,
+                                                save_probabilities)
+        if len(list_of_lists_or_source_folder) == 0:
+            return
+
+        data_iterator = self._internal_get_data_iterator_from_lists_of_filenames(list_of_lists_or_source_folder,
+                                                                                 seg_from_prev_stage_files,
+                                                                                 output_filename_truncated,
+                                                                                 num_processes_preprocessing)
+        
+        return self.predict_from_data_iterator(data_iterator, save_probabilities, num_processes_segmentation_export)
+
+    def _internal_maybe_mirror_and_predict(self, x: torch.Tensor) -> torch.Tensor:
+        mirror_axes = self.allowed_mirroring_axes if self.use_mirroring else None
+        if self.use_openvino:
+            prediction = torch.from_numpy(self.network(x)[0])
+        else:
+            prediction = self.network(x)
+
+        if mirror_axes is not None:
+            assert max(mirror_axes) <= x.ndim - 3, 'mirror_axes does not match the dimension of the input!'
+
+            mirror_axes = [m + 2 for m in mirror_axes]
+            axes_combinations = [
+                c for i in range(len(mirror_axes)) for c in itertools.combinations(mirror_axes, i + 1)
+            ]
+            for axes in axes_combinations:
+                if not self.is_openvino:
+                    prediction += torch.flip(self.network(torch.flip(x, axes)), axes)
+                else:
+                    temp_pred = torch.from_numpy(self.network(torch.flip(x, axes))[0])
+                    prediction += torch.flip(temp_pred, axes)
+
+            prediction /= (len(axes_combinations) + 1)
+        return prediction
+
+    def _internal_predict_sliding_window_return_logits(self,
+                                                       data: torch.Tensor,
+                                                       slicers,
+                                                       do_on_device: bool = True,
+                                                       ):
+        predicted_logits = n_predictions = prediction = gaussian = workon = None
+        results_device = self.device if do_on_device else torch.device('cpu')
+
+        try:
+            empty_cache(self.device)
+
+            # move data to device
+            if self.verbose:
+                print(f'move image to device {results_device}')
+            data = data.to(results_device)
+
+            # preallocate arrays
+            if self.verbose:
+                print(f'preallocating results arrays on device {results_device}')
+            predicted_logits = torch.zeros((self.label_manager.num_segmentation_heads, *data.shape[1:]),
+                                        dtype=torch.half,
+                                        device=results_device)
+            n_predictions = torch.zeros(data.shape[1:], dtype=torch.half, device=results_device)
+
+            if self.use_gaussian:
+                gaussian = compute_gaussian(tuple(self.configuration_manager.patch_size), sigma_scale=1. / 8,
+                                            value_scaling_factor=10,
+                                            device=results_device)
+            else:
+                gaussian = 1
+
+            if not self.allow_tqdm and self.verbose:
+                print(f'running prediction: {len(slicers)} steps')
+            for sl in tqdm(slicers, disable=not self.allow_tqdm):
+                workon = data[sl][None]
+                workon = workon.to(self.device)
+
+                prediction = self._internal_maybe_mirror_and_predict(workon)[0].to(results_device)
+
+                if self.use_gaussian:
+                    prediction *= gaussian
+                predicted_logits[sl] += prediction
+                n_predictions[sl[1:]] += gaussian
+
+            predicted_logits /= n_predictions
+            # check for infs
+            if torch.any(torch.isinf(predicted_logits)):
+                raise RuntimeError('Encountered inf in predicted array. Aborting... If this problem persists, '
+                                'reduce value_scaling_factor in compute_gaussian or increase the dtype of '
+                                'predicted_logits to fp32')
+        except Exception as e:
+            del predicted_logits, n_predictions, prediction, gaussian, workon
+            empty_cache(self.device)
+            empty_cache(results_device)
+            raise e
+        return predicted_logits
+
+    def predict_logits_from_preprocessed_data(self, data: torch.Tensor) -> torch.Tensor:
+        """
+        IMPORTANT! IF YOU ARE RUNNING THE CASCADE, THE SEGMENTATION FROM THE PREVIOUS STAGE MUST ALREADY BE STACKED ON
+        TOP OF THE IMAGE AS ONE-HOT REPRESENTATION! SEE PreprocessAdapter ON HOW THIS SHOULD BE DONE!
+
+        RETURNED LOGITS HAVE THE SHAPE OF THE INPUT. THEY MUST BE CONVERTED BACK TO THE ORIGINAL IMAGE SIZE.
+        SEE convert_predicted_logits_to_segmentation_with_correct_shape
+        """
+        prediction = None
+
+        if not self.use_openvino:
+            for params in self.list_of_parameters:
+
+                # messing with state dict names...
+                if not isinstance(self.network, OptimizedModule):
+                    self.network.load_state_dict(params)
+                else:
+                    self.network._orig_mod.load_state_dict(params)
+
+                # why not leave prediction on device if perform_everything_on_device? Because this may cause the
+                # second iteration to crash due to OOM. Grabbing that with try except cause way more bloated code than
+                # this actually saves computation time
+                if prediction is None:
+                    prediction = self.predict_sliding_window_return_logits(data).to('cpu')
+                else:
+                    prediction += self.predict_sliding_window_return_logits(data).to('cpu')
+
+            if len(self.list_of_parameters) > 1:
+                prediction /= len(self.list_of_parameters)
+
+        else:
+            if prediction is None:
+                prediction = self.predict_sliding_window_return_logits(data)
+            else:
+                prediction += self.predict_sliding_window_return_logits(data)
+
+        if self.verbose: print('Prediction done')
+        return prediction
+
+    @torch.inference_mode()
+    def predict_sliding_window_return_logits(self, input_image: torch.Tensor) \
+            -> Union[np.ndarray, torch.Tensor]:
+        assert isinstance(input_image, torch.Tensor)
+        if self.device not in [torch.device('cpu'), 'cpu']:
+            self.network = self.network.to(self.device)
+            self.network.eval()
+
+        empty_cache(self.device)
+
+        # Autocast can be annoying
+        # If the device_type is 'cpu' then it's slow as heck on some CPUs (no auto bfloat16 support detection)
+        # and needs to be disabled.
+        # If the device_type is 'mps' then it will complain that mps is not implemented, even if enabled=False
+        # is set. Whyyyyyyy. (this is why we don't make use of enabled=False)
+        # So autocast will only be active if we have a cuda device.
+        with torch.autocast(self.device.type, enabled=True) if self.device.type == 'cuda' else dummy_context():
+            assert input_image.ndim == 4, 'input_image must be a 4D np.ndarray or torch.Tensor (c, x, y, z)'
+
+            if self.verbose: 
+                print(f'Input shape: {input_image.shape}')
+                print("step_size:", self.tile_step_size)
+                print("mirror_axes:", self.allowed_mirroring_axes if self.use_mirroring else None)
+
+            # if input_image is smaller than tile_size we need to pad it to tile_size.
+            data, slicer_revert_padding = pad_nd_image(input_image, self.configuration_manager.patch_size,
+                                                       'constant', {'value': 0}, True,
+                                                       None)
+
+            slicers = self._internal_get_sliding_window_slicers(data.shape[1:])
+
+            if self.perform_everything_on_device and self.device != 'cpu':
+                # we need to try except here because we can run OOM in which case we need to fall back to CPU as a results device
+                try:
+                    predicted_logits = self._internal_predict_sliding_window_return_logits(data, slicers,
+                                                                                           self.perform_everything_on_device)
+                except RuntimeError:
+                    print(
+                        'Prediction on device was unsuccessful, probably due to a lack of memory. Moving results arrays to CPU')
+                    empty_cache(self.device)
+                    predicted_logits = self._internal_predict_sliding_window_return_logits(data, slicers, False)
+            else:
+                predicted_logits = self._internal_predict_sliding_window_return_logits(data, slicers,
+                                                                                       self.perform_everything_on_device)
+
+            empty_cache(self.device)
+            # revert padding
+            predicted_logits = predicted_logits[(slice(None), *slicer_revert_padding[1:])]
+        return predicted_logits
+
+    def convert_predicted_logits_to_segmentation_with_correct_shape(self, predicted_logits: Union[torch.Tensor, np.ndarray],
+                                                                    plans_manager: PlansManager,
+                                                                    configuration_manager: ConfigurationManager,
+                                                                    label_manager: LabelManager,
+                                                                    properties_dict: dict,
+                                                                    return_probabilities: bool = False,
+                                                                    num_threads_torch: int = default_num_processes):
+
+        # resample to original shape
+        current_spacing = configuration_manager.spacing if \
+            len(configuration_manager.spacing) == \
+            len(properties_dict['shape_after_cropping_and_before_resampling']) else \
+            [properties_dict['spacing'][0], *configuration_manager.spacing]
+        predicted_logits = configuration_manager.resampling_fn_probabilities(predicted_logits,
+                                                properties_dict['shape_after_cropping_and_before_resampling'],
+                                                current_spacing,
+                                                properties_dict['spacing'])
+        segmentation = predicted_logits.argmax(0)
+        del predicted_logits
+
+        # segmentation may be torch.Tensor but we continue with numpy
+        if isinstance(segmentation, torch.Tensor):
+            segmentation = segmentation.cpu().numpy()
+
+        # put segmentation in bbox (revert cropping)
+        segmentation_reverted_cropping = np.zeros(properties_dict['shape_before_cropping'],
+                                                  dtype=np.uint8 if len(label_manager.foreground_labels) < 255 else np.uint16)
+        slicer = bounding_box_to_slice(properties_dict['bbox_used_for_cropping'])
+        segmentation_reverted_cropping[slicer] = segmentation
+        del segmentation
+
+        # revert transpose
+        segmentation_reverted_cropping = segmentation_reverted_cropping.transpose(plans_manager.transpose_backward)
+        return segmentation_reverted_cropping
+
+    def export_prediction_from_logits(self, predicted_array_or_file: Union[np.ndarray, torch.Tensor], properties_dict: dict,
+                                      configuration_manager: ConfigurationManager,
+                                      plans_manager: PlansManager,
+                                      dataset_json_dict_or_file: Union[dict, str], output_file_truncated: str,
+                                      save_probabilities: bool = False):
+
+        if isinstance(dataset_json_dict_or_file, str):
+            dataset_json_dict_or_file = load_json(dataset_json_dict_or_file)
+
+        label_manager = plans_manager.get_label_manager(dataset_json_dict_or_file)
+        ret = self.convert_predicted_logits_to_segmentation_with_correct_shape(
+            predicted_array_or_file, plans_manager, configuration_manager, label_manager, properties_dict,
+            return_probabilities=save_probabilities
+        )
+        del predicted_array_or_file
+
+        segmentation_final = ret
+
+        rw = plans_manager.image_reader_writer_class()
+        rw.write_seg(segmentation_final, output_file_truncated + dataset_json_dict_or_file['file_ending'],
+                     properties_dict)
+
+    def predict_single_npy_array(self, input_image: np.ndarray, image_properties: dict,
+                                 segmentation_previous_stage: np.ndarray = None,
+                                 output_file_truncated: str = None,
+                                 save_or_return_probabilities: bool = False):
+        """
+        WARNING: SLOW. ONLY USE THIS IF YOU CANNOT GIVE NNUNET MULTIPLE IMAGES AT ONCE FOR SOME REASON.
+
+
+        input_image: Make sure to load the image in the way nnU-Net expects! nnU-Net is trained on a certain axis
+                     ordering which cannot be disturbed in inference,
+                     otherwise you will get bad results. The easiest way to achieve that is to use the same I/O class
+                     for loading images as was used during nnU-Net preprocessing! You can find that class in your
+                     plans.json file under the key "image_reader_writer". If you decide to freestyle, know that the
+                     default axis ordering for medical images is the one from SimpleITK. If you load with nibabel,
+                     you need to transpose your axes AND your spacing from [x,y,z] to [z,y,x]!
+        image_properties must only have a 'spacing' key!
+        """
+        ppa = PreprocessAdapterFromNpy([input_image], [segmentation_previous_stage], [image_properties],
+                                       [output_file_truncated],
+                                       self.plans_manager, self.dataset_json, self.configuration_manager,
+                                       num_threads_in_multithreaded=1, verbose=self.verbose)
+        if self.verbose:
+            print('preprocessing')
+        dct = next(ppa)
+
+        if self.verbose:
+            print('predicting')
+        predicted_logits = self.predict_logits_from_preprocessed_data(dct['data']).cpu()
+
+        if self.verbose:
+            print('resampling to original shape')
+        self.export_prediction_from_logits(predicted_logits, dct['data_properties'], self.configuration_manager,
+                                        self.plans_manager, self.dataset_json, output_file_truncated,
+                                        save_or_return_probabilities)
+
+
+def predict_flare(input_dir, output_dir, model_folder, save_model):
+    input_dir = Path(input_dir)
+    output_dir = Path(output_dir)
+    output_dir.mkdir(exist_ok=True, parents=True)
+    input_files = list(input_dir.glob("*.nii.gz"))
+    output_files = [str(output_dir / f.name[:-12]) for f in input_files]
+    for input_file, output_file in zip(input_files, output_files):
+        print(f"Predicting {input_file.name}")
+        start = time()
+        predictor = FlarePredictor(tile_step_size=0.5, use_mirroring=False, device=torch.device("cpu"))
+        predictor.initialize_from_trained_model_folder(model_folder, ("all",), save_model=save_model)
+        rw = predictor.plans_manager.image_reader_writer_class()
+        image, props = rw.read_images([input_file,])
+        _ = predictor.predict_single_npy_array(image, props, None, output_file, False)
+        print(f"Prediction time: {time() - start:.2f}s")
+
+
+if __name__ == '__main__':
+    parser = argparse.ArgumentParser()
+    parser.add_argument("-i", "--input", default="/workspace/inputs")
+    parser.add_argument("-o", "--output", default="/workspace/outputs")
+    parser.add_argument("-m", "--model", default="/opt/app/_trained_model")
+    parser.add_argument("-save_model", action="store_true")
+    args = parser.parse_args()
+    predict_flare(args.input, args.output, args.model, args.save_model)

nnUNet/documentation/competitions/FLARE24/Task_2/readme.md

@@ -0,0 +1,102 @@
+Authors: \
+Yannick Kirchhoff*, Ashis Ravindran*, Maximilian Rouven Rokuss, Benjamin Hamm, Constantin Ulrich, Klaus Maier-Hein<sup>&#8224;</sup>, Fabian Isensee<sup>&#8224;</sup>
+
+*: equal contribution \
+&#8224;: equal contribution
+
+# Introduction
+
+This document describes our contribution to [Task 2 of the FLARE24 Challenge](https://www.codabench.org/competitions/2320/).
+Our model is basically is a default nnU-Net with a custom low resolution setting and OpenVINO optimizations for faster CPU inference.
+
+# Experiment Planning and Preprocessing
+
+Bring the downloaded data into the [nnU-Net format](../../../nnUNet/documentation/dataset_format.md) and add the dataset.json file as given here:
+
+```json
+{
+    "name": "Dataset311_FLARE24Task2_labeled",
+    "description": "Abdominal Organ Segmentation",
+    "labels": {
+        "background": 0,
+        "liver": 1,
+        "right kidney": 2,
+        "spleen": 3,
+        "pancreas": 4,
+        "aorta": 5,
+        "ivc": 6,
+        "rag": 7,
+        "lag": 8,
+        "gallbladder": 9,
+        "esophagus": 10,
+        "stomach": 11,
+        "duodenum": 12,
+        "left kidney": 13
+    },
+    "file_ending": ".nii.gz",
+    "channel_names": {
+        "0": "CT"
+    },
+    "overwrite_image_reader_writer": "NibabelIOWithReorient",
+    "numTraining": 50
+}
+```
+
+Afterwards you can run the default nnU-Net planning and preprocessing
+
+```bash
+nnUNetv2_plan_and_preprocess -d 311 -c 3d_fullres
+```
+
+## Edit the plans files
+
+The generated `nnUNetPlans.json` file needs to be edited to incorporate the custom low resolution setting.
+
+```json
+        "3d_halfres": {
+            "inherits_from": "3d_fullres",
+            "data_identifier": "nnUNetPlans_3d_halfres",
+            "spacing": [
+                5,
+                1.6,
+                1.6
+            ]
+        },
+        "3d_halfiso": {
+            "inherits_from": "3d_fullres",
+            "data_identifier": "nnUNetPlans_3d_halfiso",
+            "spacing": [
+                2.5,
+                2.5,
+                2.5
+            ]
+        },
+```
+
+`3d_halfres` is a configuration with exactly half resolution, used as an ablation of our submission, `3d_halfiso` is the isotropic configuration we submitted as a final solution.
+
+# Model training
+
+Run one of the following commands to train the respective configurations. `3d_halfiso` yielded significantly better results in our experiments as well as on the final test set and is the recommended configuration.
+
+```bash
+nnUNetv2_train 311 3d_halfres all
+
+nnUNetv2_train 311 3d_halfiso all
+```
+
+# Inference
+
+Our inference is optimized for efficient single scan prediction. For best performance, we strongly recommend running inference using the default `nnUNetv2_predict` command!
+
+Inference using the provided script requires OpenVINO, which can easily be installed via
+
+```bash
+pip install openvino
+```
+
+To run inference simply run the following commands. `model_folder` is the folder containing the training results, i.e. for example `nnUNetTrainer__nnUNetPlans__3d_halfiso`. `-save_model` needs to be set to precompile the model once using OpenVINO. If not precompiled model exists, the inference script will fail!
+
+```bash
+python inference_flare_task1.py -i input_folder -o output_folder -m model_folder [-save_model]
+```

nnUNet/documentation/competitions/Toothfairy2/inference_script_semseg_only_customInf2.py

@@ -0,0 +1,343 @@
+import argparse
+import gc
+import os
+from pathlib import Path
+from queue import Queue
+from threading import Thread
+from typing import Union, Tuple
+
+import nnunetv2
+import numpy as np
+import torch
+from acvl_utils.cropping_and_padding.bounding_boxes import bounding_box_to_slice
+from acvl_utils.cropping_and_padding.padding import pad_nd_image
+from batchgenerators.utilities.file_and_folder_operations import load_json, join
+from nnunetv2.imageio.simpleitk_reader_writer import SimpleITKIO
+from nnunetv2.inference.predict_from_raw_data import nnUNetPredictor
+from nnunetv2.inference.sliding_window_prediction import compute_gaussian
+from nnunetv2.utilities.find_class_by_name import recursive_find_python_class
+from nnunetv2.utilities.helpers import empty_cache, dummy_context
+from nnunetv2.utilities.label_handling.label_handling import determine_num_input_channels
+from nnunetv2.utilities.plans_handling.plans_handler import PlansManager
+from torch._dynamo import OptimizedModule
+from torch.backends import cudnn
+from tqdm import tqdm
+
+
+class CustomPredictor(nnUNetPredictor):
+    def predict_single_npy_array(self, input_image: np.ndarray, image_properties: dict,
+                                 segmentation_previous_stage: np.ndarray = None):
+        torch.set_num_threads(7)
+        with torch.no_grad():
+            self.network = self.network.to(self.device)
+            self.network.eval()
+
+            if self.verbose:
+                print('preprocessing')
+            preprocessor = self.configuration_manager.preprocessor_class(verbose=self.verbose)
+            data, _, image_properties = preprocessor.run_case_npy(input_image, None, image_properties,
+                                                self.plans_manager,
+                                                self.configuration_manager,
+                                                self.dataset_json)
+
+            data = torch.from_numpy(data)
+            del input_image
+            if self.verbose:
+                print('predicting')
+
+            predicted_logits = self.predict_preprocessed_image(data)
+
+            if self.verbose: print('Prediction done')
+
+            segmentation = self.convert_predicted_logits_to_segmentation_with_correct_shape(predicted_logits,
+                                                                                            image_properties)
+        return segmentation
+
+    def initialize_from_trained_model_folder(self, model_training_output_dir: str,
+                                             use_folds: Union[Tuple[Union[int, str]], None],
+                                             checkpoint_name: str = 'checkpoint_final.pth'):
+        """
+        This is used when making predictions with a trained model
+        """
+        if use_folds is None:
+            use_folds = nnUNetPredictor.auto_detect_available_folds(model_training_output_dir, checkpoint_name)
+
+        dataset_json = load_json(join(model_training_output_dir, 'dataset.json'))
+        plans = load_json(join(model_training_output_dir, 'plans.json'))
+        plans_manager = PlansManager(plans)
+
+        if isinstance(use_folds, str):
+            use_folds = [use_folds]
+
+        parameters = []
+        for i, f in enumerate(use_folds):
+            f = int(f) if f != 'all' else f
+            checkpoint = torch.load(join(model_training_output_dir, f'fold_{f}', checkpoint_name),
+                                    map_location=torch.device('cpu'), weights_only=False)
+            if i == 0:
+                trainer_name = checkpoint['trainer_name']
+                configuration_name = checkpoint['init_args']['configuration']
+                inference_allowed_mirroring_axes = checkpoint['inference_allowed_mirroring_axes'] if \
+                    'inference_allowed_mirroring_axes' in checkpoint.keys() else None
+
+            parameters.append(join(model_training_output_dir, f'fold_{f}', checkpoint_name))
+
+        configuration_manager = plans_manager.get_configuration(configuration_name)
+        # restore network
+        num_input_channels = determine_num_input_channels(plans_manager, configuration_manager, dataset_json)
+        trainer_class = recursive_find_python_class(join(nnunetv2.__path__[0], "training", "nnUNetTrainer"),
+                                                    trainer_name, 'nnunetv2.training.nnUNetTrainer')
+        if trainer_class is None:
+            raise RuntimeError(f'Unable to locate trainer class {trainer_name} in nnunetv2.training.nnUNetTrainer. '
+                               f'Please place it there (in any .py file)!')
+        network = trainer_class.build_network_architecture(
+            configuration_manager.network_arch_class_name,
+            configuration_manager.network_arch_init_kwargs,
+            configuration_manager.network_arch_init_kwargs_req_import,
+            num_input_channels,
+            plans_manager.get_label_manager(dataset_json).num_segmentation_heads,
+            enable_deep_supervision=False
+        )
+
+        self.plans_manager = plans_manager
+        self.configuration_manager = configuration_manager
+        self.list_of_parameters = parameters
+        self.network = network
+        self.dataset_json = dataset_json
+        self.trainer_name = trainer_name
+        self.allowed_mirroring_axes = inference_allowed_mirroring_axes
+        self.label_manager = plans_manager.get_label_manager(dataset_json)
+        if ('nnUNet_compile' in os.environ.keys()) and (os.environ['nnUNet_compile'].lower() in ('true', '1', 't')) \
+                and not isinstance(self.network, OptimizedModule):
+            print('Using torch.compile')
+            self.network = torch.compile(self.network)
+
+    @torch.inference_mode(mode=True)
+    def predict_preprocessed_image(self, image):
+        empty_cache(self.device)
+        data_device = torch.device('cpu')
+        predicted_logits_device = torch.device('cpu')
+        gaussian_device = torch.device('cpu')
+        compute_device = torch.device('cuda:0')
+
+        data, slicer_revert_padding = pad_nd_image(image, self.configuration_manager.patch_size,
+                                                   'constant', {'value': 0}, True,
+                                                   None)
+        del image
+
+        slicers = self._internal_get_sliding_window_slicers(data.shape[1:])
+
+        empty_cache(self.device)
+
+        data = data.to(data_device)
+        predicted_logits = torch.zeros((self.label_manager.num_segmentation_heads, *data.shape[1:]),
+                                       dtype=torch.half,
+                                       device=predicted_logits_device)
+        gaussian = compute_gaussian(tuple(self.configuration_manager.patch_size), sigma_scale=1. / 8,
+                                    value_scaling_factor=10,
+                                    device=gaussian_device, dtype=torch.float16)
+
+        if not self.allow_tqdm and self.verbose:
+            print(f'running prediction: {len(slicers)} steps')
+
+        for p in self.list_of_parameters:
+            # network weights have to be updated outside autocast!
+            # we are loading parameters on demand instead of loading them upfront. This reduces memory footprint a lot.
+            # each set of parameters is only used once on the test set (one image) so run time wise this is almost the
+            # same
+            self.network.load_state_dict(torch.load(p, map_location=compute_device)['network_weights'], weights_only=False)
+            with torch.autocast(self.device.type, enabled=True):
+                for sl in tqdm(slicers, disable=not self.allow_tqdm):
+                    pred = self._internal_maybe_mirror_and_predict(data[sl][None].to(compute_device))[0].to(
+                        predicted_logits_device)
+                    pred /= (pred.max() / 100)
+                    predicted_logits[sl] += (pred * gaussian)
+                del pred
+        empty_cache(self.device)
+        return predicted_logits
+
+    def convert_predicted_logits_to_segmentation_with_correct_shape(self, predicted_logits, props):
+        old = torch.get_num_threads()
+        torch.set_num_threads(7)
+
+        # resample to original shape
+        spacing_transposed = [props['spacing'][i] for i in self.plans_manager.transpose_forward]
+        current_spacing = self.configuration_manager.spacing if \
+            len(self.configuration_manager.spacing) == \
+            len(props['shape_after_cropping_and_before_resampling']) else \
+            [spacing_transposed[0], *self.configuration_manager.spacing]
+        predicted_logits = self.configuration_manager.resampling_fn_probabilities(predicted_logits,
+                                                                                  props[
+                                                                                      'shape_after_cropping_and_before_resampling'],
+                                                                                  current_spacing,
+                                                                                  [props['spacing'][i] for i in
+                                                                                   self.plans_manager.transpose_forward])
+
+        segmentation = None
+        pp = None
+        try:
+            with torch.no_grad():
+                pp = predicted_logits.to('cuda:0')
+                segmentation = pp.argmax(0).cpu()
+                del pp
+        except RuntimeError:
+            del segmentation, pp
+            torch.cuda.empty_cache()
+            segmentation = predicted_logits.argmax(0)
+        del predicted_logits
+
+        # segmentation may be torch.Tensor but we continue with numpy
+        if isinstance(segmentation, torch.Tensor):
+            segmentation = segmentation.cpu().numpy()
+
+        # put segmentation in bbox (revert cropping)
+        segmentation_reverted_cropping = np.zeros(props['shape_before_cropping'],
+                                                  dtype=np.uint8 if len(
+                                                      self.label_manager.foreground_labels) < 255 else np.uint16)
+        slicer = bounding_box_to_slice(props['bbox_used_for_cropping'])
+        segmentation_reverted_cropping[slicer] = segmentation
+        del segmentation
+
+        # revert transpose
+        segmentation_reverted_cropping = segmentation_reverted_cropping.transpose(self.plans_manager.transpose_backward)
+        torch.set_num_threads(old)
+        return segmentation_reverted_cropping
+
+
+def predict_semseg(im, prop, semseg_trained_model, semseg_folds):
+    # initialize predictors
+    pred_semseg = CustomPredictor(
+        tile_step_size=0.5,
+        use_mirroring=True,
+        use_gaussian=True,
+        perform_everything_on_device=False,
+        allow_tqdm=True
+    )
+    pred_semseg.initialize_from_trained_model_folder(
+        semseg_trained_model,
+        use_folds=semseg_folds,
+        checkpoint_name='checkpoint_final.pth'
+    )
+
+    semseg_pred = pred_semseg.predict_single_npy_array(
+        im, prop, None
+    )
+    torch.cuda.empty_cache()
+    gc.collect()
+    return semseg_pred
+
+
+def map_labels_to_toothfairy(predicted_seg: np.ndarray) -> np.ndarray:
+    # Create an array that maps the labels directly
+    max_label = 42
+    mapping = np.arange(max_label + 1)
+
+    # Define the specific remapping
+    remapping = {19: 21, 20: 22, 21: 23, 22: 24, 23: 25, 24: 26, 25: 27, 26: 28,
+                 27: 31, 28: 32, 29: 33, 30: 34, 31: 35, 32: 36, 33: 37, 34: 38,
+                 35: 41, 36: 42, 37: 43, 38: 44, 39: 45, 40: 46, 41: 47, 42: 48}
+
+    # Apply the remapping
+    for k, v in remapping.items():
+        mapping[k] = v
+
+    return mapping[predicted_seg]
+
+
+def postprocess(prediction_npy, vol_per_voxel, verbose: bool = False):
+    cutoffs = {1: 0.0,
+               2: 78411.5,
+               3: 0.0,
+               4: 0.0,
+               5: 2800.0,
+               6: 1216.5,
+               7: 0.0,
+               8: 6222.0,
+               9: 1573.0,
+               10: 946.0,
+               11: 0.0,
+               12: 6783.5,
+               13: 9469.5,
+               14: 0.0,
+               15: 2260.0,
+               16: 3566.0,
+               17: 6321.0,
+               18: 4221.5,
+               19: 5829.0,
+               20: 0.0,
+               21: 0.0,
+               22: 468.0,
+               23: 1555.0,
+               24: 1291.5,
+               25: 2834.5,
+               26: 584.5,
+               27: 0.0,
+               28: 0.0,
+               29: 0.0,
+               30: 0.0,
+               31: 1935.5,
+               32: 0.0,
+               33: 0.0,
+               34: 6140.0,
+               35: 0.0,
+               36: 0.0,
+               37: 0.0,
+               38: 2710.0,
+               39: 0.0,
+               40: 0.0,
+               41: 0.0,
+               42: 970.0}
+
+    vol_per_voxel_cutoffs = 0.3 * 0.3 * 0.3
+    for c in cutoffs.keys():
+        co = cutoffs[c]
+        if co > 0:
+            mask = prediction_npy == c
+            pred_vol = np.sum(mask) * vol_per_voxel
+            if 0 < pred_vol < (co * vol_per_voxel_cutoffs):
+                prediction_npy[mask] = 0
+                if verbose:
+                    print(
+                        f'removed label {c} because predicted volume of {pred_vol} is less than the cutoff {co * vol_per_voxel_cutoffs}')
+    return prediction_npy
+
+
+if __name__ == '__main__':
+    os.environ['nnUNet_compile'] = 'f'
+
+    parser = argparse.ArgumentParser()
+    parser.add_argument('-i', '--input_folder', type=Path, default="/input/images/cbct/")
+    parser.add_argument('-o', '--output_folder', type=Path, default="/output/images/oral-pharyngeal-segmentation/")
+    parser.add_argument('-sem_mod', '--semseg_trained_model', type=str,
+                        default="/opt/app/_trained_model/semseg_trained_model")
+    parser.add_argument('--semseg_folds', type=str, nargs='+', default=[0, 1])
+    args = parser.parse_args()
+
+    args.output_folder.mkdir(exist_ok=True, parents=True)
+
+    semseg_folds = [i if i == 'all' else int(i) for i in args.semseg_folds]
+    semseg_trained_model = args.semseg_trained_model
+
+    rw = SimpleITKIO()
+
+    input_files = list(args.input_folder.glob('*.nii.gz')) + list(args.input_folder.glob('*.mha'))
+
+    for input_fname in input_files:
+        output_fname = args.output_folder / input_fname.name
+
+        # we start with the instance seg because we can then start converting that while semseg is being predicted
+        # load test image
+        im, prop = rw.read_images([input_fname])
+
+        with torch.no_grad():
+            semseg_pred = predict_semseg(im, prop, semseg_trained_model, semseg_folds)
+            torch.cuda.empty_cache()
+            gc.collect()
+
+        # now postprocess
+        semseg_pred = postprocess(semseg_pred, np.prod(prop['spacing']), True)
+
+        semseg_pred = map_labels_to_toothfairy(semseg_pred)
+
+        # now save
+        rw.write_seg(semseg_pred, output_fname, prop)

nnUNet/documentation/competitions/Toothfairy2/readme.md

@@ -0,0 +1,208 @@
+Authors: \
+Fabian Isensee*, Yannick Kirchhoff*, Lars Kraemer, Max Rokuss, Constantin Ulrich, Klaus H. Maier-Hein 
+
+*: equal contribution
+
+Author Affiliations:\
+Division of Medical Image Computing, German Cancer Research Center (DKFZ), Heidelberg \
+Helmholtz Imaging
+
+# Introduction
+
+This document describes our submission to the [Toothfairy2 Challenge](https://toothfairy2.grand-challenge.org/toothfairy2/). 
+Our model is essentially a nnU-Net ResEnc L with the patch size upscaled to 160x320x320 pixels. We disable left/right 
+mirroring and train for 1500 instead of the standard 1000 epochs. Training was either done on 2xA100 40GB or one GH200 96GB.
+
+# Dataset Conversion
+
+# Experiment Planning and Preprocessing
+Adapt and run the [dataset conversion script](../../../nnunetv2/dataset_conversion/Dataset119_ToothFairy2_All.py). 
+This script just converts the mha files to nifti (smaller file size) and removes the unused label ids.
+
+## Extract fingerprint:
+`nnUNetv2_extract_fingerprint -d 119 -np 48`
+
+## Run planning:
+`nnUNetv2_plan_experiment -d 119 -pl nnUNetPlannerResEncL_torchres`
+
+This planner not only uses the ResEncL configuration but also replaces the default resampling scheme with one that is 
+faster (but less precise). Since all images in the challenge (train and test) should already have 0.3x0.3x0.3 spacing 
+resampling is not required. This is just here as a safety measure. The speed is needed at inference time because grand 
+challenge imposes a limit of 10 minutes per case.
+
+## Edit the plans files
+Add the following configuration to the generated plans file:
+
+```json
+        "3d_fullres_torchres_ps160x320x320_bs2": {
+            "inherits_from": "3d_fullres",
+            "data_identifier": "nnUNetPlans_3d_fullres_torchres_ctnorm",
+            "patch_size": [
+                160,
+                320,
+                320
+            ],
+            "normalization_schemes": [
+                "CTNormalization"
+            ],
+            "architecture": {
+                "network_class_name": "dynamic_network_architectures.architectures.unet.ResidualEncoderUNet",
+                "arch_kwargs": {
+                    "n_stages": 7,
+                    "features_per_stage": [
+                        32,
+                        64,
+                        128,
+                        256,
+                        320,
+                        320,
+                        320
+                    ],
+                    "conv_op": "torch.nn.modules.conv.Conv3d",
+                    "kernel_sizes": [
+                        [
+                            3,
+                            3,
+                            3
+                        ],
+                        [
+                            3,
+                            3,
+                            3
+                        ],
+                        [
+                            3,
+                            3,
+                            3
+                        ],
+                        [
+                            3,
+                            3,
+                            3
+                        ],
+                        [
+                            3,
+                            3,
+                            3
+                        ],
+                        [
+                            3,
+                            3,
+                            3
+                        ],
+                        [
+                            3,
+                            3,
+                            3
+                        ]
+                    ],
+                    "strides": [
+                        [
+                            1,
+                            1,
+                            1
+                        ],
+                        [
+                            2,
+                            2,
+                            2
+                        ],
+                        [
+                            2,
+                            2,
+                            2
+                        ],
+                        [
+                            2,
+                            2,
+                            2
+                        ],
+                        [
+                            2,
+                            2,
+                            2
+                        ],
+                        [
+                            2,
+                            2,
+                            2
+                        ],
+                        [
+                            1,
+                            2,
+                            2
+                        ]
+                    ],
+                    "n_blocks_per_stage": [
+                        1,
+                        3,
+                        4,
+                        6,
+                        6,
+                        6,
+                        6
+                    ],
+                    "n_conv_per_stage_decoder": [
+                        1,
+                        1,
+                        1,
+                        1,
+                        1,
+                        1
+                    ],
+                    "conv_bias": true,
+                    "norm_op": "torch.nn.modules.instancenorm.InstanceNorm3d",
+                    "norm_op_kwargs": {
+                        "eps": 1e-05,
+                        "affine": true
+                    },
+                    "dropout_op": null,
+                    "dropout_op_kwargs": null,
+                    "nonlin": "torch.nn.LeakyReLU",
+                    "nonlin_kwargs": {
+                        "inplace": true
+                    }
+                },
+                "_kw_requires_import": [
+                    "conv_op",
+                    "norm_op",
+                    "dropout_op",
+                    "nonlin"
+                ]
+            }            
+        }
+```
+Aside from changing the patch size this makes the architecture one stage deeper (one more pooling + res blocks), enabling
+it to make effective use of the larger input
+
+# Preprocessing
+`nnUNetv2_preprocess -d 119 -c 3d_fullres_torchres_ps160x320x320_bs2 -plans_name nnUNetResEncUNetLPlans_torchres -np 48`
+
+# Training
+We train two models on all training cases:
+
+```bash
+nnUNetv2_train 119 3d_fullres_torchres_ps160x320x320_bs2 all -p nnUNetResEncUNetLPlans_torchres -tr nnUNetTrainer_onlyMirror01_1500ep
+nnUNet_results=${nnUNet_results}_2 nnUNetv2_train 119 3d_fullres_torchres_ps160x320x320_bs2 all -p nnUNetResEncUNetLPlans_torchres -tr nnUNetTrainer_onlyMirror01_1500ep
+```
+Models are trained from scratch.
+
+Note how in the second line we overwrite the nnUNet_results variable in order to be able to train the same model twice without overwriting the results
+
+We recommend to increase the number of processes used for data augmentation. Otherwise you can run into CPU bottlenecks.
+Use `export nnUNet_n_proc_DA=32` or higher (if your system permits!).
+
+# Inference
+We ensemble the two models from above. On a technical level we copy the two fold_all folders into one training output 
+directory and rename them to fold_0 and fold_1. This lets us use nnU-Net's cross-validation ensembling strategy which 
+is more computationally efficient (needed for time limit on grand-challenge.org).
+
+Run inference with the [inference script](inference_script_semseg_only_customInf2.py)
+
+# Postprocessing
+If the prediction of a class on some test case is smaller than the corresponding cutoff size then it is removed 
+(replaced with background).
+
+Cutoff values were optimized using a five-fold cross-validation on the Toothfairy2 training data. We optimize HD95 and Dice separately. 
+The final cutoff for each class is then the smaller value between the two metrics. You can find our volume cutoffs in the inference 
+script as part of our `postprocess` function.    

nnUNet/documentation/convert_msd_dataset.md

@@ -0,0 +1,3 @@
+Use `nnUNetv2_convert_MSD_dataset`.
+
+Read `nnUNetv2_convert_MSD_dataset -h` for usage instructions.

nnUNet/documentation/dataset_format.md

@@ -0,0 +1,254 @@
+# nnU-Net dataset format
+The only way to bring your data into nnU-Net is by storing it in a specific format. Due to nnU-Net's roots in the
+[Medical Segmentation Decathlon](http://medicaldecathlon.com/) (MSD), its dataset is heavily inspired but has since 
+diverged (see also [here](#how-to-use-decathlon-datasets)) from the format used in the MSD.
+
+Datasets consist of three components: raw images, corresponding segmentation maps and a dataset.json file specifying 
+some metadata. 
+
+If you are migrating from nnU-Net v1, read [this](#how-to-use-nnu-net-v1-tasks) to convert your existing Tasks.
+
+
+## What do training cases look like?
+Each training case is associated with an identifier = a unique name for that case. This identifier is used by nnU-Net to 
+connect images with the correct segmentation.
+
+A training case consists of images and their corresponding segmentation. 
+
+**Images** is plural because nnU-Net supports arbitrarily many input channels. In order to be as flexible as possible, 
+nnU-net requires each input channel to be stored in a separate image (with the sole exception being RGB natural 
+images). So these images could for example be a T1 and a T2 MRI (or whatever else you want). The different input 
+channels MUST have the same geometry (same shape, spacing (if applicable) etc.) and
+must be co-registered (if applicable). Input channels are identified by nnU-Net by their FILE_ENDING: a four-digit integer at the end 
+of the filename. Image files must therefore follow the following naming convention: {CASE_IDENTIFIER}_{XXXX}.{FILE_ENDING}. 
+Hereby, XXXX is the 4-digit modality/channel identifier (should be unique for each modality/channel, e.g., “0000” for T1, “0001” for 
+T2 MRI, …) and FILE_ENDING is the file extension used by your image format (.png, .nii.gz, ...). See below for concrete examples.
+The dataset.json file connects channel names with the channel identifiers in the 'channel_names' key (see below for details).
+
+Side note: Typically, each channel/modality needs to be stored in a separate file and is accessed with the XXXX channel identifier. 
+Exception are natural images (RGB; .png) where the three color channels can all be stored in one file (see the 
+[road segmentation](../nnunetv2/dataset_conversion/Dataset120_RoadSegmentation.py) dataset as an example). 
+
+**Segmentations** must share the same geometry with their corresponding images (same shape etc.). Segmentations are 
+integer maps with each value representing a semantic class. The background must be 0. If there is no background, then 
+do not use the label 0 for something else! Integer values of your semantic classes must be consecutive (0, 1, 2, 3, 
+...). Of course, not all labels have to be present in each training case. Segmentations are saved as {CASE_IDENTIFER}.{FILE_ENDING} .
+
+Within a training case, all image geometries (input channels, corresponding segmentation) must match. Between training 
+cases, they can of course differ. nnU-Net takes care of that.
+
+Important: The input channels must be consistent! Concretely, **all images need the same input channels in the same 
+order and all input channels have to be present every time**. This is also true for inference!
+
+
+## Supported file formats
+nnU-Net expects the same file format for images and segmentations! These will also be used for inference. For now, it 
+is thus not possible to train .png and then run inference on .jpg.
+
+One big change in nnU-Net V2 is the support of multiple input file types. Gone are the days of converting everything to .nii.gz!
+This is implemented by abstracting the input and output of images + segmentations through `BaseReaderWriter`. nnU-Net 
+comes with a broad collection of Readers+Writers and you can even add your own to support your data format! 
+See [here](../nnunetv2/imageio/readme.md).
+
+As a nice bonus, nnU-Net now also natively supports 2D input images and you no longer have to mess around with 
+conversions to pseudo 3D niftis. Yuck. That was disgusting.
+
+Note that internally (for storing and accessing preprocessed images) nnU-Net will use its own file format, irrespective 
+of what the raw data was provided in! This is for performance reasons.
+
+
+By default, the following file formats are supported:
+
+- NaturalImage2DIO: .png, .bmp, .tif
+- NibabelIO: .nii.gz, .nrrd, .mha
+- NibabelIOWithReorient: .nii.gz, .nrrd, .mha. This reader will reorient images to RAS!
+- SimpleITKIO: .nii.gz, .nrrd, .mha
+- Tiff3DIO: .tif, .tiff. 3D tif images! Since TIF does not have a standardized way of storing spacing information, 
+nnU-Net expects each TIF file to be accompanied by an identically named .json file that contains this information (see
+[here](#datasetjson)).
+
+The file extension lists are not exhaustive and depend on what the backend supports. For example, nibabel and SimpleITK 
+support more than the three given here. The file endings given here are just the ones we tested!
+
+IMPORTANT: nnU-Net can only be used with file formats that use lossless (or no) compression! Because the file 
+format is defined for an entire dataset (and not separately for images and segmentations, this could be a todo for 
+the future), we must ensure that there are no compression artifacts that destroy the segmentation maps. So no .jpg and 
+the likes! 
+
+## Dataset folder structure
+Datasets must be located in the `nnUNet_raw` folder (which you either define when installing nnU-Net or export/set every 
+time you intend to run nnU-Net commands!).
+Each segmentation dataset is stored as a separate 'Dataset'. Datasets are associated with a dataset ID, a three digit 
+integer, and a dataset name (which you can freely choose): For example, Dataset005_Prostate has 'Prostate' as dataset name and 
+the dataset id is 5. Datasets are stored in the `nnUNet_raw` folder like this:
+
+    nnUNet_raw/
+    ├── Dataset001_BrainTumour
+    ├── Dataset002_Heart
+    ├── Dataset003_Liver
+    ├── Dataset004_Hippocampus
+    ├── Dataset005_Prostate
+    ├── ...
+
+Within each dataset folder, the following structure is expected:
+
+    Dataset001_BrainTumour/
+    ├── dataset.json
+    ├── imagesTr
+    ├── imagesTs  # optional
+    └── labelsTr
+
+
+When adding your custom dataset, take a look at the [dataset_conversion](../nnunetv2/dataset_conversion) folder and 
+pick an id that is not already taken. IDs 001-010 are for the Medical Segmentation Decathlon.
+
+- **imagesTr** contains the images belonging to the training cases. nnU-Net will perform pipeline configuration, training with 
+cross-validation, as well as finding postprocessing and the best ensemble using this data. 
+- **imagesTs** (optional) contains the images that belong to the test cases. nnU-Net does not use them! This could just 
+be a convenient location for you to store these images. Remnant of the Medical Segmentation Decathlon folder structure.
+- **labelsTr** contains the images with the ground truth segmentation maps for the training cases. 
+- **dataset.json** contains metadata of the dataset.
+
+The scheme introduced [above](#what-do-training-cases-look-like) results in the following folder structure. Given 
+is an example for the first Dataset of the MSD: BrainTumour. This dataset hat four input channels: FLAIR (0000), 
+T1w (0001), T1gd (0002) and T2w (0003). Note that the imagesTs folder is optional and does not have to be present.
+
+    nnUNet_raw/Dataset001_BrainTumour/
+    ├── dataset.json
+    ├── imagesTr
+    │   ├── BRATS_001_0000.nii.gz
+    │   ├── BRATS_001_0001.nii.gz
+    │   ├── BRATS_001_0002.nii.gz
+    │   ├── BRATS_001_0003.nii.gz
+    │   ├── BRATS_002_0000.nii.gz
+    │   ├── BRATS_002_0001.nii.gz
+    │   ├── BRATS_002_0002.nii.gz
+    │   ├── BRATS_002_0003.nii.gz
+    │   ├── ...
+    ├── imagesTs
+    │   ├── BRATS_485_0000.nii.gz
+    │   ├── BRATS_485_0001.nii.gz
+    │   ├── BRATS_485_0002.nii.gz
+    │   ├── BRATS_485_0003.nii.gz
+    │   ├── BRATS_486_0000.nii.gz
+    │   ├── BRATS_486_0001.nii.gz
+    │   ├── BRATS_486_0002.nii.gz
+    │   ├── BRATS_486_0003.nii.gz
+    │   ├── ...
+    └── labelsTr
+        ├── BRATS_001.nii.gz
+        ├── BRATS_002.nii.gz
+        ├── ...
+
+Here is another example of the second dataset of the MSD, which has only one input channel:
+
+    nnUNet_raw/Dataset002_Heart/
+    ├── dataset.json
+    ├── imagesTr
+    │   ├── la_003_0000.nii.gz
+    │   ├── la_004_0000.nii.gz
+    │   ├── ...
+    ├── imagesTs
+    │   ├── la_001_0000.nii.gz
+    │   ├── la_002_0000.nii.gz
+    │   ├── ...
+    └── labelsTr
+        ├── la_003.nii.gz
+        ├── la_004.nii.gz
+        ├── ...
+
+Remember: For each training case, all images must have the same geometry to ensure that their pixel arrays are aligned. Also 
+make sure that all your data is co-registered!
+
+See also [dataset format inference](dataset_format_inference.md)!!
+
+## dataset.json
+The dataset.json contains metadata that nnU-Net needs for training. We have greatly reduced the number of required 
+fields since version 1!
+
+Here is what the dataset.json should look like at the example of the Dataset005_Prostate from the MSD:
+
+    { 
+     "channel_names": {  # formerly modalities
+       "0": "T2", 
+       "1": "ADC"
+     }, 
+     "labels": {  # THIS IS DIFFERENT NOW!
+       "background": 0,
+       "PZ": 1,
+       "TZ": 2
+     }, 
+     "numTraining": 32, 
+     "file_ending": ".nii.gz"
+     "overwrite_image_reader_writer": "SimpleITKIO"  # optional! If not provided nnU-Net will automatically determine the ReaderWriter
+     }
+
+The channel_names determine the normalization used by nnU-Net. If a channel is marked as 'CT', then a global 
+normalization based on the intensities in the foreground pixels will be used. If it is something else, per-channel 
+z-scoring will be used. Refer to the methods section in [our paper](https://www.nature.com/articles/s41592-020-01008-z) 
+for more details. nnU-Net v2 introduces a few more normalization schemes to 
+choose from and allows you to define your own, see [here](explanation_normalization.md) for more information. 
+
+Important changes relative to nnU-Net v1:
+- "modality" is now called "channel_names" to remove strong bias to medical images
+- labels are structured differently (name -> int instead of int -> name). This was needed to support [region-based training](region_based_training.md)
+- "file_ending" is added to support different input file types
+- "overwrite_image_reader_writer" optional! Can be used to specify a certain (custom) ReaderWriter class that should 
+be used with this dataset. If not provided, nnU-Net will automatically determine the ReaderWriter
+- "regions_class_order" only used in [region-based training](region_based_training.md)
+
+There is a utility with which you can generate the dataset.json automatically. You can find it 
+[here](../nnunetv2/dataset_conversion/generate_dataset_json.py). 
+See our examples in [dataset_conversion](../nnunetv2/dataset_conversion) for how to use it. And read its documentation!
+
+As described above, a json file that contains spacing information is required for TIFF files.
+An example for a 3D TIFF stack with units corresponding to 7.6 in x and y, 80 in z is:
+
+```
+{
+    "spacing": [7.6, 7.6, 80.0]
+}
+```
+
+Within the dataset folder, this file (named `cell6.json` in this example) would be placed in the following folders:
+
+    nnUNet_raw/Dataset123_Foo/
+    ├── dataset.json
+    ├── imagesTr
+    │   ├── cell6.json
+    │   └── cell6_0000.tif
+    └── labelsTr
+        ├── cell6.json
+        └── cell6.tif
+
+
+## How to use nnU-Net v1 Tasks
+If you are migrating from the old nnU-Net, convert your existing datasets with `nnUNetv2_convert_old_nnUNet_dataset`!
+
+Example for migrating a nnU-Net v1 Task:
+```bash
+nnUNetv2_convert_old_nnUNet_dataset /media/isensee/raw_data/nnUNet_raw_data_base/nnUNet_raw_data/Task027_ACDC Dataset027_ACDC 
+```
+Use `nnUNetv2_convert_old_nnUNet_dataset -h` for detailed usage instructions.
+
+
+## How to use decathlon datasets
+See [convert_msd_dataset.md](convert_msd_dataset.md)
+
+## How to use 2D data with nnU-Net
+2D is now natively supported (yay!). See [here](#supported-file-formats) as well as the example dataset in this 
+[script](../nnunetv2/dataset_conversion/Dataset120_RoadSegmentation.py).
+
+
+## How to update an existing dataset
+When updating a dataset it is best practice to remove the preprocessed data in `nnUNet_preprocessed/DatasetXXX_NAME` 
+to ensure a fresh start. Then replace the data in `nnUNet_raw` and rerun `nnUNetv2_plan_and_preprocess`. Optionally, 
+also remove the results from old trainings.
+
+# Example dataset conversion scripts
+In the `dataset_conversion` folder (see [here](../nnunetv2/dataset_conversion)) are multiple example scripts for 
+converting datasets into nnU-Net format. These scripts cannot be run as they are (you need to open them and change 
+some paths) but they are excellent examples for you to learn how to convert your own datasets into nnU-Net format. 
+Just pick the dataset that is closest to yours as a starting point.
+The list of dataset conversion scripts is continually updated. If you find that some publicly available dataset is 
+missing, feel free to open a PR to add it!

nnUNet/documentation/dataset_format_inference.md

@@ -0,0 +1,39 @@
+# Data format for Inference 
+Read the documentation on the overall [data format](dataset_format.md) first!
+
+The data format for inference must match the one used for the raw data (**specifically, the images must be in exactly 
+the same format as in the imagesTr folder**). As before, the filenames must start with a
+unique identifier, followed by a 4-digit modality identifier. Here is an example for two different datasets:
+
+1) Task005_Prostate:
+
+    This task has 2 modalities, so the files in the input folder must look like this:
+
+        input_folder
+        ├── prostate_03_0000.nii.gz
+        ├── prostate_03_0001.nii.gz
+        ├── prostate_05_0000.nii.gz
+        ├── prostate_05_0001.nii.gz
+        ├── prostate_08_0000.nii.gz
+        ├── prostate_08_0001.nii.gz
+        ├── ...
+
+    _0000 has to be the T2 image and _0001 has to be the ADC image (as specified by 'channel_names' in the 
+dataset.json), exactly the same as was used for training.
+
+2) Task002_Heart:
+
+        imagesTs
+        ├── la_001_0000.nii.gz
+        ├── la_002_0000.nii.gz
+        ├── la_006_0000.nii.gz
+        ├── ...
+    
+    Task002 only has one modality, so each case only has one _0000.nii.gz file.
+  
+
+The segmentations in the output folder will be named {CASE_IDENTIFIER}.nii.gz (omitting the modality identifier).
+
+Remember that the file format used for inference (.nii.gz in this example) must be the same as was used for training 
+(and as was specified in 'file_ending' in the dataset.json)!
+   

nnUNet/documentation/explanation_logging.md

@@ -0,0 +1,88 @@
+# Logging in nnU-Net v2
+
+## Introduction
+
+Logging in nnU-Net is intentionally simple and centralized in
+`nnunetv2/training/logging/nnunet_logger.py`.
+
+The trainer talks to one object, `MetaLogger`, and `MetaLogger` fans out logs to:
+
+- `LocalLogger` (always enabled): the source of truth for training curves, checkpoint logging state, and `progress.png`
+- optional external loggers (currently `WandbLogger`)
+
+This keeps training code clean while still allowing external tracking backends.
+
+## Default behaviour
+
+Without any setup, nnU-Net uses only `LocalLogger`.
+
+Per epoch, it stores:
+
+- `mean_fg_dice` and `ema_fg_dice` (EMA is computed automatically)
+- `dice_per_class_or_region`
+- `train_losses`, `val_losses`
+- `lrs`
+- `epoch_start_timestamps`, `epoch_end_timestamps`
+
+From these values, `progress.png` is updated in the fold output folder.
+On checkpoint save/load, the local logging state is also saved/restored, so curves continue correctly after resume.
+
+## How to enable W&B
+
+1. Install W&B:
+
+```bash
+pip install wandb
+```
+
+2. Enable the backend via environment variables:
+
+```bash
+export nnUNet_wandb_enabled=1
+export nnUNet_wandb_project=nnunet
+export nnUNet_wandb_mode=online   # or offline
+```
+
+3. Run training normally:
+
+```bash
+nnUNetv2_train DATASET_NAME_OR_ID 3d_fullres 0
+```
+
+Notes:
+
+- `nnUNet_wandb_enabled` accepts `0/1` and `false/true` (case-insensitive). Other values raise an error.
+- When resuming (`--c`), W&B resume metadata in `fold_x/wandb/latest-run` is reused and duplicate older steps are skipped.
+
+## How to integrate a custom logger
+
+Add a new logger class with the same minimal interface used by `MetaLogger`:
+
+- `update_config(self, config: dict)`
+- `log(self, key, value, step: int)`
+- `log_summary(self, key, value)`
+
+Example skeleton:
+
+```python
+class MyLogger:
+    def __init__(self, output_folder, resume):
+        self.output_folder = output_folder
+        self.resume = resume
+
+    def update_config(self, config: dict):
+        ...
+
+    def log(self, key, value, step: int):
+        ...
+
+    def log_summary(self, key, value):
+        ...
+```
+
+Then register it in `MetaLogger.__init__` (for example behind an env var switch), similar to how `WandbLogger` is added.
+
+Important integration detail:
+
+- `MetaLogger.log(...)` always writes to `LocalLogger` first.
+- If you introduce a brand-new per-epoch key, also add that key to `LocalLogger.my_fantastic_logging`, otherwise the local assertion will fail.

nnUNet/documentation/explanation_normalization.md

@@ -0,0 +1,45 @@
+# Intensity normalization in nnU-Net 
+
+The type of intensity normalization applied in nnU-Net can be controlled via the `channel_names` (former `modalities`)
+entry in the dataset.json. Just like the old nnU-Net, per-channel z-scoring as well as dataset-wide z-scoring based on 
+foreground intensities are supported. However, there have been a few additions as well.
+
+Reminder: The `channel_names` entry typically looks like this: 
+
+    "channel_names": {
+        "0": "T2",
+        "1": "ADC"
+    },
+
+It has as many entries as there are input channels for the given dataset.
+
+To tell you a secret, nnU-Net does not really care what your channels are called. We just use this to determine what normalization
+scheme will be used for the given dataset. nnU-Net requires you to specify a normalization strategy for each of your input channels! 
+If you enter a channel name that is not in the following list, the default (`zscore`) will be used.
+
+Here is a list of currently available normalization schemes:
+
+- `CT`: Perform CT normalization. Specifically, collect intensity values from the foreground classes (all but the 
+background and ignore) from all training cases, compute the mean, standard deviation as well as the 0.5 and 
+99.5 percentile of the values. Then clip to the percentiles, followed by subtraction of the mean and division with the 
+standard deviation. The normalization that is applied is the same for each training case (for this input channel).
+The values used by nnU-Net for normalization are stored in the `foreground_intensity_properties_per_channel` entry in the 
+corresponding plans file. This normalization is suitable for modalities presenting physical quantities such as CT 
+images and ADC maps.
+- `noNorm` : do not perform any normalization at all
+- `rescale_to_0_1`: rescale the intensities to [0, 1]
+- `rgb_to_0_1`: assumes uint8 inputs. Divides by 255 to rescale uint8 to [0, 1]
+- `zscore`/anything else: perform z-scoring (subtract mean and standard deviation) separately for each train case
+
+**Important:** The nnU-Net default is to perform 'CT' normalization for CT images and 'zscore' for everything else! If 
+you deviate from that path, make sure to benchmark whether that actually improves results! 
+
+# How to implement custom normalization strategies?
+- Head over to nnunetv2/preprocessing/normalization
+- implement a new image normalization class by deriving from ImageNormalization
+- register it in nnunetv2/preprocessing/normalization/map_channel_name_to_normalization.py:channel_name_to_normalization_mapping. 
+This is where you specify a channel name that should be associated with it
+- use it by specifying the correct channel_name
+
+Normalization can only be applied to one channel at a time. There is currently no way of implementing a normalization scheme 
+that gets multiple channels as input to be used jointly!

nnUNet/documentation/explanation_plans_files.md

@@ -0,0 +1,185 @@
+# Modifying the nnU-Net Configurations
+
+nnU-Net provides unprecedented out-of-the-box segmentation performance for essentially any dataset we have evaluated 
+it on. That said, there is always room for improvements. A fool-proof strategy for squeezing out the last bit of 
+performance is to start with the default nnU-Net, and then further tune it manually to a concrete dataset at hand.
+**This guide is about changes to the nnU-Net configuration you can make via the plans files. It does not cover code 
+extensions of nnU-Net. For that, take a look [here](extending_nnunet.md)**
+
+In nnU-Net V2, plans files are SO MUCH MORE powerful than they were in v1. There are a lot more knobs that you can 
+turn without resorting to hacky solutions or even having to touch the nnU-Net code at all! And as an added bonus: 
+plans files are now also .json files and no longer require users to fiddle with pickle. Just open them in your text 
+editor of choice!
+
+If overwhelmed, look at our [Examples](#examples)!
+
+# plans.json structure
+
+Plans have global and local settings. Global settings are applied to all configurations in that plans file while 
+local settings are attached to a specific configuration.
+
+## Global settings
+
+- `foreground_intensity_properties_by_modality`: Intensity statistics of the foreground regions (all labels except 
+background and ignore label), computed over all training cases. Used by [CT normalization scheme](explanation_normalization.md).
+- `image_reader_writer`: Name of the image reader/writer class that should be used with this dataset. You might want 
+to change this if, for example, you would like to run inference with files that have a different file format. The 
+class that is named here must be located in nnunetv2.imageio!
+- `label_manager`: The name of the class that does label handling. Take a look at 
+nnunetv2.utilities.label_handling.LabelManager to see what it does. If you decide to change it, place your version 
+in nnunetv2.utilities.label_handling!
+- `transpose_forward`: nnU-Net transposes the input data so that the axes with the highest resolution (lowest spacing) 
+come last. This is because the 2D U-Net operates on the trailing dimensions (more efficient slicing due to internal 
+memory layout of arrays). Future work might move this setting to affect only individual configurations. 
+- transpose_backward is what numpy.transpose gets as new axis ordering.
+- `transpose_backward`: the axis ordering that inverts "transpose_forward"
+- \[`original_median_shape_after_transp`\]: just here for your information
+- \[`original_median_spacing_after_transp`\]: just here for your information
+- \[`plans_name`\]: do not change. Used internally
+- \[`experiment_planner_used`\]: just here as metadata so that we know what planner originally generated this file
+- \[`dataset_name`\]: do not change. This is the dataset these plans are intended for
+
+## Local settings
+Plans also have a `configurations` key in which the actual configurations are stored. `configurations` are again a 
+dictionary, where the keys are the configuration names and the values are the local settings for each configuration.
+
+To better understand the components describing the network topology in our plans files, please read section 6.2 
+in the [supplementary information](https://static-content.springer.com/esm/art%3A10.1038%2Fs41592-020-01008-z/MediaObjects/41592_2020_1008_MOESM1_ESM.pdf) 
+(page 13) of our paper!
+
+Local settings:
+- `spacing`: the target spacing used in this configuration
+- `patch_size`: the patch size used for training this configuration
+- `data_identifier`: the preprocessed data for this configuration will be saved in
+  nnUNet_preprocessed/DATASET_NAME/_data_identifier_. If you add a new configuration, remember to set a unique
+  data_identifier in order to not create conflicts with other configurations (unless you plan to reuse the data from
+  another configuration, for example as is done in the cascade)
+- `batch_size`: batch size used for training
+- `batch_dice`: whether to use batch dice (pretend all samples in the batch are one image, compute dice loss over that)
+or not (each sample in the batch is a separate image, compute dice loss for each sample and average over samples)
+- `preprocessor_name`: Name of the preprocessor class used for running preprocessing. Class must be located in 
+nnunetv2.preprocessing.preprocessors
+- `use_mask_for_norm`: whether to use the nonzero mask for normalization or not (relevant for BraTS and the like, 
+probably False for all other datasets). Interacts with ImageNormalization class
+- `normalization_schemes`: mapping of channel identifier to ImageNormalization class name. ImageNormalization 
+classes must be located in nnunetv2.preprocessing.normalization. Also see [here](explanation_normalization.md)
+- `resampling_fn_data`: name of resampling function to be used for resizing image data. resampling function must be 
+callable(data, current_spacing, new_spacing, **kwargs). It must be located in nnunetv2.preprocessing.resampling
+- `resampling_fn_data_kwargs`: kwargs for resampling_fn_data
+- `resampling_fn_probabilities`: name of resampling function to be used for resizing predicted class probabilities/logits. 
+resampling function must be `callable(data: Union[np.ndarray, torch.Tensor], current_spacing, new_spacing, **kwargs)`. It must be located in 
+nnunetv2.preprocessing.resampling
+- `resampling_fn_probabilities_kwargs`: kwargs for resampling_fn_probabilities
+- `resampling_fn_seg`: name of resampling function to be used for resizing segmentation maps (integer: 0, 1, 2, 3, etc). 
+resampling function must be callable(data, current_spacing, new_spacing, **kwargs). It must be located in 
+nnunetv2.preprocessing.resampling
+- `resampling_fn_seg_kwargs`: kwargs for resampling_fn_seg
+- `network_arch_class_name`: UNet class name, can be used to integrate custom dynamic architectures
+- `UNet_base_num_features`: The number of starting features for the UNet architecture. Default is 32. Default: Features
+are doubled with each downsampling 
+- `unet_max_num_features`: Maximum number of features (default: capped at 320 for 3D and 512 for 2d). The purpose is to 
+prevent parameters from exploding too much. 
+- `conv_kernel_sizes`: the convolutional kernel sizes used by nnU-Net in each stage of the encoder. The decoder 
+  mirrors the encoder and is therefore not explicitly listed here! The list is as long as `n_conv_per_stage_encoder` has 
+  entries
+- `n_conv_per_stage_encoder`: number of convolutions used per stage (=at a feature map resolution in the encoder) in the encoder. 
+  Default is 2. The list has as many entries as the encoder has stages
+- `n_conv_per_stage_decoder`: number of convolutions used per stage in the decoder. Also see `n_conv_per_stage_encoder`
+- `num_pool_per_axis`: number of times each of the spatial axes is pooled in the network. Needed to know how to pad 
+  image sizes during inference (num_pool = 5 means input must be divisible by 2**5=32)
+- `pool_op_kernel_sizes`: the pooling kernel sizes (and at the same time strides) for each stage of the encoder
+- \[`median_image_size_in_voxels`\]: the median size of the images of the training set at the current target spacing. 
+Do not modify this as this is not used. It is just here for your information.
+
+Special local settings:
+- `inherits_from`: configurations can inherit from each other. This makes it easy to add new configurations that only
+differ in a few local settings from another. If using this, remember to set a new `data_identifier` (if needed)!
+- `previous_stage`: if this configuration is part of a cascade, we need to know what the previous stage (for example 
+the low resolution configuration) was. This needs to be specified here.
+- `next_stage`: if this configuration is part of a cascade, we need to know what possible subsequent stages are! This 
+is because we need to export predictions in the correct spacing when running the validation. `next_stage` can either 
+be a string or a list of strings
+
+# Examples
+
+## Increasing the batch size for large datasets
+If your dataset is large the training can benefit from larger batch_sizes. To do this, simply create a new 
+configuration in the `configurations` dict
+
+    "configurations": {
+      "3d_fullres_bs40": {
+        "inherits_from": "3d_fullres",
+        "batch_size": 40
+      }
+    }
+
+No need to change the data_identifier. `3d_fullres_bs40` will just use the preprocessed data from `3d_fullres`.
+No need to rerun `nnUNetv2_preprocess` because we can use already existing data (if available) from `3d_fullres`.
+
+## Using custom preprocessors
+If you would like to use a different preprocessor class then this can be specified as follows:
+
+    "configurations": {
+      "3d_fullres_my_preprocesor": {
+        "inherits_from": "3d_fullres",
+        "preprocessor_name": MY_PREPROCESSOR,
+        "data_identifier": "3d_fullres_my_preprocesor"
+      }
+    }
+
+You need to run preprocessing for this new configuration: 
+`nnUNetv2_preprocess -d DATASET_ID -c 3d_fullres_my_preprocesor` because it changes the preprocessing. Remember to 
+set a unique `data_identifier` whenever you make modifications to the preprocessed data!
+
+## Change target spacing
+
+    "configurations": {
+      "3d_fullres_my_spacing": {
+        "inherits_from": "3d_fullres",
+        "spacing": [X, Y, Z],
+        "data_identifier": "3d_fullres_my_spacing"
+      }
+    }
+
+You need to run preprocessing for this new configuration: 
+`nnUNetv2_preprocess -d DATASET_ID -c 3d_fullres_my_spacing` because it changes the preprocessing. Remember to 
+set a unique `data_identifier` whenever you make modifications to the preprocessed data!
+
+## Adding a cascade to a dataset where it does not exist
+Hippocampus is small. It doesn't have a cascade. It also doesn't really make sense to add a cascade here but hey for 
+the sake of demonstration we can do that.
+We change the following things here:
+
+- `spacing`: The lowres stage should operate at a lower resolution
+- we modify the `median_image_size_in_voxels` entry as a guide for what original image sizes we deal with
+- we set some patch size that is inspired by `median_image_size_in_voxels`
+- we need to remember that the patch size must be divisible by 2**num_pool in each axis!
+- network parameters such as kernel sizes, pooling operations are changed accordingly
+- we need to specify the name of the next stage
+- we need to add the highres stage
+
+This is how this would look like (comparisons with 3d_fullres given as reference):
+
+    "configurations": {
+      "3d_lowres": {
+        "inherits_from": "3d_fullres",
+        "data_identifier": "3d_lowres"
+        "spacing": [2.0, 2.0, 2.0], # from [1.0, 1.0, 1.0] in 3d_fullres
+        "median_image_size_in_voxels": [18, 25, 18], # from [36, 50, 35]
+        "patch_size": [20, 28, 20], # from [40, 56, 40]
+        "n_conv_per_stage_encoder": [2, 2, 2], # one less entry than 3d_fullres ([2, 2, 2, 2])
+        "n_conv_per_stage_decoder": [2, 2], # one less entry than 3d_fullres
+        "num_pool_per_axis": [2, 2, 2], # one less pooling than 3d_fullres in each dimension (3d_fullres: [3, 3, 3])
+        "pool_op_kernel_sizes": [[1, 1, 1], [2, 2, 2], [2, 2, 2]], # one less [2, 2, 2]
+        "conv_kernel_sizes": [[3, 3, 3], [3, 3, 3], [3, 3, 3]], # one less [3, 3, 3]
+        "next_stage": "3d_cascade_fullres" # name of the next stage in the cascade
+      },
+      "3d_cascade_fullres": { # does not need a data_identifier because we can use the data of 3d_fullres
+        "inherits_from": "3d_fullres",
+        "previous_stage": "3d_lowres" # name of the previous stage
+      }
+    }
+
+To better understand the components describing the network topology in our plans files, please read section 6.2 
+in the [supplementary information](https://static-content.springer.com/esm/art%3A10.1038%2Fs41592-020-01008-z/MediaObjects/41592_2020_1008_MOESM1_ESM.pdf) 
+(page 13) of our paper!

nnUNet/documentation/extending_nnunet.md

@@ -0,0 +1,37 @@
+# Extending nnU-Net
+We hope that the new structure of nnU-Net v2 makes it much more intuitive on how to modify it! We cannot give an 
+extensive tutorial on how each and every bit of it can be modified. It is better for you to search for the position 
+in the repository where the thing you intend to change is implemented and start working your way through the code from 
+there. Setting breakpoints and debugging into nnU-Net really helps in understanding it and thus will help you make the 
+necessary modifications!
+
+Here are some things you might want to read before you start:
+- Editing nnU-Net configurations through plans files is really powerful now and allows you to change a lot of things regarding 
+preprocessing, resampling, network topology etc. Read [this](explanation_plans_files.md)!
+- [Image normalization](explanation_normalization.md) and [i/o formats](dataset_format.md#supported-file-formats) are easy to extend!
+- Manual data splits can be defined as described [here](manual_data_splits.md)
+- You can chain arbitrary configurations together into cascades, see [this again](explanation_plans_files.md)
+- Read about our support for [region-based training](region_based_training.md)
+- If you intend to modify the training procedure (loss, sampling, data augmentation, lr scheduler, etc) then you need 
+to implement your own trainer class. Best practice is to create a class that inherits from nnUNetTrainer and 
+implements the necessary changes. Head over to our [trainer classes folder](../nnunetv2/training/nnUNetTrainer) for 
+inspiration! There will be similar trainers for what you intend to change and you can take them as a guide. nnUNetTrainer 
+are structured similarly to PyTorch lightning trainers, this should also make things easier!
+- Integrating new network architectures can be done in two ways:
+  - Quick and dirty: implement a new nnUNetTrainer class and overwrite its `build_network_architecture` function. 
+  Make sure your architecture is compatible with deep supervision (if not, use `nnUNetTrainerNoDeepSupervision`
+  as basis!) and that it can handle the patch sizes that are thrown at it! Your architecture should NOT apply any 
+  nonlinearities at the end (softmax, sigmoid etc). nnU-Net does that!   
+  - The 'proper' (but difficult) way: Build a dynamically configurable architecture such as the `PlainConvUNet` class 
+  used by default. It needs to have some sort of GPU memory estimation method that can be used to evaluate whether 
+  certain patch sizes and 
+  topologies fit into a specified GPU memory target. Build a new `ExperimentPlanner` that can configure your new 
+  class and communicate with its memory budget estimation. Run `nnUNetv2_plan_and_preprocess` while specifying your 
+  custom `ExperimentPlanner` and a custom `plans_name`. Implement a nnUNetTrainer that can use the plans generated by 
+  your `ExperimentPlanner` to instantiate the network architecture. Specify your plans and trainer when running `nnUNetv2_train`. 
+  It always pays off to first read and understand the corresponding nnU-Net code and use it as a template for your implementation!
+- Remember that multi-GPU training, region-based training, ignore label and cascaded training are now simply integrated 
+into one unified nnUNetTrainer class. No separate classes needed (remember that when implementing your own trainer 
+classes and ensure support for all of these features! Or raise `NotImplementedError`)
+
+[//]: # (- Read about our support for [ignore label](ignore_label.md) and [region-based training](region_based_training.md))

nnUNet/documentation/how_to_use_nnunet.md

@@ -0,0 +1,313 @@
+## **2024-04-18 UPDATE: New residual encoder UNet presets available!**
+The recommended nnU-Net presets have changed! See [here](resenc_presets.md) how to unlock them!
+
+
+## How to run nnU-Net on a new dataset
+
+
+Given some dataset, nnU-Net fully automatically configures an entire segmentation pipeline that matches its properties.
+nnU-Net covers the entire pipeline, from preprocessing to model configuration, model training, postprocessing
+all the way to ensembling. After running nnU-Net, the trained model(s) can be applied to the test cases for inference.
+
+### Dataset Format
+nnU-Net expects datasets in a structured format. This format is inspired by the data structure of
+the [Medical Segmentation Decthlon](http://medicaldecathlon.com/). Please read
+[this](dataset_format.md) for information on how to set up datasets to be compatible with nnU-Net.
+
+**Since version 2 we support multiple image file formats (.nii.gz, .png, .tif, ...)! Read the dataset_format 
+documentation to learn more!**
+
+**Datasets from nnU-Net v1 can be converted to V2 by running `nnUNetv2_convert_old_nnUNet_dataset INPUT_FOLDER 
+OUTPUT_DATASET_NAME`.** Remember that v2 calls datasets DatasetXXX_Name (not Task) where XXX is a 3-digit number.
+Please provide the **path** to the old task, not just the Task name. nnU-Net V2 doesn't know where v1 tasks were!
+
+### Experiment planning and preprocessing
+Given a new dataset, nnU-Net will extract a dataset fingerprint (a set of dataset-specific properties such as
+image sizes, voxel spacings, intensity information etc). This information is used to design three U-Net configurations. 
+Each of these pipelines operates on its own preprocessed version of the dataset.
+
+The easiest way to run fingerprint extraction, experiment planning and preprocessing is to use:
+
+```bash
+nnUNetv2_plan_and_preprocess -d DATASET_ID --verify_dataset_integrity
+```
+
+Where `DATASET_ID` is the dataset id (duh). We recommend `--verify_dataset_integrity` whenever it's the first time 
+you run this command. This will check for some of the most common error sources! Fingerprint extraction and
+preprocessing show a progress bar by default. Use `--no_pbar` if you want to suppress it, for example in
+non-interactive cluster environments.
+
+You can also process several datasets at once by giving `-d 1 2 3 [...]`. If you already know what U-Net configuration 
+you need you can also specify that with `-c 3d_fullres` (make sure to adapt -np in this case!). For more information 
+about all the options available to you please run `nnUNetv2_plan_and_preprocess -h`.
+
+nnUNetv2_plan_and_preprocess will create a new subfolder in your nnUNet_preprocessed folder named after the dataset. 
+Once the command is completed there will be a dataset_fingerprint.json file as well as a nnUNetPlans.json file for you to look at 
+(in case you are interested!). There will also be subfolders containing the preprocessed data for your UNet configurations.
+
+[Optional]
+If you prefer to keep things separate, you can also use `nnUNetv2_extract_fingerprint`, `nnUNetv2_plan_experiment` 
+and `nnUNetv2_preprocess` (in that order). `nnUNetv2_extract_fingerprint` and `nnUNetv2_preprocess` also support
+`--no_pbar`.
+
+### Model training
+#### Overview
+You pick which configurations (2d, 3d_fullres, 3d_lowres, 3d_cascade_fullres) should be trained! If you have no idea 
+what performs best on your data, just run all of them and let nnU-Net identify the best one. It's up to you!
+
+nnU-Net trains all configurations in a 5-fold cross-validation over the training cases. This is 1) needed so that 
+nnU-Net can estimate the performance of each configuration and tell you which one should be used for your 
+segmentation problem and 2) a natural way of obtaining a good model ensemble (average the output of these 5 models 
+for prediction) to boost performance.
+
+You can influence the splits nnU-Net uses for 5-fold cross-validation (see [here](manual_data_splits.md)). If you 
+prefer to train a single model on all training cases, this is also possible (see below).
+
+**Note that not all U-Net configurations are created for all datasets. In datasets with small image sizes, the U-Net
+cascade (and with it the 3d_lowres configuration) is omitted because the patch size of the full resolution U-Net 
+already covers a large part of the input images.**
+
+Training models is done with the `nnUNetv2_train` command. The general structure of the command is:
+```bash
+nnUNetv2_train DATASET_NAME_OR_ID UNET_CONFIGURATION FOLD [additional options, see -h]
+```
+
+UNET_CONFIGURATION is a string that identifies the requested U-Net configuration (defaults: 2d, 3d_fullres, 3d_lowres, 
+3d_cascade_lowres). DATASET_NAME_OR_ID specifies what dataset should be trained on and FOLD specifies which fold of 
+the 5-fold-cross-validation is trained.
+
+nnU-Net stores a checkpoint every 50 epochs. If you need to continue a previous training, just add a `--c` to the
+training command.
+
+IMPORTANT: If you plan to use `nnUNetv2_find_best_configuration` (see below) add the `--npz` flag. This makes 
+nnU-Net save the softmax outputs during the final validation. They are needed for that. Exported softmax
+predictions are very large and therefore can take up a lot of disk space, which is why this is not enabled by default.
+If you ran initially without the `--npz` flag but now require the softmax predictions, simply rerun the validation with:
+```bash
+nnUNetv2_train DATASET_NAME_OR_ID UNET_CONFIGURATION FOLD --val --npz
+```
+
+You can specify the device nnU-net should use by using `-device DEVICE`. DEVICE can only be cpu, cuda or mps. If 
+you have multiple GPUs, please select the gpu id using `CUDA_VISIBLE_DEVICES=X nnUNetv2_train [...]` (requires device to be cuda).
+
+See `nnUNetv2_train -h` for additional options.
+
+### 2D U-Net
+For FOLD in [0, 1, 2, 3, 4], run:
+```bash
+nnUNetv2_train DATASET_NAME_OR_ID 2d FOLD [--npz]
+```
+
+### 3D full resolution U-Net
+For FOLD in [0, 1, 2, 3, 4], run:
+```bash
+nnUNetv2_train DATASET_NAME_OR_ID 3d_fullres FOLD [--npz]
+```
+
+### 3D U-Net cascade
+#### 3D low resolution U-Net
+For FOLD in [0, 1, 2, 3, 4], run:
+```bash
+nnUNetv2_train DATASET_NAME_OR_ID 3d_lowres FOLD [--npz]
+```
+
+#### 3D full resolution U-Net
+For FOLD in [0, 1, 2, 3, 4], run:
+```bash
+nnUNetv2_train DATASET_NAME_OR_ID 3d_cascade_fullres FOLD [--npz]
+```
+**Note that the 3D full resolution U-Net of the cascade requires the five folds of the low resolution U-Net to be
+completed!**
+
+The trained models will be written to the nnUNet_results folder. Each training obtains an automatically generated
+output folder name:
+
+nnUNet_results/DatasetXXX_MYNAME/TRAINER_CLASS_NAME__PLANS_NAME__CONFIGURATION/FOLD
+
+For Dataset002_Heart (from the MSD), for example, this looks like this:
+
+    nnUNet_results/
+    ├── Dataset002_Heart
+        │── nnUNetTrainer__nnUNetPlans__2d
+        │    ├── fold_0
+        │    ├── fold_1
+        │    ├── fold_2
+        │    ├── fold_3
+        │    ├── fold_4
+        │    ├── dataset.json
+        │    ├── dataset_fingerprint.json
+        │    └── plans.json
+        └── nnUNetTrainer__nnUNetPlans__3d_fullres
+             ├── fold_0
+             ├── fold_1
+             ├── fold_2
+             ├── fold_3
+             ├── fold_4
+             ├── dataset.json
+             ├── dataset_fingerprint.json
+             └── plans.json
+
+Note that 3d_lowres and 3d_cascade_fullres do not exist here because this dataset did not trigger the cascade. In each
+model training output folder (each of the fold_x folder), the following files will be created:
+- debug.json: Contains a summary of blueprint and inferred parameters used for training this model as well as a 
+bunch of additional stuff. Not easy to read, but very useful for debugging ;-)
+- checkpoint_best.pth: checkpoint files of the best model identified during training. Not used right now unless you 
+explicitly tell nnU-Net to use it.
+- checkpoint_final.pth: checkpoint file of the final model (after training has ended). This is what is used for both 
+validation and inference.
+- network_architecture.pdf (only if hiddenlayer is installed!): a pdf document with a figure of the network architecture in it.
+- progress.png: Shows losses, pseudo dice, learning rate and epoch times ofer the course of the training. At the top is 
+a plot of the training (blue) and validation (red) loss during training. Also shows an approximation of
+  the dice (green) as well as a moving average of it (dotted green line). This approximation is the average Dice score 
+  of the foreground classes. **It needs to be taken with a big (!) 
+  grain of salt** because it is computed on randomly drawn patches from the validation
+  data at the end of each epoch, and the aggregation of TP, FP and FN for the Dice computation treats the patches as if
+  they all originate from the same volume ('global Dice'; we do not compute a Dice for each validation case and then
+  average over all cases but pretend that there is only one validation case from which we sample patches). The reason for
+  this is that the 'global Dice' is easy to compute during training and is still quite useful to evaluate whether a model
+  is training at all or not. A proper validation takes way too long to be done each epoch. It is run at the end of the training.
+- validation: in this folder are the predicted validation cases after the training has finished. The summary.json file in here
+  contains the validation metrics (a mean over all cases is provided at the start of the file). If `--npz` was set then 
+the compressed softmax outputs (saved as .npz files) are in here as well. 
+
+During training it is often useful to watch the progress. We therefore recommend that you have a look at the generated
+progress.png when running the first training. It will be updated after each epoch.
+
+Training times largely depend on the GPU. The smallest GPU we recommend for training is the Nvidia RTX 2080ti. With 
+that all network trainings take less than 2 days. Refer to our [benchmarks](benchmarking.md) to see if your system is 
+performing as expected.
+
+### Using multiple GPUs for training
+
+If multiple GPUs are at your disposal, the best way of using them is to train multiple nnU-Net trainings at once, one 
+on each GPU. This is because data parallelism never scales perfectly linearly, especially not with small networks such 
+as the ones used by nnU-Net.
+
+Example:
+
+```bash
+CUDA_VISIBLE_DEVICES=0 nnUNetv2_train DATASET_NAME_OR_ID 2d 0 [--npz] & # train on GPU 0
+CUDA_VISIBLE_DEVICES=1 nnUNetv2_train DATASET_NAME_OR_ID 2d 1 [--npz] & # train on GPU 1
+CUDA_VISIBLE_DEVICES=2 nnUNetv2_train DATASET_NAME_OR_ID 2d 2 [--npz] & # train on GPU 2
+CUDA_VISIBLE_DEVICES=3 nnUNetv2_train DATASET_NAME_OR_ID 2d 3 [--npz] & # train on GPU 3
+CUDA_VISIBLE_DEVICES=4 nnUNetv2_train DATASET_NAME_OR_ID 2d 4 [--npz] & # train on GPU 4
+...
+wait
+```
+
+**Important: The first time a training is run nnU-Net will extract the preprocessed data into uncompressed numpy 
+arrays for speed reasons! This operation must be completed before starting more than one training of the same 
+configuration! Wait with starting subsequent folds until the first training is using the GPU! Depending on the 
+dataset size and your System this should only take a couple of minutes at most.**
+
+If you insist on running DDP multi-GPU training, we got you covered:
+
+`nnUNetv2_train DATASET_NAME_OR_ID 2d 0 [--npz] -num_gpus X`
+
+Again, note that this will be slower than running separate training on separate GPUs. DDP only makes sense if you have 
+manually interfered with the nnU-Net configuration and are training larger models with larger patch and/or batch sizes!
+
+Important when using `-num_gpus`:
+1) If you train using, say, 2 GPUs but have more GPUs in the system you need to specify which GPUs should be used via 
+CUDA_VISIBLE_DEVICES=0,1 (or whatever your ids are).
+2) You cannot specify more GPUs than you have samples in your minibatches. If the batch size is 2, 2 GPUs is the maximum!
+3) Make sure your batch size is divisible by the numbers of GPUs you use or you will not make good use of your hardware.
+
+In contrast to the old nnU-Net, DDP is now completely hassle free. Enjoy!
+
+### Automatically determine the best configuration
+Once the desired configurations were trained (full cross-validation) you can tell nnU-Net to automatically identify 
+the best combination for you:
+
+```commandline
+nnUNetv2_find_best_configuration DATASET_NAME_OR_ID -c CONFIGURATIONS 
+```
+
+`CONFIGURATIONS` hereby is the list of configurations you would like to explore. Per default, ensembling is enabled 
+meaning that nnU-Net will generate all possible combinations of ensembles (2 configurations per ensemble). This requires 
+the .npz files containing the predicted probabilities of the validation set to be present (use `nnUNetv2_train` with 
+`--npz` flag, see above). You can disable ensembling by setting the `--disable_ensembling` flag.
+
+See `nnUNetv2_find_best_configuration -h` for more options.
+
+nnUNetv2_find_best_configuration will also automatically determine the postprocessing that should be used. 
+Postprocessing in nnU-Net only considers the removal of all but the largest component in the prediction (once for 
+foreground vs background and once for each label/region).
+
+Once completed, the command will print to your console exactly what commands you need to run to make predictions. It 
+will also create two files in the `nnUNet_results/DATASET_NAME` folder for you to inspect: 
+- `inference_instructions.txt` again contains the exact commands you need to use for predictions
+- `inference_information.json` can be inspected to see the performance of all configurations and ensembles, as well 
+as the effect of the postprocessing plus some debug information. 
+
+### Run inference
+Remember that the data located in the input folder must have the file endings as the dataset you trained the model on 
+and must adhere to the nnU-Net naming scheme for image files (see [dataset format](dataset_format.md) and 
+[inference data format](dataset_format_inference.md)!)
+
+`nnUNetv2_find_best_configuration` (see above) will print a string to the terminal with the inference commands you need to use.
+The easiest way to run inference is to simply use these commands.
+
+If you wish to manually specify the configuration(s) used for inference, use the following commands:
+
+#### Run prediction
+For each of the desired configurations, run:
+```
+nnUNetv2_predict -i INPUT_FOLDER -o OUTPUT_FOLDER -d DATASET_NAME_OR_ID -c CONFIGURATION --save_probabilities
+```
+
+Only specify `--save_probabilities` if you intend to use ensembling. `--save_probabilities` will make the command save the predicted
+probabilities alongside of the predicted segmentation masks requiring a lot of disk space.
+
+Please select a separate `OUTPUT_FOLDER` for each configuration!
+
+Note that per default, inference will be done with all 5 folds from the cross-validation as an ensemble. We very 
+strongly recommend you use all 5 folds. Thus, all 5 folds must have been trained prior to running inference. 
+
+If you wish to make predictions with a single model, train the `all` fold and specify it in `nnUNetv2_predict`
+with `-f all`
+
+#### Ensembling multiple configurations
+If you wish to ensemble multiple predictions (typically form different configurations), you can do so with the following command:
+```bash
+nnUNetv2_ensemble -i FOLDER1 FOLDER2 ... -o OUTPUT_FOLDER -np NUM_PROCESSES
+```
+
+You can specify an arbitrary number of folders, but remember that each folder needs to contain npz files that were
+generated by `nnUNetv2_predict`. Again, `nnUNetv2_ensemble -h` will tell you more about additional options.
+
+#### Apply postprocessing
+Finally, apply the previously determined postprocessing to the (ensembled) predictions: 
+
+```commandline
+nnUNetv2_apply_postprocessing -i FOLDER_WITH_PREDICTIONS -o OUTPUT_FOLDER --pp_pkl_file POSTPROCESSING_FILE -plans_json PLANS_FILE -dataset_json DATASET_JSON_FILE
+```
+
+`nnUNetv2_find_best_configuration` (or its generated `inference_instructions.txt` file) will tell you where to find 
+the postprocessing file. If not you can just look for it in your results folder (it's creatively named 
+`postprocessing.pkl`). If your source folder is from an ensemble, you also need to specify a `-plans_json` file and 
+a `-dataset_json` file that should be used (for single configuration predictions these are automatically copied 
+from the respective training). You can pick these files from any of the ensemble members.
+
+
+## How to run inference with pretrained models
+See [here](run_inference_with_pretrained_models.md)
+
+## How to Deploy and Run Inference with YOUR Pretrained Models
+To facilitate the use of pretrained models on a different computer for inference purposes, follow these streamlined steps:
+1. Exporting the Model: Utilize the `nnUNetv2_export_model_to_zip` function to package your trained model into a .zip file. This file will contain all necessary model files.
+2. Transferring the Model: Transfer the .zip file to the target computer where inference will be performed.
+3. Importing the Model: On the new PC, use the `nnUNetv2_install_pretrained_model_from_zip` to load the pretrained model from the .zip file.
+Please note that both computers must have nnU-Net installed along with all its dependencies to ensure compatibility and functionality of the model.
+
+[//]: # (## Examples)
+
+[//]: # ()
+[//]: # (To get you started we compiled two simple to follow examples:)
+
+[//]: # (- run a training with the 3d full resolution U-Net on the Hippocampus dataset. See [here](documentation/training_example_Hippocampus.md).)
+
+[//]: # (- run inference with nnU-Net's pretrained models on the Prostate dataset. See [here](documentation/inference_example_Prostate.md).)
+
+[//]: # ()
+[//]: # (Usability not good enough? Let us know!)

nnUNet/documentation/ignore_label.md

@@ -0,0 +1,104 @@
+# Ignore Label
+
+The _ignore label_ can be used to mark regions that should be ignored by nnU-Net. This can be used to 
+learn from images where only sparse annotations are available, for example in the form of scribbles or a limited 
+amount of annotated slices. Internally, this is accomplished by using partial losses, i.e. losses that are only 
+computed on annotated pixels while ignoring the rest. Take a look at our 
+[`DC_and_BCE_loss` loss](../nnunetv2/training/loss/compound_losses.py) to see how this is done.
+During inference (validation and prediction), nnU-Net will always predict dense segmentations. Metric computation in 
+validation is of course only done on annotated pixels.
+
+Using sparse annotations can be used to train a model for application to new, unseen images or to autocomplete the 
+provided training cases given the sparse labels. 
+
+(See our [paper](https://arxiv.org/abs/2403.12834) for more information)
+
+Typical use-cases for the ignore label are:
+- Save annotation time through sparse annotation schemes
+  - Annotation of all or a subset of slices with scribbles (Scribble Supervision)
+  - Dense annotation of a subset of slices 
+  - Dense annotation of chosen patches/cubes within an image
+- Coarsly masking out faulty segmentations in the reference segmentations
+- Masking areas for other reasons
+
+If you are using nnU-Net's ignore label, please cite the following paper in addition to the original nnU-net paper:
+
+```
+Gotkowski, K., Lüth, C., Jäger, P. F., Ziegler, S., Krämer, L., Denner, S., Xiao, S., Disch, N., H., K., & Isensee, F. 
+(2024). Embarrassingly Simple Scribble Supervision for 3D Medical Segmentation. ArXiv. /abs/2403.12834
+```
+
+## Usecases
+
+### Scribble Supervision
+
+Scribbles are free-form drawings to coarsly annotate an image. As we have demonstrated in our recent [paper](https://arxiv.org/abs/2403.12834), nnU-Net's partial loss implementation enables state-of-the-art learning from partially annotated data and even surpasses many purpose-built methods for learning from scribbles. As a starting point, for each image slice and each class (including background), an interior and a border scribble should be generated:
+
+- Interior Scribble: A scribble placed randomly within the class interior of a class instance
+- Border Scribble: A scribble roughly delineating a small part of the class border of a class instance
+
+An example of such scribble annotations is depicted in Figure 1 and an animation in Animation 1.
+Depending on the availability of data and their variability it is also possible to only annotated a subset of selected slices.
+
+<p align="center">
+    <img src="assets/scribble_example.png" width="1024px" />
+    <figcaption>Figure 1: Examples of segmentation types with (A) depicting a dense segmentation and (B) a scribble segmentation.</figcaption>
+</figure>
+</p>
+
+<p align="center">
+    <img width="512px" src="https://media.giphy.com/media/v1.Y2lkPTc5MGI3NjExbmdndHQwMG96M3FqZWtwbHR2enUwZXhwNHVsbndzNmNpZnVlbHJ6OSZlcD12MV9pbnRlcm5hbF9naWZfYnlfaWQmY3Q9Zw/KRJ48evmroDlIgcqcO/giphy.gif">
+    <img width="512px" src="https://media.giphy.com/media/v1.Y2lkPTc5MGI3NjExem10Z3ZqZHQ2MWNsMjdibG1zc3M2NzNqbG9mazdudG5raTk4d3h4MSZlcD12MV9pbnRlcm5hbF9naWZfYnlfaWQmY3Q9Zw/ifVxQQfco5ro1gH6bQ/giphy.gif">
+    <figcaption>Animation 1: Depiction of a dense segmentation and a scribble annotation. Background scribbles have been excluded for better visualization.</figcaption>
+</p>
+
+### Dense annotation of a subset of slices
+
+Another form of sparse annotation is the dense annotation of a subset of slices. These slices should be selected by the user either randomly, based on visual class variation between slices or in an active learning setting. An example with only 10% of slices annotated is depicted in Figure 2.
+
+<p align="center">
+    <img src="assets/amos2022_sparseseg10_2d.png" width="512px" />
+    <img src="assets/amos2022_sparseseg10.png" width="512px" />
+    <figcaption>Figure 2: Examples of a dense annotation of a subset of slices. The ignored areas are shown in red.</figcaption>
+</figure>
+</p>
+
+
+## Usage within nnU-Net
+
+Usage of the ignore label in nnU-Net is straightforward and only requires the definition of an _ignore_ label in the _dataset.json_.
+This ignore label MUST be the highest integer label value in the segmentation. Exemplary, given the classes background and two foreground classes, then the ignore label must have the integer 3. The ignore label must be named _ignore_ in the _dataset.json_. Given the BraTS dataset as an example the labels dict of the _dataset.json_ must look like this:
+
+```python
+...
+"labels": {
+    "background": 0,
+    "edema": 1,
+    "non_enhancing_and_necrosis": 2,
+    "enhancing_tumor": 3,
+    "ignore": 4
+},
+...
+```
+
+Of course, the ignore label is compatible with [region-based training](region_based_training.md):
+
+```python
+...
+"labels": {
+    "background": 0,
+    "whole_tumor": (1, 2, 3),
+    "tumor_core": (2, 3),
+    "enhancing_tumor": 3,  # or (3, )
+    "ignore": 4
+},
+"regions_class_order": (1, 2, 3),  # don't declare ignore label here! It is not predicted
+...
+```
+
+Then use the dataset as you would any other.
+
+Remember that nnU-Net runs a cross-validation. Thus, it will also evaluate on your partially annotated data. This 
+will of course work! If you wish to compare different sparse annotation strategies (through simulations for example),
+we recommend evaluating on densely annotated images by running inference and then using `nnUNetv2_evaluate_folder` or 
+`nnUNetv2_evaluate_simple`.

nnUNet/documentation/installation_instructions.md

@@ -0,0 +1,87 @@
+# System requirements
+
+## Operating System
+nnU-Net has been tested on Linux (Ubuntu 18.04, 20.04, 22.04; centOS, RHEL), Windows and MacOS! It should work out of the box!
+
+## Hardware requirements
+We support GPU (recommended), CPU and Apple M1/M2 as devices (currently Apple mps does not implement 3D 
+convolutions, so you might have to use the CPU on those devices).
+
+### Hardware requirements for Training
+We recommend you use a GPU for training as this will take a really long time on CPU or MPS (Apple M1/M2). 
+For training a GPU with at least 10 GB (popular non-datacenter options are the RTX 2080ti, RTX 3080/3090 or RTX 4080/4090) is 
+required. We also recommend a strong CPU to go along with the GPU. 6 cores (12 threads) 
+are the bare minimum! CPU requirements are mostly related to data augmentation and scale with the number of 
+input channels and target structures. Plus, the faster the GPU, the better the CPU should be!
+
+### Hardware Requirements for inference
+Again we recommend a GPU to make predictions as this will be substantially faster than the other options. However, 
+inference times are typically still manageable on CPU and MPS (Apple M1/M2). If using a GPU, it should have at least 
+4 GB of available (unused) VRAM.
+
+### Example hardware configurations
+Example workstation configurations for training:
+- CPU: Ryzen 5800X - 5900X or 7900X would be even better! We have not yet tested Intel Alder/Raptor lake but they will likely work as well.
+- GPU: RTX 3090 or RTX 4090
+- RAM: 64GB
+- Storage: SSD (M.2 PCIe Gen 3 or better!)
+
+Example Server configuration for training:
+- CPU: 2x AMD EPYC7763 for a total of 128C/256T. 16C/GPU are highly recommended for fast GPUs such as the A100!
+- GPU: 8xA100 PCIe (price/performance superior to SXM variant + they use less power)
+- RAM: 1 TB
+- Storage: local SSD storage (PCIe Gen 3 or better) or ultra fast network storage
+
+(nnU-net by default uses one GPU per training. The server configuration can run up to 8 model trainings simultaneously)
+
+### Setting the correct number of Workers for data augmentation (training only)
+Note that you will need to manually set the number of processes nnU-Net uses for data augmentation according to your 
+CPU/GPU ratio. For the server above (256 threads for 8 GPUs), a good value would be 24-30. You can do this by 
+setting the `nnUNet_n_proc_DA` environment variable (`export nnUNet_n_proc_DA=XX`). 
+Recommended values (assuming a recent CPU with good IPC) are 10-12 for RTX 2080 ti, 12 for a RTX 3090, 16-18 for 
+RTX 4090, 28-32 for A100. Optimal values may vary depending on the number of input channels/modalities and number of classes.
+
+# Installation instructions
+We strongly recommend that you install nnU-Net in a virtual environment! Pip or anaconda are both fine. If you choose to 
+compile PyTorch from source (see below), you will need to use conda instead of pip. 
+
+Use a recent version of Python! 3.9 or newer is guaranteed to work!
+
+**nnU-Net v2 can coexist with nnU-Net v1! Both can be installed at the same time.**
+
+1) Install [PyTorch](https://pytorch.org/get-started/locally/) as described on their website (conda/pip). Please 
+install the latest version with support for your hardware (cuda, mps, cpu).
+**DO NOT JUST `pip install nnunetv2` WITHOUT PROPERLY INSTALLING PYTORCH FIRST**. For maximum speed, consider 
+[compiling pytorch yourself](https://github.com/pytorch/pytorch#from-source) (experienced users only!). 
+2) Install nnU-Net depending on your use case:
+    1) For use as **standardized baseline**, **out-of-the-box segmentation algorithm** or for running 
+     **inference with pretrained models**:
+
+       ```pip install nnunetv2```
+
+    2) For use as integrative **framework** (this will create a copy of the nnU-Net code on your computer so that you
+   can modify it as needed):
+          ```bash
+          git clone https://github.com/MIC-DKFZ/nnUNet.git
+          cd nnUNet
+          pip install -e .
+          ```
+3) nnU-Net needs to know where you intend to save raw data, preprocessed data and trained models. For this you need to
+   set a few environment variables. Please follow the instructions [here](setting_up_paths.md).
+4) (OPTIONAL) Install [hiddenlayer](https://github.com/waleedka/hiddenlayer). hiddenlayer enables nnU-net to generate
+   plots of the network topologies it generates (see [Model training](how_to_use_nnunet.md#model-training)). 
+To install hiddenlayer,
+   run the following command:
+    ```bash
+    pip install --upgrade git+https://github.com/FabianIsensee/hiddenlayer.git
+    ```
+
+Installing nnU-Net will add several new commands to your terminal. These commands are used to run the entire nnU-Net
+pipeline. You can execute them from any location on your system. All nnU-Net commands have the prefix `nnUNetv2_` for
+easy identification.
+
+Note that these commands simply execute python scripts. If you installed nnU-Net in a virtual environment, this
+environment must be activated when executing the commands. You can see what scripts/functions are executed by 
+checking the project.scripts in the [pyproject.toml](../pyproject.toml) file.
+
+All nnU-Net commands have a `-h` option which gives information on how to use them.

nnUNet/documentation/manual_data_splits.md

@@ -0,0 +1,46 @@
+# How to generate custom splits in nnU-Net
+
+Sometimes, the default 5-fold cross-validation split by nnU-Net does not fit a project. Maybe you want to run 3-fold 
+cross-validation instead? Or maybe your training cases cannot be split randomly and require careful stratification. 
+Fear not, for nnU-Net has got you covered (it really can do anything <3).
+
+The splits nnU-Net uses are generated in the `do_split` function of nnUNetTrainer. This function will first look for 
+existing splits, stored as a file, and if no split exists it will create one. So if you wish to influence the split, 
+manually creating a split file that will then be recognized and used is the way to go!
+
+The split file is located in the `nnUNet_preprocessed/DATASETXXX_NAME` folder. So it is best practice to first 
+populate this folder by running `nnUNetv2_plan_and_preproccess`.
+
+Splits are stored as a .json file. They are a simple python list. The length of that list is the number of splits it 
+contains (so it's 5 in the default nnU-Net). Each list entry is a dictionary with keys 'train' and 'val'. Values are 
+again simply lists with the train identifiers in each set. To illustrate this, I am just messing with the Dataset002 
+file as an example:
+
+```commandline
+In [1]: from batchgenerators.utilities.file_and_folder_operations import load_json
+
+In [2]: splits = load_json('splits_final.json')
+
+In [3]: len(splits)
+Out[3]: 5
+
+In [4]: splits[0].keys()
+Out[4]: dict_keys(['train', 'val'])
+
+In [5]: len(splits[0]['train'])
+Out[5]: 16
+
+In [6]: len(splits[0]['val'])
+Out[6]: 4
+
+In [7]: print(splits[0])
+{'train': ['la_003', 'la_004', 'la_005', 'la_009', 'la_010', 'la_011', 'la_014', 'la_017', 'la_018', 'la_019', 'la_020', 'la_022', 'la_023', 'la_026', 'la_029', 'la_030'],
+'val': ['la_007', 'la_016', 'la_021', 'la_024']}
+```
+
+If you are still not sure what splits are supposed to look like, simply download some reference dataset from the
+[Medical Decathlon](http://medicaldecathlon.com/), start some training (to generate the splits) and manually inspect 
+the .json file with your text editor of choice!
+
+In order to generate your custom splits, all you need to do is reproduce the data structure explained above and save it as 
+`splits_final.json` in the `nnUNet_preprocessed/DATASETXXX_NAME` folder. Then use `nnUNetv2_train` etc. as usual.

nnUNet/documentation/pretraining_and_finetuning.md

@@ -0,0 +1,82 @@
+# Pretraining with nnU-Net
+
+## Intro
+
+So far nnU-Net only supports supervised pre-training, meaning that you train a regular nnU-Net on some pretraining dataset 
+and then use the final network weights as initialization for your target dataset. 
+
+As a reminder, many training hyperparameters such as patch size and network topology differ between datasets as a 
+result of the automated dataset analysis and experiment planning nnU-Net is known for. So, out of the box, it is not 
+possible to simply take the network weights from some dataset and then reuse them for another.
+
+Consequently, the plans need to be aligned between the two tasks. In this README we show how this can be achieved and 
+how the resulting weights can then be used for initialization.
+
+### Terminology
+
+Throughout this README we use the following terminology:
+
+- `pretraining dataset` is the dataset you intend to run the pretraining on
+- `finetuning dataset` is the dataset you are interested in; the one you wish to fine tune on
+
+
+## Training on the pretraining dataset
+
+In order to obtain matching network topologies we need to transfer the plans from one dataset to another. Since we are 
+only interested in the finetuning dataset, we first need to run experiment planning (and preprocessing) for it:
+
+```bash
+nnUNetv2_plan_and_preprocess -d FINETUNING_DATASET
+```
+
+Then we need to extract the dataset fingerprint of the pretraining dataset, if not yet available:
+
+```bash
+nnUNetv2_extract_fingerprint -d PRETRAINING_DATASET
+```
+
+Now we can take the plans from the finetuning dataset and transfer it to the pretraining dataset:
+
+```bash
+nnUNetv2_move_plans_between_datasets -s FINETUNING_DATASET -t PRETRAINING_DATASET -sp FINETUNING_PLANS_IDENTIFIER -tp PRETRAINING_PLANS_IDENTIFIER
+```
+
+`FINETUNING_PLANS_IDENTIFIER` is hereby probably nnUNetPlans unless you changed the experiment planner in 
+nnUNetv2_plan_and_preprocess. For `PRETRAINING_PLANS_IDENTIFIER` we recommend you set something custom in order to not 
+overwrite default plans.
+
+Note that EVERYTHING is transferred between the datasets. Not just the network topology, batch size and patch size but 
+also the normalization scheme! Therefore, a transfer between datasets that use different normalization schemes may not 
+work well (but it could, depending on the schemes!).
+
+Note on CT normalization: Yes, also the clip values, mean and std are transferred!
+
+Now you can run the preprocessing on the pretraining dataset:
+
+```bash
+nnUNetv2_preprocess -d PRETRAINING_DATASET -plans_name PRETRAINING_PLANS_IDENTIFIER
+```
+
+And run the training as usual:
+
+```bash
+nnUNetv2_train PRETRAINING_DATASET CONFIG all -p PRETRAINING_PLANS_IDENTIFIER
+```
+
+Note how we use the 'all' fold to train on all available data. For pretraining it does not make sense to split the data.
+
+## Using pretrained weights
+
+Once pretraining is completed (or you obtain compatible weights by other means) you can use them to initialize your model:
+
+```bash
+nnUNetv2_train FINETUNING_DATASET CONFIG FOLD -pretrained_weights PATH_TO_CHECKPOINT
+```
+
+Specify the checkpoint in PATH_TO_CHECKPOINT.
+
+When loading pretrained weights, all layers except the segmentation layers will be used! 
+
+So far there are no specific nnUNet trainers for fine tuning, so the current recommendation is to just use 
+nnUNetTrainer. You can however easily write your own trainers with learning rate ramp up, fine-tuning of segmentation 
+heads or shorter training time.

nnUNet/documentation/region_based_training.md

@@ -0,0 +1,79 @@
+# Region-based training
+
+## What is this about?
+In some segmentation tasks, most prominently the 
+[Brain Tumor Segmentation Challenge](http://braintumorsegmentation.org/), the target areas (based on which the metric 
+will be computed) are different from the labels provided in the training data. This is the case because for some 
+clinical applications, it is more relevant to detect the whole tumor, tumor core and enhancing tumor instead of the 
+individual labels (edema, necrosis and non-enhancing tumor, enhancing tumor). 
+
+<img src="assets/regions_vs_labels.png" width="768px" />
+
+The figure shows an example BraTS case along with label-based representation of the task (top) and region-based 
+representation (bottom). The challenge evaluation is done on the regions. As we have shown in our 
+[BraTS 2018 contribution](https://arxiv.org/abs/1809.10483), directly optimizing those 
+overlapping areas over the individual labels yields better scoring models!
+
+## What can nnU-Net do?
+nnU-Net's region-based training allows you to learn areas that are constructed by merging individual labels. For 
+some segmentation tasks this provides a benefit, as this shifts the importance allocated to different labels during training. 
+Most prominently, this feature can be used to represent **hierarchical classes**, for example when organs + 
+substructures are to be segmented. Imagine a liver segmentation problem, where vessels and tumors are also to be 
+segmented. The first target region could thus be the entire liver (including the substructures), while the remaining 
+targets are the individual substructues.
+
+Important: nnU-Net still requires integer label maps as input and will produce integer label maps as output! 
+Region-based training can be used to learn overlapping labels, but there must be a way to model these overlaps 
+for nnU-Net to work (see below how this is done).
+
+## How do you use it?
+
+When declaring the labels in the `dataset.json` file, BraTS would typically look like this:
+
+```python
+...
+"labels": {
+    "background": 0,
+    "edema": 1,
+    "non_enhancing_and_necrosis": 2,
+    "enhancing_tumor": 3
+},
+...
+```
+(we use different int values than the challenge because nnU-Net needs consecutive integers!)
+
+This representation corresponds to the upper row in the figure above.
+
+For region-based training, the labels need to be changed to the following:
+
+```python
+...
+"labels": {
+    "background": 0,
+    "whole_tumor": [1, 2, 3],
+    "tumor_core": [2, 3],
+    "enhancing_tumor": 3  # or [3]
+},
+"regions_class_order": [1, 2, 3],
+...
+```
+This corresponds to the bottom row in the figure above. Note how an additional entry in the dataset.json is 
+required: `regions_class_order`. This tells nnU-Net how to convert the region representations back to an integer map. 
+It essentially just tells nnU-Net what labels to place for which region in what order. The length of the 
+list here needs to be the same as the number of regions (excl background). Each element in the list corresponds 
+to the label that is placed instead of the region into the final segmentation. Later entries will overwrite earlier ones! 
+Concretely, for the example given here, nnU-Net 
+will firstly place the label 1 (edema) where the 'whole_tumor' region was predicted, then place the label 2 
+(non-enhancing tumor and necrosis) where the "tumor_core" was predicted and finally place the label 3 in the 
+predicted 'enhancing_tumor' area. With each step, part of the previously set pixels 
+will be overwritten with the new label! So when setting your `regions_class_order`, place encompassing regions 
+(like whole tumor etc) first, followed by substructures.
+
+**IMPORTANT** Because the conversion back to a segmentation map is sensitive to the order in which the regions are 
+declared ("place label X in the first region") you need to make sure that this order is not perturbed! When 
+automatically generating the dataset.json, make sure the dictionary keys do not get sorted alphabetically! Set 
+`sort_keys=False` in `json.dump()`!!!
+
+nnU-Net will perform the evaluation + model selection also on the regions, not the individual labels!
+
+That's all. Easy, huh?

nnUNet/documentation/resenc_presets.md

@@ -0,0 +1,126 @@
+# Residual Encoder Presets in nnU-Net
+
+When using these presets, please cite our recent paper on the need for rigorous validation in 3D medical image segmentation:
+
+> Isensee, F.<sup>* </sup>, Wald, T.<sup>* </sup>, Ulrich, C.<sup>* </sup>, Baumgartner, M.<sup>* </sup>, Roy, S., Maier-Hein, K.<sup>†</sup>, Jaeger, P.<sup>†</sup> (2024). nnU-Net Revisited: A Call for Rigorous Validation in 3D Medical Image Segmentation. arXiv preprint arXiv:2404.09556.
+
+*: shared first authors\
+<sup>†</sup>: shared last authors
+
+[PAPER LINK](https://arxiv.org/pdf/2404.09556.pdf)
+
+
+Residual Encoder UNets have been supported by nnU-Net since our participation in KiTS2019, but have flown under the radar.
+This is bound to change with our new nnUNetResEncUNet presets :raised_hands:! Especially on large datasets such as KiTS2023 and AMOS2022 
+they offer improved segmentation performance!
+
+|                        | BTCV  | ACDC  | LiTS  | BraTS | KiTS  | AMOS  |  VRAM |  RT | Arch. | nnU |
+|------------------------|-------|-------|-------|-------|-------|-------|-------|-----|-------|-----|
+|                        | n=30  | n=200 | n=131 | n=1251| n=489 | n=360 |       |     |       |     |
+| nnU-Net (org.) [1]     | 83.08 | 91.54 | 80.09 | 91.24 | 86.04 | 88.64 |  7.70 |  9  |  CNN  | Yes |
+| nnU-Net ResEnc M       | 83.31 | 91.99 | 80.75 | 91.26 | 86.79 | 88.77 |  9.10 |  12 |  CNN  | Yes |
+| nnU-Net ResEnc L       | 83.35 | 91.69 | 81.60 | 91.13 | 88.17 | 89.41 | 22.70 |  35 |  CNN  | Yes |
+| nnU-Net ResEnc XL      | 83.28 | 91.48 | 81.19 | 91.18 | 88.67 | 89.68 | 36.60 |  66 |  CNN  | Yes |
+| MedNeXt L k3 [2]       | 84.70 | 92.65 | 82.14 | 91.35 | 88.25 | 89.62 | 17.30 |  68 |  CNN  | Yes |
+| MedNeXt L k5 [2]       | 85.04 | 92.62 | 82.34 | 91.50 | 87.74 | 89.73 | 18.00 | 233 |  CNN  | Yes |
+| STU-Net S [3]          | 82.92 | 91.04 | 78.50 | 90.55 | 84.93 | 88.08 |  5.20 |  10 |  CNN  | Yes |
+| STU-Net B [3]          | 83.05 | 91.30 | 79.19 | 90.85 | 86.32 | 88.46 |  8.80 |  15 |  CNN  | Yes |
+| STU-Net L [3]          | 83.36 | 91.31 | 80.31 | 91.26 | 85.84 | 89.34 | 26.50 |  51 |  CNN  | Yes |
+| SwinUNETR [4]          | 78.89 | 91.29 | 76.50 | 90.68 | 81.27 | 83.81 | 13.10 |  15 |   TF  | Yes |
+| SwinUNETRV2 [5]        | 80.85 | 92.01 | 77.85 | 90.74 | 84.14 | 86.24 | 13.40 |  15 |   TF  | Yes |
+| nnFormer [6]           | 80.86 | 92.40 | 77.40 | 90.22 | 75.85 | 81.55 |  5.70 |  8  |   TF  | Yes |
+| CoTr [7]               | 81.95 | 90.56 | 79.10 | 90.73 | 84.59 | 88.02 |  8.20 |  18 |   TF  | Yes |
+| No-Mamba Base          | 83.69 | 91.89 | 80.57 | 91.26 | 85.98 | 89.04 |  12.0 |  24 |  CNN  | Yes |
+| U-Mamba Bot [8]        | 83.51 | 91.79 | 80.40 | 91.26 | 86.22 | 89.13 | 12.40 |  24 |  Mam  | Yes |
+| U-Mamba Enc [8]        | 82.41 | 91.22 | 80.27 | 90.91 | 86.34 | 88.38 | 24.90 |  47 |  Mam  | Yes |
+| A3DS SegResNet [9,11]  | 80.69 | 90.69 | 79.28 | 90.79 | 81.11 | 87.27 | 20.00 |  22 |  CNN  |  No |
+| A3DS DiNTS [10, 11]    | 78.18 | 82.97 | 69.05 | 87.75 | 65.28 | 82.35 | 29.20 |  16 |  CNN  |  No |
+| A3DS SwinUNETR [4, 11] | 76.54 | 82.68 | 68.59 | 89.90 | 52.82 | 85.05 | 34.50 |  9  |   TF  |  No |
+
+Results taken from our paper (see above), reported values are Dice scores computed over 5-fold cross-validation on each 
+dataset. All models trained from scratch.
+
+RT: training run time (measured on 1x Nvidia A100 PCIe 40GB)\
+VRAM: GPU VRAM used during training, as reported by nvidia-smi\
+Arch.: CNN = convolutional neural network; TF = transformer; Mam = Mamba\
+nnU: whether the architectrue was integrated and tested with the nnU-Net framework (either by us or the original authors)
+
+## How to use the new presets
+
+We offer three new presets, each targeted for a different GPU VRAM and compute budget:
+- **nnU-Net ResEnc M**: similar GPU budget to the standard UNet configuration. Best suited for GPUs with 9-11GB VRAM. Training time: ~12h on A100
+- **nnU-Net ResEnc L**: requires a GPU with 24GB VRAM. Training time: ~35h on A100
+- **nnU-Net ResEnc XL**: requires a GPU with 40GB VRAM. Training time: ~66h on A100
+
+### **:point_right: We recommend **nnU-Net ResEnc L** as the new default nnU-Net configuration! :point_left:**
+
+The new presets are available as follows ((M/L/XL) = pick one!):
+1. Specify the desired configuration when running experiment planning and preprocessing: 
+`nnUNetv2_plan_and_preprocess -d DATASET -pl nnUNetPlannerResEnc(M/L/XL)`. These planners use the same preprocessed
+data folder as the standard 2d and 3d_fullres configurations since the preprocessed data is identical. Only the
+3d_lowres differs and will be saved in a different folder to allow all configurations to coexist! If you are only 
+planning to run 3d_fullres/2d and you already have this data preprocessed, you can just run 
+`nnUNetv2_plan_experiment -d DATASET -pl nnUNetPlannerResEnc(M/L/XL)` to avoid preprocessing again! 
+2. Now, just specify the correct plans when running `nnUNetv2_train`, `nnUNetv2_predict` etc. The interface is 
+consistent across all nnU-Net commands: `-p nnUNetResEncUNet(M/L/XL)Plans`  
+
+Training results for the new presets will be stored in a dedicated folder and will not overwrite standard nnU-Net 
+results! So don't be afraid to give it a go!
+
+## Scaling ResEnc nnU-Net beyond the Presets
+The presets differ from `ResEncUNetPlanner` in two ways:
+- They set new default values for `gpu_memory_target_in_gb` to target the respective VRAM consumptions
+- They remove the batch size cap of 0.05 (= previously one batch could not cover mode pixels than 5% of the entire dataset, not it can be arbitrarily large)
+
+The presets are merely there to make life easier, and to provide standardized configurations people can benchmark with.
+You can easily adapt the GPU memory target to match your GPU, and to scale beyond 40GB of GPU memory. 
+
+Here is an example for how to scale to 80GB VRAM on Dataset003_Liver:
+
+`nnUNetv2_plan_experiment -d 3 -pl nnUNetPlannerResEncM -gpu_memory_target 80 -overwrite_plans_name nnUNetResEncUNetPlans_80G`
+
+Just use `-p nnUNetResEncUNetPlans_80G` moving forward as outlined above! Running the example above will yield a 
+warning ("You are running nnUNetPlannerM with a non-standard gpu_memory_target_in_gb"). This warning can be ignored here.
+**Always change the plans identifier with `-overwrite_plans_name NEW_PLANS_NAME` when messing with the VRAM target in 
+order to not overwrite preset plans!**
+
+Why not use `ResEncUNetPlanner` -> because that one still has the 5% cap in place!
+
+### Scaling to multiple GPUs
+When scaling to multiple GPUs, do not just specify the combined amount of VRAM to `nnUNetv2_plan_experiment` as this 
+may result in patch sizes that are too large to be processed by individual GPUs. It is best to let this command run for 
+the VRAM budget of one GPU, and then manually edit the plans file to increase the batch size. You can use [configuration inheritance](explanation_plans_files.md).
+In the configurations dictionary of the generated plans JSON file, add the following entry:
+
+```json
+        "3d_fullres_bsXX": {
+            "inherits_from": "3d_fullres",
+            "batch_size": XX
+        },
+```
+Where XX is the new batch size. If 3d_fullres has a batch size of 2 for one GPU and you are planning to scale to 8 GPUs, make the new batch size 2x8=16!
+You can then train the new configuration using nnU-Net's multi-GPU settings:
+
+```bash
+nnUNetv2_train DATASETID 3d_fullres_bsXX FOLD -p nnUNetResEncUNetPlans_80G -num_gpus 8
+```
+
+## Proposing a new segmentation method? Benchmark the right way!
+When benchmarking new segmentation methods against nnU-Net, we encourage to benchmark against the residual encoder 
+variants. For a fair comparison, pick the variant that most closely matches the GPU memory and compute 
+requirements of your method!
+
+
+## References
+ [1] Isensee, Fabian, et al. "nnU-Net: a self-configuring method for deep learning-based biomedical image segmentation." Nature methods 18.2 (2021): 203-211.\
+ [2] Roy, Saikat, et al. "Mednext: transformer-driven scaling of convnets for medical image segmentation." International Conference on Medical Image Computing and Computer-Assisted Intervention. Cham: Springer Nature Switzerland, 2023.\
+ [3] Huang, Ziyan, et al. "Stu-net: Scalable and transferable medical image segmentation models empowered by large-scale supervised pre-training." arXiv preprint arXiv:2304.06716 (2023).\
+ [4] Hatamizadeh, Ali, et al. "Swin unetr: Swin transformers for semantic segmentation of brain tumors in mri images." International MICCAI Brainlesion Workshop. Cham: Springer International Publishing, 2021.\
+ [5] He, Yufan, et al. "Swinunetr-v2: Stronger swin transformers with stagewise convolutions for 3d medical image segmentation." International Conference on Medical Image Computing and Computer-Assisted Intervention. Cham: Springer Nature Switzerland, 2023.\
+ [6] Zhou, Hong-Yu, et al. "nnformer: Interleaved transformer for volumetric segmentation." arXiv preprint arXiv:2109.03201 (2021).\
+ [7] Xie, Yutong, et al. "Cotr: Efficiently bridging cnn and transformer for 3d medical image segmentation." Medical Image Computing and Computer Assisted Intervention–MICCAI 2021: 24th International Conference, Strasbourg, France, September 27–October 1, 2021, Proceedings, Part III 24. Springer International Publishing, 2021.\
+ [8] Ma, Jun, Feifei Li, and Bo Wang. "U-mamba: Enhancing long-range dependency for biomedical image segmentation." arXiv preprint arXiv:2401.04722 (2024).\
+ [9] Myronenko, Andriy. "3D MRI brain tumor segmentation using autoencoder regularization." Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries: 4th International Workshop, BrainLes 2018, Held in Conjunction with MICCAI 2018, Granada, Spain, September 16, 2018, Revised Selected Papers, Part II 4. Springer International Publishing, 2019.\
+ [10] He, Yufan, et al. "Dints: Differentiable neural network topology search for 3d medical image segmentation." Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2021.\
+ [11] Auto3DSeg, MONAI 1.3.0, [LINK](https://github.com/Project-MONAI/tutorials/tree/ed8854fa19faa49083f48abf25a2c30ab9ac1c6b/auto3dseg)
+

nnUNet/documentation/run_inference_with_pretrained_models.md

@@ -0,0 +1,7 @@
+# How to run inference with pretrained models
+**Important:** Pretrained weights from nnU-Net v1 are NOT compatible with V2. You will need to retrain with the new 
+version. But honestly, you already have a fully trained model with which you can run inference (in v1), so 
+just continue using that!
+
+Not yet available for V2 :-(
+If you wish to run inference with pretrained models, check out the old nnU-Net for now. We are working on this full steam!

nnUNet/documentation/set_environment_variables.md

@@ -0,0 +1,78 @@
+# How to set environment variables
+
+nnU-Net requires some environment variables so that it always knows where the raw data, preprocessed data and trained 
+models are. Depending on the operating system, these environment variables need to be set in different ways.
+
+Variables can either be set permanently (recommended!) or you can decide to set them every time you call nnU-Net. 
+
+# Linux & MacOS
+
+## Permanent
+Locate the `.bashrc` file in your home folder and add the following lines to the bottom:
+
+```bash
+export nnUNet_raw="/media/fabian/nnUNet_raw"
+export nnUNet_preprocessed="/media/fabian/nnUNet_preprocessed"
+export nnUNet_results="/media/fabian/nnUNet_results"
+```
+
+(Of course you need to adapt the paths to the actual folders you intend to use).
+If you are using a different shell, such as zsh, you will need to find the correct script for it. For zsh this is `.zshrc`.
+
+## Temporary
+Just execute the following lines whenever you run nnU-Net:
+```bash
+export nnUNet_raw="/media/fabian/nnUNet_raw"
+export nnUNet_preprocessed="/media/fabian/nnUNet_preprocessed"
+export nnUNet_results="/media/fabian/nnUNet_results"
+```
+(Of course you need to adapt the paths to the actual folders you intend to use).
+
+Important: These variables will be deleted if you close your terminal! They will also only apply to the current 
+terminal window and DO NOT transfer to other terminals!
+
+Alternatively you can also just prefix them to your nnU-Net commands:
+
+`nnUNet_results="/media/fabian/nnUNet_results" nnUNet_preprocessed="/media/fabian/nnUNet_preprocessed" nnUNetv2_train[...]`
+
+## Verify that environment parameters are set
+You can always execute `echo ${nnUNet_raw}` etc to print the environment variables. This will return an empty string if 
+they were not set.
+
+# Windows
+Useful links:
+- [https://www3.ntu.edu.sg](https://www3.ntu.edu.sg/home/ehchua/programming/howto/Environment_Variables.html#:~:text=To%20set%20(or%20change)%20a,it%20to%20an%20empty%20string.)
+- [https://phoenixnap.com](https://phoenixnap.com/kb/windows-set-environment-variable)
+
+## Permanent
+See `Set Environment Variable in Windows via GUI` [here](https://phoenixnap.com/kb/windows-set-environment-variable). 
+Or read about setx (command prompt).
+
+## Temporary
+Just execute the following before you run nnU-Net:
+
+(PowerShell)
+```PowerShell
+$Env:nnUNet_raw = "C:/Users/fabian/nnUNet_raw"
+$Env:nnUNet_preprocessed = "C:/Users/fabian/nnUNet_preprocessed"
+$Env:nnUNet_results = "C:/Users/fabian/nnUNet_results"
+```
+
+(Command Prompt)
+```Command Prompt
+set nnUNet_raw=C:/Users/fabian/nnUNet_raw
+set nnUNet_preprocessed=C:/Users/fabian/nnUNet_preprocessed
+set nnUNet_results=C:/Users/fabian/fabian/nnUNet_results
+```
+
+(Of course you need to adapt the paths to the actual folders you intend to use).
+
+Important: These variables will be deleted if you close your session! They will also only apply to the current 
+window and DO NOT transfer to other sessions!
+
+## Verify that environment parameters are set
+Printing in Windows works differently depending on the environment you are in:
+
+PowerShell: `echo $Env:[variable_name]`
+
+Command Prompt: `echo %variable_name%`

nnUNet/documentation/setting_up_paths.md

@@ -0,0 +1,39 @@
+# Setting up Paths
+
+nnU-Net relies on environment variables to know where raw data, preprocessed data and trained model weights are stored. 
+To use the full functionality of nnU-Net, the following three environment variables must be set:
+
+1) `nnUNet_raw`: This is where you place the raw datasets. This folder will have one subfolder for each dataset names 
+DatasetXXX_YYY where XXX is a 3-digit identifier (such as 001, 002, 043, 999, ...) and YYY is the (unique) 
+dataset name. The datasets must be in nnU-Net format, see [here](dataset_format.md).
+
+    Example tree structure:
+    ```
+    nnUNet_raw/Dataset001_NAME1
+    ├── dataset.json
+    ├── imagesTr
+    │   ├── ...
+    ├── imagesTs
+    │   ├── ...
+    └── labelsTr
+        ├── ...
+    nnUNet_raw/Dataset002_NAME2
+    ├── dataset.json
+    ├── imagesTr
+    │   ├── ...
+    ├── imagesTs
+    │   ├── ...
+    └── labelsTr
+        ├── ...
+    ```
+
+2) `nnUNet_preprocessed`: This is the folder where the preprocessed data will be saved. The data will also be read from 
+this folder during training. It is important that this folder is located on a drive with low access latency and high 
+throughput (such as a nvme SSD (PCIe gen 3 is sufficient)).
+
+3) `nnUNet_results`: This specifies where nnU-Net will save the model weights. If pretrained models are downloaded, this 
+is where it will save them.
+
+### How to set environment variables
+
+See [here](set_environment_variables.md).

nnUNet/documentation/tldr_migration_guide_from_v1.md

@@ -0,0 +1,20 @@
+# TLDR Migration Guide from nnU-Net V1
+
+- nnU-Net V2 can be installed simultaneously with V1. They won't get in each other's way
+- The environment variables needed for V2 have slightly different names. Read [this](setting_up_paths.md). 
+- nnU-Net V2 datasets are called DatasetXXX_NAME. Not Task.
+- Datasets have the same structure (imagesTr, labelsTr, dataset.json) but we now support more 
+[file types](dataset_format.md#supported-file-formats). The dataset.json is simplified. Use `generate_dataset_json` 
+from nnunetv2.dataset_conversion.generate_dataset_json.py. 
+- Careful: labels are now no longer declared as value:name but name:value. This has to do with [hierarchical labels](region_based_training.md). 
+- nnU-Net v2 commands start with `nnUNetv2...`. They work mostly (but not entirely) the same. Just use the `-h` option.
+- You can transfer your V1 raw datasets to V2 with `nnUNetv2_convert_old_nnUNet_dataset`. You cannot transfer trained 
+models. Continue to use the old nnU-Net Version for making inference with those.
+- These are the commands you are most likely to be using (in that order)
+  - `nnUNetv2_plan_and_preprocess`. Example: `nnUNetv2_plan_and_preprocess -d 2`
+  - `nnUNetv2_train`. Example: `nnUNetv2_train 2 3d_fullres 0`
+  - `nnUNetv2_find_best_configuration`. Example: `nnUNetv2_find_best_configuration 2 -c 2d 3d_fullres`. This command
+    will now create a `inference_instructions.txt` file in your `nnUNet_preprocessed/DatasetXXX_NAME/` folder which
+    tells you exactly how to do inference.
+  - `nnUNetv2_predict`. Example: `nnUNetv2_predict -i INPUT_FOLDER -o OUTPUT_FOLDER -c 3d_fullres -d 2`
+  - `nnUNetv2_apply_postprocessing` (see inference_instructions.txt)