# NFNet

**Category:** CNN · **Code:**
[`timm/models/nfnet.py`](https://github.com/huggingface/pytorch-image-models/blob/main/timm/models/nfnet.py) ·
**Architectures:** 36 · **Pretrained weights:** 14 (14 with ImageNet-1k results)

Normalization-Free networks, trained without batch normalization using scaled weight standardization: NFNet (with
adaptive gradient clipping), NF-RegNet and pre-activation NF-ResNet models.

## Papers

* [Characterizing signal propagation to close the performance gap in unnormalized
  ResNets](https://arxiv.org/abs/2101.08692)
* [High-Performance Large-Scale Image Recognition Without Normalization](https://arxiv.org/abs/2102.06171)

## Usage

See the [Quickstart](../quickstart) for loading pretrained weights, image preprocessing and fine-tuning, and [Feature
Extraction](../feature_extraction) for using the models as backbones.

```py
>>> import timm
>>> model = timm.create_model('nf_resnet50.ra2_in1k', pretrained=True)
>>> data_config = timm.data.resolve_model_data_config(model)
```

## Model API support

| Feature | `NormFreeNet` |
|---|---|
| [`features_only=True` feature maps](../model_api#intermediate-features) | ✓ (strides 2, 4, 8, 16, 32) |
| [`forward_intermediates()`](../model_api#intermediate-features) | ✗ |
| [`prune_intermediate_layers()`](../model_api#intermediate-features) | ✗ |
| [Any input size, no changes needed](../model_api#input-size) | ✓ |
| [`set_input_size()`](../model_api#input-size) | – |
| [`dynamic_img_size=True`](../model_api#input-size) | – |
| [`set_grad_checkpointing()`](../model_api#training-hooks) | ✓ |
| [Layer-wise LR decay (`group_matcher()`)](../model_api#training-hooks) | ✓ |
| [`reset_classifier()`](../model_api#classifier) | ✓ |

✓ supported, ✓ (square) works for square sizes only, ✗ not supported, – not needed (the model already accepts any input
size). Checked by running each feature on a randomly initialized model (`NormFreeNet` with `nf_regnet_b0` at 192), other
architectures of the same class are expected to behave the same. See the [Model API](../model_api) page for details.

## Architectures

| Architecture | Params (M) | GMACs | Input size | Pretrained weights |
|---|---:|---:|---:|---:|
| `dm_nfnet_f0` | 71.49 | 7.21 | 192 | 1 |
| `dm_nfnet_f1` | 132.63 | 17.87 | 224 | 1 |
| `dm_nfnet_f2` | 193.78 | 33.76 | 256 | 1 |
| `dm_nfnet_f3` | 254.92 | 68.77 | 320 | 1 |
| `dm_nfnet_f4` | 316.07 | 122.14 | 384 | 1 |
| `dm_nfnet_f5` | 377.21 | 170.71 | 416 | 1 |
| `dm_nfnet_f6` | 438.36 | 229.70 | 448 | 1 |
| `eca_nfnet_l0` | 24.14 | 4.35 | 224 | 1 |
| `eca_nfnet_l1` | 41.41 | 9.62 | 256 | 1 |
| `eca_nfnet_l2` | 56.72 | 20.95 | 320 | 1 |
| `eca_nfnet_l3` | 72.04 | 32.57 | 352 | 0 |
| `nf_ecaresnet26` | 16.00 | 2.41 | 224 | 0 |
| `nf_ecaresnet50` | 25.56 | 4.21 | 224 | 0 |
| `nf_ecaresnet101` | 44.55 | 8.01 | 224 | 0 |
| `nf_regnet_b0` | 8.76 | 0.37 | 192 | 0 |
| `nf_regnet_b1` | 10.22 | 0.82 | 256 | 1 |
| `nf_regnet_b2` | 14.31 | 0.97 | 240 | 0 |
| `nf_regnet_b3` | 18.59 | 1.67 | 288 | 0 |
| `nf_regnet_b4` | 30.21 | 3.29 | 320 | 0 |
| `nf_regnet_b5` | 49.74 | 7.95 | 384 | 0 |
| `nf_resnet26` | 16.00 | 2.41 | 224 | 0 |
| `nf_resnet50` | 25.56 | 5.46 | 256 | 1 |
| `nf_resnet101` | 44.55 | 8.01 | 224 | 0 |
| `nf_seresnet26` | 17.40 | 2.41 | 224 | 0 |
| `nf_seresnet50` | 28.09 | 4.21 | 224 | 0 |
| `nf_seresnet101` | 49.33 | 8.02 | 224 | 0 |
| `nfnet_f0` | 71.49 | 7.21 | 192 | 0 |
| `nfnet_f1` | 132.63 | 17.87 | 224 | 0 |
| `nfnet_f2` | 193.78 | 33.76 | 256 | 0 |
| `nfnet_f3` | 254.92 | 68.77 | 320 | 0 |
| `nfnet_f4` | 316.07 | 122.14 | 384 | 0 |
| `nfnet_f5` | 377.21 | 170.71 | 416 | 0 |
| `nfnet_f6` | 438.36 | 229.70 | 448 | 0 |
| `nfnet_f7` | 499.50 | 300.08 | 480 | 0 |
| `nfnet_l0` | 35.07 | 4.36 | 224 | 1 |
| `test_nfnet` | 0.38 | 0.29 | 160 | 1 |

GMACs and parameter counts are for the default input size and classifier of each architecture, from the [benchmark
results](https://github.com/huggingface/pytorch-image-models/tree/main/results).

## Pretrained weights

Top-1 is ImageNet-1k validation accuracy from
[`results-imagenet.csv`](https://github.com/huggingface/pytorch-image-models/blob/main/results/results-imagenet.csv) at
the listed eval size, - if the weight has no ImageNet-1k result (e.g. pretraining or image-text weights). Classes is the
size of the pretrained classifier, none for weights without one.

Weights are grouped by source: the paper or origin given in the pretrained config, else a Hub organization other than
`timm`, else the first part of the pretrained tag (`architecture.tag`). Sources with a single weight are listed under
other.

### `dm` tags

| Weight | Input size | Top-1 (%) | Eval size | Classes | License | Hub |
|---|---:|---:|---:|---:|---|---|
| `dm_nfnet_f0.dm_in1k` | 192 (test 256) | 83.468 | 256 | 1000 | apache-2.0 | [timm](https://huggingface.co/timm/dm_nfnet_f0.dm_in1k) |
| `dm_nfnet_f1.dm_in1k` | 224 (test 320) | 84.700 | 320 | 1000 | apache-2.0 | [timm](https://huggingface.co/timm/dm_nfnet_f1.dm_in1k) |
| `dm_nfnet_f2.dm_in1k` | 256 (test 352) | 85.184 | 352 | 1000 | apache-2.0 | [timm](https://huggingface.co/timm/dm_nfnet_f2.dm_in1k) |
| `dm_nfnet_f3.dm_in1k` | 320 (test 416) | 85.676 | 416 | 1000 | apache-2.0 | [timm](https://huggingface.co/timm/dm_nfnet_f3.dm_in1k) |
| `dm_nfnet_f4.dm_in1k` | 384 (test 512) | 85.838 | 512 | 1000 | apache-2.0 | [timm](https://huggingface.co/timm/dm_nfnet_f4.dm_in1k) |
| `dm_nfnet_f5.dm_in1k` | 416 (test 544) | 86.102 | 544 | 1000 | apache-2.0 | [timm](https://huggingface.co/timm/dm_nfnet_f5.dm_in1k) |
| `dm_nfnet_f6.dm_in1k` | 448 (test 576) | 86.364 | 576 | 1000 | apache-2.0 | [timm](https://huggingface.co/timm/dm_nfnet_f6.dm_in1k) |

### `ra2` tags

| Weight | Input size | Top-1 (%) | Eval size | Classes | License | Hub |
|---|---:|---:|---:|---:|---|---|
| `eca_nfnet_l0.ra2_in1k` | 224 (test 288) | 82.590 | 288 | 1000 | apache-2.0 | [timm](https://huggingface.co/timm/eca_nfnet_l0.ra2_in1k) |
| `eca_nfnet_l1.ra2_in1k` | 256 (test 320) | 84.062 | 320 | 1000 | apache-2.0 | [timm](https://huggingface.co/timm/eca_nfnet_l1.ra2_in1k) |
| `nf_regnet_b1.ra2_in1k` | 256 (test 288) | 79.368 | 288 | 1000 | apache-2.0 | [timm](https://huggingface.co/timm/nf_regnet_b1.ra2_in1k) |
| `nf_resnet50.ra2_in1k` | 256 (test 288) | 80.658 | 288 | 1000 | apache-2.0 | [timm](https://huggingface.co/timm/nf_resnet50.ra2_in1k) |
| `nfnet_l0.ra2_in1k` | 224 (test 288) | 82.766 | 288 | 1000 | apache-2.0 | [timm](https://huggingface.co/timm/nfnet_l0.ra2_in1k) |

### Other

| Weight | Input size | Top-1 (%) | Eval size | Classes | License | Hub |
|---|---:|---:|---:|---:|---|---|
| `eca_nfnet_l2.ra3_in1k` | 320 (test 384) | 84.710 | 384 | 1000 | apache-2.0 | [timm](https://huggingface.co/timm/eca_nfnet_l2.ra3_in1k) |
| `test_nfnet.r160_in1k` | 160 | 47.634 | 160 | 1000 | apache-2.0 | [timm](https://huggingface.co/timm/test_nfnet.r160_in1k) |