# MaxViT and CoAtNet

**Category:** Hybrid · **Code:**
[`timm/models/maxxvit.py`](https://github.com/huggingface/pytorch-image-models/blob/main/timm/models/maxxvit.py) ·
**Architectures:** 58 · **Pretrained weights:** 52 (43 with ImageNet-1k results)

From-scratch implementations of MaxViT (MBConv blocks with multi-axis attention, blocked local and dilated grid
attention) and CoAtNet (convolution stages followed by relative attention stages), plus `timm` variants (`_rw` configs,
MaxxViT).

## Papers

* [MaxViT: Multi-Axis Vision Transformer](https://arxiv.org/abs/2204.01697)
* [CoAtNet: Marrying Convolution and Attention for All Data Sizes](https://arxiv.org/abs/2106.04803)

## Usage

See the [Quickstart](../quickstart) for loading pretrained weights, image preprocessing and fine-tuning, and [Feature
Extraction](../feature_extraction) for using the models as backbones.

```py
>>> import timm
>>> model = timm.create_model('maxxvitv2_nano_rw_256.sw_in1k', pretrained=True)
>>> data_config = timm.data.resolve_model_data_config(model)
```

## Model API support

| Feature | `MaxxVit` |
|---|---|
| [`features_only=True` feature maps](../model_api#intermediate-features) | ✓ (strides 2, 4, 8, 16, 32) |
| [`forward_intermediates()`](../model_api#intermediate-features) | ✓ |
| [`prune_intermediate_layers()`](../model_api#intermediate-features) | ✓ |
| [Any input size, no changes needed](../model_api#input-size) | ✗ |
| [`set_input_size()`](../model_api#input-size) | ✓ |
| [`dynamic_img_size=True`](../model_api#input-size) | ✗ |
| [`set_grad_checkpointing()`](../model_api#training-hooks) | ✓ |
| [Layer-wise LR decay (`group_matcher()`)](../model_api#training-hooks) | ✓ |
| [`reset_classifier()`](../model_api#classifier) | ✓ |

✓ supported, ✓ (square) works for square sizes only, ✗ not supported, – not needed (the model already accepts any input
size). Checked by running each feature on a randomly initialized model (`MaxxVit` with `maxvit_pico_rw_256` at 256),
other architectures of the same class are expected to behave the same. See the [Model API](../model_api) page for
details.

## Architectures

| Architecture | Params (M) | GMACs | Input size | Pretrained weights |
|---|---:|---:|---:|---:|
| `coatnet_0_224` | 25.04 | 4.58 | 224 | 0 |
| `coatnet_0_rw_224` | 27.44 | 4.43 | 224 | 1 |
| `coatnet_1_224` | 42.23 | 8.70 | 224 | 0 |
| `coatnet_1_rw_224` | 41.72 | 8.04 | 224 | 1 |
| `coatnet_2_224` | 74.68 | 16.50 | 224 | 0 |
| `coatnet_2_rw_224` | 73.87 | 15.09 | 224 | 2 |
| `coatnet_3_224` | 166.97 | 36.56 | 224 | 0 |
| `coatnet_3_rw_224` | 181.81 | 33.44 | 224 | 1 |
| `coatnet_4_224` | 275.43 | 62.48 | 224 | 0 |
| `coatnet_5_224` | 687.47 | 145.49 | 224 | 0 |
| `coatnet_bn_0_rw_224` | 27.44 | 4.67 | 224 | 1 |
| `coatnet_nano_cc_224` | 13.76 | 2.24 | 224 | 0 |
| `coatnet_nano_rw_224` | 15.14 | 2.41 | 224 | 1 |
| `coatnet_pico_rw_224` | 10.85 | 2.05 | 224 | 0 |
| `coatnet_rmlp_0_rw_224` | 27.45 | 4.72 | 224 | 0 |
| `coatnet_rmlp_1_rw2_224` | 41.72 | 8.11 | 224 | 2 |
| `coatnet_rmlp_1_rw_224` | 41.69 | 7.85 | 224 | 1 |
| `coatnet_rmlp_2_rw_224` | 73.88 | 15.18 | 224 | 3 |
| `coatnet_rmlp_2_rw_384` | 73.88 | 47.69 | 384 | 1 |
| `coatnet_rmlp_3_rw_224` | 165.15 | 33.56 | 224 | 0 |
| `coatnet_rmlp_nano_rw_224` | 15.15 | 2.62 | 224 | 1 |
| `coatnext_nano_rw_224` | 14.70 | 2.47 | 224 | 1 |
| `maxvit_base_tf_224` | 119.47 | 24.04 | 224 | 2 |
| `maxvit_base_tf_384` | 119.65 | 73.80 | 384 | 2 |
| `maxvit_base_tf_512` | 119.88 | 138.02 | 512 | 2 |
| `maxvit_large_tf_224` | 211.79 | 43.68 | 224 | 2 |
| `maxvit_large_tf_384` | 212.03 | 132.55 | 384 | 2 |
| `maxvit_large_tf_512` | 212.33 | 244.75 | 512 | 2 |
| `maxvit_nano_rw_256` | 15.45 | 4.46 | 256 | 1 |
| `maxvit_pico_rw_256` | 7.46 | 1.83 | 256 | 0 |
| `maxvit_rmlp_base_rw_224` | 116.14 | 23.15 | 224 | 2 |
| `maxvit_rmlp_base_rw_384` | 116.14 | 70.97 | 384 | 1 |
| `maxvit_rmlp_nano_rw_256` | 15.50 | 4.47 | 256 | 1 |
| `maxvit_rmlp_pico_rw_256` | 7.52 | 1.85 | 256 | 1 |
| `maxvit_rmlp_small_rw_224` | 64.90 | 10.75 | 224 | 1 |
| `maxvit_rmlp_small_rw_256` | 64.90 | 14.15 | 256 | 0 |
| `maxvit_rmlp_tiny_rw_256` | 29.15 | 6.77 | 256 | 1 |
| `maxvit_small_tf_224` | 68.93 | 11.66 | 224 | 1 |
| `maxvit_small_tf_384` | 69.02 | 35.87 | 384 | 1 |
| `maxvit_small_tf_512` | 69.13 | 67.26 | 512 | 1 |
| `maxvit_tiny_pm_256` | 30.09 | 6.61 | 256 | 0 |
| `maxvit_tiny_rw_224` | 29.06 | 5.11 | 224 | 1 |
| `maxvit_tiny_rw_256` | 29.07 | 6.74 | 256 | 0 |
| `maxvit_tiny_tf_224` | 30.92 | 5.60 | 224 | 1 |
| `maxvit_tiny_tf_384` | 30.98 | 17.53 | 384 | 1 |
| `maxvit_tiny_tf_512` | 31.05 | 33.49 | 512 | 1 |
| `maxvit_xlarge_tf_224` | 506.99 | 97.52 | 224 | 1 |
| `maxvit_xlarge_tf_384` | 475.32 | 292.78 | 384 | 1 |
| `maxvit_xlarge_tf_512` | 475.77 | 534.14 | 512 | 1 |
| `maxxvit_rmlp_nano_rw_256` | 16.78 | 4.37 | 256 | 1 |
| `maxxvit_rmlp_small_rw_256` | 66.01 | 14.67 | 256 | 1 |
| `maxxvit_rmlp_tiny_rw_256` | 29.64 | 6.66 | 256 | 0 |
| `maxxvitv2_nano_rw_256` | 23.70 | 6.26 | 256 | 1 |
| `maxxvitv2_rmlp_base_rw_224` | 116.09 | 24.20 | 224 | 2 |
| `maxxvitv2_rmlp_base_rw_384` | 116.09 | 72.98 | 384 | 1 |
| `maxxvitv2_rmlp_large_rw_224` | 215.42 | 44.14 | 224 | 0 |
| `test_coatnet` | 0.58 | - | 160 | 0 |
| `test_maxvit` | 1.09 | - | 160 | 0 |

GMACs and parameter counts are for the default input size and classifier of each architecture, from the [benchmark
results](https://github.com/huggingface/pytorch-image-models/tree/main/results).

## Pretrained weights

Top-1 is ImageNet-1k validation accuracy from
[`results-imagenet.csv`](https://github.com/huggingface/pytorch-image-models/blob/main/results/results-imagenet.csv) at
the listed eval size, - if the weight has no ImageNet-1k result (e.g. pretraining or image-text weights). Classes is the
size of the pretrained classifier, none for weights without one.

| Weight | Input size | Top-1 (%) | Eval size | Classes | License | Hub |
|---|---:|---:|---:|---:|---|---|
| `coatnet_0_rw_224.sw_in1k` | 224 | 82.396 | 224 | 1000 | apache-2.0 | [timm](https://huggingface.co/timm/coatnet_0_rw_224.sw_in1k) |
| `coatnet_1_rw_224.sw_in1k` | 224 | 83.620 | 224 | 1000 | apache-2.0 | [timm](https://huggingface.co/timm/coatnet_1_rw_224.sw_in1k) |
| `coatnet_2_rw_224.sw_in12k` | 224 | - | - | 11821 | apache-2.0 | [timm](https://huggingface.co/timm/coatnet_2_rw_224.sw_in12k) |
| `coatnet_2_rw_224.sw_in12k_ft_in1k` | 224 | 86.582 | 224 | 1000 | apache-2.0 | [timm](https://huggingface.co/timm/coatnet_2_rw_224.sw_in12k_ft_in1k) |
| `coatnet_3_rw_224.sw_in12k` | 224 | - | - | 11821 | apache-2.0 | [timm](https://huggingface.co/timm/coatnet_3_rw_224.sw_in12k) |
| `coatnet_bn_0_rw_224.sw_in1k` | 224 | 82.394 | 224 | 1000 | apache-2.0 | [timm](https://huggingface.co/timm/coatnet_bn_0_rw_224.sw_in1k) |
| `coatnet_nano_rw_224.sw_in1k` | 224 | 81.704 | 224 | 1000 | apache-2.0 | [timm](https://huggingface.co/timm/coatnet_nano_rw_224.sw_in1k) |
| `coatnet_rmlp_1_rw2_224.sw_in12k` | 224 | - | - | 11821 | apache-2.0 | [timm](https://huggingface.co/timm/coatnet_rmlp_1_rw2_224.sw_in12k) |
| `coatnet_rmlp_1_rw2_224.sw_in12k_ft_in1k` | 224 | 84.940 | 224 | 1000 | apache-2.0 | [timm](https://huggingface.co/timm/coatnet_rmlp_1_rw2_224.sw_in12k_ft_in1k) |
| `coatnet_rmlp_1_rw_224.sw_in1k` | 224 | 83.350 | 224 | 1000 | apache-2.0 | [timm](https://huggingface.co/timm/coatnet_rmlp_1_rw_224.sw_in1k) |
| `coatnet_rmlp_2_rw_224.sw_in12k` | 224 | - | - | 11821 | apache-2.0 | [timm](https://huggingface.co/timm/coatnet_rmlp_2_rw_224.sw_in12k) |
| `coatnet_rmlp_2_rw_224.sw_in12k_ft_in1k` | 224 | 86.538 | 224 | 1000 | apache-2.0 | [timm](https://huggingface.co/timm/coatnet_rmlp_2_rw_224.sw_in12k_ft_in1k) |
| `coatnet_rmlp_2_rw_224.sw_in1k` | 224 | 84.608 | 224 | 1000 | apache-2.0 | [timm](https://huggingface.co/timm/coatnet_rmlp_2_rw_224.sw_in1k) |
| `coatnet_rmlp_2_rw_384.sw_in12k_ft_in1k` | 384 | 87.388 | 384 | 1000 | apache-2.0 | [timm](https://huggingface.co/timm/coatnet_rmlp_2_rw_384.sw_in12k_ft_in1k) |
| `coatnet_rmlp_nano_rw_224.sw_in1k` | 224 | 82.064 | 224 | 1000 | apache-2.0 | [timm](https://huggingface.co/timm/coatnet_rmlp_nano_rw_224.sw_in1k) |
| `coatnext_nano_rw_224.sw_in1k` | 224 | 81.998 | 224 | 1000 | apache-2.0 | [timm](https://huggingface.co/timm/coatnext_nano_rw_224.sw_in1k) |
| `maxvit_base_tf_224.in1k` | 224 | 84.884 | 224 | 1000 | apache-2.0 | [timm](https://huggingface.co/timm/maxvit_base_tf_224.in1k) |
| `maxvit_base_tf_224.in21k` | 224 | - | - | 21843 | apache-2.0 | [timm](https://huggingface.co/timm/maxvit_base_tf_224.in21k) |
| `maxvit_base_tf_384.in1k` | 384 | 86.290 | 384 | 1000 | apache-2.0 | [timm](https://huggingface.co/timm/maxvit_base_tf_384.in1k) |
| `maxvit_base_tf_384.in21k_ft_in1k` | 384 | 87.930 | 384 | 1000 | apache-2.0 | [timm](https://huggingface.co/timm/maxvit_base_tf_384.in21k_ft_in1k) |
| `maxvit_base_tf_512.in1k` | 512 | 86.598 | 512 | 1000 | apache-2.0 | [timm](https://huggingface.co/timm/maxvit_base_tf_512.in1k) |
| `maxvit_base_tf_512.in21k_ft_in1k` | 512 | 88.214 | 512 | 1000 | apache-2.0 | [timm](https://huggingface.co/timm/maxvit_base_tf_512.in21k_ft_in1k) |
| `maxvit_large_tf_224.in1k` | 224 | 84.938 | 224 | 1000 | apache-2.0 | [timm](https://huggingface.co/timm/maxvit_large_tf_224.in1k) |
| `maxvit_large_tf_224.in21k` | 224 | - | - | 21843 | apache-2.0 | [timm](https://huggingface.co/timm/maxvit_large_tf_224.in21k) |
| `maxvit_large_tf_384.in1k` | 384 | 86.242 | 384 | 1000 | apache-2.0 | [timm](https://huggingface.co/timm/maxvit_large_tf_384.in1k) |
| `maxvit_large_tf_384.in21k_ft_in1k` | 384 | 87.994 | 384 | 1000 | apache-2.0 | [timm](https://huggingface.co/timm/maxvit_large_tf_384.in21k_ft_in1k) |
| `maxvit_large_tf_512.in1k` | 512 | 86.534 | 512 | 1000 | apache-2.0 | [timm](https://huggingface.co/timm/maxvit_large_tf_512.in1k) |
| `maxvit_large_tf_512.in21k_ft_in1k` | 512 | 88.236 | 512 | 1000 | apache-2.0 | [timm](https://huggingface.co/timm/maxvit_large_tf_512.in21k_ft_in1k) |
| `maxvit_nano_rw_256.sw_in1k` | 256 | 82.934 | 256 | 1000 | apache-2.0 | [timm](https://huggingface.co/timm/maxvit_nano_rw_256.sw_in1k) |
| `maxvit_rmlp_base_rw_224.sw_in12k` | 224 | - | - | 11821 | apache-2.0 | [timm](https://huggingface.co/timm/maxvit_rmlp_base_rw_224.sw_in12k) |
| `maxvit_rmlp_base_rw_224.sw_in12k_ft_in1k` | 224 | 86.908 | 224 | 1000 | apache-2.0 | [timm](https://huggingface.co/timm/maxvit_rmlp_base_rw_224.sw_in12k_ft_in1k) |
| `maxvit_rmlp_base_rw_384.sw_in12k_ft_in1k` | 384 | 87.810 | 384 | 1000 | apache-2.0 | [timm](https://huggingface.co/timm/maxvit_rmlp_base_rw_384.sw_in12k_ft_in1k) |
| `maxvit_rmlp_nano_rw_256.sw_in1k` | 256 | 82.978 | 256 | 1000 | apache-2.0 | [timm](https://huggingface.co/timm/maxvit_rmlp_nano_rw_256.sw_in1k) |
| `maxvit_rmlp_pico_rw_256.sw_in1k` | 256 | 80.550 | 256 | 1000 | apache-2.0 | [timm](https://huggingface.co/timm/maxvit_rmlp_pico_rw_256.sw_in1k) |
| `maxvit_rmlp_small_rw_224.sw_in1k` | 224 | 84.488 | 224 | 1000 | apache-2.0 | [timm](https://huggingface.co/timm/maxvit_rmlp_small_rw_224.sw_in1k) |
| `maxvit_rmlp_tiny_rw_256.sw_in1k` | 256 | 84.226 | 256 | 1000 | apache-2.0 | [timm](https://huggingface.co/timm/maxvit_rmlp_tiny_rw_256.sw_in1k) |
| `maxvit_small_tf_224.in1k` | 224 | 84.452 | 224 | 1000 | apache-2.0 | [timm](https://huggingface.co/timm/maxvit_small_tf_224.in1k) |
| `maxvit_small_tf_384.in1k` | 384 | 85.534 | 384 | 1000 | apache-2.0 | [timm](https://huggingface.co/timm/maxvit_small_tf_384.in1k) |
| `maxvit_small_tf_512.in1k` | 512 | 86.102 | 512 | 1000 | apache-2.0 | [timm](https://huggingface.co/timm/maxvit_small_tf_512.in1k) |
| `maxvit_tiny_rw_224.sw_in1k` | 224 | 83.522 | 224 | 1000 | apache-2.0 | [timm](https://huggingface.co/timm/maxvit_tiny_rw_224.sw_in1k) |
| `maxvit_tiny_tf_224.in1k` | 224 | 83.426 | 224 | 1000 | apache-2.0 | [timm](https://huggingface.co/timm/maxvit_tiny_tf_224.in1k) |
| `maxvit_tiny_tf_384.in1k` | 384 | 85.106 | 384 | 1000 | apache-2.0 | [timm](https://huggingface.co/timm/maxvit_tiny_tf_384.in1k) |
| `maxvit_tiny_tf_512.in1k` | 512 | 85.658 | 512 | 1000 | apache-2.0 | [timm](https://huggingface.co/timm/maxvit_tiny_tf_512.in1k) |
| `maxvit_xlarge_tf_224.in21k` | 224 | - | - | 21843 | apache-2.0 | [timm](https://huggingface.co/timm/maxvit_xlarge_tf_224.in21k) |
| `maxvit_xlarge_tf_384.in21k_ft_in1k` | 384 | 88.302 | 384 | 1000 | apache-2.0 | [timm](https://huggingface.co/timm/maxvit_xlarge_tf_384.in21k_ft_in1k) |
| `maxvit_xlarge_tf_512.in21k_ft_in1k` | 512 | 88.540 | 512 | 1000 | apache-2.0 | [timm](https://huggingface.co/timm/maxvit_xlarge_tf_512.in21k_ft_in1k) |
| `maxxvit_rmlp_nano_rw_256.sw_in1k` | 256 | 83.036 | 256 | 1000 | apache-2.0 | [timm](https://huggingface.co/timm/maxxvit_rmlp_nano_rw_256.sw_in1k) |
| `maxxvit_rmlp_small_rw_256.sw_in1k` | 256 | 84.634 | 256 | 1000 | apache-2.0 | [timm](https://huggingface.co/timm/maxxvit_rmlp_small_rw_256.sw_in1k) |
| `maxxvitv2_nano_rw_256.sw_in1k` | 256 | 83.108 | 256 | 1000 | apache-2.0 | [timm](https://huggingface.co/timm/maxxvitv2_nano_rw_256.sw_in1k) |
| `maxxvitv2_rmlp_base_rw_224.sw_in12k` | 224 | - | - | 11821 | apache-2.0 | [timm](https://huggingface.co/timm/maxxvitv2_rmlp_base_rw_224.sw_in12k) |
| `maxxvitv2_rmlp_base_rw_224.sw_in12k_ft_in1k` | 224 | 86.640 | 224 | 1000 | apache-2.0 | [timm](https://huggingface.co/timm/maxxvitv2_rmlp_base_rw_224.sw_in12k_ft_in1k) |
| `maxxvitv2_rmlp_base_rw_384.sw_in12k_ft_in1k` | 384 | 87.480 | 384 | 1000 | apache-2.0 | [timm](https://huggingface.co/timm/maxxvitv2_rmlp_base_rw_384.sw_in12k_ft_in1k) |

## Citation

```bibtex
@article{tu2022maxvit,
  title={MaxViT: Multi-Axis Vision Transformer},
  author={Tu, Zhengzhong and Talebi, Hossein and Zhang, Han and Yang, Feng and Milanfar, Peyman and Bovik, Alan and Li, Yinxiao},
  journal={ECCV},
  year={2022},
}

@article{DBLP:journals/corr/abs-2106-04803,
  author    = {Zihang Dai and Hanxiao Liu and Quoc V. Le and Mingxing Tan},
  title     = {CoAtNet: Marrying Convolution and Attention for All Data Sizes},
  journal   = {CoRR},
  volume    = {abs/2106.04803},
  year      = {2021}
}
```