# FastViT

**Category:** Efficient/Mobile · **Code:**
[`timm/models/fastvit.py`](https://github.com/huggingface/pytorch-image-models/blob/main/timm/models/fastvit.py) ·
**Architectures:** 12 · **Pretrained weights:** 21 (14 with ImageNet-1k results)

FastViT is a hybrid convolutional / transformer network that uses structural reparameterization to remove skip
connections at inference. The module also includes the MobileCLIP image encoders (`fastvit_mci*`).

## Papers

* [FastViT: A Fast Hybrid Vision Transformer using Structural Reparameterization](https://arxiv.org/abs/2303.14189)
* [MobileCLIP: Fast Image-Text Models through Multi-Modal Reinforced Training](https://arxiv.org/abs/2311.17049)
* [MobileCLIP2: Improving Multi-Modal Reinforced Training](https://arxiv.org/abs/2508.20691)

## Usage

See the [Quickstart](../quickstart) for loading pretrained weights, image preprocessing and fine-tuning, and [Feature
Extraction](../feature_extraction) for using the models as backbones.

```py
>>> import timm
>>> model = timm.create_model('fastvit_sa24.apple_dist_in1k', pretrained=True)
>>> data_config = timm.data.resolve_model_data_config(model)
```

## Model API support

| Feature | `FastVit` |
|---|---|
| [`features_only=True` feature maps](../model_api#intermediate-features) | ✓ (strides 4, 8, 16, 32) |
| [`forward_intermediates()`](../model_api#intermediate-features) | ✓ |
| [`prune_intermediate_layers()`](../model_api#intermediate-features) | ✓ |
| [Any input size, no changes needed](../model_api#input-size) | ✓ |
| [`set_input_size()`](../model_api#input-size) | – |
| [`dynamic_img_size=True`](../model_api#input-size) | – |
| [`set_grad_checkpointing()`](../model_api#training-hooks) | ✓ |
| [Layer-wise LR decay (`group_matcher()`)](../model_api#training-hooks) | ✓ |
| [`reset_classifier()`](../model_api#classifier) | ✓ |

✓ supported, ✓ (square) works for square sizes only, ✗ not supported, – not needed (the model already accepts any input
size). Checked by running each feature on a randomly initialized model (`FastVit` with `fastvit_t8` at 256), other
architectures of the same class are expected to behave the same. See the [Model API](../model_api) page for details.

## Architectures

| Architecture | Params (M) | GMACs | Input size | Pretrained weights |
|---|---:|---:|---:|---:|
| `fastvit_ma36` | 43.98 | 7.82 | 256 | 2 |
| `fastvit_mci0` | 11.36 | 2.39 | 256 | 2 |
| `fastvit_mci1` | 21.46 | 4.67 | 256 | 1 |
| `fastvit_mci2` | 35.70 | 7.85 | 256 | 2 |
| `fastvit_mci3` | 125.07 | 14.82 | 256 | 1 |
| `fastvit_mci4` | 321.57 | 27.78 | 256 | 1 |
| `fastvit_s12` | 9.43 | 1.80 | 256 | 2 |
| `fastvit_sa12` | 11.55 | 1.94 | 256 | 2 |
| `fastvit_sa24` | 21.50 | 3.77 | 256 | 2 |
| `fastvit_sa36` | 31.46 | 5.59 | 256 | 2 |
| `fastvit_t8` | 4.00 | 0.69 | 256 | 2 |
| `fastvit_t12` | 7.51 | 1.39 | 256 | 2 |

GMACs and parameter counts are for the default input size and classifier of each architecture, from the [benchmark
results](https://github.com/huggingface/pytorch-image-models/tree/main/results).

## Pretrained weights

Top-1 is ImageNet-1k validation accuracy from
[`results-imagenet.csv`](https://github.com/huggingface/pytorch-image-models/blob/main/results/results-imagenet.csv) at
the listed eval size, - if the weight has no ImageNet-1k result (e.g. pretraining or image-text weights). Classes is the
size of the pretrained classifier, none for weights without one.

Weights are grouped by source: the paper or origin given in the pretrained config, else a Hub organization other than
`timm`, else the first part of the pretrained tag (`architecture.tag`). Sources with a single weight are listed under
other.

### `apple` tags

| Weight | Input size | Top-1 (%) | Eval size | Classes | License | Hub |
|---|---:|---:|---:|---:|---|---|
| `fastvit_ma36.apple_dist_in1k` | 256 | 84.608 | 256 | 1000 | fastvit-license | [timm](https://huggingface.co/timm/fastvit_ma36.apple_dist_in1k) |
| `fastvit_ma36.apple_in1k` | 256 | 83.884 | 256 | 1000 | fastvit-license | [timm](https://huggingface.co/timm/fastvit_ma36.apple_in1k) |
| `fastvit_mci0.apple_mclip2_dfndr2b` | 256 | - | - | 512 | apple-amlr | [timm](https://huggingface.co/timm/fastvit_mci0.apple_mclip2_dfndr2b) |
| `fastvit_mci2.apple_mclip2_dfndr2b` | 256 | - | - | 512 | apple-amlr | [timm](https://huggingface.co/timm/fastvit_mci2.apple_mclip2_dfndr2b) |
| `fastvit_mci3.apple_mclip2_dfndr2b` | 256 | - | - | 768 | apple-amlr | [timm](https://huggingface.co/timm/fastvit_mci3.apple_mclip2_dfndr2b) |
| `fastvit_mci4.apple_mclip2_dfndr2b` | 256 | - | - | 768 | apple-amlr | [timm](https://huggingface.co/timm/fastvit_mci4.apple_mclip2_dfndr2b) |
| `fastvit_s12.apple_dist_in1k` | 256 | 81.074 | 256 | 1000 | fastvit-license | [timm](https://huggingface.co/timm/fastvit_s12.apple_dist_in1k) |
| `fastvit_s12.apple_in1k` | 256 | 79.882 | 256 | 1000 | fastvit-license | [timm](https://huggingface.co/timm/fastvit_s12.apple_in1k) |
| `fastvit_sa12.apple_dist_in1k` | 256 | 81.838 | 256 | 1000 | fastvit-license | [timm](https://huggingface.co/timm/fastvit_sa12.apple_dist_in1k) |
| `fastvit_sa12.apple_in1k` | 256 | 80.846 | 256 | 1000 | fastvit-license | [timm](https://huggingface.co/timm/fastvit_sa12.apple_in1k) |
| `fastvit_sa24.apple_dist_in1k` | 256 | 83.368 | 256 | 1000 | fastvit-license | [timm](https://huggingface.co/timm/fastvit_sa24.apple_dist_in1k) |
| `fastvit_sa24.apple_in1k` | 256 | 82.706 | 256 | 1000 | fastvit-license | [timm](https://huggingface.co/timm/fastvit_sa24.apple_in1k) |
| `fastvit_sa36.apple_dist_in1k` | 256 | 84.028 | 256 | 1000 | fastvit-license | [timm](https://huggingface.co/timm/fastvit_sa36.apple_dist_in1k) |
| `fastvit_sa36.apple_in1k` | 256 | 83.502 | 256 | 1000 | fastvit-license | [timm](https://huggingface.co/timm/fastvit_sa36.apple_in1k) |
| `fastvit_t8.apple_dist_in1k` | 256 | 77.170 | 256 | 1000 | fastvit-license | [timm](https://huggingface.co/timm/fastvit_t8.apple_dist_in1k) |
| `fastvit_t8.apple_in1k` | 256 | 76.184 | 256 | 1000 | fastvit-license | [timm](https://huggingface.co/timm/fastvit_t8.apple_in1k) |
| `fastvit_t12.apple_dist_in1k` | 256 | 80.364 | 256 | 1000 | fastvit-license | [timm](https://huggingface.co/timm/fastvit_t12.apple_dist_in1k) |
| `fastvit_t12.apple_in1k` | 256 | 79.276 | 256 | 1000 | fastvit-license | [timm](https://huggingface.co/timm/fastvit_t12.apple_in1k) |

### apple

Hosted by [`apple`](https://huggingface.co/apple) on the Hugging Face Hub.

| Weight | Input size | Top-1 (%) | Eval size | Classes | License | Hub |
|---|---:|---:|---:|---:|---|---|
| `fastvit_mci0.apple_mclip` | 256 | - | - | 512 | apple-amlr | [apple/mobileclip_s0_timm](https://huggingface.co/apple/mobileclip_s0_timm) |
| `fastvit_mci1.apple_mclip` | 256 | - | - | 512 | apple-amlr | [apple/mobileclip_s1_timm](https://huggingface.co/apple/mobileclip_s1_timm) |
| `fastvit_mci2.apple_mclip` | 256 | - | - | 512 | apple-amlr | [apple/mobileclip_s2_timm](https://huggingface.co/apple/mobileclip_s2_timm) |