repvit / README.md
Flor's picture
card: update from fastforks 1cbf1671
5f56a70 verified
|
Raw History Blame Contribute Delete
13.9 kB
metadata
library_name: fastforks
pipeline_tag: image-classification
license: apache-2.0
tags:
  - classification
  - image-classification
  - cuda
  - pytorch
  - fastforks
  - fastforks-card

RepViT (fastforks)

RepViT (Wang et al. 2024): a mobile CNN with the block layout of a ViT (3x3 depthwise token mixer, 1x1 channel mixer), re-parameterised to plain convolutions at load.

This repo is the model card only: it holds no weights. fastforks downloads the authors' weights from the links below and checks them against the SHA-256 in the model card. Code: https://github.com/kleinicke/fastforks. Kernels for the kernels backend: https://huggingface.co/fastforks/kernels. Original code: https://github.com/huggingface/pytorch-image-models at 0df212b369a5.

Author weight repos: https://huggingface.co/timm/repvit_m0_9.dist_300e_in1k, https://huggingface.co/timm/repvit_m0_9.dist_450e_in1k, https://huggingface.co/timm/repvit_m1_0.dist_300e_in1k, https://huggingface.co/timm/repvit_m1_0.dist_450e_in1k, https://huggingface.co/timm/repvit_m1_1.dist_300e_in1k, https://huggingface.co/timm/repvit_m1_1.dist_450e_in1k, https://huggingface.co/timm/repvit_m1_5.dist_300e_in1k, https://huggingface.co/timm/repvit_m1_5.dist_450e_in1k, https://huggingface.co/timm/repvit_m2_3.dist_300e_in1k, https://huggingface.co/timm/repvit_m2_3.dist_450e_in1k.

Task: classification. Tier: experimental. Outputs: logits. Backends: portable, kernels.

Input: one RGB image. Output: ImageNet-1k class logits.

Variants

Weights are the authors' files (not hosted here). SHA-256 shortened to 12 hex digits; the full hashes are in card.toml.

variant weights (author link) SHA-256 weights licence commercial use
repvit_m0_9.dist_300e_in1k@1 repvit_m0_9.dist_300e_in1k.safetensors 89440fd4da9c Apache-2.0 yes
repvit_m0_9.dist_450e_in1k@1 repvit_m0_9.dist_450e_in1k.safetensors c2dd32622df8 Apache-2.0 yes
repvit_m1_0.dist_300e_in1k@1 repvit_m1_0.dist_300e_in1k.safetensors 792cf8a2e265 Apache-2.0 yes
repvit_m1_0.dist_450e_in1k@1 repvit_m1_0.dist_450e_in1k.safetensors 9350ed15b205 Apache-2.0 yes
repvit_m1_1.dist_300e_in1k@1 repvit_m1_1.dist_300e_in1k.safetensors 0c1a155cef8f Apache-2.0 yes
repvit_m1_1.dist_450e_in1k@1 repvit_m1_1.dist_450e_in1k.safetensors 071d227f6965 Apache-2.0 yes
repvit_m1_5.dist_300e_in1k@1 repvit_m1_5.dist_300e_in1k.safetensors 1df0cc52b089 Apache-2.0 yes
repvit_m1_5.dist_450e_in1k@1 repvit_m1_5.dist_450e_in1k.safetensors 222d8399b87b Apache-2.0 yes
repvit_m2_3.dist_300e_in1k@1 repvit_m2_3.dist_300e_in1k.safetensors 6fb7b94f1607 Apache-2.0 yes
repvit_m2_3.dist_450e_in1k@1 repvit_m2_3.dist_450e_in1k.safetensors 14ae2ea27668 Apache-2.0 yes

Licence

Original-code strings, recorded as they are (research-use package; check them before any other use). Code: Apache-2.0.

Training data: ImageNet-1k (ILSVRC 2012 classification) (ImageNet-1k with distillation from a RegNetY-16GF teacher (RepViT paper), 300 or 450 epochs).

Licence evidence: https://huggingface.co/timm/repvit_m1_1.dist_450e_in1k/blob/ff441038a42982fff1c8c75be7c794f9ba76db72/README.md

Input size

fixed 224x224: the input is resized (short side 235) and centre-cropped to it Tested up to (portable, fp16, RTX 5060 Ti 16 GB): repvit_m0_9.dist_300e_in1k 224x224 (0.0 GiB peak allocated, 0.0 reserved); repvit_m0_9.dist_450e_in1k 224x224 (0.0 GiB peak allocated, 0.0 reserved); repvit_m1_0.dist_300e_in1k 224x224 (0.0 GiB peak allocated, 0.0 reserved); repvit_m1_0.dist_450e_in1k 224x224 (0.0 GiB peak allocated, 0.0 reserved); repvit_m1_1.dist_300e_in1k 224x224 (0.0 GiB peak allocated, 0.1 reserved); repvit_m1_1.dist_450e_in1k 224x224 (0.0 GiB peak allocated, 0.1 reserved); repvit_m1_5.dist_300e_in1k 224x224 (0.1 GiB peak allocated, 0.1 reserved); repvit_m1_5.dist_450e_in1k 224x224 (0.1 GiB peak allocated, 0.1 reserved); repvit_m2_3.dist_300e_in1k 224x224 (0.1 GiB peak allocated, 0.1 reserved); repvit_m2_3.dist_450e_in1k 224x224 (0.1 GiB peak allocated, 0.1 reserved)

Speed and memory

Classification, synthetic

model input time (ms): original fp32 original portable kernels kernels speed-up: vs original vs portable peak memory (MiB, allocated / reserved): original portable kernels
repvit_m0_9.dist_300e_in1k 224x224 8.45 12.7 5.35 0.776 10.9x 6.9x 65 / 84 45 / 76 45 / 76
repvit_m0_9.dist_450e_in1k 224x224 8.55 12.6 5.41 0.767 11.1x 7.1x 65 / 84 45 / 76 45 / 76
repvit_m1_0.dist_300e_in1k 224x224 8.79 12.6 5.29 0.786 11.2x 6.7x 75 / 92 48 / 78 48 / 78
repvit_m1_0.dist_450e_in1k 224x224 8.80 12.6 5.44 0.785 11.2x 6.9x 75 / 92 48 / 78 48 / 78
repvit_m1_1.dist_300e_in1k 224x224 8.14 11.7 5.03 0.765 10.6x 6.6x 84 / 98 51 / 62 51 / 64
repvit_m1_1.dist_450e_in1k 224x224 8.07 11.7 4.99 0.758 10.6x 6.6x 84 / 98 51 / 62 51 / 64
repvit_m1_5.dist_300e_in1k 224x224 13.9 20.4 8.55 1.19 11.8x 7.2x 118 / 124 62 / 94 62 / 94
repvit_m1_5.dist_450e_in1k 224x224 13.9 20.5 8.71 1.19 11.7x 7.3x 118 / 124 62 / 94 62 / 94
repvit_m2_3.dist_300e_in1k 224x224 17.9 26.1 11.1 1.66 10.8x 6.7x 170 / 184 80 / 128 79 / 130
repvit_m2_3.dist_450e_in1k 224x224 18.2 26.1 10.9 1.70 10.8x 6.5x 170 / 184 80 / 128 79 / 130

Startup and first call

model load (s): original portable kernels first call (s): original portable kernels
repvit_m0_9.dist_300e_in1k 0.20 1.65 2.01 2.43 0.28 0.08
repvit_m0_9.dist_450e_in1k 0.19 1.65 1.96 0.01 0.27 0.05
repvit_m1_0.dist_300e_in1k 0.20 1.64 2.00 1.44 0.28 0.05
repvit_m1_0.dist_450e_in1k 0.20 1.74 1.96 0.01 0.27 0.05
repvit_m1_1.dist_300e_in1k 0.20 1.86 1.97 1.36 0.28 0.06
repvit_m1_1.dist_450e_in1k 0.21 1.89 2.35 0.01 0.27 0.05
repvit_m1_5.dist_300e_in1k 0.26 1.84 2.63 0.02 0.27 0.05
repvit_m1_5.dist_450e_in1k 0.25 1.86 2.21 0.02 0.28 0.05
repvit_m2_3.dist_300e_in1k 0.33 2.00 2.41 1.54 0.28 0.05
repvit_m2_3.dist_450e_in1k 0.34 2.01 2.64 0.03 0.27 0.06

torch.compile and TensorRT

model input time: portable compile TensorRT fp16 kernels preparation: compile (cold first call) ONNX + engine build peak MiB (alloc / reserved): portable compile kernels TensorRT (engine + torch)
repvit_m0_9.dist_300e_in1k 224x224 5.35 ms 3.30 ms 0.509 ms 0.776 ms 8.62 s 33.7 s 45 / 76 77 / 96 45 / 76 75 / 108
repvit_m0_9.dist_450e_in1k 224x224 5.41 ms 3.34 ms 0.489 ms 0.767 ms 8.11 s 30.5 s 45 / 76 77 / 96 45 / 76 77 / 110
repvit_m1_0.dist_300e_in1k 224x224 5.29 ms 3.33 ms 0.503 ms 0.786 ms 8.20 s 35.2 s 48 / 78 80 / 118 48 / 78 78 / 110
repvit_m1_0.dist_450e_in1k 224x224 5.44 ms 3.21 ms 0.477 ms 0.785 ms 8.29 s 40.2 s 48 / 78 80 / 118 48 / 78 80 / 112
repvit_m1_1.dist_300e_in1k 224x224 5.03 ms 6.12 ms 0.470 ms 0.765 ms 7.72 s 33.0 s 51 / 62 83 / 102 51 / 64 83 / 96
repvit_m1_1.dist_450e_in1k 224x224 4.99 ms 6.07 ms 0.478 ms 0.758 ms 7.57 s 33.4 s 51 / 62 83 / 102 51 / 64 83 / 96
repvit_m1_5.dist_300e_in1k 224x224 8.55 ms 4.89 ms 0.766 ms 1.19 ms 10.2 s 38.9 s 62 / 94 94 / 114 62 / 94 96 / 130
repvit_m1_5.dist_450e_in1k 224x224 8.71 ms 4.93 ms 0.760 ms 1.19 ms 10.1 s 33.8 s 62 / 94 94 / 114 62 / 94 94 / 128
repvit_m2_3.dist_300e_in1k 224x224 11.1 ms 6.51 ms 1.03 ms 1.66 ms 11.9 s 41.2 s 80 / 128 112 / 148 79 / 130 149 / 200
repvit_m2_3.dist_450e_in1k 224x224 10.9 ms 6.49 ms 1.03 ms 1.70 ms 11.8 s 40.8 s 80 / 128 112 / 148 79 / 130 149 / 200

Fixed-size cells: time (ms) / peak allocated / peak reserved memory (MiB).

Hardware, protocol and caveats: benchmarks (RTX 5060 Ti 16 GB, sm_120). Original = the original code named above at the precision of portable (fp16, or (fp32) where fp16 is unsupported or not finite); original fp32 = its default fp32 run. Peak memory is the torch allocator's.

Accuracy

On public datasets

fastforks: the model's own resize / crop on the GPU; original: the library's pretrained loader and its PIL eval transform. Original = the original code named under Details, same checkpoint and protocol.

variant dataset original fp32 original fp16 portable kernels note
repvit_m0_9.dist_300e_in1k ImageNet-1k val, 5000 random images, top-1 / top-5 % 79.44 / 94.20 79.48 / 94.16 79.44 / 94.08 79.42 / 94.08
repvit_m0_9.dist_450e_in1k ImageNet-1k val, 5000 random images, top-1 / top-5 % 79.52 / 94.22 79.56 / 94.26 79.52 / 94.24 79.52 / 94.24
repvit_m1_0.dist_300e_in1k ImageNet-1k val, 5000 random images, top-1 / top-5 % 81.12 / 94.78 81.12 / 94.76 81.10 / 94.74 81.10 / 94.80
repvit_m1_0.dist_450e_in1k ImageNet-1k val, 5000 random images, top-1 / top-5 % 81.10 / 94.86 81.10 / 94.88 81.16 / 94.90 81.14 / 94.92
repvit_m1_1.dist_300e_in1k ImageNet-1k val, 5000 random images, top-1 / top-5 % 81.40 / 95.10 81.40 / 95.10 81.32 / 95.02 81.34 / 95.02
repvit_m1_1.dist_450e_in1k ImageNet-1k val, 5000 random images, top-1 / top-5 % 82.32 / 95.70 82.28 / 95.72 82.18 / 95.66 82.18 / 95.62
repvit_m1_5.dist_300e_in1k ImageNet-1k val, 5000 random images, top-1 / top-5 % 82.94 / 96.22 82.98 / 96.24 82.96 / 96.24 83.00 / 96.26
repvit_m1_5.dist_450e_in1k ImageNet-1k val, 5000 random images, top-1 / top-5 % 83.18 / 96.38 83.20 / 96.38 83.24 / 96.36 83.24 / 96.36
repvit_m2_3.dist_300e_in1k ImageNet-1k val, 5000 random images, top-1 / top-5 % 84.12 / 96.54 84.14 / 96.52 84.22 / 96.54 84.20 / 96.54
repvit_m2_3.dist_450e_in1k ImageNet-1k val, 5000 random images, top-1 / top-5 % 84.30 / 96.58 84.30 / 96.60 84.26 / 96.62 84.20 / 96.62

Measured on NVIDIA GeForce RTX 5060 Ti, torch 2.14.1+cu130, fastforks 8b2f2df5, 2026-11-10.

Parity with the original code

Every variant is checked against reference outputs of the original code at its pinned commit. CPU, float32: max |diff| <= 2.5e-05 vs the original. GPU, float16 (the default), portable and kernels: mean |diff| <= 0.0094 and p99 <= 0.036 (absolute error of the output logits).

Limitations

  • The older aliases repvit_m1 / repvit_m2 / repvit_m3 (.dist_in1k) that timm also lists are not shipped.

Usage

from fastforks import classification

model = classification.load("repvit_m0_9.dist_300e_in1k")  # mode="auto": the preferred backend from recorded measurements
result = model("photo.jpg")
result.top(5)  # top-5 classes
result.backend  # what ran, e.g. Backend(mode='portable', device='cuda:0', dtype='float16')

Install with pip install fastforks (see https://github.com/kleinicke/fastforks). Other variants: repvit_m0_9.dist_300e_in1k, repvit_m0_9.dist_450e_in1k, repvit_m1_0.dist_300e_in1k, repvit_m1_0.dist_450e_in1k, repvit_m1_1.dist_300e_in1k, repvit_m1_1.dist_450e_in1k, repvit_m1_5.dist_300e_in1k, repvit_m1_5.dist_450e_in1k, repvit_m2_3.dist_300e_in1k, repvit_m2_3.dist_450e_in1k.

Kernels backend

classification.load("repvit_m0_9.dist_300e_in1k", mode="kernels") runs our CUDA kernels (cudnn_hacc, fvm_conv, fvm_extra, fvm_mb, fvm_r3). It needs a prebuilt kernel for your GPU architecture (downloaded from https://huggingface.co/fastforks/kernels on first use) and a recent NVIDIA driver; if there is none it raises BackendUnavailableError with the reason and never falls back silently. mode="auto" picks kernels when it can run and logs the choice.