ConvNeXt (fastforks)
ConvNeXt (Liu et al. 2022) as a feature backbone: Meta's ImageNet-22k checkpoints in timm's format (tiny, small, base, large). last_hidden_state = the stride-32 stage-4 map as tokens (no prefix token, un-normalised, as timm's forward_features / features_only and HF ConvNextModel); pooler_output = the head LayerNorm of their mean (timm's pre-logits embedding).
This repo is the model card only: it holds no weights. fastforks downloads the authors' weights from the links below and checks them against the SHA-256 in the model card. Code: https://github.com/kleinicke/fastforks. Kernels for the kernels backend: https://huggingface.co/fastforks/kernels. Original code: https://github.com/huggingface/pytorch-image-models at 0df212b369a5.
Author weight repos: https://huggingface.co/timm/convnext_base.fb_in22k, https://huggingface.co/timm/convnext_large.fb_in22k, https://huggingface.co/timm/convnext_small.fb_in22k, https://huggingface.co/timm/convnext_tiny.fb_in22k.
Task: features. Tier: experimental. Outputs: last_hidden_state, pooler_output. Backends: portable, kernels.
Input: one RGB image. Output: dense patch features and a pooled image feature.
Variants
Weights are the authors' files (not hosted here). SHA-256 shortened to 12 hex digits; the full hashes are in card.toml.
| variant | weights (author link) | SHA-256 | weights licence | commercial use |
|---|---|---|---|---|
convnext_base.fb_in22k@1 |
convnext_base.fb_in22k.safetensors | 0f09df05894c |
Apache-2.0 | yes |
convnext_large.fb_in22k@1 |
convnext_large.fb_in22k.safetensors | 7e02ea57d602 |
Apache-2.0 | yes |
convnext_small.fb_in22k@1 |
convnext_small.fb_in22k.safetensors | 73a192db2a11 |
Apache-2.0 | yes |
convnext_tiny.fb_in22k@1 |
convnext_tiny.fb_in22k.safetensors | 2b40f8d53b56 |
Apache-2.0 | yes |
Licence
Original-code strings, recorded as they are (research-use package; check them before any other use). Code: Apache-2.0.
Training data: ImageNet-21k / ImageNet-22k (full ImageNet, fall 2011 release).
Licence evidence: https://huggingface.co/timm/convnext_base.fb_in22k/blob/9afd4146adabf1b6e861d0f1ac2e7baa7c8dd1b7/README.md
Input size
any H x W >= 1, resized or padded inside the model
Tested up to (portable, fp16, RTX 5060 Ti 16 GB): convnext_base.fb_in22k 3000x4000 (2.5 GiB peak allocated, 3.6 reserved); convnext_large.fb_in22k 3000x4000 (3.8 GiB peak allocated, 5.2 reserved); convnext_small.fb_in22k 3000x4000 (1.9 GiB peak allocated, 2.5 reserved); convnext_tiny.fb_in22k 3000x4000 (1.9 GiB peak allocated, 2.4 reserved)
Speed and memory
Features, synthetic, 224x224
| model | time (ms): original fp32 | original | portable | kernels | kernels speed-up: vs original | vs portable | peak memory (MiB, allocated / reserved): original | portable | kernels |
|---|---|---|---|---|---|---|---|---|---|
convnext_base.fb_in22k |
7.68 | 11.2 | 5.57 | 1.96 | 3.9x | 2.8x | 537 / 544 | 289 / 296 | 204 / 216 |
convnext_large.fb_in22k |
10.5 | 11.5 | 5.53 | 3.23 | 3.3x | 1.7x | 1160 / 1180 | 600 / 612 | 413 / 434 |
convnext_small.fb_in22k |
7.51 | 11.2 | 5.64 | 1.79 | 4.2x | 3.2x | 318 / 336 | 179 / 190 | 130 / 150 |
convnext_tiny.fb_in22k |
3.89 | 5.91 | 3.18 | 1.09 | 3.6x | 2.9x | 194 / 214 | 117 / 130 | 88 / 110 |
Startup and first call
| model | load (s): original | portable | kernels | first call (s): original | portable | kernels |
|---|---|---|---|---|---|---|
convnext_base.fb_in22k |
0.90 | 2.18 | 2.05 | 0.02 | 0.24 | 0.24 |
convnext_large.fb_in22k |
2.17 | 2.71 | 2.78 | 0.02 | 0.24 | 0.23 |
convnext_small.fb_in22k |
0.65 | 1.93 | 1.73 | 0.01 | 0.21 | 0.18 |
convnext_tiny.fb_in22k |
0.40 | 1.66 | 1.57 | 0.28 | 0.23 | 0.22 |
torch.compile and TensorRT
| model | input | time: portable | compile | TensorRT fp16 | kernels | preparation: compile (cold first call) | ONNX + engine build | peak MiB (alloc / reserved): portable | compile | kernels | TensorRT (engine + torch) |
|---|---|---|---|---|---|---|---|---|---|---|---|
convnext_base.fb_in22k |
224x224 | 5.57 ms | 3.21 ms | 1.51 ms | 1.96 ms | 10.2 s | 29.9 s | 289 / 296 | 321 / 334 | 204 / 216 | 480 / 494 |
convnext_large.fb_in22k |
224x224 | 5.53 ms | 3.34 ms | 2.50 ms | 3.23 ms | 9.87 s | 46.7 s | 600 / 612 | 639 / 650 | 413 / 434 | 983 / 1006 |
convnext_small.fb_in22k |
224x224 | 5.64 ms | 3.36 ms | 1.16 ms | 1.79 ms | 9.51 s | 35.1 s | 179 / 190 | 210 / 230 | 130 / 150 | 277 / 292 |
convnext_tiny.fb_in22k |
224x224 | 3.18 ms | 2.08 ms | 0.726 ms | 1.09 ms | 12.0 s | 33.9 s | 117 / 130 | 148 / 170 | 88 / 110 | 183 / 200 |
Fixed-size cells: time (ms) / peak allocated / peak reserved memory (MiB).
Hardware, protocol and caveats: benchmarks (RTX 5060 Ti 16 GB, sm_120). Original = the original code named above at the precision of portable (fp16, or (fp32) where fp16 is unsupported or not finite); original fp32 = its default fp32 run. Peak memory is the torch allocator's.
Accuracy
On public datasets
Two disjoint seeded subsets of the val folders; each image shorter side 256 (bicubic), centre crop 224, the CLS / pooled feature; cosine kNN, vote weighted by similarity. Original = the original code named under Details, same checkpoint and protocol.
| variant | dataset | original fp32 | original fp16 | portable | kernels | note |
|---|---|---|---|---|---|---|
convnext_tiny.fb_in22k |
ImageNet-1k val, kNN top-1 % (k = 20, 10000 labelled / 5000 test images) | 66.80 | 66.88 | 66.82 | 66.84 | |
convnext_small.fb_in22k |
ImageNet-1k val, kNN top-1 % (k = 20, 10000 labelled / 5000 test images) | 71.68 | 71.68 | 71.72 | 71.72 | |
convnext_base.fb_in22k |
ImageNet-1k val, kNN top-1 % (k = 20, 10000 labelled / 5000 test images) | 73.04 | 73.04 | 73.08 | 73.04 | |
convnext_large.fb_in22k |
ImageNet-1k val, kNN top-1 % (k = 20, 10000 labelled / 5000 test images) | 75.26 | 75.22 | 75.28 | 75.24 |
Measured on NVIDIA GeForce RTX 5060 Ti, torch 2.14.1+cu130, fastforks 8b2f2df5, 2026-11-10.
Parity with the original code
Every variant is checked against reference outputs of the original code at its pinned commit. CPU, float32: max |diff| <= 8.1e-05 vs the original. GPU, float16 (the default), portable and kernels: mean |diff| <= 1.6 % and p99 <= 9.4 % of the reference's median |value| (absolute error of the output last_hidden_state).
Limitations
- Only the last stage's map is returned: the maps of stages 1-3 (strides 4, 8, 16; timm
features_only=True) have other sizes than the token grid of this task's outputs. - kernels replays a CUDA graph per small input shape (up to 2^20 input pixels per call), captured on the shape's second call, at most 4 per model;
model.release_graphs()frees them.
Usage
from fastforks import features
model = features.load("convnext_base.fb_in22k") # mode="auto": the preferred backend from recorded measurements
result = model("photo.jpg", size=518)
result.last_hidden_state # tokens [N, C], as Hugging Face's output
result.backend # what ran, e.g. Backend(mode='portable', device='cuda:0', dtype='float16')
Install with pip install fastforks (see https://github.com/kleinicke/fastforks). Other variants: convnext_base.fb_in22k, convnext_large.fb_in22k, convnext_small.fb_in22k, convnext_tiny.fb_in22k.
Kernels backend
features.load("convnext_base.fb_in22k", mode="kernels") runs our CUDA kernels (fvm_conv, fvm_seg). It needs a prebuilt kernel for your GPU architecture (downloaded from https://huggingface.co/fastforks/kernels on first use) and a recent NVIDIA driver; if there is none it raises BackendUnavailableError with the reason and never falls back silently. mode="auto" picks kernels when it can run and logs the choice.