LW-DETR (ViT, tiny / small / medium / large / xlarge) (fastforks)
LW-DETR (Chen et al. 2024, Baidu): real-time end-to-end detection with a plain ViT encoder (interleaved window and global attention), a C2f projector and a 3-layer deformable DETR decoder over the encoder's best proposals (no NMS); boxes, scores and COCO classes at the input size. Images are resized to 640x640 (aspect ratio ignored). The portable module() is the whole network at its input size (class logits and normalised boxes).
This repo is the model card only: it holds no weights. fastforks downloads the authors' weights from the links below and checks them against the SHA-256 in the model card. Code: https://github.com/kleinicke/fastforks. Kernels for the kernels backend: https://huggingface.co/fastforks/kernels. Original code: https://github.com/huggingface/transformers at c4c4605f9618.
Author weight repos: https://huggingface.co/AnnaZhang/lwdetr_large_60e_coco, https://huggingface.co/AnnaZhang/lwdetr_medium_60e_coco, https://huggingface.co/AnnaZhang/lwdetr_small_60e_coco, https://huggingface.co/AnnaZhang/lwdetr_tiny_60e_coco, https://huggingface.co/AnnaZhang/lwdetr_xlarge_60e_coco.
Task: detection. Tier: experimental. Outputs: boxes, scores, labels. Backends: portable, kernels.
Input: one RGB image. Output: boxes, class labels, scores and instance masks.
Variants
Weights are the authors' files (not hosted here). SHA-256 shortened to 12 hex digits; the full hashes are in card.toml.
| variant | weights (author link) | SHA-256 | weights licence | commercial use |
|---|---|---|---|---|
lw-detr-large-coco@1 |
lwdetr-large.config.json lwdetr-large.model.safetensors |
4ce0d981459dc9d4a1066e29 |
Apache-2.0 | yes |
lw-detr-medium-coco@1 |
lwdetr-medium.config.json lwdetr-medium.model.safetensors |
d04681b8a5781707a8b1112d |
Apache-2.0 | yes |
lw-detr-small-coco@1 |
lwdetr-small.config.json lwdetr-small.model.safetensors |
0fda7252545d152597c899b3 |
Apache-2.0 | yes |
lw-detr-tiny-coco@1 |
lwdetr-tiny.config.json lwdetr-tiny.model.safetensors |
71b79e99f9bf42dece9592fe |
Apache-2.0 | yes |
lw-detr-xlarge-coco@1 |
lwdetr-xlarge.config.json lwdetr-xlarge.model.safetensors |
a2d2679802fa2918313e5081 |
Apache-2.0 | yes |
Licence
Original-code strings, recorded as they are (research-use package; check them before any other use). Code: Apache-2.0.
Training data: COCO 2017 (train, instance annotations), Objects365 (train, box annotations) (COCO 2017 train after detection pretraining on Objects365; the ViT encoders start from CAEv2 self-supervised pretraining (paper arXiv:2406.03459 section 4).).
Licence evidence: https://huggingface.co/AnnaZhang/lwdetr_tiny_60e_coco/blob/4b636b514dcf623f6eafc9e1ab63b8ad5c513925/README.md
Input size
any H x W >= 32, padded to a multiple of 32
Tested up to (portable, fp16, RTX 5060 Ti 16 GB): lw-detr-large-coco 3000x4000 (0.5 GiB peak allocated, 0.5 reserved); lw-detr-medium-coco 3000x4000 (0.5 GiB peak allocated, 0.5 reserved); lw-detr-small-coco 3000x4000 (0.5 GiB peak allocated, 0.5 reserved); lw-detr-tiny-coco 3000x4000 (0.5 GiB peak allocated, 0.5 reserved); lw-detr-xlarge-coco 3000x4000 (0.7 GiB peak allocated, 0.7 reserved)
Speed and memory
Detection, synthetic, 800x1344
| model | time (ms): original fp32 | original | portable | kernels | kernels speed-up: vs original | vs portable | peak memory (MiB, allocated / reserved): original | portable | kernels |
|---|---|---|---|---|---|---|---|---|---|
lw-detr-large-coco |
β | 22.8 | 10.2 | 5.64 | 4.0x | 1.8x | 355 / 382 | 204 / 230 | 192 / 254 |
lw-detr-medium-coco |
β | 23.3 | 8.86 | 3.83 | 6.1x | 2.3x | 228 / 274 | 128 / 136 | 128 / 180 |
lw-detr-small-coco |
β | 19.9 | 8.82 | 2.80 | 7.1x | 3.1x | 129 / 166 | 97 / 116 | 99 / 140 |
lw-detr-tiny-coco |
β | 16.4 | 7.45 | 2.20 | 7.5x | 3.4x | 114 / 132 | 92 / 112 | 94 / 134 |
lw-detr-xlarge-coco |
β | 25.5 | 12.1 | 9.44 | 2.7x | 1.3x | 762 / 850 | 344 / 360 | 333 / 408 |
Startup and first call
| model | load (s): original | portable | kernels | first call (s): original | portable | kernels |
|---|---|---|---|---|---|---|
lw-detr-large-coco |
3.82 | 3.20 | 3.04 | 0.72 | 0.36 | 0.34 |
lw-detr-medium-coco |
3.91 | 2.73 | 2.93 | 0.64 | 0.35 | 0.33 |
lw-detr-small-coco |
4.14 | 2.72 | 2.91 | 0.63 | 0.35 | 0.33 |
lw-detr-tiny-coco |
3.80 | 2.69 | 2.91 | 0.64 | 0.35 | 0.33 |
lw-detr-xlarge-coco |
4.72 | 2.91 | 3.05 | 0.71 | 0.37 | 0.34 |
torch.compile and TensorRT
| model | input | time: portable | compile | TensorRT fp16 | kernels | preparation: compile (cold first call) | ONNX + engine build | peak MiB (alloc / reserved): portable | compile | kernels | TensorRT (engine + torch) |
|---|---|---|---|---|---|---|---|---|---|---|---|
lw-detr-large-coco |
800x1344 | 10.2 ms | 5.62 ms | 3.45 ms | 5.64 ms | 29.6 s | 56.0 s | 204 / 230 | 195 / 208 | 192 / 254 | 270 / 296 |
lw-detr-medium-coco |
800x1344 | 8.86 ms | 3.74 ms | 2.30 ms | 3.83 ms | 22.4 s | 63.3 s | 128 / 136 | 147 / 174 | 128 / 180 | 185 / 212 |
lw-detr-small-coco |
800x1344 | 8.82 ms | 3.63 ms | 1.36 ms | 2.80 ms | 23.0 s | 63.1 s | 97 / 116 | 116 / 134 | 99 / 140 | 121 / 130 |
lw-detr-tiny-coco |
800x1344 | 7.45 ms | 2.98 ms | 0.999 ms | 2.20 ms | 21.6 s | 60.3 s | 92 / 112 | 112 / 130 | 94 / 134 | 116 / 128 |
lw-detr-xlarge-coco |
800x1344 | 12.1 ms | 10.7 ms | 6.58 ms | 9.44 ms | 30.5 s | 57.5 s | 344 / 360 | 339 / 358 | 333 / 408 | 544 / 566 |
Fixed-size cells: time (ms) / peak allocated / peak reserved memory (MiB).
Hardware, protocol and caveats: benchmarks (RTX 5060 Ti 16 GB, sm_120). Original = the original code named above at the precision of portable (fp16, or (fp32) where fp16 is unsupported or not finite); original fp32 = its default fp32 run. Peak memory is the torch allocator's.
Accuracy
On public datasets
pycocotools COCOeval of the top 100 detections per image. Original = the original code named under Details, same checkpoint and protocol.
| variant | dataset | original fp32 | portable | kernels | note |
|---|---|---|---|---|---|
lw-detr-tiny-coco |
COCO val2017, every 20th image, box mAP | 49.05 | 48.81 | 48.99 | |
lw-detr-large-coco |
COCO val2017, every 20th image, box mAP | 60.67 | 61.01 | 60.74 |
Measured on NVIDIA GeForce RTX 5060 Ti, torch 2.14.1+cu130, fastforks aee4b565, 2026-11-13.
Parity with the original code
Every variant is checked against reference outputs of the original code at its pinned commit. CPU, float32: max |diff| <= 0.002 vs the original. GPU, float16 (the default), portable and kernels: mean |diff| <= 0.28 and p99 <= 0.93 (absolute error of the output boxes).
Limitations
- Requires the optional transformers library (the network is its
LwDetrForObjectDetection). - The original in the benchmarks and goldens is the Hugging Face transformers 5.19.0 port (AnnaZhang/lwdetr_*_60e_coco weights) of the official LW-DETR (Apache-2.0); the official repository code itself was not run.
Usage
from fastforks import detection
model = detection.load("lw-detr-large-coco") # mode="auto": the preferred backend from recorded measurements
result = model("photo.jpg")
result.boxes, result.scores, result.labels # [N, 4], [N], [N]
result.backend # what ran, e.g. Backend(mode='portable', device='cuda:0', dtype='float16')
Install with pip install fastforks (see https://github.com/kleinicke/fastforks). Other variants: lw-detr-large-coco, lw-detr-medium-coco, lw-detr-small-coco, lw-detr-tiny-coco, lw-detr-xlarge-coco.
Kernels backend
detection.load("lw-detr-large-coco", mode="kernels") runs our CUDA kernels (cudnn_hacc). It needs a prebuilt kernel for your GPU architecture (downloaded from https://huggingface.co/fastforks/kernels on first use) and a recent NVIDIA driver; if there is none it raises BackendUnavailableError with the reason and never falls back silently. mode="auto" picks kernels when it can run and logs the choice.