dfine / README.md
Flor's picture
card: update from fastforks 1cbf1671
c3013a9 verified
|
Raw History Blame Contribute Delete
12.7 kB
---
library_name: "fastforks"
pipeline_tag: "object-detection"
license: "apache-2.0"
tags:
- "detection"
- "object-detection"
- "instance-segmentation"
- "cuda"
- "pytorch"
- "fastforks"
- "fastforks-card"
---
# D-FINE (HGNetV2, N / S / M / L / X) (fastforks)
D-FINE (Peng et al. 2025, USTC): real-time end-to-end detection like RT-DETR (HGNetV2 backbone, hybrid encoder, deformable-attention decoder over 300 selected queries, no NMS) whose decoder layers refine every box edge as a distribution over distances (fine-grained distribution refinement); boxes, scores and COCO classes at the input size. The portable `module()` is the whole network at 640x640 (class logits and normalised boxes).
This repo is the model card only: it holds no weights. fastforks downloads the authors' weights from the links below and checks them against the SHA-256 in the model card. Code: <https://github.com/kleinicke/fastforks>. Kernels for the `kernels` backend: <https://huggingface.co/fastforks/kernels>. Original code: <https://github.com/Peterande/D-FINE> at `956d1709314c`.
Author weight repos: <https://huggingface.co/ustc-community/dfine-large-coco>, <https://huggingface.co/ustc-community/dfine-medium-coco>, <https://huggingface.co/ustc-community/dfine-nano-coco>, <https://huggingface.co/ustc-community/dfine-small-coco>, <https://huggingface.co/ustc-community/dfine-xlarge-coco>.
Task: detection. Tier: stable. Outputs: `boxes`, `scores`, `labels`. Backends: `portable`, `kernels`.
Input: one RGB image. Output: boxes, class labels, scores and instance masks.
## Variants
Weights are the authors' files (not hosted here). SHA-256 shortened to 12 hex digits; the full hashes are in [`card.toml`](https://github.com/kleinicke/fastforks/blob/main/src/fastforks/detection/models/dfine/card.toml).
| variant | weights (author link) | SHA-256 | weights licence | commercial use |
|---|---|---|---|---|
| `deim-l-coco@1` | [deim_dfine_hgnetv2_l_coco.pth](https://drive.usercontent.google.com/download?id=1PIRf02XkrA2xAD3wEiKE2FaamZgSGTAr&export=download&confirm=t) | `a9ac1f9bf51a` | Apache-2.0 (assumed from code) | yes |
| `deim-m-coco@1` | [deim_dfine_hgnetv2_m_coco.pth](https://drive.usercontent.google.com/download?id=18Lj2a6UN6k_n_UzqnJyiaiLGpDzQQit8&export=download&confirm=t) | `2b6cd0582a4a` | Apache-2.0 (assumed from code) | yes |
| `deim-n-coco@1` | [deim_dfine_hgnetv2_n_coco.pth](https://drive.usercontent.google.com/download?id=1ZPEhiU9nhW4M5jLnYOFwTSLQC1Ugf62e&export=download&confirm=t) | `8536a97a57ba` | Apache-2.0 (assumed from code) | yes |
| `deim-s-coco@1` | [deim_dfine_hgnetv2_s_coco.pth](https://drive.usercontent.google.com/download?id=1tB8gVJNrfb6dhFvoHJECKOF5VpkthhfC&export=download&confirm=t) | `a089c9eb798f` | Apache-2.0 (assumed from code) | yes |
| `deim-x-coco@1` | [deim_dfine_hgnetv2_x_coco.pth](https://drive.usercontent.google.com/download?id=1dPtbgtGgq1Oa7k_LgH1GXPelg1IVeu0j&export=download&confirm=t) | `e729436f96d9` | Apache-2.0 (assumed from code) | yes |
| `dfine-l-coco@1` | [dfine-large-coco.safetensors](https://huggingface.co/ustc-community/dfine-large-coco/resolve/aa8f8211545e4cff5adf52ea512f9ac74a9824ec/model.safetensors) | `4e9d77ba937a` | Apache-2.0 | yes |
| `dfine-m-coco@1` | [dfine-medium-coco.safetensors](https://huggingface.co/ustc-community/dfine-medium-coco/resolve/4ab9f5e466432f4fcf9a9a023da6aa5ecc9c9829/model.safetensors) | `5a53154fb87a` | Apache-2.0 | yes |
| `dfine-n-coco@1` | [dfine-nano-coco.safetensors](https://huggingface.co/ustc-community/dfine-nano-coco/resolve/066438d3d8f0da137a37b38fdf3368fd4afceced/model.safetensors) | `19e06bdc873d` | Apache-2.0 | yes |
| `dfine-s-coco@1` | [dfine-small-coco.safetensors](https://huggingface.co/ustc-community/dfine-small-coco/resolve/f79e65b5fbb33ceb9d3ebba042955d7410c608f8/model.safetensors) | `a144baff3a27` | Apache-2.0 | yes |
| `dfine-x-coco@1` | [dfine-xlarge-coco.safetensors](https://huggingface.co/ustc-community/dfine-xlarge-coco/resolve/ea4f6be7350bbe3c199ec6febc74168346cb5a68/model.safetensors) | `3ac6ea37c6c9` | Apache-2.0 | yes |
## Licence
Original-code strings, recorded as they are (research-use package; check them before any other use). Code: Apache-2.0. Licence files: `LICENSE`, `LICENSE-DEIM`.
Training data: COCO 2017 (train, instance annotations), ImageNet-1k (ILSVRC 2012 classification) (COCO 2017 train; the HGNetV2 backbones start from ImageNet-1k pretraining).
Licence evidence: <https://github.com/Peterande/D-FINE/blob/956d1709314c2c6a4df6f34de232054578a7449f/LICENSE>
## Input size
any H x W >= 32, padded to a multiple of 32
Tested up to (portable, fp16, RTX 5060 Ti 16 GB): `deim-l-coco` 3000x4000 (0.5 GiB peak allocated, 0.6 reserved); `deim-m-coco` 3000x4000 (0.5 GiB peak allocated, 0.5 reserved); `deim-n-coco` 3000x4000 (0.5 GiB peak allocated, 0.5 reserved); `deim-s-coco` 3000x4000 (0.5 GiB peak allocated, 0.5 reserved); `deim-x-coco` 3000x4000 (0.6 GiB peak allocated, 0.6 reserved); `dfine-l-coco` 3000x4000 (0.5 GiB peak allocated, 0.7 reserved); `dfine-m-coco` 3000x4000 (0.5 GiB peak allocated, 0.6 reserved); `dfine-n-coco` 3000x4000 (0.5 GiB peak allocated, 0.6 reserved); `dfine-s-coco` 3000x4000 (0.5 GiB peak allocated, 0.6 reserved); `dfine-x-coco` 3000x4000 (0.6 GiB peak allocated, 0.7 reserved)
## Speed and memory
### Detection, synthetic, 800x1344
| model | time (ms): original fp32 | original | portable | kernels | kernels speed-up: vs original | vs portable | peak memory (MiB, allocated / reserved): original | portable | kernels |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| `deim-l-coco` | – | 36.1 | 16.2 | 6.60 | 5.5x | 2.4x | 264 / 322 | 152 / 172 | 149 / 220 |
| `deim-m-coco` | – | 31.4 | 13.6 | 5.17 | 6.1x | 2.6x | 210 / 250 | 126 / 142 | 127 / 192 |
| `deim-n-coco` | – | 24.6 | 10.2 | 2.50 | 9.8x | 4.1x | 93 / 124 | 88 / 116 | 93 / 122 |
| `deim-s-coco` | – | 26.4 | 11.4 | 3.79 | 7.0x | 3.0x | 154 / 180 | 105 / 128 | 106 / 172 |
| `deim-x-coco` | – | 41.3 | 18.6 | 9.65 | 4.3x | 1.9x | 462 / 502 | 227 / 258 | 228 / 304 |
| `dfine-l-coco` | 37.1 | 44.9 | 16.2 | 6.64 | 5.6x | 2.4x | 278 / 350 | 152 / 172 | 149 / 220 |
| `dfine-m-coco` | 30.8 | 38.2 | 13.5 | 5.16 | 6.0x | 2.6x | 209 / 256 | 126 / 142 | 127 / 192 |
| `dfine-n-coco` | 24.3 | 29.1 | 10.3 | 2.52 | 9.7x | 4.1x | 92 / 126 | 88 / 116 | 93 / 122 |
| `dfine-s-coco` | 25.7 | 31.9 | 11.1 | 3.83 | 6.7x | 2.9x | 152 / 184 | 105 / 128 | 106 / 172 |
| `dfine-x-coco` | 44.9 | 54.6 | 18.5 | 9.71 | 4.6x | 1.9x | 476 / 536 | 227 / 258 | 228 / 304 |
### Startup and first call
| model | load (s): original | portable | kernels | first call (s): original | portable | kernels |
| --- | --- | --- | --- | --- | --- | --- |
| `deim-l-coco` | 2.34 | 1.57 | 1.36 | 0.52 | 0.69 | 0.48 |
| `deim-m-coco` | 2.24 | 1.31 | 1.31 | 0.58 | 0.72 | 0.55 |
| `deim-n-coco` | 2.39 | 1.61 | 1.25 | 0.62 | 1.35 | 0.58 |
| `deim-s-coco` | 2.08 | 1.05 | 1.54 | 0.53 | 0.71 | 0.57 |
| `deim-x-coco` | 2.63 | 1.57 | 1.52 | 0.48 | 0.65 | 0.44 |
| `dfine-l-coco` | 3.68 | 1.09 | 1.46 | 1.41 | 0.67 | 0.47 |
| `dfine-m-coco` | 3.63 | 1.04 | 1.20 | 1.55 | 0.69 | 0.56 |
| `dfine-n-coco` | 5.24 | 1.25 | 1.21 | 4.07 | 0.85 | 0.62 |
| `dfine-s-coco` | 2.99 | 1.05 | 1.16 | 1.44 | 0.71 | 0.52 |
| `dfine-x-coco` | 3.88 | 1.39 | 1.41 | 1.45 | 0.65 | 0.45 |
### torch.compile and TensorRT
| model | input | time: portable | compile | TensorRT fp16 | kernels | preparation: compile (cold first call) | ONNX + engine build | peak MiB (alloc / reserved): portable | compile | kernels | TensorRT (engine + torch) |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| `deim-l-coco` | 800x1344 | 16.2 ms | 7.99 ms | 3.39 ms | 6.60 ms | 49.8 s | 58.1 s | 152 / 172 | 175 / 198 | 149 / 220 | 229 / 266 |
| `deim-m-coco` | 800x1344 | 13.6 ms | 6.73 ms | 2.54 ms | 5.17 ms | 39.1 s | 58.2 s | 126 / 142 | 149 / 166 | 127 / 192 | 204 / 232 |
| `deim-n-coco` | 800x1344 | 10.2 ms | 5.44 ms | 1.01 ms | 2.50 ms | 28.0 s | 50.8 s | 88 / 116 | 101 / 118 | 93 / 122 | 113 / 128 |
| `deim-s-coco` | 800x1344 | 11.4 ms | 5.63 ms | 1.68 ms | 3.79 ms | 31.3 s | 57.5 s | 105 / 128 | 131 / 152 | 106 / 172 | 163 / 176 |
| `deim-x-coco` | 800x1344 | 18.6 ms | 11.0 ms | 5.10 ms | 9.65 ms | 48.8 s | 66.5 s | 227 / 258 | 245 / 284 | 228 / 304 | 352 / 394 |
| `dfine-l-coco` | 800x1344 | 16.2 ms | 7.91 ms | 3.19 ms | 6.64 ms | 45.7 s | 58.7 s | 152 / 172 | 175 / 198 | 149 / 220 | 219 / 256 |
| `dfine-m-coco` | 800x1344 | 13.5 ms | 6.79 ms | 2.49 ms | 5.16 ms | 36.4 s | 54.6 s | 126 / 142 | 149 / 166 | 127 / 192 | 204 / 232 |
| `dfine-n-coco` | 800x1344 | 10.3 ms | 5.43 ms | 0.942 ms | 2.52 ms | 26.7 s | 51.5 s | 88 / 116 | 101 / 118 | 93 / 122 | 113 / 128 |
| `dfine-s-coco` | 800x1344 | 11.1 ms | 5.63 ms | 1.65 ms | 3.83 ms | 31.2 s | 51.4 s | 105 / 128 | 131 / 152 | 106 / 172 | 147 / 160 |
| `dfine-x-coco` | 800x1344 | 18.5 ms | 11.0 ms | 5.03 ms | 9.71 ms | 47.7 s | 65.7 s | 227 / 258 | 245 / 284 | 228 / 304 | 342 / 374 |
Fixed-size cells: time (ms) / peak allocated / peak reserved memory (MiB).
Hardware, protocol and caveats: [benchmarks](https://github.com/kleinicke/fastforks/blob/main/docs/benchmarks.md) (RTX 5060 Ti 16 GB, sm_120). Original = the original code named above at the precision of portable (fp16, or `(fp32)` where fp16 is unsupported or not finite); original fp32 = its default fp32 run. Peak memory is the torch allocator's.
## Accuracy
### On public datasets
pycocotools COCOeval of the top 100 detections per image. Original = the original code named under Details, same checkpoint and protocol.
| variant | dataset | original fp32 | original fp16 | portable | kernels | note |
|---|---|---|---|---|---|---|
| `dfine-n-coco` | COCO val2017, every 10th image, box mAP | 42.94 | 42.91 | 42.91 | 42.97 | |
| `dfine-s-coco` | COCO val2017, every 10th image, box mAP | 49.84 | 49.83 | 49.88 | 49.89 | |
| `dfine-m-coco` | COCO val2017, every 10th image, box mAP | 51.92 | 51.95 | 52.03 | 52.00 | |
| `dfine-l-coco` | COCO val2017, every 10th image, box mAP | 54.17 | 54.18 | 54.14 | 54.17 | |
| `dfine-x-coco` | COCO val2017, every 10th image, box mAP | 56.40 | 56.38 | 56.30 | 56.32 | |
| `deim-n-coco` | COCO val2017, every 10th image, box mAP | 44.38 | 44.37 | 44.40 | 44.38 | |
| `deim-s-coco` | COCO val2017, every 10th image, box mAP | 49.46 | 49.45 | 49.50 | 49.52 | |
| `deim-m-coco` | COCO val2017, every 10th image, box mAP | 52.38 | 52.43 | 52.43 | 52.43 | |
| `deim-l-coco` | COCO val2017, every 10th image, box mAP | 54.67 | 54.65 | 54.67 | 54.67 | |
| `deim-x-coco` | COCO val2017, every 10th image, box mAP | 56.87 | 56.91 | 56.86 | 56.90 | |
Measured on NVIDIA GeForce RTX 5060 Ti, torch 2.14.1+cu130, fastforks 675d6e93 / a981ff86, 2026-11-11 to 2026-11-12.
### Parity with the original code
Every variant is checked against reference outputs of the original code at its pinned commit. CPU, float32: max |diff| <= 0.002 vs the original. GPU, float16 (the default), portable and kernels: mean |diff| <= 0.2 and p99 <= 1.0 (absolute error of the output `boxes`).
## Limitations
- The original in the benchmarks and goldens is the Hugging Face transformers 5.19.0 port (DFineForObjectDetection, ustc-community weights) of the official D-FINE (Apache-2.0); the official repository code itself was not run.
- deim-* variants: DEIM (Huang et al. 2025), the D-FINE network with SiLU decoder MLPs, trained with dense one-to-one matching; weights and the original in the benchmarks are from the official DEIM repository (@09d35d53).
## Usage
```python
from fastforks import detection
model = detection.load("deim-l-coco") # mode="auto": the preferred backend from recorded measurements
result = model("photo.jpg")
result.boxes, result.scores, result.labels # [N, 4], [N], [N]
result.backend # what ran, e.g. Backend(mode='portable', device='cuda:0', dtype='float16')
```
Install with `pip install fastforks` (see <https://github.com/kleinicke/fastforks>). Other variants: `deim-l-coco`, `deim-m-coco`, `deim-n-coco`, `deim-s-coco`, `deim-x-coco`, `dfine-l-coco`, `dfine-m-coco`, `dfine-n-coco`, `dfine-s-coco`, `dfine-x-coco`.
## Kernels backend
`detection.load("deim-l-coco", mode="kernels")` runs our CUDA kernels (`cudnn_hacc`, `fvm_conv`). It needs a prebuilt kernel for your GPU architecture (downloaded from <https://huggingface.co/fastforks/kernels> on first use) and a recent NVIDIA driver; if there is none it raises `BackendUnavailableError` with the reason and never falls back silently. `mode="auto"` picks `kernels` when it can run and logs the choice.
<!-- staged from fastforks 1cbf1671 (2026-11-15) -->