D-FINE-seg

Real-time object detection, keypoint estimation, instance and semantic segmentation - one codebase, one config flag (task: detect | segment | sem_seg | kpts), five model sizes (N -> X), and fused end-to-end deployment graphs for ONNX, TensorRT, OpenVINO, CoreML and LiteRT.

This repo hosts the released weights; the code, full benchmark tables and docs live in the GitHub repo. See every task on real footage at dfine-seg.argoai.space. πŸ“„ Paper (arXiv:2602.23043) Β· 🎬 Video tutorial Β· πŸ§ͺ Colab

One frame, three tasks: detection, instance segmentation and semantic segmentation

One frame, three tasks, one config flag.

Cityscapes semantic segmentation (released)

Five semantic-segmentation checkpoints trained on Cityscapes at native 1024x2048, benchmarked against 23 official checkpoints of four other families - each through its own framework's shipped code, one validator, 500 val images, TensorRT fp16, batch 1, RTX 5070 Ti. Full 28-model table, per-class IoU and protocol: cityscapes-benchmark.

Cityscapes val mIoU vs end-to-end latency at native 1024x2048

model params (M) mIoU pixel acc engine ms e2e ms
D-FINE-seg-N 3.36 77.99 96.03 3.02 4.21
D-FINE-seg-S 7.99 81.12 96.47 5.38 6.37
D-FINE-seg-M 15.95 81.41 96.55 7.42 8.26
D-FINE-seg-L 24.97 83.80 96.93 8.98 9.77
D-FINE-seg-X 56.3 84.84 97.10 14.18 14.64
YOLO26x-sem 40.13 83.58 96.79 10.37 13.58
EfficientViT-L2 52.61 83.20 96.87 9.56 19.87
Mask2Former-Swin-L 215.46 83.33 96.91 64.51 94.07
SegFormer-B5 84.61 81.41 96.62 38.81 42.12

D-FINE-seg-X: 84.84 mIoU at 14.6 ms end-to-end, the highest mIoU of the 28 models - +1.26 mIoU over YOLO26-x, and +1.5 mIoU over Mask2Former Swin-L with 6.4x lower latency and a quarter of the parameters. D-FINE-seg-L matches or beats YOLO26-x on every axis.

Load with one line (weights auto-download):

model = load_model("x", task="sem_seg")   # names and 1024x2048 input size come from the checkpoint

They are also a strong init for your own driving/outdoor data: point train.pretrained_model_path at one with a different label_to_name and the class heads re-initialize while the rest keeps the Cityscapes training (N initialized from COCO, S/M/L/X from obj2coco). A model fine-tuned from them is still derived from Cityscapes and inherits its non-commercial terms (see License) - for commercial use, start from the COCO / obj2coco weights instead.

COCO person keypoints (released)

dfine_kpts_s_coco.pt: the S model trained on COCO person keypoints (17 points). COCO val2017 at 640, official COCO AP; end-to-end latency with each framework's own pipeline, TensorRT fp16, batch 1, RTX 5070 Ti.

model keypoint AP AP50 / AP75 person box AP e2e ms
D-FINE-seg S (40 epochs) 65.4 88.0 / 71.1 64.2 2.10
YOLO26s-pose (released, 300 epochs, Objects365 init) 62.7 85.8 / 68.8 54.3 2.16
model = load_model("s", task="kpts")      # out["kpts"]: [N, 17, 4] = x, y, presence, visible

The keypoint head is not tied to human pose: every detection predicts K points with a presence and a visibility score, so it learns whatever points your labels define (YOLO-pose or COCO keypoint labels). Use these weights as the init for your own person keypoint data, or start from the detection weights for other objects. Larger sizes are coming.

Highlights

  • Four tasks, one model family - DETR-style detection (queries -> boxes), plus a lightweight Mask-DINO-style mask head for instance segmentation, a query-free head for semantic segmentation, and a keypoint head that predicts K points per detection (refined by a keypoint decoder, one token per point).
  • Five sizes (N/S/M/L/X) - pretrained weights for every task: detection on COCO (plain and Objects365-pretrained), COCO instance segmentation, Cityscapes semantic segmentation, COCO person keypoints (S for now).
  • Multi-backend deployment - one export pipeline produces ONNX / TensorRT / OpenVINO / CoreML / LiteRT graphs with the postprocessor fused; INT8 quantization (OpenVINO, CoreML, LiteRT); ByteTrack tracking; SAM3 auto-labeling; Gradio demo.
  • Fine-tuning - on 640 Cityscapes, beats Ultralytics YOLO26 and RF-DETR on detection & instance-segmentation F1; on TACO and VisDrone, beats YOLO26 across all sizes (TensorRT FP16, end-to-end protocol).
  • Not a fork - the detection core follows the D-FINE paper; segmentation heads, training, export and inference were reimplemented from scratch, with the mask-head design following the Mask DINO paradigm.

Checkpoints

All files are PyTorch .pt; weights auto-download on first use (no manual download needed).

Task Files Params (M)
Detection (COCO-pretrained) dfine_{n,s,m,l,x}_coco.pt 3.8 / 10.4 / 19.7 / 31.4 / 62.9
Detection (Objects365 -> COCO) dfine_{s,m,l,x}_obj2coco.pt 10.4 / 19.7 / 31.4 / 62.9
Instance segmentation (COCO-pretrained) dfine_seg_{n,s,m,l,x}_coco.pt 4.2 / 12.0 / 21.3 / 33.0 / 64.6
Semantic segmentation (Cityscapes) dfine_sem_{n,s,m,l,x}_cityscapes.pt 3.36 / 7.99 / 15.95 / 24.97 / 56.3
Keypoints (COCO person, 17 points) dfine_kpts_s_coco.pt 13.5

Usage

pip install dfine-seg           # inference + training
pip install 'dfine-seg[all]'    # + every export backend, SAM3, Gradio demo
from dfine_seg import load_model, read_image

model = load_model("s")                        # COCO detection, weights auto-downloaded
model = load_model("s", task="segment")        # COCO instance segmentation
model = load_model("x", task="sem_seg")        # Cityscapes semantic segmentation
model = load_model("s", task="kpts")           # COCO person keypoints (17 points)
model = load_model("path/to/model.pt")         # your checkpoint - size/task/classes auto-detected
model = load_model("path/to/model.engine")     # any exported artifact, picked by extension

out = model(read_image("path/to/image.jpg"))[0]
print(out["boxes"], out["scores"], [model.names[int(i)] for i in out["labels"]])

Train, export and bench from a config file:

dfine init          # writes ./config.yaml (task: detect | segment | sem_seg | kpts)
dfine split && dfine train && dfine export && dfine bench

Everything else - data formats, config keys, export knobs, inference flags - is in the GitHub README.

Example outputs

Detection on a VisDrone aerial scene Instance segmentation on a TACO image

Detection on VisDrone (aerial) Β· instance segmentation on TACO (waste).

Architecture

D-FINE-seg architecture: shared backbone/encoder, three heads

Notes

  • Weights are an init, not a final model - the COCO checkpoints cover detection, instance segmentation and person keypoints, the Cityscapes ones semantic segmentation; fine-tune on your own data for the task and classes you need. load_model detects task, classes and input size from the weights.
  • TensorRT engines are GPU-specific - re-export on the target GPU. INT8 quantization shows a noticeable F1 drop on segmentation; FP16 is the recommended trade-off.
  • Mosaic augmentation (mosaic_augs.mosaic_prob) is not recommended for the segmentation task - lower it toward 0 if masks look wrong.
  • Older fine-tuning benchmarks (640 Cityscapes detection/instance segmentation, VisDrone, TACO, COCO-style APs, cross-format latency): GitHub README -> Benchmarks.

License

  • The code and the COCO-pretrained checkpoints (detection, instance segmentation, person keypoints) are Apache-2.0.
  • The Cityscapes semantic-segmentation checkpoints (dfine_sem_*_cityscapes.pt) are derivative works of the Cityscapes dataset and inherit its terms of use: non-commercial purposes only (academic research, teaching, scientific publications, personal experimentation), no commercial gain. Train on your own data and the resulting model is yours to ship - the code stays Apache-2.0 (no AGPL).

Citation

@article{saakyan2026dfineseg,
  title   = {D-FINE-seg: Object Detection and Instance Segmentation Framework with Multi-Backend Deployment},
  author  = {Saakyan, Argo and Solntsev, Dmitry},
  journal = {arXiv preprint arXiv:2602.23043},
  year    = {2026},
  eprint  = {2602.23043}
}

@misc{peng2024dfine,
  title         = {D-FINE: Redefine Regression Task in DETRs as Fine-grained Distribution Refinement},
  author        = {Yansong Peng and Hebei Li and Peixi Wu and Yueyi Zhang and Xiaoyan Sun and Feng Wu},
  year          = {2024},
  eprint        = {2410.13842},
  archivePrefix = {arXiv},
  primaryClass  = {cs.CV}
}
Downloads last month
1,091
Inference Providers NEW
This model isn't deployed by any Inference Provider. πŸ™‹ Ask for provider support

Space using ArgoSA/D-FINE-seg 1

Papers for ArgoSA/D-FINE-seg

Evaluation results