---
license: apache-2.0
base_model:
- ustc-community/dfine-small-coco
library_name: executorch
pipeline_tag: object-detection
inference: false
tags:
- executorch
- webgpu
- on-device
- browser
---
# D-FINE small object detection for ExecuTorch WebGPU

This repository contains an [ExecuTorch](https://github.com/pytorch/executorch) conversion of
[ustc-community/dfine-small-coco](https://huggingface.co/ustc-community/dfine-small-coco) for the ExecuTorch **WebGPU** backend, so the model runs entirely in a web browser on
the visitor's GPU. It powers the corresponding demo in the ExecuTorch WebGPU Playground (launching soon on [executorch.ai](https://executorch.ai)).

- **Base model:** [ustc-community/dfine-small-coco](https://huggingface.co/ustc-community/dfine-small-coco)
- **Format:** ExecuTorch `.pte` programs delegated to the WebGPU backend
- **Quantization:** fp32 (as exported)
- **Total size:** 40.3 MB

> **Note:** The weights are the Hugging Face transformers port of the authors' checkpoint; every tensor was checked to be bit-identical to the official `dfine_s_coco.pth` published by the first author (see Provenance and `export/verify_official.py`).

## About the original model

The small COCO model of D-FINE, a real-time DETR-style object detector that refines boxes as probability distributions.

- **Developed by:** Yansong Peng et al. (USTC)
- **Paper:** [D-FINE: Redefine Regression Task in DETRs as Fine-grained Distribution Refinement](https://arxiv.org/abs/2410.13842)
- 10.3M parameters
- Detects the 80 COCO object classes
- No non-maximum suppression needed

For training data, evaluation results and full details, see the original model card: [ustc-community/dfine-small-coco](https://huggingface.co/ustc-community/dfine-small-coco).

## Intended use and limitations

Detecting everyday objects in photos, video and camera frames.

Only knows the 80 COCO classes. Small, unusual or heavily occluded objects are often missed, and confidence scores are not calibrated probabilities.

## What changed in this conversion

- Graph rewritten with output-preserving substitutions so it runs as a single WebGPU delegate (checked against upstream on a real image: boxes within 0.001 px)
- Fixed 640x640 input; the 300 query outputs are filtered by score outside the program

## Inputs and outputs

- Input `[1, 3, 640, 640]` float32 RGB in [0, 1] (`x / 255`, no mean/std); letterboxed or plain-resized
- Output `[1, 84, 300]` float32: rows 0-3 are box center x, center y, width, height in input pixels; rows 4-83 are sigmoid scores for the 80 COCO classes

## Files

| File | Size | SHA-256 |
|---|---|---|
| `dfine_s_coco_webgpu.pte` | 40.3 MB | `96cc65f615dc2c79…` |

Full checksums are in `manifest.json`. The playground verifies them after download.

## Running it

These programs were exported with the ExecuTorch WebGPU backend
([backends/webgpu](https://github.com/pytorch/executorch/tree/main/backends/webgpu)) and validated with the
WebAssembly runtime build used by the ExecuTorch WebGPU Playground (launching soon on [executorch.ai](https://executorch.ai)). That runtime build is not published yet; until it is,
the playground is the supported way to run them. It downloads these files once, caches them in the browser,
and runs everything locally.

Requirements: a browser with WebGPU enabled.

## Provenance

- **Weights:** `ustc-community/dfine-small-coco` @ `f79e65b5fbb33ceb9d3ebba042955d7410c608f8`, file `model.safetensors`, sha256 `a144baff3a27bfd538fe37b11fa8d4fca5744b8c054b5301e7c699a1379cd18f` (from a community mirror of the official release)
- **Checked against the official checkpoint:** `Peterande/D-FINE` @ `19214e91f5ce4c1e609022c38b1fc953c47404e7`, file `dfine_s_coco.pth` (sha256 `48a6c8cc43eb57186843f752e2e8461ddd3326e0d3c575e71e6e960844683e89`): all 764 tensors bit-identical (verify_official.py)
- **Exported with:** executorch 1.5.1+cpu, torch 2.14.1+cpu, torchao 0.18.0+cpu, transformers 5.19.0 (executorch git `3b6068392324`)
- **Reproduce:** `python recipes/detector-dfine-small.py --out out/detector-dfine-small --image photo.jpg` (recipe in `export/`)
- Re-running the recipe produces byte-identical files.

## Citation

If you use this model, please cite the original work:

```bibtex
@misc{peng2024dfine,
  title={D-FINE: Redefine Regression Task in DETRs as Fine-grained Distribution Refinement},
  author={Yansong Peng and Hebei Li and Peixi Wu and Yueyi Zhang and Xiaoyan Sun and Feng Wu},
  year={2024},
  eprint={2410.13842},
  archivePrefix={arXiv},
  primaryClass={cs.CV}
}
```

## License

This is a derivative of the upstream model and is distributed under the same terms
(`apache-2.0`); see `LICENSE`.
Modifications (conversion to the ExecuTorch format) by Meta Platforms, Inc.