| --- |
| license: apache-2.0 |
| base_model: |
| - ustc-community/dfine-small-coco |
| library_name: executorch |
| pipeline_tag: object-detection |
| inference: false |
| tags: |
| - executorch |
| - webgpu |
| - on-device |
| - browser |
| --- |
| # D-FINE small object detection for ExecuTorch WebGPU |
|
|
| This repository contains an [ExecuTorch](https://github.com/pytorch/executorch) conversion of |
| [ustc-community/dfine-small-coco](https://huggingface.co/ustc-community/dfine-small-coco) for the ExecuTorch **WebGPU** backend, so the model runs entirely in a web browser on |
| the visitor's GPU. It powers the corresponding demo in the ExecuTorch WebGPU Playground (launching soon on [executorch.ai](https://executorch.ai)). |
|
|
| - **Base model:** [ustc-community/dfine-small-coco](https://huggingface.co/ustc-community/dfine-small-coco) |
| - **Format:** ExecuTorch `.pte` programs delegated to the WebGPU backend |
| - **Quantization:** fp32 (as exported) |
| - **Total size:** 40.3 MB |
|
|
| > **Note:** The weights are the Hugging Face transformers port of the authors' checkpoint; every tensor was checked to be bit-identical to the official `dfine_s_coco.pth` published by the first author (see Provenance and `export/verify_official.py`). |
| |
| ## About the original model |
| |
| The small COCO model of D-FINE, a real-time DETR-style object detector that refines boxes as probability distributions. |
| |
| - **Developed by:** Yansong Peng et al. (USTC) |
| - **Paper:** [D-FINE: Redefine Regression Task in DETRs as Fine-grained Distribution Refinement](https://arxiv.org/abs/2410.13842) |
| - 10.3M parameters |
| - Detects the 80 COCO object classes |
| - No non-maximum suppression needed |
| |
| For training data, evaluation results and full details, see the original model card: [ustc-community/dfine-small-coco](https://huggingface.co/ustc-community/dfine-small-coco). |
| |
| ## Intended use and limitations |
| |
| Detecting everyday objects in photos, video and camera frames. |
| |
| Only knows the 80 COCO classes. Small, unusual or heavily occluded objects are often missed, and confidence scores are not calibrated probabilities. |
| |
| ## What changed in this conversion |
| |
| - Graph rewritten with output-preserving substitutions so it runs as a single WebGPU delegate (checked against upstream on a real image: boxes within 0.001 px) |
| - Fixed 640x640 input; the 300 query outputs are filtered by score outside the program |
| |
| ## Inputs and outputs |
| |
| - Input `[1, 3, 640, 640]` float32 RGB in [0, 1] (`x / 255`, no mean/std); letterboxed or plain-resized |
| - Output `[1, 84, 300]` float32: rows 0-3 are box center x, center y, width, height in input pixels; rows 4-83 are sigmoid scores for the 80 COCO classes |
| |
| ## Files |
| |
| | File | Size | SHA-256 | |
| |---|---|---| |
| | `dfine_s_coco_webgpu.pte` | 40.3 MB | `96cc65f615dc2c79…` | |
|
|
| Full checksums are in `manifest.json`. The playground verifies them after download. |
|
|
| ## Running it |
|
|
| These programs were exported with the ExecuTorch WebGPU backend |
| ([backends/webgpu](https://github.com/pytorch/executorch/tree/main/backends/webgpu)) and validated with the |
| WebAssembly runtime build used by the ExecuTorch WebGPU Playground (launching soon on [executorch.ai](https://executorch.ai)). That runtime build is not published yet; until it is, |
| the playground is the supported way to run them. It downloads these files once, caches them in the browser, |
| and runs everything locally. |
|
|
| Requirements: a browser with WebGPU enabled. |
|
|
| ## Provenance |
|
|
| - **Weights:** `ustc-community/dfine-small-coco` @ `f79e65b5fbb33ceb9d3ebba042955d7410c608f8`, file `model.safetensors`, sha256 `a144baff3a27bfd538fe37b11fa8d4fca5744b8c054b5301e7c699a1379cd18f` (from a community mirror of the official release) |
| - **Checked against the official checkpoint:** `Peterande/D-FINE` @ `19214e91f5ce4c1e609022c38b1fc953c47404e7`, file `dfine_s_coco.pth` (sha256 `48a6c8cc43eb57186843f752e2e8461ddd3326e0d3c575e71e6e960844683e89`): all 764 tensors bit-identical (verify_official.py) |
| - **Exported with:** executorch 1.5.1+cpu, torch 2.14.1+cpu, torchao 0.18.0+cpu, transformers 5.19.0 (executorch git `3b6068392324`) |
| - **Reproduce:** `python recipes/detector-dfine-small.py --out out/detector-dfine-small --image photo.jpg` (recipe in `export/`) |
| - Re-running the recipe produces byte-identical files. |
| |
| ## Citation |
| |
| If you use this model, please cite the original work: |
| |
| ```bibtex |
| @misc{peng2024dfine, |
| title={D-FINE: Redefine Regression Task in DETRs as Fine-grained Distribution Refinement}, |
| author={Yansong Peng and Hebei Li and Peixi Wu and Yueyi Zhang and Xiaoyan Sun and Feng Wu}, |
| year={2024}, |
| eprint={2410.13842}, |
| archivePrefix={arXiv}, |
| primaryClass={cs.CV} |
| } |
| ``` |
| |
| ## License |
| |
| This is a derivative of the upstream model and is distributed under the same terms |
| (`apache-2.0`); see `LICENSE`. |
| Modifications (conversion to the ExecuTorch format) by Meta Platforms, Inc. |
| |
