README.md

metadata

license: apache-2.0
base_model: litert-community/D-FINE-S-LiteRT
base_model_relation: quantized
library_name: openvino
pipeline_tag: object-detection
tags:
  - openvino
  - onnx
  - intel-npu

D-FINE-S-OpenVINO

D-FINE-S for Intel NPUs, GPUs and CPUs: dfine_graphA_fp16 and dfine_graphB_fp16 from litert-community/D-FINE-S-LiteRT as ONNX, OpenVINO IR with fp32 weights, and precompiled Intel NPU blobs. Call graph A (backbone and encoder) first, on a 640×640 image. A host step then builds graph B's inputs from A's outputs and that repo's host_params.bin. Graph B (decoder) returns 300 boxes and their class logits. The NPU of a Core Ultra 9 386H runs the blobs in 11.14 ms (A) and 96.94 ms (B).

Scripts, raw records, and the same table for all 52 graphs of LiteRT-Models: https://github.com/john-rocky/windows-zoo

Files

file	bytes	sha256
dfine_graphA_fp16.onnx	25,224,269	a17ae288f3885fc9fac1947b809804ec22fcc7a2e349769157df0852c9342f3a
openvino/dfine_graphA_fp16.xml	1,733,404	d826377da28183918bbc2acd6e89ae8d0d6c826efdbebc34b907c949aea0680c
openvino/dfine_graphA_fp16.bin	24,222,828	a8784deaba145f7186fdfd005108eba308eada86e9be98c587146430a393c402
npu/dfine_graphA_fp16.blob	18,370,696	7a1041fd1917e0122a9c41b723027e607290dc08d0d3012047d6c89eb635fb0e
dfine_graphB_fp16.onnx	17,758,341	440bc1c6db016bc069027df3ce791b09d0b36727dcabfb340c31c7e158cfdb12
openvino/dfine_graphB_fp16.xml	1,467,491	7dc7752dd3e923b95df0b4ff75843df16b153f8f76510f8ad501bbad2566969d
openvino/dfine_graphB_fp16.bin	16,665,968	91b4a6e2eb7246d1ba77d6669ff5594f209f05e9b15588fafbf938906819bf87
npu/dfine_graphB_fp16.blob	60,096,662	077cfe34c93d875266a5bbfa383bda686fd670cc3259d0c08528b704ca6af0d0

The NPU blobs were compiled with OpenVINO 2026.4.1, Intel NPU driver 32.0.100.4841 (OpenVINO reports NPU_DRIVER_VERSION 1004841, NPU_COMPILER_VERSION 524292). If import_model refuses a blob on another driver or OpenVINO version, compile the IR instead (the commented line in the snippet).

Conversion

The ONNX files were converted from the .tflite files with tf2onnx 1.17.0 (opset 17). tf2onnx 1.17.0 drops the bias of 15 FULLY_CONNECTED layers of dfine_graphA_fp16 and dfine_graphB_fp16; these files were converted with a one-line fix of that handler (convert/patch_tf2onnx.py in the GitHub repo), and their outputs match the .tflite.

How it was measured

MSI Stealth 16 AI+ (Core Ultra 9 386H, RTX 5080 Laptop GPU at a 95 W power limit, Windows 11 build 26200.9711, on AC power), 2026-11-07. On the laptop, each row is one process: load, one run, 5 warmup runs, then the median of 30 timed runs (2 warmup + 10 timed runs when the first run takes more than 3 s), numpy input to numpy output, every runtime at its default settings, one process at a time, each started after the CPU, the RTX and the Intel GPU had gone idle (the thresholds are in METHOD.md of the GitHub repo). Output check: each output against the CPU output in fp32 on the same input (ONNX Runtime's CPU EP); PASS = cosine ≥ 0.99, no NaN or Inf, and top-1 agreement ≥ 0.99 for class scores. No Snapdragon rows: Qualcomm AI Hub measured the files before the bias fix (its numbers are in the GitHub table).

dfine_graphA_fp16

processor	runtime	file	ms	load s	output check
Intel AI Boost NPU	OpenVINO 2026.4.1	npu/ blob (import_model)	11.14	0.01	PASS (cos ≥ 0.9973)
Intel AI Boost NPU	ONNX Runtime 1.31.1 + OpenVINO EP 1.8.1	.onnx, compiled when the session loads	8.61	12.75	PASS (cos ≥ 0.9973)
Intel Graphics (iGPU)	ONNX Runtime 1.31.1 + OpenVINO EP 1.8.1	.onnx	8.89	1.36	PASS (cos ≥ 0.9999)
RTX 5080 Laptop (95 W limit)	ONNX Runtime 1.31.1, CUDA EP	fp16 copy of the .onnx (not in this repo)	6.80	1.29	PASS (cos ≥ 0.9999)
RTX 5080 Laptop (95 W limit)	ONNX Runtime 1.31.1, CUDA EP	.onnx	7.22	1.37	PASS (cos ≥ 0.9999)
Core Ultra 9 386H CPU	ONNX Runtime 1.31.1, CPU EP	.onnx	44.49	0.26	reference

The ONNX file against the source .tflite (LiteRT interpreter on the CPU, the same input): PASS, cosine ≥ 0.9999 for every output.

dfine_graphB_fp16

processor	runtime	file	ms	load s	output check
Intel AI Boost NPU	OpenVINO 2026.4.1	npu/ blob (import_model)	96.94	0.04	PASS (cos ≥ 0.9999)
Intel AI Boost NPU	ONNX Runtime 1.31.1 + OpenVINO EP 1.8.1	.onnx, compiled when the session loads	97.99	51.92	PASS (cos ≥ 0.9999)
Intel Graphics (iGPU)	ONNX Runtime 1.31.1 + OpenVINO EP 1.8.1	.onnx	88.00	3.14	FAIL: output_1_output: 0 NaN, 80 Inf; output_1_output: argmax agreement 189/300
RTX 5080 Laptop (95 W limit)	ONNX Runtime 1.31.1, CUDA EP	fp16 copy of the .onnx (not in this repo)	9.60	1.27	FAIL: output_1_output: 0 NaN, 80 Inf
RTX 5080 Laptop (95 W limit)	ONNX Runtime 1.31.1, CUDA EP	.onnx	9.66	1.23	PASS (cos ≥ 0.9999)
Core Ultra 9 386H CPU	ONNX Runtime 1.31.1, CPU EP	.onnx	67.99	0.11	reference

The ONNX file against the source .tflite (LiteRT interpreter on the CPU, the same input): PASS, cosine ≥ 0.9999 for every output.

Run it on the Intel NPU

Download this repo, then run the snippets from its folder:

pip install huggingface_hub
hf download mlboydaisuke/D-FINE-S-OpenVINO --local-dir D-FINE-S-OpenVINO
cd D-FINE-S-OpenVINO

OpenVINO (pip install openvino==2026.4.1):

import numpy as np
import openvino as ov

core = ov.Core()
if "NPU" not in core.available_devices:
    raise SystemExit(f"no Intel NPU visible to OpenVINO: {core.available_devices}")
with open("npu/dfine_graphA_fp16.blob", "rb") as f:
    compiled = core.import_model(f.read(), "NPU")  # precompiled: no compile at load
# or compile the IR on this machine: compiled = core.compile_model("openvino/dfine_graphA_fp16.xml", "NPU")
# replace the zeros with your preprocessed input (float32, the shapes printed by compiled.inputs)
feeds = {i.get_any_name(): np.zeros(list(i.shape), np.float32) for i in compiled.inputs}
outputs = compiled(feeds)
print({o.get_any_name(): outputs[o].shape for o in compiled.outputs})

ONNX Runtime with the OpenVINO EP (pip install onnxruntime==1.31.1 onnxruntime-ep-openvino==1.8.1):

import numpy as np
import onnxruntime as ort
import onnxruntime_ep_openvino as ov_ep

ort.register_execution_provider_library(ov_ep.get_ep_name(), ov_ep.get_library_path())
npu = [d for d in ort.get_ep_devices()
       if d.ep_name == ov_ep.get_ep_name() and d.ep_metadata.get("ov_device") == "NPU"]
if not npu:
    raise SystemExit("no Intel NPU visible to the OpenVINO EP")
so = ort.SessionOptions()
so.add_provider_for_devices(npu, {})
sess = ort.InferenceSession("dfine_graphA_fp16.onnx", sess_options=so)  # compiles for the NPU here
# replace the zeros with your preprocessed input (float32)
feeds = {i.name: np.zeros(i.shape, np.float32) for i in sess.get_inputs()}
outputs = sess.run(None, feeds)
print([o.shape for o in outputs])

The same lines run dfine_graphB_fp16 with its own file names.

The Python on the LiteRT card has the rest: the preprocessing (img, x), the loading of host_params.bin from that repo (p, take), and the host step between the graphs (from enc_cls, mem = to ref =). Swap its run() for the lines above and give each graph its inputs by shape.

Branch windows-model-zoo

The branch windows-model-zoo adds the 5 files that windows-model-zoo runs. unfixed/: the conversion without the bias fix above, with its fp16 copy and NPU blob. fp16/: the fp16 copy that the RTX fp16 row of dfine_graphB_fp16 above ran. To use the model, take main's files.

Licence

Apache-2.0, the licence of litert-community/D-FINE-S-LiteRT. The upstream model: ustc-community/dfine-small-coco.