DAC-16kHz-OpenVINO

DAC-16kHz for Intel NPUs, GPUs and CPUs: dac_16khz_deconly_zs_fp16 and dac_16khz_encoder_fp16 from mlboydaisuke/DAC-16kHz-LiteRT as ONNX (tf2onnx 1.17.0, opset 17), OpenVINO IR with fp32 weights, and precompiled Intel NPU blobs. On the NPU of a Core Ultra 9 386H the blob runs in 59.10 ms (dac_16khz_deconly_zs_fp16) and 23.91 ms (dac_16khz_encoder_fp16) per run, numpy input to numpy output.

Scripts, raw records, and the same table for all 52 graphs of LiteRT-Models: https://github.com/john-rocky/windows-zoo

Files

file	bytes	sha256
dac_16khz_deconly_zs_fp16.onnx	209,727,641	965d11e2e7a38094249e44926ca18cf5e3a0caff2eb093842cc35ebfae3ee2a4
openvino/dac_16khz_deconly_zs_fp16.xml	611,912	394fb94d5d5fbd4a52a7017a93ae53159a1a2a85b6ed34fc40c68cf161202e55
openvino/dac_16khz_deconly_zs_fp16.bin	209,451,688	0cc8cbf682061fad31f502eebcbfec2847c002913a0463b67c6e19d8bedab122
npu/dac_16khz_deconly_zs_fp16.blob	118,108,289	412c9345c54d8312345bed055435b3742c741086db627773366b01826c2e238d
dac_16khz_encoder_fp16.onnx	86,354,740	544231058e78b9228b00a6bfd3f451ea45e561acb46423d7f6c24ac229eac732
openvino/dac_16khz_encoder_fp16.xml	572,863	31922a9f01d1dd0350c56cd90d0a73758b5457440e454d349bea0d7561b7374b
openvino/dac_16khz_encoder_fp16.bin	86,090,404	0122fdb095750e1cb4d418b1ccab28680d1f7891bbacaa565565d7a1373b7a24
npu/dac_16khz_encoder_fp16.blob	47,616,129	8a2a98e7252ba521823188faff4b9d16a477ea998b8ea2b29420f83ea6a5dfa0

The NPU blobs were compiled with OpenVINO 2026.4.1, Intel NPU driver 32.0.100.4841 (OpenVINO reports NPU_DRIVER_VERSION 1004841, NPU_COMPILER_VERSION 524292). If import_model refuses a blob on another driver or OpenVINO version, compile the IR instead (the commented line in the snippet).

How it was measured

MSI Stealth 16 AI+ (Core Ultra 9 386H, RTX 5080 Laptop GPU at a 95 W power limit, Windows 11 build 26200.9711, on AC power), 2026-11-06 to 2026-11-07. On the laptop, each row is one process: load, one run, 5 warmup runs, then the median of 30 timed runs (2 warmup + 10 timed runs when the first run takes more than 3 s), numpy input to numpy output, every runtime at its default settings, one process at a time, each started after the CPU, the RTX and the Intel GPU had gone idle (the thresholds are in METHOD.md of the GitHub repo). Output check: each output against the CPU output in fp32 on the same input (ONNX Runtime's CPU EP); PASS = cosine ≥ 0.99, no NaN or Inf, and top-1 agreement ≥ 0.99 for class scores. The Snapdragon rows are Qualcomm AI Hub profile jobs of the same ONNX files (their own timer).

dac_16khz_deconly_zs_fp16

processor	runtime	file	ms	load s	output check
Intel AI Boost NPU	OpenVINO 2026.4.1	npu/ blob (import_model)	59.10	0.07	PASS (cos ≥ 0.9999)
Intel AI Boost NPU	ONNX Runtime 1.31.1 + OpenVINO EP 1.8.1	.onnx, compiled when the session loads	59.49	26.72	PASS (cos ≥ 0.9999)
Intel Graphics (iGPU)	ONNX Runtime 1.31.1 + OpenVINO EP 1.8.1	.onnx	29.22	1.71	FAIL: output_0_output: cos 0.5917 < 0.99
RTX 5080 Laptop (95 W limit)	ONNX Runtime 1.31.1, CUDA EP	fp16 copy of the .onnx (not in this repo)	6.20	1.31	PASS (cos ≥ 0.9999)
RTX 5080 Laptop (95 W limit)	ONNX Runtime 1.31.1, CUDA EP	.onnx	9.14	1.34	PASS (cos ≥ 0.9999)
Core Ultra 9 386H CPU	ONNX Runtime 1.31.1, CPU EP	.onnx	578.78	0.61	reference
Snapdragon X2 Elite NPU	Qualcomm AI Hub, precompiled QNN ONNX	.onnx compiled by AI Hub	155.58	0.52	not compared (AI Hub profile job, 2026-10-30)
Snapdragon X2 Elite NPU	Qualcomm AI Hub, ONNX Runtime QNN EP	.onnx, compiled when it loads	159.35 (4 layers on the CPU)	88.34	not compared (AI Hub profile job, 2026-10-30)
Galaxy S26 NPU	Qualcomm AI Hub, precompiled QNN ONNX	.onnx compiled by AI Hub	150.21	0.42	not compared (AI Hub profile job, 2026-10-30)

The ONNX file against the source .tflite (LiteRT interpreter on the CPU, the same input): PASS, cosine ≥ 0.9999 for every output.

dac_16khz_encoder_fp16

processor	runtime	file	ms	load s	output check
Intel AI Boost NPU	OpenVINO 2026.4.1	npu/ blob (import_model)	23.91	0.03	PASS (cos ≥ 0.9927)
Intel AI Boost NPU	ONNX Runtime 1.31.1 + OpenVINO EP 1.8.1	.onnx, compiled when the session loads	24.37	11.22	PASS (cos ≥ 0.9927)
Intel Graphics (iGPU)	ONNX Runtime 1.31.1 + OpenVINO EP 1.8.1	.onnx	8.24	1.10	PASS (cos ≥ 0.9930)
RTX 5080 Laptop (95 W limit)	ONNX Runtime 1.31.1, CUDA EP	fp16 copy of the .onnx (not in this repo)	4.04	1.12	PASS (cos ≥ 0.9929)
RTX 5080 Laptop (95 W limit)	ONNX Runtime 1.31.1, CUDA EP	.onnx	5.62	1.16	PASS (cos ≥ 0.9999)
Core Ultra 9 386H CPU	ONNX Runtime 1.31.1, CPU EP	.onnx	79.90	0.17	reference
Snapdragon X2 Elite NPU	Qualcomm AI Hub, precompiled QNN ONNX	.onnx compiled by AI Hub	69.64	0.45	not compared (AI Hub profile job, 2026-10-30)
Snapdragon X2 Elite NPU	Qualcomm AI Hub, ONNX Runtime QNN EP	.onnx, compiled when it loads	74.74	24.80	not compared (AI Hub profile job, 2026-10-30)
Galaxy S26 NPU	Qualcomm AI Hub, precompiled QNN ONNX	.onnx compiled by AI Hub	70.31	0.30	not compared (AI Hub profile job, 2026-10-30)

The ONNX file against the source .tflite (LiteRT interpreter on the CPU, the same input): PASS, cosine ≥ 0.9999 for every output.

Run it on the Intel NPU

Download this repo, then run the snippets from its folder:

pip install huggingface_hub
hf download mlboydaisuke/DAC-16kHz-OpenVINO --local-dir DAC-16kHz-OpenVINO
cd DAC-16kHz-OpenVINO

OpenVINO (pip install openvino==2026.4.1):

import numpy as np
import openvino as ov

core = ov.Core()
if "NPU" not in core.available_devices:
    raise SystemExit(f"no Intel NPU visible to OpenVINO: {core.available_devices}")
with open("npu/dac_16khz_deconly_zs_fp16.blob", "rb") as f:
    compiled = core.import_model(f.read(), "NPU")  # precompiled: no compile at load
# or compile the IR on this machine: compiled = core.compile_model("openvino/dac_16khz_deconly_zs_fp16.xml", "NPU")
# replace the zeros with your preprocessed input (float32, the shapes printed by compiled.inputs)
feeds = {i.get_any_name(): np.zeros(list(i.shape), np.float32) for i in compiled.inputs}
outputs = compiled(feeds)
print({o.get_any_name(): outputs[o].shape for o in compiled.outputs})

ONNX Runtime with the OpenVINO EP (pip install onnxruntime==1.31.1 onnxruntime-ep-openvino==1.8.1):

import numpy as np
import onnxruntime as ort
import onnxruntime_ep_openvino as ov_ep

ort.register_execution_provider_library(ov_ep.get_ep_name(), ov_ep.get_library_path())
npu = [d for d in ort.get_ep_devices()
       if d.ep_name == ov_ep.get_ep_name() and d.ep_metadata.get("ov_device") == "NPU"]
if not npu:
    raise SystemExit("no Intel NPU visible to the OpenVINO EP")
so = ort.SessionOptions()
so.add_provider_for_devices(npu, {})
sess = ort.InferenceSession("dac_16khz_deconly_zs_fp16.onnx", sess_options=so)  # compiles for the NPU here
# replace the zeros with your preprocessed input (float32)
feeds = {i.name: np.zeros(i.shape, np.float32) for i in sess.get_inputs()}
outputs = sess.run(None, feeds)
print([o.shape for o in outputs])

The same lines run dac_16khz_encoder_fp16 with its own file names.

Licence

MIT, the licence of mlboydaisuke/DAC-16kHz-LiteRT.