anira-project/steerable-nafx

Steerable NAFX for anira

Christian Steinmetz's steerable neural audio effects (a temporal convolutional network) in three sizes: the full model, 3 blocks and 2 blocks, for LibTorch, ONNX Runtime, LiteRT and ExecuTorch. anira's CNN benchmark and example model.

Files

File	Engine	Size	SHA256
onnx/steerable-nafx-2_blocks-libtorch-dynamic.onnx	ONNX Runtime	6,409	e59a4eb53b0a22ec0a1e6c456cbf6538a166173ba780a5403bf3295c72a7b281
onnx/steerable-nafx-3_blocks-libtorch-dynamic.onnx	ONNX Runtime	65,192	64f79fea832b9b50798bf4a05c2d807d06294e2322b784565a404f73c5986289
onnx/steerable-nafx-libtorch-dynamic.onnx	ONNX Runtime	123,975	4cbcdaffef8e6fab2a19ed138559fd7772e7c811fe4b15cf94a58b0bd0cbb52f
litert/steerable-nafx-2-blocks.tflite	LiteRT, TensorFlow Lite	13,528	ba74cdb66a046b45ed44140e529d5202005499c64a46b79d8449e39822311b38
litert/steerable-nafx-3-blocks.tflite	LiteRT, TensorFlow Lite	75,628	6529df74ba2fe664c641e1e248cc5d6b252f6bce4b2ea893750b6e4d36418233
litert/steerable-nafx.tflite	LiteRT, TensorFlow Lite	137,728	c6e7fb6d5778cf52703b779733710ebfc08b4f226e14cadfcc47f38a53c17106
libtorch/steerable-nafx-2_blocks-dynamic.pt	LibTorch	20,573	266159e1d2aa7bba88754732d4ad7c47655f7ec41ad2c80d1505a36e597ebf66
libtorch/steerable-nafx-3_blocks-dynamic.pt	LibTorch	82,967	bb4cc880ba407dcbdc82e8fa94846c08c78bf8209885ce64c51e67b1324ae813
libtorch/steerable-nafx-dynamic.pt	LibTorch	144,072	36a4bfde003141cb4060ce9eb9941138a42578ac130decddd92a9c17d879892b
executorch/steerable-nafx-2-blocks.pte	ExecuTorch 1.3.1, CPU	12,928	49544305d0db6b4645f1a85297a8a36b33a67a96ca34252a474d3828b158c3fc
executorch/steerable-nafx-2-blocks_coreml.pte	ExecuTorch 1.3.1, Core ML (Apple)	35,392	e9a0b9eb64bf011f433039c2de88c6406e6d625e7cbaf93e503cc7ba81ffa27d
executorch/steerable-nafx-2-blocks_vulkan.pte	ExecuTorch 1.3.1, Vulkan (GPUs)	12,292	06e821c571a60f5f97b04f8e024c657237734e01ef6b0795ccd8a3fe9e95f42b
executorch/steerable-nafx-3-blocks.pte	ExecuTorch 1.3.1, CPU	74,112	f29622b181dec86241287ceb19c6f5f7e0e1d5089722bec20b4d2b84c44ca3ad
executorch/steerable-nafx-3-blocks_coreml.pte	ExecuTorch 1.3.1, Core ML (Apple)	106,892	dd0d4c3701909876aa0568f914dac7954c0eff8d948cce72c8c8fb42205f0c4a
executorch/steerable-nafx-3-blocks_vulkan.pte	ExecuTorch 1.3.1, Vulkan (GPUs)	73,348	96dfc9fe25bc590a4b4c4007c14bf2139e24f4d6b04af8c3fc206b99bbf8b443
executorch/steerable-nafx.pte	ExecuTorch 1.3.1, CPU	135,296	a9a087ec3e78a2d2a9514c8a541f07859f4da2ced058f104d8971eeb966c388b
executorch/steerable-nafx_coreml.pte	ExecuTorch 1.3.1, Core ML (Apple)	178,397	356903c6b482baa628ec88417f244081c746e4cdcc0b24d5e89ce2355ec9ee20
executorch/steerable-nafx_vulkan.pte	ExecuTorch 1.3.1, Vulkan (GPUs)	134,276	2fc383bc1c8e9c140e0f36608aa763559e31118a8bbc9ad61a29d253309c01e3
coreml/steerable-nafx-2-blocks.mlpackage	Core ML (native)	12,515	per file in the catalog
coreml/steerable-nafx-3-blocks.mlpackage	Core ML (native)	73,829	per file in the catalog
coreml/steerable-nafx.mlpackage	Core ML (native)	135,155	per file in the catalog

Source

faressc/steerable-nafx @ e495467 (a fork of csteinmetz1/steerable-nafx), models/model_0/; the export code is in that repository.

Formats and checks

Every model here runs on ONNX Runtime, LiteRT and TensorFlow Lite, LibTorch, ExecuTorch (XNNPACK or portable kernels on the CPU, Core ML on Apple, Vulkan on GPUs) and as a native Core ML ML program, with the ONNX models' tensor names. anira foundation models streams every format of every description over twelve consecutive calls with the state fed back and checks that they agree (tools/check_streaming.py). Deviations below are from ONNX Runtime, per model, in the order of the files.

Format	Files	Max deviation
ExecuTorch, XNNPACK (re-exported)	executorch/{model}.pte	2.1e-07, 1.0e-07, 9.7e-08
LiteRT, TensorFlow Lite (re-exported)	litert/*.tflite	2.1e-07, 1.0e-07, 9.7e-08
ExecuTorch, Core ML (Apple, float32)	executorch/*_coreml.pte	2.0e-07, 9.7e-08, 1.1e-07
ExecuTorch, Vulkan (GPUs)	executorch/*_vulkan.pte	exported; not run here (no Vulkan runtime)
Core ML ML program (anira's native engine, coming with v3; macOS 13, iOS 16)	coreml/*.mlpackage	2.0e-07, 9.7e-08, 1.1e-07

uv run --no-project export/export_litert.py <model.onnx> <out.tflite> [<description>] [--guitar-lstm <pt>]
uv run --no-project export/export_executorch.py <model.onnx> <out.pte> [<description>] [--guitar-lstm <pt>] [--coreml | --vulkan]
uv run --no-project export/export_coreml.py <model.onnx> <out.mlpackage> [<description>] [--guitar-lstm <pt>]
uv run --no-project export/export_libtorch.py <model.onnx> <out.pt> [<description>] [--guitar-lstm <pt>]

<description> is the model's description in anira foundation models, for the input shapes where the ONNX model's time axis is dynamic. The PyTorch module comes from the ONNX model (export/onnx_to_torch.py: onnxsim to static shapes, onnx2torch with traceable converters); each export is checked against ONNX Runtime over eight consecutive calls of random input with the state fed back, relative to the output's peak (export/check.py).

upstream/ keeps anira's original LiteRT files (upstream/litert/) and ExecuTorch programs (upstream/executorch/), which the descriptions no longer name: the LiteRT files compute something else than the other formats, 17 to 52 % off from the first call, and the ExecuTorch programs run on portable kernels, 427 ms per call of the single-block model against a 42.7 ms budget. Their replacements in litert/ and executorch/ are exported from the same PyTorch module as the other formats; the XNNPACK program takes 3.7 ms per call.

This repository holds the weights, the export scripts and the licence. The anira model descriptions live with the code in anira foundation models (models/steerable-nafx), versioned with git. Pin a revision and check each file's SHA256 when you download:

https://huggingface.co/anira-project/steerable-nafx/resolve/<revision>/onnx/steerable-nafx-2_blocks-libtorch-dynamic.onnx

Licence

Apache-2.0: LICENSE, copied from the source repository.