Orvena/Supertonic-3-ANE

Supertonic-3 vector estimator for the Apple Neural Engine

This is the vector estimator of Supertone's Supertonic-3 text-to-speech model, converted so it runs entirely on the Neural Engine of an iPhone or Mac. The estimator is the part Supertonic runs eight times per sentence, so it is where almost all of the time goes. On an iPhone 17 Pro Max this makes a whole sentence 11 to 15 times faster than ONNX Runtime on the CPU and leaves both the CPU and the GPU free. Orvena uses it for its voice in the languages Kokoro doesn't speak.

Speech length	ONNX Runtime, CPU	This model, Neural Engine	Neural Engine, GPU busy
3.1 s	937 ms	82 ms	141 ms
5.8 s	1,822 ms	121 ms	208 ms
7.4 s	2,092 ms	137 ms	186 ms

Times are for a whole sentence: Supertonic's ONNX duration predictor, text encoder and vocoder, with this model in place of the ONNX estimator.

What changed

The Core ML port this starts from keeps the estimator in FP32 with a flexible length, which Core ML runs on the CPU. This version computes in FP16 and fixes the length to four buckets, one function each: latent_32, latent_64, latent_128 and latent_256. In each, the text and the latent are both that many frames long. With fixed shapes, every operation lands on the Neural Engine. convert.py reproduces the package.

Using it

Run Supertonic as usual, and for the eight flow-matching steps pick the smallest bucket that holds both the text length and the latent length:

1. Pad the text embedding (text_emb, 1 × 256 × T) and text_mask with zeros to the bucket.
2. Pad the noisy latent (1 × 144 × L) and latent_mask with zeros to the bucket.
3. Run the bucket's function for each step with style_ttl, current_step and total_step as usual, feeding denoised_latent back in.
4. Crop the result back to L frames before the vocoder.

Load a bucket with MLModelConfiguration.functionName = "latent_128" and computeUnits = .cpuAndNeuralEngine. Text longer than 256 frames needs the original estimator.

How close it is

Padding the text is exact. Padding the latent changes the result in the same way batching does in the original model: you get a different take on the sentence from the same noise, not a degraded one. FP16 then adds a little drift of its own. To check that the speech is still the same speech, we transcribed 10 sentences in 7 languages (English, German, Spanish, French, Korean, Russian, Polish) with three noise seeds each using Whisper large-v3-turbo. The ONNX original and this model gave identical transcripts for all 30.

License

Supertonic-3 is released under the BigScience OpenRAIL-M license, and this conversion is too. The use-based restrictions in LICENSE, Attachment A, apply to it and to anything built from it. Thanks to Supertone for the model and to the Core ML port it was converted from.