thuongvv/Supertonic-3-VectorEstimator-CoreML-W8

Supertonic 3 vector estimator — Core ML, int8 weights, for the Apple Neural Engine

The vector estimator of Supertonic 3 (Supertone Inc.) as a compiled Core ML model that runs on the Apple Neural Engine. It is what the Quotica iOS reader downloads to read books aloud on the device. Quotica runs it with soniqo's int8 duration predictor and text encoder (soniqo/Supertonic-3-ONNX-INT8) and a weight-only int8 vocoder (thuongvv/Supertonic-3-Vocoder-ONNX-W8), on the CPU with ONNX Runtime.

Modified files. This is not an official Supertone release. It is made from Supertone/supertonic-3 at commit 3cadd1ee (onnx/vector_estimator.onnx) by make_coreml_vector_estimator.py, which is in this repository.

What is in it

vector_estimator.mlmodelc (65 MB) is a compiled multifunction Core ML model, for MLModel(contentsOf:) with MLModelConfiguration.functionName (iOS 18 or later). Each function is the vector estimator at a fixed length:

Function	Latent frames	Text code points	Speech up to
length64	64	96	4.5 s
length160 (default)	160	224	11 s
length416	416	320	29 s

The functions share their weights, which are kept once.

Inputs and output are Supertone's, in float32:

• Pad text_emb and text_mask up to the function's text length, and noisy_latent and latent_mask up to its latent length. The padding must be zero in the masks.
• current_step and total_step are as in Supertone's reference code.
• Padding the text changes nothing. Padding the latent moves the output as much as another noise does; it is heard as the same voice.

How it was made

1. Fixed to each length, with its shape arithmetic folded by ONNX Runtime.
2. The ops Core ML lacks rewritten: edge Pad as Gather, the attention masks' Where as Mul and Add, Tile as Concat.
3. Converted to PyTorch with onnx2torch, traced, and converted to a Core ML ML Program in float16 with coremltools 9.
4. Weights quantized to int8, symmetric, per channel.
5. Merged into one multifunction model and compiled.

Against Supertone's float32 graph over 8 steps, it moves the output by 4 % (relative RMS). That is heard as the same voice.

Speed

Measured on an iPhone 15 Pro, with the rest of the pipeline:

	Speech made per second of compute	Memory
soniqo's int8 vector estimator, on the CPU	0.56 s when cool, 0.80–0.98 s once warm	245 MB
this model, on the Neural Engine	0.05 s	155 MB loaded, ~400 MB at most

• It loads in about 0.4 s.
• The first time on a device, Core ML compiles each function for the Neural Engine, in about 5 s.

License

BigScience OpenRAIL-M, as Supertone's model (LICENSE). Copyright Supertone Inc. Whoever uses or redistributes it, including in an app, must pass on the use-based restrictions of its Attachment A.