wegotomars/VoxCPM2-CoreAI-ANE

VoxCPM2 on Apple Neural Engine

Core AI conversion of openbmb/VoxCPM2 for multilingual 48 kHz speech synthesis, voice design, reference cloning, and audio continuation, using tinkerbox-ane.

The MiniCPM main and residual language models, the local audio encoder, and the local diffusion transformer run fully on the ANE. Tokenization, small projections, scalar quantization, and the AudioVAE V2 decoder run on the CPU while your GPU stays idle and cool 😎❄️.

Requires macOS 27, Apple Silicon M1–M6, and Python 3.12 or 3.13. A lookup-table variant that uses less memory is published as VoxCPM2-LUT8-CoreAI-ANE.

Usage

Install tinkerbox-ane[speech]>=2.2.0. The downloader fetches only this Mac's bundle: its checkpoint folder carries the tokenizer, configuration, CPU lookup tables, and AudioVAE weights.

Generate speech and save it as WAV:

import soundfile as sf
from tinkerbox_ane import voxcpm2
from tinkerbox_ane.bundle import download_bundle

bundle = download_bundle("voxcpm2", "wegotomars/VoxCPM2-CoreAI-ANE", "v2.2.0")
with voxcpm2.load(bundle) as model:
    audio = model.generate("Hello! Welcome to Tinkerbox.", seed=42)[0]
    sf.write("speech.wav", audio, model.sampling_rate)

Design a voice from a description, or clone one from a recording:

with voxcpm2.load(bundle) as model:
    designed = model.generate("A familiar voice can tell a new story.",
                              mode="design", instruct="A warm, calm female voice", seed=7)[0]
    voice = model.create_voice_clone_prompt("reference.m4a", mode="clone")
    voice.save("voice.pt")
    cloned = model.generate("A new sentence in the same voice.",
                            mode="clone", voice_clone_prompt=voice, seed=23)[0]

mode="continuation" continues the reference recording and needs its transcript (ref_text=); without one, the transcript comes from Apple's on-device speech recognition. mode="clone_continuation" combines both. Language is detected from the text.

Play speech while it is generated; the pieces join into exactly what generate returns:

with voxcpm2.load(bundle) as model:
    for piece in model.stream("Playback can begin with the first words.", seed=42):
        ...  # float32 pieces at model.sampling_rate

Text, reference audio, and generated audio share an 8,192-position context, as in the original model; each generated position is 160 ms of audio.

Supported architectures

Architecture	SoC name
h13g	Apple M1
h13s	Apple M1 Pro
h13c	Apple M1 Max
h13d	Apple M1 Ultra
h14g	Apple M2
h14s	Apple M2 Pro
h14c	Apple M2 Max
h14d	Apple M2 Ultra
h15g	Apple M3
h15s	Apple M3 Pro
h15m	Apple M3 Max (14-core CPU)
h15c	Apple M3 Max (16-core CPU)
h15d	Apple M3 Ultra
h16g	Apple M4
h16s	Apple M4 Pro
h16c	Apple M4 Max
h17g	Apple M5
h17s	Apple M5 Pro
h17c	Apple M5 Max
h17d	Apple M5 Ultra
h18g	Apple M6

Benchmarks

Measured October 26, 2026 on Apple M5 Pro (h17s, 24 GiB), macOS 27.0.1, warm. Real-time multiplier is audio duration divided by generation time (higher is faster); RTF is its inverse (lower is faster).

Request	Audio	Generation time	Real-time multiplier	RTF	ANE energy
Greeting, automatic voice	4.5 s	1.21 s	3.7×	0.27	2.6 J
Short reference clone	3.8 s	1.04 s	3.7×	0.27	2.1 J
Long passage, automatic voice	21.9 s	5.73 s	3.8×	0.26	10.7 J

Memory	Peak
Wired, including ANE weights	3.9 GiB
Python process	0.46 GiB

Streaming playback starts after 0.26 s for a short greeting and 0.32 s for a 51-second passage.

Accuracy

Compared with the original VoxCPM2 in CPU FP16 on identical inputs:

Check	Result
Main-model prefill and decode, cosine	≥ 0.99998
Diffusion output, cosine	≥ 0.9996
Cache growth through 8,192 positions, cosine	0.99999
16-clip transcription, mean character error	0.028

Original checkpoint: openbmb/VoxCPM2@32279effe8c19989596f05d353d1447f51d9e915. Weights, converted assets, and upstream code are Apache-2.0. See NOTICE.