Instructions to use mlboydaisuke/VoxCPM2-CoreAI with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- VoxCPM
How to use mlboydaisuke/VoxCPM2-CoreAI with VoxCPM:
import soundfile as sf from voxcpm import VoxCPM model = VoxCPM.from_pretrained("mlboydaisuke/VoxCPM2-CoreAI") wav = model.generate( text="VoxCPM is an innovative end-to-end TTS model from ModelBest, designed to generate highly expressive speech.", prompt_wav_path=None, # optional: path to a prompt speech for voice cloning prompt_text=None, # optional: reference text cfg_value=2.0, # LM guidance on LocDiT, higher for better adherence to the prompt, but maybe worse inference_timesteps=10, # LocDiT inference timesteps, higher for better result, lower for fast speed normalize=True, # enable external TN tool denoise=True, # enable external Denoise tool retry_badcase=True, # enable retrying mode for some bad cases (unstoppable) retry_badcase_max_times=3, # maximum retrying times retry_badcase_ratio_threshold=6.0, # maximum length restriction for bad case detection (simple but effective), it could be adjusted for slow pace speech ) sf.write("output.wav", wav, 16000) print("saved: output.wav") - Notebooks
- Google Colab
- Kaggle
README: the 512-position KV cache and what CoreAIKit does at its end; iPhone 18 Pro: the ios/ graphs did not load (2026-11-13)
Browse files
README.md
CHANGED
|
@@ -110,6 +110,10 @@ continuous-feedback path is quant-sensitive — same split mlx-community uses).
|
|
| 110 |
|
| 111 |
## iPhone 18 Pro, the JIT bundles in `ios/` (load only)
|
| 112 |
|
|
|
|
|
|
|
|
|
|
|
|
|
| 113 |
Measured 2026-09-26 on an iPhone 18 Pro (iPhone19,2, iOS 27.0 build 24A437, h19p) with the zoo's
|
| 114 |
DecideGate app in its load-only mode, without the increased-memory entitlement. Each first load was the
|
| 115 |
first after a fresh install of the app. The call is one run on all-zero inputs; a graph with a KV state
|
|
@@ -139,6 +143,10 @@ let tts = try await VoxCPM2TTS(paths: .standard(artifactsRoot: root, lm: .int8))
|
|
| 139 |
let wav = try await tts.synthesize("On device speech synthesis, running entirely on your iPhone.") // 48 kHz Float PCM
|
| 140 |
```
|
| 141 |
|
|
|
|
|
|
|
|
|
|
|
|
|
| 142 |
## Verification
|
| 143 |
|
| 144 |
Reimplemented in exportable Core AI overlays and gated end-to-end against the official model: backbone /
|
|
|
|
| 110 |
|
| 111 |
## iPhone 18 Pro, the JIT bundles in `ios/` (load only)
|
| 112 |
|
| 113 |
+
On an iPhone 18 Pro with iOS 27.2 beta (build 24B5099f), a test app ended twice on 2026-11-13 while CoreAIKit's `VoxCPM2TTS` loaded these `ios/` graphs, so VoxCPM2 has not produced audio on that phone. One run left a crash report: an abort from `MTLReportFailure` in `-[MPSGraphExecutable serializeToMPSGraphPackageAtURL:descriptor:]`, during the phone's on-device compile, with no assertion message. Both runs had loaded VoxCPM-0.5B and VibeVoice in the same process first; VoxCPM2 alone was not tried, and the cause is not known. The `macos/` graphs are the same seven files as `ios/` (the same sha256), and on a Mac (M4 Max, macOS 27.0) `VoxCPM2TTS` loaded them and produced audio on 2026-11-14.
|
| 114 |
+
|
| 115 |
+
For speech on an iPhone 18 Pro, use [VoxCPM-0.5B-CoreAI](https://huggingface.co/mlboydaisuke/VoxCPM-0.5B-CoreAI) (16 kHz): it loaded and produced audio in the same runs.
|
| 116 |
+
|
| 117 |
Measured 2026-09-26 on an iPhone 18 Pro (iPhone19,2, iOS 27.0 build 24A437, h19p) with the zoo's
|
| 118 |
DecideGate app in its load-only mode, without the increased-memory entitlement. Each first load was the
|
| 119 |
first after a fresh install of the app. The call is one run on all-zero inputs; a graph with a KV state
|
|
|
|
| 143 |
let wav = try await tts.synthesize("On device speech synthesis, running entirely on your iPhone.") // 48 kHz Float PCM
|
| 144 |
```
|
| 145 |
|
| 146 |
+
A clip's text and speech share the base and residual decode graphs' fixed KV cache of 512 positions: one position per text token and one per speech frame. A clip is at most 300 frames (48 s) by default (`maxFrames`), so a text of up to 211 tokens (about 800 characters of English) always fits, and a longer one can run out of room. From the CoreAIKit release after 0.7.3, the clip then ends where the cache ends, before the end of the text, and a text that leaves no room for speech throws `VoxCPMError.textTooLong` (VoxCPM2 shares VoxCPM's errors). CoreAIKit 0.7.3 and earlier do not stop at the end of the cache, and in a Mac test the kit raised no error and kept producing audio past it, which has not been listened to.
|
| 147 |
+
|
| 148 |
+
`KitSpeaker` and `CoreAI.speak` pass the whole text at once: split a long text into sentences and synthesize them one by one.
|
| 149 |
+
|
| 150 |
## Verification
|
| 151 |
|
| 152 |
Reimplemented in exportable Core AI overlays and gated end-to-end against the official model: backbone /
|