Hugging Face's logo

mlboydaisuke
/
VoxCPM2-CoreAI

Text-to-Speech
VoxCPM
English
Chinese
tts
core-ai
on-device
ios
Model card Files
xet
 Community
VoxCPM2-CoreAI
File size: 4,795 Bytes
91f9bad
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
adad084
0cbf395
 
d62a886
 
 
 
 
 
 
 
 
0cbf395
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
adad084
 
91f9bad
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
---
license: apache-2.0
language:
- en
- zh
pipeline_tag: text-to-speech
tags:
- text-to-speech
- tts
- core-ai
- on-device
- ios
- voxcpm
base_model: openbmb/VoxCPM2
---

# VoxCPM2 2B — Core AI (on-device, 48 kHz)

[OpenBMB **VoxCPM2** (2B)](https://huggingface.co/openbmb/VoxCPM2) converted to **Apple Core AI**, running
fully **on-device** on iPhone (A19 Pro / iPhone 17 Pro) and Mac — no network. The 2B, 48 kHz successor to
[VoxCPM-0.5B-CoreAI](https://huggingface.co/mlboydaisuke/VoxCPM-0.5B-CoreAI).

A tokenizer-free diffusion TTS: a **MiniCPM4 28-layer** text-semantic LM + an **8-layer residual** acoustic
LM drive a **12-layer LocDiT** flow-matching diffusion head, decoded by a **48 kHz AudioVAE**. Five Core AI
bundles + a few host-side projections.

<!-- gen-cards:use-it begin id=voxcpm2-2b (managed by scripts/gen-cards — edit cards.json / QuickStart.swift, not this block) -->
## Use it

⚡ **One line** — run the kit's task op on this model
(`import CoreAIOps`; no session, no model plumbing, downloads on first use):

```swift
let audio = try await CoreAI.speak(text, options: .model("voxcpm2-2b"))
```

Twenty ops, one shape — [Cookbook](https://github.com/john-rocky/coreai-kit/blob/main/docs/COOKBOOK.md).

▶️ **Run it (source)** — the [Speak runner](https://github.com/john-rocky/coreai-kit/tree/main/Examples/Speak)
(GUI + CLI, one app for every text-to-speech model in the catalog):

```bash
git clone https://github.com/john-rocky/coreai-kit
open coreai-kit/Examples/Speak/Speak.xcodeproj
# → Run, then pick "VoxCPM2 2B" in the model picker

# agents / headless (macOS):
cd coreai-kit/Examples/Speak
swift run speak-cli --model voxcpm2-2b --text "Hello from Core AI." --output hello.wav
```

💻 **Build with it** — complete; the glue is kit API, copy-paste runs:

```swift
import CoreAIKit

let speaker = try await KitSpeaker(catalog: "voxcpm2-2b")
let audio = try await speaker.synthesize(text)
// audio.samples: 48 kHz mono PCM in [-1, 1] — play it or write a WAV
```

The take-home is [`Examples/Speak/Sources/QuickStart.swift`](https://github.com/john-rocky/coreai-kit/blob/main/Examples/Speak/Sources/QuickStart.swift)
— this exact code as one typed function, no UI; the CLI is an argument shell over it, and
the GUI drives the same `KitSpeaker(catalog:)` and plays the samples.
Live playback? `synthesizeStreaming(_:onChunk:)` hands you ~0.5 s chunks as they decode,
so audio starts before the whole clip exists. The WAV container is your app's territory
(the runner ships a 20-line writer).

**Integration checklist**

- SPM: `https://github.com/john-rocky/coreai-kit` → product **CoreAIKit**
- Info.plist: none needed
- Entitlements: none needed
- First run downloads the model — 4.7 GB (Mac) / 5.7 GB (iPhone) — then it loads from the
  local cache (Application Support; progress via the `downloadProgress` callback)
- Measure in Release — Debug is ~3× slower on per-token host work
<!-- gen-cards:use-it end -->

## What's inside

| dir | contents |
|---|---|
| `macos/` | JIT `.aimodel` bundles (Mac): int8 base/res decode + prefill, fp16 feat_decoder / feat_encoder / vocoder |
| `ios/` | AOT `.aimodelc` bundles (iOS `h18p`, GPU): same five + the two int8 prefill bundles |
| `voxcpm2_host_glue/` | embed table + projections / FSQ-512 / stop-head / fusion (`.bin` + manifest) |
| `tokenizer/` | the VoxCPM2 tokenizer (Llama fast) |

The backbone LMs are **weight-only int8** (the size driver); the diffusion + VAE stay **fp16** (the
continuous-feedback path is quant-sensitive — same split mlx-community uses).

## On-device numbers (iPhone 17 Pro, int8 + prefill + streaming)

- **RTF 1.19**, **first-audio 0.65 s**, 48 kHz, ~4.9 GB resident (increased-memory entitlement).
- Streaming starts after the first ~0.65 s; the 2B is ~4× the 0.5B, so RTF sits just above realtime.

## Use it

Runs through **[coreai-kit](https://github.com/john-rocky/coreai-kit)** `VoxCPM2TTS`, wired into the
**[coreai-model-zoo](https://github.com/john-rocky/coreai-model-zoo)** `coreai-audio` app ("Voice 2B" tab).
Conversion + gates + export scripts: `coreai-model-zoo/conversion/voxcpm/` (`*_v2.py`).

```swift
let tts = try await VoxCPM2TTS(paths: .standard(artifactsRoot: root, lm: .int8))
let wav = try await tts.synthesize("On device speech synthesis, running entirely on your iPhone.") // 48 kHz Float PCM
```

## Verification

Reimplemented in exportable Core AI overlays and gated end-to-end against the official model: backbone /
feat_decoder / feat_encoder **cos 1.0**, full chain **magspec 0.996**; every exported bundle engine-gated
**cos ≥ 0.9999**.

## License

Apache-2.0 (commercial OK), inherited from [openbmb/VoxCPM2](https://huggingface.co/openbmb/VoxCPM2).
Not affiliated with OpenBMB or Apple. Community port.