| --- |
| license: mit |
| tags: |
| - text-to-speech |
| - executorch |
| - react-native-executorch |
| - pocket-tts |
| --- |
| |
| # react-native-executorch-pocket-tts |
|
|
| An [ExecuTorch](https://pytorch.org/executorch/) (`.pte`) export of |
| [kyutai-labs/pocket-tts](https://github.com/kyutai-labs/pocket-tts) for |
| [react-native-executorch](https://github.com/software-mansion/react-native-executorch) |
| (RNE). Independent export, not affiliated with Kyutai or Software Mansion. |
|
|
| pocket-tts is licensed MIT by Kyutai; this repo redistributes derived model weights |
| under the same terms. Voice files are pocket-tts's own predefined voices |
| (`kyutai/tts-voices`) — check that repo for per-voice licensing before redistributing |
| further. |
|
|
| ## Status: XNNPACK-delegated, verified on-device |
|
|
| These `.pte` files are **XNNPACK-delegated** and verified end-to-end on an iPhone 17 |
| simulator through RNE's generic `ExecutorchModule` binding (react-native-executorch |
| 0.9.2, completely stock — no patches, no custom native modules): all three voices |
| generate real speech, with the model signaling its own EOS, ~0.33x realtime on the |
| simulator. Full pipeline, export scripts and the debugging writeup live in the |
| `pocket-tts-export` project this was built from |
| (`notes/plan.md`, `notes/phase0-2-findings.md`). |
|
|
| Export-side fixes that make the XNNPACK artifacts on-device-safe (see the export |
| project's notes for the whole story — the original failure was a broken `.pte` |
| serialization plus on-device-only XNNPACK kernel crashes, both fixed in the export): |
| - Mimi decode is **two graphs** (`mimi_latent_step` + `mimi_pcm_step`), split at the |
| sub-frame/PCM boundary — a single merged graph at this context does not survive a |
| fresh-process load. |
| - Every `.pte` is smoke-tested in a fresh process before shipping. |
| - Small streaming-tail op families (and the SEANet conv stack in `mimi_pcm_step`) are |
| kept off the XNNPACK delegate, where RNE's bundled XNNPACK build crashes; the bulk |
| linear/matmul work stays delegated. |
|
|
| ## Layout |
|
|
| Files live at the repo root on the `v1` tag (the `resolve/v1/<file>` convention |
| `react-native-executorch` uses): |
|
|
| ``` |
| text_conditioner.pte 16MB - text tokens -> embeddings (dynamic seq len) |
| flow_lm_prompt_step.pte 264MB - text embeddings -> primed KV cache (dynamic prompt len) |
| flow_lm_decode_step.pte 341MB - one autoregressive decode step (6-layer transformer) |
| mimi_latent_step.pte 24MB - latent frame -> 16 decoder sub-frames + state |
| mimi_pcm_step.pte 16MB - sub-frames + conv state -> 1920 PCM samples |
| state_manifest.json - exact state tensor shapes + emb_mean/emb_std, for the JS driver |
| tokenizer.json - SentencePiece tokenizer (HF format) |
| voices/ |
| alba.safetensors 12MB |
| george.safetensors 12MB |
| eve.safetensors 12MB |
| ``` |
|
|
| ## Consuming these |
|
|
| `.pte` files here are **not** loadable through RNE's `TextToSpeechModule` — that's |
| hardcoded to Kokoro's graph shape. Load each file through RNE's generic |
| `ExecutorchModule` binding instead and drive the prompt → decode → Mimi loop |
| yourself: text_conditioner → flow_lm_prompt_step (on the voice-primed caches) → |
| [flow_lm_decode_step + mimi_latent_step + mimi_pcm_step] per autoregressive step. |
| `pocket-tts-export`'s `mobile-demo/App.tsx` is a complete reference implementation. |
| |
