File size: 3,325 Bytes
b30e24d 34c877a b30e24d 34c877a b30e24d 34c877a b30e24d 15999ec b30e24d 15999ec b30e24d 34c877a | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 | ---
license: mit
tags:
- text-to-speech
- executorch
- react-native-executorch
- pocket-tts
---
# react-native-executorch-pocket-tts
An [ExecuTorch](https://pytorch.org/executorch/) (`.pte`) export of
[kyutai-labs/pocket-tts](https://github.com/kyutai-labs/pocket-tts) for
[react-native-executorch](https://github.com/software-mansion/react-native-executorch)
(RNE). Independent export, not affiliated with Kyutai or Software Mansion.
pocket-tts is licensed MIT by Kyutai; this repo redistributes derived model weights
under the same terms. Voice files are pocket-tts's own predefined voices
(`kyutai/tts-voices`) — check that repo for per-voice licensing before redistributing
further.
## Status: XNNPACK-delegated, verified on-device
These `.pte` files are **XNNPACK-delegated** and verified end-to-end on an iPhone 17
simulator through RNE's generic `ExecutorchModule` binding (react-native-executorch
0.9.2, completely stock — no patches, no custom native modules): all three voices
generate real speech, with the model signaling its own EOS, ~0.33x realtime on the
simulator. Full pipeline, export scripts and the debugging writeup live in the
`pocket-tts-export` project this was built from
(`notes/plan.md`, `notes/phase0-2-findings.md`).
Export-side fixes that make the XNNPACK artifacts on-device-safe (see the export
project's notes for the whole story — the original failure was a broken `.pte`
serialization plus on-device-only XNNPACK kernel crashes, both fixed in the export):
- Mimi decode is **two graphs** (`mimi_latent_step` + `mimi_pcm_step`), split at the
sub-frame/PCM boundary — a single merged graph at this context does not survive a
fresh-process load.
- Every `.pte` is smoke-tested in a fresh process before shipping.
- Small streaming-tail op families (and the SEANet conv stack in `mimi_pcm_step`) are
kept off the XNNPACK delegate, where RNE's bundled XNNPACK build crashes; the bulk
linear/matmul work stays delegated.
## Layout
Files live at the repo root on the `v1` tag (the `resolve/v1/<file>` convention
`react-native-executorch` uses):
```
text_conditioner.pte 16MB - text tokens -> embeddings (dynamic seq len)
flow_lm_prompt_step.pte 264MB - text embeddings -> primed KV cache (dynamic prompt len)
flow_lm_decode_step.pte 341MB - one autoregressive decode step (6-layer transformer)
mimi_latent_step.pte 24MB - latent frame -> 16 decoder sub-frames + state
mimi_pcm_step.pte 16MB - sub-frames + conv state -> 1920 PCM samples
state_manifest.json - exact state tensor shapes + emb_mean/emb_std, for the JS driver
tokenizer.json - SentencePiece tokenizer (HF format)
voices/
alba.safetensors 12MB
george.safetensors 12MB
eve.safetensors 12MB
```
## Consuming these
`.pte` files here are **not** loadable through RNE's `TextToSpeechModule` — that's
hardcoded to Kokoro's graph shape. Load each file through RNE's generic
`ExecutorchModule` binding instead and drive the prompt → decode → Mimi loop
yourself: text_conditioner → flow_lm_prompt_step (on the voice-primed caches) →
[flow_lm_decode_step + mimi_latent_step + mimi_pcm_step] per autoregressive step.
`pocket-tts-export`'s `mobile-demo/App.tsx` is a complete reference implementation.
|