vtts Vietnamese prototype voices v0.3.0 (int8, ESP32-P4 / ESP32-S3)
Browse files- .gitattributes +2 -0
- README.md +71 -0
- SHA256SUMS +2 -0
- vtts-vi-m-v0.2.q8.vtts +3 -0
- vtts-vi-s-v0.1.q8.vtts +3 -0
.gitattributes
CHANGED
|
@@ -33,3 +33,5 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
|
| 33 |
*.zip filter=lfs diff=lfs merge=lfs -text
|
| 34 |
*.zst filter=lfs diff=lfs merge=lfs -text
|
| 35 |
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
|
|
|
|
|
|
|
|
| 33 |
*.zip filter=lfs diff=lfs merge=lfs -text
|
| 34 |
*.zst filter=lfs diff=lfs merge=lfs -text
|
| 35 |
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
| 36 |
+
vtts-vi-m-v0.2.q8.vtts filter=lfs diff=lfs merge=lfs -text
|
| 37 |
+
vtts-vi-s-v0.1.q8.vtts filter=lfs diff=lfs merge=lfs -text
|
README.md
ADDED
|
@@ -0,0 +1,71 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: cc-by-nc-4.0
|
| 3 |
+
language:
|
| 4 |
+
- vi
|
| 5 |
+
pipeline_tag: text-to-speech
|
| 6 |
+
tags:
|
| 7 |
+
- text-to-speech
|
| 8 |
+
- tts
|
| 9 |
+
- vietnamese
|
| 10 |
+
- esp32
|
| 11 |
+
- esp32-p4
|
| 12 |
+
- esp32-s3
|
| 13 |
+
- microcontroller
|
| 14 |
+
- int8
|
| 15 |
+
- on-device
|
| 16 |
+
- tinyml
|
| 17 |
+
library_name: vtts
|
| 18 |
+
---
|
| 19 |
+
|
| 20 |
+
# vtts — Vietnamese prototype voices for ESP32
|
| 21 |
+
|
| 22 |
+
Tiny int8 text-to-speech models that run **entirely on an ESP32 microcontroller** with the
|
| 23 |
+
[vtts](https://github.com/tuanpmt/vtts) C99 runtime: text normalization, pronunciation, the neural
|
| 24 |
+
voice and the iSTFT vocoder all run on the chip. Listen to audio generated on an ESP32-P4:
|
| 25 |
+
**https://tuanpmt.github.io/vtts/**
|
| 26 |
+
|
| 27 |
+
| File | Tier | Params | Size | ESP32-P4 RTF | ESP32-S3 RTF |
|
| 28 |
+
|---|---|---:|---:|---:|---:|
|
| 29 |
+
| `vtts-vi-m-v0.2.q8.vtts` | Quality (M-vocos, pitch/energy predictor) | 1.73M | 2.6 MB | 0.34–0.42 | 0.85–0.99 |
|
| 30 |
+
| `vtts-vi-s-v0.1.q8.vtts` | Lite (S-vocos) | 0.77M | 1.1 MB | 0.20–0.24 | 0.42–0.47 |
|
| 31 |
+
|
| 32 |
+
RTF below 1 means faster than real time (two cores, int8 SIMD). Verify downloads with `SHA256SUMS`.
|
| 33 |
+
|
| 34 |
+
## Quality
|
| 35 |
+
|
| 36 |
+
50 out-of-domain sentences, audio produced by the int8 C runtime; CER from PhoWhisper-large
|
| 37 |
+
(tones included), UTMOS predicts naturalness.
|
| 38 |
+
|
| 39 |
+
| System | CER ↓ | UTMOS ↑ |
|
| 40 |
+
|---|---:|---:|
|
| 41 |
+
| vtts M v0.2 | 0.012 | 2.19 |
|
| 42 |
+
| vtts S v0.1 | 0.023 | 1.86 |
|
| 43 |
+
| sanoTTS `vi` | 0.064 | 1.49 |
|
| 44 |
+
| Piper `vi_VN-25hours_single` | 0.089 | 2.06 |
|
| 45 |
+
|
| 46 |
+
## Use
|
| 47 |
+
|
| 48 |
+
```bash
|
| 49 |
+
git clone https://github.com/tuanpmt/vtts && cd vtts/firmware/esp32
|
| 50 |
+
BOARD=nano ./build.sh /path/vtts-vi-s-v0.1.q8.vtts /path/vtts-vi-m-v0.2.q8.vtts -p PORT flash monitor
|
| 51 |
+
```
|
| 52 |
+
|
| 53 |
+
`BOARD` is `panel` (ESP32-P4-86-Panel), `nano` (ESP32-P4-NANO) or `s3` (ESP32-S3-Touch-LCD-2.8, 8 MB PSRAM).
|
| 54 |
+
On a PC: `make -C runtime build/vtts_cli`, then pipe phoneme ids from the Python front-end. The file
|
| 55 |
+
format is documented in `docs/model-format.md` of the code repository.
|
| 56 |
+
|
| 57 |
+
## Training
|
| 58 |
+
|
| 59 |
+
Distilled from a VITS teacher into 0.8–1.7M-parameter students (duration, acoustic + pitch,
|
| 60 |
+
Vocos-style ConvNeXt + iSTFT decoder), quantized to int8 per output channel. Single male speaker,
|
| 61 |
+
Northern Vietnamese, spelling-based token set (65 tokens).
|
| 62 |
+
|
| 63 |
+
## License and limitations
|
| 64 |
+
|
| 65 |
+
- **CC BY-NC 4.0, non-commercial only.** The teacher was initialized from a non-commercial Piper
|
| 66 |
+
checkpoint and the commercial terms of the training corpus are unclear. The runtime code is
|
| 67 |
+
Apache-2.0; that license does not cover these model files.
|
| 68 |
+
- Prototype quality: intonation is still somewhat flat; rare syllables may be misread.
|
| 69 |
+
- Bilingual Vietnamese–English voices and other languages are in development; see the
|
| 70 |
+
[roadmap](https://github.com/tuanpmt/vtts#roadmap).
|
| 71 |
+
- Do not use the voice to impersonate people; label generated audio as synthetic.
|
SHA256SUMS
ADDED
|
@@ -0,0 +1,2 @@
|
|
|
|
|
|
|
|
|
|
| 1 |
+
9b4b78ed10c08ba0ccbb0af2cb78a1c353189be38f2ef7fca180d83fb8c585ee vtts-vi-m-v0.2.q8.vtts
|
| 2 |
+
914a70e3aa41e510744447991916f9df1f160df6bcbbfa2911a2d6a63eca3020 vtts-vi-s-v0.1.q8.vtts
|
vtts-vi-m-v0.2.q8.vtts
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:9b4b78ed10c08ba0ccbb0af2cb78a1c353189be38f2ef7fca180d83fb8c585ee
|
| 3 |
+
size 2553408
|
vtts-vi-s-v0.1.q8.vtts
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:914a70e3aa41e510744447991916f9df1f160df6bcbbfa2911a2d6a63eca3020
|
| 3 |
+
size 1137856
|