tuanpm commited on
Commit
3f2bd7f
·
verified ·
1 Parent(s): 6fd7cb9

vtts Vietnamese prototype voices v0.3.0 (int8, ESP32-P4 / ESP32-S3)

Browse files
Files changed (5) hide show
  1. .gitattributes +2 -0
  2. README.md +71 -0
  3. SHA256SUMS +2 -0
  4. vtts-vi-m-v0.2.q8.vtts +3 -0
  5. vtts-vi-s-v0.1.q8.vtts +3 -0
.gitattributes CHANGED
@@ -33,3 +33,5 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
 
 
 
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
36
+ vtts-vi-m-v0.2.q8.vtts filter=lfs diff=lfs merge=lfs -text
37
+ vtts-vi-s-v0.1.q8.vtts filter=lfs diff=lfs merge=lfs -text
README.md ADDED
@@ -0,0 +1,71 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: cc-by-nc-4.0
3
+ language:
4
+ - vi
5
+ pipeline_tag: text-to-speech
6
+ tags:
7
+ - text-to-speech
8
+ - tts
9
+ - vietnamese
10
+ - esp32
11
+ - esp32-p4
12
+ - esp32-s3
13
+ - microcontroller
14
+ - int8
15
+ - on-device
16
+ - tinyml
17
+ library_name: vtts
18
+ ---
19
+
20
+ # vtts — Vietnamese prototype voices for ESP32
21
+
22
+ Tiny int8 text-to-speech models that run **entirely on an ESP32 microcontroller** with the
23
+ [vtts](https://github.com/tuanpmt/vtts) C99 runtime: text normalization, pronunciation, the neural
24
+ voice and the iSTFT vocoder all run on the chip. Listen to audio generated on an ESP32-P4:
25
+ **https://tuanpmt.github.io/vtts/**
26
+
27
+ | File | Tier | Params | Size | ESP32-P4 RTF | ESP32-S3 RTF |
28
+ |---|---|---:|---:|---:|---:|
29
+ | `vtts-vi-m-v0.2.q8.vtts` | Quality (M-vocos, pitch/energy predictor) | 1.73M | 2.6 MB | 0.34–0.42 | 0.85–0.99 |
30
+ | `vtts-vi-s-v0.1.q8.vtts` | Lite (S-vocos) | 0.77M | 1.1 MB | 0.20–0.24 | 0.42–0.47 |
31
+
32
+ RTF below 1 means faster than real time (two cores, int8 SIMD). Verify downloads with `SHA256SUMS`.
33
+
34
+ ## Quality
35
+
36
+ 50 out-of-domain sentences, audio produced by the int8 C runtime; CER from PhoWhisper-large
37
+ (tones included), UTMOS predicts naturalness.
38
+
39
+ | System | CER ↓ | UTMOS ↑ |
40
+ |---|---:|---:|
41
+ | vtts M v0.2 | 0.012 | 2.19 |
42
+ | vtts S v0.1 | 0.023 | 1.86 |
43
+ | sanoTTS `vi` | 0.064 | 1.49 |
44
+ | Piper `vi_VN-25hours_single` | 0.089 | 2.06 |
45
+
46
+ ## Use
47
+
48
+ ```bash
49
+ git clone https://github.com/tuanpmt/vtts && cd vtts/firmware/esp32
50
+ BOARD=nano ./build.sh /path/vtts-vi-s-v0.1.q8.vtts /path/vtts-vi-m-v0.2.q8.vtts -p PORT flash monitor
51
+ ```
52
+
53
+ `BOARD` is `panel` (ESP32-P4-86-Panel), `nano` (ESP32-P4-NANO) or `s3` (ESP32-S3-Touch-LCD-2.8, 8 MB PSRAM).
54
+ On a PC: `make -C runtime build/vtts_cli`, then pipe phoneme ids from the Python front-end. The file
55
+ format is documented in `docs/model-format.md` of the code repository.
56
+
57
+ ## Training
58
+
59
+ Distilled from a VITS teacher into 0.8–1.7M-parameter students (duration, acoustic + pitch,
60
+ Vocos-style ConvNeXt + iSTFT decoder), quantized to int8 per output channel. Single male speaker,
61
+ Northern Vietnamese, spelling-based token set (65 tokens).
62
+
63
+ ## License and limitations
64
+
65
+ - **CC BY-NC 4.0, non-commercial only.** The teacher was initialized from a non-commercial Piper
66
+ checkpoint and the commercial terms of the training corpus are unclear. The runtime code is
67
+ Apache-2.0; that license does not cover these model files.
68
+ - Prototype quality: intonation is still somewhat flat; rare syllables may be misread.
69
+ - Bilingual Vietnamese–English voices and other languages are in development; see the
70
+ [roadmap](https://github.com/tuanpmt/vtts#roadmap).
71
+ - Do not use the voice to impersonate people; label generated audio as synthetic.
SHA256SUMS ADDED
@@ -0,0 +1,2 @@
 
 
 
1
+ 9b4b78ed10c08ba0ccbb0af2cb78a1c353189be38f2ef7fca180d83fb8c585ee vtts-vi-m-v0.2.q8.vtts
2
+ 914a70e3aa41e510744447991916f9df1f160df6bcbbfa2911a2d6a63eca3020 vtts-vi-s-v0.1.q8.vtts
vtts-vi-m-v0.2.q8.vtts ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:9b4b78ed10c08ba0ccbb0af2cb78a1c353189be38f2ef7fca180d83fb8c585ee
3
+ size 2553408
vtts-vi-s-v0.1.q8.vtts ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:914a70e3aa41e510744447991916f9df1f160df6bcbbfa2911a2d6a63eca3020
3
+ size 1137856