Add LingTTS model card
Browse files
README.md
ADDED
|
@@ -0,0 +1,64 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
language:
|
| 3 |
+
- zh
|
| 4 |
+
pipeline_tag: text-to-speech
|
| 5 |
+
license: other
|
| 6 |
+
license_name: lingtts-community-license-1-0
|
| 7 |
+
license_link: https://github.com/LingCoreX-AI/LingTTS/blob/main/licenses/LingTTS-Community-License-1.0.md
|
| 8 |
+
tags:
|
| 9 |
+
- esp32
|
| 10 |
+
- esp32-s3
|
| 11 |
+
- embedded
|
| 12 |
+
- offline-tts
|
| 13 |
+
- chinese-tts
|
| 14 |
+
- edge-ai
|
| 15 |
+
---
|
| 16 |
+
|
| 17 |
+
# LingTTS: Offline Chinese Neural Text-to-Speech for ESP32-S3
|
| 18 |
+
|
| 19 |
+
[简体中文说明](https://github.com/LingCoreX-AI/LingTTS/blob/main/README.md) | English
|
| 20 |
+
|
| 21 |
+
The model and engine are distributed through the community SDK on GitHub; this repository contains the model card, not standalone weights. It is not a Transformers-ready checkpoint.
|
| 22 |
+
|
| 23 |
+
LingTTS is a neural text-to-speech model for offline Mandarin speech synthesis on ESP32-S3-class devices. It generates speech locally from Chinese text, including stories, short passages and device information. The ESP-IDF SDK provides an integration interface.
|
| 24 |
+
|
| 25 |
+
[Try the online demo](https://edge.lingcorex.cn) — the demo runs the same C code and model compiled for a PC server; use a board to evaluate device performance.
|
| 26 |
+
|
| 27 |
+
## Can ESP32-S3 run neural TTS offline?
|
| 28 |
+
|
| 29 |
+
Yes. LingTTS runs the synthesis locally on ESP32-S3 without a cloud connection. The current community preview is 0.2.0-rc1. It supports Mandarin Chinese; English support is limited to letters and around 80 common words, not full English sentences.
|
| 30 |
+
|
| 31 |
+
## ESP32-S3 performance and memory
|
| 32 |
+
|
| 33 |
+
The measurements below are for ESP32-S3 N16R8 at 240 MHz with ESP-IDF 5.5.6. Results depend on the application and memory configuration. RTF is synthesis time divided by generated audio duration; lower is faster.
|
| 34 |
+
|
| 35 |
+
| Metric | Measured result |
|
| 36 |
+
| --- | --- |
|
| 37 |
+
| Audio output | 16 kHz, 16-bit mono PCM, delivered in chunks |
|
| 38 |
+
| RTF | 0.42 with PSRAM / 0.48 using internal RAM, hello example including text front end; 0.53–0.54 in the Wi-Fi demo |
|
| 39 |
+
| First PCM | 0.27–0.32 seconds from invocation, not speaker playback latency |
|
| 40 |
+
| Memory | 2.63 MB PSRAM plus about 1 KB internal RAM, or 151 KB internal RAM without PSRAM; task stack, DMA and application memory are extra |
|
| 41 |
+
| Flash | About 2.6 MB for library, model and dictionary; fits within 4 MB Flash subject to application and partition needs |
|
| 42 |
+
|
| 43 |
+
See the [performance details](https://github.com/LingCoreX-AI/LingTTS/blob/main/docs/资源与性能.md) and [integration guide](https://github.com/LingCoreX-AI/LingTTS/blob/main/docs/集成指南.md) (Chinese).
|
| 44 |
+
|
| 45 |
+
## Try it on a board
|
| 46 |
+
|
| 47 |
+
The demo requires an ESP32-S3 N16R8 board, MAX98357A amplifier and speaker. Connect BCLK to GPIO 15, LRC to GPIO 16, and DIN to GPIO 7. Download the community demo package from [Releases](https://github.com/LingCoreX-AI/LingTTS/releases), extract it and run:
|
| 48 |
+
|
| 49 |
+
```bash
|
| 50 |
+
python -m pip install esptool==4.12.0
|
| 51 |
+
python flash.py --port COM5 # Linux / WSL: python3 flash.py --port /dev/ttyUSB0
|
| 52 |
+
```
|
| 53 |
+
|
| 54 |
+
Connect a phone to the `LingTTS-Demo` Wi-Fi hotspot (password `lingtts-demo`), open http://192.168.4.1, enter Chinese text and play it. For ESP-IDF integration, download the community SDK package from Releases and see the [Chinese README](https://github.com/LingCoreX-AI/LingTTS/blob/main/README.md#在自己的工程里用).
|
| 55 |
+
|
| 56 |
+
## Limitations
|
| 57 |
+
|
| 58 |
+
The current release offers a Mandarin voice. English is limited as described above; SSML, emotion and speed controls are not supported. Polyphonic characters, names and model numbers may require text adaptation. Other chips have not yet been adapted.
|
| 59 |
+
|
| 60 |
+
## License
|
| 61 |
+
|
| 62 |
+
Examples, public headers and integration files use the [MIT license](https://github.com/LingCoreX-AI/LingTTS/blob/main/licenses/MIT.txt). The precompiled engine, model, dictionary and demo firmware use the separate [LingTTS Community License 1.0](https://github.com/LingCoreX-AI/LingTTS/blob/main/licenses/LingTTS-Community-License-1.0.md). The core and model are not open-source software. Personal projects, education, and internal company R&D, testing and prototypes are free; commercial deployment requires written authorization. Contact service@lingcorex.cn.
|
| 63 |
+
|
| 64 |
+
Report board, documentation or pronunciation issues via [GitHub Issues](https://github.com/LingCoreX-AI/LingTTS/issues), including the board, SDK version, memory configuration and relevant logs. Do not include secrets or personal information.
|