| --- |
| license: openrail++ |
| --- |
| |
|
|
|
|
|
|
|
|
|
|
| <p align="center"> |
| <img src="https://raw.githubusercontent.com/BudEcosystem/Tansen/main/Instagram%20post%20-%204.png" alt="Tensen Logo" width="300" height="300"/> |
| </p> |
|
|
| --- |
|
|
| <p align="center"><i>Democratizing access to LLMs, Multi-Modal Gen AI models for the open-source community.<br>Let's advance AI, together. </i></p> |
|
|
| --- |
|
|
|
|
| Tansen is a text-to-speech program built with the following priorities: |
|
|
| 1. Strong multi-voice capabilities. |
| 2. Highly realistic prosody and intonation. |
| 3. Speaking rate control |
|
|
|
|
| <a href="https://github.com/BudEcosystem/Tansen"><img src="https://img.shields.io/badge/GitHub-100000?style=for-the-badge&logo=github&logoColor=white" /> </a> |
|
|
|
|
| <h2 align="left">🎧 Demos </h2> |
|
|
|
|
|
|
| ### Demos |
|
|
| [random_0_0.webm](https://github.com/BudEcosystem/Tansen/assets/4546714/9a6ce191-2646-497e-bf48-003f2bf0bb8d) |
|
|
| [random_0_1.webm](https://github.com/BudEcosystem/Tansen/assets/4546714/87bf5f7c-ae47-4aa4-a110-b5c9899e4446) |
|
|
| [random_0_2.webm](https://github.com/BudEcosystem/Tansen/assets/4546714/5549c464-c670-4e7a-987c-c5d79b32bf4b) |
|
|
| <h2 align="left">💻 Getting Started on GitHub </h2> |
|
|
| Ready to dive in? Here's how you can get started with our repo on GitHub. |
|
|
| <h3 align="left">1️⃣ : Clone our GitHub repository</h3> |
|
|
| First things first, you'll need to clone our repository. Open up your terminal, navigate to the directory where you want the repository to be cloned, and run the following command: |
|
|
| ```bash |
| conda create --name Tansen python=3.9 numba inflect |
| conda activate Tansen |
| conda install pytorch torchvision torchaudio pytorch-cuda=11.7 -c pytorch -c nvidia |
| conda install transformers=4.29.2 |
| git clone https://github.com/BudEcosystem/Tansen.git |
| cd Tansen |
| ``` |
|
|
| <h3 align="left">2️⃣ : Install dependencies</h3> |
|
|
| ```bash |
| python setup.py install |
| ``` |
|
|
| <h3 align="left">3️⃣ : Generate Audio</h3> |
|
|
| ### do_tts.py |
| |
| This script allows you to speak a single phrase with one or more voices. |
| |
| ```shell |
| python do_tts.py --text "I'm going to speak this" --voice random --preset fast |
| ``` |
| |
| ### read.py |
| |
| This script provides tools for reading large amounts of text. |
| |
| ```shell |
| python Tansen/read.py --textfile <your text to be read> --voice random |
| ``` |
| |
| This will break up the textfile into sentences, and then convert them to speech one at a time. It will output a series |
| of spoken clips as they are generated. Once all the clips are generated, it will combine them into a single file and |
| output that as well. |
| |
| Sometimes Tansen screws up an output. You can re-generate any bad clips by re-running `read.py` with the --regenerate |
| argument. |
| |
| Intrested in running as as API ? |
| |
| ### 🐍 Usage in Python |
| |
| Tansen can be used programmatically : |
| |
| ```python |
| reference_clips = [utils.audio.load_audio(p, 22050) for p in clips_paths] |
| tts = api.TextToSpeech(use_deepspeed=True, kv_cache=True, half=True) |
| pcm_audio = tts.tts_with_preset("your text here", voice_samples=reference_clips, preset='fast') |
| ``` |
| |
| ## Loss Curves |
| |
| <p align="center"> |
| <img src="https://raw.githubusercontent.com/BudEcosystem/Tansen/main/results/images/loss_mel_ce.png" alt="" width="500"/> |
| <span>loss_mel_ce</span> |
| <p> |
| |
| <p align="center"> |
| <img src="https://raw.githubusercontent.com/BudEcosystem/Tansen/main/results/images/loss_text_ce.png" alt="" width="500" /> |
| <span>loss_text_ce</span> |
| <p> |
| |
| |
| ## Training Information |
| |
| Device : A Single A100 |
| |
| Dataset : 876 hours |
