Update README.md
Browse files
README.md
CHANGED
|
@@ -8,50 +8,31 @@ tags:
|
|
| 8 |
- text-generation-inference
|
| 9 |
- TTS
|
| 10 |
---
|
| 11 |
-
# 🎙️ MamreVoice: State-of-the-Art Hebrew TTS
|
| 12 |
|
| 13 |
-
|
| 14 |
-
<img src="https://raw.githubusercontent.com/maxmelichov/mamre-site/main/static/images/logo.png" alt="Mamre Logo" width="200"/>
|
| 15 |
-
</p>
|
| 16 |
|
| 17 |
-
[**Project Site**](https://maxmelichov.github.io/mamre-site/) | [**GitHub**](https://github.com/maxmelichov)
|
| 18 |
|
| 19 |
-
MamreVoice is a high-performance, end-to-end Text-to-Speech (TTS)
|
| 20 |
|
| 21 |
## ✨ Features
|
| 22 |
|
| 23 |
-
* **
|
| 24 |
-
* **
|
| 25 |
-
* **
|
| 26 |
-
* **
|
| 27 |
-
* **Multi-Lingual Foundation**: While specialized for Hebrew, the model maintains high-fidelity across various acoustic environments.
|
| 28 |
-
|
| 29 |
-
---
|
| 30 |
-
|
| 31 |
-
## 🚀 Performance Benchmarks
|
| 32 |
-
|
| 33 |
-
MamreVoice is optimized for extreme throughput without sacrificing quality.
|
| 34 |
-
|
| 35 |
-
| Hardware | RTF (Real-Time Factor) | Description |
|
| 36 |
-
| :--- | :--- | :--- |
|
| 37 |
-
| **NVIDIA RTX 4090** | ~1,500x | Near-instant generation |
|
| 38 |
-
| **NVIDIA RTX 3090** | 1,260x | High-end workstation |
|
| 39 |
-
| **CPU (i7-12700K)** | 35x | Faster than real-time |
|
| 40 |
-
| **Apple M2 Max** | 25x | Efficient mobile performance |
|
| 41 |
|
| 42 |
---
|
| 43 |
|
| 44 |
## 🛠️ Technical Overview
|
| 45 |
|
| 46 |
-
###
|
| 47 |
-
|
| 48 |
-
1. **Normalization**: Handling numbers, dates, and symbols.
|
| 49 |
-
2. **Phonikud**: Automatic vowelization and phonetic transcription.
|
| 50 |
-
3. **DiffMamba Backbone**: The core model predicts the mel-spectrogram using a State Space Model (SSM) backbone.
|
| 51 |
-
4. **Vocoder**: High-fidelity reconstruction into 24kHz or 44.1kHz audio.
|
| 52 |
|
|
|
|
|
|
|
|
|
|
|
|
|
| 53 |
|
| 54 |
|
| 55 |
-
### Requirements
|
| 56 |
```bash
|
| 57 |
-
pip install torch torchaudio phonikud onnxruntime-gpu
|
|
|
|
| 8 |
- text-generation-inference
|
| 9 |
- TTS
|
| 10 |
---
|
|
|
|
| 11 |
|
| 12 |
+
# 🎙️ MamreVoice: State-of-the-Art Hebrew TTS
|
|
|
|
|
|
|
| 13 |
|
| 14 |
+
[**Project Site**](https://maxmelichov.github.io/mamre-site/) | [**GitHub**](https://github.com/maxmelichov)
|
| 15 |
|
| 16 |
+
MamreVoice is a high-performance, end-to-end Text-to-Speech (TTS) and Voice Conversion system specifically optimized for the Hebrew language. Built on the innovative **DiffMamba** architecture, it delivers expressive, clear, and lifelike synthetic speech.
|
| 17 |
|
| 18 |
## ✨ Features
|
| 19 |
|
| 20 |
+
* **Expressive Hebrew Synthesis**: Specialized optimization for Hebrew prosody, rhythm, and naturalness.
|
| 21 |
+
* **Voice Conversion**: High-quality voice cloning and identity preservation.
|
| 22 |
+
* **Phonetics-Aware**: Seamless integration with **Phonikud** for accurate Hebrew diacritization (Niqqud) and vowelization.
|
| 23 |
+
* **Efficiency**: Designed for low computational overhead, enabling real-time performance on modern Nvidia GPU hardware.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 24 |
|
| 25 |
---
|
| 26 |
|
| 27 |
## 🛠️ Technical Overview
|
| 28 |
|
| 29 |
+
### Architecture
|
| 30 |
+
MamreVoice utilizes **DiffMamba**, which addresses the computational heavy-lifting of traditional diffusion models by using Mamba blocks. This allows for long-context handling and faster inference without sacrificing audio quality.
|
|
|
|
|
|
|
|
|
|
|
|
|
| 31 |
|
| 32 |
+
### Components
|
| 33 |
+
1. **Text Processing**: Advanced normalization and diacritization via [Phonikud](https://github.com/thewh1teagle/phonikud).
|
| 34 |
+
2. **BackBone Model**: DiffMamba-based Backbone
|
| 35 |
+
---
|
| 36 |
|
| 37 |
|
|
|
|
| 38 |
```bash
|
|
|