Text-to-Speech
Hebrew
text-generation-inference
TTS
notmax123 commited on
Commit
536541d
·
verified ·
1 Parent(s): 47d0d09

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +13 -32
README.md CHANGED
@@ -8,50 +8,31 @@ tags:
8
  - text-generation-inference
9
  - TTS
10
  ---
11
- # 🎙️ MamreVoice: State-of-the-Art Hebrew TTS
12
 
13
- <p align="center">
14
- <img src="https://raw.githubusercontent.com/maxmelichov/mamre-site/main/static/images/logo.png" alt="Mamre Logo" width="200"/>
15
- </p>
16
 
17
- [**Project Site**](https://maxmelichov.github.io/mamre-site/) | [**GitHub**](https://github.com/maxmelichov) | [**Demo**](https://huggingface.co/spaces/maxmelichov/mamre-demo)
18
 
19
- MamreVoice is a high-performance, end-to-end Text-to-Speech (TTS) engine specifically optimized for the Hebrew language. By combining the **DiffMamba** architecture with advanced Hebrew diacritization, it achieves studio-quality synthesis with remarkably low latency.
20
 
21
  ## ✨ Features
22
 
23
- * **DiffMamba Architecture**: Utilizes a 2B parameter model merging Diffusion generation with Mamba (SSM) efficiency.
24
- * **Zero-Shot Voice Cloning**: Clone any Hebrew voice with just a 5-10 second reference clip.
25
- * **Advanced Diacritization**: Built-in support for **Phonikud**, ensuring correct pronunciation of Niqqud (vowels).
26
- * **Production Ready**: Optimized for ONNX and TensorRT, making it viable for real-time applications.
27
- * **Multi-Lingual Foundation**: While specialized for Hebrew, the model maintains high-fidelity across various acoustic environments.
28
-
29
- ---
30
-
31
- ## 🚀 Performance Benchmarks
32
-
33
- MamreVoice is optimized for extreme throughput without sacrificing quality.
34
-
35
- | Hardware | RTF (Real-Time Factor) | Description |
36
- | :--- | :--- | :--- |
37
- | **NVIDIA RTX 4090** | ~1,500x | Near-instant generation |
38
- | **NVIDIA RTX 3090** | 1,260x | High-end workstation |
39
- | **CPU (i7-12700K)** | 35x | Faster than real-time |
40
- | **Apple M2 Max** | 25x | Efficient mobile performance |
41
 
42
  ---
43
 
44
  ## 🛠️ Technical Overview
45
 
46
- ### The Pipeline
47
- The model processes text through a specialized Hebrew pipeline:
48
- 1. **Normalization**: Handling numbers, dates, and symbols.
49
- 2. **Phonikud**: Automatic vowelization and phonetic transcription.
50
- 3. **DiffMamba Backbone**: The core model predicts the mel-spectrogram using a State Space Model (SSM) backbone.
51
- 4. **Vocoder**: High-fidelity reconstruction into 24kHz or 44.1kHz audio.
52
 
 
 
 
 
53
 
54
 
55
- ### Requirements
56
  ```bash
57
- pip install torch torchaudio phonikud onnxruntime-gpu
 
8
  - text-generation-inference
9
  - TTS
10
  ---
 
11
 
12
+ # 🎙️ MamreVoice: State-of-the-Art Hebrew TTS
 
 
13
 
14
+ [**Project Site**](https://maxmelichov.github.io/mamre-site/) | [**GitHub**](https://github.com/maxmelichov)
15
 
16
+ MamreVoice is a high-performance, end-to-end Text-to-Speech (TTS) and Voice Conversion system specifically optimized for the Hebrew language. Built on the innovative **DiffMamba** architecture, it delivers expressive, clear, and lifelike synthetic speech.
17
 
18
  ## ✨ Features
19
 
20
+ * **Expressive Hebrew Synthesis**: Specialized optimization for Hebrew prosody, rhythm, and naturalness.
21
+ * **Voice Conversion**: High-quality voice cloning and identity preservation.
22
+ * **Phonetics-Aware**: Seamless integration with **Phonikud** for accurate Hebrew diacritization (Niqqud) and vowelization.
23
+ * **Efficiency**: Designed for low computational overhead, enabling real-time performance on modern Nvidia GPU hardware.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
24
 
25
  ---
26
 
27
  ## 🛠️ Technical Overview
28
 
29
+ ### Architecture
30
+ MamreVoice utilizes **DiffMamba**, which addresses the computational heavy-lifting of traditional diffusion models by using Mamba blocks. This allows for long-context handling and faster inference without sacrificing audio quality.
 
 
 
 
31
 
32
+ ### Components
33
+ 1. **Text Processing**: Advanced normalization and diacritization via [Phonikud](https://github.com/thewh1teagle/phonikud).
34
+ 2. **BackBone Model**: DiffMamba-based Backbone
35
+ ---
36
 
37
 
 
38
  ```bash