itriedcoding commited on
Commit
d77d328
·
verified ·
1 Parent(s): 4dedf7f

Add model card for Sol

Browse files
Files changed (1) hide show
  1. README.md +130 -0
README.md ADDED
@@ -0,0 +1,130 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ language:
3
+ - en
4
+ - es
5
+ - fr
6
+ - hi
7
+ - it
8
+ - pt
9
+ - zh
10
+ license: apache-2.0
11
+ library_name: transformers
12
+ pipeline_tag: text-to-speech
13
+ tags:
14
+ - text-to-speech
15
+ - tts
16
+ - multilingual
17
+ - speecht5
18
+ - kokoro
19
+ - sol
20
+ ---
21
+
22
+ # Sol
23
+
24
+ **Sol** is an open text-to-speech engine: type text, get clear speech.
25
+
26
+ It comes in two parts:
27
+
28
+ 1. **The live engine** — a multilingual, multi-voice TTS service running on
29
+ [Kokoro-82M](https://huggingface.co/hexgrad/Kokoro-82M) with **49 voices across
30
+ 8 languages**. Self-hosted, CPU-only, and reachable 24/7.
31
+ 2. **Sol v1** — a `SpeechT5` acoustic model we fine-tuned on public
32
+ LibriSpeech voice data, whose weights live in **this repository**.
33
+
34
+ **Live demo:** <https://sol-tts.vercel.app> — type a prompt and hear it speak.
35
+
36
+ ## Voices & languages
37
+
38
+ | Language | Code | Voices |
39
+ |---|---|---|
40
+ | English (US) | `en-us` | 20 |
41
+ | English (UK) | `en-gb` | 8 |
42
+ | Chinese (Mandarin) | `zh` | 8 |
43
+ | Hindi | `hi` | 4 |
44
+ | Spanish | `es` | 3 |
45
+ | Portuguese (Brazil) | `pt-br` | 3 |
46
+ | Italian | `it` | 2 |
47
+ | French | `fr-fr` | 1 |
48
+
49
+ **49 voices total.** Voice ids look like `af_heart` (American female), `am_michael`
50
+ (American male), `bf_emma` (British female), `ef_dora` (Spanish female), and so on.
51
+
52
+ ## Use the live API
53
+
54
+ ```bash
55
+ curl -X POST "$SOL_API/api/tts" \
56
+ -H "Content-Type: application/json" \
57
+ -d '{"text":"hello from sol","voice":"af_heart","speed":1.0}' \
58
+ --output sol.wav
59
+ ```
60
+
61
+ | Endpoint | Method | Purpose |
62
+ |---|---|---|
63
+ | `/api/health` | GET | service status |
64
+ | `/api/languages` | GET | supported languages + voice counts |
65
+ | `/api/voices` | GET | every voice (optional `?lang=en-us`) |
66
+ | `/api/tts` | POST | `{text, voice, speed}` → `audio/wav` (24 kHz) |
67
+
68
+ Audio is silence-trimmed and peak-normalised, so output is clean with no static.
69
+
70
+ ### Multilingual samples
71
+
72
+ A sample in each language lives in
73
+ [`examples/multilingual/`](./examples/multilingual):
74
+
75
+ <audio controls src="examples/multilingual/en-us.wav"></audio>
76
+ <audio controls src="examples/multilingual/es.wav"></audio>
77
+ <audio controls src="examples/multilingual/zh.wav"></audio>
78
+
79
+ ## Sol v1 — the fine-tuned SpeechT5 model
80
+
81
+ This repository also ships **Sol v1**, a `SpeechT5` model fine-tuned on a
82
+ single-speaker subset of [LibriSpeech](https://www.openslr.org/12) (87 utterances,
83
+ light fine-tune, lr 2e-6) and paired with the `speecht5_hifigan` vocoder.
84
+
85
+ ```python
86
+ import numpy as np, soundfile as sf, torch
87
+ from transformers import SpeechT5ForTextToSpeech, SpeechT5HifiGan, SpeechT5Processor
88
+
89
+ repo = "itriedcoding/Sol"
90
+
91
+ processor = SpeechT5Processor.from_pretrained(repo)
92
+ model = SpeechT5ForTextToSpeech.from_pretrained(repo)
93
+ vocoder = SpeechT5HifiGan.from_pretrained("microsoft/speecht5_hifigan")
94
+
95
+ speaker = torch.tensor(np.load("speaker_embedding.npy")).unsqueeze(0)
96
+
97
+ inputs = processor(text="hello, i am sol.", return_tensors="pt")
98
+ with torch.no_grad():
99
+ speech = model.generate_speech(inputs["input_ids"], speaker, vocoder=vocoder)
100
+
101
+ sf.write("sol.wav", speech.numpy(), samplerate=16000)
102
+ ```
103
+
104
+ > Keep the speaker embedding at its original scale — it is a raw x-vector with
105
+ > norm ≈ 320. Do **not** unit-normalise it, or the voice degrades.
106
+
107
+ ## Self-hosting
108
+
109
+ The engine is served from a FastAPI app (`server/app.py`) that loads Kokoro-82M
110
+ locally and exposes the endpoints above on `127.0.0.1:8000`, then published with a
111
+ Cloudflare tunnel. It starts automatically at logon and runs without a GPU.
112
+
113
+ ## Intended use & limitations
114
+
115
+ - **Intended use:** research, prototyping, accessibility and demos.
116
+ - **Limitations:** 8 languages; may mispronounce rare names, numbers and unusual
117
+ punctuation; Japanese is not currently enabled (G2P dependency).
118
+ - Output is not a human voice and must not be used for impersonation or deceptive
119
+ audio.
120
+
121
+ ## Credits
122
+
123
+ - Engine: [Kokoro-82M](https://huggingface.co/hexgrad/Kokoro-82M) by hexgrad.
124
+ - SpeechT5 v1 voice data: [LibriSpeech](https://www.openslr.org/12), CC BY 4.0.
125
+ - Base model & vocoder: Microsoft Research Asia, via 🤗 Transformers.
126
+ - Fine-tuned, hosted and packaged by the **Sol** project.
127
+
128
+ ## License
129
+
130
+ Apache-2.0. Please respect the original dataset, base-model and engine licenses.