File size: 10,901 Bytes
b9fa092
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
---

language:
- he
license: other
license_name: private-research-only
license_link: https://huggingface.co/useless-ai/pitput/blob/main/README.md#license
pipeline_tag: text-to-speech
library_name: pytorch
tags:
- text-to-speech
- tts
- hebrew
- fastspeech2
- ddsp
---


# pitput

**pitput** (ืคึดึผื˜ึฐืคึผื•ึผื˜) is a Hebrew take on [tiny-voca](https://github.com/n4ze3m/tiny-voca): a tiny TTS model
built to run on a CPU. The whole model, vocoder included, is **only 18 MB** (4.5M parameters). It outputs 24 kHz
mono and runs faster than real time on a laptop CPU. It reads plain Hebrew without niqqud, in one voice with
**8 delivery styles** and **8 expression tags** (`<laugh>`, `<sigh>`, `<short pause>` โ€ฆ).

It is a personal learning project, trained from scratch on synthetic speech. This is **try 1** (step 104,000,
November 18, 2026). **Private research use only; see [License](#license).**

## Samples

The 10 listening texts that were synthesized after every epoch, here from the final weights.

| Style | Text | Audio |
|---|---|---|
| default | ืœืคื™ ื ืชื•ื ื™ ื”ืœืฉื›ื” ื”ืžืจื›ื–ื™ืช ืœืกื˜ื˜ื™ืกื˜ื™ืงื”, ืื•ื›ืœื•ืกื™ื™ืช ื™ืฉืจืืœ ื’ื“ืœื” ื‘ืฉื ื” ื”ืื—ืจื•ื ื” ื‘ื›ืžืื” ื•ืฉืžื•ื ื™ื ืืœืฃ ืื™ืฉ. | <audio controls src="https://huggingface.co/useless-ai/pitput/resolve/main/samples/01-default.wav"></audio> |
| default, addressee f | ืžื” ืฉืœื•ืžืš? ืฉืžืขืชื™ ืฉืœื ื”ืจื’ืฉืช ื˜ื•ื‘ ื‘ืฉื‘ื•ืข ืฉืขื‘ืจ. | <audio controls src="https://huggingface.co/useless-ai/pitput/resolve/main/samples/02-default.wav"></audio> |
| default | ื”ืจื›ื‘ืช ืœื—ื™ืคื” ื™ื•ืฆืืช ื‘ืฉื‘ืข ื•ืขืฉืจื™ื ื•ืžื’ื™ืขื” ื‘ืฉืžื•ื ื” ื•ื—ืฆื™, `<chuckle>` ืื ื”ื™ื ื‘ื›ืœืœ ืชื’ื™ืข. | <audio controls src="https://huggingface.co/useless-ai/pitput/resolve/main/samples/03-default.wav"></audio> |
| excited | `<laugh>` ืื ื™ ืœื ืžืืžื™ื ื”, ืงื™ื‘ืœื ื• ืืช ื”ื“ื™ืจื”! | <audio controls src="https://huggingface.co/useless-ai/pitput/resolve/main/samples/04-excited.wav"></audio> |
| sad | `<sigh>` ื—ื™ื›ื™ืชื™ ืœื• ื›ืœ ื”ืขืจื‘, ื•ื”ื•ื ืคืฉื•ื˜ ืœื ื”ื’ื™ืข. | <audio controls src="https://huggingface.co/useless-ai/pitput/resolve/main/samples/05-sad.wav"></audio> |
| angry, addressee m | ื›ืžื” ืคืขืžื™ื ืืžืจืชื™ ืœืš ืœืกื’ื•ืจ ืืช ื”ื“ืœืช?! | <audio controls src="https://huggingface.co/useless-ai/pitput/resolve/main/samples/06-angry.wav"></audio> |
| fearful | ืฉืžืขืช ืืช ื–ื”? `<gasp>` ื™ืฉ ืžื™ืฉื”ื• ื‘ื—ื•ืฅ. | <audio controls src="https://huggingface.co/useless-ai/pitput/resolve/main/samples/07-fearful.wav"></audio> |
| surprised | ืจื’ืข, `<short pause>` ื”ื•ื ื‘ืืžืช ื”ืชืคื˜ืจ? ืื—ืจื™ ืขืฉืจื™ื ืฉื ื”? | <audio controls src="https://huggingface.co/useless-ai/pitput/resolve/main/samples/08-surprised.wav"></audio> |
| calm | `<breath>` ืงื—ื• ื ืฉื™ืžื” ืขืžื•ืงื”, `<long pause>` ื•ืชื ื• ืœื’ื•ืฃ ืœื”ื™ืจื’ืข ืœืื˜ ืœืื˜. | <audio controls src="https://huggingface.co/useless-ai/pitput/resolve/main/samples/09-calm.wav"></audio> |
| whisper | ื‘ื•ื ื ื“ื‘ืจ ื‘ืฉืงื˜, ื›ื•ืœื ื›ื‘ืจ ื™ืฉื ื™ื. | <audio controls src="https://huggingface.co/useless-ai/pitput/resolve/main/samples/10-whisper.wav"></audio> |

## Usage

```bash

huggingface-cli login                  # the repository is private

git clone https://huggingface.co/useless-ai/pitput

cd pitput

pip install -r requirements.txt        # PyTorch, Phonikud, onnxruntime, soundfile, huggingface_hub



python -m pitput.infer --style excited --text "<laugh> ืื ื™ ืœื ืžืืžื™ื ื”, ืงื™ื‘ืœื ื• ืืช ื”ื“ื™ืจื”!" --output out.wav

python -m pitput.infer --style default --addressee f --text "ืžื” ืฉืœื•ืžืš? ืฉืžืขืชื™ ืฉืœื ื”ืจื’ืฉืช ื˜ื•ื‘." --output hello.wav

```

On first use the Phonikud model (~300 MB) is downloaded to the Hugging Face cache. Options:

| Option | Meaning |
|---|---|
| `--style` | `default`, `excited`, `sad`, `angry`, `fearful`, `surprised`, `calm`, `whisper` |
| `--addressee m` / `f` | Gender of the person spoken to (ืœืš, ืื•ืชืš, ืืžืจืช โ€ฆ). Without it Phonikud reads the masculine form |
| `--text-file` | Read the text from a UTF-8 file instead of `--text` |
| `--speed` | 1.0 = as trained; 0.9 is 10% slower |
| `--pitch-shift` | Added to log F0 (0.058 โ‰ˆ one semitone) |
| `--pitch-range` | Scales the melody around each sentence's mean pitch (1.0 = as predicted) |
| `--device` | `cpu` (default) or `cuda` |

From Python:

```python

import soundfile as sf

from pitput.infer import CHECKPOINT, load_model, speak_text



model = load_model(CHECKPOINT)  # inference.pt next to the package

wave, utterances = speak_text(model, "ืฉืœื•ื, <sigh> ืžื” ืฉืœื•ืžืš?", "calm", addressee="f")

sf.write("out.wav", wave.numpy(), model.config.sample_rate)

```

### What the input must look like

- **Plain Hebrew in spoken form.** There is no text normalizer: write numbers, dates, times, acronyms and
  foreign words out in Hebrew words (`ื‘ืฉื‘ืข ื•ืขืฉืจื™ื`, not `7:20`). Characters it can't read are skipped.
- **No niqqud needed.** [Phonikud](https://github.com/thewh1teagle/phonikud) vocalizes each sentence in context
  and adds stress; the model reads its phonemes.
- **Tags** go where the sound should be, in English inside the Hebrew: `<laugh>`, `<chuckle>`, `<sigh>`,
  `<breath>`, `<gasp>`, `<throat-clearing>`, `<short pause>`, `<long pause>`.
- **Punctuation** shapes the delivery (`, . ? ! โ€ฆ`). Long texts are read sentence by sentence.

## Model

A fork of [fiato](https://huggingface.co/cfahlgren1/fiato), itself a fork of
[tiny-voca](https://github.com/n4ze3m/tiny-voca): a FastSpeech2-style model with a built-in
differentiable source-filter vocoder, so there is no separate neural vocoder.

- **Input:** 47 tokens: 29 Phonikud phonemes, 5 punctuation marks, word boundary, utterance edges and the 8
  expression tags. Stress is a parallel feature, not a token.
- **Text encoder:** 4 transformer layers, 192 channels, with the style embedding added to every token.
- **Duration predictor:** frames per token, plus a "pause present" switch for punctuation, word boundaries and tags.
- **Frame level:** the style embedding is added again; 2 layers predict pitch (log F0), voicing and energy, and 5
  layers predict the acoustic features.
- **Vocoder:** harmonic pulse train (from the predicted pitch) plus noise, each shaped by a complex cepstrum per frame.
- **Size:** 4,534,027 parameters at inference (18 MB checkpoint). Training also uses 2.7M parameters that are not
  in the checkpoint (discriminators and an auxiliary mel head).

## Training data

The dataset is **synthetic only**, with no recorded human speech: 65,722 clips (139.3 h) of one synthetic voice,
of which 63,745 training clips (136.4 h) passed forced alignment. 160 held-out sentences (20 per style) were kept
for validation and 160 for test.

- **Text:** 65,728 Hebrew sentences from news (archive, HeSum, RSS), HeDC4 web text and stories, Hebrew Wikipedia,
  ivrit.ai transcripts and LLM-written scenes (dialogue for the expressive styles).
- **Spoken form:** an LLM rewrote numbers, dates, money, acronyms and Latin script as Hebrew words, and every answer
  was checked by code.
- **Addressee gender:** an LLM decided the gender of ambiguous second-person words (ืœืš, ืื•ืชืš, ืืžืจืช) in context.
  The audio and the training phonemes follow the same decision.
- **Styles:** an LLM picked a style for each dialogue sentence and inserted the tags; calm and whisper were assigned
  to plain statements. Each style was rendered with one fixed delivery instruction; default is a bright, friendly
  delivery.

  | Style | Clips synthesized | Rejected at alignment |
  |---|---|---|
  | default | 52,193 | 1% |
  | calm | 4,900 | 1% |
  | whisper | 4,900 | 1% |
  | fearful | 1,104 | 42% |
  | angry | 1,027 | 12% |
  | sad | 685 | 19% |
  | excited | 470 | 18% |
  | surprised | 449 | 14% |

  4,582 clips contain tags (at most 2 per clip); `<laugh>` appears in only about 90 of them.
- **Labels:** Phonikud phonemes with the addressee decisions applied, phonemized word by word with the tags at
  their positions. Durations come from forced alignment with
  [facebook/wav2vec2-lv-60-espeak-cv-ft](https://huggingface.co/facebook/wav2vec2-lv-60-espeak-cv-ft), which is used
  only for labeling and is not part of the model. A tag owns the sound between the silences around it.

## Limitations

- **Laughs and other non-speech tags are weak.** The model gives each tag its time, but fills it with a quiet,
  mostly unvoiced sound: a laugh comes out about 15 dB quieter than in the recordings and is heard as a soft breath.
  Chuckle, throat-clearing and gasp suffer the same way. Pauses and breaths work better.
- **Intonation can sound a little robotic.** Pitch range is close to the recordings for most styles, but fearful
  and surprised have only 70โ€“75% of it. The model speaks 3โ€“17% faster than the recordings, with shorter pauses.
- **Fearful is the weakest style:** it has the fewest usable training clips and the most pronunciation errors.
- **Pronunciation follows Phonikud,** including its mistakes (e.g. ื•ื•ืื• read as "vaส”u").
- **Second person defaults to masculine** unless `--addressee f` is given; there is no per-word decision at
  inference.
- One voice only. Sentences are synthesized independently, with no prosody carried between them.

## License

- **Code** (`pitput/`): Apache License 2.0. It is a fork of fiato by cfahlgren1 and tiny-voca by Muhammed Nazeem,
  both Apache-2.0; see `NOTICE`, `LICENSE` and `LICENSE.tiny-voca`.
- **Weights and audio samples: private research use only, not licensed for redistribution or any commercial use.**
  Keep this repository private.
- **Phonikud** ([thewh1teagle/phonikud-onnx](https://huggingface.co/thewh1teagle/phonikud-onnx), MIT) is downloaded
  at first use and is not included here.

## Files

| File | Contents |
|---|---|
| `inference.pt` | Weights, model configuration and front-end fingerprint (loaded with `weights_only=True`) |
| `pitput/` | Inference code: `infer.py` (command line and `speak_text`), `model.py`, `text.py` (phoneme tokens), `g2p.py` (Phonikud and addressee gender), `labels.py`, `config.py` |
| `samples/` | The audio above |
| `config.json`, `recipe.json` | Model and training settings of this run |
| `requirements.txt` | Python dependencies |
| `NOTICE`, `LICENSE`, `LICENSE.tiny-voca` | Attribution and license of the code |

## Acknowledgements

[tiny-voca](https://github.com/n4ze3m/tiny-voca) for the architecture and training recipe,
[fiato](https://huggingface.co/cfahlgren1/fiato) for expression tags on it, [Phonikud](https://github.com/thewh1teagle/phonikud)
for Hebrew vocalization and phonemes, and the Hebrew text sources: HeDC4, HeSum, Hebrew Wikipedia and ivrit.ai.