--- language: - he license: other license_name: private-research-only license_link: https://huggingface.co/useless-ai/pitput/blob/main/README.md#license pipeline_tag: text-to-speech library_name: pytorch tags: - text-to-speech - tts - hebrew - fastspeech2 - ddsp --- # pitput **pitput** (פִּטְפּוּט) is a Hebrew take on [tiny-voca](https://github.com/n4ze3m/tiny-voca): a tiny TTS model built to run on a CPU. The whole model, vocoder included, is **only 18 MB** (4.5M parameters). It outputs 24 kHz mono and runs faster than real time on a laptop CPU. It reads plain Hebrew without niqqud, in one voice with **8 delivery styles** and **8 expression tags** (``, ``, `` …). It is a personal learning project, trained from scratch on synthetic speech. This is **try 1** (step 104,000, November 18, 2026). **Private research use only; see [License](#license).** ## Samples The 10 listening texts that were synthesized after every epoch, here from the final weights. | Style | Text | Audio | |---|---|---| | default | לפי נתוני הלשכה המרכזית לסטטיסטיקה, אוכלוסיית ישראל גדלה בשנה האחרונה בכמאה ושמונים אלף איש. | | | default, addressee f | מה שלומך? שמעתי שלא הרגשת טוב בשבוע שעבר. | | | default | הרכבת לחיפה יוצאת בשבע ועשרים ומגיעה בשמונה וחצי, `` אם היא בכלל תגיע. | | | excited | `` אני לא מאמינה, קיבלנו את הדירה! | | | sad | `` חיכיתי לו כל הערב, והוא פשוט לא הגיע. | | | angry, addressee m | כמה פעמים אמרתי לך לסגור את הדלת?! | | | fearful | שמעת את זה? `` יש מישהו בחוץ. | | | surprised | רגע, `` הוא באמת התפטר? אחרי עשרים שנה? | | | calm | `` קחו נשימה עמוקה, `` ותנו לגוף להירגע לאט לאט. | | | whisper | בוא נדבר בשקט, כולם כבר ישנים. | | ## Usage ```bash huggingface-cli login # the repository is private git clone https://huggingface.co/useless-ai/pitput cd pitput pip install -r requirements.txt # PyTorch, Phonikud, onnxruntime, soundfile, huggingface_hub python -m pitput.infer --style excited --text " אני לא מאמינה, קיבלנו את הדירה!" --output out.wav python -m pitput.infer --style default --addressee f --text "מה שלומך? שמעתי שלא הרגשת טוב." --output hello.wav ``` On first use the Phonikud model (~300 MB) is downloaded to the Hugging Face cache. Options: | Option | Meaning | |---|---| | `--style` | `default`, `excited`, `sad`, `angry`, `fearful`, `surprised`, `calm`, `whisper` | | `--addressee m` / `f` | Gender of the person spoken to (לך, אותך, אמרת …). Without it Phonikud reads the masculine form | | `--text-file` | Read the text from a UTF-8 file instead of `--text` | | `--speed` | 1.0 = as trained; 0.9 is 10% slower | | `--pitch-shift` | Added to log F0 (0.058 ≈ one semitone) | | `--pitch-range` | Scales the melody around each sentence's mean pitch (1.0 = as predicted) | | `--device` | `cpu` (default) or `cuda` | From Python: ```python import soundfile as sf from pitput.infer import CHECKPOINT, load_model, speak_text model = load_model(CHECKPOINT) # inference.pt next to the package wave, utterances = speak_text(model, "שלום, מה שלומך?", "calm", addressee="f") sf.write("out.wav", wave.numpy(), model.config.sample_rate) ``` ### What the input must look like - **Plain Hebrew in spoken form.** There is no text normalizer: write numbers, dates, times, acronyms and foreign words out in Hebrew words (`בשבע ועשרים`, not `7:20`). Characters it can't read are skipped. - **No niqqud needed.** [Phonikud](https://github.com/thewh1teagle/phonikud) vocalizes each sentence in context and adds stress; the model reads its phonemes. - **Tags** go where the sound should be, in English inside the Hebrew: ``, ``, ``, ``, ``, ``, ``, ``. - **Punctuation** shapes the delivery (`, . ? ! …`). Long texts are read sentence by sentence. ## Model A fork of [fiato](https://huggingface.co/cfahlgren1/fiato), itself a fork of [tiny-voca](https://github.com/n4ze3m/tiny-voca): a FastSpeech2-style model with a built-in differentiable source-filter vocoder, so there is no separate neural vocoder. - **Input:** 47 tokens: 29 Phonikud phonemes, 5 punctuation marks, word boundary, utterance edges and the 8 expression tags. Stress is a parallel feature, not a token. - **Text encoder:** 4 transformer layers, 192 channels, with the style embedding added to every token. - **Duration predictor:** frames per token, plus a "pause present" switch for punctuation, word boundaries and tags. - **Frame level:** the style embedding is added again; 2 layers predict pitch (log F0), voicing and energy, and 5 layers predict the acoustic features. - **Vocoder:** harmonic pulse train (from the predicted pitch) plus noise, each shaped by a complex cepstrum per frame. - **Size:** 4,534,027 parameters at inference (18 MB checkpoint). Training also uses 2.7M parameters that are not in the checkpoint (discriminators and an auxiliary mel head). ## Training data The dataset is **synthetic only**, with no recorded human speech: 65,722 clips (139.3 h) of one synthetic voice, of which 63,745 training clips (136.4 h) passed forced alignment. 160 held-out sentences (20 per style) were kept for validation and 160 for test. - **Text:** 65,728 Hebrew sentences from news (archive, HeSum, RSS), HeDC4 web text and stories, Hebrew Wikipedia, ivrit.ai transcripts and LLM-written scenes (dialogue for the expressive styles). - **Spoken form:** an LLM rewrote numbers, dates, money, acronyms and Latin script as Hebrew words, and every answer was checked by code. - **Addressee gender:** an LLM decided the gender of ambiguous second-person words (לך, אותך, אמרת) in context. The audio and the training phonemes follow the same decision. - **Styles:** an LLM picked a style for each dialogue sentence and inserted the tags; calm and whisper were assigned to plain statements. Each style was rendered with one fixed delivery instruction; default is a bright, friendly delivery. | Style | Clips synthesized | Rejected at alignment | |---|---|---| | default | 52,193 | 1% | | calm | 4,900 | 1% | | whisper | 4,900 | 1% | | fearful | 1,104 | 42% | | angry | 1,027 | 12% | | sad | 685 | 19% | | excited | 470 | 18% | | surprised | 449 | 14% | 4,582 clips contain tags (at most 2 per clip); `` appears in only about 90 of them. - **Labels:** Phonikud phonemes with the addressee decisions applied, phonemized word by word with the tags at their positions. Durations come from forced alignment with [facebook/wav2vec2-lv-60-espeak-cv-ft](https://huggingface.co/facebook/wav2vec2-lv-60-espeak-cv-ft), which is used only for labeling and is not part of the model. A tag owns the sound between the silences around it. ## Limitations - **Laughs and other non-speech tags are weak.** The model gives each tag its time, but fills it with a quiet, mostly unvoiced sound: a laugh comes out about 15 dB quieter than in the recordings and is heard as a soft breath. Chuckle, throat-clearing and gasp suffer the same way. Pauses and breaths work better. - **Intonation can sound a little robotic.** Pitch range is close to the recordings for most styles, but fearful and surprised have only 70–75% of it. The model speaks 3–17% faster than the recordings, with shorter pauses. - **Fearful is the weakest style:** it has the fewest usable training clips and the most pronunciation errors. - **Pronunciation follows Phonikud,** including its mistakes (e.g. וואו read as "vaʔu"). - **Second person defaults to masculine** unless `--addressee f` is given; there is no per-word decision at inference. - One voice only. Sentences are synthesized independently, with no prosody carried between them. ## License - **Code** (`pitput/`): Apache License 2.0. It is a fork of fiato by cfahlgren1 and tiny-voca by Muhammed Nazeem, both Apache-2.0; see `NOTICE`, `LICENSE` and `LICENSE.tiny-voca`. - **Weights and audio samples: private research use only, not licensed for redistribution or any commercial use.** Keep this repository private. - **Phonikud** ([thewh1teagle/phonikud-onnx](https://huggingface.co/thewh1teagle/phonikud-onnx), MIT) is downloaded at first use and is not included here. ## Files | File | Contents | |---|---| | `inference.pt` | Weights, model configuration and front-end fingerprint (loaded with `weights_only=True`) | | `pitput/` | Inference code: `infer.py` (command line and `speak_text`), `model.py`, `text.py` (phoneme tokens), `g2p.py` (Phonikud and addressee gender), `labels.py`, `config.py` | | `samples/` | The audio above | | `config.json`, `recipe.json` | Model and training settings of this run | | `requirements.txt` | Python dependencies | | `NOTICE`, `LICENSE`, `LICENSE.tiny-voca` | Attribution and license of the code | ## Acknowledgements [tiny-voca](https://github.com/n4ze3m/tiny-voca) for the architecture and training recipe, [fiato](https://huggingface.co/cfahlgren1/fiato) for expression tags on it, [Phonikud](https://github.com/thewh1teagle/phonikud) for Hebrew vocalization and phonemes, and the Hebrew text sources: HeDC4, HeSum, Hebrew Wikipedia and ivrit.ai.