|
Download README.md from useless-ai/pitput: direct link, hf CLI and curl.
- Browser
- Download file 10.9 kB
-
https://huggingface.co/useless-ai/pitput/resolve/main/README.md
- Command line
-
hf download hf://useless-ai/pitput/README.md
-
curl -L -o README.md https://huggingface.co/useless-ai/pitput/resolve/main/README.md
10.9 kB
| language: | |
| - he | |
| license: other | |
| license_name: private-research-only | |
| license_link: https://huggingface.co/useless-ai/pitput/blob/main/README.md#license | |
| pipeline_tag: text-to-speech | |
| library_name: pytorch | |
| tags: | |
| - text-to-speech | |
| - tts | |
| - hebrew | |
| - fastspeech2 | |
| - ddsp | |
| # pitput | |
| **pitput** (ืคึดึผืึฐืคึผืึผื) is a Hebrew take on [tiny-voca](https://github.com/n4ze3m/tiny-voca): a tiny TTS model | |
| built to run on a CPU. The whole model, vocoder included, is **only 18 MB** (4.5M parameters). It outputs 24 kHz | |
| mono and runs faster than real time on a laptop CPU. It reads plain Hebrew without niqqud, in one voice with | |
| **8 delivery styles** and **8 expression tags** (`<laugh>`, `<sigh>`, `<short pause>` โฆ). | |
| It is a personal learning project, trained from scratch on synthetic speech. This is **try 1** (step 104,000, | |
| November 18, 2026). **Private research use only; see [License](#license).** | |
| ## Samples | |
| The 10 listening texts that were synthesized after every epoch, here from the final weights. | |
| | Style | Text | Audio | | |
| |---|---|---| | |
| | default | ืืคื ื ืชืื ื ืืืฉืื ืืืจืืืืช ืืกืืืืกืืืงื, ืืืืืืกืืืช ืืฉืจืื ืืืื ืืฉื ื ืืืืจืื ื ืืืืื ืืฉืืื ืื ืืืฃ ืืืฉ. | <audio controls src="https://huggingface.co/useless-ai/pitput/resolve/main/samples/01-default.wav"></audio> | | |
| | default, addressee f | ืื ืฉืืืื? ืฉืืขืชื ืฉืื ืืจืืฉืช ืืื ืืฉืืืข ืฉืขืืจ. | <audio controls src="https://huggingface.co/useless-ai/pitput/resolve/main/samples/02-default.wav"></audio> | | |
| | default | ืืจืืืช ืืืืคื ืืืฆืืช ืืฉืืข ืืขืฉืจืื ืืืืืขื ืืฉืืื ื ืืืฆื, `<chuckle>` ืื ืืื ืืืื ืชืืืข. | <audio controls src="https://huggingface.co/useless-ai/pitput/resolve/main/samples/03-default.wav"></audio> | | |
| | excited | `<laugh>` ืื ื ืื ืืืืื ื, ืงืืืื ื ืืช ืืืืจื! | <audio controls src="https://huggingface.co/useless-ai/pitput/resolve/main/samples/04-excited.wav"></audio> | | |
| | sad | `<sigh>` ืืืืืชื ืื ืื ืืขืจื, ืืืื ืคืฉืื ืื ืืืืข. | <audio controls src="https://huggingface.co/useless-ai/pitput/resolve/main/samples/05-sad.wav"></audio> | | |
| | angry, addressee m | ืืื ืคืขืืื ืืืจืชื ืื ืืกืืืจ ืืช ืืืืช?! | <audio controls src="https://huggingface.co/useless-ai/pitput/resolve/main/samples/06-angry.wav"></audio> | | |
| | fearful | ืฉืืขืช ืืช ืื? `<gasp>` ืืฉ ืืืฉืื ืืืืฅ. | <audio controls src="https://huggingface.co/useless-ai/pitput/resolve/main/samples/07-fearful.wav"></audio> | | |
| | surprised | ืจืืข, `<short pause>` ืืื ืืืืช ืืชืคืืจ? ืืืจื ืขืฉืจืื ืฉื ื? | <audio controls src="https://huggingface.co/useless-ai/pitput/resolve/main/samples/08-surprised.wav"></audio> | | |
| | calm | `<breath>` ืงืื ื ืฉืืื ืขืืืงื, `<long pause>` ืืชื ื ืืืืฃ ืืืืจืืข ืืื ืืื. | <audio controls src="https://huggingface.co/useless-ai/pitput/resolve/main/samples/09-calm.wav"></audio> | | |
| | whisper | ืืื ื ืืืจ ืืฉืงื, ืืืื ืืืจ ืืฉื ืื. | <audio controls src="https://huggingface.co/useless-ai/pitput/resolve/main/samples/10-whisper.wav"></audio> | | |
| ## Usage | |
| ```bash | |
| huggingface-cli login # the repository is private | |
| git clone https://huggingface.co/useless-ai/pitput | |
| cd pitput | |
| pip install -r requirements.txt # PyTorch, Phonikud, onnxruntime, soundfile, huggingface_hub | |
| python -m pitput.infer --style excited --text "<laugh> ืื ื ืื ืืืืื ื, ืงืืืื ื ืืช ืืืืจื!" --output out.wav | |
| python -m pitput.infer --style default --addressee f --text "ืื ืฉืืืื? ืฉืืขืชื ืฉืื ืืจืืฉืช ืืื." --output hello.wav | |
| ``` | |
| On first use the Phonikud model (~300 MB) is downloaded to the Hugging Face cache. Options: | |
| | Option | Meaning | | |
| |---|---| | |
| | `--style` | `default`, `excited`, `sad`, `angry`, `fearful`, `surprised`, `calm`, `whisper` | | |
| | `--addressee m` / `f` | Gender of the person spoken to (ืื, ืืืชื, ืืืจืช โฆ). Without it Phonikud reads the masculine form | | |
| | `--text-file` | Read the text from a UTF-8 file instead of `--text` | | |
| | `--speed` | 1.0 = as trained; 0.9 is 10% slower | | |
| | `--pitch-shift` | Added to log F0 (0.058 โ one semitone) | | |
| | `--pitch-range` | Scales the melody around each sentence's mean pitch (1.0 = as predicted) | | |
| | `--device` | `cpu` (default) or `cuda` | | |
| From Python: | |
| ```python | |
| import soundfile as sf | |
| from pitput.infer import CHECKPOINT, load_model, speak_text | |
| model = load_model(CHECKPOINT) # inference.pt next to the package | |
| wave, utterances = speak_text(model, "ืฉืืื, <sigh> ืื ืฉืืืื?", "calm", addressee="f") | |
| sf.write("out.wav", wave.numpy(), model.config.sample_rate) | |
| ``` | |
| ### What the input must look like | |
| - **Plain Hebrew in spoken form.** There is no text normalizer: write numbers, dates, times, acronyms and | |
| foreign words out in Hebrew words (`ืืฉืืข ืืขืฉืจืื`, not `7:20`). Characters it can't read are skipped. | |
| - **No niqqud needed.** [Phonikud](https://github.com/thewh1teagle/phonikud) vocalizes each sentence in context | |
| and adds stress; the model reads its phonemes. | |
| - **Tags** go where the sound should be, in English inside the Hebrew: `<laugh>`, `<chuckle>`, `<sigh>`, | |
| `<breath>`, `<gasp>`, `<throat-clearing>`, `<short pause>`, `<long pause>`. | |
| - **Punctuation** shapes the delivery (`, . ? ! โฆ`). Long texts are read sentence by sentence. | |
| ## Model | |
| A fork of [fiato](https://huggingface.co/cfahlgren1/fiato), itself a fork of | |
| [tiny-voca](https://github.com/n4ze3m/tiny-voca): a FastSpeech2-style model with a built-in | |
| differentiable source-filter vocoder, so there is no separate neural vocoder. | |
| - **Input:** 47 tokens: 29 Phonikud phonemes, 5 punctuation marks, word boundary, utterance edges and the 8 | |
| expression tags. Stress is a parallel feature, not a token. | |
| - **Text encoder:** 4 transformer layers, 192 channels, with the style embedding added to every token. | |
| - **Duration predictor:** frames per token, plus a "pause present" switch for punctuation, word boundaries and tags. | |
| - **Frame level:** the style embedding is added again; 2 layers predict pitch (log F0), voicing and energy, and 5 | |
| layers predict the acoustic features. | |
| - **Vocoder:** harmonic pulse train (from the predicted pitch) plus noise, each shaped by a complex cepstrum per frame. | |
| - **Size:** 4,534,027 parameters at inference (18 MB checkpoint). Training also uses 2.7M parameters that are not | |
| in the checkpoint (discriminators and an auxiliary mel head). | |
| ## Training data | |
| The dataset is **synthetic only**, with no recorded human speech: 65,722 clips (139.3 h) of one synthetic voice, | |
| of which 63,745 training clips (136.4 h) passed forced alignment. 160 held-out sentences (20 per style) were kept | |
| for validation and 160 for test. | |
| - **Text:** 65,728 Hebrew sentences from news (archive, HeSum, RSS), HeDC4 web text and stories, Hebrew Wikipedia, | |
| ivrit.ai transcripts and LLM-written scenes (dialogue for the expressive styles). | |
| - **Spoken form:** an LLM rewrote numbers, dates, money, acronyms and Latin script as Hebrew words, and every answer | |
| was checked by code. | |
| - **Addressee gender:** an LLM decided the gender of ambiguous second-person words (ืื, ืืืชื, ืืืจืช) in context. | |
| The audio and the training phonemes follow the same decision. | |
| - **Styles:** an LLM picked a style for each dialogue sentence and inserted the tags; calm and whisper were assigned | |
| to plain statements. Each style was rendered with one fixed delivery instruction; default is a bright, friendly | |
| delivery. | |
| | Style | Clips synthesized | Rejected at alignment | | |
| |---|---|---| | |
| | default | 52,193 | 1% | | |
| | calm | 4,900 | 1% | | |
| | whisper | 4,900 | 1% | | |
| | fearful | 1,104 | 42% | | |
| | angry | 1,027 | 12% | | |
| | sad | 685 | 19% | | |
| | excited | 470 | 18% | | |
| | surprised | 449 | 14% | | |
| 4,582 clips contain tags (at most 2 per clip); `<laugh>` appears in only about 90 of them. | |
| - **Labels:** Phonikud phonemes with the addressee decisions applied, phonemized word by word with the tags at | |
| their positions. Durations come from forced alignment with | |
| [facebook/wav2vec2-lv-60-espeak-cv-ft](https://huggingface.co/facebook/wav2vec2-lv-60-espeak-cv-ft), which is used | |
| only for labeling and is not part of the model. A tag owns the sound between the silences around it. | |
| ## Limitations | |
| - **Laughs and other non-speech tags are weak.** The model gives each tag its time, but fills it with a quiet, | |
| mostly unvoiced sound: a laugh comes out about 15 dB quieter than in the recordings and is heard as a soft breath. | |
| Chuckle, throat-clearing and gasp suffer the same way. Pauses and breaths work better. | |
| - **Intonation can sound a little robotic.** Pitch range is close to the recordings for most styles, but fearful | |
| and surprised have only 70โ75% of it. The model speaks 3โ17% faster than the recordings, with shorter pauses. | |
| - **Fearful is the weakest style:** it has the fewest usable training clips and the most pronunciation errors. | |
| - **Pronunciation follows Phonikud,** including its mistakes (e.g. ืืืื read as "vaสu"). | |
| - **Second person defaults to masculine** unless `--addressee f` is given; there is no per-word decision at | |
| inference. | |
| - One voice only. Sentences are synthesized independently, with no prosody carried between them. | |
| ## License | |
| - **Code** (`pitput/`): Apache License 2.0. It is a fork of fiato by cfahlgren1 and tiny-voca by Muhammed Nazeem, | |
| both Apache-2.0; see `NOTICE`, `LICENSE` and `LICENSE.tiny-voca`. | |
| - **Weights and audio samples: private research use only, not licensed for redistribution or any commercial use.** | |
| Keep this repository private. | |
| - **Phonikud** ([thewh1teagle/phonikud-onnx](https://huggingface.co/thewh1teagle/phonikud-onnx), MIT) is downloaded | |
| at first use and is not included here. | |
| ## Files | |
| | File | Contents | | |
| |---|---| | |
| | `inference.pt` | Weights, model configuration and front-end fingerprint (loaded with `weights_only=True`) | | |
| | `pitput/` | Inference code: `infer.py` (command line and `speak_text`), `model.py`, `text.py` (phoneme tokens), `g2p.py` (Phonikud and addressee gender), `labels.py`, `config.py` | | |
| | `samples/` | The audio above | | |
| | `config.json`, `recipe.json` | Model and training settings of this run | | |
| | `requirements.txt` | Python dependencies | | |
| | `NOTICE`, `LICENSE`, `LICENSE.tiny-voca` | Attribution and license of the code | | |
| ## Acknowledgements | |
| [tiny-voca](https://github.com/n4ze3m/tiny-voca) for the architecture and training recipe, | |
| [fiato](https://huggingface.co/cfahlgren1/fiato) for expression tags on it, [Phonikud](https://github.com/thewh1teagle/phonikud) | |
| for Hebrew vocalization and phonemes, and the Hebrew text sources: HeDC4, HeSum, Hebrew Wikipedia and ivrit.ai. | |