dignome
/

kitten_tts2

@@ -28,75 +28,79 @@ tags:
 **Powered by Stellon Labs.**
-Native audio.cpp GGUF conversion of [KittenML/kitten-tts-2](https://huggingface.co/KittenML/kitten-tts-2), with the full S3 meanflow decoder, speaker encoders, and prepared voice presets included in one file.
-Use [dignome/audio.cpp-custom, branch `kittentts2`](https://github.com/dignome/audio.cpp-custom/tree/kittentts2), or an audio.cpp build that includes the same port. The published implementation is commit [`a96e66c6`](https://github.com/dignome/audio.cpp-custom/commit/a96e66c6bc4d53398563cfff8399d1f9a6e35b40). The GGUF uses audio.cpp's model packaging and is intended for this loader; generic GGUF support in another application does not imply compatibility.
-## Download
-- [`kitten-tts2-native-q8-multilingual.gguf`](./kitten-tts2-native-q8-multilingual.gguf): **3,282,123,776 bytes** (3.28 GB / 3.06 GiB).
-- [`SHA256SUMS`](./SHA256SUMS): checksum for verifying the download.
-The GGUF includes all model components, tokenizer, configuration, model specification, 48 prepared voice entries, and license/attribution files. Separate reference WAVs, voice JSON files, or model downloads are not needed for preset synthesis or custom voice cloning.
-The language-model projections use Q8_0, with F16 embeddings/tied output head. The speaker projection retains its original BF16 storage, and the S3 decoder and speaker encoders retain F32 weights. This is a mixed-precision package, not an all-Q8 conversion.
-## Features
-- Native C++/GGML inference on CPU and NVIDIA CUDA; no Python, LibTorch, TorchScript, or ONNX Runtime dependency at inference time.
-- All **47 upstream presets**, plus the additional `PreparedBruno` entry.
-- Named English voices such as `Bruno`, `Bella`, `Jasper`, and `Luna`.
-- Language presets: `Arabic`, `Chinese`, `French`, `German`, `Hindi`, `Italian`, `Portuguese`, `Russian`, and `Spanish`.
-- Custom voice cloning from a **1–30 second reference recording and its transcript**.
-- Chunked long-form synthesis and mono **24 kHz** output.
-Ten languages have been validated: English and the nine languages listed above. Upstream advertises additional languages, but those are not validated by this port. Select a matching language preset or supply a reference recording and transcript in the target language; there is no separate language switch.
-## Build and run
-See the [port documentation](https://github.com/dignome/audio.cpp-custom/blob/kittentts2/docs/community_models/kitten_tts2.md) for full instructions.
 ```sh
-git clone --branch kittentts2 https://github.com/dignome/audio.cpp-custom.git
-cd audio.cpp-custom
-cmake -S . -B build-kitten2 -DAUDIOCPP_MODEL_SET=custom -DAUDIOCPP_MODELS=kitten_tts2 -DENGINE_ENABLE_CUDA=OFF
-cmake --build build-kitten2 --config Release --target audiocpp_cli audiocpp_server --parallel 8
 ```
-For NVIDIA, configure with `-DENGINE_ENABLE_CUDA=ON` and use `--backend cuda`. Visual Studio executables are under `build-kitten2/bin/Release/`; single-configuration builds use `build-kitten2/bin/`. The examples below assume the executable is on your PATH and the GGUF is in the current directory.
 ```sh
-audiocpp_cli --family kitten_tts2 --model kitten-tts2-native-q8-multilingual.gguf --backend cpu --threads 8 --task tts --voice-id Bruno --text "Hello from native Kitten TTS 2." --out hello.wav
 ```
-German preset:
-```sh
-audiocpp_cli --family kitten_tts2 --model kitten-tts2-native-q8-multilingual.gguf --backend cpu --task tts --voice-id German --text "Guten Tag! Dies ist ein Beispiel." --out german.wav
-```
-Voice cloning:
-```sh
-audiocpp_cli --family kitten_tts2 --model kitten-tts2-native-q8-multilingual.gguf --backend cpu --threads 8 --task clon --voice-ref reference.wav --reference-text "The exact words spoken in the reference recording." --text "The new words to say in this voice." --out cloned.wav
-```
-For server configuration, the model family is `kitten_tts2`. A model entry may use `"id": "kitten-tts2"`, `"family": "kitten_tts2"`, `"task": "tts"`, `"mode": "offline"`, and `"path"` pointing to the downloaded GGUF. API requests use the configured model ID; for example, `"model": "kitten-tts2"` and `"voice": "Bruno"`.
 ## Validation and limitations
-CPU and NVIDIA CUDA synthesis and cloning were validated on Windows. The target branch was freshly built and tested on CPU; the earlier CUDA validation used the same model and CUDA implementation before transfer to this branch. See the [validation record](https://github.com/dignome/audio.cpp-custom/blob/kittentts2/tests/kitten_tts2/validation.md) for exact environments and results.
-The UI estimates **7 GB VRAM** for this package, including headroom. This is guidance, not a guaranteed minimum or maximum for every request. CUDA was tested on a 16 GB RTX 4060 Ti; AMD and other GPU backends have not been validated.
-This is an experimental port. Streaming, automatic reference transcription, the upstream English text normalizer, and the smaller student decoders are not supported here. Write numbers and abbreviations as spoken words when needed. The full default decoder is included.
 ## Sources and license
-- Kitten TTS 2: [KittenML/kitten-tts-2](https://huggingface.co/KittenML/kitten-tts-2), snapshot `baa41e5d2c5f64be0095365672a7858542261271`.
-- S3 decoder: [ResembleAI/chatterbox-turbo](https://huggingface.co/ResembleAI/chatterbox-turbo), revision `749d1c1a46eb10492095d68fbcf55691ccf137cd`, `s3gen_meanflow.safetensors`.
-- Speaker encoder: the `speaker/model.safetensors` distributed with the Kitten snapshot, attributed upstream to [pyannote/embedding](https://huggingface.co/pyannote/embedding).
-Kitten model materials retain the **Stellon Labs Community License** in [LICENSE](./LICENSE). Commercial use requires registration under that agreement, and exceeding its revenue or funding limits requires a separate license. The bundled decoder and speaker components retain their respective MIT terms in [native/LICENSE](./native/LICENSE) and [speaker/LICENSE](./speaker/LICENSE). See [NOTICE](./NOTICE) for attribution and conversion details.
-Conversion and native integration published by **dignome**. Bulk of the work was started with Sol 6.1 and finished with Astra.

 **Powered by Stellon Labs.**
+Self-contained multilingual GGUF exports of KittenML/kitten-tts-2 weights at
+revision `f2eee74f4467855ad58fe16f1fb7b7df473263fd`. Each file includes the default
+S3 MeanFlow decoder, speaker encoders, tokenizer, and all 48 prepared voice entries.
+Use an audio.cpp build containing the updated Kitten TTS 2 BF16/Q8/Q4 support.
+The model family is `kitten_tts2`. Model-manager installations share
+`models/kitten-tts2/`, with a distinct filename for each precision. Leave `kitten_tts2.weight_type` unset, or use
+`native`, to preserve each file's stored precision. Inference needs no Python
+or LibTorch installation.
+## Downloads
+**Q8 is the default and recommended package.** Q4 is experimental.
+| File | LM body / embedding | Size |
+| --- | --- | --- |
+| [kitten-tts2-bf16-multilingual.gguf](./kitten-tts2-bf16-multilingual.gguf) | BF16 / BF16 | 4,603,407,040 bytes |
+| [kitten-tts2-q8-v2-multilingual.gguf](./kitten-tts2-q8-v2-multilingual.gguf) | Q8_0 / BF16 | 3,282,201,248 bytes |
+| [kitten-tts2-q4-multilingual.gguf](./kitten-tts2-q4-multilingual.gguf) | Q4_0 / Q8_0 | 2,273,735,456 bytes |
+Verify downloads with [SHA256SUMS](./SHA256SUMS). All three packages use standard
+GGML tensor types. Their names describe LM storage; decoder and reference
+components retain their original precision.
+Q8 and Q4 preserve the supplied ternary body weights exactly. Q4 quantizes the
+embedding directly from full BF16 to Q8_0, with about 0.44% relative L2 weight
+error. This is a weight reconstruction measurement, not an audio quality score.
+On CUDA, Q4 down projections load as BF16 to avoid non-finite computation,
+adding about 483 MiB compared with retaining those projections as Q4.
+## Speech and cloning
 ```sh
+audiocpp_cli --family kitten_tts2 --model models/kitten-tts2/kitten-tts2-q8-v2-multilingual.gguf --backend cpu --threads 8 --task tts --voice-id Bruno --text "Hello from Kitten TTS 2." --out hello.wav
 ```
+Use `--backend cuda` for a CUDA-enabled build. For custom voice cloning:
 ```sh
+audiocpp_cli --family kitten_tts2 --model models/kitten-tts2/kitten-tts2-q8-v2-multilingual.gguf --backend cpu --threads 8 --task clon --voice-ref reference.wav --reference-text "The exact words in the reference recording." --text "The new words to say." --out cloned.wav
 ```
+Cloning requires a 1–30 second reference recording and its transcript. Preset
+synthesis needs no separate voice recordings. Output is mono 24 kHz.
+The package includes 47 upstream presets and the `PreparedBruno` alias. Named
+English voices include Bruno, Bella, Jasper and Luna. Language presets cover
+Arabic, Chinese, French, German, Hindi, Italian, Portuguese, Russian and Spanish.
+Choose a matching preset or reference clip for the target language. Generation
+checks cover these ten languages including English; they do not establish
+native-speaker pronunciation quality.
 ## Validation and limitations
+CPU and NVIDIA CUDA synthesis and cloning are checked locally. Q4 remains
+experimental: longer samples sometimes omitted words. Broader listening review
+is needed before relying on it for long-form speech. File sizes do not describe
+runtime RAM or VRAM requirements.
+The default decoder is used in every export. Student decoders and the upstream
+TL2/emb4 format are not included. Streaming and automatic reference transcription
+are not supported. Write numbers and abbreviations as spoken words when needed.
 ## Sources and license
+- Kitten weights: KittenML/kitten-tts-2, revision `f2eee74f4467855ad58fe16f1fb7b7df473263fd`.
+- Default S3 decoder: ResembleAI/chatterbox-turbo, revision `749d1c1a46eb10492095d68fbcf55691ccf137cd`.
+- Speaker encoder: the Kitten-distributed speaker model, attributed upstream to pyannote/embedding.
+Model materials retain the Stellon Labs Community License in [LICENSE](./LICENSE).
+Bundled components retain their respective terms in [native/LICENSE](./native/LICENSE)
+and [speaker/LICENSE](./speaker/LICENSE). See [NOTICE](./NOTICE) for attribution
+and conversion details.
+Conversion and native integration by **dignome**. Prepared with AI assistance
+and validated locally.