Instructions to use OpenBloom/MOSS-TTS-Realtime-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use OpenBloom/MOSS-TTS-Realtime-GGUF with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf OpenBloom/MOSS-TTS-Realtime-GGUF:Q4_K_M # Run inference directly in the terminal: llama cli -hf OpenBloom/MOSS-TTS-Realtime-GGUF:Q4_K_M
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf OpenBloom/MOSS-TTS-Realtime-GGUF:Q4_K_M # Run inference directly in the terminal: llama cli -hf OpenBloom/MOSS-TTS-Realtime-GGUF:Q4_K_M
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-app # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama serve -hf OpenBloom/MOSS-TTS-Realtime-GGUF:Q4_K_M # Run inference directly in the terminal: ./build/bin/llama cli -hf OpenBloom/MOSS-TTS-Realtime-GGUF:Q4_K_M
Use Docker
docker model run hf.co/OpenBloom/MOSS-TTS-Realtime-GGUF:Q4_K_M
- Llama new
- LM Studio
- Jan
- Ollama
How to use OpenBloom/MOSS-TTS-Realtime-GGUF with Ollama:
ollama run hf.co/OpenBloom/MOSS-TTS-Realtime-GGUF:Q4_K_M
- Unsloth Desktop
- Docker Model Runner
How to use OpenBloom/MOSS-TTS-Realtime-GGUF with Docker Model Runner:
docker model run hf.co/OpenBloom/MOSS-TTS-Realtime-GGUF:Q4_K_M
- Lemonade
How to use OpenBloom/MOSS-TTS-Realtime-GGUF with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull OpenBloom/MOSS-TTS-Realtime-GGUF:Q4_K_M
Run and chat with the model
lemonade run user.MOSS-TTS-Realtime-GGUF-Q4_K_M
List all available models
lemonade list
- Atomic Chat
|
Download README.md from OpenBloom/MOSS-TTS-Realtime-GGUF: direct link, hf CLI and curl.
- Browser
- Download file 1.7 kB
-
https://huggingface.co/OpenBloom/MOSS-TTS-Realtime-GGUF/resolve/main/README.md
- Command line
-
hf download hf://OpenBloom/MOSS-TTS-Realtime-GGUF/README.md
-
curl -L -o README.md https://huggingface.co/OpenBloom/MOSS-TTS-Realtime-GGUF/resolve/main/README.md
1.7 kB
| license: apache-2.0 | |
| base_model: | |
| - OpenMOSS/MOSS-TTS-Realtime | |
| pipeline_tag: text-to-speech | |
| library_name: gguf | |
| tags: | |
| - gguf | |
| - tts | |
| - moss | |
| - speech | |
| - openbloom | |
| - kriskross | |
| - quantized | |
| language: | |
| - en | |
| - zh | |
| # MOSS-TTS-Realtime 2.0.0 — GGUF (Q4_K_M) | |
| OpenBloom Labs' GGUF quantization of [MOSS-TTS-Realtime 2.0.0](https://huggingface.co/OpenMOSS/MOSS-TTS-Realtime) | |
| — a realtime streaming text-to-speech model from the OpenMOSS team. | |
| ## Files | |
| | File | Size | sha256 | | |
| |---|---|---| | |
| | `moss-tts-realtime-q4_k_m.gguf` | 1,335,751,072 B (~1.24 GiB) | `ccd2ab76524affbc3a33a54f56224e24eabb008e73dd2b4142e8bf48cf492ed6` | | |
| The quantization keeps the local codebook embeddings at f16 (the host | |
| lookups require f16/f32 rows); everything else rides Q4_K_M. | |
| ## Usage | |
| The conversion targets llama.cpp via **[KrisKross](https://huggingface.co/OpenBloom/runtime-releases)** | |
| — OpenBloom's build of the OpenMOSS llama.cpp fork (`llama-moss-tts` + | |
| `llama-moss-tts-realtime`, the streaming duplex lane). Any | |
| llama.cpp-compatible loader that carries the MOSS-TTS architecture can | |
| serve it too. | |
| ``` | |
| # verify before use | |
| sha256sum moss-tts-realtime-q4_k_m.gguf | |
| ccd2ab76524affbc3a33a54f56224e24eabb008e73dd2b4142e8bf48cf492ed6 | |
| ``` | |
| ## License | |
| The MOSS-TTS model is **Apache-2.0** (© the OpenMOSS team and | |
| contributors); this quantization inherits the license verbatim. | |
| ## Attribution | |
| The model, the architecture and the training belong to | |
| [OpenMOSS](https://github.com/OpenMOSS/MOSS-TTS). OpenBloom Labs | |
| produced the GGUF conversion and publishes it for the OpenBloom | |
| installer's local TTS lanes (one-shot synthesis + the realtime duplex | |
| voice loop). Not affiliated with or endorsed by the OpenMOSS team. | |