Instructions to use OpenBloom/MOSS-TTS-Realtime-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use OpenBloom/MOSS-TTS-Realtime-GGUF with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf OpenBloom/MOSS-TTS-Realtime-GGUF:Q4_K_M # Run inference directly in the terminal: llama cli -hf OpenBloom/MOSS-TTS-Realtime-GGUF:Q4_K_M
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf OpenBloom/MOSS-TTS-Realtime-GGUF:Q4_K_M # Run inference directly in the terminal: llama cli -hf OpenBloom/MOSS-TTS-Realtime-GGUF:Q4_K_M
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-app # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama serve -hf OpenBloom/MOSS-TTS-Realtime-GGUF:Q4_K_M # Run inference directly in the terminal: ./build/bin/llama cli -hf OpenBloom/MOSS-TTS-Realtime-GGUF:Q4_K_M
Use Docker
docker model run hf.co/OpenBloom/MOSS-TTS-Realtime-GGUF:Q4_K_M
- Llama new
- LM Studio
- Jan
- Ollama
How to use OpenBloom/MOSS-TTS-Realtime-GGUF with Ollama:
ollama run hf.co/OpenBloom/MOSS-TTS-Realtime-GGUF:Q4_K_M
- Unsloth Desktop
- Docker Model Runner
How to use OpenBloom/MOSS-TTS-Realtime-GGUF with Docker Model Runner:
docker model run hf.co/OpenBloom/MOSS-TTS-Realtime-GGUF:Q4_K_M
- Lemonade
How to use OpenBloom/MOSS-TTS-Realtime-GGUF with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull OpenBloom/MOSS-TTS-Realtime-GGUF:Q4_K_M
Run and chat with the model
lemonade run user.MOSS-TTS-Realtime-GGUF-Q4_K_M
List all available models
lemonade list
- Atomic Chat
MOSS-TTS-Realtime 2.0.0 β GGUF (Q4_K_M)
OpenBloom Labs' GGUF quantization of MOSS-TTS-Realtime 2.0.0 β a realtime streaming text-to-speech model from the OpenMOSS team.
Files
| File | Size | sha256 |
|---|---|---|
moss-tts-realtime-q4_k_m.gguf |
1,335,751,072 B (~1.24 GiB) | ccd2ab76524affbc3a33a54f56224e24eabb008e73dd2b4142e8bf48cf492ed6 |
The quantization keeps the local codebook embeddings at f16 (the host lookups require f16/f32 rows); everything else rides Q4_K_M.
Usage
The conversion targets llama.cpp via KrisKross
β OpenBloom's build of the OpenMOSS llama.cpp fork (llama-moss-tts +
llama-moss-tts-realtime, the streaming duplex lane). Any
llama.cpp-compatible loader that carries the MOSS-TTS architecture can
serve it too.
# verify before use
sha256sum moss-tts-realtime-q4_k_m.gguf
ccd2ab76524affbc3a33a54f56224e24eabb008e73dd2b4142e8bf48cf492ed6
License
The MOSS-TTS model is Apache-2.0 (Β© the OpenMOSS team and contributors); this quantization inherits the license verbatim.
Attribution
The model, the architecture and the training belong to OpenMOSS. OpenBloom Labs produced the GGUF conversion and publishes it for the OpenBloom installer's local TTS lanes (one-shot synthesis + the realtime duplex voice loop). Not affiliated with or endorsed by the OpenMOSS team.
- Downloads last month
- 88
4-bit