Instructions to use herimor/voxtream2.5 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use herimor/voxtream2.5 with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf herimor/voxtream2.5:Q8_0 # Run inference directly in the terminal: llama cli -hf herimor/voxtream2.5:Q8_0
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf herimor/voxtream2.5:Q8_0 # Run inference directly in the terminal: llama cli -hf herimor/voxtream2.5:Q8_0
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-app # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama serve -hf herimor/voxtream2.5:Q8_0 # Run inference directly in the terminal: ./build/bin/llama cli -hf herimor/voxtream2.5:Q8_0
Use Docker
docker model run hf.co/herimor/voxtream2.5:Q8_0
- Llama new
- LM Studio
- Jan
- Ollama
How to use herimor/voxtream2.5 with Ollama:
ollama run hf.co/herimor/voxtream2.5:Q8_0
- Unsloth Desktop
- Docker Model Runner
How to use herimor/voxtream2.5 with Docker Model Runner:
docker model run hf.co/herimor/voxtream2.5:Q8_0
- Lemonade
How to use herimor/voxtream2.5 with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull herimor/voxtream2.5:Q8_0
Run and chat with the model
lemonade run user.voxtream2.5-Q8_0
List all available models
lemonade list
- Atomic Chat
Download config.json from herimor/voxtream2.5: direct link, hf CLI and curl.
- Browser
- Download file 921 Bytes
-
https://huggingface.co/herimor/voxtream2.5/resolve/main/config.json
- Command line
-
hf download hf://herimor/voxtream2.5/config.json
-
curl -L -o config.json https://huggingface.co/herimor/voxtream2.5/resolve/main/config.json
921 Bytes
| { | |
| "phone_former": "phone_former", | |
| "temp_former": "temp_former", | |
| "dep_former": "dep_former", | |
| "audio_vocab_size": 2050, | |
| "audio_pad_size": 0, | |
| "embedding_dim": 1024, | |
| "spk_embedding_dim": 192, | |
| "num_codebooks": 32, | |
| "num_phone_states": 6, | |
| "amortization_divisor": 4, | |
| "max_look_ahead": 5, | |
| "audio_window_size": 625, | |
| "phone_window_size": 625, | |
| "phone_root_vocab_size": 62, | |
| "phone_attr_vocab_sizes": [ | |
| 3, | |
| 2, | |
| 5, | |
| 2 | |
| ], | |
| "phone_la_weights": [ | |
| 0.25, | |
| 0.15, | |
| 0.15, | |
| 0.15, | |
| 0.3 | |
| ], | |
| "train_prompt_sink": true, | |
| "dep_former_head": "mog", | |
| "mog_codebook_size": 2048, | |
| "mog_latent_dim": 256, | |
| "mog_num_components": 1024, | |
| "mog_low_rank": 64, | |
| "mog_num_layers": 3, | |
| "mog_intermediate_dim": 4096, | |
| "mog_min_log_std": -4.0, | |
| "mog_label_smoothing": 0.01, | |
| "mog_ar_levels": 3 | |
| } | |