Text Generation
GGUF
Chinese
English
llama.cpp
speech-to-text
text-rewriting
voice-input
conversational
Instructions to use ThermalEX/voxflow-rewrite with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use ThermalEX/voxflow-rewrite with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf ThermalEX/voxflow-rewrite:Q4_K_M # Run inference directly in the terminal: llama cli -hf ThermalEX/voxflow-rewrite:Q4_K_M
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf ThermalEX/voxflow-rewrite:Q4_K_M # Run inference directly in the terminal: llama cli -hf ThermalEX/voxflow-rewrite:Q4_K_M
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-app # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama serve -hf ThermalEX/voxflow-rewrite:Q4_K_M # Run inference directly in the terminal: ./build/bin/llama cli -hf ThermalEX/voxflow-rewrite:Q4_K_M
Use Docker
docker model run hf.co/ThermalEX/voxflow-rewrite:Q4_K_M
- Llama new
- LM Studio
- Jan
- vLLM
How to use ThermalEX/voxflow-rewrite with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "ThermalEX/voxflow-rewrite" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "ThermalEX/voxflow-rewrite", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/ThermalEX/voxflow-rewrite:Q4_K_M
- Ollama
How to use ThermalEX/voxflow-rewrite with Ollama:
ollama run hf.co/ThermalEX/voxflow-rewrite:Q4_K_M
- Unsloth Desktop
- Pi
How to use ThermalEX/voxflow-rewrite with Pi:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf ThermalEX/voxflow-rewrite:Q4_K_M
Configure the model in Pi
# Install Pi: npm install -g @earendil-works/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "llama-cpp": { "baseUrl": "http://localhost:8080/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "ThermalEX/voxflow-rewrite:Q4_K_M" } ] } } }Run Pi
# Start Pi in your project directory: pi
- Docker Model Runner
How to use ThermalEX/voxflow-rewrite with Docker Model Runner:
docker model run hf.co/ThermalEX/voxflow-rewrite:Q4_K_M
- Lemonade
How to use ThermalEX/voxflow-rewrite with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull ThermalEX/voxflow-rewrite:Q4_K_M
Run and chat with the model
lemonade run user.voxflow-rewrite-Q4_K_M
List all available models
lemonade list
- Hermes Agent
How to use ThermalEX/voxflow-rewrite with Hermes Agent:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf ThermalEX/voxflow-rewrite:Q4_K_M
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:8080/v1 hermes config set model.default ThermalEX/voxflow-rewrite:Q4_K_M
Run Hermes
hermes
- Atomic Chat
- OpenClaw
How to use ThermalEX/voxflow-rewrite with OpenClaw:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf ThermalEX/voxflow-rewrite:Q4_K_M
Configure OpenClaw
# Install OpenClaw: npm install -g openclaw@latest # Register the local server and set it as the default model: openclaw onboard --non-interactive --mode local \ --auth-choice custom-api-key \ --custom-base-url http://127.0.0.1:8080/v1 \ --custom-model-id "ThermalEX/voxflow-rewrite:Q4_K_M" \ --custom-provider-id llama-cpp \ --custom-compatibility openai \ --custom-text-input \ --accept-risk \ --skip-health
Run OpenClaw
openclaw agent --local --agent main --message "Hello from Hugging Face"
Download catalog.json from ThermalEX/voxflow-rewrite: direct link, hf CLI and curl.
- Browser
- Download file 7.09 kB
-
https://huggingface.co/ThermalEX/voxflow-rewrite/resolve/main/catalog.json
- Command line
-
hf download hf://ThermalEX/voxflow-rewrite/catalog.json
-
curl -L -o catalog.json https://huggingface.co/ThermalEX/voxflow-rewrite/resolve/main/catalog.json
7.09 kB
| { | |
| "version": 1, | |
| "items": [ | |
| { | |
| "id": "asr-sensevoice", | |
| "kind": "asr", | |
| "required": true, | |
| "recommended": true, | |
| "title": { | |
| "en": "Speech recognition model", | |
| "zh": "语音识别模型" | |
| }, | |
| "desc": { | |
| "en": "SenseVoice · Chinese, English, Japanese, Korean, Cantonese · required for voice input", | |
| "zh": "SenseVoice · 中、英、日、韩、粤语 · 语音输入必需" | |
| }, | |
| "check": "models/sherpa-onnx-sense-voice-zh-en-ja-ko-yue-int8-2024-07-17/model.int8.onnx", | |
| "files": [ | |
| { | |
| "repo": "csukuangfj/sherpa-onnx-sense-voice-zh-en-ja-ko-yue-2024-07-17", | |
| "revision": "2365baeacb507f821a0c8120fcee3d484dba7a07", | |
| "path": "model.int8.onnx", | |
| "size": 239233841, | |
| "sha256": "c71f0ce00bec95b07744e116345e33d8cbbe08cef896382cf907bf4b51a2cd51", | |
| "dest": "models/sherpa-onnx-sense-voice-zh-en-ja-ko-yue-int8-2024-07-17/model.int8.onnx" | |
| }, | |
| { | |
| "repo": "csukuangfj/sherpa-onnx-sense-voice-zh-en-ja-ko-yue-2024-07-17", | |
| "revision": "2365baeacb507f821a0c8120fcee3d484dba7a07", | |
| "path": "tokens.txt", | |
| "size": 315894, | |
| "sha256": "f449eb28dc567533d7fa59be34e2abca8784f771850c78a47fb731a31429a1dc", | |
| "dest": "models/sherpa-onnx-sense-voice-zh-en-ja-ko-yue-int8-2024-07-17/tokens.txt" | |
| }, | |
| { | |
| "repo": "csukuangfj/sherpa-onnx-sense-voice-zh-en-ja-ko-yue-2024-07-17", | |
| "revision": "2365baeacb507f821a0c8120fcee3d484dba7a07", | |
| "path": "LICENSE", | |
| "size": 71, | |
| "sha256": "221c6df10b0931a5629adad671ea48fb7747e034c414b6d2bfa275bc3dd4ea17", | |
| "dest": "models/sherpa-onnx-sense-voice-zh-en-ja-ko-yue-int8-2024-07-17/LICENSE" | |
| } | |
| ] | |
| }, | |
| { | |
| "id": "llama-vulkan", | |
| "kind": "runtime", | |
| "group": "llama", | |
| "recommended": true, | |
| "title": { | |
| "en": "llama.cpp · Universal (Vulkan)", | |
| "zh": "llama.cpp · 通用版(Vulkan)" | |
| }, | |
| "desc": { | |
| "en": "Runs smart editing · NVIDIA, AMD and Intel graphics · first start takes about 8 s, then as fast as the NVIDIA build", | |
| "zh": "运行智能编辑 · NVIDIA / AMD / Intel 显卡都能用 · 第一次启动约 8 秒,之后与加速版一样快" | |
| }, | |
| "check": "runtime/llama.cpp/ggml-vulkan.dll", | |
| "files": [ | |
| { | |
| "repo": "ThermalEX/voxflow-rewrite", | |
| "revision": "llama-b11596", | |
| "path": "runtime/llama-b11596-bin-win-vulkan-x64.zip", | |
| "size": 32128506, | |
| "sha256": "67eb3043f1070dba2021c5b9bd279041c003866548edfc6d5bfbd81842780811", | |
| "dest": "runtime/llama.cpp", | |
| "extract": true | |
| } | |
| ] | |
| }, | |
| { | |
| "id": "llama-cuda", | |
| "kind": "runtime", | |
| "group": "llama", | |
| "needs_nvidia": true, | |
| "title": { | |
| "en": "llama.cpp · NVIDIA build (CUDA)", | |
| "zh": "llama.cpp · NVIDIA 加速版(CUDA)" | |
| }, | |
| "desc": { | |
| "en": "Runs smart editing · NVIDIA graphics only · measured as fast as the universal build, 20× larger", | |
| "zh": "运行智能编辑 · 仅 NVIDIA 显卡 · 实测速度与通用版相同,体积大 20 倍" | |
| }, | |
| "check": "runtime/llama.cpp/ggml-cuda.dll", | |
| "files": [ | |
| { | |
| "repo": "ThermalEX/voxflow-rewrite", | |
| "revision": "llama-b11596", | |
| "path": "runtime/llama-b11596-bin-win-cuda-12.4-x64.zip", | |
| "size": 265222683, | |
| "sha256": "16df7d14d06a178c92baf621e21ba62c516183710bbf8e0ee9e0d47afe72a341", | |
| "dest": "runtime/llama.cpp", | |
| "extract": true | |
| }, | |
| { | |
| "repo": "ThermalEX/voxflow-rewrite", | |
| "revision": "llama-b11596", | |
| "path": "runtime/cudart-llama-bin-win-cuda-12.4-x64.zip", | |
| "size": 391443627, | |
| "sha256": "8c79a9b226de4b3cacfd1f83d24f962d0773be79f1e7b75c6af4ded7e32ae1d6", | |
| "dest": "runtime/llama.cpp", | |
| "extract": true | |
| } | |
| ] | |
| }, | |
| { | |
| "id": "rewrite-v6-q8", | |
| "kind": "rewrite", | |
| "recommended": true, | |
| "title": { | |
| "en": "Smart editing model v6 · Q8", | |
| "zh": "智能编辑模型 v6 · Q8" | |
| }, | |
| "desc": { | |
| "en": "Smart editing and memory · needs about 2.5 GB of graphics memory", | |
| "zh": "智能编辑与记忆 · 约需 2.5 GB 显存" | |
| }, | |
| "check": "runtime/models/voxflow-rewrite-v6-q8_0.gguf", | |
| "files": [ | |
| { | |
| "repo": "ThermalEX/voxflow-rewrite", | |
| "revision": "v6", | |
| "path": "voxflow-rewrite-q8_0.gguf", | |
| "size": 2076674496, | |
| "sha256": "7470f6aaea0dafefb6cff626686842d9de5f6a10c7666a2a43b05f6824ef0587", | |
| "dest": "runtime/models/voxflow-rewrite-v6-q8_0.gguf" | |
| } | |
| ] | |
| }, | |
| { | |
| "id": "rewrite-v6-q4", | |
| "kind": "rewrite", | |
| "title": { | |
| "en": "Smart editing model v6 · Q4", | |
| "zh": "智能编辑模型 v6 · Q4" | |
| }, | |
| "desc": { | |
| "en": "Less graphics memory · slightly lower quality", | |
| "zh": "更省显存 · 质量略低" | |
| }, | |
| "check": "runtime/models/voxflow-rewrite-v6-q4_k_m.gguf", | |
| "files": [ | |
| { | |
| "repo": "ThermalEX/voxflow-rewrite", | |
| "revision": "v6", | |
| "path": "voxflow-rewrite-q4_k_m.gguf", | |
| "size": 1312164288, | |
| "sha256": "cf9af334b88247ac8af045dc663eb51be970bf72bdf99aeb729519f6112ba2b6", | |
| "dest": "runtime/models/voxflow-rewrite-v6-q4_k_m.gguf" | |
| } | |
| ] | |
| }, | |
| { | |
| "id": "rewrite-v4-q8", | |
| "kind": "rewrite", | |
| "title": { | |
| "en": "Smart editing model v4 (previous) · Q8", | |
| "zh": "智能编辑模型 v4(上一版) · Q8" | |
| }, | |
| "desc": { | |
| "en": "Needs about 2.5 GB of graphics memory", | |
| "zh": "约需 2.5 GB 显存" | |
| }, | |
| "check": "runtime/models/voxflow-rewrite-v4-q8_0.gguf", | |
| "files": [ | |
| { | |
| "repo": "ThermalEX/voxflow-rewrite", | |
| "revision": "v4", | |
| "path": "voxflow-rewrite-q8_0.gguf", | |
| "size": 2076674496, | |
| "sha256": "0d07504034d1fbea23dcf64466c6ca31cd2593ba9a17c8c9443d51fe75d447b5", | |
| "dest": "runtime/models/voxflow-rewrite-v4-q8_0.gguf" | |
| } | |
| ] | |
| }, | |
| { | |
| "id": "rewrite-v4-q4", | |
| "kind": "rewrite", | |
| "title": { | |
| "en": "Smart editing model v4 (previous) · Q4", | |
| "zh": "智能编辑模型 v4(上一版) · Q4" | |
| }, | |
| "desc": { | |
| "en": "Less graphics memory · slightly lower quality", | |
| "zh": "更省显存 · 质量略低" | |
| }, | |
| "check": "runtime/models/voxflow-rewrite-v4-q4_k_m.gguf", | |
| "files": [ | |
| { | |
| "repo": "ThermalEX/voxflow-rewrite", | |
| "revision": "v4", | |
| "path": "voxflow-rewrite-q4_k_m.gguf", | |
| "size": 1312164288, | |
| "sha256": "94b067f09a653d126fedb21356039ee94dd9f05b172b52fc447056bc879f9d84", | |
| "dest": "runtime/models/voxflow-rewrite-v4-q4_k_m.gguf" | |
| } | |
| ] | |
| } | |
| ] | |
| } | |