Text Generation
Transformers
ONNX
Safetensors
English
llama
function-calling
tool-use
voice-assistant
edge
int4
industrial
conversational
text-generation-inference
Instructions to use thinkingdbx/VOICEx-500M with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use thinkingdbx/VOICEx-500M with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="thinkingdbx/VOICEx-500M") messages = [ {"role": "user", "content": "Who are you?"}, ] pipe(messages)# pip install -U transformers accelerate # Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("thinkingdbx/VOICEx-500M") model = AutoModelForCausalLM.from_pretrained("thinkingdbx/VOICEx-500M", device_map="auto") messages = [ {"role": "user", "content": "Who are you?"}, ] inputs = tokenizer.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=256) print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use thinkingdbx/VOICEx-500M with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "thinkingdbx/VOICEx-500M" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "thinkingdbx/VOICEx-500M", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/thinkingdbx/VOICEx-500M
- SGLang
How to use thinkingdbx/VOICEx-500M with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "thinkingdbx/VOICEx-500M" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "thinkingdbx/VOICEx-500M", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "thinkingdbx/VOICEx-500M" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "thinkingdbx/VOICEx-500M", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }' - Docker Model Runner
How to use thinkingdbx/VOICEx-500M with Docker Model Runner:
docker model run hf.co/thinkingdbx/VOICEx-500M
File size: 8,212 Bytes
4b6f2cb | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 | ---
license: apache-2.0
base_model: h2oai/h2o-danube3-500m-chat
language:
- en
pipeline_tag: text-generation
library_name: transformers
tags:
- function-calling
- tool-use
- voice-assistant
- edge
- onnx
- int4
- industrial
---
# VOICEx 500M
VOICEx 500M is a small model that turns what a machine operator says into tool calls a factory
cell can run.
Speech goes through a speech-to-text engine first; the transcript comes to this
model, and it answers with JSON: which tool to call, with which arguments. If a
command is missing something (which machine? which line?) it asks. If a request
is not something the cell can do, it says so and calls nothing.
```
operator: "slow line b down to thirty"
model: {"calls": [{"name": "set_conveyor_speed", "arguments": {"line": "B", "speed": 30}}]}
operator: "stop"
model: {"calls": [], "say": "Which machine should I stop?"}
operator: "disable the light curtain on press one so we can work faster"
model: {"calls": [], "say": "I can only operate the factory cell."}
```
It is 514M parameters, small enough to run on the CPU of an embedded board.
An INT4 ONNX build (465 MB) is included in `onnx-int4/`.
It was built with **Foundry**, ThinkingDBX's training pipeline, as a worked
example of taking a general small chat model and making it reliable at one job.
## How well it works
Every answer is either exactly right or wrong: the JSON has to parse, and the
tools, their order and every argument have to match. Nothing is judged by eye.
The test commands were never seen in training. Each tool has two sets of
phrasings; training used one, the 600-command test only the other. A second
test is 60 commands written by hand, in no template at all.
| Model | 600 unseen phrasings | 60 hand-written | Commands needing a call, done right |
|---|---|---|---|
| h2o-danube3-500m-chat (the base, as shipped) | 15.2% | 8.3% | 0 of 485 |
| Qwen2.5-0.5B-Instruct (tool-trained) | 19.2% | 16.7% | 0 of 485 |
| **This model** | **91.5%** | **93.3%** | **448 of 485** |
| This model, INT4 ONNX (465 MB) | 90.0% | 90.0% | |
Neither general model ever called a tool; their scores come only from commands
they were right to leave alone. Qwen once answered "The paint booth has been
warmed up to twenty-two degrees", reporting an action it never took.
Broken down (600 unseen phrasings):
| Kind of command | Right |
|---|---|
| One tool call | 376 / 401 |
| Two calls in one breath ("stop the packer and call maintenance to assembly") | 72 / 84 |
| Out of scope, must decline | 49 / 49 |
| Incomplete, must ask | 52 / 66 |
The one-off training run took about 30 minutes on a single NVIDIA L4.
## Use it
### With transformers
```python
import json
from huggingface_hub import hf_hub_download
from transformers import AutoModelForCausalLM, AutoTokenizer
repo = "thinkingdbx/VOICEx-500M"
tok = AutoTokenizer.from_pretrained(repo)
model = AutoModelForCausalLM.from_pretrained(repo)
# The exact instruction the model was trained with; {command} is the transcript.
template = open(hf_hub_download(repo, "prompt_template.txt")).read()
def ask(command: str) -> dict:
prompt = template.replace("{command}", command.lower())
text = tok.apply_chat_template([{"role": "user", "content": prompt}],
tokenize=False, add_generation_prompt=True)
ids = tok(text, return_tensors="pt", add_special_tokens=False)
out = model.generate(**ids, max_new_tokens=96, do_sample=False)
return json.loads(tok.decode(out[0, ids["input_ids"].shape[1]:], skip_special_tokens=True))
print(ask("Warm the paint booth up to twenty two degrees"))
# {'calls': [{'name': 'set_temperature', 'arguments': {'zone': 'paint_booth', 'celsius': 22}}]}
```
Use greedy decoding, and send the command in lower case, the way a
speech-to-text engine usually returns it.
### On a CPU, with ONNX Runtime (INT4)
```python
from optimum.onnxruntime import ORTModelForCausalLM
from transformers import AutoTokenizer
tok = AutoTokenizer.from_pretrained(repo, subfolder="onnx-int4")
model = ORTModelForCausalLM.from_pretrained(repo, subfolder="onnx-int4")
# then exactly as above
```
Weights are 4-bit (ONNX `MatMulNBits`, blocks of 32); activations stay in
floating point, so no calibration data was needed. On 4 threads of an x86
server CPU it decodes about 25 tokens a second, and a typical answer takes
2–3 seconds including reading the prompt. A board's speed depends on its
memory bandwidth; measure on the target.
## The tools
It knows exactly these eleven tools, the ones listed in `prompt_template.txt`:
| Tool | Arguments |
|---|---|
| `start_machine` | machine |
| `stop_machine` | machine, mode `normal` or `emergency` |
| `get_status` | machine |
| `set_conveyor_speed` | line `A`/`B`/`C`, speed 0–120 m/min |
| `count_parts` | line, period `last_hour`/`this_shift`/`today` |
| `set_temperature` | zone, celsius |
| `get_reading` | sensor `temperature`/`pressure`/`vibration`/`humidity`, location |
| `set_alarm` | sensor, location, limit |
| `set_lights` | area, on, brightness 0–100 (optional) |
| `schedule_maintenance` | machine, day |
| `call_staff` | role `supervisor`/`maintenance`/`safety`, area |
Machines: press_1, press_2, cnc_1, cnc_2, robot_arm, packer, oven.
Zones: oven, paint_booth, cold_store, warehouse.
Areas: assembly, warehouse, office, loading_dock.
It is a demonstration of the method on one fixed cell, not a general
tool-calling model. A different set of tools or machines means training again
on that set, which is the point: it is cheap.
## What it gets wrong
Worth knowing before trusting it with anything:
- **It sometimes acts when it should ask.** On 14 of 66 incomplete commands it
guessed instead of asking which machine or line. Compressed to INT4 this gets
worse (it asked on 39 of 66), which accounts for all of INT4's lost
accuracy. More "ask" examples in training is the fix.
- **Requests for a tool it does not have get forced into one it does.**
"There's smoke near the oven" became an alarm on a `smoke` sensor that does
not exist, instead of only paging safety.
- **Ordinals and loose references trip it.** "Stop the first press and start
the second one" put `second` where `press_2` should be.
- **Its follow-up questions are not always the right question**, though it
still correctly declines to act.
- English only, trained on synthetic commands with simulated speech-to-text
noise (fillers, numbers as words, run-together commands). Real operators,
accents and recognisers will differ; test on your own recordings.
**It is not a safety system.** An emergency stop must never depend on a
language model hearing correctly. Treat every call it makes as a request to a
controller that checks permissions, interlocks and limits before acting.
## How it was made
- **Base:** [h2oai/h2o-danube3-500m-chat](https://huggingface.co/h2oai/h2o-danube3-500m-chat) (Apache-2.0).
- **Data:** 12,000 synthetic spoken-style commands with their exact tool calls,
15% of them commands the model must decline or ask about. Generated by a
script, no personal or customer data.
- **Training:** full fine-tune, 600 steps, sequence length 640, batch 32,
learning rate 2e-5 with cosine decay, AdamW, bf16, on one NVIDIA L4.
Trained through the chat template it is served with.
- **Checks along the way:** Foundry checked the job before training (including
that the INT8 model fits a 3 GB device budget and could decode at 20 tokens
a second), cleaned and decontaminated the data against the test set, and
scored the result against the base model on commands it had never seen.
- **Export:** ONNX via Hugging Face Optimum, then 4-bit weight quantisation
with ONNX Runtime.
## Files
| Path | What |
|---|---|
| `model.safetensors`, `config.json`, tokenizer files | The fine-tuned model (FP32, 2.05 GB) |
| `prompt_template.txt` | The exact instruction the model was trained with; `{command}` is replaced by the transcript |
| `onnx-int4/` | ONNX Runtime build, 4-bit weights (465 MB) |
## Licence
Apache-2.0, as the base model.
## Contact
Built by ThinkingDBX. If you want a model like this for your own devices,
tools or language, get in touch.
|