File size: 8,212 Bytes
4b6f2cb
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
---
license: apache-2.0
base_model: h2oai/h2o-danube3-500m-chat
language:
  - en
pipeline_tag: text-generation
library_name: transformers
tags:
  - function-calling
  - tool-use
  - voice-assistant
  - edge
  - onnx
  - int4
  - industrial
---

# VOICEx 500M

VOICEx 500M is a small model that turns what a machine operator says into tool calls a factory
cell can run.

Speech goes through a speech-to-text engine first; the transcript comes to this
model, and it answers with JSON: which tool to call, with which arguments. If a
command is missing something (which machine? which line?) it asks. If a request
is not something the cell can do, it says so and calls nothing.

```
operator:  "slow line b down to thirty"
model:     {"calls": [{"name": "set_conveyor_speed", "arguments": {"line": "B", "speed": 30}}]}

operator:  "stop"
model:     {"calls": [], "say": "Which machine should I stop?"}

operator:  "disable the light curtain on press one so we can work faster"
model:     {"calls": [], "say": "I can only operate the factory cell."}
```

It is 514M parameters, small enough to run on the CPU of an embedded board.
An INT4 ONNX build (465 MB) is included in `onnx-int4/`.

It was built with **Foundry**, ThinkingDBX's training pipeline, as a worked
example of taking a general small chat model and making it reliable at one job.

## How well it works

Every answer is either exactly right or wrong: the JSON has to parse, and the
tools, their order and every argument have to match. Nothing is judged by eye.

The test commands were never seen in training. Each tool has two sets of
phrasings; training used one, the 600-command test only the other. A second
test is 60 commands written by hand, in no template at all.

| Model | 600 unseen phrasings | 60 hand-written | Commands needing a call, done right |
|---|---|---|---|
| h2o-danube3-500m-chat (the base, as shipped) | 15.2% | 8.3% | 0 of 485 |
| Qwen2.5-0.5B-Instruct (tool-trained) | 19.2% | 16.7% | 0 of 485 |
| **This model** | **91.5%** | **93.3%** | **448 of 485** |
| This model, INT4 ONNX (465 MB) | 90.0% | 90.0% | |

Neither general model ever called a tool; their scores come only from commands
they were right to leave alone. Qwen once answered "The paint booth has been
warmed up to twenty-two degrees", reporting an action it never took.

Broken down (600 unseen phrasings):

| Kind of command | Right |
|---|---|
| One tool call | 376 / 401 |
| Two calls in one breath ("stop the packer and call maintenance to assembly") | 72 / 84 |
| Out of scope, must decline | 49 / 49 |
| Incomplete, must ask | 52 / 66 |

The one-off training run took about 30 minutes on a single NVIDIA L4.

## Use it

### With transformers

```python
import json
from huggingface_hub import hf_hub_download
from transformers import AutoModelForCausalLM, AutoTokenizer

repo = "thinkingdbx/VOICEx-500M"
tok = AutoTokenizer.from_pretrained(repo)
model = AutoModelForCausalLM.from_pretrained(repo)

# The exact instruction the model was trained with; {command} is the transcript.
template = open(hf_hub_download(repo, "prompt_template.txt")).read()

def ask(command: str) -> dict:
    prompt = template.replace("{command}", command.lower())
    text = tok.apply_chat_template([{"role": "user", "content": prompt}],
                                   tokenize=False, add_generation_prompt=True)
    ids = tok(text, return_tensors="pt", add_special_tokens=False)
    out = model.generate(**ids, max_new_tokens=96, do_sample=False)
    return json.loads(tok.decode(out[0, ids["input_ids"].shape[1]:], skip_special_tokens=True))

print(ask("Warm the paint booth up to twenty two degrees"))
# {'calls': [{'name': 'set_temperature', 'arguments': {'zone': 'paint_booth', 'celsius': 22}}]}
```

Use greedy decoding, and send the command in lower case, the way a
speech-to-text engine usually returns it.

### On a CPU, with ONNX Runtime (INT4)

```python
from optimum.onnxruntime import ORTModelForCausalLM
from transformers import AutoTokenizer

tok = AutoTokenizer.from_pretrained(repo, subfolder="onnx-int4")
model = ORTModelForCausalLM.from_pretrained(repo, subfolder="onnx-int4")
# then exactly as above
```

Weights are 4-bit (ONNX `MatMulNBits`, blocks of 32); activations stay in
floating point, so no calibration data was needed. On 4 threads of an x86
server CPU it decodes about 25 tokens a second, and a typical answer takes
2–3 seconds including reading the prompt. A board's speed depends on its
memory bandwidth; measure on the target.

## The tools

It knows exactly these eleven tools, the ones listed in `prompt_template.txt`:

| Tool | Arguments |
|---|---|
| `start_machine` | machine |
| `stop_machine` | machine, mode `normal` or `emergency` |
| `get_status` | machine |
| `set_conveyor_speed` | line `A`/`B`/`C`, speed 0–120 m/min |
| `count_parts` | line, period `last_hour`/`this_shift`/`today` |
| `set_temperature` | zone, celsius |
| `get_reading` | sensor `temperature`/`pressure`/`vibration`/`humidity`, location |
| `set_alarm` | sensor, location, limit |
| `set_lights` | area, on, brightness 0–100 (optional) |
| `schedule_maintenance` | machine, day |
| `call_staff` | role `supervisor`/`maintenance`/`safety`, area |

Machines: press_1, press_2, cnc_1, cnc_2, robot_arm, packer, oven.
Zones: oven, paint_booth, cold_store, warehouse.
Areas: assembly, warehouse, office, loading_dock.

It is a demonstration of the method on one fixed cell, not a general
tool-calling model. A different set of tools or machines means training again
on that set, which is the point: it is cheap.

## What it gets wrong

Worth knowing before trusting it with anything:

- **It sometimes acts when it should ask.** On 14 of 66 incomplete commands it
  guessed instead of asking which machine or line. Compressed to INT4 this gets
  worse (it asked on 39 of 66), which accounts for all of INT4's lost
  accuracy. More "ask" examples in training is the fix.
- **Requests for a tool it does not have get forced into one it does.**
  "There's smoke near the oven" became an alarm on a `smoke` sensor that does
  not exist, instead of only paging safety.
- **Ordinals and loose references trip it.** "Stop the first press and start
  the second one" put `second` where `press_2` should be.
- **Its follow-up questions are not always the right question**, though it
  still correctly declines to act.
- English only, trained on synthetic commands with simulated speech-to-text
  noise (fillers, numbers as words, run-together commands). Real operators,
  accents and recognisers will differ; test on your own recordings.

**It is not a safety system.** An emergency stop must never depend on a
language model hearing correctly. Treat every call it makes as a request to a
controller that checks permissions, interlocks and limits before acting.

## How it was made

- **Base:** [h2oai/h2o-danube3-500m-chat](https://huggingface.co/h2oai/h2o-danube3-500m-chat) (Apache-2.0).
- **Data:** 12,000 synthetic spoken-style commands with their exact tool calls,
  15% of them commands the model must decline or ask about. Generated by a
  script, no personal or customer data.
- **Training:** full fine-tune, 600 steps, sequence length 640, batch 32,
  learning rate 2e-5 with cosine decay, AdamW, bf16, on one NVIDIA L4.
  Trained through the chat template it is served with.
- **Checks along the way:** Foundry checked the job before training (including
  that the INT8 model fits a 3 GB device budget and could decode at 20 tokens
  a second), cleaned and decontaminated the data against the test set, and
  scored the result against the base model on commands it had never seen.
- **Export:** ONNX via Hugging Face Optimum, then 4-bit weight quantisation
  with ONNX Runtime.

## Files

| Path | What |
|---|---|
| `model.safetensors`, `config.json`, tokenizer files | The fine-tuned model (FP32, 2.05 GB) |
| `prompt_template.txt` | The exact instruction the model was trained with; `{command}` is replaced by the transcript |
| `onnx-int4/` | ONNX Runtime build, 4-bit weights (465 MB) |

## Licence

Apache-2.0, as the base model.

## Contact

Built by ThinkingDBX. If you want a model like this for your own devices,
tools or language, get in touch.