zugetor/Lyra-Car-Assistant-enth

Lyra Car Assistant (Thai + English)

Say "เปิดแอร์ 23 องศา", "Open the tailgate" or "เพลงถัดไป". Get back climate/temperature = 23, tailgate = OPEN, media = next. Offline, on the device or in a browser tab.

Try the demo. Your browser pulls the files from this repo and runs the model itself, no server in between. Code, training data and the eval sets live on GitHub.

Under the hood it's Laya Multilingual: mmBERT-base plus Laya's decision head. We cut the vocabulary down to Thai and English, fine-tuned it on car commands and exported it to ONNX with 8-bit weights. Laya doesn't write text. It scores a question's options in one forward pass, so it can't make up a control that isn't on the list.

What it understands

control	key	payloads
Climate mode	climate/mode	off, auto
Climate temperature	climate/temperature	17–33 °C, or warmer / cooler when there's no number
Climate fan	climate/fan_mode	0–7, or up / down when there's no number
Windows	windows_all	OPEN, CLOSE, STOP
Tailgate	tailgate	OPEN, CLOSE, STOP
Sunroof	sunroof	OPEN, CLOSE, STOP
Sunshade	sunshade	OPEN, CLOSE, STOP
Seat ventilation	seat_vent_driver, seat_vent_passenger	off, low, medium, high
Daytime running lights	drl	on / off
Wireless phone charger	wireless_charging	1 / 0
Media	media	play, pause, next, previous, volume_up, volume_down, mute
Open app	open_app	carplay, android_auto

"Call mom" or "lock the doors" should come back as none.

Two model calls per command, at most. First: which control, out of 14 options. Then a rule reads the number for temperature and fan speed, and everything else gets that control's payload question. Both questions are in car_questions.json.

Files

file
laya.onnx	the graph, ~1 MB
laya.onnx.data	the 8-bit weights, ~211 MB. laya.onnx refers to it by name, so keep the two together
tokenizer/	Gemma BPE, cut from 256,000 tokens to 104,761
car_questions.json	the control question, the payload questions and label_to_key (the model says aircon, the car gets climate/mode)
rl_agent_config.json	Laya's config: calibrated temperatures, head length
finetune_summary.json	training and calibration report, with Laya's ask-again thresholds
encoder/config.json	encoder config
export_report.json	how far the ONNX file drifts from torch

Use it

In the browser

web_browser/lyra.js has the tokenizer, the prompt builder and the runtime, on ONNX Runtime Web 1.30.0. Point it at this repo:

import { CarAssistant } from "./lyra.js";   // needs ONNX Runtime Web loaded as the global `ort`
const car = await CarAssistant.load({ base: "https://huggingface.co/zugetor/Lyra-Car-Assistant-enth/resolve/main/" });
const r = await car.decide("เปิดฝาท้ายหน่อย");
r.action;   // { tailgate: "OPEN" }

In Python

The full runtime (both calls, the number rule, the ask-again gate) is runtime.py in the GitHub repo. Python 3.11–3.14:

git clone https://github.com/zugetor/Lyra-Car-Assistant && cd Lyra-Car-Assistant
python -m venv .venv && .venv/bin/pip install -r requirements.txt

from huggingface_hub import snapshot_download
from runtime import CarAssistant

folder = snapshot_download("zugetor/Lyra-Car-Assistant-enth", local_dir="lyra-car-model")
car = CarAssistant(path=f"{folder}/laya.onnx")
car.decide("ตั้งแอร์ 24 องศา")["action"]   # {'climate/temperature': 24}

Use local_dir. The default Hugging Face cache keeps each file as a symlink into its own blob folder, and ONNX Runtime refuses to load laya.onnx.data from a folder other than the one laya.onnx resolves to.

Only need one question? laya alone does it:

import json, laya
from huggingface_hub import snapshot_download

folder = snapshot_download("zugetor/Lyra-Car-Assistant-enth", local_dir="lyra-car-model")
agent = laya.load(folder, backend="onnx", onnx_path=f"{folder}/laya.onnx")
q = json.load(open(f"{folder}/car_questions.json", encoding="utf-8"))
agent.predict("เปิดฝาท้ายหน่อย", {"control": q["control"]})["answers"]["control"]["choice"]   # 'tailgate'

How it was made

Pruning. Laya has exactly one vocab-sized weight, the embedding table. We kept every token with a Thai character, English tokens that appear at least 20 times in wikitext-103, single characters, byte fallback, anything the car commands and questions need, and every merge piece required to build those. 104,761 of 256,000 tokens survive, and the model drops from 322 M to 206 M parameters. 11,327 / 11,327 number strings and 47,622 / 47,622 pure-Thai runs from Wikipedia tokenize exactly as before. So do 99.8% of English words in wikitext-103 test. Other scripts fall back to bytes.

Fine-tuning. 3,708 synthetic sentences (6,828 questions), half Thai, half English, generated from a phrase bank. Each control gets several object words (ฝาท้าย / ประตูหลัง / ท้ายรถ, tailgate / trunk / boot / liftgate), times verbs, politeness words and numbers. Another 300 sentences teach none. Then laya.train.finetune: 4 epochs, options shuffled every epoch, 400 items held back to calibrate the temperature. About an hour on Apple MPS.

Export. ONNX opset 18, encoder and head together. Weights are int8 (MatMulNBits, 8 bits, block 128; the embedding int8 per row), math in float. We tried dynamic INT8 first. Its answers changed with the runtime, so it went.

Results

There are two hand-written test sets. 89 "familiar" commands sit close to the training sentences. 52 "unseen" ones are worded differently, never appear verbatim in training, and weren't used to tune anything.

	Laya, pruned, zero-shot	this model
familiar: control / whole command	69.7% / 62.9%	100% / 100%
unseen: control / whole command	53.8% / 38.5%	96.2% / 94.2%
unseen, Thai / English control	50% / 58%	96% / 96%

Torch FP32, this ONNX file in Python and ONNX Runtime Web agree on all 141 commands. The browser's probabilities are at most 0.0001 off.

On a Mac it's 212 MB on disk. The Python runtime adds ~300 MB of private RAM after load. About half of that is torch, which laya imports even for ONNX, and the weight file is memory-mapped on top. In the browser, ONNX Runtime Web's WASM memory sits at 269 MB. A command takes ~150–210 ms on native CPU, ~450 ms in the browser with 4 WASM threads. Car hardware: not measured yet.

Limitations

One person typed both test sets. A real speech recognizer adds odd spacing, numbers spelled out as words and plain mishearings. The unseen set also borrows plenty of vocabulary from the phrase bank, so read 94% as a ceiling, not a promise.

Some misses are already known:

• "ช่วยเปิดแอร์ให้หน่อย ร้อนจัง" lands on temperature instead of AC auto.
• "Blow harder" goes to media. "It's too loud" turns the volume up.
• "เบาแอร์" and "หรี่แอร์" come out inconsistent, and the misspelling "เบาแอ" lowers the media volume.

Worse, the mistakes are confident. Wrong answers win by a margin of 0.98–0.99, and no confidence threshold will catch that. The car has to decide what deserves a confirmation, like moving a window or the tailgate with people nearby.

It covers the 13 controls above, nothing more. A new control means retraining, and the 256-token question budget has room for maybe 5–7 more. Thai and English only. Keep it away from anything safety-critical.

License

Apache-2.0, like the base model. Derived from convaiinnovations/laya-multilingual by Convai Innovations (revision 1720e3e). Changes: vocabulary pruned, fine-tuned on car commands, exported to ONNX, weights quantized to 8 bits.