mlaik09/abdul-laya-decider

Abdul AI — Laya decider (int8 ONNX)

A small on-device "decider" for the Abdul AI phone assistant: it reads what the user said (English or Emirati Arabic) and picks one of the options it is given — a phone skill or chat — in one pass. Options are given per request, so new skills need no retraining.

• Base: convaiinnovations/laya-multilingual (mmBERT-base + decision head, Apache-2.0).
• Fine-tune: ~800 synthetic English + Emirati Arabic phone commands across 22 skills, with random option sets (3–9 options, shuffled, sometimes without the right skill so the answer is chat). 3 epochs, top 10 encoder layers + head trained, embeddings frozen.
• Quantization: embeddings int8 (Gather), every MatMul weight-only int8 in blocks of 128 (MatMulNBits). Built and checked with onnxruntime 1.28. 337 MB.
• Input (same as the laya package, choice question): input_ids, attention_mask, marker_pos, marker_mask, qtype (0); output logits [options] (+ act_probs). Instruction used in training: What does the user want the assistant to do?; each option is "<id>: <description>, e.g. <English> / <Arabic>".
• tokenizer.bin: compact copy of the official tokenizer for the app's Kotlin tokenizer (same ids).

Held-out test (hand-written, not from the training templates), int8 on onnxruntime 1.28:

Test	Accuracy
App's 5 options, 24 typical requests	24/24
App's 5 options, 185 mixed requests (incl. skills the app lacks → chat)	76%
Random 4–9 option sets	76% (Arabic 74%)
Skills never seen in training	94%

Code: tools/laya/ in the Abdul AI repository.