README.md

metadata

license: apache-2.0
base_model: convaiinnovations/laya
library_name: llamadart
tags:
  - laya
  - decision-model
  - intent-classification
  - safetensors

Laya command head

A decision head for Laya, fine-tuned to read the intent of a command typed into an app. It is used by llamadart's laya_command_bar example through DecisionEngine.

laya-head-commands.safetensors holds the 36 head tensors under Laya's PyTorch names, in F32, with no encoder tensors. Its laya.config metadata is the unmodified rl_agent_config.json of convaiinnovations/laya at revision 1c5edc17a7acd8701df6fc341c0d179f1c62c982, so DecisionEngine.load needs no configPath. Pair it with a Laya ModernBERT backbone GGUF, such as laya-Q8_0.gguf from fr0stbit3/laya-gguf.

final decisions = await DecisionEngine.load(
  engine, // a LlamaEngine with the Laya backbone GGUF loaded
  headPath: 'laya-head-commands.safetensors',
);

The head was trained on one choice question, with the typed text as the state: "What does the user want to do with this text typed into the app?", over eight options: search, task, event, reminder, message, calculate, ask and settings, each with the one-line criterion in the example's lib/src/intents.dart.

Training

• Base: the head of convaiinnovations/laya at 1c5edc17, with the encoder frozen. Only head.*, type_emb.* and scorer.* were trained; the act head and the temperatures are unchanged.
• Data: 3,063 labelled commands from the example's bin/make_dataset.dart: 48 seed commands, 1,346 from templates, and 1,669 generated by Qwen3.8-27B (Q4_K_M) with bin/generate_commands.dart and kept only where the same model, asked again with bin/verify_commands.dart, gave the same intent (1,949 generated). Commands that normalize to a development or held-out command were dropped. Targets are one-hot.
• Recipe: 60 epochs; AdamW at lr 3e-4, weight decay 0.01; 50 warmup steps, then cosine decay; batch 32; three seeds. The epoch with the best accuracy on the 48 development commands was kept: seed 0, epoch 29. This is the example's training/laya_head_tuning.ipynb.

Results

Top intent correct on the example's 48 development commands, which chose the checkpoint and the gate, and 32 held-out commands, which did not:

Head	Development	Held-out
Laya base head, PyTorch F32 encoder	29	18
Laya base head, llamadart with laya-Q8_0.gguf on Metal	28	18
This head, PyTorch F32 encoder	46	27
This head, llamadart with laya-Q8_0.gguf on Metal	46	27

In the example's bin/bench.dart with its 0.3 confidence gate, the bar shows the right intent for 26 held-out commands, the wrong one for 4, and stays plain for 2. The notebook's other seeds kept heads with up to four fewer held-out commands right in PyTorch, so a single run is noisy at this size.

Limitations

• Trained only on the question above and its eight options. Other questions and option sets have not been evaluated with this head.
• Short English commands only, like the base model.
• The example's EmbeddingGemma nearest-example reader and small LLM readers are more accurate on the same commands; this head is a fine-tuning example, not the best intent reader.

License

Apache-2.0, like the base model. This is a modified version of the convaiinnovations/laya head: the weights listed under Training were fine-tuned on the command data.