LakoreAI/PhoLaya-base

PhoLaya — fine-tuned from convaiinnovations/laya-multilingual (mmBERT-base). Not affiliated with VinAI (PhoBERT) or Convai Innovations (Laya).

PhoLaya (first version)

A Vietnamese typed-decision model (choice / noul; score is untrained): given a Vietnamese text and a question with options, it returns a calibrated probability per option in one forward pass. Architecture is laya-multilingual unchanged (mmBERT-base encoder + Laya decision head, 322M parameters); only the weights were fine-tuned.

Training

• Data: LakoreAI/PhoLaya-base train split — recast ViSFD, VSFC, ViNLI, MASSIVE vi-VN (179,231 cases). Soft cross-entropy only (w_rl = 0), no RL.
• Budget 1024 tokens total / 256 option tokens; 3 epochs, lr 2e-5 (head 1e-4), batch 16, bf16.
• Train-time augmentation: 30% diacritic stripping, option-order permutation, question paraphrases (20% English).
• Calibration: per-(type, K-bucket) temperature fit on a 10% slice carved from train (temperatures.json). Nothing was fit on test.

Accuracy (%) by source — accented test

model	all	visfd	vsfc	vinli	massive
E0 majority class	57.59	69.23	60.98	49.78	10.56
E0 zero-shot Laya	36.64	26.46	50.72	65.33	34.42
E2 best (epoch 2)	89.19	91.78	90.47	83.02	83.00
E2 epoch 3	89.36	91.76	90.87	83.17	83.75

Accuracy (%) by source — diacritic-stripped test

model	all	visfd	vsfc	vinli	massive
E0 majority class	57.59	69.23	60.98	49.78	10.56
E0 zero-shot Laya	19.06	19.37	13.47	30.73	11.35
E2 best (epoch 2)	85.43	89.56	86.31	77.20	75.12
E2 epoch 3	85.91	89.87	86.66	77.15	77.07

Calibration (all sources)

model	variant	n	acc	raw ECE	post-T ECE	post-T NLL	post-T Brier
E0 zero-shot Laya	test	42062	36.64	44.58	10.71	1.414	0.7076
E0 zero-shot Laya	test_stripped	42062	19.06	52.36	24.36	1.912	0.8795
E2 best (epoch 2)	test	42062	89.19	2.68	1.61	0.317	0.1576
E2 best (epoch 2)	test_stripped	42062	85.43	1.86	0.61	0.410	0.2062
E2 epoch 3	test	42062	89.36	4.44	1.98	0.323	0.1577
E2 epoch 3	test_stripped	42062	85.91	3.79	0.62	0.403	0.2012

Seeds

seeds available: 1 (need >= 2 for mean ± std)

Limitations

• Trained only on student feedback, phone reviews, NLI and voice-assistant commands; short texts (median ~150 characters). Out-of-domain behaviour is untested.
• No score data; option counts seen in training are 2, 3, 4, 18, 60.
• One seed unless the Seeds section says otherwise. Labels are recast human annotations (one-hot); the UIT-derived sources are research-licensed — research use only.
• Zero-shot Laya (E0) is the baseline row; gains are relative to it, not to other decision models.

Reproduce

Code and configs: configs/train/E2_main.yaml; tables above are regenerated by scripts/make_tables.py from results/*/report*.json.