--- language: en license: apache-2.0 tags: - decision - system-1 - hdc - vsa - neuro-symbolic - laya base_model: convaiinnovations/laya --- # hd-laya **hd-laya** removes laya's ~20-option choice ceiling with a hyperdimensional (HDC/VSA) option index, keeping laya's System-1 engine unmodified. This repo carries the **v4 calibrated checkpoint** (`hd_laya_trained_calibrated.pt`, 106 MB) which achieves a measured +3.1 pt accuracy gain over the base model on K=20 choice tasks and 80% accuracy at K=500/1000 (vs 1.6% base). ## Comparison: hd-laya vs base laya ![Comparison chart](comparison_chart.png) ### High-K stress suite (synthetic, 256 items per K) | K | base acc | hd-laya acc | improvement | |---|---|---|---| | 20 | 47.3% | 46.5% | −0.8 (flat) | | 77 | 14.5% | 26.6% | +12.1 | | 150 | 5.1% | 44.5% | +39.4 | | 500 | 1.6% | 80.5% | +78.9 | | 1000 | 0.78% | 79.7% | +78.9 | The HDC index narrows K options to 20 before the forward pass. At K=500/1000, the trained model matches keys in topically-similar sets with 80% accuracy, vs 1.6% for base laya. The z-gate is disabled (z_bar=0) for these numbers. ### Real-world benchmarks | dataset | intents | base laya | hd-laya | delta | index recall@20 | |---|---|---|---|---|---| | BANKING77 | 77 | 4.0% | 24.0% | +20.0% | ~1.0 | | CLINC150 | 151 | 20.0% | 3.3% | −16.7% | 0.75 | **BANKING77** (50 items): base laya is at chance (4% for 77-way). hd-laya narrows to 20 via HDC and achieves 24% — a 6× improvement. **CLINC150** (30 items): base laya gets 20% (chance is 0.67% for 151-way, so it's actually doing well). hd-laya gets 3.3% because the HDC index only achieves 75% recall@20 on CLINC150's more semantically diverse intents — when the correct intent isn't in the narrowed set, no amount of model training helps. ## Training | | | |---|---| | base model | `convaiinnovations/laya` (encoder partially unfrozen: last 2 layers) | | hardware | Kaggle T4 x2 | | task | synthetic high-K choice, K=20, 2048 items | | epochs | 12 (6144 steps) | | budgets | `max_len=512`, `head_max_len=192` | | trainable params | 26.5M head + 24.5M encoder (2 layers) | | held-out delta | +3.1 pts (base 52.3% → trained 55.5%) | | calibration | temp=1.427 for choice bucket (fit on 400 items) | ## Usage ```python from huggingface_hub import hf_hub_download from laya.agent import Agent from hd_laya.agent import HDLayaAgent from hd_laya.laya_agent import TrainedLayaAgent ckpt = hf_hub_download("geraldaton20/hd-laya", "hd_laya_trained_calibrated.pt") laya = Agent("convaiinnovations/laya", device="cuda") trained = TrainedLayaAgent.from_checkpoint(ckpt, agent=laya, max_len=512, head_max_len=192) agent = HDLayaAgent(trained, max_options=20, z_bar=0.0) result = agent.predict("customer wants to refund order", { "q": {"type": "choice", "instructions": "Choose the matching option.", "criteria": {"opt_0": "refund order (variant 0)", "opt_1": "reset password (variant 1)"}}, }) print(result["answers"]["q"]["choice"]) ``` ## Limitations - **z-gate needs per-model calibration.** The trained model's confidence distribution differs from base; fixed thresholds over-refuse. Using z_bar=0 (no refusals) gives the best accuracy. - **K=20 performance is flat vs base.** Training improved high-K key matching but not K=20 diverse-option accuracy. - **HDC index recall varies by dataset.** 100% on BANKING77, 75% on CLINC150. The index relies on lexical similarity; semantically diverse datasets are harder. - **Synthetic task.** Topics are 12 templates; real-world BANKING77 shows +20% improvement over base (4% → 24%). - **ECE is high** (0.80 at K=500). The model is overconfident on wrong answers. ## License Apache-2.0 (derived from laya; attribution to NandhaKishorM/laya). ## Links - GitHub: https://github.com/sensesVoid/hd-laya - Upstream: https://github.com/NandhaKishorM/laya