r38: ppo_cho2 chain (choose wire 0.800), ft_skill5_ppo (stage-2 skill heads), refreshed card measurements

Chains: base, ft_lang, ft_gr14_ppo, ft_tool5v2_ppo, ft_skill5_ppo, ft_triage5, ft_mail8, ft_sup3, ppo_cho2, ft_cmp2, ft_rrk3. ANE-pure f16 static, 0 fp32 islands, acceptance PASS on ne.

README.md

@@ -13,7 +13,7 @@ tags:
 pipeline_tag: text-classification
 ---
 
-# Laya Decision-Plugin — combined Core AI decision model (r37)
+# Laya Decision-Plugin — combined Core AI decision model (r38)
 
 A **typed decision classifier** for coding agents: given an agent state and a
 typed question, it answers *which tool*, *which skill*, *allow/ask/block*,
@@ -45,7 +45,7 @@ exactly the dialect it was trained on); **temperatures ship per
 option-bucket**, matching each RL head's optimization point (builder aborts
 on stale configs).
 
-## Scores — wire battery, frozen r36/r37 (`bench/reports/r36-dialect-fix/`)
+## Scores — wire battery, r38 (plugin repo `bench/reports/r38-final/`)
 
 Accuracy is measured **through the deployed daemon over MCP** (train==serve
 strings), on held-out oracles. "wire = head" means the serving path adds no
@@ -55,13 +55,13 @@ loss versus offline evaluation.
 |---|----------|-------|---------------------------|-----------------|--------|
 | 1 | guardrail `guard_command` | ft_gr14_ppo | 0.315 exact-disposition, adversarial n=184; **red-line holds 11/91 never-trained reds** | 34/38 ms | shipped |
 | 2 | lang `lang_route` | ft_lang | **0.829** (open-massive, n=2606; base 0.159) | 36/39 ms | shipped |
-| 3 | skill `skill_select` | ft_skill4_ppo | 0.360 wire (n=161); head offline 0.615 @ 100 % coverage | 71/107 ms | shipped |
+| 3 | skill `skill_select` | ft_skill5_ppo | 0.348 wire (n=161); stage-2 fit-judgment heads added: needs_skill **0.887**, skill_needed **0.881** (dual-blind ho, n=801; previous chain was at chance there) | 68/101 ms | shipped |
 | 4 | triage `triage_message` | ft_triage5 | 0.575 wire = 0.625 head (n=40; teacher's own pack 0.700 caps end-to-end) | 215/219 ms | shipped |
 | 5 | mail_sort `mail_sort` | ft_mail8 | 0.625–0.708 wire (two harnesses, n=24) = 0.70 head champion (teacher ceiling 0.95) | 35/37 ms | shipped |
 | 6 | supervise_run | ft_sup3 | **0.679 wire** (n=28; head 0.643, base 0.607) | 178/182 ms | shipped |
 | 7 | rerank_noul | ft_rrk3 | 0.100 exact = head (n=10; jaccard 0.765; base 0.000; teacher ceiling 8/10) | 215/281 ms | shipped |
 | 8 | compact_context | ft_cmp2 | 0.300 wire = head (n=10 exact-invariant golds; base 0.000) | 324/340 ms | shipped |
-| 9 | choose_action | ppo_cho | **0.650 wire = head = champion** (n=20; base 0.550) | 34/38 ms | shipped |
+| 9 | choose_action | ppo_cho2 | **0.800 wire = head** (n=20; base 0.550, r37 chain 0.650) | 21/22 ms | shipped |
 | 10 | ladder_plan / plan_request / ack_gate | none by design | code-only use cases (arithmetic, rails, local rule) — zero model calls | 0 | by design |
 
 Three further use cases (ladder planning, request planning, ack gate) are
@@ -77,11 +77,11 @@ identical held-out golds:**
 | guardrail | 0.194 (raw red 10/35 unseen) | **0.465** routed; red-line holds 11/91 never-trained reds at floor | 2.4× |
 | lang | 0.159 | **0.829** wire (0.958 head) | 5.2× |
 | tool_route (20-slate) | 0.000 (acted 2/28, all wrong) | **0.750 @ 80 % coverage** (frozen oc2) | ∞ |
-| skill_select (107-slate) | 0.242 | **0.615 @ 100 % coverage** (dual-blind ho) | 2.5× |
+| skill_select (107-slate) | 0.242 | **0.615 @ 100 % coverage** stage-1 (dual-blind ho); stage-2 fit-judgment **0.887 / 0.881** added r38 | 2.5× |
 | triage | 0.250 | **0.625** head / 0.575 wire | 2.5× |
 | mail_sort | 0.250 | **0.70** champion (wire 0.625–0.708) | 2.8× |
 | supervise | 0.607 | **0.679** wire (head 0.643) | +12 % |
-| choose | 0.550 | **0.650** wire = head | +18 % |
+| choose | 0.550 | **0.800** wire = head (ppo_cho2; r37 chain 0.650) | +45 % |
 | compact | 0.000 | **0.300** wire = head | ∞ |
 | rerank | 0.000 | **0.100** exact / 0.765 jaccard | ∞ |
 
@@ -167,8 +167,12 @@ reads wire = head.
 - Skill-route real in-harness traffic is sparse; the verdict rests on a
   dual-blind-graded public holdout.
 - rerank/compact are early-loop chains: exact-match accuracy is low but each
-  beats the base chain (0.000) and the frozen head; corpus expansion is the
-  known lever.
+  beats the base chain (0.000). Public-corpus expansion was attempted r38
+  and honestly reverted: the harvested faces taught a generic prior that
+  contradicted the product register (sharegpt turns score "summarize" under
+  the dual-blind rubric while the product keeps most turns), so the wire
+  pack regressed despite winning offline shootouts. Face construction, not
+  face volume, is the lever for these two chains.
 - Trained for the 20-tool / 107-slate coding-agent harness; foreign tool
   slates degrade gracefully to escalation, not to correct guesses.
 - Base checkpoint `convaiinnovations/laya-multilingual` is **Apache-2.0**;

combined_provenance.json

@@ -33,8 +33,8 @@
     "tool_route"
    ]
   },
-  "ft_skill4_ppo": {
-   "sha256": "795d407003097d573f7fb63a453a68d6a412a625751b6f5d318bcfc09288eeca",
+  "ft_skill5_ppo": {
+   "sha256": "0969146cf115ad9ce5a68fce69e6ea92eb83413a784592cbc68620a022922a55",
    "chain_index": 4,
    "tasks": [
     "skill_route"
@@ -61,8 +61,8 @@
     "supervise"
    ]
   },
-  "ppo_cho": {
-   "sha256": "41f87018d0589e045c2c148d1bdc0983adc97735df65ef84bd382a866527dd2a",
+  "ppo_cho2": {
+   "sha256": "92daac9c9c679c70a03296095d5780420bafa6d53f2c0508c3aa13458697faf1",
    "chain_index": 8,
    "tasks": [
     "choose"
@@ -90,10 +90,51 @@
   "dynamic": "static (padded serving)"
  },
  "torch_parity": {
-  "per_head": {},
+  "per_head": {
+   "ft_lang": {
+    "rows": 24,
+    "maxabs": 0.0
+   },
+   "ft_gr14_ppo": {
+    "rows": 24,
+    "maxabs": 0.0
+   },
+   "ft_tool5v2_ppo": {
+    "rows": 24,
+    "maxabs": 0.0
+   },
+   "ft_skill5_ppo": {
+    "rows": 24,
+    "maxabs": 0.0
+   },
+   "ft_triage5": {
+    "rows": 24,
+    "maxabs": 0.0
+   },
+   "ft_mail8": {
+    "rows": 24,
+    "maxabs": 0.0
+   },
+   "ft_sup3": {
+    "rows": 24,
+    "maxabs": 0.0
+   },
+   "ppo_cho2": {
+    "rows": 24,
+    "maxabs": 0.0
+   },
+   "ft_cmp2": {
+    "rows": 24,
+    "maxabs": 0.0
+   },
+   "ft_rrk3": {
+    "rows": 24,
+    "maxabs": 0.0
+   }
+  },
   "maxabs_f32": 0.0,
   "gate": 0.0001,
-  "verdict": "SKIPPED"
+  "verdict": "PASS"
  },
  "build_seconds": 6.0,
  "note": "raw un-tempered logits; deployment temperatures in models/<head>/rl_agent_config.json (temperature_by_options)",
@@ -121,10 +162,10 @@
  "runtime_acceptance": {
   "unit": "ne",
   "argmax_agree": "100% all buckets",
-  "mean_KL_max_bucket": 0.00054,
+  "mean_KL_max_bucket": 0.00027,
   "nonfinite": 0,
   "verdict": "PASS",
-  "main_mlirb_sha256": "d94bfcf029353d16a939147e0b895bccf0a55af05fa103ac7aef943ca0358620"
+  "main_mlirb_sha256": "b573110e1c1bb1f2e440912db5cbd6457ce58160f8689471c76f36dced9bb30d"
  },
- "asset_note": "r37 ANE swap: this bundle is the ANE-pure static export (zero fp32 islands); the previous dynamic fp32-island asset is archived as laya-combined-f16-pre-ane-dynamic.aimodel.bak."
+ "asset_note": "promoted ANE-pure static export (previous asset archived as laya-combined-f16-prev-02.aimodel.bak)"
 }

configs/ft_skill5_ppo.rl_agent_config.json

@@ -0,0 +1,60 @@
+{
+ "encoder": "jhu-clsp/mmBERT-base",
+ "head_layers": 2,
+ "max_len": 1024,
+ "head_max_len": 256,
+ "max_prefixes": 6,
+ "act_costs": {
+  "escalate": 0.5
+ },
+ "cost_wrong_act": 3.0,
+ "amp_dtype": "bf16",
+ "model_name": "rl-agent",
+ "temperature": [
+  1.0,
+  1.0,
+  1.0
+ ],
+ "temperature_by_options": {
+  "choice:11+": 1.0,
+  "choice:2": 1.391,
+  "choice:3-5": 1.0,
+  "choice:6-10": 3.555,
+  "noul:2": 1.0,
+  "score:3-5": 1.0
+ },
+ "training": {
+  "updates": 15987,
+  "epochs_completed": 4,
+  "hours": 4.97,
+  "world_size": 1,
+  "fine_tuned_from_checkpoint": false
+ },
+ "fine_tuned": {
+  "what": "PPO on skill_route conf-correct reward (encoder frozen)",
+  "task": "skill_route",
+  "init_from": "models/ft_skill5",
+  "ref_from": "models/ft_skill5",
+  "episodes_seen": 12000,
+  "lr": 1e-05,
+  "eps": 0.2,
+  "beta": 0.06,
+  "inner": 2,
+  "reward": {
+   "conf_correct": 3.0,
+   "hedge": -0.5,
+   "wrong": -2.0,
+   "floor": 0.7,
+   "version": "choice-v1"
+  },
+  "select": "val reward, in-dist val subset, argmax semantics",
+  "val_acc": 0.28,
+  "val_cov": 0.196,
+  "train_rows": 986,
+  "confidence_calibration": {
+   "method": "coverage-targeted: sharpest T meeting zero-wrong (guardrail) or acc>=0.85 at floor 0.7 on val rows; NLL grid replaced (adversarial to entropy confidence)",
+   "floor": 0.7,
+   "tag": "skill5_final"
+  }
+ }
+}

configs/ppo_cho2.rl_agent_config.json

@@ -0,0 +1,48 @@
+{
+ "encoder": "jhu-clsp/mmBERT-base",
+ "head_layers": 2,
+ "max_len": 1024,
+ "head_max_len": 256,
+ "max_prefixes": 6,
+ "act_costs": {
+  "escalate": 0.5
+ },
+ "cost_wrong_act": 3.0,
+ "amp_dtype": "bf16",
+ "model_name": "rl-agent",
+ "temperature": [
+  1.0,
+  1.0,
+  1.0
+ ],
+ "temperature_by_options": {
+  "choice:11+": 5.0,
+  "choice:2": 2.131,
+  "choice:3-5": 1.0,
+  "choice:6-10": 4.216,
+  "noul:2": 1.0,
+  "score:3-5": 1.0
+ },
+ "training": {
+  "updates": 15987,
+  "epochs_completed": 4,
+  "hours": 4.97,
+  "world_size": 1,
+  "fine_tuned_from_checkpoint": false
+ },
+ "fine_tuned": {
+  "what": "type_emb+head+scorer+act_head (encoder frozen)",
+  "train_rows": 1461,
+  "val_rows": 434,
+  "epochs": 10,
+  "lr": 3e-05,
+  "select_by": "acc",
+  "init_from": "source",
+  "train_file": "ft/train.jsonl (sessions disjoint from val/test)",
+  "confidence_calibration": {
+   "method": "coverage-targeted: sharpest T meeting zero-wrong (guardrail) or acc>=0.85 at floor 0.7 on val rows; NLL grid replaced (adversarial to entropy confidence)",
+   "floor": 0.7,
+   "tag": "cho2_final"
+  }
+ }
+}

laya-combined-f16.aimodel/main.hash

@@ -1 +1 @@
-�K��)5=�9~�[��Z�_��z�<�5� 
+�s���@�-���E|�`�h�q�o6�훳

laya-combined-f16.aimodel/main.mlirb

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:d94bfcf029353d16a939147e0b895bccf0a55af05fa103ac7aef943ca0358620
-size 944629276
+oid sha256:b573110e1c1bb1f2e440912db5cbd6457ce58160f8689471c76f36dced9bb30d
+size 945026529

laya-combined-f16.aimodel/metadata.json

@@ -1,5 +1,5 @@
 {
   "assetVersion" : "2.0",
   "producer" : "coreai-core 1.0.0b2",
-  "creationDate" : "20260927T184829Z"
+  "creationDate" : "20260928T201128Z"
 }

sha256manifest.json

@@ -4,27 +4,27 @@
   "bytes": 2215
  },
  "README.md": {
-  "sha256": "3eb0dfa5ab31a8e3da640f4c4f0e050317c2668db3bdcfdb7fa4d014094b71cb",
-  "bytes": 10226
+  "sha256": "cf3dae816cbe0a596df4a19d89d6bd8638c860a0f9836c7aee214e5002dc799d",
+  "bytes": 10747
  },
  "combined_provenance.json": {
-  "sha256": "1ceb68468121eb495015fc2ef073e9a8421c5331fd0b833c91fbe65dee0b7db0",
-  "bytes": 3564
+  "sha256": "ebf4b4280e4a2d89bb453125f6f0e83297f76e20957d7d849a8e7a98c2b6e55f",
+  "bytes": 4067
  },
  ".gitattributes": {
   "sha256": "49acd61f1b29a348e72ca8de4f0653322d2459882bf8ddc9a774570b66ab7360",
   "bytes": 509
  },
  "laya-combined-f16.aimodel/metadata.json": {
-  "sha256": "3b0afc4a5a61707d4ffff0f6ae9144b009c1862b70843517aff8623c8c064e52",
+  "sha256": "f3f87ddef489cd0041c0e268f991d127ab16fb0014119f06d5f16bbba4252388",
   "bytes": 105
  },
  "laya-combined-f16.aimodel/main.mlirb": {
-  "sha256": "d94bfcf029353d16a939147e0b895bccf0a55af05fa103ac7aef943ca0358620",
-  "bytes": 944629276
+  "sha256": "b573110e1c1bb1f2e440912db5cbd6457ce58160f8689471c76f36dced9bb30d",
+  "bytes": 945026529
  },
  "laya-combined-f16.aimodel/main.hash": {
-  "sha256": "1c24a3f3abfeb033f825b18212f65c71bc7493d5d24f8efbe0d44c19baef5fb9",
+  "sha256": "47528c61f91f226c1a8d7fce44195671036d17c8fbd54c625ec3c6f1a04072be",
   "bytes": 32
  },
  "configs/ft_mail8.rl_agent_config.json": {
@@ -47,18 +47,14 @@
   "sha256": "86e36a591780982b12cb484a2775aae30dec4db6c8783e51ffbad327aab47c6c",
   "bytes": 728
  },
- "configs/ppo_cho.rl_agent_config.json": {
-  "sha256": "0efbf6ad08338a1e1190343f4333a1bfafb80b52b2b4ea40ea0e55b5650f7df1",
-  "bytes": 894
- },
- "configs/ft_skill4_ppo.rl_agent_config.json": {
-  "sha256": "195cad99f9da575cf86769344a0151e980237db3f6550b7455c33b5eb125364b",
-  "bytes": 1305
- },
  "configs/ft_triage5.rl_agent_config.json": {
   "sha256": "8867e5e82d5366a098056906cd00c735a3d6d40f2cf1e014be6d9d4d1613b331",
   "bytes": 996
  },
+ "configs/ppo_cho2.rl_agent_config.json": {
+  "sha256": "82f759a45f82112be0f557189c16128066d6a94d321cbf1f69e6f29d3c2c9745",
+  "bytes": 1076
+ },
  "configs/ft_cmp2.rl_agent_config.json": {
   "sha256": "950d88e63309ebcb614587dbd433e3033fa176490f478bff32150b2cbbfb50c8",
   "bytes": 728
@@ -67,6 +63,10 @@
   "sha256": "25061739243b617ad88d1219ba6f8a9c86c5881ca28df024fa2d9b3b2fcc30c6",
   "bytes": 472
  },
+ "configs/ft_skill5_ppo.rl_agent_config.json": {
+  "sha256": "351dff6c463a868775f7395f002e5f8cf16d98386d5ef3eeb38a1928aa5155c9",
+  "bytes": 1325
+ },
  "configs/ft_sup3.rl_agent_config.json": {
   "sha256": "34a696eebff50074b420cbe523f45f5dcb7a932656159d3f41a424fbd31cdc8a",
   "bytes": 751