Text Classification
Laya
multilingual
English
Japanese
coreai
modernbert
apple-silicon
on-device
calibrated-decisions
Instructions to use mlboydaisuke/Laya-Multilingual-CoreAI with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Laya
How to use mlboydaisuke/Laya-Multilingual-CoreAI with Laya:
# No code snippets available yet for this library. # To use this model, check the repository files and the library's documentation. # Want to help? PRs adding snippets are welcome at: # https://github.com/huggingface/huggingface.js
- Notebooks
- Google Colab
- Kaggle
README: the iPhone 17 Pro rows (201/201, 47 ms per decision, the Neural Engine preference)
Browse files
README.md
CHANGED
|
@@ -31,8 +31,8 @@ A **typed decision model** that answers in one forward pass: give it a state (a
|
|
| 31 |
JSON record, a conversation) and a typed question β `choice` (pick one of named options), `score`
|
| 32 |
(an ordered scale) or `noul` (yes/no) β and it returns a probability for every option, no generation.
|
| 33 |
[`convaiinnovations/laya`](https://huggingface.co/convaiinnovations/laya), subfolder `multilingual/`
|
| 34 |
-
(Apache-2.0, revision `1c5edc17β¦`, laya 0.3.4), as a static `.aimodel` for macOS 27 and
|
| 35 |
-
|
| 36 |
decision model**: the other
|
| 37 |
decision models here are language models read out after a prefill; this one scores every option at its
|
| 38 |
own marker position in a single encoder call.
|
|
@@ -99,8 +99,24 @@ On the CPU, wfp16 and fp32 return bit-identical marker and act logits on all 402
|
|
| 99 |
weights in fp16 β exact, the checkpoint is F16 β and computes in fp32. On this Mac's GPU both stay within
|
| 100 |
1e-4 of the CPU run as well. **Use the GPU, and request it explicitly**: with the Neural Engine preference
|
| 101 |
the Mac returned the GPU's results bit for bit for fp32, but for wfp16 it returned different results that
|
| 102 |
-
changed from run to run (next section). **iPhone
|
| 103 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 104 |
|
| 105 |
The publisher's own model on the SemIf `authored144` fixture (144 three-option evidence / rule /
|
| 106 |
candidate questions it was not trained for), as `laya.load(...).predict` answers them: mean family
|
|
@@ -217,8 +233,8 @@ at the final norm (0.13 at S=256 in torch), the state that normalizes those larg
|
|
| 217 |
|
| 218 |
## License and limits
|
| 219 |
|
| 220 |
-
Apache-2.0 at the pinned upstream revision. Not tested:
|
| 221 |
-
Macs or OS builds, dynamic or batched shapes, windows other than 256 and 512, more than 20 options, the
|
| 222 |
fitted calibration's quality at S = 512, languages beyond the fixture's English, Japanese and mixed rows,
|
| 223 |
sustained thermals. The act probability is saturated at 1.0 on every fixture row, in the publisher's
|
| 224 |
model as here: it is carried through the graph, not evidence of when to escalate.
|
|
|
|
| 31 |
JSON record, a conversation) and a typed question β `choice` (pick one of named options), `score`
|
| 32 |
(an ordered scale) or `noul` (yes/no) β and it returns a probability for every option, no generation.
|
| 33 |
[`convaiinnovations/laya`](https://huggingface.co/convaiinnovations/laya), subfolder `multilingual/`
|
| 34 |
+
(Apache-2.0, revision `1c5edc17β¦`, laya 0.3.4), as a static `.aimodel` for macOS 27 and for the iPhone
|
| 35 |
+
17 Pro (run there 2026-09-23: the same 201 rows pass, 47 ms per decision on the GPU). It is the catalog's first **encoder-type
|
| 36 |
decision model**: the other
|
| 37 |
decision models here are language models read out after a prefill; this one scores every option at its
|
| 38 |
own marker position in a single encoder call.
|
|
|
|
| 99 |
weights in fp16 β exact, the checkpoint is F16 β and computes in fp32. On this Mac's GPU both stay within
|
| 100 |
1e-4 of the CPU run as well. **Use the GPU, and request it explicitly**: with the Neural Engine preference
|
| 101 |
the Mac returned the GPU's results bit for bit for fp32, but for wfp16 it returned different results that
|
| 102 |
+
changed from run to run (next section). **iPhone 17 Pro** (iOS 27.0 24A437, the `ios/wfp16-s256` JIT bundle,
|
| 103 |
+
2026-09-23, a headless harness over CoreAIKit's `EncoderDecider.decideRow` on the 201 fixture rows, the
|
| 104 |
+
fixture's own token ids, T = 1): on the GPU **201/201 rows within 1e-3, argmax 81/81, max |Ξp| 8.1e-6**,
|
| 105 |
+
act probability delta 0, no drift over three passes; CPU-only the same verdict at 9.4e-6. The kit's builder
|
| 106 |
+
renders every row identically on the phone (201/201 tokens and markers). One decision on the phone's GPU:
|
| 107 |
+
**53 ms median** (p90 57 ms, 30 warm-up calls, 603 timed calls) with the row's tokens given, 69β70 ms
|
| 108 |
+
through `TypedDecisions` with the state's tokens kept (the question tokenized on the phone, 5 ms median),
|
| 109 |
+
73 ms CPU-only; the bundle loads in 1.2β2.5 s, the process peaks at 357 MB (670 MB CPU-only). With a Neural
|
| 110 |
+
Engine preference the phone misses the bar the way the Mac does β 196/201 within 1e-3, argmax 80/81, max
|
| 111 |
+
|Ξp| 0.33 on five rows β at 82 ms and 1.4 GB, so the kit keeps refusing that preference for this bundle.
|
| 112 |
+
Thermal state during those runs: "fair" for the GPU rows, "serious" for the CPU-only and Neural Engine
|
| 113 |
+
rows; the GPU gate run again at "nominal" (2026-09-24) gave the same verdict at 54.5 ms median (p90 59.6),
|
| 114 |
+
so the row cost is not thermal-bound. Through the kit's own `decide-cli bench --repeat 3` on the cool phone
|
| 115 |
+
(a 109-token state, eight questions, the measure of the Mac's 11.5 ms): **47.1 ms per decision** with the
|
| 116 |
+
state shared, 46.4 from scratch, 390 / 458 ms for the state and its eight. [`gate-laya-multilingual-iphone-gpu.json`](https://github.com/john-rocky/coreai-model-zoo/blob/main/models/laya-multilingual/gate-laya-multilingual-iphone-gpu.json)
|
| 117 |
+
in the zoo repository is the first GPU run's record. The AOT `ios-h18p/wfp16-s256` bundle, run the same way on the cool phone
|
| 118 |
+
(2026-09-24): the same verdict (201/201, argmax 81/81, max |Ξp| 8.1e-6) at **51.0 ms** median (p90 54.4), first call
|
| 119 |
+
68.6 ms after a 1.7 s load, peak 357 MB β no faster to load or to run than the JIT bundle the catalog ships.
|
| 120 |
|
| 121 |
The publisher's own model on the SemIf `authored144` fixture (144 three-option evidence / rule /
|
| 122 |
candidate questions it was not trained for), as `laya.load(...).predict` answers them: mean family
|
|
|
|
| 233 |
|
| 234 |
## License and limits
|
| 235 |
|
| 236 |
+
Apache-2.0 at the pinned upstream revision. Not tested: the 512 window on the phone (the 256 window was, JIT
|
| 237 |
+
and AOT h18p, on one iPhone 17 Pro), other Macs or OS builds, dynamic or batched shapes, windows other than 256 and 512, more than 20 options, the
|
| 238 |
fitted calibration's quality at S = 512, languages beyond the fixture's English, Japanese and mixed rows,
|
| 239 |
sustained thermals. The act probability is saturated at 1.0 on every fixture row, in the publisher's
|
| 240 |
model as here: it is carried through the graph, not evidence of when to escalate.
|