Text Classification
Laya
multilingual
English
Japanese
coreai
modernbert
apple-silicon
on-device
calibrated-decisions
Instructions to use mlboydaisuke/Laya-Multilingual-CoreAI with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Laya
How to use mlboydaisuke/Laya-Multilingual-CoreAI with Laya:
# No code snippets available yet for this library. # To use this model, check the repository files and the library's documentation. # Want to help? PRs adding snippets are welcome at: # https://github.com/huggingface/huggingface.js
- Notebooks
- Google Colab
- Kaggle
JevBench public 231 accuracy and latency (Mac, 2026-09-24)
Browse filesSame subsection as coreai-model-zoo a708c2e (models/laya-multilingual/README.md).
README.md
CHANGED
|
@@ -123,6 +123,23 @@ candidate questions it was not trained for), as `laya.load(...).predict` answers
|
|
| 123 |
balanced accuracy **0.6114**, accuracy 0.5903 (the same at T = 1 and with the fitted calibration, and at
|
| 124 |
both windows). That is the reference a port's decisions are compared with, not a claim about the port.
|
| 125 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 126 |
## Numerics gate
|
| 127 |
|
| 128 |
The oracle is the publisher's package itself (`laya.load(<pinned snapshot>, subfolder="multilingual")`,
|
|
|
|
| 123 |
balanced accuracy **0.6114**, accuracy 0.5903 (the same at T = 1 and with the fitted calibration, and at
|
| 124 |
both windows). That is the reference a port's decisions are compared with, not a claim about the port.
|
| 125 |
|
| 126 |
+
### JevBench public 231 (Mac, 2026-09-24)
|
| 127 |
+
|
| 128 |
+
| easy 48 | standard 72 | hard 111 | ECE hard | p50 | p95 | hard max |
|
| 129 |
+
|---:|---:|---:|---:|---:|---:|---:|
|
| 130 |
+
| 0.896 | 0.403 | 0.342 | 0.273 | 0.02 s | 0.20 s | 0.3 s |
|
| 131 |
+
|
| 132 |
+
The benchmark's own harness ([fstandhartinger/jevbench](https://github.com/fstandhartinger/jevbench)
|
| 133 |
+
`2fa63fa`, v1.4.0, `typesafe` adapter) ran the 231 public items against coreai-kit `adbc755`
|
| 134 |
+
`decide-cli serve` with the `macos/wfp16-s256` bundle, one question per request. The bundle's input
|
| 135 |
+
window is 256 tokens: 95 of the 111 hard items were truncated to it (hard states are up to 3,677 tokens
|
| 136 |
+
long), and no easy or standard item was (the longest is 107 tokens). Accuracy per tier and the hard
|
| 137 |
+
tier's ECE are JevBench's own scoring (argmax of the returned probabilities); p50 and p95 are per-request
|
| 138 |
+
latency over all 231 requests, hard max the maximum over the hard tier. Latency was measured without an
|
| 139 |
+
exclusive GPU window (contended), with this model's server running alone. JevBench's published scores
|
| 140 |
+
(Intelligence and the rest) are chance-corrected over 534 items, sealed ones included, and are not
|
| 141 |
+
comparable to these accuracies.
|
| 142 |
+
|
| 143 |
## Numerics gate
|
| 144 |
|
| 145 |
The oracle is the publisher's package itself (`laya.load(<pinned snapshot>, subfolder="multilingual")`,
|