| --- |
| title: IDU |
| emoji: 🧠 |
| colorFrom: purple |
| colorTo: gray |
| sdk: gradio |
| sdk_version: 5.49.1 |
| app_file: app.py |
| pinned: false |
| license: apache-2.0 |
| short_description: Typed decisions (choice/score/noul) on open weights |
| --- |
| |
| # IDU |
|
|
| **JEV** shifted one letter back — the same shift that turns **IBM** into **HAL**. |
|
|
| A typed-decision service in Jev's shape: give it a `state` and typed `questions`, get typed |
| answers with probabilities and confidence. No text generation, so nothing to parse and |
| nothing to hallucinate. |
|
|
| | type | question | answer | |
| |---|---|---| |
| | **choice** | which of these options? | the option, a probability per option, confidence | |
| | **score** | where on this rubric? | a position along your levels (can land between them), distribution | |
| | **noul** | is this true? | one number: the probability that it is | |
|
|
| ## Backend |
|
|
| [`convaiinnovations/laya`](https://huggingface.co/convaiinnovations/laya) — 421M |
| ModernBERT-large with a decision head, trained by RL against strictly proper scoring rules |
| (RLCD). Apache 2.0. |
|
|
| Why this one: |
|
|
| - **It already is the Jev shape.** `state` + typed questions in, typed decisions out. No |
| adapter, no tool-calling workaround, no trigger vocabulary. |
| - **Options are judged independently.** Each option is scored at its own `[MASK]` token and |
| then softmaxed over that question's options. That is what Jev documents ("each level is |
| judged on its own against the state"), and it removes the polarity problem a tool-calling |
| model has: a `noul` question is two markers, not two competing armed tools. |
| - **One forward pass** answers every question in a call. |
| - **PyTorch**, so it fits ZeroGPU natively — none of the JAX dtype/preallocation work it took |
| to get a JAX model running on this runtime. |
|
|
| ## Endpoints |
|
|
| - `POST /systemone` (Gradio API name `systemone`) — the Jev shape. |
| - `/triage` — four-question support triage in one call, with a confidence-gated action line. |
| - `/compare` — head-to-head against the tuned Needle 3 adapter on `dkappe/needle3-gpu`. |
| - `/health` — device, temperature in force, warm-up result. |
|
|
| ## Honest status |
|
|
| **The base checkpoints are near chance zero-shot** on their own typed-decisions benchmark: |
| 0.362 against a 0.318 random and a **0.461 majority-class baseline**. The card's headline |
| 0.766 belongs to a checkpoint fine-tuned on that benchmark's own training split, and it says |
| so itself: *"Laya is a fast base to specialise, not a zero-shot decision engine."* |
|
|
| **The probabilities ship already calibrated.** The model card reports mean ECE 0.466 falling |
| to 0.081 after fitting one temperature per (question type, option count) — and that fitting |
| is baked into the checkpoint: `rl_agent_config.json` carries `temperature [1.6369, 1.2514, |
| 1.9834]` for choice/score/noul plus per-option-count buckets. So the defaults here *are* the |
| calibrated path. The temperature controls use **0 = keep the checkpoint's value** as their |
| sentinel; setting a positive number overrides deliberately. |
|
|
| **`score` is the weakest primitive** by the card's own numbers (SST-5 0.372). |
|
|
| **High-cardinality `choice` degrades** because options share a fixed token budget: 0.425 on |
| Banking77 (77 options) vs Jev's 0.870. Fine at our 3–4 options; raise `head_max_len` or use a |
| coarse-to-fine split well before 50. |
|
|
| **English only** at the repo root. `laya-multilingual` (mmBERT-base) covers 100+ languages |
| and is ~2.2x faster. |
|
|
| ## Implementation notes worth keeping |
|
|
| - **`spaces` must be imported before `torch`.** The spaces library patches torch's CUDA |
| initialisation so a GPU can be attached per call; import torch first and every decorated |
| call dies inside `worker_init` with `RuntimeError: No CUDA GPUs are available`. Nothing |
| that pulls in torch — gradio, transformers, laya, safetensors — may appear above that |
| guard block. Taken from the model's own demo Space, which documents the same trap. |
| - **Assets load at module scope**, before the UI is reachable, because network access inside |
| a `@spaces.GPU` function is restricted. Warm-up runs at startup so the first click is not |
| paying model construction. |
| - **One decorated entry point** for every tab, so the start-up warm-up covers all of them |
| instead of each tab paying its own GPU attach. |
|
|
| Apache 2.0 — weights and SDK both. |
|
|
