---
license: apache-2.0
base_model: convaiinnovations/laya
language:
  - en
library_name: pytorch
pipeline_tag: text-classification
tags:
  - typed-decisions
  - classification
  - routing
  - modernbert
  - coreml
  - on-device
---

# Snap 1

**Fast, local, typed decisions.** Give Snap a state (text or JSON) and a few typed questions —
yes/no, pick-one, or a level — and it answers all of them in a single forward pass, with
probabilities. It never generates text, so there is nothing to parse and nothing to hallucinate.
It runs on your machine: no API key, no network, nothing leaves the device.

`Otron/snap-1` · 421M parameters · English · up to 4,096 tokens per pass

## Quick start

```python
# pip install "laya==0.3.4" torch transformers safetensors huggingface_hub
# then download snap.py from this repo
from snap import Snap

snap = Snap("Otron/snap-1")
snap.decide(
    state={"message": "my card got charged twice for the same order"},
    questions={
        "topic": {"type": "choice", "instructions": "What is the message about?",
                  "criteria": {"billing": "Payments, charges, refunds",
                               "shipping": "Delivery and tracking",
                               "other": "Anything else"}},
        "urgent": {"type": "noul", "instructions": "The customer needs a reply today"},
        "mood": {"type": "score", "instructions": "How upset is the customer?",
                 "criteria": ["calm", "annoyed", "angry", "furious"]},
    },
)
# {"topic":  {"choice": "billing", "confidence": 0.997, "probabilities": {...}},
#  "urgent": {"noul": 0.42, "confidence": 0.58},
#  "mood":   {"score": 2, "confidence": 0.41, "probabilities": {"0": 0.05, "1": 0.38, "2": 0.41, "3": 0.15}}}
```

Question types:

| type | answers | `criteria` |
|---|---|---|
| `noul` | probability the statement in `instructions` is true | — |
| `choice` | one key | dict of key → description (or a list of keys) |
| `score` | one ordered level (0-based) | list of level descriptions, lowest first |

Descriptions matter: Snap reads every option's text, so options it has never seen work as well as
familiar ones. Option lists can be long (hundreds of options in one question).

### Core ML (macOS 14+)

`coreml/Snap.mlpackage` is the same model converted for Core ML (fp16 weights, 819 MB). It
takes one sequence (`input_ids`, `tok_type`) and returns a score per token; `coreml/snap.json`
holds the token ids, limits and confidence temperatures needed to build the input and read the
answers, and `coreml/tokenizer.json` is the tokenizer. The input format is exactly the one
`snap.py` builds (`Snap._block` / `decide`). Run it on `.cpuAndGPU`: the Neural Engine
recompiles for every new input length.

Apps that download the model on demand can use `coreml/manifest.json`: every Core ML file with
its size and SHA-256, at URLs pinned to the commit they were published in.

## Speed

Measured on an Apple M5 (32 GB), one decision at a time, model already loaded:

| | PyTorch (MPS, fp16) | Core ML (GPU, fp16) |
|---|---|---|
| short decision — one sentence, 3 questions (109 tokens) | 25 ms | **16 ms** |
| long decision — a spoken request routed over ~100 options plus context (1.6k tokens median, up to 3.7k) | 214 ms median · 545 ms p90 | 191 ms median · 557 ms p90 |
| load | 18.9 s | 5.8 s first time (compiles), then faster |

Cost grows with the total length of the state and the options, not with the number of questions:
every question about the same state shares one pass when they fit together.

## Results

Accuracy against each dataset's own labels, on 500 examples per dataset from the **official**
evaluation split (never trained on). Questions are phrased as plain typed decisions with the
labels' names and descriptions as options; no task-specific heads or prompts.

| dataset (split) | majority class | Laya (base) | **Snap 1** |
|---|---|---|---|
| Banking77 (test) — 77 intents | 0.028 | 0.464 | **0.812** |
| CLINC150 plus (test) — 151 intents incl. out-of-scope | 0.196 | 0.514 | **0.838** |
| ContractNLI (test) — whole NDAs | 0.488 | 0.488 | **0.712** |
| MNLI (validation matched) | 0.356 | **0.872** | 0.870 |
| ANLI (test r1–r3) | 0.366 | 0.506 | **0.520** |
| BoolQ (validation) — accuracy · AUROC | 0.626 | 0.850 · 0.913 | **0.860 · 0.933** |
| Civil Comments (test) — AUROC | 0.5 | **0.944** | 0.923 |

Laya is scored on the same examples in its own input format, with its 512-token limit and with
Snap's 4,096; the better of the two is shown (with its own 192-token option budget it cannot fit
Banking77's or CLINC150's option lists at all). Snap's gains are where the fine-tuning aimed —
many-option routing and long documents; it is level on reasoning-style NLI and slightly behind the
base model at ranking toxicity.

Civil Comments is reported as AUROC: its label (≥50% of raters said toxic) is narrower than a plain
"is this rude or disrespectful" question, so raw accuracy mostly measures the cut-off.

**Unseen tools.** On held-out synthetic decisions for 300 invented tools (spoken requests routed to
each tool's own actions, none seen in training), Snap picks the reference action 69.8% of the time,
with expected calibration error 0.025.

**Calibration.** Confidences are temperature-calibrated per question type and option count; across
all held-out decisions the expected calibration error is 0.026.

## JevBench (public half)

[JevBench](https://github.com/fstandhartinger/jevbench) is an independent benchmark for typed-decision
models. We ran Snap 1 on its 231 public items with the benchmark's own harness and scorer (commit
`609a664`). The Laya and Jev columns are the per-item results JevBench publishes for the same items
(release v1.3.0). This is a self-run of the public half, not an official board entry: the board also
scores sealed items, calibration, speed and cost. Snap 1 was not trained on any JevBench item.

| tier (items) | Laya (base) | **Snap 1** | Jev 1.13 |
|---|---|---|---|
| easy (48) | 0.958 | **1.000** | 1.000 |
| standard (72) | 0.694 | **0.806** | 0.986 |
| hard (111) | 0.351 | **0.342** | 0.730 |
| **all (231)** | 0.584 | **0.623** | 0.866 |
| median latency per decision | — | **24 ms** on-device (M5) | 665 ms over the network |

The hard tier (long insurance policies, contracts and rule sets with multi-step reasoning, written
by frontier models) is where a 421M encoder falls behind; none of those items exceed Snap's 4,096-
token window. Jev's latency is as JevBench measured it, over the network; Snap's runs locally.

## How it was made

- **Base:** [Laya](https://huggingface.co/convaiinnovations/laya) by ConvAI Innovations, a typed-
  decision model on [ModernBERT-large](https://huggingface.co/answerdotai/ModernBERT-large).
- **Training:** full-parameter fine-tuning (all weights) on the data below.
- **Data:** training splits of Banking77, CLINC150, MNLI, ANLI, BoolQ, Civil Comments, ContractNLI
  and QuALITY, plus ~30k synthetic decisions for 300 invented tools generated with GPT-6 Luna, and a
  small amount of private data that is not released. Examples were labelled with TypeSafe's Jev,
  combined with each dataset's own labels where they exist.
- **Layout:** Snap packs all questions about one state into a single sequence (each question's
  options are `[MASK]` markers that the model scores), instead of one sequence per question. This
  is what makes many questions cost one pass. `extra.safetensors` holds the extra token-type
  embedding this layout uses; `snap.py` implements it.

## Limitations

- **Snap learned from model-generated labels and inherits their mistakes.** It is weakest on
  hard multi-step reasoning (ANLI; JevBench's hard tier) and less sure of itself on tools it has
  never seen.
- **English only**, and 4,096 tokens per pass (state + questions); longer states are truncated.
- **Text in the state can steer the answer** (prompt injection). Treat decisions on untrusted input
  as advice, and keep a human or a hard rule in the loop for anything consequential.
- **Not for safety-critical, medical, legal or financial decisions.**

## License and attribution

Apache 2.0 (see `LICENSE` and `NOTICE`). Built on Laya (Apache 2.0) and ModernBERT (Apache 2.0).
Trained on public datasets under their own licenses: ANLI is CC BY-NC 4.0 and BoolQ CC BY-SA 3.0;
the datasets themselves are not redistributed here.

Not affiliated with or endorsed by TypeSafe or ConvAI Innovations.