Horopter 0.4B (horopter-0.4b-20261104)

An encoder-only decision model built on jhu-clsp/ettin-encoder-400m. Give it a state (a document, a record or a JSON object) and typed questions about it (yes/no, choose one, score). It answers them in one pass, and returns three things with each answer:

• evidence spans: whole-word spans in the state that support or refute each option;
• relation: whether the state bears on the question at all, and with what stance;
• abstention: a calibrated probability that the answer is wrong, with a reason.

The state is encoded once, without seeing any question, and cached. Questions never change each other's answers, and many questions about one state cost about one encoding.

Code, wire format and training recipe: https://github.com/jmccardle/horopter. Larger sibling: horopter-1B, stronger on decisions.

What's new in this release (beta-3). Rule-application data now includes the non-decisive outcomes: "more information needed", "this rule does not apply" and default answers. The previous release never gave the first two on real rule text. ShARC 4-way accuracy rises 48 → 79. This release also bundles an extract adapter for typed field extraction.

Use

import horopter
m = horopter.load("jmccardle/horopter-0.4B")
out = m.answer("Refunds within 30 days. The lamp was bought 12 days ago.",
               {"q": {"type": "noul", "instructions": "Is a refund allowed?"}}, evidence=True)

Server: python -m horopter.server --model jmccardle/horopter-0.4B --port 8420. Model id horopter is the router, which picks a family adapter per question. horopter-<adapter> (e.g. horopter-base, horopter-check2) forces one adapter. --prefix horopter-0.4b renames them.

The previous release stays available as a revision: horopter.load("jmccardle/horopter-0.4B@horopter-0.4b-20261020").

Files

• model.safetensors: the trunk with its answer, relation and evidence heads.
• adapters/:
  • router;
  • family LoRAs: typed, nli, relation, classification, knowledge, clause;
  • evidence (deciding-rule evidence);
  • check2 (single-condition checks);
  • extract (field extraction, used by fields requests; new in this release);
  • router_q (optional, new): a router for the question layout, --route_layout question --router router_q. It reads the request's own cached state instead of a separate router state. It routes the bundled families as well as router does, is 2–3 points worse on task descriptions it wasn't trained on, and is no faster on a desktop GPU, so router stays the default. It may pay off on edge devices, where the extra state pass costs more.
• abstain_head.json: the calibrated abstention head.
• config.json and the tokenizer (from Ettin).

There is no quantized file for this release yet; it will follow once evaluated. The Q8 file of horopter-0.4b-20261020 remains at that revision.

Results

Horopter rows use the router + adapters, in one pass with no generation. The reference decoders were scored on the same items.

	Horopter 0.4B (this release)	Horopter 1B (this release)	horopter-0.4b-20261020	clef-flash (9B)	decider-2b	openjev v5 (4B)
LLM-AggreFact, balanced accuracy (macro, 11 datasets)	75.0	75.2	74.8	77.2	72.6	74.2
4-way relation (support / refute / undetermined / unrelated), macro-F1	77.0	78.9	76.6	75.3	72.3	71.0
irrelevant-state detection, AUROC	0.949	0.955	0.948	0.961	0.935	0.941
JevBench original (72 decisions)	63.9	79.2	66.7	97.2	88.9	98.6
JevBench hard (111 multi-step policy decisions)	39.6	40.5	36.0	64.0	57.7	62.2
JF100 (100 paired counterfactual and formal decisions)	42.0	54.0	52.0	71.0	63.0	60.0

• Grounding. Horopter is level with 4–9B decoders at telling whether a document supports a claim, and whether a text bears on a question at all. These scores are within half a point of the previous release.
• Multi-step decisions. It trails the decoders by 20 or more points. JF100 is 10 points below the previous 0.4B release (see Limits).

Real rule text (dev sets, accuracy). Qwen3-30B-A3B answers each question directly, with thinking off.

set	Horopter 0.4B	Horopter 0.4B + check2	Horopter 1B	horopter-0.4b-20261020	Qwen3-30B-A3B
ShARC, 4-way (yes / no / irrelevant / more info)	78.6	78.9	82.3	48.1	64.8
ShARC, recall of "irrelevant" / "more info"	95.7 / 77.0	96.4 / 75.4	100 / 82.6	0.0 / 0.0	55.8 / 32.4
ShARC, yes/no items	91.5	91.5	93.1	71.2	72.2
ProofWriter OWA depth 0–5, true/false items	62.3	62.3	61.7	58.4	75.5
ConditionalQA, yes/no (95)	69.5	68.4	74.7	71.6	78.9

ShARC's training split is in this release's data, so its dev split is held out but not a new distribution. ProofWriter and ConditionalQA are not trained on.

Generated policies. Trunk, no adapters. Two held-out sets:

• the release-1 set: 1,868 rule sets and cases at depth 1–5, every outcome determined;
• the beta-3 set: 1,177 cases that include the non-decisive outcomes.

One domain (robot SOPs) is held out of training entirely.

Horopter 0.4B	one-shot, JSON policy	one-shot, prose policy	interpreter (compiled per-rule checks via check2)
release-1 set, all	82.7	96.6	99.5
release-1 set, held-out domain	58.8	88.5	99.1
beta-3 set (with non-decisive outcomes), prose policy		97.0
beta-3 set, held-out domain, prose policy		90.0

The evidence span lands on the deciding rule 96–99% of the time.

Field extraction (fields requests, extract adapter), on generated documents held out from training:

• 87.2% of typed fields extracted exactly (phone numbers, dates, amounts, emails, ...).
• When the asked-for role is missing but a value of the same type is present for another role (the sender's phone number, but not the recipient's), it returns no value 84% of the time on free-text kinds and 79% on typed kinds.

Training data

• A committee-distilled typed-decision mixture (NLI, relation, classification, knowledge and clause questions, with evidence spans).
• Rule application:
  • 12,002 generated rule sets with cases (211k decisions), labelled by executing the rules. Of these, 3,082 end in "more information needed", 1,012 in "does not apply" and 2,228 in a default;
  • the ShARC training split, twice (71k decisions);
  • 1,620 single-condition check sets;
  • 392 counterfactual minimal pairs;
  • 3,353 deciding-rule evidence targets.
• Labels come from human annotations, a committee of open decoder models (Gemma-family models among them), and TypeSafe's Jev API.
• Some sources are licensed non-commercial or research-only. The per-source list, with its permissions column, is in the recipe.
• LLM-AggreFact and the other benchmark sets were not trained on.

Limits

• Multi-step policy decisions trail 2B+ decoders by about 25 points in one pass (the 1B by about 20). The interpreter path closes the gap on generated policies, but only partly on real policy text.
• JF100 fell 52 → 42 and ConditionalQA 71.6 → 69.5 against the previous 0.4B release. The same data at 1B did not repeat either drop (JF100 rose 49 → 54), and both sets are small (100 and 95 items). Use the 1B where these counterfactual and conditional decisions matter.
• ProofWriter "unknown". At 0.4B the model still rarely answers "unknown" on ProofWriter (48 of 1,800 items).
• Field extraction of free-text values (file names, versions, handles) is weaker than for typed values. About half of present free-text values come back as no value.
• English only. States up to 2,048 tokens.
• Not for safety-critical use without a human in the loop.

Licence

• Weights: CC BY-NC-SA 4.0. Research and non-commercial use only; share adaptations under the same terms.
• Code: MIT.
• Base model: Ettin is MIT, © its authors.

A permissively licensed variant, trained only on commercially usable sources, is in training.

The training data includes outputs of TypeSafe's Jev API. TypeSafe's terms do not permit using them to imitate their model or to build a competing service.

Versions

• horopter-0.4b-20261104 (this revision): beta-3. Non-decisive rule outcomes, extract adapter.
• horopter-0.4b-20261020: first release.