Horopter 1B (horopter-1b-20261104)

An encoder-only decision model built on jhu-clsp/ettin-encoder-1b. Give it a state (a document, a record or a JSON object) and typed questions about it (yes/no, choose one, score). It answers them in one pass, and returns three things with each answer:

• evidence spans: whole-word spans in the state that support or refute each option;
• relation: whether the state bears on the question at all, and with what stance;
• abstention: a calibrated probability that the answer is wrong, with a reason.

The state is encoded once, without seeing any question, and cached. Questions never change each other's answers, and many questions about one state cost about one encoding.

Code, wire format and training recipe: https://github.com/jmccardle/horopter. Smaller sibling: horopter-0.4B.

What's new in this release (beta-3). Rule-application data now includes the non-decisive outcomes: "more information needed", "this rule does not apply" and default answers. The previous release almost never gave these on real rule text. ShARC 4-way accuracy rises 54 → 82. This release also bundles an extract adapter for typed field extraction.

Use

import horopter
m = horopter.load("jmccardle/horopter-1B")
out = m.answer("Refunds within 30 days. The lamp was bought 12 days ago.",
               {"q": {"type": "noul", "instructions": "Is a refund allowed?"}}, evidence=True)

Server: python -m horopter.server --model jmccardle/horopter-1B --port 8420. Model id horopter is the router, which picks a family adapter per question. horopter-<adapter> (e.g. horopter-base, horopter-check2) forces one adapter. --prefix horopter-1b renames them.

The previous release stays available as a revision: horopter.load("jmccardle/horopter-1B@horopter-1b-20261020").

Files

• model.safetensors: the trunk with its answer, relation and evidence heads.
• adapters/:
  • router;
  • family LoRAs: typed, nli, relation, classification, knowledge, clause;
  • evidence (deciding-rule evidence);
  • check2 (single-condition checks);
  • extract (field extraction, used by fields requests; new in this release).
• abstain_head.json: the calibrated abstention head.
• config.json and the tokenizer (from Ettin).

There are no quantized files for this release yet; they will follow once evaluated. The Q8 and Q4 files of horopter-1b-20261020 remain at that revision.

Results

Horopter rows use the router + adapters, in one pass with no generation. The reference decoders were scored on the same items.

	Horopter 1B (this release)	Horopter 0.4B (this release)	horopter-1b-20261020	clef-flash (9B)	decider-2b	openjev v5 (4B)
LLM-AggreFact, balanced accuracy (macro, 11 datasets)	75.2	75.0	75.7	77.2	72.6	74.2
4-way relation (support / refute / undetermined / unrelated), macro-F1	78.9	77.0	78.6	75.3	72.3	71.0
irrelevant-state detection, AUROC	0.955	0.949	0.956	0.961	0.935	0.941
JevBench original (72 decisions)	79.2	63.9	80.6	97.2	88.9	98.6
JevBench hard (111 multi-step policy decisions)	40.5	39.6	40.5	64.0	57.7	62.2
JF100 (100 paired counterfactual and formal decisions)	54.0	42.0	49.0	71.0	63.0	60.0

• Grounding. Horopter is level with 4–9B decoders at telling whether a document supports a claim, and whether a text bears on a question at all. These scores are within a point of the previous release.
• Multi-step decisions. It trails the decoders by 20 or more points. JF100 is up 5 on the previous release and JevBench original down 1 item, both within item-level noise on sets this small.

Real rule text (dev sets, accuracy). Qwen3-30B-A3B answers each question directly, with thinking off.

set	Horopter 1B	Horopter 1B + check2	Horopter 0.4B	horopter-1b-20261020	Qwen3-30B-A3B
ShARC, 4-way (yes / no / irrelevant / more info)	82.3	82.2	78.6	54.3	64.8
ShARC, recall of "irrelevant" / "more info"	100 / 82.6	100 / 81.5	95.7 / 77.0	36.2 / 0.5	55.8 / 32.4
ShARC, yes/no items	93.1	93.1	91.5	74.9	72.2
ProofWriter OWA depth 0–5, true/false items	61.7	61.4	62.3	62.7	75.5
ConditionalQA, yes/no (95)	74.7	72.6	69.5	73.7	78.9

ShARC's training split is in this release's data, so its dev split is held out but not a new distribution. ProofWriter and ConditionalQA are not trained on.

Generated policies. Trunk, no adapters. Two held-out sets:

• the release-1 set: 1,868 rule sets and cases at depth 1–5, every outcome determined;
• the beta-3 set: 1,177 cases that include the non-decisive outcomes.

One domain (robot SOPs) is held out of training entirely.

Horopter 1B	one-shot, JSON policy	one-shot, prose policy	interpreter (compiled per-rule checks via check2)
release-1 set, all	87.2	99.6	99.8
release-1 set, held-out domain	68.2	98.9	100.0
beta-3 set (with non-decisive outcomes), prose policy		99.3
beta-3 set, held-out domain, prose policy		97.7

The evidence span lands on the deciding rule 99.7–99.9% of the time.

Field extraction (fields requests, extract adapter), on generated documents held out from training:

• 87.6% of typed fields extracted exactly (phone numbers, dates, amounts, emails, ...).
• When the asked-for role is missing but a value of the same type is present for another role (the sender's phone number, but not the recipient's), the 1B returns no value 96% of the time on free-text kinds and 78% on typed kinds. On typed kinds it returns the other role's value 17% of the time (55% for the earlier extraction adapter).

Training data

• A committee-distilled typed-decision mixture (NLI, relation, classification, knowledge and clause questions, with evidence spans).
• Rule application:
  • 12,002 generated rule sets with cases (211k decisions), labelled by executing the rules. Of these, 3,082 end in "more information needed", 1,012 in "does not apply" and 2,228 in a default;
  • the ShARC training split, twice (71k decisions);
  • 1,620 single-condition check sets;
  • 392 counterfactual minimal pairs;
  • 3,353 deciding-rule evidence targets.
• Labels come from human annotations, a committee of open decoder models (Gemma-family models among them), and TypeSafe's Jev API.
• Some sources are licensed non-commercial or research-only. The per-source list, with its permissions column, is in the recipe.
• LLM-AggreFact and the other benchmark sets were not trained on.

Limits

• Multi-step policy decisions trail 2B+ decoders by about 20 points in one pass. The interpreter path closes the gap on generated policies, but only partly on real policy text.
• ProofWriter "unknown". The model now says "unknown" more often, including where the answer is determined. On ProofWriter OWA true/false items, the 3-way answer falls 58.4 → 53.4; the true-vs-false readout is flat.
• Field extraction of free-text values (file names, versions, handles) is weaker than for typed values. About half of present free-text values come back as no value.
• English only. States up to 2,048 tokens.
• Not for safety-critical use without a human in the loop.

Licence

• Weights: CC BY-NC-SA 4.0. Research and non-commercial use only; share adaptations under the same terms.
• Code: MIT.
• Base model: Ettin is MIT, © its authors.

A permissively licensed variant, trained only on commercially usable sources, is in training.

The training data includes outputs of TypeSafe's Jev API. TypeSafe's terms do not permit using them to imitate their model or to build a competing service.

Versions

• horopter-1b-20261104 (this revision): beta-3. Non-decisive rule outcomes, extract adapter.
• horopter-1b-20261020: first release.