Cogent-Large

Cogent-Large is the largest of the four Cogent models (Nano, Small, Base, Large). Cogent models answer typed multi-question decisions: you give the model a state (any text, or JSON) and a list of typed questions about it, and it returns a probability for every option of every question. The state is encoded once and shared by all the questions, and the models are built to run on a CPU.

• Parameters: 395.8M (backbone 394.8M; the reader head is under 1M), read from the weights.
• Default format: bf16, with an fp32 copy beside it.
• Inference code: the cogent package, https://github.com/glim-oss/cogent-infer. The model does not run with transformers alone.
• Paper: to appear; the link will be added here.
• Licence: GPL-3.0. The backbone derives from an Apache-2.0 encoder (see "Architecture").
• Other sizes: glim-oss/cogent-nano, glim-oss/cogent-small, glim-oss/cogent-base

What it returns

For each question, a probability for each of its options. The options keep your order and their probabilities sum to 1. There are three question types:

Type	Meaning	Options
choice	Pick one of several options.	Any ids, in any order.
score	Rate on an ordered scale.	The levels in order, lowest first.
noul	A yes/no statement: does it hold?	["false", "true"].

Usage

Install the inference package and download this repository into a folder:

git clone https://github.com/glim-oss/cogent-infer && cd cogent-infer && pip install .
hf download glim-oss/cogent-large --local-dir cogent-large

That downloads all six copies in this repository (three seeds, each in bf16 and fp32), 6.7 GB. For the default model alone, 0.8 GB, add --exclude "seed*/*" --exclude "fp32/*". Keep fp32/ if your CPU has no avx512_bf16, and keep the seed folders for the 3-seed ensemble.

It needs Python 3.10 or later, torch, transformers>=5.22, safetensors and numpy.

from cogent import Cogent

model = Cogent.from_pretrained("cogent-large")      # dtype="auto", threads=None
probs = model.decide(
    state={"task": "Resolve the customer's billing complaint.", "last_action": "issued refund of 850 EUR"},
    questions=[
        {"key": "action", "type": "choice", "instructions": "What should happen to this trace?",
         "options": ["continue", "human_review", "stop"],
         "descriptions": {"continue": "Let the agent proceed.", "human_review": "Queue it for a human.",
                          "stop": "Halt the agent."}},
        {"key": "review", "type": "noul", "instructions": "This trace requires human review.",
         "options": ["false", "true"]},
    ],
)
# {"action": {"continue": ..., "human_review": ..., "stop": ...}, "review": {"false": ..., "true": ...}}

The request format

• state: a string is read as it is. Any other JSON value is serialised to text, and that text is what the model reads.
• key: the name of the question in the answer. Optional (q0, q1, ... by position); keys must be unique.
• type: choice, score or noul.
• instructions: the question, as text.
• options: the option ids, as strings, unique within a question.
• descriptions: optional, option id to the meaning of that option. Give them when you have them: an option's id and its description are all the model reads about it.

Limits, in tokens (longer inputs are truncated on the right): state 2,048 with [CLS] and [SEP], question 512, option 128. There is no limit on the number of questions or options. The package's README documents model.scores, the command line and the exact text the model reads.

The 3-seed ensemble

This repository holds three independently trained copies of Cogent-Large (seed 0 at the top level, seed1/, seed2/). The ensemble answers with the mean of their probabilities, option by option:

from cogent import Cogent

members = [Cogent.from_pretrained(p) for p in ("cogent-large", "cogent-large/seed1", "cogent-large/seed2")]
ensemble = Cogent.ensemble(members)
probs = ensemble.decide(state, questions)

It runs three models, so it costs three times the time and the memory of one. On the typed-decisions test set it has the lowest Brier and KL of the configurations we measured. Its accuracy, .8000, equals that of seed 0 alone in bf16 and is one decision below the fp32 file of seed 0 (.8005), so it is not the highest accuracy we measured. See "Results" for what that does and does not show.

Architecture

Cogent-Large is a ModernBERT-style encoder (28 layers, hidden size 1024, 16 attention heads) run under a tree attention mask, with a small reader head. Its backbone descends from the public Ettin reranker encoder of this size, cross-encoder/ettin-reranker-400m-v1; the paper describes the training method. One request is one token sequence: the state, then each question followed by its options. The state reads only itself, so it is encoded once and shared by all the questions of the request (a request longer than 1,024 tokens is split into several sequences, each holding the whole state). A question reads the state and itself. An option reads the state, its own question and itself, and never another option or another question. The reader head pools each option's tokens into one vector, lets it read the pooled state through two layers of cross-attention, and gives one logit per option; a softmax over the options of each question gives the probabilities.

Because no option reads anything outside its own branch, a question asked alone gets the same answer as the same question asked together with others, in any order. This is a property of the architecture, not of the weights, and it is exact in exact arithmetic. In fp32 the two runs differ only by float rounding: the largest total-variation distance we measured across the family is 2.5e-6. In bf16 the rounding is coarser; see "Files and formats".

Files and formats

Folder	Format	Seed	model.safetensors	Folder total
top level (default)	bf16	0	789.6 MB	797.1 MB
seed1/	bf16	1	789.6 MB	797.1 MB
seed2/	bf16	2	789.6 MB	797.1 MB
fp32/	fp32	0	1,579.1 MB	1,586.7 MB
fp32/seed1/	fp32	1	1,579.1 MB	1,586.7 MB
fp32/seed2/	fp32	2	1,579.1 MB	1,586.7 MB

Every model folder holds config.json and model.safetensors (the backbone), cogent.json and head.safetensors (the reader head, float32 in every format) and the tokeniser files. MANIFEST.json lists the sha256 and size of every file.

• bf16 stores and runs the backbone in bfloat16. It is the default because it is faster and about half the size.
• bf16 needs a CPU with avx512_bf16. Its fidelity and speed were measured on such a CPU. On other CPUs its answers differ slightly and it is slow: use the fp32 folder there. dtype="auto" loads the fp32 copy for you on a CPU without native bfloat16, with a warning, and cogent.cpu_has_bf16() tells you what your CPU has.
• fp32 and bf16 are two formats of the same model and do not give identical numbers. Every result on this card names its format.

bf16 against its own fp32, measured on the 2,000 typed-decisions test decisions (fitted; second read, see "Results"):

	Seed 0 (the default file)	Seeds 0, 1, 2
Decisions whose top option differs between the formats, of 2,000	13	13, 7, 9
Mean total-variation distance between the formats	.0024	.0024, .0024, .0024
Accuracy, bf16 and fp32	.8000 and .8005	pooled difference -.0002 [-.0018, +.0015]
Brier, bf16 and fp32	.04645 and .04639	pooled difference +.00005 [+7e-7, +.00011]
Correct decisions, bf16 minus fp32, of 2,000	-1	-1, -2, +2
Largest distance between a question asked alone and asked jointly, bf16, 2,000 test questions	.0145	.0145, .0374, .0148

On accuracy no difference between the formats is shown. The pooled Brier rise is about .00005 and its interval just excludes zero. These rows compare two formats of one model. They are not a claim that either format is the better model on the benchmark.

Alone against joint in bf16, said plainly. We hold bf16 to a bound of .025 on the largest distance between a question asked alone and the same question asked jointly. Seed 0, the default file, reads .0145 on the test inputs and seed 2 reads .0148. Seed 1 reads .0374: it exceeds the bound on 1 of 2,000 questions. Seed 1 is a member of the 3-seed ensemble. If you need alone and joint answers to agree to float rounding, use the fp32 folders.

Results

Read this first:

• Fitted. Every Cogent row on typed-decisions is a fitted (in-distribution) result; see "Training data". Laya Typed-Decisions is the one fitted rival we measured ourselves, at its defaults, on the same machine. No row here is zero-shot, and no zero-shot model is ranked beside these rows.
• Second read. These are the final models. The same test set and the same JevBench suite had been read once before, for earlier versions of these models (the paper's first evaluation), and development continued after those scores were seen. The numbers below are from the second evaluation.
• Three seeds. Each tier was trained three times. The headline is the mean of the three, with the lowest and highest seed beside it. Seed 0 is the file this repository loads by default. That was fixed by convention before any of these scores was read, so no seed was picked on test. It happens to be the highest of the three seeds on test accuracy here, so read the tier by its mean.
• Format. All rows of Cogent-Large are for the bf16 files, measured on a CPU with avx512_bf16.
• Code. All results were produced with the authors' own evaluation code on these weights. The public cogent package is a standalone port of its inference path.
• Intervals are 95% paired bootstrap intervals over cases (20,000 resamples). "After Holm" means after Holm's correction across the 36 comparisons planned for this evaluation, 9 for each tier.

typed-decisions test set

400 cases, 2,000 decisions (LocalLLaMA/typed-decisions, test). Accuracy is agreement with the top gold option. Brier is summed over options and KL is from the gold distribution; lower is better for Brier, KL, ECE and dSCE (the debiased squared calibration error).

Row	Track	Accuracy	Brier	KL	ECE	dSCE
Cogent-Large, mean of 3 seeds (lowest to highest seed)	fitted	.7960 (.7915 to .8000)	.0472 (.0465 to .0485)	.0868 (.0856 to .0886)	.1537 (.1476 to .1612)	.0262 (.0241 to .0286)
Cogent-Large, seed 0 (the default file)	fitted	.8000	.0465	.0856	.1612	.0286
Cogent-Large, 3-seed ensemble	fitted	.8000	.0436	.0799	.1606	.0292
Laya Typed-Decisions, measured by us	fitted rival	.7660	.0615	.1170	.2132	.0466

Cogent-Large minus Laya Typed-Decisions, mean of 3 seeds, paired by decision:

• Accuracy: +.0300 [+.0145, +.0453]: in Cogent-Large's favour, and it holds after Holm.
• Brier: -.0142 [-.0180, -.0104]: in Cogent-Large's favour, and it holds after Holm.
• dSCE: -.0203 [-.0269, -.0140]: in Cogent-Large's favour, and it holds after Holm.

Cogent-Large is better than Laya Typed-Decisions on accuracy, Brier and calibration (dSCE) after Holm.

The 3-seed ensemble. It is one extra measured row, in no Holm family, so no claim of beating a rival rests on it.

• Against Laya Typed-Decisions: accuracy +.0340 [+.0175, +.0500], Brier -.0178 [-.0216, -.0140].
• Against its own three members, pooled: accuracy +.0040 [-.0023, +.0103], so an accuracy gain over a single model is not shown; Brier -.0036 [-.0039, -.0033] and KL -.0069 [-.0075, -.0063], both better.
• Against seed 0, the default file: the same accuracy, .8000 and .8000. The fp32 file of seed 0 reads .8005 (1,601 of 2,000), one decision more than the ensemble. An independent audit of this reading computed the paired difference from seed 0 alone: accuracy +.0000 [-.0105, +.0105], Brier -.0028 [-.0046, -.0011].
• Calibration is slightly worse than its members': dSCE +.0030 [+.0011, +.0050], ECE +.0070 [+.0010, +.0134].
• The ensemble is therefore the configuration with the lowest Brier and KL that we measured, and not the one with the highest accuracy. What it reliably buys is a lower Brier, at three times the cost.

How to read the accuracy. As the benchmark's own card describes, its gold labels come from a model: each is the mean of three distributions sampled from a labelling model. A score on it measures agreement with that labeller, not correctness. The benchmark's card reports that a fresh sample of the labeller itself scores accuracy .735 against gold built from its other samples (published, not measured by us). Every accuracy in the table, Laya's included, is above that, so accuracy here partly measures agreement with the labeller's quirks. Brier and KL carry the comparison better. ECE is computed against the top gold option and is not a clean quality score on soft labels.

JevBench-mini

JevBench (JevBench/jevbench) tests whether a model's probabilities hang together: 50 metamorphic relations in five dimensions, 24 tests each on JevBench-mini, over 240 synthetic English cases. Its overall coherence score is the mean of the five dimensions. Accuracy, Brier and log loss are read on the 1,938 decisions of the unchanged requests that have known gold. Brier here is summed over options against a one-hot gold, a different scale from the test set's Brier above. These rows are not entries on the JevBench board and are not ranked against it.

Row	Accuracy	Brier (summed over options)	Log loss	Coherence, overall
Cogent-Large, mean of 3 seeds (lowest to highest seed)	.7124 (.7002 to .7281)	.4114 (.4026 to .4195)	.7061 (.6998 to .7130)	.7107 (.6991 to .7179)
Cogent-Large, seed 0 (the default file)	.7002	.4195	.7130	.7179 [.7057, .7305]
Laya Typed-Decisions, measured by us	.7118	.4155	.7238	.6483 [.6248, .6709]
Uniform reference (equal probability for every option)	.3982	.6018	.9761	.8941

For a Cogent seed and for Laya, the bracket after the coherence score is the benchmark package's own interval.

Cogent-Large minus Laya Typed-Decisions, mean of 3 seeds, paired by decision:

• Accuracy: +.0006 [-.0192, +.0205]: no difference shown.
• Brier: -.0042 [-.0157, +.0072]: no difference shown.

Cogent-Large ties Laya Typed-Decisions on JevBench-mini. .7124 against .7118 is a tie, not a win. The three smaller Cogent tiers trail Laya on this benchmark, on accuracy and on Brier, after Holm.

Coherence, and why it must sit beside accuracy. By dimension, mean of 3 seeds: BAT 1.000, CHO 1.000, LOG .753, MEA .181, REP .620 (BAT: batch independence, CHO: choice-set conditioning, LOG: logical order, MEA: probability measure, REP: representation). Laya reads BAT 1.000, CHO .792, LOG .750, MEA .147, REP .553. Every seed of Cogent-Large has a coherence interval above Laya's (lowest lower end .6863 against Laya's upper end .6709); that comparison is unpaired and no claim rests on it. The uniform reference scores .8941, above every model here, at chance accuracy: coherence alone ranks nothing. Cogent-Large is the most accurate of the four tiers on this benchmark and the least coherent by this score.

The four sizes side by side

For orientation. Every row is fitted and from the second evaluation, mean of 3 seeds, in the tier's default format. No comparison between tiers was tested.

Model	Parameters	Default format	Test accuracy	Test Brier	JevBench-mini accuracy	JevBench-mini coherence	S2 p95 at 4 threads, ms
Cogent-Nano	32.6M	bf16	.7482	.0666	.4943	.7171	55.2
Cogent-Small	68.9M	bf16	.7782	.0531	.5941	.7217	147.9
Cogent-Base	149.9M	fp32	.7770	.0508	.6192	.7342	621.2
Cogent-Large	395.8M	bf16	.7960	.0472	.7124	.7107	788.8
Laya Typed-Decisions, measured by us (fitted rival)	421M as published		.7660	.0615	.7118	.6483	5,783.0

The last column is the five-question shape (S2). On the long label list (S3: one question, 77 options) Cogent-Large is slower than Laya, at 1.20 to 1.46 times its p95, and Cogent-Base is at parity (.933 to 1.005); see "CPU latency".

Against the earlier version of this tier

The paper's first evaluation reports an earlier version of this tier. Final version minus earlier version, mean of 3 seeds, paired seed by seed:

• typed-decisions test: accuracy +.0108 [+.0032, +.0188] (raw interval excludes zero, better, not after Holm); Brier -.0021 [-.0039, -.0004] (raw interval excludes zero, better, not after Holm). Earlier version's means: accuracy .7852, Brier .0493.
• JevBench-mini: accuracy -.0136 [-.0235, -.0037] (raw interval excludes zero, worse, not after Holm); Brier +.0009 [-.0033, +.0050] (no difference shown). Earlier version's means: accuracy .7260, Brier .4104.
• dSCE on the test set against the earlier version: +.0027 [+.0002, +.0053]. This comparison is outside the Holm family and is not corrected.

No difference from the earlier Cogent-Large holds after Holm, on either benchmark. On JevBench-mini the final version reads lower accuracy, and the raw interval excludes zero: stated as measured.

CPU latency

Milliseconds per request, for the default file of this repository (seed 0, bf16). Machine: a laptop with an AMD Ryzen AI 9 HX 370 (it has avx512_bf16), Linux, torch 2.15.0 (CPU build), transformers 5.22.1. Batch 1, pinned cores, a fresh process for every row, 5 warm-up requests, no result cache, warm page cache, on mains power. Each shape is a fixed set of 40 requests (8 for S4, so its p95 says little more than its p50).

Request shape	Threads	Cogent-Large p50, ms	Cogent-Large p95, ms	Laya p95, ms	Cogent-Large p95 as a share of Laya's
S1: short message, 3 questions	1	262.7	264.5	827.9	.320
S1: short message, 3 questions	4	86.7	114.8	332.5	.345
S2: 5 questions	1	1,917.1	2,728.2	15,137.4	.180
S2: 5 questions	4	579.3	788.8	5,783.0	.136
S3: 1 question, 77 options	1	3,441.2	3,590.1	2,451.8	1.464
S3: 1 question, 77 options	4	1,053.7	1,108.2	922.3	1.201
S4: long state, 5 questions	1	12,735.5	13,074.8	37,641.8	.347
S4: long state, 5 questions	4	3,929.8	4,037.9	14,357.2	.281

• On the long label list (S3) Cogent-Large is slower than Laya: 1.46 times Laya's p95 at 1 thread and 1.20 times at 4 threads (p50: 1.52 and 1.26 times).
• On S1, S2 and S4 its p95 is .136 to .347 of Laya's.
• S3 and S4 are identical requests but not the same work. Laya's default label budget of 256 tokens leaves each of S3's 77 labels 3 to 4 tokens, so it reads 328 tokens on the first request where Cogent reads 926. Laya's default input limit of 1,024 tokens truncates the S4 state, of which Cogent reads 2,048 tokens. We did not run Laya with larger budgets.
• Laya's rows are reused, not retimed. Laya 0.4.22, convaiinnovations/laya-typed-decisions at its defaults, was timed on this laptop six days before these rows, under the same protocol. The check that allows the reuse is a control row of ours reading within 5% of its earlier value. It failed on its first reading (+25.7% at 1 thread) and passed when that row was timed again (+1.4%). The paper gives the full account.
• Serving peak memory on S2 at 4 threads: 1,256 MB, against 2,304 MB for Laya.
• One timed seed. Only seed 0 was timed. Timing used the authors' own inference code on these weights; the public cogent package was not timed. The 4-thread rows ran at the laptop's thermal ceiling.

The 3-seed ensemble was not timed. It runs three models: expect three times these numbers.

Intended use

• Typed decisions over a piece of text or a JSON record, on a CPU: routing and triage (choice), ordered ratings (score) and yes/no checks (noul), several at once on the same state.
• Workloads where many questions are asked about one state, since the state is encoded once.
• A scoring component whose output a person or a downstream rule reviews. The model returns probabilities, not explanations.

Not intended for: decisions that affect people without human review; inputs in languages other than English; free-text generation; tasks far from the typed-decisions format without your own evaluation.

Limitations

• Fitted, in-distribution. The typed-decisions results above are fitted results. They say how the model does on that benchmark's kind of request, not how it does on yours.
• No zero-shot claim. Nothing on this card measures zero-shot behaviour, and no zero-shot Cogent row exists. On JevBench-mini, the one other benchmark we read, Cogent-Large ties Laya and the smaller tiers trail it.
• The benchmark's labels come from a model. High scores on typed-decisions mean agreement with that labeller, quirks included. They do not show that a decision is right.
• Second read. The test set had been read once before for earlier versions, so these numbers are not a first, untouched read.
• One rival measured. Only Laya Typed-Decisions was run by us. Published rows of other models are in the paper, in separate tables, as published and not measured. This card claims no state of the art.
• One timed seed, one machine. Latency and memory were measured for seed 0 on one laptop CPU. Every result on this card is a CPU result; running on a GPU was not checked.
• bf16 portability. bf16's fidelity and speed were measured on a CPU with avx512_bf16. On other CPUs its answers differ slightly and it is slow: use fp32 there. bf16 and fp32 never give identical numbers.
• The ensemble. Its accuracy gain over one model is not shown, its calibration is slightly worse than its members', it was not timed, and one of its members (seed 1) exceeds our alone-against-joint bound in bf16 on one test question.
• Probabilities are scores against a soft gold. Check calibration on your own data before you put a threshold on them.
• Truncation. A state longer than 2,048 tokens, a question longer than 512 or an option longer than 128 is cut on the right without an error.
• Settings. The training settings were tuned for one configuration and reused for the others.

Training data

Cogent's training data is proprietary and is not described. It is in-distribution for the typed-decisions benchmark, so every Cogent result on that benchmark is a fitted (in-distribution) result.

Citation

@misc{cogent2026,
  title  = {Cogent: CPU-First Typed Decision Models with Option Independence by Construction},
  author = {Esposito, Carlo},
  year   = {2026},
  note   = {Preprint; link to follow}
}