Cogent-Small

Cogent-Small is the second smallest of the four Cogent models (Nano, Small, Base, Large). Cogent models answer typed multi-question decisions: you give the model a state (any text, or JSON) and a list of typed questions about it, and it returns a probability for every option of every question. The state is encoded once and shared by all the questions, and the models are built to run on a CPU.

• Parameters: 68.9M (backbone 68.1M; the reader head is under 1M), read from the weights.
• Default format: bf16, with an fp32 copy beside it.
• Inference code: the cogent package, https://github.com/glim-oss/cogent-infer. The model does not run with transformers alone.
• Paper: to appear; the link will be added here.
• Licence: GPL-3.0. The backbone derives from an Apache-2.0 encoder (see "Architecture").
• Other sizes: glim-oss/cogent-nano, glim-oss/cogent-base, glim-oss/cogent-large

What it returns

For each question, a probability for each of its options. The options keep your order and their probabilities sum to 1. There are three question types:

Type	Meaning	Options
choice	Pick one of several options.	Any ids, in any order.
score	Rate on an ordered scale.	The levels in order, lowest first.
noul	A yes/no statement: does it hold?	["false", "true"].

Usage

Install the inference package and download this repository into a folder:

git clone https://github.com/glim-oss/cogent-infer && cd cogent-infer && pip install .
hf download glim-oss/cogent-small --local-dir cogent-small

It needs Python 3.10 or later, torch, transformers>=5.22, safetensors and numpy.

from cogent import Cogent

model = Cogent.from_pretrained("cogent-small")      # dtype="auto", threads=None
probs = model.decide(
    state={"task": "Resolve the customer's billing complaint.", "last_action": "issued refund of 850 EUR"},
    questions=[
        {"key": "action", "type": "choice", "instructions": "What should happen to this trace?",
         "options": ["continue", "human_review", "stop"],
         "descriptions": {"continue": "Let the agent proceed.", "human_review": "Queue it for a human.",
                          "stop": "Halt the agent."}},
        {"key": "review", "type": "noul", "instructions": "This trace requires human review.",
         "options": ["false", "true"]},
    ],
)
# {"action": {"continue": ..., "human_review": ..., "stop": ...}, "review": {"false": ..., "true": ...}}

The request format

• state: a string is read as it is. Any other JSON value is serialised to text, and that text is what the model reads.
• key: the name of the question in the answer. Optional (q0, q1, ... by position); keys must be unique.
• type: choice, score or noul.
• instructions: the question, as text.
• options: the option ids, as strings, unique within a question.
• descriptions: optional, option id to the meaning of that option. Give them when you have them: an option's id and its description are all the model reads about it.

Limits, in tokens (longer inputs are truncated on the right): state 2,048 with [CLS] and [SEP], question 512, option 128. There is no limit on the number of questions or options. The package's README documents model.scores, the command line and the exact text the model reads.

Architecture

Cogent-Small is a ModernBERT-style encoder (19 layers, hidden size 512, 8 attention heads) run under a tree attention mask, with a small reader head. Its backbone descends from the public Ettin reranker encoder of this size, cross-encoder/ettin-reranker-68m-v1; the paper describes the training method. One request is one token sequence: the state, then each question followed by its options. The state reads only itself, so it is encoded once and shared by all the questions of the request (a request longer than 1,024 tokens is split into several sequences, each holding the whole state). A question reads the state and itself. An option reads the state, its own question and itself, and never another option or another question. The reader head pools each option's tokens into one vector, lets it read the pooled state through two layers of cross-attention, and gives one logit per option; a softmax over the options of each question gives the probabilities.

Because no option reads anything outside its own branch, a question asked alone gets the same answer as the same question asked together with others, in any order. This is a property of the architecture, not of the weights, and it is exact in exact arithmetic. In fp32 the two runs differ only by float rounding: the largest total-variation distance we measured across the family is 2.5e-6. In bf16 the rounding is coarser; see "Files and formats".

Files and formats

Folder	Format	Seed	model.safetensors	Folder total
top level (default)	bf16	0	136.3 MB	143.1 MB
fp32/	fp32	0	272.6 MB	279.4 MB

Every model folder holds config.json and model.safetensors (the backbone), cogent.json and head.safetensors (the reader head, float32 in every format) and the tokeniser files. MANIFEST.json lists the sha256 and size of every file.

• bf16 stores and runs the backbone in bfloat16. It is the default because it is faster and about half the size.
• bf16 needs a CPU with avx512_bf16. Its fidelity and speed were measured on such a CPU. On other CPUs its answers differ slightly and it is slow: use the fp32 folder there. dtype="auto" loads the fp32 copy for you on a CPU without native bfloat16, with a warning, and cogent.cpu_has_bf16() tells you what your CPU has.
• fp32 and bf16 are two formats of the same model and do not give identical numbers. Every result on this card names its format.

bf16 against its own fp32, measured on the 2,000 typed-decisions test decisions (fitted; second read, see "Results"):

	Seed 0 (the default file)	Seeds 0, 1, 2
Decisions whose top option differs between the formats, of 2,000	8	8, 4, 7
Mean total-variation distance between the formats	.0022	.0022, .0023, .0025
Accuracy, bf16 and fp32	.7800 and .7800	pooled difference -.0002 [-.0017, +.0013]
Brier, bf16 and fp32	.05202 and .05209	pooled difference -.00004 [-.00008, +.00001]
Correct decisions, bf16 minus fp32, of 2,000	0	0, +2, -3
Largest distance between a question asked alone and asked jointly, bf16, 2,000 test questions	.0119	.0119, .0096, .0121

No difference between the formats is shown: both pooled intervals include zero. These rows compare two formats of one model. They are not a claim that either format is the better model on the benchmark.

Results

Read this first:

• Fitted. Every Cogent row on typed-decisions is a fitted (in-distribution) result; see "Training data". Laya Typed-Decisions is the one fitted rival we measured ourselves, at its defaults, on the same machine. No row here is zero-shot, and no zero-shot model is ranked beside these rows.
• Second read. These are the final models. The same test set and the same JevBench suite had been read once before, for earlier versions of these models (the paper's first evaluation), and development continued after those scores were seen. The numbers below are from the second evaluation.
• Three seeds. Each tier was trained three times. The headline is the mean of the three, with the lowest and highest seed beside it. Seed 0 is the file this repository loads by default. That was fixed by convention before any of these scores was read, so no seed was picked on test. It happens to be the highest of the three seeds on test accuracy here, so read the tier by its mean.
• Format. All rows of Cogent-Small are for the bf16 files, measured on a CPU with avx512_bf16.
• Code. All results were produced with the authors' own evaluation code on these weights. The public cogent package is a standalone port of its inference path.
• Intervals are 95% paired bootstrap intervals over cases (20,000 resamples). "After Holm" means after Holm's correction across the 36 comparisons planned for this evaluation, 9 for each tier.

typed-decisions test set

400 cases, 2,000 decisions (LocalLLaMA/typed-decisions, test). Accuracy is agreement with the top gold option. Brier is summed over options and KL is from the gold distribution; lower is better for Brier, KL, ECE and dSCE (the debiased squared calibration error).

Row	Track	Accuracy	Brier	KL	ECE	dSCE
Cogent-Small, mean of 3 seeds (lowest to highest seed)	fitted	.7782 (.7760 to .7800)	.0531 (.0520 to .0543)	.0964 (.0944 to .0985)	.1614 (.1592 to .1631)	.0289 (.0284 to .0295)
Cogent-Small, seed 0 (the default file)	fitted	.7800	.0520	.0944	.1631	.0295
Laya Typed-Decisions, measured by us	fitted rival	.7660	.0615	.1170	.2132	.0466

Cogent-Small minus Laya Typed-Decisions, mean of 3 seeds, paired by decision:

• Accuracy: +.0122 [-.0042, +.0287]: no difference shown.
• Brier: -.0083 [-.0118, -.0047]: in Cogent-Small's favour, and it holds after Holm.
• dSCE: -.0177 [-.0245, -.0111]: in Cogent-Small's favour, and it holds after Holm.

Cogent-Small is better than Laya Typed-Decisions on Brier and on calibration (dSCE) after Holm. Its accuracy is not shown to differ from Laya's.

How to read the accuracy. As the benchmark's own card describes, its gold labels come from a model: each is the mean of three distributions sampled from a labelling model. A score on it measures agreement with that labeller, not correctness. The benchmark's card reports that a fresh sample of the labeller itself scores accuracy .735 against gold built from its other samples (published, not measured by us). Every accuracy in the table, Laya's included, is above that, so accuracy here partly measures agreement with the labeller's quirks. Brier and KL carry the comparison better. ECE is computed against the top gold option and is not a clean quality score on soft labels.

JevBench-mini

JevBench (JevBench/jevbench) tests whether a model's probabilities hang together: 50 metamorphic relations in five dimensions, 24 tests each on JevBench-mini, over 240 synthetic English cases. Its overall coherence score is the mean of the five dimensions. Accuracy, Brier and log loss are read on the 1,938 decisions of the unchanged requests that have known gold. Brier here is summed over options against a one-hot gold, a different scale from the test set's Brier above. These rows are not entries on the JevBench board and are not ranked against it.

Row	Accuracy	Brier (summed over options)	Log loss	Coherence, overall
Cogent-Small, mean of 3 seeds (lowest to highest seed)	.5941 (.5614 to .6176)	.4938 (.4878 to .5034)	.8024 (.7960 to .8096)	.7217 (.7202 to .7233)
Cogent-Small, seed 0 (the default file)	.6176	.4878	.7960	.7215 [.7078, .7359]
Laya Typed-Decisions, measured by us	.7118	.4155	.7238	.6483 [.6248, .6709]
Uniform reference (equal probability for every option)	.3982	.6018	.9761	.8941

For a Cogent seed and for Laya, the bracket after the coherence score is the benchmark package's own interval.

Cogent-Small minus Laya Typed-Decisions, mean of 3 seeds, paired by decision:

• Accuracy: -.1177 [-.1391, -.0961]: in Laya's favour, and it holds after Holm.
• Brier: +.0782 [+.0658, +.0905]: in Laya's favour, and it holds after Holm.

Cogent-Small trails Laya Typed-Decisions on JevBench-mini, clearly: .1177 lower accuracy, and a worse Brier, both after Holm. The same is true of the other two smaller tiers; only Cogent-Large ties Laya there. The result on the typed-decisions test set does not carry over to JevBench's requests for this tier.

Coherence, and why it must sit beside accuracy. By dimension, mean of 3 seeds: BAT 1.000, CHO 1.000, LOG .812, MEA .169, REP .627 (BAT: batch independence, CHO: choice-set conditioning, LOG: logical order, MEA: probability measure, REP: representation). Laya reads BAT 1.000, CHO .792, LOG .750, MEA .147, REP .553. Every seed of Cogent-Small has a coherence interval above Laya's (lowest lower end .7069 against Laya's upper end .6709); that comparison is unpaired and no claim rests on it. The uniform reference scores .8941, above every model here, at chance accuracy: coherence alone ranks nothing.

The four sizes side by side

For orientation. Every row is fitted and from the second evaluation, mean of 3 seeds, in the tier's default format. No comparison between tiers was tested.

Model	Parameters	Default format	Test accuracy	Test Brier	JevBench-mini accuracy	JevBench-mini coherence	S2 p95 at 4 threads, ms
Cogent-Nano	32.6M	bf16	.7482	.0666	.4943	.7171	55.2
Cogent-Small	68.9M	bf16	.7782	.0531	.5941	.7217	147.9
Cogent-Base	149.9M	fp32	.7770	.0508	.6192	.7342	621.2
Cogent-Large	395.8M	bf16	.7960	.0472	.7124	.7107	788.8
Laya Typed-Decisions, measured by us (fitted rival)	421M as published		.7660	.0615	.7118	.6483	5,783.0

The last column is the five-question shape (S2). On the long label list (S3: one question, 77 options) Cogent-Large is slower than Laya, at 1.20 to 1.46 times its p95, and Cogent-Base is at parity (.933 to 1.005); see "CPU latency".

Against the earlier version of this tier

The paper's first evaluation reports an earlier version of this tier. Final version minus earlier version, mean of 3 seeds, paired seed by seed:

• typed-decisions test: accuracy +.0275 [+.0157, +.0395] (better after Holm); Brier -.0113 [-.0144, -.0084] (better after Holm). Earlier version's means: accuracy .7507, Brier .0644.
• JevBench-mini: accuracy +.0569 [+.0407, +.0727] (better after Holm); Brier -.0503 [-.0603, -.0405] (better after Holm). Earlier version's means: accuracy .5372, Brier .5441.
• dSCE on the test set against the earlier version: +.0127 [+.0089, +.0166]. This comparison is outside the Holm family and is not corrected.

The final Cogent-Small is better than the earlier version on both benchmarks, on accuracy and on Brier, after Holm. Its test-set dSCE is higher than the earlier version's.

CPU latency

Milliseconds per request, for the default file of this repository (seed 0, bf16). Machine: a laptop with an AMD Ryzen AI 9 HX 370 (it has avx512_bf16), Linux, torch 2.15.0 (CPU build), transformers 5.22.1. Batch 1, pinned cores, a fresh process for every row, 5 warm-up requests, no result cache, warm page cache, on mains power. Each shape is a fixed set of 40 requests (8 for S4, so its p95 says little more than its p50).

Request shape	Threads	Cogent-Small p50, ms	Cogent-Small p95, ms	Laya p95, ms	Cogent-Small p95 as a share of Laya's
S1: short message, 3 questions	1	40.4	40.6	827.9	.049
S1: short message, 3 questions	4	16.0	20.2	332.5	.061
S2: 5 questions	1	340.2	493.6	15,137.4	.033
S2: 5 questions	4	104.7	147.9	5,783.0	.026
S3: 1 question, 77 options	1	671.1	695.7	2,451.8	.284
S3: 1 question, 77 options	4	203.9	211.9	922.3	.230
S4: long state, 5 questions	1	3,135.2	3,243.4	37,641.8	.086
S4: long state, 5 questions	4	975.5	1,003.0	14,357.2	.070

• On these rows Cogent-Small's p95 is .026 to .284 of Laya's. Its weakest shape is the long label list (S3): .284 and .230 at 1 and 4 threads.
• The family is not faster than Laya everywhere. On S3, Cogent-Large is slower than Laya (1.46 and 1.20 times its p95 at 1 and 4 threads) and Cogent-Base is at parity (1.005 and .933).
• S3 and S4 are identical requests but not the same work. Laya's default label budget of 256 tokens leaves each of S3's 77 labels 3 to 4 tokens, so it reads 328 tokens on the first request where Cogent reads 926. Laya's default input limit of 1,024 tokens truncates the S4 state, of which Cogent reads 2,048 tokens. We did not run Laya with larger budgets.
• Laya's rows are reused, not retimed. Laya 0.4.22, convaiinnovations/laya-typed-decisions at its defaults, was timed on this laptop six days before these rows, under the same protocol. The check that allows the reuse is a control row of ours reading within 5% of its earlier value. It failed on its first reading (+25.7% at 1 thread) and passed when that row was timed again (+1.4%). The rows of Cogent-Small were timed again in that same second pass, each once, because another job had overlapped the first pass; the second readings are the ones reported. The paper gives the full account.
• Serving peak memory on S2 at 4 threads: 629 MB, against 2,304 MB for Laya.
• One timed seed. Only seed 0 was timed. Timing used the authors' own inference code on these weights; the public cogent package was not timed. The 4-thread rows ran at the laptop's thermal ceiling.

Intended use

• Typed decisions over a piece of text or a JSON record, on a CPU: routing and triage (choice), ordered ratings (score) and yes/no checks (noul), several at once on the same state.
• Workloads where many questions are asked about one state, since the state is encoded once.
• A scoring component whose output a person or a downstream rule reviews. The model returns probabilities, not explanations.

Not intended for: decisions that affect people without human review; inputs in languages other than English; free-text generation; tasks far from the typed-decisions format without your own evaluation.

Limitations

• Fitted, in-distribution. The typed-decisions results above are fitted results. They say how the model does on that benchmark's kind of request, not how it does on yours.
• No zero-shot claim. Nothing on this card measures zero-shot behaviour, and no zero-shot Cogent row exists. On JevBench-mini, the one other benchmark we read, Cogent-Small trails Laya clearly.
• The benchmark's labels come from a model. High scores on typed-decisions mean agreement with that labeller, quirks included. They do not show that a decision is right.
• Second read. The test set had been read once before for earlier versions, so these numbers are not a first, untouched read.
• One rival measured. Only Laya Typed-Decisions was run by us. Published rows of other models are in the paper, in separate tables, as published and not measured. This card claims no state of the art.
• One timed seed, one machine. Latency and memory were measured for seed 0 on one laptop CPU. Every result on this card is a CPU result; running on a GPU was not checked.
• bf16 portability. bf16's fidelity and speed were measured on a CPU with avx512_bf16. On other CPUs its answers differ slightly and it is slow: use fp32 there. bf16 and fp32 never give identical numbers.
• Probabilities are scores against a soft gold. Check calibration on your own data before you put a threshold on them.
• Truncation. A state longer than 2,048 tokens, a question longer than 512 or an option longer than 128 is cut on the right without an error.
• Settings. The training settings were tuned for one configuration and reused for the others.

Training data

Cogent's training data is proprietary and is not described. It is in-distribution for the typed-decisions benchmark, so every Cogent result on that benchmark is a fitted (in-distribution) result.

Citation

@misc{cogent2026,
  title  = {Cogent: CPU-First Typed Decision Models with Option Independence by Construction},
  author = {Esposito, Carlo},
  year   = {2026},
  note   = {Preprint; link to follow}
}