KnowLine-4B-Gen5.5

KnowLine-4B-Gen5 with a new way of serving it: the same weights, and a front end that reads short requests twice before it answers.

Inference guide · 中文说明 · weights Apache-2.0 · previous version: KnowLine-4B-Gen5 · also: KnowLine-35B-A3B-Gen1.5, KnowLine-26B-A4B-Gen2.5

We raise the minor version when only inference changes and the major version (Gen) when the weights change. Every weight, config and tokenizer file of Gen5.5 is byte-identical to Gen5.

Highlights:

• Same weights, read twice. For a text-only request of up to 3,000 tokens, the front end writes the whole input a second time before the model answers. Nothing was trained; see What changed in Gen5.5.
• Decision Index 0.3 public suite: 66.75 (self-run, fresh full run), up from 65.67 for Gen5 with the same weights. The gains are mostly in reasoning and inference over long inputs (knowledge and reasoning 45.0 → 47.3, language 66.8 → 68.2); tools and agents are unchanged (87.4) and arts and taste slightly lower (61.2 → 61.0). See Comparison.
• Private scores: the board's full score comes from the maintainers' private run. Gen5 is listed at 53.68. Gen5.5 has the same weights as Gen5; its private scores will be produced by the maintainers' run.
• Other public benchmarks: C-Eval 80.5 (Gen5 on the same files: 79.2) and JevBench / JevBench-hard 85.3 / 70.3 (83.1 / 66.7).
• Calibration: the temperature folded into the weights is unchanged (T = 1.50). Computed with the board's method, ECE is about 0.035-0.046 (Gen5: 0.037-0.049) and the Brier score about 0.27-0.30 (Gen5: 0.28-0.31); see Calibration.
• Cost: about 1.56 times the prompt tokens of Gen5 on the Decision Index suite. One request at a time on one H20: 42.1 ms median and 83 ms mean, against 31.6 ms and 58.8 ms for Gen5 (our measurement, not the board's reference hardware); see INFERENCE.md.
• Images: unchanged. Requests with images are sent exactly as in Gen5, so Gen5's image results apply; see Images.

KnowLine is an independent model. It is not affiliated with, endorsed by, or derived from TypeSafe or Jev.

What it does

You send a state (text or a chat) and up to 64 typed questions: yes/no, choose one of k, or score on a rubric. The model answers every question with a probability distribution over its options:

• one forward pass per question, with no text generation;
• the output is the probability of each option's label token;
• existing Jev clients only need to change the base URL, and users of earlier versions only need to change the model name, because the interface is unchanged.

Base model	Qwen/Qwen3.5-4B (Apache-2.0)
Weights	byte-identical to KnowLine-4B-Gen5: a LoRA SFT model (rank 32, alpha 64, language model only), merged into bf16 weights, with a readout temperature T = 1.50, fitted on held-out data, folded into the final norm weight
Serving	serve_knowline.sh: SGLang with FP8 at load, as for Gen5; knowline_server.py reads short requests twice (on by default; --no-read-twice gives the Gen5 prompts). We ran every evaluation with one server on one 96 GB H20.
Languages	English, Simplified Chinese, Traditional Chinese

Quickstart

pip install "sglang==0.5.21" "transformers==5.12.1" requests
bash serve_knowline.sh PelaAI/KnowLine-4B-Gen5.5 0 8080     # SGLang (FP8 at load) on :9080 + /v1/systemone on :8080
curl -s http://127.0.0.1:8080/v1/systemone -H 'Content-Type: application/json' -d '{
  "model": "m",
  "state": "Customer: my order arrived broken, I want my money back.",
  "questions": {"refund": {"type": "noul", "instructions": "Should the agent offer a refund?"},
                "tone": {"type": "choice", "instructions": "Customer tone?",
                         "criteria": {"angry": "Angry", "neutral": "Neutral", "happy": "Happy"}}}}'

• Front end: knowline_server.py with knowline_layout.py next to it; together they depend only on transformers and requests.
• Full settings: the exact settings used for our Decision Index run are in INFERENCE.md.

What changed in Gen5.5

1. Read twice. Gen5 saw each request once: the state, then the instruction, the question and its options. Gen5.5 writes all of that a second time, introduced by the line "Read the same input again before answering:", and then reads the answer from the label token as before. The rule is mechanical: it applies to every text-only request of at most 3,000 tokens (94.6% of the Decision Index requests); longer requests and requests with images are sent as in Gen5.
2. A likely reason. With causal attention, the first copy of a long state is read before the model knows the question; in the second copy every token can also attend to the first copy, question included.
3. No training. The weights, the temperature and the label-token readout are those of Gen5. We checked the change on a fresh full run of the Decision Index public suite before release. It is the same change as in KnowLine-35B-A3B-Gen1.5 and KnowLine-26B-A4B-Gen2.5.
4. Credit. The read-twice layout follows FlexDM (github.com/1000luca/FlexDM, MIT); repeating the input goes back to RE2 (Xu et al., 2023, arXiv:2309.06275). knowline_layout.py is our own implementation.

Comparison

Scores on the Decision Index. Since Decision Index 0.4, the board's full score is 60% private tests of the same skills and 40% private tasks from new domains; the public suite is shown but no longer counts. Gen5.5 has no board entry yet.

model	size	DI 0.3 public	DI 0.4.1 full	source
KnowLine-4B-Gen5.5	4B	66.75	not yet scored	self-run, official kit
KnowLine-4B-Gen5 (same weights)	4B	65.67	53.68	board
StartLux-Decision v1.1 4B	4.7B	67.81	54.80	board
Perplexity Decider Flash v1.2	4.5B	60.06	54.29	board
vLLM-SR d3-mini	4.5B	54.90	49.97	board
OneJev 1.6	4.5B	61.95	48.88	board
Jev 1.14	(API)	60.70	60.24	board
KnowLine-35B-A3B-Gen1	35B-A3B	69.49	61.20	board

Board rows from the board's data of 2026-10-21 (22:34 UTC, Decision Index 0.4.1).

Public-suite skill by area (DI 0.3, same weights):

area	Gen5.5	Gen5
Knowledge & reasoning	47.3	45.0
Language	68.2	66.8
Retrieval & routing	74.0	73.3
Tools & agents	87.4	87.4
Arts & taste	61.0	61.2

Releases

The model is trained in a self-evolving loop, and new versions will follow.

model	date	DI 0.3 public	notes
KnowLine-4B-Gen5.5 (this model)	2026-10-22	66.75	Gen5 weights, read-twice serving
KnowLine-4B-Gen5	2026-10-21	65.67	fifth release
KnowLine-4B-Gen4	2026-10-10	64.90	fourth release
KnowLine-4B-Gen3	2026-10-09	63.11	third release
KnowLine-4B-Gen2	2026-10-08	62.54	second release
KnowLine-4B-Gen1	2026-10-07	60.47	first release

Evaluation

All results are self-run and have not been verified by a third party. Every Gen5.5 number below was measured through the Gen5.5 front end (read twice on).

Other public benchmarks

suite	Gen5.5	Gen5
C-Eval (4 categories, macro)	80.5	79.2
JevBench / JevBench-hard	85.3 / 70.3	83.1 / 66.7

• In a paired test against Gen5, p = 0.02 for C-Eval; the differences on JevBench (p = 0.23) and JevBench-hard (p = 0.34) are not significant.

Web operation (Mind2Web official test splits, evaluation only)

split	Gen5.5 element selection	Gen5.5 operation (balanced)	Gen5 element selection	Gen5 operation (balanced)
test_task (Cross-Task)	92.7	96.9	92.6	97.1
test_website (Cross-Website)	91.2	96.3	90.9	96.9
test_domain (Cross-Domain)	92.0	98.0	91.8	98.4

• Within noise of Gen5 on all three splits (paired test, p ≥ 0.06); no request failed.

• The task is to pick the target element from a candidate set: the target plus up to 5 other elements sampled from the page. This is easier than the original Mind2Web protocol, so do not compare these results with the Mind2Web leaderboard.

Images (zero-shot)

• Unchanged from Gen5: read twice applies only to text-only requests. A request with images is sent exactly as in Gen5, byte for byte, to the same weights, so Gen5's image behaviour holds for Gen5.5.
• No image training: the vision tower is the base model's and image use is zero-shot; see the Gen5 card.

Calibration

Computed on the 0.3 public suite using the method described by the Decision Index board: each field is scored right or wrong, confidence is the probability of the chosen option, benchmarks are weighted equally, and 10 equal-width bins are used.

metric	Gen5.5	Gen5
ECE (lower is better)	0.035-0.046	0.037-0.049
Brier score (lower is better)	0.27-0.30	0.28-0.31
Confidence ≥95% but wrong	1.5-1.6%	1.5-1.6%
Mean confidence / accuracy	0.83 / 0.78-0.80	0.82-0.83 / 0.78-0.80

• Temperature in the weights: unchanged from Gen5: the final norm weight includes T = 1.50 (fitted on held-out data), so every logit is divided by 1.50 before the softmax over the option labels.
• Why ranges: the board does not publish which benchmarks it uses for calibration, so we report ranges over the plausible benchmark sets, as for Gen5.
• Recommendation: if your use depends on probability thresholds, check calibration on your own data.

Disclosures

• Training data: the weights are those of Gen5, so Gen5's disclosures on training data and decontamination apply unchanged; see the Gen5 card.
• Public suite: the Decision Index public suite is public; the board's private tests may show a lower score.

Limitations

• Slower than Gen5: about 1.56 times the prompt tokens on the Decision Index suite; one request at a time on one H20, 42.1 ms median and 83 ms mean (Gen5: 31.6 ms and 58.8 ms), and 297 ms at the 95th percentile (Gen5: 144 ms). Use --no-read-twice if latency matters more than these gains.
• Only short requests change: text-only requests over 3,000 tokens, and requests with images, are answered exactly as by Gen5.
• Not better everywhere: arts and taste on the public suite is slightly lower than with Gen5 (61.2 → 61.0).
• Everything else as Gen5: knowledge-heavy reasoning is still the weakest area and a 4B model is limited there; the board's private new-domain tasks favour larger models; prompt-injection attacks may still work; image use is zero-shot.

Citation

@misc{knowline4bgen55,
  title  = {KnowLine-4B-Gen5.5: a 4B decision model served with read twice},
  author = {PelaAI},
  year   = {2026},
  url    = {https://huggingface.co/PelaAI/KnowLine-4B-Gen5.5}
}

License

• Weights: Apache-2.0, the same as the base model.
• Code: knowline_server.py, knowline_layout.py and knowline_engine.py are MIT (see the file headers).