Text Classification
Transformers
Safetensors
English
Chinese
qwen3_5
image-text-to-text
decision-model
system-one
decision-index
lora-merged
Instructions to use PelaAI/KnowLine-4B-Gen5 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use PelaAI/KnowLine-4B-Gen5 with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-classification", model="PelaAI/KnowLine-4B-Gen5")# pip install -U transformers accelerate # Load model directly from transformers import AutoProcessor, AutoModelForMultimodalLM processor = AutoProcessor.from_pretrained("PelaAI/KnowLine-4B-Gen5") model = AutoModelForMultimodalLM.from_pretrained("PelaAI/KnowLine-4B-Gen5", device_map="auto") - Notebooks
- Google Colab
- Kaggle
|
Download README.md from PelaAI/KnowLine-4B-Gen5: direct link, hf CLI and curl.
- Browser
- Download file 9.62 kB
-
https://huggingface.co/PelaAI/KnowLine-4B-Gen5/resolve/main/README.md
- Command line
-
hf download hf://PelaAI/KnowLine-4B-Gen5/README.md
-
curl -L -o README.md https://huggingface.co/PelaAI/KnowLine-4B-Gen5/resolve/main/README.md
9.62 kB
| license: apache-2.0 | |
| base_model: | |
| - Qwen/Qwen3.5-4B | |
| base_model_relation: finetune | |
| library_name: transformers | |
| pipeline_tag: text-classification | |
| language: | |
| - en | |
| - zh | |
| tags: | |
| - decision-model | |
| - system-one | |
| - decision-index | |
| - lora-merged | |
| # KnowLine-4B-Gen5 | |
| **The fifth release of PelaAI's 4B System One decision model.** | |
| [Inference guide](INFERENCE.md) · [中文说明](README.zh.md) · weights Apache-2.0 · previous version: | |
| [KnowLine-4B-Gen4](https://huggingface.co/PelaAI/KnowLine-4B-Gen4) | |
| Highlights: | |
| - **Decision Index 0.3 public suite: 65.67** (self-run; Gen4: 64.90). The board's full score comes from the maintainers' | |
| private tests; see [Comparison](#comparison). | |
| - **Calibrated probabilities:** a temperature is folded into the weights. Computed with the board's method, ECE is about | |
| 0.04-0.05 and the Brier score about 0.28-0.31; see [Calibration](#calibration). | |
| - **Fast:** one request at a time on one H20, 31.6 ms median (see [INFERENCE.md](INFERENCE.md)). | |
| - **Images without any image training:** the vision tower is the base model's and image input is zero-shot. | |
| KnowLine is an independent model. It is not affiliated with, endorsed by, or derived from TypeSafe or Jev. | |
| ## What it does | |
| You send a state (text or a chat) and up to 64 typed questions: yes/no, choose one of k, or score on a rubric. The model | |
| answers every question with a probability distribution over its options: | |
| - one forward pass per question, with no text generation; | |
| - the output is the probability of each option's label token; | |
| - existing Jev clients only need to change the base URL, and users of earlier versions only need to change the model | |
| name, because the interface and serving settings are unchanged. | |
| | | | | |
| |---|---| | |
| | Base model | [Qwen/Qwen3.5-4B](https://huggingface.co/Qwen/Qwen3.5-4B) (Apache-2.0) | | |
| | Training | LoRA SFT (rank 32, alpha 64, language model only), merged into bf16 weights (see [How Gen5 was made](#how-gen5-was-made)) | | |
| | Release format | bf16 weights, with a readout temperature T = 1.50 folded into the final norm weight (see [Calibration](#calibration)). The vision tower and MTP head are unchanged from the base model; the config, tokenizer and chat template are identical to those of Gen1-Gen4. | | |
| | Languages | English, Simplified Chinese, Traditional Chinese | | |
| ## Quickstart | |
| ```bash | |
| pip install "sglang==0.5.21" "transformers==5.12.1" requests | |
| bash serve_knowline.sh PelaAI/KnowLine-4B-Gen5 0 8080 # SGLang (FP8 at load) on :9080 + /v1/systemone on :8080 | |
| curl -s http://127.0.0.1:8080/v1/systemone -H 'Content-Type: application/json' -d '{ | |
| "model": "m", | |
| "state": "Customer: my order arrived broken, I want my money back.", | |
| "questions": {"refund": {"type": "noul", "instructions": "Should the agent offer a refund?"}, | |
| "tone": {"type": "choice", "instructions": "Customer tone?", | |
| "criteria": {"angry": "Angry", "neutral": "Neutral", "happy": "Happy"}}}}' | |
| ``` | |
| - **Front end:** `knowline_server.py` is a single file that depends only on transformers and requests. It is the same | |
| file as in Gen4. | |
| - **Full settings:** the exact settings used for our Decision Index run are in [INFERENCE.md](INFERENCE.md). | |
| ## How Gen5 was made | |
| - **Base model:** [Qwen/Qwen3.5-4B](https://huggingface.co/Qwen/Qwen3.5-4B). | |
| - **Training:** one epoch of LoRA SFT (rank 32, alpha 64, language model only), merged into bf16 weights. Training | |
| examples include decision tasks written and labelled by LLM teachers. | |
| - **Readout temperature:** T = 1.50, fitted on held-out data and folded into the final norm weight. It changes the size | |
| of the probabilities, but not which option ranks first. | |
| The training data is not included in this repository. | |
| ## Comparison | |
| Scores on the Decision Index. Since Decision Index 0.4 (2026-10-19), the board's full score is 60% private tests of the | |
| same skills and 40% private tasks from new domains; the public suite is shown but no longer counts. Our full score is not | |
| available yet. | |
| | model | size | DI 0.3 public | DI 0.4 full | source | | |
| |---|---|---|---|---| | |
| | **KnowLine-4B-Gen5** | 4B | **65.67** | not yet scored | self-run, official kit | | |
| | KnowLine-4B-Gen4 | 4B | 64.90 | not yet scored | self-run, official kit | | |
| | StartLux-Decision v1.1 4B | 4.7B | 67.81 | 56.35 | board | | |
| | Perplexity Decider Flash v1.2 | 4.5B | 60.06 | 55.39 | board | | |
| | vLLM-SR d3-mini | 4.5B | 54.90 | 51.25 | board | | |
| | OneJev 1.6 | 4.5B | 61.95 | 49.54 | board | | |
| | Jev 1.14 | (API) | 60.70 | 62.22 | board | | |
| | KnowLine-35B-A3B-Gen1 | 35B-A3B | 69.49 | 62.28 | board | | |
| Board rows as of the board's latest update (Decision Index 0.4). | |
| Public-suite scores by area (DI 0.3): | |
| | area | KnowLine-4B-Gen5 | KnowLine-4B-Gen4 | | |
| |---|---|---| | |
| | Knowledge & reasoning | 45.0 | 44.6 | | |
| | Language | 66.8 | 65.5 | | |
| | Retrieval & routing | 73.3 | 72.5 | | |
| | Tools & agents | 87.4 | 87.6 | | |
| | Arts & taste | 61.2 | 59.2 | | |
| ## Releases | |
| The model is trained in a self-evolving loop, and new versions will follow. | |
| | model | date | DI 0.3 public | notes | | |
| |---|---|---|---| | |
| | KnowLine-4B-Gen5 (this model) | 2026-10-21 | 65.67 | fifth release | | |
| | [KnowLine-4B-Gen4](https://huggingface.co/PelaAI/KnowLine-4B-Gen4) | 2026-10-10 | 64.90 | fourth release | | |
| | [KnowLine-4B-Gen3](https://huggingface.co/PelaAI/KnowLine-4B-Gen3) | 2026-10-09 | 63.11 | third release | | |
| | [KnowLine-4B-Gen2](https://huggingface.co/PelaAI/KnowLine-4B-Gen2) | 2026-10-08 | 62.54 | second release | | |
| | [KnowLine-4B-Gen1](https://huggingface.co/PelaAI/KnowLine-4B-Gen1) | 2026-10-07 | 60.47 | first release | | |
| ## Evaluation | |
| All results are self-run and have not been verified by a third party. | |
| ### Other public benchmarks | |
| | suite | Gen5 | Gen4 | | |
| |---|---|---| | |
| | C-Eval (4 categories, macro) | 79.2 | 78.7 | | |
| | JevBench / JevBench-hard | 83.1 / 66.7 | 86.1 / 72.1 | | |
| ### Web operation (Mind2Web official test splits, evaluation only) | |
| | split | Gen5 element selection | Gen5 operation (balanced) | Gen4 element selection | Gen4 operation (balanced) | | |
| |---|---|---|---|---| | |
| | test_task (Cross-Task) | 92.6 | 97.1 | 92.9 | 97.0 | | |
| | test_website (Cross-Website) | 90.9 | 96.9 | 91.9 | 95.7 | | |
| | test_domain (Cross-Domain) | 91.8 | 98.4 | 91.7 | 97.2 | | |
| - About the same as Gen4; the differences are within noise. | |
| - The task is to pick the target element from a candidate set: the target plus up to 5 other elements sampled from the | |
| page. This is easier than the original Mind2Web protocol, so do not compare these results with the Mind2Web | |
| leaderboard. | |
| ### Images (zero-shot) | |
| - **No image training:** Gen5 keeps the base model's vision tower unchanged and saw no images in training. | |
| `knowline_server.py` accepts OpenAI-style `image` / `image_url` content parts in a chat state and passes them to | |
| SGLang; each question is still one prefill. | |
| ### Calibration | |
| Computed on the 0.3 public suite using the method described by the Decision Index board: each field is scored right or | |
| wrong, confidence is the probability of the chosen option, benchmarks are weighted equally, and 10 equal-width bins are | |
| used. | |
| | metric | Gen5 | Gen4 | | |
| |---|---|---| | |
| | ECE (lower is better) | 0.04-0.05 | 0.04-0.05 | | |
| | Brier score (lower is better) | 0.28-0.31 | 0.29-0.31 | | |
| | Confidence ≥95% but wrong | 1.5-1.6% | 1.6-1.8% | | |
| | Mean confidence / accuracy | 0.82-0.83 / 0.78-0.80 | 0.82 / 0.77 | | |
| - **Temperature in the weights:** the final norm weight includes a readout temperature T = 1.50, so every logit is | |
| divided by 1.50 before the softmax over the option labels. The top-ranked option does not change; the probabilities | |
| are just less extreme. T was fitted on held-out data. | |
| - **Why ranges:** the board does not publish which 32 benchmarks it uses, so we report ranges over the plausible | |
| benchmark sets, as for the earlier releases. | |
| - **Recommendation:** if your use depends on probability thresholds, check calibration on your own data. | |
| ## Disclosures | |
| - **Decontamination:** all training data was checked against every Decision Index 0.3 row and the public benchmarks we | |
| report (Intern-Decision, S1MB, JevBench, MMDM). About 22,000 rows were removed: rows that nearly | |
| duplicate a benchmark item, repeat its question or claim, or contain its whole state, and, in a stricter second pass, | |
| most rows that share a source passage with a benchmark item. | |
| - **Teacher-labelled synthetic data:** LLM teachers wrote and labelled our synthetic tasks. The labels were filtered, but | |
| not all of them were checked by a human. | |
| - **Public suite:** the Decision Index public suite is public; the board's private tests may show a lower score. | |
| ## Limitations | |
| - **Knowledge-heavy reasoning:** weaker than larger models. The knowledge area score is 45.0 (DI 0.3); a 4B model is | |
| limited here, and the new-domain part of the board favours larger models. | |
| - **JevBench-hard:** 66.7 against 72.1 for Gen4; the drop is not significant, but check it if your use is close to | |
| JevBench-hard. | |
| - **New domains:** the board's private new-domain tasks favour larger models; check the model on your own domain. | |
| - **Prompt injection:** an instruction planted in the state can change the answer. Keep untrusted text clearly | |
| delimited. | |
| - **Images:** there was no image training; image use is zero-shot, so check it on your own data before relying on it. | |
| ## Citation | |
| ```bibtex | |
| @misc{knowline4bgen5, | |
| title = {KnowLine-4B-Gen5: a 4B decision model}, | |
| author = {PelaAI}, | |
| year = {2026}, | |
| url = {https://huggingface.co/PelaAI/KnowLine-4B-Gen5} | |
| } | |
| ``` | |
| ## License | |
| - **Weights:** Apache-2.0, the same as the base model. | |
| - **Code:** `knowline_server.py` is MIT (see the file header). | |