Model card: README.md (English) and README.zh.md (Chinese)

README.md

@@ -0,0 +1,180 @@
+---
+license: apache-2.0
+base_model:
+  - Qwen/Qwen3.5-4B
+base_model_relation: finetune
+library_name: transformers
+pipeline_tag: text-classification
+language:
+  - en
+  - zh
+tags:
+  - decision-model
+  - system-one
+  - decision-index
+  - lora-merged
+---
+
+# KnowLine-4B-Gen1
+
+**A 4B System One decision model from PelaAI, trained on a single all-in-one machine by an agent-driven data loop.**
+
+[Inference guide](INFERENCE.md) · [中文说明](README.zh.md) · weights Apache-2.0
+
+Highlights:
+
+- **Decision Index 0.3, public suite: 60.47** (self-run). For comparison: Jev 1.13 scores 57.96 (board), and the best
+  ≤5B model on the board has a public score of 50.82; see [Comparison](#comparison).
+- **An AI agent ran the whole loop.** In each round it:
+  - found the areas where the model was weak;
+  - proposed a targeted data group;
+  - built and decontaminated the data;
+  - trained and evaluated on it;
+  - kept or rejected the change based on the evidence.
+- **KOF '98 harness: 17 wins, 0 losses, 1 draw against Jev 1.13** in a single-bout mirror round robin. For the play
+  style, see [Game harness](#game-harness-kof-98).
+
+KnowLine is an independent model. It is not affiliated with, endorsed by, or derived from TypeSafe or Jev.
+
+## What it does
+
+You send a state (text or a chat) and up to 64 typed questions: yes/no, choose one of k, or score on a rubric. The model
+answers every question with a probability distribution over its options:
+
+- one forward pass per question, with no generated text;
+- the output is the probability of each option's label token;
+- existing Jev clients only need a new base URL.
+
+| | |
+|---|---|
+| Base model | [Qwen/Qwen3.5-4B](https://huggingface.co/Qwen/Qwen3.5-4B) (Apache-2.0) |
+| Training | LoRA SFT (rank 32, alpha 64) on the language model, one epoch over about 713k rows. This release is step 5,650 of 5,655, the lowest validation loss. |
+| Release format | LoRA merged into the base; bf16 weights. The vision tower and MTP head are unchanged. |
+| Languages | English, Simplified Chinese, Traditional Chinese |
+
+## Quickstart
+
+```bash
+pip install "sglang==0.5.21" "transformers==5.12.1" requests
+bash serve_knowline.sh PelaAI/KnowLine-4B-Gen1 0 8080     # SGLang (FP8 at load) on :9080 + /v1/systemone on :8080
+curl -s http://127.0.0.1:8080/v1/systemone -H 'Content-Type: application/json' -d '{
+  "model": "m",
+  "state": "Customer: my order arrived broken, I want my money back.",
+  "questions": {"refund": {"type": "noul", "instructions": "Should the agent offer a refund?"},
+                "tone": {"type": "choice", "instructions": "Customer tone?",
+                         "criteria": {"angry": "Angry", "neutral": "Neutral", "happy": "Happy"}}}}'
+```
+
+- **Without SGLang:** `python knowline_server.py --model PelaAI/KnowLine-4B-Gen1 --backend hf --port 8080` uses
+  transformers only. It is slower and runs bf16.
+- **Front end:** `knowline_server.py` is a single file and needs only transformers and requests.
+- **Full settings:** the exact settings of our Decision Index runs are in [INFERENCE.md](INFERENCE.md).
+
+## Comparison
+
+Decision Index 0.3, public suite. The full 0.3 score adds private tests that only the maintainers run (0.5 same-skill,
+0.3 new-domain); ours is not available yet.
+
+| model | size | DI 0.3 public | DI 0.3 full | source |
+|---|---|---|---|---|
+| **KnowLine-4B-Gen1** | 4B | **60.47** | not yet scored | self-run, official kit |
+| Clef | 27B | 61.71 | 53.08 | board |
+| Jev 1.13 | (API) | 57.96 | 60.11 | board |
+| RSI-Jev v6.1-VL | 4B | 50.98 | not listed | self-reported |
+| ezjev 4B s2 | 4B | 50.82 | 46.95 | board |
+| jiwo 4B | 4B | 45.76 | 42.86 | board |
+| Nox 4B | 4B | 44.21 | 44.95 | board |
+
+Our public-suite scores by area:
+
+| area | KnowLine-4B-Gen1 (0.3 public) | Jev 1.13 (0.2.1) |
+|---|---|---|
+| Knowledge & reasoning | 37.7 | 51.4 |
+| Language | 60.8 | 62.0 |
+| Retrieval & routing | 70.9 | 55.4 |
+| Tools & agents | 86.8 | 75.1 |
+| Arts & taste | 49.5 | 37.7 |
+
+## Releases
+
+The model is trained in a self-evolving loop, and new versions will follow.
+
+| model | date | DI 0.3 public | DI 0.2.1 | golden held-out (en / zh-Hans / zh-Hant) | notes |
+|---|---|---|---|---|---|
+| KnowLine-4B-Gen1 (this model) | 2026-10-07 | 60.47 | 60.92 | 68.7 / 69.9 / 64.1 | first release (internal run "mix E", step 5,650) |
+
+## Evaluation
+
+All results are self-run and not verified by a third party.
+
+### Held-out and Chinese evaluations
+
+| suite | accuracy |
+|---|---|
+| In-house evaluation set, English | 68.7 |
+| In-house evaluation set, Simplified Chinese | 69.9 |
+| In-house evaluation set, Traditional Chinese | 64.1 |
+| C-Eval (4 categories, macro) | 77.9 |
+| Open-Jev 1.1 test / OOD | 87.4 / 86.6 |
+
+### Web operation (Mind2Web official test splits, evaluation only)
+
+| split | element selection | operation (balanced) |
+|---|---|---|
+| test_task (websites seen in training, new tasks) | 93.1 | 96.6 |
+| test_website (new websites) | 90.7 | 96.5 |
+| test_domain (new domains) | 91.5 | 98.5 |
+
+- So far this is only used to explore the model in RPA-style automation and to check that it generalises to some degree.
+- The task is to pick the target element among it and up to 5 other candidates sampled from the page. This is easier
+  than the original Mind2Web protocol, so do not compare it with the Mind2Web leaderboard.
+
+### Game harness (KOF '98)
+
+- **Setup:** single-bout character-mirror round robin, 11 players, 18 games per pair (990 games), argmax actions.
+- **Result:** win rate 0.883 [0.83, 0.92], Elo 1910, tied for first of 11 with another internal checkpoint.
+- **Head-to-head:** 17-0-1 against Jev 1.13, 15-3 against StartLux-Decision-4B, 18-0 against Clef-Flash.
+- **Caveat:** the policy relies heavily on one move, a 623C anti-air uppercut used about 55% of the time. This is a
+  measured result in this harness, not general fighting-game skill.
+
+## Disclosures
+
+- **Decision Index training data:** about 25% of the data is in Decision Index format, that is, synthetic items written
+  in the benchmarks' request formats. No Decision Index test item is included.
+- **Decontamination:**
+  - Every component text (state, instructions, option texts) was checked against every Decision Index row and all of
+    our evaluation sets.
+  - Rows that are identical, or share any run of 50 consecutive characters, were removed.
+- **Game data:** labels come from simulator rollouts or engines. Seeds and start states are disjoint from the evaluation
+  states.
+- **No model outputs as labels:** no output of Jev or any other decision model was used as a training label.
+- **Teacher-labelled synthetic data:** LLM teachers wrote and labelled our synthetic tasks. The labels were filtered, but
+  not all were checked by a human.
+
+## Limitations
+
+- **Knowledge-heavy reasoning:** weaker than larger models. The knowledge area is 37.7 (0.3); MMLU-Pro, GPQA and HLE are
+  close to the base model.
+- **Math:** answered without reasoning. The rebuilt 0.3 GSM8K scores 29.5.
+- **Prompt injection:** an instruction planted in the state changes the answer about 9% of the time on our injection
+  set. Keep untrusted text clearly delimited.
+- **Calibration:** ECE is about 0.05-0.06 on choice and yes/no questions and about 0.12 on score questions. For score
+  questions, fit a temperature on your own data.
+- **Private tests:** part of our public-suite advantage comes from adapting to the question formats, so the 0.3 private
+  tests may score lower.
+
+## Citation
+
+```bibtex
+@misc{knowline4bgen1,
+  title  = {KnowLine-4B-Gen1: a 4B decision model},
+  author = {PelaAI},
+  year   = {2026},
+  url    = {https://huggingface.co/PelaAI/KnowLine-4B-Gen1}
+}
+```
+
+## License
+
+- **Weights:** Apache-2.0, the same as the base model.
+- **Code:** `knowline_server.py` is MIT (see the file header).

README.zh.md

@@ -0,0 +1,151 @@
+# KnowLine-4B-Gen1
+
+**PelaAI 推出的 4B System One 决策模型，在单台一体机上，由 Agent 驱动的数据迭代循环训练出来。**
+
+[推理说明](INFERENCE.md) · [English](README.md) · 权重 Apache-2.0 
+
+亮点：
+
+- **Decision Index 0.3 公开套件：60.47**（自测）。作为对照：Jev 1.13 为 57.96（榜单）；榜单上 ≤5B 模型的最高公开分为 50.82，见[横向对比](#横向对比)。
+- 整个迭代循环由 AI Agent 端到端执行，包括分析模型在哪些领域效果不佳、提出针对性的数据组、构建数据并去污染，并针对其进行训练和评测，最终根据证据决定保留还是否决这次改动
+- **拳皇 '98 评测：对 Jev 1.13 为 17 胜 0 负 1 平**（单回合同角色镜像循环赛），打法特点见[游戏评测](#游戏评测拳皇-98)。
+
+KnowLine 是独立开发的模型，与 TypeSafe 或 Jev 没有隶属关系，未获其背书，也不是基于它们开发的。
+
+## 它做什么
+
+你发送一段状态（文本或对话）和最多 64 道带类型的问题：是/否、k 选一、按细则打分。模型对每道题给出各选项的概率分布：
+
+- 每道题只做一次前向计算，不生成文本；
+- 生成的是各选项标签 token 的概率；
+- 现有 Jev 客户端只需修改 base URL 即可接入。
+
+|      |                                                                                                 |
+| ---- | ----------------------------------------------------------------------------------------------- |
+| 底座模型 | [Qwen/Qwen3.5-4B](https://huggingface.co/Qwen/Qwen3.5-4B)（Apache-2.0）                           |
+| 训练方式 | 在语言模型部分做 LoRA 监督微调（rank 32，alpha 64），在约 71.3 万行数据上训练 1 轮。本次发布的是第 5,650 步（共 5,655 步），验证 loss 最低。 |
+| 发布格式 | LoRA 已合并进底座，bf16 权重；视觉部分和 MTP 头不变                                                               |
+| 语言   | 英文、简体中文、繁体中文                                                                                    |
+
+
+## 快速开始
+
+```bash
+pip install "sglang==0.5.21" "transformers==5.12.1" requests
+bash serve_knowline.sh PelaAI/KnowLine-4B-Gen1 0 8080     # SGLang（加载时转 FP8）监听 :9080，/v1/systemone 监听 :8080
+curl -s http://127.0.0.1:8080/v1/systemone -H 'Content-Type: application/json' -d '{
+  "model": "m",
+  "state": "Customer: my order arrived broken, I want my money back.",
+  "questions": {"refund": {"type": "noul", "instructions": "Should the agent offer a refund?"},
+                "tone": {"type": "choice", "instructions": "Customer tone?",
+                         "criteria": {"angry": "Angry", "neutral": "Neutral", "happy": "Happy"}}}}'
+```
+
+- **不用 SGLang**：`python knowline_server.py --model PelaAI/KnowLine-4B-Gen1 --backend hf --port 8080` 只依赖 transformers，速度较慢，精度为 bf16。
+- **前端**：`knowline_server.py` 是单个文件，只依赖 transformers 和 requests。
+- **完整设置**：我们跑 Decision Index 时的完整设置见 [INFERENCE.md](INFERENCE.md)。
+
+## 横向对比
+
+下表为 Decision Index 0.3 公开套件的分数。0.3 的总分还要加上只由维护者运行的私有测试（同技能 0.5、新领域 0.3），我们的总分暂时还没有。
+
+| 模型                   | 规模    | DI 0.3 公开分 | DI 0.3 总分 | 来源       |
+| -------------------- | ----- | ---------- | --------- | -------- |
+| **KnowLine-4B-Gen1** | 4B    | **60.47**  | 尚未评测      | 自测，官方工具包 |
+| Clef                 | 27B   | 61.71      | 53.08     | 榜单       |
+| Jev 1.13             | （API） | 57.96      | 60.11     | 榜单       |
+| RSI-Jev v6.1-VL      | 4B    | 50.98      | 未上榜       | 自报       |
+| ezjev 4B s2          | 4B    | 50.82      | 46.95     | 榜单       |
+| jiwo 4B              | 4B    | 45.76      | 42.86     | 榜单       |
+| Nox 4B               | 4B    | 44.21      | 44.95     | 榜单       |
+
+我们的模型在各子领域的公开套件评分如下：
+
+| 领域        | KnowLine-4B-Gen1（0.3 公开） | Jev 1.13（0.2.1） |
+| --------- | ------------------------ | --------------- |
+| 知识与推理     | 37.7                     | 51.4            |
+| 语言        | 60.8                     | 62.0            |
+| 检索与路由     | 70.9                     | 55.4            |
+| 工具与 Agent | 86.8                     | 75.1            |
+| 艺术与品味     | 49.5                     | 37.7            |
+
+
+## 版本
+
+本模型使用自进化的模式进行训练，将持续产生新版本
+
+
+| 模型                    | 日期         | DI 0.3 公开 | DI 0.2.1 | 黄金集留出（英 / 简 / 繁）   | 说明                           |
+| --------------------- | ---------- | --------- | -------- | ------------------ | ---------------------------- |
+| KnowLine-4B-Gen1（本模型） | 2026-10-07 | 60.47     | 60.92    | 68.7 / 69.9 / 64.1 | 首次发布（内部运行 "mix E"，第 5,650 步） |
+
+
+## 评测
+
+以下结果均为自测，未经第三方复核。
+
+### 留出评测与中文评测
+
+
+| 测试集                   | 准确率         |
+| --------------------- | ----------- |
+| 自建评测数据集（英文部分）         | 68.7        |
+| 自建评测数据集（简体中文部分）       | 69.9        |
+| 自建评测数据集（繁体中文部分）       | 64.1        |
+| C-Eval（4 大类宏平均）       | 77.9        |
+| Open-Jev 1.1 测试 / OOD | 87.4 / 86.6 |
+
+
+### 网页操作（Mind2Web 官方测试集，仅用于评测）
+
+
+| 划分                       | 选元素  | 选操作（三类平均） |
+| ------------------------ | ---- | --------- |
+| test\_task（训练中见过的网站、新任务） | 93.1 | 96.6      |
+| test\_website（新网站）       | 90.7 | 96.5      |
+| test\_domain（新领域）        | 91.5 | 98.5      |
+
+
+- 目前仅用于探索该模型在 RPA 自动化等领域的情况，并验证存在一定的泛化性
+- 任务是从目标元素和页面上随机抽取的最多 5 个其他候选里选出目标，比 Mind2Web 原始评测简单，因此请勿与其排行榜比较
+
+### 游戏评测（拳皇 '98）
+
+- **赛制**：单回合同角色镜像循环赛，11 方参赛，每组 18 局（共 990 局），每步选概率最高的动作。
+- **结果**：胜率 0.883 [0.83, 0.92]，Elo 1910，与另一个内部检查点并列第一。
+- **对战成绩**：对 Jev 1.13 为 17 胜 0 负 1 平，对 StartLux-Decision-4B 为 15 胜 3 负，对 Clef-Flash 为 18 胜 0 负。
+- **说明**：策略高度依赖一招，即 623C 对空升龙，使用率约 55%。这是在该评测环境中的实测结果，不代表具备全面的格斗游戏能力。
+
+## 披露
+
+- **Decision Index 训练集**：约 25% 的数据是 Decision Index 的格式，即按基准请求格式编写的合成题。不含任何测试题。
+- **去污染**：
+  - 每个文本片段（状态、题干、选项文本）都与所有 Decision Index 题目以及我们全部评测集做了比对。
+  - 完全相同或存在任意连续 50 个字符相同的行都已删除。
+- **游戏数据**：标签来自模拟器推演或引擎，随机种子和起始局面都与评测局面不重叠。
+- **不用模型输出作标签**：从未把 Jev 或其他任何决策模型的输出用作训练标签。
+- **教师模型标注的合成数据**：合成任务由大语言模型教师编写和标注。标签经过筛选，但并非全部经过人工核对。
+
+## 局限
+
+- **知识密集型推理**：弱于更大的模型。知识领域 37.7（0.3），MMLU-Pro、GPQA、HLE 接近底座水平。
+- **数学**：没有推理过程，0.3 重建版 GSM8K 为 29.5。
+- **提示注入**：在我们的注入测试集上，状态里植入的指令约有 9% 的概率改变答案。请把不可信文本清楚地隔开。
+- **校准**：选择题和是/否题的 ECE 约为 0.05-0.06，打分题约为 0.12。打分题建议用自己的数据拟合温度。
+- **私有测试**：我们在公开套件上的优势有一部分来自对题目格式的适应，0.3 私有测试的分数可能更低。
+
+## 引用
+
+```bibtex
+@misc{knowline4bgen1,
+  title  = {KnowLine-4B-Gen1: a 4B decision model},
+  author = {PelaAI},
+  year   = {2026},
+  url    = {https://huggingface.co/PelaAI/KnowLine-4B-Gen1}
+}
+```
+
+## 许可证
+
+- **权重**：Apache-2.0，与底座模型相同。
+- **代码**：`knowline_server.py` 为 MIT 许可（见文件开头）。