K-IM-Deiction — judgment (triage) model for group-chat messages (standalone + LoRA adapter)

The kai-decision V2 model — a Sage2-style structured judgment model for group-chat messages — in one repo, two interchangeable forms:

• Standalone (repo root): the LoRA adapter already merged into the Qwen/Qwen3.5-0.8B backbone. Download and it runs — no separate base-model download, no PEFT setup for the backbone.
• LoRA adapter (adapter/ folder): the same model as an adapter (102.4 MB), for loading on top of your own Qwen/Qwen3.5-0.8B.

Both forms share the seven judgment heads and calibration files in this repo and produce the same judgments. The model reads one message window (recent context + the target message) and produces seven judgments in a single forward pass — median latency 41.0 ms per window on an RTX 4090 D (batch 1, ≤512 tokens).

Judgment layer, not decision layer. This model answers typed questions about a window (what kind of message is it, how likely is it to need attention, does it contain a to-do) and returns calibrated scores. It never decides or acts by itself: the decision — surface this message, create that task, escalate or ignore — belongs to the surrounding system: deterministic rules, attention budgets and thresholds you set, and ultimately people. Keeping the probabilistic model in the judgment layer and the decision in deterministic code (or human hands) is deliberate: it is what makes the scores safe to automate around.

Why this model exists

Group chats produce far more messages than anyone can read. The scarce resource is not information but attention: which of these hundreds of windows deserve a person's eyes, a reply, a follow-up?

Goal. Turn every message window into structured, calibrated judgment signals — fast enough and cheap enough to run on all of them, locally, before any human (or any large model) spends attention. Instead of asking a large language model to read every window at ~1.5k tokens apiece, a 0.8B specialist distilled from such a teacher makes the routine judgments in 40 ms on your own hardware, and only the windows it flags ever need a closer look.

What it does

For each window the model emits seven calibrated judgments in one pass (definitions in the Outputs section): a 7-class category, sentiment, importance / urgency / intensity scores, a needs-attention probability, and a has-todo probability. Downstream, these plug into software directly:

• Attention triage — rank each person's incoming windows by the calibrated needs-attention probability and surface only the top few per day (an attention budget), instead of an unread-badge firehose.
• To-do mining — windows with high has-todo probability become task-list candidates with an owner and a deadline to confirm, instead of actions dissolving into chat history.
• Routing and filing — the category head files windows into help / task / decision / info / discussion streams for dashboards, digests and retrieval.
• Priority queues — importance × urgency orders follow-up lists and escalation checks.
• Team-sentiment monitoring — sentiment and intensity aggregate into a daily pulse (spikes of negative intensity are visible without reading the raw chats).
• A pre-filter in front of large models — in an LLM pipeline, this model screens every window for ~0 marginal cost; only flagged windows are sent to a large model for deep reading or drafting. The comparison below quantifies what that trade buys.

Where it fits — and where it does not

Good fits

• Enterprise IM triage at volume: thousands of windows a day that no one can read end-to-end.
• Notification prioritization and daily digests in team-chat products.
• To-do / action-item extraction pipelines and follow-up reminders.
• Sentiment and escalation monitoring for support or operations groups.
• On-premise or edge deployment where chat content must not leave the network (this model runs fully locally; nothing is sent to an API).
• High-volume first-pass screening in front of an LLM, where per-window API cost would dominate.

Poor fits

• Open-ended reading comprehension, summarization or reply drafting — it emits judgments, not text; pair it with a large model for those.
• Languages, platforms or chat cultures beyond Chinese enterprise group chat (its training domain).
• Judging content it cannot see: attachments, images, links and quoted documents enter only as placeholder tokens in the window text.
• Being the sole arbiter of consequential decisions. Needs-attention is a distilled, partly subjective judgment operating close to its label-noise ceiling — use it to rank and filter, keep humans (or a large model) on the flagged minority.

Performance & evaluation at a glance

Item	Value
Total repo size	1.83 GB — standalone weights 1.71 GB + LoRA adapter 0.10 GB; tokenizer, heads and calibration files make up the rest
Latency	median 41.0 ms, p90 44.1 ms per window (RTX 4090 D, batch 1, ≤512 tokens, all 7 heads in one forward pass)
Category macro-F1	0.611 (7 classes) · 0.673 (classes with support ≥10)
needs-attention	AUC 0.822 · ECE 0.053 (calibrated; RTX 4090 D, torch)
has-todo	AUC 0.927 · ECE 0.069 (calibrated; RTX 4090 D, torch)
Sentiment accuracy	92.8%
Importance / urgency within ±1	99.9% / 97.7%

Scores were measured on the adapter form of this same model (base + LoRA) on the held-out set described below. Before packaging, the merged weights in this repo were verified against the adapter form on 16 held-out windows: the largest absolute difference across all seven heads' logits was 0.049 (bf16 rounding), i.e. the bundle is output-equivalent within bf16 precision.

The identical V2 bundle was also run end-to-end on an Apple-silicon Mac (MPS, bf16) over the same 1,000 held-out windows: category F1 (support ≥10) 0.671, needs-attention AUC 0.822, binary-head ECE 0.055 / 0.070, median latency 48.2 ms (p90 81.6 ms), and 99.6% category agreement with the RTX 4090 D run — the same bundle gives the same judgments on both machines.

Judgment model vs. prompting a large model (measured, 2026-11-05)

Why not just prompt a large model for these judgments? We measured it for the previous (v1.3) release, on that release's frozen held-out set (258 windows) and the same gold labels. It has not been re-run for V2 — the table below is the v1.3 measurement, kept as a reference for the specialist-vs-large-model trade-off; V2's own held-out results are in the Performance and Evaluation sections. The large-model baseline is DeepSeek (deepseek-chat API) prompted zero-shot with the full labeling specification as its system prompt — the same spec the teacher labels were produced under — at temperature 0.

Metric	K-IM-Deiction (v1.3, 0.8B)	DeepSeek zero-shot (large model)	Difference
Output format compliance	100% (structured heads, by construction)	100.0% (258/258 valid JSON)	0% (tie)
Category macro-F1 (7 classes)	0.637	0.620	+2.7%
Category macro-F1 (support ≥10)	0.744	0.724	+2.8%
needs-attention AUC	0.801	0.781	+2.6%
has-todo AUC	0.965	0.959	+0.6%
Sentiment accuracy	89.1%	94.2%	−5.4%
Importance / urgency within ±1	100% / 99.6%	98.8% / 98.5%	+1.2% / +1.1%
Latency per window	40.3 ms median (local, RTX 4090 D)	1.91 s median (API round trip, incl. network)	−97.9%
Tokens per window	186 processed on avg (median 139; windows left-truncated at 512), 0 generated — it emits numbers, not text	1,496 consumed on avg (1,312 in + 184 out); 385,961 for the 258 windows	−87.6%
Data exposure	at inference, fully on-premise — no window leaves the machine	every window is sent to a cloud API	—

Difference = (this model − large model) / large model. Positive means this model scores higher; for latency and tokens, negative means this model uses less (better).

How to read this. On the triage judgments this model was distilled for, the 0.8B specialist matches or slightly beats zero-shot large-model prompting — at roughly 1/47 of the latency, zero marginal cost, and with inference fully on-premise. The large model is clearly better at sentiment (94.2% vs 89.1%) and remains the right tool for open-ended reading and drafting; the specialist is a filter in front of it, not a replacement for it.

About tokens. Tokens are the unit any transformer processes — this model has them too; the difference is how many and for what. It reads one window (186 tokens on average after truncation, measured on this held-out set) and generates zero tokens, because its outputs are numbers from the decision heads, not text. The prompting baseline re-sends the same ~1,000 tokens of labeling specification with every window (its 1,312 average input tokens = window + spec), then generates ~184 tokens of JSON per window. So even if this model were deployed on a cloud inference service billed per token, a window would consume only its ~186 input tokens and nothing else — the specification lives in its weights, not in a prompt it must re-read forever.

Two more reference points, also measured:

• Prompting the untrained Qwen3.5-0.8B base with the same specification produced parseable output on only 1.6% of the 258 windows — a small model cannot simply be prompted into this job; the ability comes from the distillation.
• The teacher (GLM-5.3) that produced the gold labels consumed ≈1,526 tokens per window while labeling the v1.3 training round (3,440,553 tokens / 2,255 windows) — the same cost class as the DeepSeek baseline, except it is paid once during dataset construction instead of on every window, forever.

Contents

File	What it is
model.safetensors	Full merged model weights (Qwen3.5-0.8B backbone with the V2 LoRA merged in)
adapter/adapter_model.safetensors	The same V2 model as a LoRA adapter (r=64, alpha=128, dropout 0.05; language-model projections only) — alternative to the merged weights
adapter/adapter_config.json	PEFT config for the adapter (base: Qwen/Qwen3.5-0.8B)
config.json / generation_config.json	Model configuration (stock Qwen3.5)
tokenizer.json, tokenizer_config.json, vocab.json, merges.txt, chat_template.jinja	The base model's tokenizer files, copied verbatim from Qwen/Qwen3.5-0.8B
preprocessor_config.json, video_preprocessor_config.json	Stock Qwen3.5 processor configs
mlx/	MLX edition for Apple Silicon (slimmed text-only weights, heads as heads.npz, inference + service scripts); see the MLX section below
heads.pt	The 7 judgment heads (plain nn.Linear layers on the 1024-d pooled hidden state) plus hidden/base metadata
temps.json	Per-head temperature for probability calibration
calib.json	Category bias+temperature and isotonic regression tables for the two binary heads
LICENSE	Apache-2.0 license text from the base model

Input format

Plain text, one window per inference. The target message is preceded by up to 10 context messages from the previous 30 minutes; senders are anonymized as S####. The window is left-truncated to 512 tokens. The window includes two marker lines, 【前文窗口】 (context) and 【目标消息】 (target), exactly as used in training.

Example (synthetic):

群：【示例】项目周会群
【前文窗口】
[S0002] 明天下午三点过一下发布清单
[S0005] 好的，我把清单整理成文档发出来
【目标消息】
[S0003] 收到，我今晚把测试报告补齐

Outputs

Head	Type	Label space
cat	7-class	求助/提问 (help/request) · 任务/待办 (task) · 决策/结论 (decision) · 信息分享 (info) · 讨论观点 (discussion) · 闲聊寒暄 (chitchat) · 无关/其他 (other)
sen	3-class	积极 (positive) · 中性 (neutral) · 消极 (negative)
imp	ordinal 1–5	importance
urg	ordinal 1–5	urgency
inten	ordinal 1–5	sentiment intensity
att	binary	needs the user's attention (probability, calibrated)
todo	binary	contains a to-do / action item (probability, calibrated)

Ordinal heads are trained as 5-way soft distributions; read them as an expected score or argmax. Binary-head probabilities should pass through the temperature + isotonic calibration in calib.json before thresholding.

Usage

import json
import numpy as np
import torch
from transformers import AutoModelForImageTextToText, AutoTokenizer

REPO = "Songkai/K-IM-Deiction"

tok = AutoTokenizer.from_pretrained(REPO)
tok.truncation_side = "left"
if tok.pad_token is None:
    tok.pad_token = tok.eos_token
model = AutoModelForImageTextToText.from_pretrained(REPO, dtype=torch.bfloat16).cuda().eval()

blob = torch.load("heads.pt", map_location="cpu", weights_only=True) # from this repo
W = blob["heads"] # state_dict of the 7 nn.Linear heads (hidden size in blob["hidden"])
def lin(name, h): # one nn.Linear per head
    return h @ W[name + ".weight"].float().cuda().T + W[name + ".bias"].float().cuda()

calib = json.load(open("calib.json")) # from this repo
temps = calib["temps"]

def pooled(m, input_ids, attention_mask):
    mm = m.get_base_model() if hasattr(m, "get_base_model") else m
    core = getattr(mm, "model", mm)
    h = core(input_ids=input_ids, attention_mask=attention_mask).last_hidden_state
    idx = attention_mask.sum(1) - 1
    return h[torch.arange(h.size(0), device=h.device), idx]

@torch.no_grad()
def decide(text):
    enc = tok(text, truncation=True, max_length=512, return_tensors="pt").to("cuda")
    h = pooled(model, enc["input_ids"], enc["attention_mask"]).float()
    out = {n: lin(n, h) for n in ["cat", "sen", "imp", "urg", "inten", "att", "todo"]}
    res = {
        "category": torch.softmax((out["cat"] + torch.tensor(calib["cat_bias"]).cuda())
            / calib["cat_T"], -1)[0].cpu().numpy(),
        "sentiment": torch.softmax(out["sen"] / temps["sen"], -1)[0].cpu().numpy(),
        "importance": torch.softmax(out["imp"] / temps["imp"], -1)[0].cpu().numpy(),
        "urgency": torch.softmax(out["urg"] / temps["urg"], -1)[0].cpu().numpy(),
        "intensity": torch.softmax(out["inten"] / temps["inten"], -1)[0].cpu().numpy(),
    }
    for name, key in [("needs_attention", "att"), ("has_todo", "todo")]:
        p = torch.sigmoid(out[key] / temps[key])[0].item()
        iso = calib[key + "_iso"]
        res[name] = float(np.interp(p, iso["x"], iso["y"])) # calibrated probability
    return res

Adapter form

If you already run Qwen/Qwen3.5-0.8B, load the adapter from the adapter/ subfolder instead of the merged weights. The heads, calibration and the pooled / decide logic are exactly as above (download heads.pt and calib.json from this repo's root):

import torch
from transformers import AutoModelForImageTextToText, AutoTokenizer
from peft import PeftModel
BASE = "Qwen/Qwen3.5-0.8B"
tok = AutoTokenizer.from_pretrained(BASE)
tok.truncation_side = "left"
tok.pad_token = tok.pad_token if tok.pad_token is not None else tok.eos_token
backbone = AutoModelForImageTextToText.from_pretrained(BASE, dtype=torch.bfloat16)
model = PeftModel.from_pretrained(backbone, "Songkai/K-IM-Deiction", subfolder="adapter").cuda().eval()

Serving it locally

This is not a generative model, so generative serving stacks do not fit it: Ollama and vLLM serve text generation and cannot run the custom judgment heads or the calibration in this repo. The intended deployment is a thin HTTP wrapper around the loading code above — load the model and heads once, expose a JSON endpoint. The author's own setup is a small zero-dependency Python HTTP server on the training machine (RTX 4090 D) with this contract:

• GET /health → service status
• POST /decide with {"text": "<window>"} → one window's judgments (named fields: category and sentiment distributions with top-1, the three ordinal distributions with expected scores, calibrated needs-attention and has-todo probabilities)
• POST /decide_batch with {"texts": ["<window>", "..."]} → {"results": [...]}

Measured through the HTTP layer: about 0.1 s per window end-to-end in small batches (41.0 ms of that is the forward pass). A reference Skill for agent-driven triage (window construction, attention thresholds, a daily-digest workflow) is maintained in the author's assistant setup; the contract above is all an integration needs.

MLX edition (Apple Silicon)

The mlx/ folder in this repo is a native MLX port of the same V2 model for Apple Silicon Macs, built on mlx-lm 0.32.2 (which supports the Qwen3.5 text architecture; the vision tower is unused, exactly as in the torch version). It holds a slimmed weight file (the 153 vision-tower keys removed, 1,504,827,581 bytes; the 320 text keys are identical to the merged model), the judgment heads as both heads.pt and heads.npz, calib.json / temps.json, the tokenizer files, and two scripts: mlx_decision.py (the full inference chain: tokenization with left truncation to 512 tokens, last-token hidden state, the 7 heads, temperature + isotonic calibration) and serve_decision_mlx.py (a zero-dependency HTTP service with the same /health, /decide, /decide_batch contract as the torch service). See mlx/README_MLX.md for details.

Measured on the author's Mac on the same frozen held-out set of 1,000 windows: macro-F1 (support>=10) 0.673, needs-attention AUC 0.822, ECE 0.050 / 0.067 for the two binary heads (ECE is reported per platform: the RTX 4090 D torch package measures 0.053 / 0.069), with 99.6% category agreement against the torch-Mac run (bf16 rounding level). Latency is a median of 13.5 ms / p90 of 23.5 ms per window including tokenization (the slim package alone measures 14.9 / 24.8 ms), about 3.6x faster than the torch MPS run on the same machine (48.2 / 81.6 ms). To use it, download the mlx/ folder, install mlx and mlx-lm, and start the service manually in the foreground (it is intentionally not installed as a daemon):

pip install mlx mlx-lm numpy
python serve_decision_mlx.py --package ./mlx --port 8765

Resource footprint (measured 2026-11-07)

Both editions are small enough to run as an always-on background service. Measured with batch = 1, bf16 weights, steady state after warm-up inferences:

Edition	Machine	Process RAM (RSS)	GPU / unified memory	On-disk size
PyTorch package (this repo root)	Windows machine, WSL2, RTX 4090 D	~1.1 GB right after load, ~1.8 GB steady state	~2.2 GB VRAM	1.83 GB
MLX edition (mlx/, slim package)	Author's Mac (Apple Silicon)	~1.7 GB total process footprint (unified memory)	MLX active ~1.5 GB, peak ~1.6 GB	1.53 GB

The 7 judgment heads and calibration files are a few MB and are included in the figures above. The PyTorch service additionally needs the PyTorch runtime itself on disk; the MLX service needs mlx / mlx-lm.

Evaluation (held-out set, 1,000 windows, measured 2026-11-06)

The held-out set is 1,000 windows drawn from seven groups the previous release never saw; it was frozen before V2 training under a pre-registered re-check design and never used for training or for calibration/model selection. Labels are teacher labels (GLM-5.3 primary, frozen labeling spec, soft distributions; DeepSeek re-labeled the same windows for agreement reporting). Trained on 6,580 windows.

Metric	Result
Category macro-F1 (7 classes)	0.611
Category macro-F1 (classes with support ≥10)	0.673
Per-class F1	help 0.836 (support 268) · task 0.754 (239) · decision 0.462 (101) · info 0.672 (182) · discussion 0.694 (158) · chitchat 0.620 (43) · other 0.235 (support 9)
needs-attention AUC	0.822
needs-attention positives	247 of 1,000 windows (24.7%)
needs-attention ECE (calibrated; RTX 4090 D, torch)	0.053
has-todo AUC	0.927
has-todo ECE (calibrated; RTX 4090 D, torch)	0.069
Category confidence ECE	0.095 (diagnostic only, see below)
Sentiment accuracy	92.8%
Importance / urgency within ±1	99.9% / 97.7%
Latency	median 41.0 ms, p90 44.1 ms (RTX 4090 D, batch 1)
Needs-attention ranking quality	Precision@10 0.70 · Precision@20 0.80 · Precision@50 0.76

Metrics in plain words.

• AUC — did it rank right? Pick one message that truly needs attention and one that does not: AUC is the chance the model scores the important one higher. 0.5 is a coin flip, 1.0 is perfect ranking. The 0.822 above means the right message wins about 82% of the time. For a judgment this subjective, even two teachers labeling the same windows agree with each other on needs-attention only ~79% of the time (measured on this set), which puts the practical ceiling near 0.85.
• ECE — are the probabilities honest? Among windows the model calls "70% likely to need attention", do roughly 70% actually need it? ECE is the average gap between what the model says and what is true, across probability bands. The 0.053 / 0.069 above means the stated probabilities are off by only about 5–7 percentage points on average — close enough to use the probability as a threshold dial.

For reference, prompting the untrained base model to emit this label schema directly produced parseable output on only 1.6% of the earlier 258-window held-out set.

About the release gates. The original gates (macro-F1 ≥ 0.65, needs-attention AUC ≥ 0.85, ECE ≤ 0.08) were set before label quality was measured. A second teacher re-labeled a pilot sample and agreed with the primary teacher on needs-attention only 75% of the time; under a symmetric independent-error assumption that puts the achievable AUC against single-teacher gold labels at ≈ 0.854 — the 0.85 gate sat essentially on the label-noise ceiling. On 2026-11-05 the gates were revised with the rationale documented (support-aware macro-F1 ≥ 0.65; needs-attention AUC ≥ 0.80; ECE gate kept on the two binary heads, category ECE demoted to a diagnostic because the head is trained on soft teacher distributions). v1.3 passed the revised gates borderline (needs-attention AUC 0.801 on only 35 positive examples) and was recorded for re-check on a larger set. That re-check ran on this 1,000-window set from seven unseen groups: v1.3 scored needs-attention AUC 0.807 there, confirming the borderline pass under the pre-registered rule. V2 passes all three revised gates on the same set — F1 (support ≥10) 0.673 ≥ 0.65, needs-attention AUC 0.822 ≥ 0.80, binary-head ECE 0.053 / 0.069 ≤ 0.08.

Training data and privacy

• Trained on 6,580 message windows from private enterprise group chats (Chinese) — the v1.3 training set plus 1,507 newly labeled windows from additional groups and hard-example mining. Senders were replaced by pseudonymous IDs (S####) before labeling and training; labels were produced by a teacher model (GLM-5.3) under a frozen labeling specification, as soft distributions. The 1,000-window V2 held-out set was never used for training.
• The raw data has never been publicly released. This repo contains no messages, no group names and no personal identifiers; the example window above is synthetic, and the tokenizer files are Qwen's stock files, copied verbatim from the base model.
• Labeling did involve third-party APIs. During dataset construction, the windows — sender pseudonyms (S####) in place, but message body text verbatim at the time — were processed through BigModel's API (GLM-5.3, the primary teacher) and DeepSeek's API (second-pass labels and the large-model comparison). Anyone using this model should know its training labels were produced that way. Running the model itself is fully local (see the comparison table above).
• The retained local copies were body-scrubbed on 2026-11-07: person names (harvested from message metadata, @-mentions and structured name labels), phone numbers, company names, ID-card numbers and emails in message bodies were replaced by placeholders across the training set, every held-out set and the raw pulls (about 127,000 replacements in total; post-scrub verification scans for all of these patterns return zero matches). Window headers still carry the real group names, and free-text nicknames cannot be guaranteed exhaustively caught — so the local datasets are pseudonymized and body-scrubbed, not provably fully anonymized, a further reason they exist only on the author's machines and are never released.
• Scrubbing the retained copies does not retroactively change the already-trained weights. The model emits only numbers, not text, but no zero-leakage guarantee is claimed for the weights.
• Approaches measured and rejected during development: class weighting / oversampling (v1.1, metrics went down), more history from the same groups (marginal AUC gain +0.010 in v1.3), alternative category calibrators (statistically tied); hard-example mining in V2 surfaced only 20 usable candidate windows (limited effect). The shipped recipe is the plain one.

Known limitations

• Domain-specific: Chinese enterprise group-chat windows in the format above. Not evaluated on other languages, platforms, or chat styles.
• needs_attention is a distilled, partly subjective judgment close to its label-noise ceiling — use it as a triage ranking signal, not as ground truth about what matters.
• The other class is still rare in the held-out set (support 9), so its F1 (0.235) is noisy; the decision class remains the weakest of the main classes (F1 0.462, support 101) — its misses go mostly to info, discussion and task, i.e. conclusions stated implicitly inside a discussion.
• Scores inherit the teacher model's biases and the labeling spec's definitions; a different spec (e.g. a stricter notion of "needs attention") would produce a different model.

Roadmap

Next steps, in phases. Acceptance criteria for each experiment are fixed before it runs, from the actual loss and budget at that point — no hard numeric gates are pre-committed here.

Phase 1 — no new training required.

• Decision-boundary arbitration: pin down the labeling rules for boundary cases (implicit sign-offs, pending confirmations, vetoes and withdrawals) and freeze them before collecting any more decision-class examples; new examples must come from outside the evaluation pools.
• User-day shadow-trial design: keep the 20-per-day needs-attention budget and the manual Mac service unchanged, and design what a shadow run records (daily Precision@K and Recall@K, missed critical tasks, duplicate notifications, reading time), with a rule-based baseline (@-mentions / explicit assignment / deadlines) for comparison.
• Human anchor sample: a small double-blind human-labeled sample (annotators do not see teacher labels first) to calibrate what "generally worth attention" means; its size follows positive counts and interval width, not a round number.
• Evaluation hygiene, institutionalized: from the next round on, development, calibration and final blind-test sets are kept separate; the frozen 1,000-window set used to diagnose V2 becomes a regression set and is no longer treated as a fresh blind test.

Phase 2 — depending on Phase 1 results.

• Calibration and low-confidence escalation: fit calibration on an independent calibration set; report PR-AUC, Brier score and reliability diagrams for the binary heads; add a low-confidence escalation path evaluated by risk–coverage. Escalation defaults to a human or an authorized local path — group chats are never sent to a cloud API automatically.
• Personalization groundwork: study conditioning on relation signals relative to the target user (@-mentions, roles), with the label definitions conditioned the same way, plus counterfactual checks.

Phase 3 — once experiments justify it.

• Targeted top-up training against the remaining errors, and one-at-a-time structural ablations (pooling, ordinal loss, multi-teacher labels, new heads, quantization) — never bundled together.

License

Apache-2.0, following the Qwen3.5 base model.

---

中文说明

kai-decision V2——Sage2 式群聊消息结构化判断（分诊）模型，一个仓库、两种形态：整包（仓库根目录）是 LoRA 已合并进 Qwen/Qwen3.5-0.8B 基座的完整权重，下载即用、不用再单独下基座；adapter/ 子目录是同一模型的 LoRA 适配器（102.4 MB），可加载到自备的 Qwen3.5-0.8B 上。两种形态共用本仓的 7 个判断头与校准文件，输出一致。输入一个消息窗口（前文 + 目标消息），一次前向给出七项判断；在 RTX 4090 D 上单窗口延迟中位 41.0 ms（batch 1，≤512 token）。

判断层，不是决策层。 本模型回答的是带类型的问题（这条消息属于哪类、需关注的可能性多大、含不含待办），输出的是经校准的判断分数；它自己不做决策、不执行动作。决策——哪条冒泡、建不建任务、升级还是忽略——由外围系统做：你设定的确定性规则、注意力预算与阈值，以及最终的人。把概率模型放在判断层、把决策放在确定性代码（或人）手里，是有意为之：这样这些分数才可以放心地被自动化流程使用。

这个模型是为了解决什么

群聊每天产生的消息远超任何人能读完的量。真正稀缺的不是信息，而是注意力：几百个窗口里，哪些值得本人看、值得回复、值得跟进？

目标：把每一个消息窗口变成结构化、经校准的判断信号，快到、便宜到可以对全部窗口都跑一遍，且全程在本地完成——在人（或大模型）花注意力之前先做完分诊。与其让大模型以每窗口约 1.5k token 的成本逐条读，不如用一个由大模型蒸馏出的 0.8B 专用小模型在自家硬件上 40 ms 做完常规判断，只把被它标出来的少数窗口留给细看。

它具体干什么

每个窗口一次前向输出七项经校准的判断（定义见上方输出节）：七分类、情绪、重要/紧急/情绪强度打分、需关注概率、含待办概率。下游软件可以直接拿去用：

• 注意力分诊——按校准后的需关注概率给每个人当天的窗口排序，只冒泡前几条（注意力预算），而不是未读红点大水漫灌。
• 待办挖掘——含待办概率高的窗口自动变成任务候选（附负责人和时间点待确认），而不是让行动项沉进聊天记录。
• 分流归档——分类头把窗口分进求助/任务/决策/信息/讨论等流，供看板、日报摘要和检索使用。
• 优先级队列——重要 × 紧急决定跟进顺序和升级检查。
• 团队情绪监测——情绪与强度按天聚合出脉搏（消极强度突增看得见，不必去翻原始聊天）。
• 大模型的前置过滤器——在 LLM 流水线里先用它近乎零边际成本筛全部窗口，只把标出的送大模型深读或起草。下节的对比数据就是在量化这笔账。

适合与不适合的场景

适合

• 企业 IM 大流量分诊：每天几千个窗口，没人能逐条读完。
• 团队聊天产品里的通知优先级排序与每日摘要。
• 待办/行动项抽取流水线与跟进提醒。
• 客服、运营类群的情绪与升级监测。
• 聊天内容不能出内网的本地/边缘部署（本模型全程本地运行，不调任何 API）。
• 大模型流水线的高流量第一道筛——逐窗口 API 成本会吃掉预算的那种。

不适合

• 开放式阅读理解、总结、代写回复——它只输出判断、不生成文本，这类活请搭配大模型。
• 中文企业群聊之外的语言、平台或聊天文化（它的训练域）。
• 判断它看不见的内容：附件、图片、链接、引用文档在窗口里只是占位符。
• 独自裁决重要事项。「需关注」是蒸馏来的、带主观性的判断，且已接近其标签噪声天花板——请用它排序和过滤，把被标出的少数留给人或大模型定夺。

性能与评测速览

项目	数值
仓库总大小	1.83 GB——整包权重 1.71 GB + LoRA 适配器 0.10 GB，其余为 tokenizer、决策头与校准文件
延迟	单窗口中位 41.0 ms、p90 44.1 ms（RTX 4090 D，batch 1，≤512 token，7 个头一次前向）
分类 macro-F1	0.611（7 类）· 0.673（support ≥10 的类）
需关注	AUC 0.822 · ECE 0.053（校准后；RTX 4090 D，torch 口径）
含待办	AUC 0.927 · ECE 0.069（校准后；RTX 4090 D，torch 口径）
情绪准确率	92.8%
重要 / 紧急 ±1 内	99.9% / 97.7%

分数是同一模型在适配器形态下（基座 + LoRA）于下述保密集上实测的。 打包前已核验本包的合并权重与适配器形态的等价性：在 16 条保密集窗口 上，七个头的 logits 最大绝对差为 0.049（bf16 舍入），即整包与 适配器形态在 bf16 精度内输出等价。同一 V2 整包已在 Apple 芯片 Mac（MPS、bf16）上用同一 1,000 条保密集全量实测：F1（support ≥10）0.671、需关注 AUC 0.822、两二值头 ECE 0.055 / 0.070、延迟中位 48.2 ms（p90 81.6 ms）、与 4090 分类一致率 99.6%——同包同数，两台机器结果一致。

与大模型直判的对比（实测，2026-11-05）

为什么不直接让大模型来做这些判断？这张表是上一版（v1.3）在其冻结保密集（258 窗口）、同一份金标上的实测，V2 未重跑此对比，保留在此只为说明「专用小模型对大模型直判」这笔账的量级；V2 自己的保密集数字见上方速览与下方评测节。大模型基线为 DeepSeek（deepseek-chat API）零样本直判：系统提示给的就是教师打标用的同一份完整打标规范，温度 0。

指标	K-IM-Deiction（v1.3，0.8B）	DeepSeek 零样本（大模型）	差异
输出格式合规率	100%（结构化决策头，构造上保证）	100.0%（258/258 合法 JSON）	0%（持平）
分类 macro-F1（7 类）	0.637	0.620	+2.7%
分类 macro-F1（support ≥10）	0.744	0.724	+2.8%
需关注 AUC	0.801	0.781	+2.6%
含待办 AUC	0.965	0.959	+0.6%
情绪准确率	89.1%	94.2%	−5.4%
重要 / 紧急 ±1 内	100% / 99.6%	98.8% / 98.5%	+1.2% / +1.1%
单窗口延迟	40.3 ms 中位（本地，RTX 4090 D）	1.91 s 中位（API 往返，含网络）	−97.9%
单窗口 token	平均处理 186（中位 139；窗口左截断至 512），生成 0——它输出的是数字，不是文本	平均消耗 1,496（入 1,312 + 出 184）；258 个窗口共 385,961	−87.6%
数据暴露	推理时全程本地，窗口不出机器	每个窗口都要发到云端 API	—

差异 =（本模型 − 大模型）/ 大模型。正值表示本模型更高；延迟与 token 的负值表示本模型更低（更好）。

怎么读这张表。 在它被蒸馏来做的分诊判断上，这个 0.8B 专用模型与大模型零样本直判打平或略胜——而延迟只有约 1/47、边际成本为零、推理全程在本地。大模型在情绪上明显更好（94.2% 对 89.1%），开放式阅读与起草也仍是大模型的活；专用小模型是大模型前面的过滤器，不是替代品。

关于 token。 token 是任何 Transformer 处理文本的单位，本模型也有——区别在多少、用来干什么。它读一个窗口（截断后平均 186 token，本保密集实测）且生成 0 token，因为它的输出是决策头给出的数字，不是文本。而提示词路线每个窗口都要重发一遍约 1,000 token 的打标规范（其平均入 1,312 = 窗口 + 规范），再生成约 184 token 的 JSON。所以即便把本模型部署到按 token 计费的云端推理服务，一个窗口也只消耗约 186 个输入 token、别无其他——规范已经长在它的权重里，不用每次重读。

另外两个同样实测的参照点：

• 用同一份规范提示未训练的 Qwen3.5-0.8B 基座，258 个窗口里可解析率仅 **1.6%**——小模型靠提示词干不了这活，能力来自蒸馏。
• 产出这批金标的教师（GLM-5.3）在 v1.3 打标轮的用量为每窗口约 1,526 token（3,440,553 token / 2,255 窗口）——与 DeepSeek 基线同一成本量级，区别是它只在建数据集时付一次，而不是每个窗口永远付。

内容

文件	说明
model.safetensors	合并后的完整权重（Qwen3.5-0.8B 基座 + V2 LoRA 已合并）
adapter/adapter_model.safetensors	同一模型的 LoRA 适配器（r=64、alpha=128、dropout 0.05；只训语言模型投影层），与合并权重二选一
adapter/adapter_config.json	适配器的 PEFT 配置（基座 Qwen/Qwen3.5-0.8B）
config.json / generation_config.json	模型配置（Qwen3.5 原厂）
tokenizer.json、tokenizer_config.json、vocab.json、merges.txt、chat_template.jinja	基座 tokenizer 文件，自 Qwen/Qwen3.5-0.8B 原样复制
preprocessor_config.json、video_preprocessor_config.json	Qwen3.5 原厂处理器配置
mlx/	MLX 版（Apple Silicon 用；瘦身纯文本权重、heads.npz、推理与服务脚本），见下文 MLX 版一节
heads.pt	7 个判断头（1024 维池化隐状态上的普通 nn.Linear）及 hidden/base 元数据
temps.json	各头的校准温度
calib.json	分类头的 bias+温度参数、两个二值头的 isotonic 校准表
LICENSE	基座的 Apache-2.0 许可证文本

输入格式

纯文本窗口：目标消息之前是 30 分钟内最多 10 条前文，发送者一律脱敏为 S#### 编号，整体左截断至 512 token。窗口含【前文窗口】与【目标消息】两行标记，与训练格式完全一致。示例（虚构）：

群：【示例】项目周会群
【前文窗口】
[S0002] 明天下午三点过一下发布清单
[S0005] 好的，我把清单整理成文档发出来
【目标消息】
[S0003] 收到，我今晚把测试报告补齐

输出

• cat 七分类：求助/提问、任务/待办、决策/结论、信息分享、讨论观点、 闲聊寒暄、无关/其他
• sen 三分类：积极 / 中性 / 消极
• imp / urg / inten：重要性、紧急性、情绪强度，各为 1–5 序数软分布
• att：是否需要用户关注（二值，概率经校准）
• todo：是否含待办（二值，概率经校准）

使用代码见上方 Usage：整包直接 from_pretrained 加载，再取最后一个 非填充 token 的隐状态过 7 个头，按 temps.json / calib.json 做温度 与 isotonic 校准。适配器形态见上方 Adapter form 一节：基座加载自备的 Qwen/Qwen3.5-0.8B，用 PeftModel 从本仓 adapter/ 子目录挂适配器， 其余步骤相同。

本地服务方式

本模型不是生成式模型，Ollama、vLLM 这类生成推理服务套不上——它们管不了自定义决策头和校准逻辑。正确形态是把上面的加载代码包一层薄 HTTP 服务：模型与决策头加载一次，对外提供 JSON 接口。作者自己的部署是训练机（RTX 4090 D）上一个零依赖的 Python 小服务，契约如下：

• GET /health：服务状态
• POST /decide，body {"text": "<窗口>"}：返回单个窗口的判断（带字段名：分类与情绪分布及 top1、三个序数分布及期望分、校准后的需关注与含待办概率）
• POST /decide_batch，body {"texts": ["<窗口>", "..."]}：返回 {"results": [...]}

HTTP 层实测：小批量端到端约 0.1 秒/窗口（其中纯前向 41.0 ms）。作者的助手环境中另有一份调用该服务的分诊 Skill（窗口构造、注意力阈值、每日摘要流程）；第三方集成只需要上面的接口契约。

MLX 版（Apple Silicon）

本仓的 mlx/ 子目录是同一款 V2 模型的 MLX 原生移植版，面向 Apple Silicon Mac，基于 mlx-lm 0.32.2（支持 Qwen3.5 文本架构；vision 部分与 torch 版一样不使用）。子目录内含瘦身权重文件（剔除 153 个 vision 权重键、1,504,827,581 字节；320 个文本键与合并整包一致）、判断头的 heads.pt 与 heads.npz 两种形态、calib.json / temps.json、tokenizer 全套，以及两个脚本：mlx_decision.py（完整推理链：左截断 512 token 分词、末 token 隐状态、7 个判断头、温度 + isotonic 校准）与 serve_decision_mlx.py（零依赖 HTTP 服务，/health、/decide、/decide_batch 契约与 torch 版服务一致）。细节见 mlx/README_MLX.md。

在作者 Mac 上、同一批冻结保密集 1,000 窗口实测：macro-F1（support≥10）0.673、需关注 AUC 0.822、两二值头 ECE 0.050 / 0.067（ECE 按平台标注：RTX 4090 D 的 torch 整包为 0.053 / 0.069）；与同机 torch-Mac 版的分类一致率 99.6%（bf16 舍入量级）。延迟中位 13.5 ms、p90 23.5 ms每窗口（含分词；瘦身包单独口径为 14.9 / 24.8 ms），比同机 torch MPS 版（48.2 / 81.6 ms）快约 3.6 倍。用时下载 mlx/ 子目录、装好 mlx 与 mlx-lm 后，手动在前台启动服务（有意不做常驻守护）：

pip install mlx mlx-lm numpy
python serve_decision_mlx.py --package ./mlx --port 8765

资源占用（2026-11-07 实测）

两个版本都小到可以常驻后台运行。口径：batch = 1、bf16 权重、预热推理后的稳态：

版本	机器	进程内存（RSS）	显存 / 统一内存	磁盘占用
PyTorch 整包（本仓根目录）	Windows 机器，WSL2，RTX 4090 D	加载后约 1.1 GB，稳态约 1.8 GB	显存约 2.2 GB	1.83 GB
MLX 版（mlx/ 瘦身包）	作者的 Mac（Apple Silicon）	进程总占用约 1.7 GB（统一内存）	MLX 活跃约 1.5 GB、峰值约 1.6 GB	1.53 GB

7 个判断头与校准文件只有几 MB，已包含在上表数字内。PyTorch 版另需 PyTorch 运行时本身的磁盘空间；MLX 版另需 mlx / mlx-lm。

评测（保密集 1,000 窗口，2026-11-06 实测）

保密集为 1,000 个窗口，来自上一版从未见过的 7 个新群；按预注册的复核 设计在 V2 训练前冻结，从未用于训练、校准或选型。金标为教师标签 （GLM-5.3 主标，按冻结的打标规范产出软分布；DeepSeek 同集复标用于 一致率报告）。训练集 6,580 个窗口。

指标	结果
分类 macro-F1（7 类）	0.611
分类 macro-F1（support ≥10 的类）	0.673
逐类 F1	求助 0.836（268）· 任务 0.754（239）· 决策 0.462（101）· 信息 0.672（182）· 讨论 0.694（158）· 闲聊 0.620（43）· 无关 0.235（support 9）
需关注 AUC	0.822
需关注正例数	1,000 窗口中 247 条（24.7%）
需关注 ECE（校准后；RTX 4090 D，torch 口径）	0.053
含待办 AUC	0.927
含待办 ECE（校准后；RTX 4090 D，torch 口径）	0.069
分类置信 ECE	0.095（仅作诊断，见下）
情绪准确率	92.8%
重要 / 紧急 ±1 内	99.9% / 97.7%
延迟	中位 41.0 ms、p90 44.1 ms（RTX 4090 D，batch 1）
需关注排序质量	Precision@10 0.70 · Precision@20 0.80 · Precision@50 0.76

指标怎么读（大白话）。

• AUC —— 排序准不准。 随机抽一条真正需要关注的消息、再抽一条不需要的，模型给前者打分更高的概率就是 AUC。0.5 等于抛硬币，1.0 是完美排序。上表的 0.822 即约 82% 的时候能把该看的排在前面。这类主观判断连两位教师在同一批窗口上对「需关注」的一致率也只有约 79%（本集实测），实际上限约 0.85。
• ECE —— 概率虚不虚。 把模型说「70% 需要关注」的所有消息挑出来，看实际上是不是真有约 70% 需要关注；ECE 就是各概率档上「说的」与「实际的」平均偏差。上表的 0.053 / 0.069 即报出的概率平均只偏 5–7 个百分点，拿它当阈值旋钮用是可信的。

对照：未训练的基座模型被直接要求按此标签体系输出时，上一版 258 窗口 保密集上可解析率仅 **1.6%**。

关于发布门线。 原门线（macro-F1 ≥0.65、需关注 AUC ≥0.85、ECE ≤0.08） 定于标签质量被测量之前。试点中第二位教师对「需关注」的复标与主教师 一致率只有 75%；在错误对称独立的假设下，以单教师标签为金标可达的 AUC 上限约为 0.854——原 0.85 门线基本压在标签噪声天花板上。 2026-11-05 门线经修订并留档（macro-F1 改为 support 感知口径 ≥0.65； 需关注 AUC 改 ≥0.80；ECE 门线只保留两个二值头，分类 ECE 因其拟合的是 教师软分布而降为诊断指标）。v1.3 在修订门线下贴线通过（需关注 AUC 0.801，正样本仅 35 条），明确记为须用更大的保密集复核；该复核已在本 集（1,000 窗口、7 个未见新群）完成：v1.3 需关注 AUC 0.807，按预 注册规则判定贴线结论确认、不回退。V2 在同一集上三项修订门线全过： F1（support ≥10）0.673 ≥0.65、需关注 AUC 0.822 ≥0.80、两二值头 ECE 0.053 / 0.069 ≤0.08。

训练数据与隐私

• 训练数据为企业内部群聊（中文）的 6,580 个消息窗口（v1.3 训练集 + 新打标 1,507 个，来自新增群与难例挖掘）；发送者在打标与训练前已全部替换为假名编号（S####）；标签由教师模型（GLM-5.3）按冻结规范产出，为软分布。V2 保密集 1,000 窗口从未进入训练。
• 原始数据从未公开发布。 本仓库不含任何消息原文、群名或个人标识；上方示例窗口为虚构；tokenizer 文件为 Qwen 原厂文件、原样复制。
• 标注阶段确有第三方 API 处理。 数据集构建期间，窗口文本（发送者已换为 S#### 假名，但正文当时为原文）经 BigModel 接口（GLM-5.3，主教师）与 DeepSeek 接口（二标与大模型对照）处理。使用本模型的人应当知道，其训练标签是这样产出的。模型本身的运行则完全在本地（见上方对比表）。
• 留存的本地副本已于 2026-11-07 完成正文级脱敏：正文中的人名（取自消息元数据、@ 提及与结构化姓名标注）、电话、公司名、身份证号与邮箱，已在训练集、全部保密/评测集与原始拉取数据中统一替换为占位符（合计约 12.7 万处；脱敏后对上述模式的复扫为零命中）。窗口头仍为真实群名，正文中的昵称类称呼无法保证穷尽——因此本地数据集准确的说法是已假名化并经正文脱敏，但不宣称可证明的完全匿名化；这也是它们只存放在作者机器上、从不对外发布的原因。
• 留存副本的清洗不能追溯改变已经训练好的权重。本模型只输出数字、不生成文本，但对权重不宣称零泄漏风险。
• 开发中实测无效、已放弃的路线：类加权/过采样（v1.1，指标反降）、继续深拉同一批群的历史（v1.3 AUC 仅 +0.010）、更换分类校准方法（三种候选统计并列）；V2 的难例挖掘最终只捞到 20 条可用候选（收效有限）。最终发布的是最朴素的配方。

已知限制

• 领域专用：上述格式的中文企业群聊窗口；未在其他语言、平台或聊天 风格上评测。
• 「需关注」是蒸馏来的、带主观性的判断，且已接近其标签噪声天花板—— 请把它当作分诊排序信号，而不是「什么重要」的真值。
• 「无关/其他」在保密集中仅 9 条，其 F1（0.235）噪声大；「决策/结论」 仍是主要类别里最弱的一类（F1 0.462，support 101），主要漏到信息分享、 讨论观点与任务/待办——多为讨论中夹带的隐性结论。
• 各项分数继承教师模型的偏见与打标规范的定义；换一套规范（例如更 严格的「需关注」定义）会得到不同的模型。

下一步计划

分三期推进。每项实验的验收标准在开跑前按当时的实际损失与预算确定，这里不预设拍脑袋的硬门槛。

第一期——不需要新训练。

• decision 边界仲裁：先厘清隐式拍板、待确认、否决/撤回等边界情形的标注口径并冻结，再谈补决策类样本；补例来源必须在评测池之外。
• 用户日级影子试验设计：保持每天 20 条的「需关注」预算与 Mac 手动启动方式不变，设计影子运行要记录的项（逐日 Precision@K、Recall@K、关键任务漏报、重复通知、阅读时间），并设规则基线（@ 提及/明确指派/期限）作对照。
• 人工锚点小样：双人盲标/仲裁、标注者不先看教师答案，校准「通用关注价值」的定义；样本量按正例数与区间宽度定，不机械凑整。
• 评测隔离制度化：下一轮起开发集、校准集、最终盲测集分开；已用于 V2 诊断的冻结 1,000 集降为回归集，不再充当新的盲测。

第二期——视第一期结果决定。

• 校准与低置信升级：在独立校准集上拟合校准；为二值头补报 PR-AUC、Brier 分数与可靠性图；低置信升级通道按风险—覆盖权衡评估，默认走人工或经授权的本地路径，绝不自动把群聊发往云端。
• 个性化条件预研：研究相对目标用户的关系信号（@ 提及、角色）如何条件化，并让标签定义同步条件化，辅以反事实检验。

第三期——有实验结论再定。

• 按剩余错误定向补训；结构消融逐项做（池化、序数损失、多教师、新头、量化），不捆绑上。

许可证

Apache-2.0，随 Qwen3.5 基座模型。