---
license: apache-2.0
base_model: Qwen/Qwen3-4B-Instruct-2507
library_name: peft
pipeline_tag: text-generation
language:
  - zh
  - en
tags:
  - lora
  - peft
  - qwen3
  - slide-creator
  - kai-report-creator
---

# kai-skill LoRA adapters

Two LoRA adapters fine-tuned on **Qwen/Qwen3-4B-Instruct-2507** for two content
generation Skills: `slide-creator` (slide decks) and `kai-report-creator`
(structured reports). They generate each Skill's intermediate representation
(IR / BRIEF) from a task prompt, which the Skill's own deterministic renderer
then turns into the final artifact.

## Contents

- `content/` — content adapter (`4090-qlora-full-v4fix3`): writes the IR for
  both Skills.
- `router/` — router adapter (`4090-router-lora-fix1`): routes a request to
  the right Skill (including rejecting out-of-scope requests).

These are LoRA adapters, not a full model — load them on top of the base
model. Each subfolder contains `adapter_model.safetensors`,
`adapter_config.json`, and the base model's tokenizer files.

## Evaluation (gate-3, held-out secret set, 60 cases)

Evaluation calibre: kai-report-creator v1.28.1, slide-creator v2.34.2.
The held-out set was never used in training.

- Routing: **60/60** (within this 60-case set)
- IR first-pass validation: **46/52 (88.5%)**, against the pre-set ≥45/52
  release gate
  - slide: 26/26 (24/26 under the stricter production pre-write calibre)
  - report: 20/26

## Small model vs. large model (measured 2026-11-01)

Same 60-case held-out set, same validators, three configurations. The large
model is GLM-5.3 — the teacher that generated this model's training data
(BigModel coding endpoint, temperature 0.3, reasoning effort low).

| | This model (Qwen3-4B + LoRA) | GLM-5.3, bare prompts | GLM-5.3, full Skill run |
|---|---|---|---|
| Routing | 60/60 | 0/60 | 60/60 |
| IR first-pass | **46/52 (88.5%)** | 0/52 | 45/52 (86.5%) |
| IR after repair | — (one-shot) | — | 52/52 |
| Slide strict tier | 24/26 | — | 26/26 |

- **Bare prompts**: GLM-5.3 received exactly this model's evaluation inputs,
  with no format specification. It answered the requests directly (final
  HTML) and never entered the routing/IR contract: 0/60 and 0/52. The
  contract is not emergent — it has to be supplied in the prompt or trained
  into the model.
- **Full Skill run**: GLM-5.3 received the complete Skill contract in
  context (routing rules, the IR format specification, the BRIEF template /
  report example — its own training-data production prompts) and up to 2
  rounds of validator-feedback repair. Under the full Skill contract, on
  the same set and the same validators, its first-pass quality is close
  to the fine-tuned 4B in observed values (45 vs 46 of 52; observed
  closeness is not a claim of statistical equivalence, and neither side
  is claimed to lead); per skill — slide 24/26 vs 26/26,
  report 21/26 vs 20/26). It pulls ahead only through the repair loop: all
  7 first-pass failures were fixed in a single repair round, reaching 52/52. The LoRA model's own repair-loop effect has not been measured.
- **Cost**: the full GLM-5.3 run used a median of 15.7 s of API time and
  ~3.0k tokens per triggered case (157,153 tokens for the whole set). This
  model generates ~1.5k output tokens per triggered case locally (≈29
  routing + ≈1.5k IR; estimated by tokenizing its trained output
  distribution with this model's tokenizer), at a measured median of
  77.7 s of routing + IR generation per triggered case on the evaluation
  RTX 4090 (routing 2.1 s + IR 75.6 s, 4-bit, single-stream) — this
  figure covers routing and IR generation only, not the full
  render-and-deliver pipeline — not faster
  than the API, but with no per-call cost, no quota and no network
  dependency.

## Known limitations

Accepted with the release decision (2026-11-01); no further fix round was run.

- report-05 / 09 / 17: stubborn kpi-semantics failures (kpi written as prose,
  no real number) — survived two rounds of targeted repair data.
- report-18 / 26: chart syntax — illegal chart `type=table`.
- report-24: chart syntax — bar block missing a required marker.
- slide-01 / 09: pass ordinary validation but are rejected by the stricter
  production pre-write gate; a production-gate issue, not a model failure on
  the ordinary calibre.

## Usage

```python
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3-4B-Instruct-2507")
tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen3-4B-Instruct-2507")

# Content generation
model = PeftModel.from_pretrained(base, "Songkai/kai-skill-lora", subfolder="content")
# Routing: subfolder="router"
```

## Roadmap

Next steps, in phases. Acceptance criteria are fixed before each experiment, from the actual costs and budget at that point — no hard numeric gates are pre-committed here.

**Phase 1 — latency profile and production-contract alignment (no new training).**
- Segmented latency profile: record prefill / decode time, true token counts, adapter loading and switching, validation and rendering separately, to establish what the median ~78 s of routing + IR generation per case is actually made of. No speed-up figure is promised in advance.
- Production-contract alignment: unify the routing / IR / strict-validation / rendering-acceptance contract and its version notes; then have a non-author run the whole pipeline from the documentation alone and record at which layer any failure occurs.

**Phase 2 — content loop and inference stack (driven by Phase 1 evidence).**
- E1, content-loop experiment: base/LoRA × guardrails/repair comparison — chart safety constraints (type enums, required markers, safe normalization) plus one budgeted local validation-repair pass. Where no real KPI data exists, the KPI block stays explicitly missing / to-be-filled; numbers are never fabricated.
- E2, inference-stack comparison: after profiling, compare 1–2 compatible stacks (merged adapters, quantization, an optimized runtime) on the same hardware and the same quality calibre (tokens/s, p50/p95), and re-measure factual quality after any stack change.

**Phase 3 — targeted training.**
- Top-up training aimed at the remaining semantic errors; earlier failure sets are kept as regression sets only.

## License

Apache-2.0, following the Qwen3 base model.

---

## 中文说明

在 **Qwen/Qwen3-4B-Instruct-2507** 上微调的两个 LoRA 适配器，服务于两个内容
生成 Skill：`slide-creator`（幻灯片）和 `kai-report-creator`（结构化报告）。
模型根据任务提示生成各 Skill 的中间表示（IR / BRIEF），再由 Skill 自带的
确定性渲染器生成最终成品。

### 内容

- `content/` —— 内容适配器（`4090-qlora-full-v4fix3`）：为两个 Skill 撰写 IR。
- `router/` —— 路由适配器（`4090-router-lora-fix1`）：把请求路由到正确的
  Skill（含拒绝超出范围的请求）。

这是 LoRA 适配器，不是完整模型，需要加载到基座模型上使用。每个子目录包含
`adapter_model.safetensors`、`adapter_config.json` 和基座模型的 tokenizer 文件。

### 评测（gate-3，60 例保密集）

评测口径：kai-report-creator v1.28.1、slide-creator v2.34.2。保密集从未用于训练。

- 路由：**60/60**（当前 60 例集内结果）
- IR 一次通过：**46/52（88.5%）**，发布门槛为 ≥45/52
  - slide：26/26（生产严格口径 24/26）
  - report：20/26

### 与大模型的对比（2026-11-01 实测）

同一 60 例保密集、同一校验器，三种配置。大模型为 GLM-5.3（即生成本模型
训练数据的教师；BigModel coding 端点，temperature 0.3，reasoning effort low）。

| | 本模型（Qwen3-4B + LoRA） | GLM-5.3 裸提示 | GLM-5.3 完整跑 |
|---|---|---|---|
| 路由 | 60/60 | 0/60 | 60/60 |
| IR 一次通过 | **46/52（88.5%）** | 0/52 | 45/52（86.5%） |
| 返修后通过 | —（一次直出） | — | 52/52 |
| slide 严格口径 | 24/26 | — | 26/26 |

- **裸提示**：GLM-5.3 拿到与本模型评测完全相同的输入、不给任何格式说明。
  它把请求当普通需求直接作答（产最终 HTML），根本没进入路由/IR 契约：
  0/60、0/52。契约不会凭空出现——要么写进提示，要么训进模型。
- **完整跑**：GLM-5.3 在上下文中拿到全套 Skill 契约（路由规则、IR 格式
  规范、BRIEF 模板/报告示例，即其产训练数据时的生产提示词），并可按校验器
  报错返修最多 2 轮。在完整 Skill 契约、同集同验证器下，其一次通过的观察值与微调后的 4B 接近（45 对 46；不宣称统计等效、也不宣称任一方领先；
  分技能：slide 24/26 对 26/26，report 21/26 对 20/26）。它只靠返修环节
  胜出：首轮失败的 7 例各返修 1 轮即全部修好，达到 52/52。本地模型自身的返修效果尚未测。
- **成本**：GLM-5.3 完整跑每触发例中位 15.7 秒 API 时间、约 3.0k tokens
  （全集 157,153 tokens）。本模型每触发例本地生成约 1.5k 输出 tokens
  （路由 ≈29 + IR ≈1.5k；以本模型 tokenizer 对其训练输出分布估算），
  在评测机 RTX 4090 上路由＋IR 生成中位 77.7 秒/例（路由 2.1 秒 + IR 75.6 秒，
  4-bit、单流，实测；此口径只含路由与 IR 生成，不含完整渲染交付）——单例速度不占优，但没有 API 费用、额度和网络依赖。

### 已知限制

以下问题随 2026-11-01 的发布决定一并接受，不再追加修复轮次。

- report-05 / 09 / 17：kpi 语义顽固失败（kpi 写成文字、没有真实数值），
  两轮定向修复数据未能解决。
- report-18 / 26：chart 语法问题，chart `type=table` 不合法。
- report-24：chart 语法问题，bar 块缺少必备标记。
- slide-01 / 09：普通校验通过，但被更严格的生产预写门拒绝；属生产侧问题，
  不是模型在普通口径下的失败。

### 使用方式

见上方 Usage 代码示例（内容生成用 `subfolder="content"`，路由用
`subfolder="router"`）。

### 下一步计划

分三期推进。每项实验的验收标准在开跑前按当时的实际成本与预算确定，这里不预设拍脑袋的硬门槛。

**第一期——时延 profile 与生产契约对齐（不需要新训练）。**
- 分段时延 profile：把 prefill/decode、真实 token 数、适配器加载与切换、校验、渲染分别计时，定位每例路由＋IR 生成中位约 78 秒的构成；不预先承诺提速数字。
- 生产契约对齐与非作者试跑：统一路由/IR/严格校验/渲染验收的契约与版本说明；再请一位非作者只按说明把全流程跑一遍，记录失败发生在哪一层。

**第二期——内容闭环与推理栈（由第一期证据驱动）。**
- E1 内容闭环实验：基座/LoRA × 约束/修复对照——图表安全约束（类型枚举、必需标记、安全规范化）加一次有预算的本地校验修复；KPI 没有真实数据时就明确写缺失/待补，绝不编数。
- E2 推理栈比较：profile 之后选 1–2 种兼容栈（适配器合并、量化、优化运行时），在同硬件、同质量口径下比 tokens/s 与 p50/p95；换栈后重新实测事实质量。

**第三期——定向补训。**
- 按剩余语义错误定向补训练数据；旧失败集只作回归集使用。

### 许可证

Apache-2.0，随 Qwen3 基座模型。