onw 0.12.0: Ornith-1.5-9B, earlier turns' reasoning_content to the template, --deltanet-fmt q8g128

CHANGELOG.md

@@ -5,6 +5,14 @@
 日付は日本時間です。各版は Hugging Face の `ryugyosoft/onw` にタグ（`v0.4.2` 以降）付きで残っています。
 onw ウィンドウは、新しい版があるとこのファイルから変更点を表示します（`## v…` の見出しで区切っています）。
 
+## v0.12.0 (2026-10-01)
+- **Ornith-1.5-9B に対応**（ornith-ai、MIT）。Qwen3.5-9B をエージェント・ツール呼び出し向けに鍛えたモデルで、モデル一覧に追加しました。
+  構造は Qwen3.5 と同じで、テキストと画像に対応します。MoE の 35B-A3B は NPU での精度を調整中で、次の版で追加する予定です。
+- `onw convert --deltanet-fmt q8g128`：Qwen3.5／3.6 系の DeltaNet の `in_proj_qkv`・`out_proj` を INT8 にする変換オプション
+  （量子化に敏感なモデル向け。INT4 と同じグループ 128 の形にしたので、チャネル単位 INT8 よりずっと速い）。
+- 履歴の `reasoning_content`（前のターンの思考）をテンプレートに渡すように。Ornith のテンプレートは過去のターンの思考も残す作りで、
+  エージェント用途ではこれが効きます（Qwen は最新のターンだけ、gpt-oss は `thinking` として渡ります）。
+
 ## v0.11.3 (2026-09-30)
 - Ubuntu：ランチャー（アプリ一覧の onw）から起動するとすぐ落ちる問題を修正。GNOME はランチャーの中身をログインし直すまで
   使い回し、しかもホームフォルダで起動するため、`python -m onw` がインストール先の `~/onw` を空の `onw` パッケージとして

CHANGELOG_en.md

@@ -4,6 +4,15 @@
 
 Dates are JST. Every version is on Hugging Face `ryugyosoft/onw`, tagged from `v0.4.2` on.
 
+## v0.12.0 (2026-10-01)
+- **Ornith-1.5-9B** (ornith-ai, MIT): Qwen3.5-9B trained further for agents and tool calls, added to the model list.
+  Same structure as Qwen3.5, text + image. The 35B-A3B MoE is still being tuned for accuracy on the NPU and should come
+  in the next version.
+- `onw convert --deltanet-fmt q8g128`: INT8 DeltaNet `in_proj_qkv` / `out_proj` for Qwen3.5 / 3.6-family models
+  (for quantization-sensitive ones; in INT4's group-128 layout, much faster than per-channel INT8).
+- `reasoning_content` in the history (an earlier turn's thinking) reaches the template. Ornith's template keeps earlier
+  turns' thinking, which agents rely on (Qwen keeps the latest turn's; gpt-oss gets it as `thinking`).
+
 ## v0.11.3 (2026-09-30)
 - Ubuntu: starting onw from its launcher crashed at once. GNOME keeps a launcher's command until the next login and
   starts it in the home folder, where `python -m onw` imported the install folder `~/onw` as an empty `onw` package that

README.md

@@ -106,6 +106,7 @@ Python や git の知識は要りません。Python が入っていなければ
 | [ryugyosoft/Qwen3.5-9B-onw](https://huggingface.co/ryugyosoft/Qwen3.5-9B-onw) | `qwen3_5`（Gated DeltaNet＋ゲート付きアテンション、dense） | テキスト＋画像 | 4.1 tok/s | 5.3 GB | 約 11 GB |
 | [ryugyosoft/Qwen3.6-REAP-18B-A3B-onw](https://huggingface.co/ryugyosoft/Qwen3.6-REAP-18B-A3B-onw) | `qwen3_5_moe`、256 個中 128 個のエキスパートを残す（REAP） | テキスト＋画像 | 6.8 tok/s | 9.6 GB | 約 12 GB |
 | [ryugyosoft/Qwen3.6-35B-A3B-onw](https://huggingface.co/ryugyosoft/Qwen3.6-35B-A3B-onw) | `qwen3_5_moe`（Gated DeltaNet＋ゲート付きアテンション＋MoE 256 個から 8 個＋共有エキスパート） | テキスト＋画像 | 6.8 tok/s | 17 GB | 約 20 GB（SSD から読めば 8〜13 GB） |
+| [ryugyosoft/Ornith-1.5-9B-onw](https://huggingface.co/ryugyosoft/Ornith-1.5-9B-onw) | `qwen3_5`（Qwen3.5-9B をエージェント・ツール呼び出し向けに鍛えた Ornith-1.5） | テキスト＋画像 | 4.9 tok/s | 5.3 GB | 約 11 GB |
 | [ryugyosoft/gpt-oss-20b-onw](https://huggingface.co/ryugyosoft/gpt-oss-20b-onw) | `gpt_oss`（OpenAI gpt-oss-20b：MoE 32 個から 4 個、attention sink、YaRN、harmony 形式。エキスパートはチャネル単位 INT4） | テキスト | 8.5 tok/s | 11.4 GB | 約 12 GB |
 | [ryugyosoft/gpt-oss-20b-int8-onw](https://huggingface.co/ryugyosoft/gpt-oss-20b-int8-onw) | 同上、エキスパート INT8（bf16 に近い精度） | テキスト | 6 tok/s | 21 GB | 約 21 GB（SSD から読めば 8〜13 GB） |
 

README_en.md

@@ -93,6 +93,7 @@ window keeps the server running; "Unload" stops it, "Quit" ends onw.
 | [ryugyosoft/Qwen3.5-9B-onw](https://huggingface.co/ryugyosoft/Qwen3.5-9B-onw) | `qwen3_5` (Gated DeltaNet + gated attention, dense) | text + image | 4.1 tok/s | 5.3 GB | ~11 GB |
 | [ryugyosoft/Qwen3.6-REAP-18B-A3B-onw](https://huggingface.co/ryugyosoft/Qwen3.6-REAP-18B-A3B-onw) | `qwen3_5_moe`, 128 of 256 experts kept (REAP) | text + image | 6.8 tok/s | 9.6 GB | ~12 GB |
 | [ryugyosoft/Qwen3.6-35B-A3B-onw](https://huggingface.co/ryugyosoft/Qwen3.6-35B-A3B-onw) | `qwen3_5_moe` (Gated DeltaNet + gated attention + MoE 256x top-8 + shared expert) | text + image | 6.8 tok/s | 17 GB | ~20 GB (8-13 GB with experts from the SSD) |
+| [ryugyosoft/Ornith-1.5-9B-onw](https://huggingface.co/ryugyosoft/Ornith-1.5-9B-onw) | `qwen3_5` (Ornith-1.5: Qwen3.5-9B trained further for agents and tool calls) | text + image | 4.9 tok/s | 5.3 GB | ~11 GB |
 | [ryugyosoft/gpt-oss-20b-onw](https://huggingface.co/ryugyosoft/gpt-oss-20b-onw) | `gpt_oss` (OpenAI gpt-oss-20b: MoE 32x top-4, attention sinks, YaRN, harmony format; channel-wise INT4 experts) | text | 8.5 tok/s | 11.4 GB | ~12 GB |
 | [ryugyosoft/gpt-oss-20b-int8-onw](https://huggingface.co/ryugyosoft/gpt-oss-20b-int8-onw) | same with INT8 experts (close to bf16) | text | 6 tok/s | 21 GB | ~21 GB (8-13 GB with experts from the SSD) |
 

TECHNICAL.md

@@ -100,6 +100,14 @@ Qwen3.5／Qwen3.6 は、Gated DeltaNet（線形アテンションの一種で、
 - **大きすぎる値の前処理**：1024 倍した出力を複数トークンのブロックで RMSNorm に入れると、今度は二乗和が FP16 で
   溢れます。x／c を正規化し eps を c² で割る形に前処理しています。
 - 実際の状態の大きさは CPU（FP32）で確かめています（畳み込みの状態の最大値は 14 で、FP16 の限界からは遠い）。
+- **量子化に敏感なモデル（onw 0.12、Ornith-1.5-35B-A3B）**：Qwen3.6-35B-A3B の追加学習版ですが、全重みグループ 128 の INT4
+  では最初のトークンを外しました（NPU でなく CPU の FP32 でも同じ）。bf16 のモデルと層ごとに比べると、層 0 の MoE 入力で
+  すでに 22% ずれていて、その大半が DeltaNet の `out_proj` と `in_proj_qkv` でした（グループ 32 の INT4 でも 19%）。この 2 つ
+  だけを INT8 にすると回答の終わりまで一致します。INT8 は INT4 と同じ **グループ 128 の形（q8g128）** にしています。同じ重みを
+  チャネル単位 INT8 にすると 1 トークンの区間が 157 → 364 ms と倍以上遅くなりましたが、q8g128 なら 195 ms で済みます
+  （`onw convert --deltanet-fmt q8g128`）。
+  35B-A3B は、これとは別に NPU だけで出るずれ（改行などのトークンで残差がとても小さく、RMSNorm の二乗が FP16 の
+  非正規化数に落ちる）を調査中のため、まだモデル一覧には載せていません。
 
 ## 画像エンコーダーまで NPU に載せた経緯
 

TECHNICAL_en.md

@@ -99,6 +99,14 @@ attention 3:1. The NPU computes in fp16, which needed:
 - **Prescaling large values**: with that 1024x output, multi-token blocks overflow the RMSNorm's sum of squares in
   fp16, so the norm is prescaled (normalize x / c with eps / c²).
 - The real state magnitudes were checked on the CPU in fp32 (conv state max 14, far from fp16 limits).
+- **A quantization-sensitive model (onw 0.12, Ornith-1.5-35B-A3B)**: a Qwen3.6-35B-A3B fine-tune, but with every weight
+  in group-128 INT4 it missed the first token (on the CPU in fp32 too, not just the NPU). Compared layer by layer with
+  the bf16 model, layer 0's MoE input was already 22% off, most of it from the DeltaNet `out_proj` and `in_proj_qkv`
+  (group-32 INT4 still 19%). With just those two in INT8 it matches to the end of the answer. The INT8 uses INT4's
+  **group-128 layout (q8g128)**: the same weights in per-channel INT8 made the 1-token segments 157 -> 364 ms, q8g128
+  195 ms (`onw convert --deltanet-fmt q8g128`). The
+  35B-A3B is not in the model list yet: a separate NPU-only drift (line-break tokens have a tiny residual whose squares
+  in RMSNorm fall into fp16's subnormal range) is still being worked on.
 
 ## How the vision encoders got onto the NPU
 

onw/__init__.py

@@ -2,4 +2,4 @@
 
 A small engine that runs LLMs entirely on the Intel NPU: static segment graphs + host-managed state.
 """
-__version__ = "0.11.3"
+__version__ = "0.12.0"

onw/app.py

@@ -16,6 +16,8 @@ CATALOG = [  # repo, title, takes images, download GB, decode speed (NPU 3720),
     ("ryugyosoft/Qwen3.6-REAP-18B-A3B-onw", "Qwen3.6-REAP-18B-A3B", True, 9.6, "6.8 tok/s", "35B MoE を半分に刈り込み"),
     ("ryugyosoft/Qwen3.6-35B-A3B-onw", "Qwen3.6-35B-A3B", True, 17.1, "6.8 tok/s",
      "最大（メモリが足りない PC ではエキスパートを SSD から読みます）"),
+    ("ryugyosoft/Ornith-1.5-9B-onw", "Ornith-1.5-9B", True, 5.3, "5 tok/s",
+     "エージェント・ツール呼び出し向け（Qwen3.5-9B ベース）"),
     ("ryugyosoft/gpt-oss-20b-onw", "gpt-oss-20b", False, 11.4, "8.5 tok/s", "OpenAI の MoE。考えてから答えます（思考モードで思考を表示）"),
     ("ryugyosoft/gpt-oss-20b-int8-onw", "gpt-oss-20b (INT8)", False, 21.0, "6 tok/s",
      "gpt-oss-20b の高精度版（エキスパート INT8。メモリ 24 GB 以上向け）"),
@@ -30,6 +32,7 @@ MEMORY = {
     "ryugyosoft/Qwen3.5-9B-onw": (10.9, 0.0, 16),
     "ryugyosoft/Qwen3.6-REAP-18B-A3B-onw": (12.3, 0.0, 10),
     "ryugyosoft/Qwen3.6-35B-A3B-onw": (19.9, 0.0, 10),
+    "ryugyosoft/Ornith-1.5-9B-onw": (10.9, 0.0, 16),            # the shape of Qwen3.5-9B
     "ryugyosoft/gpt-oss-20b-onw": (12.4, 0.0, 12),
     "ryugyosoft/gpt-oss-20b-int8-onw": (20.6, 0.0, 12),
 }

onw/chat.py

@@ -224,7 +224,11 @@ class ChatEngine:
         """-> token ids (image placeholders expanded), B = end of the conversation before the generation prompt,
         images in order of appearance. Messages may carry tool_calls / tool_call_id / name (tool use)."""
         images, tmpl = [], []
-        extra = lambda m: {k: m[k] for k in ("tool_calls", "tool_call_id", "name") if m.get(k) is not None}
+        # an earlier turn's thinking goes back to the template (Qwen keeps the latest turn's, Ornith every turn's -
+        # agents rely on it); gpt-oss's harmony template calls it "thinking"
+        rc = "thinking" if self.think_style == "harmony" else "reasoning_content"
+        extra = lambda m: {**{k: m[k] for k in ("tool_calls", "tool_call_id", "name") if m.get(k) is not None},
+                           **({rc: m["reasoning_content"]} if m.get("reasoning_content") else {})}
         for m in messages:
             c = m["content"]
             if isinstance(c, str) or c is None:

onw/convert.py

@@ -1,5 +1,5 @@
 """Convert a Hugging Face checkpoint into an onw model directory.
-usage: onw convert HF_DIR OUT_DIR [--sizes 1,16] [--T 1024] [--fmt q4g128|q8] [--expert-bits 4|8] [--graphs-only]
+usage: onw convert HF_DIR OUT_DIR [--sizes 1,16] [--T 1024] [--fmt q4g128|q8] [--deltanet-fmt q8g128] [--expert-bits 4|8] [--graphs-only]
                    [--compact] [--prune SALIENCY.json --keep 0.5]"""
 import argparse, importlib, inspect, json, os, shutil, time
 
@@ -17,6 +17,8 @@ def main(argv=None):
     ap.add_argument("--sizes", default="1,16", help="compiled block sizes (1 = decoding, largest = prompt blocks)")
     ap.add_argument("--T", type=int, default=1024, help="static context length")
     ap.add_argument("--fmt", default=None, help="non-expert weights: q4g128 | q8 (default: the architecture's, q4g128 or q8)")
+    ap.add_argument("--deltanet-fmt", default=None,
+                    help="Qwen3.5 / 3.6: DeltaNet in_proj_qkv / out_proj weights, e.g. q8g128 (Ornith-1.5-35B-A3B)")
     ap.add_argument("--graphs-only", action="store_true", help="rebuild the segment graphs, keep the weight files")
     ap.add_argument("--expert-bits", type=int, choices=(4, 8), default=None,
                     help="MoE expert bank precision (gpt-oss: 4 = 9.6 GB, fits 16 GB machines; 8 = 19 GB, closer to bf16)")
@@ -40,11 +42,14 @@ def main(argv=None):
             raise SystemExit(f"--prune is not supported for {arch}")
         from onw import prune
         prune.SETTING = (a.prune, a.keep)
+    if a.deltanet_fmt and "dn_fmt" not in takes:
+        raise SystemExit(f"--deltanet-fmt is not supported for {arch}")
     if a.expert_bits and "bits" not in takes:
         raise SystemExit(f"--expert-bits is not supported for {arch}")
     t0 = time.time()
     mod.export(a.src, a.out, sizes=tuple(int(x) for x in a.sizes.split(",")), T=a.T, graphs_only=a.graphs_only,
-               **({"fmt": a.fmt} if a.fmt else {}), **({"bits": a.expert_bits} if a.expert_bits else {}))
+               **({"fmt": a.fmt} if a.fmt else {}), **({"bits": a.expert_bits} if a.expert_bits else {}),
+               **({"dn_fmt": a.deltanet_fmt} if a.deltanet_fmt else {}))
     for f in COPY:
         if os.path.exists(os.path.join(a.src, f)):
             shutil.copy(os.path.join(a.src, f), os.path.join(a.out, f))

onw/i18n.json

@@ -61,6 +61,8 @@
 "アンロード（サーバー停止）": "Unload (stop the server)",
 "エキスパートのメモリ": "Expert memory",
 "エラー: {0}": "Error: {0}",
+"エージェント・ツール呼び出し向け": "For agents and tool calls",
+"エージェント・ツール呼び出し向け（Qwen3.5-9B ベース）": "For agents and tool calls (based on Qwen3.5-9B)",
 "キャッシュを削除（{0} GB）": "Delete the compile cache ({0} GB)",
 "キャンセル": "Cancel",
 "コピー": "Copy",
@@ -290,6 +292,8 @@
 "アンロード（サーバー停止）": "卸载（停止服务器）",
 "エキスパートのメモリ": "专家内存",
 "エラー: {0}": "错误：{0}",
+"エージェント・ツール呼び出し向け": "面向智能体与工具调用",
+"エージェント・ツール呼び出し向け（Qwen3.5-9B ベース）": "面向智能体与工具调用（基于 Qwen3.5-9B）",
 "キャッシュを削除（{0} GB）": "删除编译缓存（{0} GB）",
 "キャンセル": "取消",
 "コピー": "复制",

onw/ir.py

@@ -66,7 +66,7 @@ def quant_i8(w):
 
 def weight(w, fmt="q4g128"):
     """Constant weight subgraph for w [out, in] (float array) -> f32 node [out, in].
-    fmt: q4g<N> (INT4 sym, group N), q8 (INT8 per channel), f16."""
+    fmt: q4g<N> (INT4 sym, group N), q8g<N> (INT8 sym, group N), q8 (INT8 per channel), f16."""
     w = np.asarray(w, np.float32)
     o, i = w.shape
     if fmt == "f16":
@@ -83,6 +83,18 @@ def weight(w, fmt="q4g128"):
         c = ops.constant(pack_i4(q.reshape(o, i // g, g)))
         d = ops.multiply(ops.convert(c, ov.Type.f16), ops.constant(s[..., None]))
         return ops.convert(ops.reshape(d, i64([o, i]), False), F32)
+    if fmt.startswith("q8g"):
+        # INT8 in q4g's grouped layout: on NPU 3720 a 1-token Qwen3.5 segment with its DeltaNet qkv / out weights in
+        # q8g128 took 195 ms against 157 ms all-q4g128, the same weights in per-channel q8 364 ms
+        g = int(fmt[3:])
+        while i % g:
+            g //= 2
+        x = w.reshape(o, i // g, g)
+        sc = np.abs(x).max(-1, keepdims=True) / 127.0
+        sc[sc == 0] = 1e-8
+        q = np.clip(np.round(x / sc), -127, 127).astype(np.int8)
+        d = ops.multiply(ops.convert(ops.constant(q), ov.Type.f16), ops.constant(sc.astype(np.float16)))
+        return ops.convert(ops.reshape(d, i64([o, i]), False), F32)
     raise ValueError(fmt)
 
 

onw/models/qwen3_5_moe.py

@@ -37,7 +37,8 @@ def rotary(cfg):
     return int(cfg["head_dim"] * frac), rp.get("rope_theta", cfg.get("rope_theta", 10000.0))
 
 
-def build_segments(cfg, W, S, T, fmt="q4g128"):
+def build_segments(cfg, W, S, T, fmt="q4g128", dn_fmt=None):
+    """dn_fmt: format of the DeltaNet in_proj_qkv / out_proj weights (default fmt); see export()."""
     H, E, K, I = cfg["hidden_size"], cfg.get("num_experts", 0), cfg.get("num_experts_per_tok", 0), cfg.get("moe_intermediate_size")
     nh, nkv, D = cfg["num_attention_heads"], cfg["num_key_value_heads"], cfg["head_dim"]
     rot = rotary(cfg)[0]
@@ -53,7 +54,7 @@ def build_segments(cfg, W, S, T, fmt="q4g128"):
 
     def deltanet(l, x, params, outs, shared):
         a = f"{P}layers.{l}.linear_attn."
-        qkv = ir.linear(x, W[a + "in_proj_qkv.weight"], fmt)                   # [1,S,2*kh*dk + vh*dv]
+        qkv = ir.linear(x, W[a + "in_proj_qkv.weight"], dn_fmt or fmt)         # [1,S,2*kh*dk + vh*dv]
         C = 2 * kh * dk + vh * dv
         z = ops.reshape(ir.linear(x, W[a + "in_proj_z.weight"], fmt), i64([1, S, vh, dv]), False)
         b = ir.linear(x, W[a + "in_proj_b.weight"], "f16")                     # [1,S,vh]
@@ -96,7 +97,7 @@ def build_segments(cfg, W, S, T, fmt="q4g128"):
         outs[f"ssm_new.{l}"] = s1
         o = ops.transpose(o, i64([0, 2, 1, 3]))                                 # [1,S,vh,dv]
         o = ops.multiply(ir.rmsnorm(o, W[a + "norm.weight"], eps * QSCALE ** 2, prescale=QSCALE / 16), ir.silu(z))
-        return ir.linear(ops.reshape(o, i64([1, S, vh * dv]), False), W[a + "out_proj.weight"], fmt)
+        return ir.linear(ops.reshape(o, i64([1, S, vh * dv]), False), W[a + "out_proj.weight"], dn_fmt or fmt)
 
     def attention(l, x, params, outs, shared):
         a = f"{P}layers.{l}.self_attn."
@@ -216,18 +217,23 @@ def export_weights(cfg, W, out):
     common.export_embed_head(W, out, f"{P}embed_tokens.weight", "lm_head.weight")
 
 
-def export(src, out, sizes=(1, 16), T=1024, fmt="q4g128", graphs_only=False):
+def export(src, out, sizes=(1, 16), T=1024, fmt="q4g128", graphs_only=False, dn_fmt=None):
+    """dn_fmt="q8g128": Ornith-1.5-35B-A3B (a Qwen3.6-35B-A3B fine-tune) loses its first answer token with every
+    weight in q4g128 (the MoE input of layer 0 is 22% off the bf16 model's, the DeltaNet out_proj and in_proj_qkv
+    being most of it); with those two in INT8 it matches the bf16 model to the end of the answer."""
     os.makedirs(out, exist_ok=True)
     from ..prune import maybe_prune
     common.restore_prune(out, graphs_only)
     W, cfg = maybe_prune(Checkpoint(src), text_config(src))
-    metas = common.save_segments(lambda S: build_segments(cfg, W, S, T, fmt), out, sizes)
+    metas = common.save_segments(lambda S: build_segments(cfg, W, S, T, fmt, dn_fmt), out, sizes)
     rp = cfg.get("rope_parameters") or {}
     rot, theta = rotary(cfg)
     full = json.load(open(os.path.join(src, "config.json")))
     meta = {"arch": full["model_type"], "T": T, "sizes": list(sizes), "config": cfg, "segments": metas,
             "rope": {"dim": rot, "theta": theta,
                      "mrope_section": rp.get("mrope_section"), "mrope_interleaved": rp.get("mrope_interleaved", False)}}
+    if dn_fmt:
+        meta["deltanet_fmt"] = dn_fmt
     if common.prune_meta():
         meta["prune"] = common.prune_meta()
     if "vision_config" in full and not graphs_only:                                                # static vision tower, fixed 512x512

onw/server.py

@@ -136,7 +136,7 @@ def to_hf_messages(messages):
     """OpenAI chat messages -> HF chat format with PIL images."""
     out = []
     for m in tool_calls.history(messages):
-        keep = {k: m[k] for k in ("tool_calls", "tool_call_id", "name") if m.get(k) is not None}
+        keep = {k: m[k] for k in ("tool_calls", "tool_call_id", "name", "reasoning_content") if m.get(k) is not None}
         c = m.get("content")
         if c is None or isinstance(c, str):
             out.append({"role": m["role"], "content": c or "", **keep})

pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "onw"
-version = "0.11.3"
+version = "0.12.0"
 description = "Ore no NPU ga konna ni ugoku wake nai - LLMs entirely on the Intel NPU"
 readme = "README.md"
 license = {text = "Apache-2.0"}