onw 1.4.0: MTP draft tokens (~1.5x answers on Lunar Lake, NF4 Qwen3.5 family), 64-token prompt blocks on Lunar Lake, less host work per step, fix all-zero logit rows

CHANGELOG.md

@@ -5,6 +5,29 @@
 日付は日本時間です。各版は Hugging Face の `ryugyosoft/onw` にタグ（`v0.4.2` 以降）付きで残っています。
 onw ウィンドウは、新しい版があるとこのファイルから変更点を表示します（`## v…` の見出しで区切っています）。
 
+## v1.4.0 (2026-10-06)
+- **MTP 先読みで回答が約 1.5 倍に（Lunar Lake 以降、Qwen3.5 系の NF4 版）**：Qwen3.5 系のモデルに付いている MTP 層
+  （次の次のトークンを予想する小さな層）で 2 トークン先まで下書きし、本体で一度に確かめます。NPU は数トークンを 1 トークンと
+  ほぼ同じ時間で計算できるので、当たった分だけ速くなります。温度ありの生成でも、1 トークンずつ生成したときと同じ確率でトークンを
+  選びます。Lunar Lake（8 種類の質問の平均）で Qwen3.5-9B（NF4）9.7 → 15.1 tok/s（温度 0.6：8.6 → 13.8）、Ornith-1.5-9B（NF4）
+  10.2 → 15.1 tok/s（8.7 → 13.8）。確かめるのは 4 トークン用のグラフで、外れた下書きの分だけ正確に巻き戻します。
+  NPU 3720 では確かめるブロックが重く、かえって遅くなるため使いません。
+- **Lunar Lake 以降で長いプロンプトの処理が約 2 倍に**：「64 トークン処理」の auto が、Lunar Lake 以降でもオンになりました
+  （これまでは RAM 24 GB 以上）。重みを共有するのでメモリはほぼ増えません（+0.1〜0.3 GB）。Gemma 4 E4B 73 → 140 tok/s、
+  Qwen3.5-9B（NF4）119 → 220 tok/s、Ornith-1.5-9B（NF4）117 → 212 tok/s（約 2,000 トークン）。
+- Qwen3.5-9B（NF4）と Ornith-1.5-9B（NF4）のモデルを、MTP 層・4 トークン版・64 トークン版の入りに更新しました。入れてあるモデルは
+  モデル一覧に「更新あり」と出るので、「更新」で変わったファイルだけ取り直してください（約 3.9 GB）。
+- 回答の 1 ステップごとのホスト側の処理を約 3 割削減（NPU への入力をコピーせずに渡す、DeltaNet の状態を 2 つのバッファで交互に
+  使う、アテンションの出力を次の区間の形へ直接書く、など）。生成結果は変わりません（全モデルで、変更前と同じトークン列を確認）。
+- **修正**：複数トークンのブロック（先読み検証、MTP、プロンプトの最後のブロック）で、活性の大きい行の最終正規化が fp16 で溢れて
+  ロジットが行ごと 0 になり、意味のないトークン（他の言語の単語など）がまれに混ざっていました。その行だけ縮めて計算し直します。
+- **修正**：NPU 4000 以降で、ブロックサイズごとのコンパイル結果が別々の回に作られていると（64 トークン処理を後からオンにした、
+  onw の更新で一部だけコンパイルし直した、など）、重みの共有が外れてメモリを 2.5〜3.8 GB 余計に使っていました。混ざっていたら
+  一度だけまとめてコンパイルし直します（Lunar Lake で約 10 秒）。
+- 検証画面のモデル情報に「MTP 先読み」を追加。
+- `python -m onw.server` で直接起動したとき、Anthropic・Responses の API（`/v1/messages`、`/v1/responses`）が 404 になって
+  いたのを修正（onw ウィンドウや `onw serve` から起動した場合は問題ありませんでした）。
+
 ## v1.3.0 (2026-10-04)
 - **Anthropic の Messages API（`/v1/messages`）と OpenAI の Responses API（`/v1/responses`）に対応**。Claude Code や Codex CLI など、
   OpenAI のチャット API 以外の形式で話すアプリからも使えます（ストリーミング、ツール呼び出し、思考に対応。`count_tokens`、

CHANGELOG_en.md

@@ -4,6 +4,32 @@
 
 Dates are JST. Every version is on Hugging Face `ryugyosoft/onw`, tagged from `v0.4.2` on.
 
+## v1.4.0 (2026-10-06)
+- **MTP draft tokens: answers ~1.5x faster (Lunar Lake and newer, the NF4 Qwen3.5-family builds)**: the MTP layer that
+  Qwen3.5-family models carry (a small layer predicting the token after next) drafts two tokens ahead, and the model checks
+  them in one pass. The NPU computes a few tokens in about the time of one, so every kept draft is time saved. With
+  temperature the tokens are drawn with the same probabilities as one at a time. On Lunar Lake (average over 8 kinds of
+  prompts): Qwen3.5-9B (NF4) 9.7 -> 15.1 tok/s (temperature 0.6: 8.6 -> 13.8), Ornith-1.5-9B (NF4) 10.2 -> 15.1 tok/s
+  (8.7 -> 13.8). The checking runs on 4-token graphs and cuts back exactly past a rejected draft. Not used on NPU 3720,
+  where the checking block is expensive and MTP came out slower.
+- **Long prompts ~2x faster on Lunar Lake and newer**: "64-token blocks" on auto now turns on from Lunar Lake as well (until
+  now: 24 GB+ RAM). The weights are shared, so memory hardly grows (+0.1-0.3 GB). Gemma 4 E4B 73 -> 140 tok/s,
+  Qwen3.5-9B (NF4) 119 -> 220 tok/s, Ornith-1.5-9B (NF4) 117 -> 212 tok/s (~2,000 tokens).
+- The Qwen3.5-9B (NF4) and Ornith-1.5-9B (NF4) models now include the MTP layer and the 4- and 64-token graphs. Installed
+  copies show "Update available" in the model list: "Update" fetches just the changed files (~3.9 GB).
+- About 30% less host-side work per answer step (inputs handed to the NPU without copies, DeltaNet states in two
+  alternating buffers, attention output written straight in the next part's layout, ...). Same output: every model gives
+  the same tokens as before.
+- **Fixed**: in multi-token blocks (prompt lookup and MTP verification, a prompt's last block) the final norm of rows with
+  large activations overflowed fp16 and the whole row of logits came back as zeros - now and then a meaningless token (a
+  word from another language) slipped in. Those rows are computed again, scaled down.
+- **Fixed**: on NPU 4000 and newer, block sizes compiled in different runs (64-token blocks switched on later, a few graphs
+  compiled again after an onw update, ...) stopped sharing their weights and used 2.5-3.8 GB more memory. A mixed cache is now
+  compiled again once, in one run (~10 s on Lunar Lake).
+- "MTP draft tokens" in the model information of the checks page.
+- Started directly with `python -m onw.server`, the Anthropic and Responses APIs (`/v1/messages`, `/v1/responses`) answered
+  404 (started from the onw window or `onw serve` they were fine).
+
 ## v1.3.0 (2026-10-04)
 - **Anthropic's Messages API (`/v1/messages`) and OpenAI's Responses API (`/v1/responses`)**: apps that speak these instead of
   OpenAI's chat API, such as Claude Code and Codex CLI, work too (streaming, tool calls, thinking; `count_tokens` and

MANUAL.md

@@ -165,7 +165,7 @@
 | メモリ節約モード | 1 トークン用と 16 トークン用のグラフを入れ替えて、片方だけを持ちます。待機中のメモリが約 4 割減る代わりに、回答の出だし数秒は遅めで、先読み検証は使いません。効くモデルがあるときだけ表示され、この PC で効くモデルの名前が出ます |
 | 厳格 | メモリ節約モードの追加。2 種類のグラフを同時に持たないので、メモリは最小になりますが、回答のたびに書き始めるまで数秒待ちます |
 | エキスパートのメモリ | Qwen3.6 系の MoE 用。上限を超えるエキスパートは SSD に置いたまま、必要なときだけ読みます（メモリが少なくても動きますが、遅くなります） |
-| 64 トークン処理 | 長いプロンプトの処理が速くなる代わりに、そのぶんメモリを使います（Gemma）。auto はメモリ 24 GB 以上でオン |
+| 64 トークン処理 | 長いプロンプトの処理が約 2 倍速くなります（Gemma、Qwen3.5 系の NF4 版）。Lunar Lake 以降は重みを共有するのでメモリはほぼ増えません。NPU 3720 ではそのぶんメモリを使います。auto は Lunar Lake 以降か、メモリ 24 GB 以上でオン |
 
 ### 詳細
 

MANUAL_en.md

@@ -175,7 +175,7 @@ applies them ("Discard" throws them away). For settings that need a server resta
 | Memory-saving mode | Swaps the 1-token and 16-token graphs so only one set is held: about 40% less memory when idle, but the first seconds of an answer are slower and prompt lookup is not used. Shown only when it applies, with the models it works for on this PC |
 | Strict | An addition to the memory-saving mode: never holds both sets of graphs, for the least memory, but every answer waits a few seconds before it starts |
 | Expert memory | For the Qwen3.6 MoE models: experts beyond the limit stay on the SSD and are read when needed (works with less memory, but slower) |
-| 64-token blocks | Long prompts are processed faster, at the cost of more memory (Gemma). auto turns it on with 24 GB+ RAM |
+| 64-token blocks | Long prompts are processed ~2x faster (Gemma, the NF4 Qwen3.5-family builds). From Lunar Lake the weights are shared, so memory hardly grows; on NPU 3720 it costs that much memory. auto turns it on from Lunar Lake or with 24 GB+ RAM |
 
 ### Advanced
 

README.md

@@ -24,9 +24,6 @@ LLM を **Intel NPU だけで** 動かす小さなエンジンです。MoE（専
 llama.cpp と GGUF の関係と同じく、エンジンとモデルは別々にダウンロードします。エンジンは一つ、コマンドも一つで、
 変換済みのモデルならどれでも動きます。
 
-低電力の NPU で動かすので、ファンが低回転のままでも静かに LLM を使えます。Lunar Lake で計測したところ、1 トークンあたりの電力は
-iGPU（llama.cpp）より少なく、CPU の約 4 割でした。詳しい数字と測り方は [電力効率と温度](#電力効率と温度) にあります。
-
 ## インストール（コマンドを 1 行貼るだけ）
 
 Intel Core Ultra（NPU 付き）の PC なら、次の 1 行で入ります（詳しい条件は下の「必要なもの」）。
@@ -88,11 +85,11 @@ Python や git の知識は要りません。Python が入っていなければ
 |---|---|---|---|---|---|---|
 | [ryugyosoft/gemma-4-E4B-it-onw](https://huggingface.co/ryugyosoft/gemma-4-E4B-it-onw) | `gemma4`（検証済みの Gemma NPU グラフを取り込み） | テキスト＋画像 | 切替 | 7.2 tok/s | 4.3 GB | 約 6 GB（画像処理中 8.5 GB） |
 | [ryugyosoft/Qwen3.5-9B-onw](https://huggingface.co/ryugyosoft/Qwen3.5-9B-onw) | `qwen3_5`（Gated DeltaNet＋ゲート付きアテンション、dense） | テキスト＋画像 | 切替 | 4.1 tok/s | 5.3 GB | 約 11 GB |
-| [ryugyosoft/Qwen3.5-9B-nf4-onw](https://huggingface.co/ryugyosoft/Qwen3.5-9B-nf4-onw) | 同上、重みはチャネル単位 NF4（**Lunar Lake 以降専用**） | テキスト＋画像 | 切替 | —（Lunar Lake で 9.4 tok/s、通常版の約 1.3 倍） | 5.2 GB | 通常版の約 8 割 |
+| [ryugyosoft/Qwen3.5-9B-nf4-onw](https://huggingface.co/ryugyosoft/Qwen3.5-9B-nf4-onw) | 同上、重みはチャネル単位 NF4、MTP 先読み付き（**Lunar Lake 以降専用**） | テキスト＋画像 | 切替 | —（Lunar Lake で約 15 tok/s、通常版の約 2 倍） | 5.6 GB | 通常版の約 85% |
 | [ryugyosoft/Qwen3.6-REAP-18B-A3B-onw](https://huggingface.co/ryugyosoft/Qwen3.6-REAP-18B-A3B-onw) | `qwen3_5_moe`、256 個中 128 個のエキスパートを残す（REAP） | テキスト＋画像 | 切替 | 6.8 tok/s | 9.6 GB | 約 12 GB |
 | [ryugyosoft/Qwen3.6-35B-A3B-onw](https://huggingface.co/ryugyosoft/Qwen3.6-35B-A3B-onw) | `qwen3_5_moe`（Gated DeltaNet＋ゲート付きアテンション＋MoE 256 個から 8 個＋共有エキスパート） | テキスト＋画像 | 切替 | 6.8 tok/s | 17 GB | 約 20 GB（SSD から読めば 8〜13 GB） |
 | [ryugyosoft/Ornith-1.5-9B-onw](https://huggingface.co/ryugyosoft/Ornith-1.5-9B-onw) | `qwen3_5`（Qwen3.5-9B をエージェント・ツール呼び出し向けに鍛えた Ornith-1.5） | テキスト＋画像 | 切替 | 4.9 tok/s | 5.3 GB | 約 11 GB |
-| [ryugyosoft/Ornith-1.5-9B-nf4-onw](https://huggingface.co/ryugyosoft/Ornith-1.5-9B-nf4-onw) | 同上、重みはチャネル単位 NF4（**Lunar Lake 以降専用**） | テキスト＋画像 | 切替 | —（Lunar Lake で 9.3 tok/s、通常版の約 1.3 倍） | 5.2 GB | 通常版の約 8 割 |
+| [ryugyosoft/Ornith-1.5-9B-nf4-onw](https://huggingface.co/ryugyosoft/Ornith-1.5-9B-nf4-onw) | 同上、重みはチャネル単位 NF4、MTP 先読み付き（**Lunar Lake 以降専用**） | テキスト＋画像 | 切替 | —（Lunar Lake で約 15 tok/s、通常版の約 2 倍） | 5.6 GB | 通常版の約 85% |
 | [ryugyosoft/NeoHorse-1-4B-onw](https://huggingface.co/ryugyosoft/NeoHorse-1-4B-onw) | `qwen3_5`（Qwen3.5-4B をエージェント・ツール呼び出し向けに追加学習した NeoHorse-1） | テキスト | 切替 | 6.4 tok/s | 2.4 GB | 約 6 GB |
 | [ryugyosoft/granite-4.2-3b-onw](https://huggingface.co/ryugyosoft/granite-4.2-3b-onw) | `granite`（IBM Granite 4.2：思考とツール呼び出しを強化した 3B。Llama 型） | テキスト | 切替 | 6.7 tok/s（チャット約 7.7） | 2.1 GB | 約 6 GB |
 | [ryugyosoft/granite-4.2-8b-onw](https://huggingface.co/ryugyosoft/granite-4.2-8b-onw) | `granite`（同上の 8B） | テキスト | 切替 | 3.6 tok/s（チャット約 3.9） | 5.0 GB | 約 13 GB |
@@ -104,9 +101,15 @@ Python や git の知識は要りません。Python が入っていなければ
 | [ryugyosoft/gpt-oss-20b-int8-onw](https://huggingface.co/ryugyosoft/gpt-oss-20b-int8-onw) | 同上、エキスパート INT8（bf16 に近い精度） | テキスト | **常に** | 6 tok/s | 21 GB | 約 21 GB（SSD から読めば 8〜13 GB） |
 
 **NF4 版（Lunar Lake 以降専用）**：Lunar Lake（NPU 4000）以降の NPU は、NF4（正規分布に合わせた 4 ビット）の重みを
-ハードウェアで計算できます。NF4 版はチャネル単位の NF4 にしたモデルで、Lunar Lake では通常版より生成が約 3 割・プロンプト処理が
-約 2 倍速く、メモリは約 2 割少なくて済みます（精度はほぼ同等）。NPU 3720（Meteor Lake・Arrow Lake）では動かないため、onw ウィンドウは
-Lunar Lake 以降の PC でだけ一覧に出します。
+ハードウェアで計算できます。NF4 版はチャネル単位の NF4 にしたモデルで、Lunar Lake では通常版より生成が約 2 倍（MTP 先読み込み）・
+プロンプト処理が約 3 倍速く、メモリは約 15% 少なくて済みます（精度はほぼ同等）。NPU 3720（Meteor Lake・Arrow Lake）では動かないため、
+onw ウィンドウは Lunar Lake 以降の PC でだけ一覧に出します。
+
+**MTP 先読み（onw 1.4.0、Lunar Lake 以降）**：Qwen3.5 系のモデルには、次の次のトークンを予想する小さな層（MTP）が付いています。
+onw はこれで 2 トークン先まで下書きし、本体の 4 トークン用のグラフで一度に確かめます（外れた下書きの分だけ正確に巻き戻します）。
+NPU は数トークンを 1 トークンとほぼ同じ時間で計算できるので、下書きが当たった分だけ速くなります（8 種類の質問の平均で約 1.5 倍）。
+温度ありの生成でも、1 トークンずつ生成したときと同じ確率でトークンを選びます。NF4 版に入っています。NPU 3720 では確かめるブロックが
+2 トークンでも 1 トークンの 1.7 倍かかり、かえって遅くなるため使いません。
 
 使用メモリは、テスト機（Core Ultra 9 285HX、NPU 3720）で、64 トークン処理なし・画像エンコーダーは使ったら解放（メモリ
 24 GB 未満の PC の既定）で測ったプロセスのピークです。onw ウィンドウの「メモリ目安」は、この値にその PC の設定
@@ -174,7 +177,8 @@ onw list                                                      # 対応アーキ
 | temperature、top_p、top_k、繰り返し抑制（repetition / presence / frequency penalty）、max_tokens、stop、seed、思考モード | リクエストごと（API またはチャット画面） | 不要 |
 | ポート、待ち受けアドレス、モデル、デバイス、先読み検証のオン／オフ | サーバーのオプション／設定画面、再起動（約 20 秒） | 不要 |
 | **コンテキスト長**（`--context`、既定 32K、モデルの最大まで） | サーバーのオプション／設定画面 | 不要（KV キャッシュの上限が変わるだけ） |
-| 64 トークン処理（Gemma） | `ONW_S64=1/0`／設定画面 | 初回のみ |
+| 64 トークン処理（Gemma、Qwen3.5 系 NF4 版。auto は Lunar Lake 以降か RAM 24 GB 以上でオン） | `ONW_S64=1/0`／設定画面 | 初回のみ |
+| MTP 先読み（Qwen3.5 系 NF4 版。auto は Lunar Lake 以降でオン） | `ONW_MTP=1/0` | 初回のみ |
 | エキスパートのメモリ（Qwen3.6 系。上限を超える分は SSD から読む） | `ONW_EXPERT_GB=5`／設定画面 | 不要 |
 | NPUW 重み共有（試験的。auto は Lunar Lake 以降でオン、MoE モデル（LFM2・Qwen3.6）と区間の多いモデル（Granite・llm-jp）は対象外：重みを 1 コピーにしてメモリとコンパイル時間を削減） | `ONW_NPUW=1/0`／設定画面 | 初回のみ |
 | メモリ節約モード（1 トークン用と 16 トークン用で重みを共有できない dense モデル：NPU 3720 ではすべて、Lunar Lake 以降は Granite・llm-jp と NPUW オフのとき。普段は 16 トークン用のグラフだけを持ち、回答を書く間だけ 1 トークン用と入れ替える。待機中のメモリ約 4〜6 割減、回答の出だし数秒は遅め、先読み検証なし） | `ONW_LAZY16=1`／設定画面（効くモデルがあるときだけ表示） | 初回のみ |
@@ -217,43 +221,6 @@ KV キャッシュは INT8 で、会話が実際に使った長さの分だけ
 | `setup.*`、`install.*`、`find_python.bat` | インストーラー（Python の検出と winget） |
 | `check_ref.py`、`check_vision.py`、`test_*.py`、`probe_segments.py`、`prof_*.py`、`mem_*.py`、`bench_*.py` | 元の HF モデルとの比較、生成・画像・複数ターンの検証、区間ごとの NPU と CPU の比較、速度・メモリ計測 |
 
-## 電力効率と温度
-
-同じ PC（LattePanda Mu Ultra、Core Ultra 5 226V＝Lunar Lake）で、Qwen3.5-9B の生成中のパッケージ電力と温度を、NPU・iGPU・CPU で比べました。
-日本語の長文生成（768 トークン、temperature 0）を各 3 回測った平均です。
-
-| デバイス | 実行環境 | 生成速度 | パッケージ電力 | パッケージ温度（平均 / 最大） | 1 トークンあたり | 同（アイドル分を引いた値） |
-|---|---|---|---|---|---|---|
-| **NPU** | onw（Qwen3.5-9B-onw、INT4、5.3 GB） | 7.3 tok/s | **12.0 W** | 52 / **67 ℃** | **1.66 J** | **1.27 J** |
-| iGPU（Arc） | llama.cpp SYCL（Q4_K_M、6.2 GB） | **12.6 tok/s** | 23.5 W | 66 / 83 ℃ | 1.91 J | 1.67 J |
-| CPU | llama.cpp（Q4_K_M、6.2 GB） | 5.7 tok/s | 22.3 W | 93 / 96 ℃ | 3.99 J | 3.47 J |
-
-アイドル時のパッケージ電力は約 2.9 W、温度は約 42〜46 ℃ で��。
-
-1 万トークンを生成したとき（上の平均値からの計算）：
-
-| デバイス | かかる時間 | 消費エネルギー | 同（アイドル分を引いた値） |
-|---|---|---|---|
-| **NPU** | 約 23 分 | **4.6 Wh** | 3.5 Wh |
-| iGPU | 約 13 分 | 5.3 Wh | 4.6 Wh |
-| CPU | 約 30 分 | 11.1 Wh | 9.6 Wh |
-
-NPU は CPU の約 4 割、iGPU より約 13% 少ないエネルギーで済みます。生成中の電力は 12 W で iGPU の約半分、最大温度は 67 ℃ で、ファンも低回転のまま動かせました（ファンの回転数は測れていないため、目視・体感の観察です）。
-1 万トークンの値は 768 トークンの測定からの計算で、実測ではありません。長く回すと、熱で CPU の電力が下がって遅くなり、時間が延びることがあります。
-
-
-- 1 トークンあたりの電力は、NPU が最も少なく、iGPU より 13〜24% 少なく、CPU の約 4 割以下です。
-- 速さは iGPU が最も速く、NPU の約 1.7 倍です。ただし電力は約 2 倍かかります。
-- NPU は最大でも 67 ℃ に収まります。iGPU は 83 ℃、CPU は 96 ℃ まで上がりました。
-
-測り方と注意：
-- 電力は Linux の RAPL（`intel-rapl:0`、package-0）の電力量を 0.5 秒ごとに読んだものです。SoC（CPU・iGPU・NPU）だけの値で、DRAM（約 0.4〜0.8 W）や PC 全体は含みません。温度は `x86_pkg_temp` です。
-- 測るのは生成の区間だけで、モデルの読み込みは除いています。毎回、温度が下がるのを待ってからアイドル電力を 30 秒測りました。デバイスの順番による偏りが出ないよう、NPU、iGPU、CPU を 3 周しています。
-- ファン回転数は、この PC では CPU クーラーの値を OS から読めないため、測っていません。NPU の生成中、ファンは止まらず低回転のままでした（観察）。
-- onw と llama.cpp ではエンジンが違い、重みの量子化も同一ではありません（onw は独自の INT4、llama.cpp は Q4_K_M）。モデルと実行環境をまとめた比較として読んでください。
-- ばらつき：NPU と CPU は 3 回でほぼ揃いました（NPU の 1 トークンあたり 1.65〜1.67 J）。iGPU は生成速度が 10.9〜13.5 tok/s と回ごとに差があり、1 トークンあたり 1.84〜2.05 J でした。
-- 別の PC、別のプロンプト、別の冷却では値が変わります。
-
 ## 注意点と制限
 
 - コンテキスト長の既定は 32K トークンで、モデルの最大（Qwen3.5／3.6 は 262K、Gemma 4 E4B は 128K）まで設定できます。
@@ -261,8 +228,8 @@ NPU は CPU の約 4 割、iGPU より約 13% 少ないエネルギーで済み
   （最大 280 トークン）。
 - エキスパートはチャネル単位 INT4（四捨五入量子化）です。確認した範囲では、最初のトークンのロジットは bf16 との差が
   約 33% で、1 位の予測は一致しました。回答は自然ですが、数トークン先から言い回しが bf16 モデルと分かれます。
-- 16 トークンより大きいプロンプトブロック（Gemma の 64 トークン版）は RAM 24 GB 以上のときだけ読み込みます
-  （`ONW_S64=1/0` で強制）。
+- 16 トークンより大きいプロンプトブロック（Gemma と Qwen3.5 系 NF4 版の 64 トークン版）は、Lunar Lake 以降（重みを共有するので
+  メモリはほぼ増えません）か RAM 24 GB 以上のときに読み込みます（`ONW_S64=1/0` で強制）。
 - OpenVINO のプレリリース版（2026.5.0b1。OpenVINO の pre-release 索引から入ります）が必要です。NPU 3720（Windows 11）と NPU 4000（Lunar Lake、Ubuntu、
   メモリ 16 GB）で全モデルの動作を確認しています。NPU ドライバーがない環境では CPU で動きます。
 

README_en.md

@@ -10,9 +10,6 @@ A small engine that runs LLMs — including MoE and hybrid DeltaNet models, with
 Intel NPU**, with an OpenAI-compatible server and a browser chat UI. Engine and models are separate downloads,
 like llama.cpp + a GGUF: one engine, one command, any converted model.
 
-It runs on the low-power NPU, so you can use an LLM quietly, with the fan staying at low speed. Measured on Lunar Lake, it used
-less energy per token than the iGPU (llama.cpp) and about 40% of the CPU. The numbers and how they were measured are in [Power efficiency and temperature](#power-efficiency-and-temperature).
-
 ## Install (paste one line)
 
 On a PC with an Intel Core Ultra (it has the NPU) this one line installs it (details under "What you need" below).
@@ -75,11 +72,11 @@ troubleshooting), is the **[user guide](MANUAL_en.md)**.
 |---|---|---|---|---|---|---|
 | [ryugyosoft/gemma-4-E4B-it-onw](https://huggingface.co/ryugyosoft/gemma-4-E4B-it-onw) | `gemma4` (imported from the validated Gemma NPU graphs) | text + image | switchable | 7.2 tok/s | 4.3 GB | ~6 GB (8.5 GB while reading an image) |
 | [ryugyosoft/Qwen3.5-9B-onw](https://huggingface.co/ryugyosoft/Qwen3.5-9B-onw) | `qwen3_5` (Gated DeltaNet + gated attention, dense) | text + image | switchable | 4.1 tok/s | 5.3 GB | ~11 GB |
-| [ryugyosoft/Qwen3.5-9B-nf4-onw](https://huggingface.co/ryugyosoft/Qwen3.5-9B-nf4-onw) | the same, channel-wise NF4 weights (**Lunar Lake and newer only**) | text + image | switchable | - (9.4 tok/s on Lunar Lake, ~1.3x the regular build) | 5.2 GB | ~80% of the regular build |
+| [ryugyosoft/Qwen3.5-9B-nf4-onw](https://huggingface.co/ryugyosoft/Qwen3.5-9B-nf4-onw) | the same, channel-wise NF4 weights, MTP draft tokens (**Lunar Lake and newer only**) | text + image | switchable | - (~15 tok/s on Lunar Lake, ~2x the regular build) | 5.6 GB | ~85% of the regular build |
 | [ryugyosoft/Qwen3.6-REAP-18B-A3B-onw](https://huggingface.co/ryugyosoft/Qwen3.6-REAP-18B-A3B-onw) | `qwen3_5_moe`, 128 of 256 experts kept (REAP) | text + image | switchable | 6.8 tok/s | 9.6 GB | ~12 GB |
 | [ryugyosoft/Qwen3.6-35B-A3B-onw](https://huggingface.co/ryugyosoft/Qwen3.6-35B-A3B-onw) | `qwen3_5_moe` (Gated DeltaNet + gated attention + MoE 256x top-8 + shared expert) | text + image | switchable | 6.8 tok/s | 17 GB | ~20 GB (8-13 GB with experts from the SSD) |
 | [ryugyosoft/Ornith-1.5-9B-onw](https://huggingface.co/ryugyosoft/Ornith-1.5-9B-onw) | `qwen3_5` (Ornith-1.5: Qwen3.5-9B trained further for agents and tool calls) | text + image | switchable | 4.9 tok/s | 5.3 GB | ~11 GB |
-| [ryugyosoft/Ornith-1.5-9B-nf4-onw](https://huggingface.co/ryugyosoft/Ornith-1.5-9B-nf4-onw) | the same, channel-wise NF4 weights (**Lunar Lake and newer only**) | text + image | switchable | - (9.3 tok/s on Lunar Lake, ~1.3x the regular build) | 5.2 GB | ~80% of the regular build |
+| [ryugyosoft/Ornith-1.5-9B-nf4-onw](https://huggingface.co/ryugyosoft/Ornith-1.5-9B-nf4-onw) | the same, channel-wise NF4 weights, MTP draft tokens (**Lunar Lake and newer only**) | text + image | switchable | - (~15 tok/s on Lunar Lake, ~2x the regular build) | 5.6 GB | ~85% of the regular build |
 | [ryugyosoft/NeoHorse-1-4B-onw](https://huggingface.co/ryugyosoft/NeoHorse-1-4B-onw) | `qwen3_5` (NeoHorse-1: Qwen3.5-4B trained further for agents and tool calls) | text | switchable | 6.4 tok/s | 2.4 GB | ~6 GB |
 | [ryugyosoft/granite-4.2-3b-onw](https://huggingface.co/ryugyosoft/granite-4.2-3b-onw) | `granite` (IBM Granite 4.2: a 3B with thinking and stronger tool calls; Llama-type) | text | switchable | 6.7 tok/s (~7.7 in chat) | 2.1 GB | ~6 GB |
 | [ryugyosoft/granite-4.2-8b-onw](https://huggingface.co/ryugyosoft/granite-4.2-8b-onw) | `granite` (the same, 8B) | text | switchable | 3.6 tok/s (~3.9 in chat) | 5.0 GB | ~13 GB |
@@ -91,9 +88,16 @@ troubleshooting), is the **[user guide](MANUAL_en.md)**.
 | [ryugyosoft/gpt-oss-20b-int8-onw](https://huggingface.co/ryugyosoft/gpt-oss-20b-int8-onw) | same with INT8 experts (close to bf16) | text | **always** | 6 tok/s | 21 GB | ~21 GB (8-13 GB with experts from the SSD) |
 
 **NF4 builds (Lunar Lake and newer only)**: NPUs from Lunar Lake (NPU 4000) on compute NF4 weights (4 bits placed like a
-normal distribution) in hardware. The NF4 builds use channel-wise NF4: on Lunar Lake they decode ~30% faster, process
-prompts ~2x faster and need ~20% less memory than the regular builds, at about the same accuracy. NPU 3720 (Meteor Lake,
-Arrow Lake) cannot run them, so the onw window lists them only on Lunar Lake and newer PCs.
+normal distribution) in hardware. The NF4 builds use channel-wise NF4: on Lunar Lake they decode ~2x faster (with MTP
+draft tokens), process prompts ~3x faster and need ~15% less memory than the regular builds, at about the same accuracy.
+NPU 3720 (Meteor Lake, Arrow Lake) cannot run them, so the onw window lists them only on Lunar Lake and newer PCs.
+
+**MTP draft tokens (onw 1.4.0, Lunar Lake and newer)**: Qwen3.5-family models carry a small layer (MTP) that predicts the
+token after next. onw uses it to draft two tokens ahead and checks them in one pass on the model's 4-token graphs
+(cutting back exactly past a rejected draft). The NPU computes a few tokens in about the time of one, so every kept draft
+is time saved (~1.5x on average over 8 kinds of prompts). With temperature the tokens are still drawn with the same
+probabilities as one at a time. The NF4 builds include it. Not used on NPU 3720, where even a 2-token block costs 1.7
+single steps and MTP came out slower.
 
 Memory is the process peak on the test machine (Core Ultra 9 285HX, NPU 3720) without 64-token blocks and with the
 vision encoder freed after each image (the default below 24 GB of RAM). The "Memory (est.)" column in the onw
@@ -160,7 +164,8 @@ What needs what:
 | temperature, top_p, top_k, repetition / presence / frequency penalty, max_tokens, stop, seed, thinking | per request (API or chat UI) | no |
 | port, address, model, device, prompt lookup on/off | server option / settings, restart (~20 s) | no |
 | **context length** (`--context`, 32K by default, up to the model's maximum) | server option / settings | no (it only bounds the KV cache) |
-| 64-token prompt blocks (Gemma) | `ONW_S64=1/0` / settings | once, the first time |
+| 64-token prompt blocks (Gemma, the NF4 Qwen3.5-family builds; auto = on from Lunar Lake or with 24 GB+ RAM) | `ONW_S64=1/0` / settings | once, the first time |
+| MTP draft tokens (the NF4 Qwen3.5-family builds; auto = on from Lunar Lake) | `ONW_MTP=1/0` | once, the first time |
 | expert memory (Qwen3.6 models; the rest streams from the SSD) | `ONW_EXPERT_GB=5` / settings | no |
 | NPUW weight sharing (experimental; auto = on from Lunar Lake, not for the MoE models LFM2 / Qwen3.6 or models with many parts (Granite, llm-jp): one copy of the weights, less memory, much faster compile) | `ONW_NPUW=1/0` / settings | once, the first time |
 | Memory-saving mode (dense models whose 1-token and 16-token graphs cannot share weights: all of them on NPU 3720; from Lunar Lake on Granite / llm-jp, and any with NPUW off. Only the 16-token graphs are kept, swapped for the 1-token ones while an answer is written; 40-60% less memory when idle, the first seconds of an answer slower, no prompt lookup) | `ONW_LAZY16=1` / settings (shown only when a model it helps is there) | once, the first time |
@@ -202,43 +207,6 @@ install it anywhere with `pip install -e . --extra-index-url https://storage.ope
 | `setup.*`, `install.*`, `find_python.bat` | installers (Python detection + winget) |
 | `check_ref.py`, `check_vision.py`, `test_*.py`, `probe_segments.py`, `prof_*.py`, `mem_*.py`, `bench_*.py` | vs the HF model, generation / image / multi-turn checks, NPU-vs-CPU per segment, timing and memory |
 
-## Power efficiency and temperature
-
-Package power and temperature while generating with Qwen3.5-9B on the same PC (LattePanda Mu Ultra, Core Ultra 5 226V / Lunar Lake), on the NPU, iGPU and CPU.
-Average of 3 runs each of a long Japanese generation (768 tokens, temperature 0).
-
-| Device | Engine | Decode | Package power | Package temp (avg / max) | Energy per token | Same, idle subtracted |
-|---|---|---|---|---|---|---|
-| **NPU** | onw (Qwen3.5-9B-onw, INT4, 5.3 GB) | 7.3 tok/s | **12.0 W** | 52 / **67 °C** | **1.66 J** | **1.27 J** |
-| iGPU (Arc) | llama.cpp SYCL (Q4_K_M, 6.2 GB) | **12.6 tok/s** | 23.5 W | 66 / 83 °C | 1.91 J | 1.67 J |
-| CPU | llama.cpp (Q4_K_M, 6.2 GB) | 5.7 tok/s | 22.3 W | 93 / 96 °C | 3.99 J | 3.47 J |
-
-Idle package power is about 2.9 W at 42-46 °C.
-
-For 10,000 generated tokens (calculated from the averages above):
-
-| Device | Time | Energy | Same, idle subtracted |
-|---|---|---|---|
-| **NPU** | about 23 min | **4.6 Wh** | 3.5 Wh |
-| iGPU | about 13 min | 5.3 Wh | 4.6 Wh |
-| CPU | about 30 min | 11.1 Wh | 9.6 Wh |
-
-The NPU needs about 40% of the CPU's energy and about 13% less than the iGPU. It draws 12 W during generation, about half the iGPU, peaks at 67 °C, and ran with the fan at low speed (fan speed was not measured; this is an observation).
-The 10,000-token figures are calculated from the 768-token measurements, not measured. Over a longer run the CPU may throttle and take longer.
-
-
-- The NPU uses the least energy per token: 13-24% less than the iGPU, and under 40% of the CPU.
-- The iGPU is the fastest, about 1.7x the NPU, at about twice the power.
-- The NPU peaks at 67 °C; the iGPU reaches 83 °C and the CPU 96 °C.
-
-Method and caveats:
-- Power is the RAPL package-0 energy counter (`intel-rapl:0`) sampled every 0.5 s. It covers the SoC (CPU, iGPU, NPU) only, not DRAM (about 0.4-0.8 W) or the whole PC. Temperature is `x86_pkg_temp`.
-- Only the generation phase is measured, not model loading. Each run starts after a cool-down, with 30 s of idle power measured first. The devices were run NPU, iGPU, CPU in three rounds to avoid order effects.
-- Fan speed was not measured: this PC does not expose the CPU cooler's RPM to the OS. During NPU generation the fan kept running at low speed (observation).
-- The engines differ (onw vs llama.cpp) and the weights are not the same quantization (onw's own INT4 vs Q4_K_M), so read this as a comparison of model plus runtime.
-- Spread: NPU and CPU were consistent across runs (NPU 1.65-1.67 J/token). The iGPU varied from run to run: 10.9-13.5 tok/s and 1.84-2.05 J/token.
-- Other machines, prompts and cooling will give different numbers.
-
 ## Notes and limitations
 
 - The context is 32K tokens by default and goes up to the model's maximum (262K for Qwen3.5 / 3.6, 128K for
@@ -246,7 +214,8 @@ Method and caveats:
   ratio (up to 280 tokens).
 - Experts are channel-wise INT4 (round-to-nearest): first-token logits within ~33% of bf16 with the same top-1 on
   our checks; answers are fluent, word choices diverge from the bf16 model after a few tokens.
-- Prompt blocks above 16 tokens (Gemma's 64-token block) are loaded only with >= 24 GB RAM (`ONW_S64=1/0`).
+- Prompt blocks above 16 tokens (the 64-token blocks of Gemma and the NF4 Qwen3.5-family builds) are loaded from Lunar Lake
+  on (the weights are shared, so memory hardly grows) or with >= 24 GB RAM (`ONW_S64=1/0`).
 - Needs a pre-release OpenVINO (2026.5.0b1, from OpenVINO's pre-release index). All models are tested on NPU 3720 (Windows 11) and NPU 4000
   (Lunar Lake, Ubuntu, 16 GB RAM). Without an NPU driver everything runs on the CPU.
 

TECHNICAL.md

@@ -13,6 +13,9 @@ onw をどうやって Intel NPU に載せたかの説明です。使い方は [
 - 長い文脈：ページ化した INT8 KV キャッシュ
 - onw 0.7 でのメモリ削減
 - モデルを小さくする（onw convert --compact / --prune）
+- MTP 先読みと 64 トークン処理（onw 1.4、Lunar Lake）
+- 1 ステップのホスト側の処理（onw 1.4）
+- 複数トークンのブロックで、ロジットが行ごと 0 になる（onw 1.4 で修正）
 - NPU で分かった制約（実測）
 
 ## NPU で初めて動かした 4 つ
@@ -308,6 +311,52 @@ onw 0.10 では、読み込み時に各区間をアテンションの位置で
   残してパープレキシティ +8.9%（75% 残しなら +1.4%。Qwen3.6-REAP-18B-A3B-onw として公開）。エキスパートが 32 個しかない
   LFM2 は半分で +35% と落ち方が大きいため、刈り込んでいません。
 
+## MTP 先読みと 64 トークン処理（onw 1.4、Lunar Lake）
+
+Lunar Lake（NPU 4000、Ornith-1.5-9B NF4）で 1 トークンの生成を測ると、85% が NPU の計算そのもので、16 トークンのブロックは
+1 トークンとほぼ同じ時間（区間 1 つあたり 7.2 ms と 6.3 ms）でした。NPU は重みを読む時間で決まっていて、計算は余っています。
+そこで、余りを使う 2 つのことをしました。
+
+- **64 トークンのプロンプトブロック**：DeltaNet の chunk 計算は 16 トークンのまま（`ir.deltanet_chunks`：64 トークンを 16 ずつ、
+  状態を引き継いで計算。FP16 での累積減衰と三角行列の逆行列が長いブロックで崩れないように）、射影は 64 行まとめて計算します。
+  プロンプト処理は約 2 倍。bf16 モデルとの一致率は 16 トークン版と同じでした（教師強制で 69.5% と 71.3%）。
+- **MTP 先読み**：Qwen3.5 系には MTP 層（`mtp.*`：fc で [埋め込み, 最終正規化後の hidden] をまとめ、ゲート付きアテンションの
+  デコーダー層 1 つ、mtp.norm、LM ヘッド）が付いています。位置 i の hidden と i+1 のトークンから i+2 を予想します（bf16 で 78% 一致。
+  連結の順を逆にすると 0%）。onw はこれを別のグラフ（INT8、約 0.25 GB、NPUW なし）にして、16 トークンのグラフ 1 つで KV 行の
+  書き込みと下書きを兼ね、2 トークン先まで下書きします。ヘッドは出現頻度の高い 3.2 万語だけ（Wikipedia 日英とコードで数えた
+  上位。ほかの文章の 96% を占める）で、全語彙の 1/8 の読み込み（11.5 → 2.3 ms）。温度ありでは下書き d を確率 p(d) で採り、
+  外れたら d を除いた p から選び直します（下書きが決定的なときの投機的サンプリング。1 トークンずつ生成したのと同じ分布）。
+- **4 トークンの確認グラフと正確な巻き戻し**：確かめるブロックは「x＋下書き 2 つ」なので、4 トークン用のグラフで十分です
+  （16 トークン用より 1 ブロック約 15% 安い）。4 トークンでは DeltaNet を chunk 形式にできない（NPU 4000 のコンパイラーが
+  [1, h, T, T] のテンソルで失敗する）ので、1 トークンの式を 4 回つなぎます。この形だと各行の後の状態もグラフの中にあるので、
+  それも出力し（`ssm_rows`）、外れた下書きの後ろだけを捨てます（これまでは DeltaNet の状態を途中まで戻せず、ブロックを丸ごと
+  戻して採ったトークンを次のブロックで計算し直していた）。NPU 4000 のコンパイラーは、最後の行以外の状態を転置したまま
+  出力する（CPU では正しい。グラフ側で先に転置しても打ち消される）ので、読み込み時に各層・各行の並びを、その行数だけのブロックの
+  最終状態とビット単位で照らし合わせて決めます（`_check_rows`。合わなければ従来の丸ごと戻す方式）。
+  8 種類の質問の平均で、MTP なし 10.2 → 16 トークン確認 13.9 → 4 トークン確認 15.1 tok/s（Ornith NF4、greedy）。
+- NPU 3720 では 2 トークンのブロックでも 1 トークンの 1.7 倍（16 トークンは 2.4 倍）かかり、MTP は遅くなった（3.6〜4.7 対
+  4.2〜4.8 tok/s）ので使いません。小さい確認グラフを足すと、NPUW のない 3720 では重みのコピーも 1 つ増えます（約 4 GB）。
+- MTP を使うときは 1・16 トークン用のグラフを読み込みません（回答はすべて確認ブロックで書くため）。NPU ドライバーの
+  コマンドキューの枠（doorbell）を区間の数が超えないようにするためでもあります（下の「制約」）。
+
+## 1 ステップのホスト側の処理（onw 1.4）
+
+MTP の 1 ブロックのうち、NPU 以外（Python・numpy）が約 47 ms ありました。生成結果を 1 ビットも変えずに、次のことで約 32 ms に
+しました（Gemma、LFM2、Granite、llm-jp、Qwen3.5／3.6、gpt-oss、Ornith で、変更前と同じトークン列を確認）。
+
+- 推論の入力をコピーしない（`share_inputs`）。入力の型が違うときだけ変換した配列を渡す。
+- DeltaNet の状態を 2 つのバッファで交互に使う：区間は今の方を読み、新しい状態をもう一方へ直接書く（出力に `set_tensor`）。
+  ブロックを戻すときは入れ替えを戻すだけで、25 MB の状態のコピーが要らない。
+- アテンションの出力（KV のブロックが 1 つのとき）を、割り算と並べ替えを 1 回で次の区間の形（f16）へ書く。マスクはブロックごとに
+  1 回だけ作る。greedy でペナルティなしのときは、確率分布を作らずに argmax だけで決める。
+
+## 複数トークンのブロックで、ロジットが行ごと 0 になる（onw 1.4 で修正）
+
+先読み検証・MTP のブロックで、活性の大きい行（数式の `\times` や数字の後など）の最終 RMSNorm の二乗和が NPU の fp16 で溢れ、
+その行のロジットがすべて 0 で返ることがありました（1,400 トークンに 16 回ほど）。温度ありではこれが全語彙の一様分布になり、他の言語の
+単語などが混ざります。RMSNorm は入力の倍率によらないので、0 の行だけ行の最大値で割ってからヘッドをもう一度実行します
+（正常な行はそのまま）。
+
 ## NPU で分かった制約（実測）
 
 onw を作る途中で、NPU 3720／4000（OpenVINO 2026.5.0b1）について分かったことです。
@@ -333,3 +382,10 @@ onw を作る途中で、NPU 3720／4000（OpenVINO 2026.5.0b1）について分
 - LFM2.5-2.6B は残差の値がとても小さい（RMS 約 0.02）トークンと値の大きい BOS が同じブロックに並び、RMSNorm の二乗平均と
   eps（1e-5）が FP16 の正規化数の範囲（6e-5 以上）を下回って、NPU の区間出力が 50〜77% ずれた。固定の倍率では両立しない
   ため、各行を自分の最大値で割ってから二乗平均を取る（必ず 1/次元数以上になる）。eps も (√eps／最大値)² として計算（差は 1〜5% に）。
+- NPUW の重み共有（NPUW_WEIGHTS_BANK）は、同じプロセスでコンパイルしたグラフどうしでだけ効く。別々の回にコンパイルして
+  キャッシュから読んだグラフは、それぞれ重みを持つ（Ornith 9B NF4：1・16 トークン版を一緒にコンパイル 4.6 GB、16 トークン版まで
+  と 64 トークン版を別の回にコンパイル 8.7 GB）。onw はキャッシュのフォルダーにコンパイルした回を記録し、混ざっていたら
+  まとめてコンパイルし直す（1.4.0）。ドライバーのメモリは `/proc/meminfo` の Shmem（Unevictable）に出て、プロセスの RSS には出ない。
+- NPU 4000 の Linux ドライバーは、1 つのプロセスで推論リクエスト（コマンドキュー）を約 40 を超えて作ると doorbell の割り当てに
+  失敗し（`Failed to allocate doorbell ID: -16`）、以後は再起動するまでどのプロセスも NPU で推論できなくなった
+  （`Failed to create command queue: -74`）。

TECHNICAL_en.md

@@ -13,6 +13,9 @@ test machine (Core Ultra 9 285HX, NPU 3720, Windows 11, OpenVINO 2026.5.0b1) unl
 - Long contexts: the paged INT8 KV cache
 - Memory savings in onw 0.7
 - Smaller models (onw convert --compact / --prune)
+- MTP draft tokens and 64-token blocks (onw 1.4, Lunar Lake)
+- Host-side work per step (onw 1.4)
+- Logit rows of all zeros in multi-token blocks (fixed in onw 1.4)
 - What we learned about the NPU (measured)
 
 ## Four firsts on the NPU
@@ -303,6 +306,54 @@ Same models, engine-side only: 20-60% less memory (test machine, peak).
   experts at +8.9% perplexity (+1.4% keeping 75%; published as Qwen3.6-REAP-18B-A3B-onw); LFM2 with only 32 experts
   loses +35% at 50%, so it is not pruned.
 
+## MTP draft tokens and 64-token blocks (onw 1.4, Lunar Lake)
+
+Measured on Lunar Lake (NPU 4000, Ornith-1.5-9B NF4), 85% of a decoding step is the NPU's own work, and a 16-token block
+takes about as long as one token (7.2 ms against 6.3 ms per part): the NPU waits on reading weights and has compute to
+spare. Two things use it.
+
+- **64-token prompt blocks**: the DeltaNet chunk math stays at 16 tokens (`ir.deltanet_chunks`: 64 tokens as four chunks of
+  16 passing the state on, so fp16's cumulative decays and the triangular inverse stay sound), the projections run on all 64
+  rows. Prompts ~2x faster, with the same agreement with the bf16 model as the 16-token blocks (69.5% and 71.3% teacher-forced).
+- **MTP draft tokens**: Qwen3.5-family models carry an MTP layer (`mtp.*`: fc over [embedding, final-normed hidden state],
+  one gated-attention decoder layer, mtp.norm, the LM head): the hidden state at i and the token at i+1 predict i+2 (78%
+  agreement in bf16; 0% with the concatenation the other way round). onw makes it a graph of its own (INT8, ~0.25 GB, no
+  NPUW); one 16-token graph both writes its KV rows and drafts, two tokens ahead. Its head scores only the 32K most frequent
+  tokens (counted on Japanese / English Wikipedia and code; 96% of other text), an eighth of the LM head's reads (11.5 ->
+  2.3 ms). With temperature a draft d is kept with probability p(d) and otherwise the token is drawn from p without d
+  (speculative sampling with a deterministic draft: the same distribution as one token at a time).
+- **4-token checking graphs, cut back exactly**: a checking block is x + 2 drafts, so 4-token graphs do (a block ~15% cheaper
+  than on the 16-token ones). At 4 tokens DeltaNet cannot take the chunk form (the NPU 4000 compiler fails on [1, h, T, T]
+  tensors), so it is the 1-token step four times - which has every row's state in the graph: those are output too
+  (`ssm_rows`) and only the rows past a rejected draft are dropped (before, the recurrent state could not be cut back, so a
+  block with a miss was undone and its kept tokens computed again in the next block). The NPU 4000 compiler hands out the
+  states of all rows but the last transposed (correct on the CPU; transposing them in the graph first gets cancelled), so at
+  load each layer's and row's layout is found by a bit-exact comparison with the final state of a block of just that many
+  rows (`_check_rows`; without a match, blocks are undone as before). Averages over 8 kinds of prompts: no MTP 10.2 ->
+  16-token checks 13.9 -> 4-token checks 15.1 tok/s (Ornith NF4, greedy).
+- On NPU 3720 even a 2-token block costs 1.7 single steps (16 tokens: 2.4) and MTP came out slower (3.6-4.7 against
+  4.2-4.8 tok/s): not used there. A small checking graph would also add a weight copy there (no NPUW: ~4 GB).
+- With MTP the 1- and 16-token graphs are not loaded (every answer step is a checking block), which also keeps the number of
+  parts under the NPU driver's command-queue (doorbell) budget (see below).
+
+## Host-side work per step (onw 1.4)
+
+Of an MTP block, ~47 ms was Python / numpy. Without changing a single output bit, it is now ~32 ms (the same tokens as
+before checked on Gemma, LFM2, Granite, llm-jp, Qwen3.5 / 3.6, gpt-oss and Ornith):
+
+- inference inputs are not copied (`share_inputs`; converted arrays only where an input's type differs);
+- DeltaNet states in two alternating buffers: a segment reads the current one and writes its new state straight into the
+  other (`set_tensor` on the output); undoing a block swaps back instead of copying 25 MB of state;
+- the attention output (one KV block) is divided and laid out for the next part (f16) in one pass, masks are built once per
+  block, and greedy decoding without penalties takes the argmax without a full distribution.
+
+## Logit rows of all zeros in multi-token blocks (fixed in onw 1.4)
+
+In prompt-lookup and MTP checking blocks, the final RMSNorm's sum of squares overflowed fp16 on the NPU for rows with large
+activations (after `\times` or digits in formulas, ...), and the whole row of logits came back as zeros (about 16 times in
+1,400 tokens). With temperature that is a uniform draw over the vocabulary: words from other languages slipped in. RMSNorm
+does not care about scale, so just those rows run the head again divided by their max (the others are untouched).
+
 ## What we learned about the NPU (measured)
 
 Found while building onw, on NPU 3720 / 4000 with OpenVINO 2026.5.0b1:
@@ -332,3 +383,11 @@ Found while building onw, on NPU 3720 / 4000 with OpenVINO 2026.5.0b1:
   square and eps (1e-5) fell below fp16's normal range (6e-5) and the NPU's segment outputs were off by 50-77%. No fixed
   scale fits both, so each row is divided by its own max before the mean square (always >= 1/dim), and eps is applied as
   (sqrt(eps) / max)² (now within 1-5%).
+- NPUW's weight sharing (NPUW_WEIGHTS_BANK) works only between graphs compiled in the same process: graphs compiled in
+  different runs and read from the cache hold their own copy each (Ornith 9B NF4: 1- and 16-token graphs compiled together
+  4.6 GB, up to 16 tokens and the 64-token graphs from different runs 8.7 GB). onw records the run in each cache folder and
+  compiles a mixed set again in one run (1.4.0). The driver's memory shows as Shmem (Unevictable) in `/proc/meminfo`, not in
+  the process's RSS.
+- With the NPU 4000 Linux driver, a process creating more than about 40 inference requests (command queues) failed to get a
+  doorbell (`Failed to allocate doorbell ID: -16`), and from then until a reboot no process could run inference on the NPU
+  (`Failed to create command queue: -74`).

onw/__init__.py

@@ -2,4 +2,4 @@
 
 A small engine that runs LLMs entirely on the Intel NPU: static segment graphs + host-managed state.
 """
-__version__ = "1.3.0"
+__version__ = "1.4.0"

onw/app.py

@@ -13,15 +13,15 @@ import json, os, shutil, subprocess, sys, threading, time, urllib.request, webbr
 CATALOG = [  # repo, title, takes images, download GB, decode speed (NPU 3720), note (shown through i18n.T)
     ("ryugyosoft/gemma-4-E4B-it-onw", "Gemma 4 E4B", True, 4.3, "7 tok/s", "画像も読めます"),
     ("ryugyosoft/Qwen3.5-9B-onw", "Qwen3.5-9B", True, 5.3, "4 tok/s", "画像も読めます"),
-    ("ryugyosoft/Qwen3.5-9B-nf4-onw", "Qwen3.5-9B (NF4)", True, 5.2, "9.4 tok/s",
-     "Lunar Lake 以降専用の軽量・高速版（重み NF4。通常版よりメモリ約 2 割減・生成約 3 割速い）"),
+    ("ryugyosoft/Qwen3.5-9B-nf4-onw", "Qwen3.5-9B (NF4)", True, 5.6, "15 tok/s",
+     "Lunar Lake 以降専用の軽量・高速版（重み NF4・MTP 先読み付き。通常版よりメモリ約 15% 減・生成約 2 倍）"),
     ("ryugyosoft/Qwen3.6-REAP-18B-A3B-onw", "Qwen3.6-REAP-18B-A3B", True, 9.6, "6.8 tok/s", "35B MoE を半分に刈り込み"),
     ("ryugyosoft/Qwen3.6-35B-A3B-onw", "Qwen3.6-35B-A3B", True, 17.1, "6.8 tok/s",
      "最大（メモリが足りない PC ではエキスパートを SSD から読みます）"),
     ("ryugyosoft/Ornith-1.5-9B-onw", "Ornith-1.5-9B", True, 5.3, "5 tok/s",
      "エージェント・ツール呼び出し向け（Qwen3.5-9B ベース）"),
-    ("ryugyosoft/Ornith-1.5-9B-nf4-onw", "Ornith-1.5-9B (NF4)", True, 5.2, "9.3 tok/s",
-     "Lunar Lake 以降専用の軽量・高速版（重み NF4。通常版よりメモリ約 2 割減・生成約 3 割速い）"),
+    ("ryugyosoft/Ornith-1.5-9B-nf4-onw", "Ornith-1.5-9B (NF4)", True, 5.6, "15 tok/s",
+     "Lunar Lake 以降専用の軽量・高速版（重み NF4・MTP 先読み付き。通常版よりメモリ約 15% 減・生成約 2 倍）"),
     ("ryugyosoft/NeoHorse-1-4B-onw", "NeoHorse-1-4B", False, 2.4, "6.4 tok/s",
      "エージェント・ツール呼び出し向けの軽量モデル（Qwen3.5-4B ベース）"),
     ("ryugyosoft/granite-4.2-3b-onw", "Granite 4.2 3B", False, 2.1, "7.7 tok/s",
@@ -57,12 +57,13 @@ def catalog():
 MEMORY = {
     "ryugyosoft/gemma-4-E4B-it-onw": (8.4, 1.2, 8),
     "ryugyosoft/Qwen3.5-9B-onw": (10.9, 0.0, 16),
-    "ryugyosoft/Qwen3.5-9B-nf4-onw": (8.3, 0.0, 16),         # NF4: 5.1 GB against 6.6 GB on NPU 4000, scaled
+    # NF4: 5.1 GB against 6.6 GB on NPU 4000, scaled; onw 1.4.0 adds the MTP layer and the 64-token blocks (+0.7 GB)
+    "ryugyosoft/Qwen3.5-9B-nf4-onw": (9.0, 0.0, 16),
     "ryugyosoft/Qwen3.6-REAP-18B-A3B-onw": (12.3, 0.0, 10),
     "ryugyosoft/Qwen3.6-35B-A3B-onw": (19.9, 0.0, 10),
     "ryugyosoft/Ornith-1.5-9B-onw": (10.9, 0.0, 16),            # the shape of Qwen3.5-9B
     # NF4: on NPU 4000 5.1 GB against 6.5 GB for the regular build (system memory after load and an answer), scaled
-    "ryugyosoft/Ornith-1.5-9B-nf4-onw": (8.6, 0.0, 16),
+    "ryugyosoft/Ornith-1.5-9B-nf4-onw": (9.3, 0.0, 16),
     "ryugyosoft/NeoHorse-1-4B-onw": (6.0, 0.0, 16),             # RSS 5.75 GB measured (onw 0.12.1)
     "ryugyosoft/granite-4.2-3b-onw": (5.6, 0.0, 80),            # 40 layers x 8 kv heads x 128: 80 KB per token
     "ryugyosoft/granite-4.2-8b-onw": (13.1, 0.0, 80),           # (NPU 3720: one weight copy per block size)
@@ -500,13 +501,16 @@ def model_title(key):
 def outdated(repo, p):
     """A downloaded catalog model whose files were replaced on the hub since (downloading again into the same folder
     fetches just the changed files): the gpt-oss-20b models published with an INT4 embedding table (v0.11.0 era) -
-    the INT8 one is noticeably more accurate."""
-    if not repo.startswith("ryugyosoft/gpt-oss-20b") or not downloaded_by_onw(p):
+    the INT8 one is noticeably more accurate; the NF4 Qwen3.5 models published before the MTP layer and the 64-token
+    blocks (onw 1.4.0: ~1.3x the answer and 2x the prompt speed on Lunar Lake)."""
+    gpt, nf4 = repo.startswith("ryugyosoft/gpt-oss-20b"), repo in NF4_ONLY
+    if not (gpt or nf4) or not downloaded_by_onw(p):
         return False
     try:
-        return "embed" in (json.load(open(os.path.join(p, "engine.json"))).get("tables") or {})
+        meta = json.load(open(os.path.join(p, "engine.json")))
     except (OSError, ValueError):
         return False
+    return "embed" in (meta.get("tables") or {}) if gpt else "mtp" not in meta
 
 
 def downloaded_by_onw(p):

onw/chat.py

@@ -33,8 +33,25 @@ def total_ram_gb():
         return None
 
 
-def block_sizes(meta):
-    """Block sizes to load. Prompt blocks above 16 tokens prefill faster but every size holds its own copy of the
+def shares_weights(core, device, engine_dir, meta):
+    """Every block size can read one copy of the weights: NPUW on NPU 4000+ (Lunar Lake), for dense models (MoE
+    models hold their experts once anyway). Whether the sizes picked fit what the driver holds is checked by the
+    caller (ChatEngine; runtime.SegmentedModel turns NPUW off past that)."""
+    from .runtime import use_npuw
+    return device == "NPU" and use_npuw(core) and not os.path.exists(os.path.join(engine_dir, "experts.bin"))
+
+
+def use_mtp(meta, shared):
+    """Draft tokens from the model's MTP layer (Qwen3.5 dense): where a 16-token verification block costs about one
+    single step - NPU 4000+ with NPUW (Lunar Lake: ~1.3x the answer speed). On NPU 3720 a block costs 2.4 steps and
+    MTP came out slower than without. ONW_MTP=1 / 0 forces it on / off."""
+    env = os.environ.get("ONW_MTP")
+    return bool(meta.get("mtp")) and (env == "1" if env in ("0", "1") else shared)
+
+
+def block_sizes(meta, shared=False, mtp=False):
+    """Block sizes to load. Prompt blocks above 16 tokens prefill faster (Lunar Lake: 64-token blocks ~2x); with the
+    weights shared between the sizes (NPUW) they cost next to nothing, else every size holds its own copy of the
     in-graph weights, so they are used only with >= 24 GB RAM; ONW_S64=1 / 0 forces them on / off."""
     sizes = meta["sizes"]
     only = os.environ.get("ONW_SIZES")                  # e.g. "16": decode with the 16-token graphs only
@@ -42,8 +59,23 @@ def block_sizes(meta):
         sizes = [S for S in sizes if str(S) in only.split(",")] or sizes
     env = os.environ.get("ONW_S64")
     ram = total_ram_gb()
-    big = env == "1" if env in ("0", "1") else (ram is not None and ram >= 24)
-    return [S for S in sizes if S <= 16 or big]
+    big = env == "1" if env in ("0", "1") else (shared or (ram is not None and ram >= 24))
+    sizes = [S for S in sizes if S <= 16 or big]
+    if not only:
+        small = [S for S in sizes if S <= 16]
+        verify = [S for S in small if S >= 4]
+        if shared and mtp and verify:
+            # MTP drafts: every answer step is a verification block (x + 2 drafts, + rows to redo) on the smallest
+            # graph of >= 4 rows (Lunar Lake: the 4-token one ~15% cheaper than the 16-token one); no 1-token graphs
+            # (no single steps are left, and the NPU driver runs out of command queues past ~40 compiled parts:
+            # Ornith 9B has 13 per size + 3 for MTP)
+            keep = {min(verify)}
+        else:
+            # otherwise the 1- and 16-token graphs (4 / 8-token ones stay out: without NPUW each size holds its own
+            # copy of the weights)
+            keep = {S for S in small if S in (1, 16)} or set(small)
+        sizes = [S for S in sizes if S > 16 or S in keep]
+    return sizes
 
 
 # Thinking levels ("low" / "medium" / "high", the API's reasoning_effort): harmony models (gpt-oss, llm-jp) were trained
@@ -58,6 +90,7 @@ class ChatEngine:
     swap_strict = False
     _split_ids = None                          # harmony tokens to cut decoding at (llm-jp-4), see __init__
     _force = ()                                # tokens _plain emits next instead of sampling (see THINK_BUDGET)
+    _mtp_pend = None                           # MTP: the prompt's last row (normalized hidden state, position)
 
     def __init__(self, engine_dir, device=None, pld=True, cache_dir=None, context=None):
         core = ov.Core()
@@ -65,9 +98,16 @@ class ChatEngine:
         cfg = {"CACHE_DIR": cache_dir or os.path.join(engine_dir, "npu_cache")} if device == "NPU" else \
             {"INFERENCE_PRECISION_HINT": "f32"}
         t0 = time.time()
-        self.model = SegmentedModel(core, engine_dir, device, cfg,
-                                    sizes=block_sizes(json.load(open(os.path.join(engine_dir, "engine.json")))),
-                                    context=context)
+        meta = json.load(open(os.path.join(engine_dir, "engine.json")))
+        shared = shares_weights(core, device, engine_dir, meta)
+        mtp = use_mtp(meta, shared)
+        sizes = block_sizes(meta, shared, mtp)
+        from . import parts
+        if shared and parts.estimate(meta, sizes) > parts.MANY:   # (NPUW goes off for so many parts: no sharing)
+            shared = False
+            mtp = use_mtp(meta, shared)
+            sizes = block_sizes(meta, shared, mtp)
+        self.model = SegmentedModel(core, engine_dir, device, cfg, sizes=sizes, context=context, mtp=mtp)
         self.tok = AutoTokenizer.from_pretrained(engine_dir)
         # thinking: "switch" = the template takes enable_thinking (Qwen); "forced" = its generation prompt always opens
         # <think> (LFM2.5), skipped by closing it at once unless thinking is asked for
@@ -390,9 +430,22 @@ class ChatEngine:
                 k = min(m.S, stop - s)
                 lg = m.step(ids[s:s + k], None if embeds is None else embeds[s:s + k],
                             None if pos is None else pos[:, s:s + k], want_logits=(s + k >= stop and need_logits))
+                if m.mtp is not None:
+                    self._mtp_prompt(ids, s, k)
             s += k
         return lg[-1] if lg is not None else None
 
+    def _mtp_prompt(self, ids, s, k):
+        """After a prompt block ids[s:s+k]: the MTP layer's KV rows for its positions (row j: the main model's hidden
+        state at j and the token at j + 1). The prompt's last row waits for the first answer token (_mtp_pend)."""
+        m = self.model
+        hn = m.mtp_norm(m.last_h, 0)
+        full = min(k, len(ids) - 1 - s)                # rows whose next token is known
+        for a in range(0, full, 16):
+            b = min(a + 16, full)
+            m.mtp_layer(hn[a:b], ids[s + a + 1:s + b + 1], s + a)
+        self._mtp_pend = (hn[full:full + 1], s + full) if full < k else None
+
     # ---- token sources ----
     @staticmethod
     def _sample(lg, s, rng, seen=None, counts=None):
@@ -418,6 +471,115 @@ class ChatEngine:
         k = o[:max(1, int(np.searchsorted(np.cumsum(p[o]), s["top_p"]) + 1))]
         return int(k[rng.choice(len(k), p=p[k] / p[k].sum())])
 
+    @staticmethod
+    def _probs(lg, s, seen=None, counts=None):
+        """The distribution _sample draws from, over the whole vocabulary (float64, sums to 1); greedy: one-hot."""
+        z = lg.astype(np.float64)
+        if s["repetition_penalty"] != 1.0 and seen:
+            i = np.fromiter(seen, np.int64)
+            z[i] = np.where(z[i] > 0, z[i] / s["repetition_penalty"], z[i] * s["repetition_penalty"])
+        if counts and (s["presence_penalty"] or s["frequency_penalty"]):
+            i = np.fromiter(counts.keys(), np.int64)
+            c = np.fromiter(counts.values(), np.float64)
+            z[i] -= s["presence_penalty"] + s["frequency_penalty"] * c
+        p = np.zeros(len(z))
+        if s["temperature"] <= 0:
+            p[int(z.argmax())] = 1.0
+            return p
+        z /= s["temperature"]
+        if 0 < s["top_k"] < len(z):
+            cut = np.partition(z, -s["top_k"])[-s["top_k"]]
+            z[z < cut] = -np.inf
+        q = np.exp(z - z.max())
+        q /= q.sum()
+        o = np.argsort(-q)
+        k = o[:max(1, int(np.searchsorted(np.cumsum(q[o]), s["top_p"]) + 1))]
+        p[k] = q[k] / q[k].sum()
+        return p
+
+    def _mtp(self, lg, s, rng):
+        """Decoding with draft tokens from the model's MTP layer, verified in one block pass: a draft d is kept with
+        probability p(d) under the sampling distribution p of its row (greedy: when it is the argmax), else the row's
+        token is drawn from p without d - the same distribution as sampling token by token (speculative sampling with
+        a deterministic draft). Rows after a rejection are dropped like _pld does. The MTP layer drafts from the main
+        model's hidden state of the last accepted position and the token after it, then from its own output."""
+        m, K = self.model, max(1, int(os.environ.get("ONW_MTP_K", "2")))
+        # verification blocks: on the smallest loaded graph of >= 4 rows (x + 2 drafts + a row to redo), at most 16
+        V = min(min((S for S in m.segs if S >= 4), default=m.S), 16)
+        seen, counts = set(self.ctx), {}
+
+        plain = s["temperature"] <= 0 and s["repetition_penalty"] == 1.0 and not s["presence_penalty"] and \
+            not s["frequency_penalty"]
+
+        def take(row, d=None):
+            """The token of a row: (token, the draft was kept)."""
+            if plain:                                  # (what the general path below gives for a one-hot p)
+                t = int(row.argmax())
+                return t, d is not None and t == d
+            p = self._probs(row, s, seen, counts)
+            if d is not None and (p[d] >= 1.0 or (p[d] > 0 and rng.random() < p[d])):
+                return d, True
+            if d is not None:
+                p[d] = 0.0
+                if p.sum() <= 0:
+                    return d, True
+                p /= p.sum()
+            return (int(p.argmax()) if s["temperature"] <= 0 else int(rng.choice(len(p), p=p))), False
+
+        def note(t):
+            seen.add(t)
+            counts[t] = counts.get(t, 0) + 1
+
+        pend, self._mtp_pend = self._mtp_pend, None   # the prompt's last row: (normalized hidden, position)
+        fill_h, fs = (pend[0], pend[1]) if pend is not None else (None, None)
+        x = self._force.pop(0) if self._force else take(lg)[0]
+        fill_t, redo = [x], []
+        while True:
+            forced = self._force[:V - 1 - len(redo)]
+            del self._force[:len(forced)]
+            room = V - 1 - len(redo) - len(forced)
+            drafts = []
+            if fill_h is not None and not forced and room > 0 and m.cur + V <= m.T:
+                z = m.mtp_layer(fill_h, fill_t, fs)[-1]
+                pos = fs + len(fill_t)
+                for j in range(min(K, room)):
+                    zn = m.mtp_norm(z, 1)
+                    drafts.append(m.mtp_draft(zn))
+                    if j + 1 < min(K, room):
+                        z = m.mtp_layer(zn[None], drafts[-1:], pos)[0]
+                        pos += 1
+            start = m.cur
+            block, o = redo + [x] + forced + drafts, len(redo)
+            lgs = m.step(block)
+            hn = m.mtp_norm(m.last_h, 0)
+            acc = [x] + forced
+            note(x)
+            for t in forced:
+                note(t)
+            r = o + len(acc) - 1                       # the row deciding the next token
+            for d in drafts:
+                t, kept = take(lgs[r], d)
+                if not kept:
+                    y = t
+                    break
+                note(d)                                # (the next row's context; y is noted as the next x)
+                acc.append(d)
+                r += 1
+            else:
+                y, _ = take(lgs[r]) if not self._force else (self._force.pop(0), True)
+            a = len(acc) - 1 - len(forced)             # drafts kept
+            if drafts:
+                self.mtp_stats[0] += 1
+                self.mtp_stats[1] += a
+            redo = []
+            if len(acc) - 1 < len(forced) + len(drafts) and not m.truncate(o + len(acc)):
+                redo = block[:o + len(acc)]
+            # the MTP rows for the accepted positions (and the draft of the next block from the last one)
+            fill_h, fs, fill_t = hn[o:o + len(acc)], start + o, acc[1:] + [y]
+            self.ctx.extend(acc)
+            yield from acc
+            x = y
+
     def _plain(self, lg, s, rng):
         seen, counts = set(self.ctx), {}
         while True:
@@ -549,8 +711,11 @@ class ChatEngine:
             self.model.start_size(1, need_gb=self.swap_gb + 1.0, then_drop=16)
         # exact only for plain greedy; not while block sizes are swapped (the verification blocks go away mid-answer)
         use_pld = self.pld and plain_greedy and self.model.S > 1 and not swap and budget is None
-        self.pld_stats = [0, 0]
-        src = self._pld(lg) if use_pld else self._plain(lg, samp, rng)
+        # MTP drafts (Qwen3.5 dense): every sampling setting (exact), the thinking budget included
+        mtp = getattr(self.model, "mtp", None) is not None and not swap and any(S >= 4 for S in self.model.segs)
+        use_pld = use_pld and not mtp
+        self.pld_stats, self.mtp_stats = [0, 0], [0, 0]
+        src = self._mtp(lg, samp, rng) if mtp else self._pld(lg) if use_pld else self._plain(lg, samp, rng)
         out, finish, held = [], "length", 0
         sent = {"r": "", "t": ""}                            # what has gone out: thinking, answer
         stop = [x for x in stop if x]
@@ -637,4 +802,5 @@ class ChatEngine:
         yield {"finish_reason": finish, "prompt_tokens": n, "cached_tokens": reuse, "completion_tokens": len(out),
                "reasoning_tokens": min(len(out), len(self.tok(thought, add_special_tokens=False)["input_ids"])) if thought else 0,
                "vision_ms": t_vis * 1000, "prefill_ms": t_pre * 1000, "decode_tok_s": len(out) / max(time.time() - t1, 1e-9),
-               "pld": {"blocks": self.pld_stats[0], "accepted": self.pld_stats[1]} if use_pld else None}
+               "pld": {"blocks": self.pld_stats[0], "accepted": self.pld_stats[1]} if use_pld else None,
+               "mtp": {"blocks": self.mtp_stats[0], "accepted": self.mtp_stats[1]} if mtp else None}

onw/i18n.json

@@ -18,7 +18,7 @@
 "IBM の軽量モデル。思考とツール呼び出しに対応（日本語はやや弱め）": "IBM's light model, with thinking and tool calls (Japanese a bit weaker)",
 "LAN からの接続を許可する": "Allow connections from the LAN",
 "Lunar Lake 以降専用の軽量・高速版": "Lighter, faster build for Lunar Lake and newer only",
-"Lunar Lake 以降専用の軽量・高速版（重み NF4。通常版よりメモリ約 2 割減・生成約 3 割速い）": "Lighter, faster build for Lunar Lake and newer only (NF4 weights: ~20% less memory, ~30% faster decode than the regular one)",
+"Lunar Lake 以降専用の軽量・高速版（重み NF4・MTP 先読み付き。通常版よりメモリ約 15% 減・生成約 2 倍）": "Lighter, faster build for Lunar Lake and newer only (NF4 weights, MTP draft tokens: ~15% less memory, ~2x the decode speed of the regular one)",
 "MoE モデルでも NPUW 重み共有を使う（実験的）": "Use NPUW weight sharing for MoE models too (experimental)",
 "NPU 向けの準備 {0}/{1}（初回だけ数分〜15 分ほどかかります）": "Preparing for the NPU {0}/{1} (the first time only, a few to ~15 minutes)",
 "NPUW の非同期実行（FUNCALL_ASYNC、実験的）": "NPUW asynchronous execution (FUNCALL_ASYNC, experimental)",
@@ -30,7 +30,7 @@
 "URL をコピーしました": "URL copied",
 "auto（Lunar Lake 以降でオン）": "auto (on for Lunar Lake+)",
 "auto（OS に合わせる）": "auto (follow the OS)",
-"auto（RAM 24 GB 以上でオン）": "auto (on at 24 GB+ RAM)",
+"auto（Lunar Lake 以降か RAM 24 GB 以上でオン）": "auto (on with Lunar Lake or later, or 24 GB+ RAM)",
 "auto（足りないときだけ SSD から）": "auto (SSD when RAM is short)",
 "gpt-oss-20b の高精度版": "The higher-precision gpt-oss-20b",
 "gpt-oss-20b の高精度版（エキスパート INT8。メモリ 24 GB 以上向け）": "The higher-precision gpt-oss-20b (INT8 experts; for 24 GB+ of RAM)",
@@ -189,6 +189,7 @@
 "停止中": "Stopped",
 "元に戻す": "Discard",
 "先読み検証（PLD）": "Prompt lookup (PLD)",
+"MTP 先読み": "MTP draft tokens",
 "先読み検証（PLD）で高速化する": "Speed up with prompt lookup (PLD)",
 "入れ替え中": "Replacing files",
 "再生成": "Regenerate",
@@ -340,7 +341,7 @@
 "IBM の軽量モデル。思考とツール呼び出しに対応（日本語はやや弱め）": "IBM 的轻量模型，支持思考和工具调用（日语稍弱）",
 "LAN からの接続を許可する": "允许来自局域网的连接",
 "Lunar Lake 以降専用の軽量・高速版": "仅限 Lunar Lake 及更新机型的轻量高速版",
-"Lunar Lake 以降専用の軽量・高速版（重み NF4。通常版よりメモリ約 2 割減・生成約 3 割速い）": "仅限 Lunar Lake 及更新机型的轻量高速版（NF4 权重：比普通版内存约少 20%、生成约快 30%）",
+"Lunar Lake 以降専用の軽量・高速版（重み NF4・MTP 先読み付き。通常版よりメモリ約 15% 減・生成約 2 倍）": "仅限 Lunar Lake 及更新机型的轻量高速版（NF4 权重、MTP 草稿预测：比普通版内存约少 15%、生成约 2 倍）",
 "MoE モデルでも NPUW 重み共有を使う（実験的）": "MoE 模型也使用 NPUW 权重共享（实验性）",
 "NPU 向けの準備 {0}/{1}（初回だけ数分〜15 分ほどかかります）": "正在为 NPU 准备 {0}/{1}（仅首次，需要几分钟到约 15 分钟）",
 "NPUW の非同期実行（FUNCALL_ASYNC、実験的）": "NPUW 异步执行（FUNCALL_ASYNC，实验性）",
@@ -352,7 +353,7 @@
 "URL をコピーしました": "已复制 URL",
 "auto（Lunar Lake 以降でオン）": "auto（Lunar Lake 起开启）",
 "auto（OS に合わせる）": "auto（跟随系统）",
-"auto（RAM 24 GB 以上でオン）": "auto（内存 24 GB 以上时开启）",
+"auto（Lunar Lake 以降か RAM 24 GB 以上でオン）": "auto（Lunar Lake 及更新机型或内存 24 GB 以上时开启）",
 "auto（足りないときだけ SSD から）": "auto（内存不足时用 SSD）",
 "gpt-oss-20b の高精度版": "gpt-oss-20b 的高精度版",
 "gpt-oss-20b の高精度版（エキスパート INT8。メモリ 24 GB 以上向け）": "gpt-oss-20b 的高精度版（INT8 专家；面向 24 GB 以上内存）",
@@ -511,6 +512,7 @@
 "停止中": "已停止",
 "元に戻す": "撤销",
 "先読み検証（PLD）": "提示查找（PLD）",
+"MTP 先読み": "MTP 草稿预测",
 "先読み検証（PLD）で高速化する": "使用提示查找（PLD）加速",
 "入れ替え中": "正在替换文件",
 "再生成": "重新生成",

onw/ir.py

@@ -367,6 +367,34 @@ def deltanet_chunk(q, k, v, g, beta, S0, T):
     return O, ST
 
 
+def deltanet_chunks(q, k, v, g, beta, S0, T, chunk=16, rows=False):
+    """deltanet_chunk over T tokens as consecutive chunks of `chunk` (the state carried from one to the next): blocks
+    above 16 tokens keep the numerics of the 16-token chunk (fp16 on the NPU: one long chunk's cumulative decays and
+    T x T inverse drift), while the projections around it still run on all T rows at once. rows=True (1 < T < chunk):
+    also the state after every row, [T, h, dk, dv] - a verification block can then be cut back exactly."""
+    assert not rows or 1 < T < chunk
+    if 1 < T < chunk:
+        # blocks below one chunk (the 4-token verification graphs): token by token in the 1-token form (the S=1
+        # graphs' math). The chunk form at T < 16 fails in the NPU 4000 compiler ([1, h, T, T] tensors: "failed to
+        # legalize unresolved materialization"), and padded to a 16-token chunk it costs as much as the 16-token graph
+        sl1 = lambda t, a: ops.slice(t, np.array([a]), np.array([a + 1]), np.array([1]), np.array([2]))
+        outs, S, Ss = [], S0, []
+        for a in range(T):
+            o, S = deltanet_step(sl1(q, a), sl1(k, a), sl1(v, a), sl1(g, a), sl1(beta, a), S)
+            outs.append(o)
+            Ss.append(S)
+        return (ops.concat(outs, 2), S, ops.concat(Ss, 0)) if rows else (ops.concat(outs, 2), S)
+    if T <= chunk:
+        return deltanet_chunk(q, k, v, g, beta, S0, T)
+    sl = lambda t, a, b_, ax: ops.slice(t, np.array([a]), np.array([b_]), np.array([1]), np.array([ax]))
+    outs, S = [], S0
+    for a in range(0, T, chunk):
+        b_ = min(a + chunk, T)
+        o, S = deltanet_chunk(sl(q, a, b_, 2), sl(k, a, b_, 2), sl(v, a, b_, 2), sl(g, a, b_, 2), sl(beta, a, b_, 2), S, b_ - a)
+        outs.append(o)
+    return ops.concat(outs, 2), S
+
+
 def rope_partial(x, cos, sin, rot):
     """Rotate the first `rot` dims of x [1,h,S,D] (rotate_half convention); cos/sin [1,1,S,rot]."""
     D = dim(x, 3)

onw/kvcache.py

@@ -96,6 +96,7 @@ class PagedKV:
         self.layers = {n: _Layer(s["kv"], s["D"], s.get("Dv", s["D"]), s["scale"], s["G"])
                        for n, s in layers.items() if self.owner[n] == n}
         self.graphs, self._open = {}, {}
+        self._masks = {}                                # (R, C) -> (key, mask): the last partly visible block's mask
 
     # ---- storage ----
     def _tensor(self, t, shape):
@@ -183,16 +184,18 @@ class PagedKV:
                 self.graphs[key] = self.core.compile_model(m, self.device, self.config).create_infer_request()
         return self.graphs[key]
 
-    def attend(self, name, q, pos, S, n, sink=None):
+    def attend(self, name, q, pos, S, n, sink=None, rows_out=False):
         """q [kv, G*S, D] (rows g*S + s) for tokens at pos..pos+S-1 (the first n real) -> o [kv, G*S, Dv] f32, over
         the stored positions <= each row's own (sliding-window layers are not paged: they keep a ring in the graph).
-        sink [kv, G]: a learned logit per head that takes part in the softmax without a value (gpt-oss)."""
+        sink [kv, G]: a learned logit per head that takes part in the softmax without a value (gpt-oss).
+        rows_out: o as the next segment takes it instead - [1, S, kv * G * Dv] f16 (token-major)."""
         L = self.layers[self.owner[name]]
         R = q.shape[1]
         rows = pos + (np.arange(R) % S)                                     # each row's position
         end = pos + n                                                        # stored positions: [0, end)
         q = np.ascontiguousarray(q, np.float16)
         M = Lsum = O = None
+        used = sum(1 for b in L.blocks if b["start"] < end)                  # blocks holding visible positions
         for b in L.blocks:
             s0, C = b["start"], b["C"]
             if s0 >= end:
@@ -201,14 +204,31 @@ class PagedKV:
                 mask = self._open.get((R, C))                              # every slot visible to every row
                 if mask is None:
                     mask = self._open[(R, C)] = np.zeros((R, C), np.float16)
-            else:
-                j = s0 + np.arange(C)
-                ok = (j[None, :] <= rows[:, None]) & (j[None, :] < end)
-                mask = np.where(ok, 0.0, MASKED).astype(np.float16)
+            else:                                    # (the same for every layer of a block: built once)
+                key = (s0, pos, S, n)
+                hit = self._masks.get((R, C))
+                if hit is not None and hit[0] == key:
+                    mask = hit[1]
+                else:
+                    j = s0 + np.arange(C)
+                    ok = (j[None, :] <= rows[:, None]) & (j[None, :] < end)
+                    mask = np.where(ok, 0.0, MASKED).astype(np.float16)
+                    self._masks[(R, C)] = (key, mask)
             req = self._graph(L, R, C)
             req.set_tensor("k", b["k"])
             req.set_tensor("v", b["v"])
-            r = req.infer({"q": q, "ks": b["ks"], "vs": b["vs"], "mask": mask})
+            # (shared, not copied: every input is a contiguous f16 array, as the graph takes them)
+            r = req.infer({"q": q, "ks": b["ks"], "vs": b["vs"], "mask": mask}, share_inputs=True)
+            if rows_out and sink is None and used == 1:
+                # one block, nothing to merge: o / l straight into the token-major f16 layout (computed in f32 and
+                # rounded once, the same values as the general path below)
+                kvh, G = q.shape[0], R // S
+                Dv = r["o"].shape[2]
+                out = np.empty((S, kvh, G, Dv), np.float16)
+                np.divide(r["o"].reshape(kvh, G, S, Dv).transpose(2, 0, 1, 3),
+                          np.asarray(r["l"], np.float32).reshape(kvh, G, S, 1).transpose(2, 0, 1, 3),
+                          out=out, casting="unsafe")
+                return out.reshape(1, S, kvh * G * Dv)
             m, l, o = (np.asarray(r[x], np.float32) for x in ("m", "l", "o"))
             if M is None:
                 M, Lsum, O = m, l, o
@@ -218,4 +238,8 @@ class PagedKV:
                 M, Lsum, O = nm, Lsum * a + l * c, O * a + o * c
         if sink is not None:                                                 # one more softmax term, no value
             Lsum = Lsum + np.exp(np.repeat(np.asarray(sink, np.float32), S, 1)[:, :, None] - M)
+        if rows_out:
+            kvh, R, Dv = O.shape
+            return (O / Lsum).reshape(kvh, R // S, S, Dv).transpose(2, 0, 1, 3).reshape(1, S, kvh * (R // S) * Dv) \
+                .astype(np.float16)
         return O / Lsum

onw/models/qwen3_5_draft_vocab.npy

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9c288259be0973c6800bc3ff92054c1f0e1335cf0cbd1b5ecd9f16873d85337f
+size 131200

onw/models/qwen3_5_moe.py

@@ -37,11 +37,25 @@ def rotary(cfg):
     return int(cfg["head_dim"] * frac), rp.get("rope_theta", cfg.get("rope_theta", 10000.0))
 
 
-def build_segments(cfg, W, S, T, fmt="q4g128", dn_fmt=None, norm_maxscale=False, P=P, attn_fmt=None):
+class _MTPWeights:
+    """The checkpoint with the MTP layer (mtp.layers.0.*) under the name of a layer after the last one, so the
+    builders below make it like any full-attention layer (its paged KV cache gets that layer number)."""
+    def __init__(self, W, P, L):
+        self.W, self.src, self.where = W, f"{P}layers.{L}.", W.where
+
+    def __getitem__(self, k):
+        return self.W[k.replace(self.src, "mtp.layers.0.", 1)]
+
+
+def build_segments(cfg, W, S, T, fmt="q4g128", dn_fmt=None, norm_maxscale=False, P=P, attn_fmt=None, mtp=False,
+                   draft_vocab=None):
     """dn_fmt: format of the DeltaNet in_proj_qkv / out_proj weights (default fmt); attn_fmt: the full-attention
     q/k/v_proj (default fmt; NF4 models keep them INT8); norm_maxscale: residual-stream
     RMSNorms with ir.rmsnorm(maxscale=True); P: checkpoint key prefix (text-only "qwen3_5_text" repos use "model.").
+    mtp=True: the multi-token-prediction layer instead (see export_mtp; draft_vocab: the token ids its head scores).
     See export()."""
+    if mtp:
+        W = _MTPWeights(W, P, cfg["num_hidden_layers"])
     H, E, K, I = cfg["hidden_size"], cfg.get("num_experts", 0), cfg.get("num_experts_per_tok", 0), cfg.get("moe_intermediate_size")
     nh, nkv, D = cfg["num_attention_heads"], cfg["num_key_value_heads"], cfg["head_dim"]
     rot = rotary(cfg)[0]
@@ -99,7 +113,13 @@ def build_segments(cfg, W, S, T, fmt="q4g128", dn_fmt=None, norm_maxscale=False,
         beta = ops.transpose(beta, i64([0, 2, 1]))
         s0 = ops.parameter([1, vh, dk, dv], ov.Type.f32, name=f"ssm.{l}")
         params.append(s0)
-        o, s1 = ir.deltanet_step(q, k, v, g, beta, s0) if S == 1 else ir.deltanet_chunk(q, k, v, g, beta, s0, S)
+        if S == 1:
+            o, s1 = ir.deltanet_step(q, k, v, g, beta, s0)
+        elif S < 16:                            # verification blocks: every row's state too (exact truncation)
+            o, s1, srows = ir.deltanet_chunks(q, k, v, g, beta, s0, S, rows=True)
+            outs[f"ssm_rows.{l}"] = srows
+        else:
+            o, s1 = ir.deltanet_chunks(q, k, v, g, beta, s0, S)
         outs[f"ssm_new.{l}"] = s1
         o = ops.transpose(o, i64([0, 2, 1, 3]))                                 # [1,S,vh,dv]
         o = ops.multiply(ir.rmsnorm(o, W[a + "norm.weight"], eps * QSCALE ** 2, prescale=QSCALE / 16), ir.silu(z))
@@ -160,6 +180,29 @@ def build_segments(cfg, W, S, T, fmt="q4g128", dn_fmt=None, norm_maxscale=False,
         u = ir.linear(xn, W[m + "up_proj.weight"], fmt)
         return ops.add(h, ir.linear(ops.multiply(ir.silu(g), u), W[m + "down_proj.weight"], fmt))
 
+    if mtp:
+        # draft tokens (Qwen3.5 MTP, as in vLLM's qwen3_next_mtp): for position i, the main model's final-normed
+        # hidden state h_i and the embedding of token i+1 -> fc([norm(e), norm(h)]) -> one gated full-attention
+        # decoder layer (its own KV cache) -> a hidden state whose mtp.norm feeds the LM head: token i+2. Both inputs
+        # come normalized-or-raw from the host: carry.mtp_e = the raw embedding row, carry.mtp_h = h after the final
+        # norm (or, chaining drafts, the previous MTP output after mtp.norm); next.mtp_y = before mtp.norm
+        params, outs, shared = [], {}, {}
+        e = ops.parameter([1, S, H], ov.Type.f32, name="carry.mtp_e")
+        hn = ops.parameter([1, S, H], ov.Type.f32, name="carry.mtp_h")
+        params += [e, hn]
+        x = ops.concat([norm1(e, "mtp.pre_fc_norm_embedding.weight"), norm1(hn, "mtp.pre_fc_norm_hidden.weight")], 2)
+        h = ir.linear(x, W["mtp.fc.weight"], fmt)
+        x = rnorm(h, f"{P}layers.{L}.input_layernorm.weight")
+        h = dense_mlp(L, ops.add(h, attention(L, x, params, outs, shared)))
+        outs["next.mtp_y"] = h
+        meta = {"index": 0, "consume_moe": None, "router_layer": None, "S": S, "slots": 0, "mtp": True}
+        hp = ops.parameter([1, 1, H], ov.Type.f32, name="carry.mtp_hn")   # (normalized on the host)
+        # the draft head scores only the most frequent tokens (draft_vocab, 32K of 248K: ~96% of what the model
+        # writes): 1/8 of the LM head's reads - a draft outside them is just a draft that is not kept
+        head = W["lm_head.weight" if "lm_head.weight" in W.where else f"{P}embed_tokens.weight"]
+        lg = ir.linear(hp, np.ascontiguousarray(head[np.asarray(draft_vocab)]), "q8")
+        return [(ir.finish(outs, params, f"q35_mtp_S{S}"), meta),
+                (ir.finish({"logits": lg}, [hp], "q35_mtp_head_S1"), dict(meta, index=1, S=1, head=True))]
     result = []
     if not E:                                                                   # dense: a few layers per segment
         per = int(os.environ.get("ONW_DENSE_PER", "8"))
@@ -231,7 +274,7 @@ def prefix(W):
 
 
 def export(src, out, sizes=(1, 16), T=1024, fmt="q4g128", graphs_only=False, dn_fmt=None, norm_maxscale=False,
-           attn_fmt=None):
+           attn_fmt=None, mtp_fmt="q8"):
     """dn_fmt="q8g128": Ornith-1.5-35B-A3B (a Qwen3.6-35B-A3B fine-tune) loses its first answer token with every
     weight in q4g128 (the MoE input of layer 0 is 22% off the bf16 model's, the DeltaNet out_proj and in_proj_qkv
     being most of it); with those two in INT8 it matches the bf16 model to the end of the answer.
@@ -273,4 +316,27 @@ def export(src, out, sizes=(1, 16), T=1024, fmt="q4g128", graphs_only=False, dn_
         print("vision tower saved", flush=True)
     if not graphs_only:
         export_weights(cfg, W, out, pre)
+    if mtp_fmt and cfg.get("mtp_num_hidden_layers") and "mtp.fc.weight" in W.where and not cfg.get("num_experts"):
+        meta["mtp"] = export_mtp(cfg, W, out, T, mtp_fmt, norm_maxscale, pre)
     common.write_engine(out, meta, graphs_only)             # (with graphs_only the vision entry is kept)
+
+
+def export_mtp(cfg, W, out, T, fmt, norm_maxscale, pre, S=16):
+    """The MTP layer for draft tokens (dense models): mtp_S16.xml (one 16-token graph does every MTP call - its cost
+    hardly depends on the rows) and mtp_head_S1.xml (an INT8 head over the 32K most frequent tokens, mtp_vocab.npy,
+    on one normalized row); mtp_norm.npy: the
+    final norm and mtp.norm weights (applied on the host, (1 + w) form). Compiled without NPUW (its own small copy of
+    the weights: INT8 ~0.25 GB for 9B) - it stays out of NPUW's part budget. Not in "segments": onw versions without
+    MTP do not see it."""
+    segs = {}
+    vocab = np.load(os.path.join(os.path.dirname(os.path.abspath(__file__)), "qwen3_5_draft_vocab.npy"))
+    vocab = vocab[vocab < cfg["vocab_size"]]
+    np.save(os.path.join(out, "mtp_vocab.npy"), vocab.astype(np.int32))
+    for m, sm in build_segments(cfg, W, S, T, fmt, None, norm_maxscale, pre, None, mtp=True, draft_vocab=vocab):
+        name = "mtp_head_S1.xml" if sm.get("head") else f"mtp_S{S}.xml"
+        ov.save_model(m, os.path.join(out, name))
+        segs[name] = sm
+    np.save(os.path.join(out, "mtp_norm.npy"),
+            np.stack([W[f"{pre}norm.weight"], W["mtp.norm.weight"]]).astype(np.float32))
+    print("MTP layer saved", flush=True)
+    return {"segments": segs, "norms": "mtp_norm.npy", "vocab": "mtp_vocab.npy", "fmt": fmt, "eps": cfg["rms_norm_eps"]}

onw/probe.py

@@ -17,13 +17,14 @@ def main():
     meta = json.load(open(f"{d}/engine.json"))
     bins = {}                                             # keeps the .bin buffer alive while compiling
     try:
-        model = read_segment(core, d, name, meta["segments"][name], npuw, bins, T, meta["T"])
+        segs = {**meta["segments"], **(meta.get("mtp") or {}).get("segments", {})}     # (the MTP graphs too)
+        model = read_segment(core, d, name, segs[name], npuw, bins, T, meta["T"])
         if os.environ.get("ONW_KV", "paged") != "legacy":  # the same graph the parent compiles (host-side rope)
             from . import paged
             paged.host_rope(model)
         if os.environ.get("ONW_RING"):                    # paged mode: sliding-window caches live in a ring
             from .runtime import ring_shapes
-            model.reshape(ring_shapes(model, int(os.environ["ONW_RING"]), meta["segments"][name]["S"]))
+            model.reshape(ring_shapes(model, int(os.environ["ONW_RING"]), segs[name]["S"]))
     except Exception as e:
         print(f"{type(e).__name__}: {e}", flush=True)
         sys.exit(2)

onw/runtime.py

@@ -207,6 +207,49 @@ def blob_path(cache_dir, name, parts):
 
 USED_BLOBS = set()                                    # onw_blobs files this process imported or wrote (see prune_cache)
 USED_NPUW = set()                                     # npuw/<graph>-<key> cache folders this process compiled from
+# NPUW's weights bank shares one copy of the weights only between graphs compiled in the same process: cache entries
+# written by different runs (the 64-token graphs switched on later, a few graphs compiled again after an update)
+# each come back with a copy of their own (Lunar Lake, Ornith-1.5-9B NF4: 1+16 compiled together 4.6 GB of driver
+# memory, 1+16 from one run + 64 from another 8.7 GB). Each folder records the run that compiled it (onw_session);
+# a load that finds more than one run, or folders without the record, compiles all of its graphs again in one run.
+NPUW_SESSION = os.urandom(8).hex()
+NPUW_SEEN = {}                                        # npuw folder -> the run that compiled it (None: unknown)
+LIGHT = "light"     # the record of a graph without weights to speak of (the LM-head segment): any run will do
+
+
+def weight_bytes(model):
+    """Bytes of an ov.Model's constants (its weights)."""
+    return sum(op.get_byte_size() for op in model.get_ordered_ops() if op.get_type_name() == "Constant")
+
+
+def npuw_check_runs(cache, graphs):
+    """Before compiling: the npuw folders of these graphs (seg00_S1, ...; any key) all from one run? If not - folders
+    of more than one run or without the record, or a block size with none at all next to sizes that have them (the
+    64-token graphs switched on later) - they are removed, so this load compiles every graph in one run instead of
+    holding a second copy of the weights (freed requests do not give theirs back)."""
+    nd = os.path.join(cache or "", "npuw")
+    if not cache or not os.path.isdir(nd):
+        return
+    dirs = [os.path.join(nd, f) for f in os.listdir(nd) if f.rsplit("-", 1)[0].split(".p")[0] in graphs]
+    runs, by_size = set(), {}             # (graphs without weights - LIGHT - do not count)
+    for d in dirs:
+        try:
+            r = open(os.path.join(d, "onw_session")).read().strip()
+        except OSError:
+            r = None
+        if r != LIGHT:
+            runs.add(r)
+            by_size.setdefault(os.path.basename(d).rsplit("-", 1)[0].split(".p")[0].rsplit("_S", 1)[1], set())
+    if not runs:
+        return
+    sizes = {g.rsplit("_S", 1)[1] for g in graphs}
+    if len(runs) == 1 and None not in runs and set(by_size) >= sizes:
+        return
+    import shutil
+    print(f"[npuw] compiled graphs from different runs (each would hold its own copy of the weights): compiling all "
+          f"{len(graphs)} segments again in one run", flush=True)
+    for d in dirs:
+        shutil.rmtree(d, ignore_errors=True)
 
 
 def npuw_cache_dir(cache, gname, key_parts):
@@ -397,7 +440,7 @@ class _Tried:
 
 
 class SegmentedModel:
-    def __init__(self, core, d, device="NPU", config=None, sizes=None, context=None):
+    def __init__(self, core, d, device="NPU", config=None, sizes=None, context=None, mtp=False):
         self.meta = json.load(open(os.path.join(d, "engine.json")))
         if self.meta.get("needs") == "nf4" and device == "NPU" and npu_arch(core) < 4000:
             raise ValueError("this model's weights are NF4, which only NPU 4 (Lunar Lake) and later can run - this NPU "
@@ -506,30 +549,61 @@ class SegmentedModel:
         except ImportError:
             rss0 = None
         todo = sum(1 for m in self.meta["segments"].values() if m["S"] in self.segs or m["S"] in precompile)
-        k = 0
-        for name, m in sorted(self.meta["segments"].items()):
-            if not (m["S"] in self.segs or m["S"] in precompile):   # (a block size not used: not counted either)
-                continue
-            # (read by the onw window: the first load compiles for 5-15 minutes, segments take about equally long)
-            print(f"[progress] segments {k}/{todo}", flush=True)
-            k += 1
-            if m["S"] in precompile:                   # compiled into the blob store and let go at once
-                reqs = self._build(name, m)
-                if reqs is None:
-                    precompile.discard(m["S"])
-                    deferred.remove(m["S"])
-                del reqs
-                self._trim()
-                continue
-            if m["S"] in self.segs and m["S"] not in dropped:
-                reqs = self._build(name, m)
-                if reqs is None:                       # no way to compile this block size here: run without it
-                    print(f"[compile] no {m['S']}-token graphs on this NPU / driver: running without them", flush=True)
-                    self.segs.pop(m["S"])
-                    dropped.add(m["S"])
+        NPUW_SEEN.clear()
+        if npuw and device == "NPU" and os.environ.get("ONW_OWN_BLOBS") != "0":
+            npuw_check_runs((config or {}).get("CACHE_DIR"),
+                            {n[:-4] for n, m in self.meta["segments"].items() if m["S"] in self.segs})
+        for attempt in range(2):
+            k = 0
+            for name, m in sorted(self.meta["segments"].items()):
+                if not (m["S"] in self.segs or m["S"] in precompile):   # (a block size not used: not counted either)
                     continue
-                for req, pm_ in reqs:
-                    self.segs[m["S"]].append((req, pm_, {i.get_any_name() for i in req.model_inputs}))
+                # (read by the onw window: the first load compiles for 5-15 minutes, segments take about equally long)
+                print(f"[progress] segments {k}/{todo}", flush=True)
+                k += 1
+                if m["S"] in precompile:                   # compiled into the blob store and let go at once
+                    reqs = self._build(name, m)
+                    if reqs is None:
+                        precompile.discard(m["S"])
+                        deferred.remove(m["S"])
+                    del reqs
+                    self._trim()
+                    continue
+                if m["S"] in self.segs and m["S"] not in dropped:
+                    reqs = self._build(name, m)
+                    if reqs is None:                       # no way to compile this block size here: run without it
+                        print(f"[compile] no {m['S']}-token graphs on this NPU / driver: running without them", flush=True)
+                        self.segs.pop(m["S"])
+                        dropped.add(m["S"])
+                        continue
+                    for req, pm_ in reqs:
+                        self.segs[m["S"]].append((req, pm_, {i.get_any_name() for i in req.model_inputs}))
+            runs = set(NPUW_SEEN.values())
+            if attempt or not (npuw and NPUW_SEEN and (len(runs) > 1 or None in runs)):
+                break
+            # graphs of different runs hold one copy of the weights each: compile them all again, in this run
+            print(f"[npuw] parts compiled by different runs (a copy of the weights each): compiling the "
+                  f"{len(NPUW_SEEN)} parts again", flush=True)
+            import shutil
+            self.segs = {S: [] for S in self.segs}
+            self._trim()
+            for p_ in NPUW_SEEN:
+                shutil.rmtree(p_, ignore_errors=True)
+            NPUW_SEEN.clear()
+        # the MTP layer (draft tokens, Qwen3.5 dense): compiled without NPUW, before the paged KV cache is set up
+        # (its attention layer gets a cache of its own); mtp: wanted (chat.use_mtp)
+        self.mtp, self.last_h = None, None
+        import weakref
+        self._dts = weakref.WeakKeyDictionary()         # request -> {input name: numpy dtype} (_run_segment)
+        self._dead_note = False                         # (_head_again has said so once)
+        if mtp and self.meta.get("mtp") and any(S >= 4 for S in self.segs):     # (verification blocks: chat._mtp)
+            # (the NPU 4000 driver runs out of command queues - doorbells - past ~40 parts in one process, and then
+            # no process gets the NPU until a reboot: MTP's 3 parts must fit next to the model's)
+            from . import parts as _parts
+            if npuw and _parts.estimate(self.meta, sorted(self.segs), self.paged) + 3 > _parts.MANY:
+                print("[mtp] off: too many compiled parts with these block sizes", flush=True)
+            else:
+                self.mtp = self._build_mtp(self.meta["mtp"])
         tried.save()
         print(f"[progress] segments {todo}/{todo}", flush=True)
         print(f"[segments] compiled {sum(len(v) for v in self.segs.values())} in {time.time()-t0:.1f}s", flush=True)
@@ -572,6 +646,18 @@ class SegmentedModel:
         for S, lst in self.segs.items():
             self._bind(S, lst)
         self.exact_truncate = not any(k.startswith("ssm.") for k in self.state)
+        # DeltaNet states in two buffers each: a segment reads the current one and writes its new state straight into
+        # the other (set_tensor, no copy), then the two swap roles; undoing a verification block swaps back instead of
+        # copying the states back (Ornith 9B: 25 MB per block). {name: [[buffer, buffer], index of the current one]},
+        # the first buffer being the state tensor above. ONW_PINGPONG=0: copies as before
+        self._pp = {}
+        if os.environ.get("ONW_PINGPONG", "1") != "0":
+            hm = host_tensor(core, device)
+            for n, t in self._owner.items():
+                if n.startswith("ssm."):
+                    t2 = hm(t.get_element_type(), list(t.get_shape()))
+                    t2.data[:] = 0
+                    self._pp[n] = [[t, t2], 0]
         self.S = max(self.segs)
         import threading
         self._mutex = threading.RLock()                # a block size is swapped in only between two blocks
@@ -585,12 +671,128 @@ class SegmentedModel:
                     self._sink[l] = v.reshape(Lk.kv, Lk.G)
         self._last = None
         self.reset()
+        self._rows_t = None             # {DeltaNet state: [row given transposed?]} - exact truncation (_check_rows)
+        if not self.exact_truncate and any(1 < S < 16 for S in self.segs):
+            self._check_rows(min(S for S in self.segs if 1 < S < 16))
+
+    def _check_rows(self, S):
+        """The 4-token graphs give every row's DeltaNet state, so a verification block can be cut back exactly. The
+        NPU 4000 compiler hands out the states of the rows before the last with their last two dimensions transposed
+        (a layout it keeps between the steps; on the CPU they come as built, and NPUW's default pipeline differs again),
+        so each layer's layout per row is found once per load: against the state after a block of just that many rows
+        (the final state is always as built). Unless every row of every layer matches one way or the other exactly,
+        truncation keeps undoing the whole block (as without these graphs)."""
+        ids = list(range(1000, 1000 + 3 * S))
+        self.reset()
+        self._step(ids[:S], want_logits=False, size=S)
+        self._step(ids[S:2 * S], want_logits=False, size=S)
+        snap = self.snapshot()
+        self._step(ids[2 * S:3 * S], want_logits=False, size=S)
+        rows = {k: np.array(v) for k, v in self._last["rows"].items()}
+        names = [k for k in self.state if k.startswith("ssm.")]
+        if not rows:                                   # (graphs from before onw 1.4.0: no row states)
+            self.reset()
+            return
+        lay = {}
+        if rows and all(k in rows for k in names):
+            want = {k: np.array(self.state[k])[0] for k in names}
+            for n in range(S, 0, -1):
+                if n < S:
+                    self.restore(snap)
+                    self._step(ids[2 * S:2 * S + n], want_logits=False, size=S)
+                    want = {k: np.array(self.state[k])[0] for k in names}
+                for k in names:
+                    r = rows[k][n - 1]
+                    if np.array_equal(r, want[k]):
+                        lay.setdefault(k, {})[n - 1] = False
+                    elif np.array_equal(r.transpose(0, 2, 1), want[k]):
+                        lay.setdefault(k, {})[n - 1] = True
+                    else:
+                        lay = None
+                        break
+                if lay is None:
+                    break
+        self.reset()
+        if lay:
+            self._rows_t = {k: [v[i] for i in range(S)] for k, v in lay.items()}
+            t = sum(sum(v) for v in self._rows_t.values())
+            print(f"[mtp] exact truncation of {S}-token blocks ({t} of {S * len(names)} row states transposed)", flush=True)
+        else:
+            print(f"[mtp] the {S}-token graphs' row states do not match: verification blocks are undone on a miss",
+                  flush=True)
 
-    def _build(self, name, m):
-        """Read, split and compile one segment: [(request, part meta)], or None (this block size cannot run here)."""
+    def _build_mtp(self, mt):
+        """The MTP layer's parts and its head: {"layer": [(request, meta, input names)], "head": ..., "norm": [2, H]
+        (final norm, mtp.norm), "eps"}, or None when it does not compile here."""
+        out = {"layer": [], "head": None}
+        for name, m in sorted(mt["segments"].items()):
+            try:
+                reqs = self._build(name, m, npuw=False)
+            except RuntimeError as e:                  # (a transient failure: tried again next load)
+                print(f"[mtp] {name}: {' '.join(str(e).split())[-200:]}", flush=True)
+                reqs = None
+            if reqs is None:
+                print("[mtp] the MTP layer does not compile on this NPU / driver: no draft tokens", flush=True)
+                return None
+            lst = [(r, pm, {i.get_any_name() for i in r.model_inputs}) for r, pm in reqs]
+            for r, _, names in lst:
+                for n in names:
+                    if n.startswith("shared.") and n[7:] in self.shared:
+                        r.set_tensor(n, self.shared[n[7:]])
+            if m.get("head"):
+                out["head"] = lst[0]
+            else:
+                out["layer"] = lst
+        out["norm"] = np.load(os.path.join(self._ctx["d"], mt["norms"])).astype(np.float32)
+        out["vocab"] = np.load(os.path.join(self._ctx["d"], mt["vocab"])).astype(np.int64)
+        out["eps"] = float(mt["eps"])
+        print(f"[mtp] draft layer loaded ({len(out['layer'])} parts)", flush=True)
+        return out
+
+    def mtp_norm(self, h, which):
+        """(1 + w) RMSNorm on the host: which 0 = the final norm (the main model's hidden states), 1 = mtp.norm."""
+        x = np.asarray(h, np.float32)
+        return x / np.sqrt((x * x).mean(-1, keepdims=True) + self.mtp["eps"]) * (1.0 + self.mtp["norm"][which])
+
+    def mtp_layer(self, hn, toks, start):
+        """The MTP layer on n <= 16 rows at positions start.. (row j: the normalized hidden state of position
+        start + j and the token at start + j + 1): writes their rows of its KV cache, returns its outputs [n, H]
+        (before mtp.norm)."""
+        with self._mutex:
+            cur = self.cur
+            self.cur = start
+            try:
+                S, n, x, common = self._block_inputs(toks, None, None, S=16)
+            finally:
+                self.cur = cur
+            h = np.zeros_like(x)
+            h[0, :n] = hn
+            carry = {"carry.mtp_e": x, "carry.mtp_h": h}
+            for req, m, names in self.mtp["layer"]:
+                new, _, _ = self._run_segment(req, m, names, carry, common, None, S, n, start)
+                carry.update(new)
+            return np.asarray(carry["carry.mtp_y"][0, :n], np.float32)
+
+    def mtp_draft(self, hn):
+        """The draft token of one MTP output row after mtp.norm: the best of the draft head's tokens."""
+        req = self.mtp["head"][0]
+        with self._mutex:
+            lg = np.array(req.infer({"carry.mtp_hn": np.ascontiguousarray(np.asarray(hn, np.float16).reshape(1, 1, -1))},
+                                    share_inputs=True)["logits"]).reshape(-1)
+        if self.banned:
+            lg[np.isin(self.mtp["vocab"], self.banned)] = -np.inf
+        return int(self.mtp["vocab"][int(lg.argmax())])
+
+    def _build(self, name, m, npuw=None):
+        """Read, split and compile one segment: [(request, part meta)], or None (this block size cannot run here).
+        npuw=False: without NPUW whatever the model's setting (the MTP layer)."""
         c = self._ctx
-        core, d, device, config, npuw, tried, built_T = (c[k] for k in ("core", "d", "device", "config", "npuw", "tried",
-                                                                        "built_T"))
+        core, d, device, config, npuw_, tried, built_T = (c[k] for k in ("core", "d", "device", "config", "npuw", "tried",
+                                                                         "built_T"))
+        npuw = npuw_ if npuw is None else npuw
+        if npuw_ and not npuw:                         # (the model's config carries the NPUW options)
+            config = {k: v for k, v in config.items() if not k.startswith(
+                ("NPUW", "NPU_USE_NPUW", "NPU_COMPILATION_MODE_PARAMS", "NPU_COMPILER_", "CACHE_MODE"))}
         model = read_segment(core, d, name, m, npuw, c["bins"], built_T if self.paged else self.T, built_T)
         if self.streaming and self.predict_k and m["S"] == 1 and m.get("router_layer") is not None:
             rw_ = router_weights(model)
@@ -819,6 +1021,7 @@ class SegmentedModel:
                                                                                           default=str)])}
                 os.makedirs(cfg["CACHE_DIR"], exist_ok=True)
                 USED_NPUW.add(cfg["CACHE_DIR"])
+            npuw_dir = cfg.get("CACHE_DIR") if how.startswith("npuw") and not own else None
             if own and os.path.exists(bp):
                 try:
                     req = import_blob(core, bp, device, nocache).create_infer_request()
@@ -854,7 +1057,24 @@ class SegmentedModel:
                             pass                           # fall back to compiling here
                     if r is not True:
                         raise RuntimeError(r)
+                if npuw_dir:                           # a compile of this run, or an entry of which one?
+                    empty = not [f for f in os.listdir(npuw_dir) if f != "onw_session"]
+                    light = weight_bytes(model) < (1 << 20)
                 cm = core.compile_model(model, device, nocache if own else cfg)
+                if npuw_dir:
+                    mark = os.path.join(npuw_dir, "onw_session")
+                    if light:                          # (nothing for the weights bank to share)
+                        with open(mark, "w") as f:
+                            f.write(LIGHT)
+                    elif empty:
+                        with open(mark, "w") as f:
+                            f.write(NPUW_SESSION)
+                        NPUW_SEEN[npuw_dir] = NPUW_SESSION
+                    else:
+                        try:
+                            NPUW_SEEN[npuw_dir] = open(mark).read().strip()
+                        except OSError:
+                            NPUW_SEEN[npuw_dir] = None
                 if own:
                     os.makedirs(os.path.dirname(bp), exist_ok=True)
                     import io
@@ -943,7 +1163,7 @@ class SegmentedModel:
         """Row lookup rows -> f16 [n, dim] for a host table: INT8 per row, or INT4 with a scale per group."""
         spec = self.meta.get("tables", {}).get(t)
         if spec:
-            q, s, g = self.shared[t + "_q4"].data, self.shared[t + "_s4"].data, spec["group"]
+            q, s, g = np.asarray(self.shared[t + "_q4"].data), np.asarray(self.shared[t + "_s4"].data), spec["group"]
             D = q.shape[1] * 2
 
             def rows4(r):
@@ -956,7 +1176,7 @@ class SegmentedModel:
             return rows4
         if t + "_q" not in self.shared:
             return None
-        q, s = self.shared[t + "_q"].data, self.shared[t + "_s"].data
+        q, s = np.asarray(self.shared[t + "_q"].data), np.asarray(self.shared[t + "_s"].data)
         return lambda r: q[r].astype(np.float16) * s[r, None]
 
     # ---- state ----
@@ -987,8 +1207,23 @@ class SegmentedModel:
         False is returned: the caller must feed those `keep` tokens again."""
         last = self._last
         assert last is not None and last["prev"] is not None, "truncate() follows a multi-token step"
+        rows = last.get("rows") or {}
+        if not self.exact_truncate and self._rows_t and rows and all(k in rows for k in self._rows_t):
+            # a block from graphs that give every row's recurrent state (4-token verification): cut back exactly
+            for k, t in self._rows_t.items():
+                r = rows[k][keep - 1]
+                self.state[k][:] = (r.transpose(0, 2, 1) if t[keep - 1] else r)[None]
+            for l, cx in last["cx"].items():
+                st = self.state["conv." + l]
+                st[:] = np.concatenate([last["prev"]["conv." + l], cx[:, :keep]], 1)[:, -st.shape[1]:]
+            self.cur = last["start"] + keep
+            return True
         if not self.exact_truncate:
             self.restore((last["prev"], last["start"], self.delta))
+            for k, c in (last.get("pp") or {}).items():      # the states before the block: still in the old buffers
+                e = self._pp[k]
+                e[1] = c
+                self.state[k] = e[0][c].data
             return False
         for l, cx in last["cx"].items():
             st = self.state["conv." + l]
@@ -1053,10 +1288,11 @@ class SegmentedModel:
         return rw
 
     # ---- run ----
-    def _block_inputs(self, ids, embeds, pos):
-        """Host-built inputs of one block starting at self.cur: (S, n, embeddings, common inputs)."""
+    def _block_inputs(self, ids, embeds, pos, S=None):
+        """Host-built inputs of one block starting at self.cur: (S, n, embeddings, common inputs). S: this block size
+        (default: the smallest loaded one that holds the block)."""
         n = len(ids)
-        S = min((s for s in self.segs if s >= n), default=None)
+        S = S or min((s for s in self.segs if s >= n), default=None)
         assert S is not None and self.cur + n <= self.T, "block too large or context full"
         x = np.zeros((1, S, self.cfg["hidden_size"]), np.float16)
         x[0, :n] = embeds if embeds is not None else self.embed_rows(np.asarray(ids))
@@ -1102,7 +1338,21 @@ class SegmentedModel:
             feed["route.slot_w"] = self._bind_experts(req, m, route[0], route[1], S, n)
         if S == 1 and self.routers and self.predict_k:
             self._predict(m, carry)
-        res = {o.get_any_name(): v for o, v in req.infer(feed, share_outputs=True).items()}   # no copies: consumed right away
+        # no copies either way: the inputs are read from our arrays (in the dtype each input wants - a converted copy
+        # only when it differs), the outputs are the request's own buffers, consumed right away. Every call of these
+        # requests shares its inputs (a copying call would write into the arrays a sharing call left bound)
+        ent = self._dts.get(req)
+        if ent is None:
+            outs = {o.get_any_name() for o in req.model_outputs}
+            dts = {i.get_any_name(): i.get_element_type().to_dtype() for i in req.model_inputs}
+            ent = self._dts[req] = (dts, [k for k in dts if k in self._pp and "ssm_new." + k[4:] in outs])
+        dts, pp = ent
+        feed = {k: np.ascontiguousarray(v, dts[k]) for k, v in feed.items()}
+        for k in pp:                                  # read the current state buffer, write the other one
+            bufs, c = self._pp[k]
+            req.set_tensor(k, bufs[c])
+            req.set_tensor("ssm_new." + k[4:], bufs[1 - c])
+        res = {o.get_any_name(): v for o, v in req.infer(feed, share_inputs=True, share_outputs=True).items()}
         cp = (lambda v: v.copy()) if keep else (lambda v: v)
         nxt = {"carry." + k[5:]: cp(v) for k, v in res.items() if k.startswith("next.")}
         if "route.idx" in res:
@@ -1112,10 +1362,8 @@ class SegmentedModel:
         if attn is not None:                          # paged: store this layer's new rows, run its attention
             if f"new_k.{attn}" in res:                # (a layer sharing another's cache only reads it)
                 self.kv.write(attn, cur, res[f"new_k.{attn}"][0, :, :n], res[f"new_v.{attn}"][0, :, :n])
-            o = self.kv.attend(attn, res[f"q.{attn}"][0], cur, S, n, sink=self._sink.get(attn))  # [kv, G*S, Dv]
-            kvh, R, Dv = o.shape
-            o = o.reshape(kvh, R // S, S, Dv).transpose(2, 0, 1, 3).reshape(1, S, kvh * (R // S) * Dv)
-            nxt[f"carry.attn.{attn}"] = o.astype(np.float16)
+            nxt[f"carry.attn.{attn}"] = self.kv.attend(attn, res[f"q.{attn}"][0], cur, S, n, sink=self._sink.get(attn),
+                                                       rows_out=True)                    # [1, S, heads * Dv] f16
         for k, v in res.items():
             if k.startswith(("new_k.", "new_v.")):
                 if attn is not None and k[6:] == attn:        # the paged layer's rows are in the paged cache
@@ -1130,10 +1378,22 @@ class SegmentedModel:
                     self._last["cx"][k[3:]] = v[:, :n].copy()
                 st = self.state["conv." + k[3:]]
                 st[:] = np.concatenate([st, v[:, :n]], 1)[:, -st.shape[1]:]
+            elif k.startswith("ssm_rows."):           # (the request's own buffer: read by truncate() before the
+                if n > 1 and self._last is not None and self._last["start"] == cur:   # next block runs it)
+                    self._last["rows"]["ssm." + k[9:]] = v
             elif k.startswith("ssm_new."):
-                self.state["ssm." + k[8:]][:] = v
+                name = "ssm." + k[8:]
+                if name in pp:                        # written into the other buffer: it is the state now
+                    e = self._pp[name]
+                    e[1] = 1 - e[1]
+                    self.state[name] = e[0][e[1]].data
+                else:
+                    self.state[name][:] = v
             elif k == "logits":
                 logits = v[0, :n].copy()
+                dead = ~logits.any(axis=1)
+                if dead.any() and "carry.h" in feed:
+                    logits[dead] = self._head_again(req, feed, n)[dead]
                 if not np.isfinite(logits).all():     # else argmax picks id 0 (<pad>) forever, silently
                     raise FloatingPointError(f"the {S}-token graph returned NaN/inf logits (a numeric problem "
                                              "on this NPU / driver; try ONW_SIZES=16 or the CPU device)")
@@ -1141,6 +1401,21 @@ class SegmentedModel:
                     logits[:, self.banned] = -np.inf
         return nxt, route, logits
 
+    def _head_again(self, req, feed, n):
+        """Logits [n, V] for a head segment's rows that came back all zero: on the NPU (seen on 4000 and 3720) the final
+        RMSNorm's sum of squares overflows fp16 on rows with large activations in multi-token blocks (a whole row of
+        zeros - sampled from, a uniform draw over the vocabulary). The norm does not care about scale, so the head runs
+        again on the hidden states divided by their max |h|."""
+        h = np.asarray(feed["carry.h"], np.float32)
+        h = h / np.maximum(np.abs(h).max(-1, keepdims=True), 1e-2)
+        f2 = dict(feed)
+        f2["carry.h"] = np.ascontiguousarray(h, self._dts[req][0]["carry.h"])
+        out = req.infer(f2, share_inputs=True, share_outputs=True)
+        if not self._dead_note:
+            print("[head] logit rows of all zeros (fp16 overflow in the final norm) computed again, scaled", flush=True)
+            self._dead_note = True
+        return np.array(out["logits"][0, :n])
+
     def _predict(self, m, carry):
         """Hand the prediction for this segment's router to a helper thread (the NPU starts at once)."""
         L = m.get("router_layer")
@@ -1172,18 +1447,23 @@ class SegmentedModel:
         with self._mutex:
             return self._prefill_layers(ids, embeds, pos, want_logits)
 
-    def _step(self, ids, embeds=None, pos=None, want_logits=True):
+    def _step(self, ids, embeds=None, pos=None, want_logits=True, size=None):
         """ids: 1-D token ids (n <= largest S) -> logits [n, V] (f16); appends n tokens to the state.
         embeds [n, H]: use these rows instead of the token embeddings (image features); pos [3, n]: explicit
         MRoPE positions (default: slot + delta on all rows). want_logits=False skips the LM head segment (prompt
-        blocks whose logits are not needed) and returns None."""
-        S, n, x, common = self._block_inputs(ids, embeds, pos)
+        blocks whose logits are not needed) and returns None. size: this block size (default: the smallest that
+        holds the block)."""
+        S, n, x, common = self._block_inputs(ids, embeds, pos, S=size)
         self._tok += 1
         # what truncate() needs to undo part of a multi-token (verification) block
-        self._last = {"start": self.cur, "cx": {},
-                      "prev": {k: v.copy() for k, v in self.state.items() if not k.startswith("past_")} if n > 1 else None}
+        self._last = {"start": self.cur, "cx": {}, "rows": {},
+                      "prev": {k: v.copy() for k, v in self.state.items()
+                               if not k.startswith("past_") and k not in self._pp} if n > 1 else None,
+                      "pp": {k: e[1] for k, e in self._pp.items()} if n > 1 else None}
         carry, route, logits = {"inputs_embeds": x}, None, None
         for i, (req, m, names) in enumerate(self.segs[S]):
+            if m.get("head") and self.mtp is not None and "carry.h" in carry:
+                self.last_h = np.asarray(carry["carry.h"][0, :n], np.float32)   # (what the MTP layer drafts from)
             if m.get("head") and not want_logits:
                 continue
             new, route, lg = self._run_segment(req, m, names, carry, common, route, S, n, self.cur)

onw/server.py

@@ -606,7 +606,8 @@ def main(argv=None):
                 image_px=v.get("image_px") if v else None,
                 think=pipe.think_style is not None, max_context=pipe.max_context,
                 always_thinks=pipe.think_style == "harmony",   # (gpt-oss, llm-jp: the thinking counts against max_tokens)
-                note=f"文脈は約{pipe.max_context}トークン", context=pipe.max_context, pld=pipe.pld)
+                note=f"文脈は約{pipe.max_context}トークン", context=pipe.max_context, pld=pipe.pld,
+                mtp=getattr(pipe.model, "mtp", None) is not None)
     try:
         with open(os.path.join(a.engine, "config.json"), encoding="utf8") as f:
             arch = json.load(f).get("model_type") or ""
@@ -636,4 +637,6 @@ def main(argv=None):
 
 
 if __name__ == "__main__":
-    main()
+    # (python -m onw.server: run the module that onw.compat put its routes on - this one is a second copy of it)
+    from onw.server import main as _main
+    _main()

onw/web/check.html

@@ -117,7 +117,8 @@ fetch("/api/info").then(r => r.json()).then(i => {
   INFO = i;
   const rows = [[T("モデル名"), i.model], [T("デバイス"), Object.values(i.devices || {}).join(", ")],
     [T("コンテキスト長"), T("{0} トークン", i.context || i.max_context)], [T("画像入力"), i.vision ? T("対応") : T("なし")],
-    [T("先読み検証（PLD）"), i.pld ? T("有効") : T("無効")], [T("思考モード"), i.think ? T("対応") : T("なし")]];
+    [T("先読み検証（PLD）"), i.pld && !i.mtp ? T("有効") : T("無効")], [T("MTP 先読み"), i.mtp ? T("有効") : T("なし")],
+    [T("思考モード"), i.think ? T("対応") : T("なし")]];
   $("info").innerHTML = rows.map(([k, v]) => `<tr><td>${k}</td><td>${v}</td></tr>`).join("");
   $("base").textContent = location.origin + "/v1"; $("mid").textContent = i.model;
   if (i.rep_hint) { $("rephint").textContent = T("{0} 以上推奨（1 では同じ言葉を繰り返すことがあります）", i.rep_hint); $("rephint").style.display = ""; }

onw/web/manage.html

@@ -240,7 +240,7 @@
       </div>
       <div class="opt"><span class="k">エキスパートのメモリ</span><select id="f-expert_gb" style="width:220px"><option value="auto">auto（足りないときだけ SSD から）</option><option value="off">すべてメモリに置く</option><option value="2">2 GB</option><option value="3">3 GB</option><option value="4">4 GB</option><option value="5">5 GB</option><option value="6">6 GB</option><option value="8">8 GB</option></select>
         <span class="desc">Qwen3.6 系の MoE 用。上限を超えるエキスパートは SSD に置いたまま必要なときだけ読みます（動きますが遅くなります）。</span></div>
-      <div class="opt"><span class="k">64 トークン処理</span><select id="f-big_blocks" style="width:220px"><option value="auto">auto（RAM 24 GB 以上でオン）</option><option value="on">on</option><option value="off">off</option></select>
+      <div class="opt"><span class="k">64 トークン処理</span><select id="f-big_blocks" style="width:300px"><option value="auto">auto（Lunar Lake 以降か RAM 24 GB 以上でオン）</option><option value="on">on</option><option value="off">off</option></select>
         <span class="desc">長いプロンプトの処理が速くなる代わりに、そのぶんメモリを使います（Gemma）。</span></div>
     </div>