onw 3.0.0: onw NPU kit (Windows and Linux) - weightless graphs, shared weights, compiler fixes; fused graphs on by default from Lunar Lake on, up to 32K positions

.gitattributes

@@ -45,3 +45,7 @@ images/onw-models.png filter=lfs diff=lfs merge=lfs -text
 images/onw-settings-speed-en.png filter=lfs diff=lfs merge=lfs -text
 images/onw-settings-speed.png filter=lfs diff=lfs merge=lfs -text
 images/onw-server.png filter=lfs diff=lfs merge=lfs -text
+onw/npukit/openvino_intel_npu_compiler.dll filter=lfs diff=lfs merge=lfs -text
+onw/npukit/openvino_intel_npu_compiler_loader.dll filter=lfs diff=lfs merge=lfs -text
+onw/npukit/openvino_intel_npu_plugin.dll filter=lfs diff=lfs merge=lfs -text
+onw/npukit/openvino_intel_npu_vm_runtime.dll filter=lfs diff=lfs merge=lfs -text

CHANGELOG.md

@@ -5,6 +5,25 @@
 日付は日本時間です。各版は Hugging Face の `ryugyosoft/onw` にタグ（`v0.4.2` 以降）付きで残っています。
 onw ウィンドウは、新しい版があるとこのファイルから変更点を表示します（`## v…` の見出しで区切っています）。
 
+## v3.0.0 (2026-10-19)
+- **onw 独自の NPU キット（Windows・Linux）を同梱：NPU のコンパイラーとプラグインを、Intel の公開ソースから onw 用に作り直したもの**（`onw/npukit`、Apache 2.0、変更点は `onw/npukit/patches`）。onw が入れる OpenVINO 2026.5.0 のときだけ使い、インストール済みの OpenVINO には手を加えません（`ONW_NPUKIT=0` で OpenVINO 自身のものに戻せます）。これにより次のことができるようになりました。
+  - **重みを持たないコンパイル済みグラフ**：キャッシュにはグラフだけを置き、重みはロードのたびにモデルのファイルから NPU 上で組み立てます。キャッシュがモデル本体の 1/10 以下になります（Granite 4.2 8B NF4：10.5 → 0.8 GB、llm-jp-4.1-8B NF4：9.2 → 0.65 GB、Granite 4.2 3B NF4：4.2 → 0.5 GB）。OpenVINO 2026.5 のコンパイラーでは NF4 の重みでこれができませんでした。
+  - **同じ重みを NPU 上で 1 つだけ持つ**：グラフの種類（ブロックサイズ、文脈の長さごとのグラフ）がいくつあっても、重みは 1 組です。
+  - コンパイラーの修正：Qwen3.5 系 NF4 の 4 トークンの検証グラフ（MTP 先読み）、2 ビットの重み（Bonsai）を 2 行以上で計算すると 0 になる不具合、コンパイルの高速化（Panther Lake で Ternary Bonsai 2 9B の初回のロード約 7.5 分、生成約 17 tok/s など）。
+- **Lunar Lake 以降の PC（Windows・Ubuntu）で、融合グラフを既定で使うように**（2.1.0 では試験的・既定オフ）。区間ごとに NPU を 1 回呼ぶだけにして、生成もプロンプトも速くなります。重みを共有するので、メモリは増えず、むしろ減ります。Panther Lake（Core Ultra X7 358H）、2.1.0 の既定との比較：
+  - Granite 4.2 8B NF4：生成 約 9 → 約 13 tok/s、約 900 トークンのプロンプト 約 160 → 約 400 tok/s、専用メモリ 11.4 → 約 1.6 GB（ピーク 12.5 → 約 9.8 GB）、1 トークンあたりの電力量 約 1.3 → 約 0.9 J。
+  - llm-jp-4.1-8B NF4：生成 約 9.9 → 約 14 tok/s、プロンプト 約 170 → 約 440 tok/s、専用メモリ 11.0 → 約 2.4 GB。
+  - Lunar Lake（Ubuntu、16 GB）の Granite 4.2 8B NF4：生成 11〜13 tok/s、約 900 トークンのプロンプト約 385 tok/s、専用メモリ約 2 GB（ピーク 3.3 GB）、キャッシュ 1.8 GB、初回のロード 3.5 分（残りの融合グラフは裏で約 14 分）。
+  - Granite 4.2 3B NF4：生成 約 14.3 → 約 23 tok/s、プロンプト 約 235 → 約 730 tok/s、専用メモリ 5.1 → 約 1.4 GB。
+  - **長い会話も速く**：融合グラフは文脈の長さまで 8K・16K・32K 位置の種類も作ります（2.1.0 は 4K 位置まで、その先は従来の経路）。Granite 4.2 3B NF4、約 7.3K／15K／22K トークンのプロンプト（従来の経路との比較）：生成 4.6 → 13.6／4.0 → 8.8／2.6 → 4.9 tok/s、プロンプト 94 → 170／62 → 146／43 → 74 tok/s。
+  - 文脈の長さごとのグラフ（1K〜32K 位置）をすべて読み込んだままにするので、会話の途中で止まらず、同じ質問には同じ答えが返ります。初回のロードは 2.1.0 とほぼ同じで（8B で約 10〜12 分）、残りのグラフは回答しながら裏でコンパイルします。
+  - 省メモリモードでは、今までどおり使う長さのグラフだけを読み込みます。
+- Qwen3.5 系・Ornith・gemma-4（NPUW で動くモデル）もキャッシュが小さくなります（Qwen3.5-9B NF4：約 0.3 GB）。Qwen3.5-9B NF4 の生成は MTP 先読み付きで約 14〜19 tok/s。
+- **Ubuntu でもキットを使います**（`onw/npukit/linux`、32 MB の圧縮ファイルを最初の起動で展開）。Lunar Lake（16 GB）の Qwen3.5-9B NF4：初回のロード約 2.2 分、生成 15.5〜20.6 tok/s、プロンプト約 240 tok/s、キャッシュ 0.26 GB、専用メモリ約 1.1 GB。Granite 4.2 3B INT4 は初回のロード 4.7 → 2.7 分（答え・速さは 2.1.0 と同じ）。
+- INT4 版のモデル（Granite・llm-jp など区間に分けて動くもの）は、重みを持たないグラフにできない（重みが分離されない）ので、Lunar Lake 以降でも重みを持つグラフ・融合グラフなしの 2.1.0 と同じ動作です。Lunar Lake 以降では NF4 版を勧めます。
+- Meteor Lake・Arrow Lake（NPU 3720）は、2.1.0 と同じ動作です（重みを持たないグラフに対応していないため）。
+- OpenVINO は 2026.5.0b1 に固定しました（キットがこの版専用のため）��
+
 ## v2.1.0 (2026-10-18)
 - **Lunar Lake 以降専用の NF4 版を 3 つ追加：Granite 4.2 3B／8B、llm-jp-4.1-8B-thinking**（ryugyosoft/granite-4.2-3b-nf4-onw、granite-4.2-8b-nf4-onw、llm-jp-4.1-8b-thinking-nf4-onw）。重みはチャネル単位の NF4 を基本に、誤差の出やすいアテンションの入力（q/k/v_proj）と down_proj だけをチャネル単位の INT8 にしました。
   - Panther Lake（Core Ultra X7 358H）：Granite 4.2 8B は生成 5.4 → 約 9 tok/s（INT4 版・2.0.3 との比較）、約 900 トークンのプロンプト約 160 tok/s。llm-jp-4.1-8B は生成約 9.9 tok/s・プロンプト約 170 tok/s、Granite 4.2 3B は生成約 14.3 tok/s・プロンプト約 235 tok/s。1 トークンあたりの電力量（パッケージ全体）は 8B で約 1.3 J、3B で約 0.7 J。

CHANGELOG_en.md

@@ -4,6 +4,25 @@
 
 Dates are JST. Every version is on Hugging Face `ryugyosoft/onw`, tagged from `v0.4.2` on.
 
+## v3.0.0 (2026-10-19)
+- **onw's own NPU kit (Windows and Linux): the NPU compiler and the NPU plugin, rebuilt for onw from Intel's open sources** (`onw/npukit`, Apache 2.0, changes in `onw/npukit/patches`). Used only with the OpenVINO 2026.5.0 that onw installs; nothing of the installed OpenVINO is modified (`ONW_NPUKIT=0` goes back to OpenVINO's own). What it brings:
+  - **Compiled graphs without weights**: the cache holds the graphs only; the weights are rebuilt on the NPU from the model's own files at every load. The cache shrinks to a tenth of the model or less (Granite 4.2 8B NF4: 10.5 -> 0.8 GB, llm-jp-4.1-8B NF4: 9.2 -> 0.65 GB, Granite 4.2 3B NF4: 4.2 -> 0.5 GB). OpenVINO 2026.5's compiler cannot do this with NF4 weights.
+  - **One copy of identical weights on the NPU**: however many graphs (block sizes, context-length variants), the weights are held once.
+  - Compiler fixes: the 4-token verification graphs of the Qwen3.5 NF4 models (MTP drafts), 2-bit weights (Bonsai) computing zeros with 2+ rows, faster compiles (Panther Lake: Ternary Bonsai 2 9B loads the first time in ~7.5 min and answers at ~17 tok/s, ...).
+- **Fused graphs on by default on PCs from Lunar Lake on (Windows and Ubuntu)** (experimental and off in 2.1.0): one NPU call per segment, faster answers and prompts. The weights are shared, so memory does not grow - it shrinks. Panther Lake (Core Ultra X7 358H), against 2.1.0's default:
+  - Granite 4.2 8B NF4: answers ~9 -> ~13 tok/s, a ~900-token prompt ~160 -> ~400 tok/s, private memory 11.4 -> ~1.6 GB (peak 12.5 -> ~9.8 GB), energy per token ~1.3 -> ~0.9 J.
+  - llm-jp-4.1-8B NF4: answers ~9.9 -> ~14 tok/s, prompt ~170 -> ~440 tok/s, private memory 11.0 -> ~2.4 GB.
+  - Lunar Lake (Ubuntu, 16 GB), Granite 4.2 8B NF4: answers 11-13 tok/s, a ~900-token prompt ~385 tok/s, private memory ~2 GB (peak 3.3 GB), cache 1.8 GB, first load 3.5 min (the remaining fused graphs ~14 min in the background).
+  - Granite 4.2 3B NF4: answers ~14.3 -> ~23 tok/s, prompt ~235 -> ~730 tok/s, private memory 5.1 -> ~1.4 GB.
+  - **Long chats are fast too**: the fused graphs go on up to the context, with 8K / 16K / 32K-position variants (2.1.0: up to 4K positions, the paged path beyond). Granite 4.2 3B NF4, prompts of ~7.3K / 15K / 22K tokens (against the paged path): answers 4.6 -> 13.6 / 4.0 -> 8.8 / 2.6 -> 4.9 tok/s, prompts 94 -> 170 / 62 -> 146 / 43 -> 74 tok/s.
+  - Every context-length variant (1K to 32K positions) stays loaded: no pauses mid-chat, and the same question gets the same answer. The first load takes about as long as with 2.1.0 (8B: ~10-12 min); the remaining variants compile in the background while you chat.
+  - The memory-saving mode keeps loading only the variant in use, as before.
+- The Qwen3.5 family, Ornith and gemma-4 (models that run through NPUW) get small caches too (Qwen3.5-9B NF4: ~0.3 GB). Qwen3.5-9B NF4 answers at ~14-19 tok/s with MTP drafts.
+- **The kit on Ubuntu too** (`onw/npukit/linux`, a 32 MB archive unpacked at the first start). Lunar Lake (16 GB), Qwen3.5-9B NF4: first load ~2.2 min, answers 15.5-20.6 tok/s, prompts ~240 tok/s, cache 0.26 GB, private memory ~1.1 GB. Granite 4.2 3B INT4 loads the first time in 2.7 instead of 4.7 min (same answers and speed as 2.1.0).
+- The INT4 builds (Granite, llm-jp and the other models run in paged parts) cannot have graphs without weights (their weights are not separated), so from Lunar Lake on too they run as in 2.1.0: graphs with their weights, no fused graphs. From Lunar Lake on, use the NF4 builds.
+- Meteor Lake / Arrow Lake (NPU 3720) behave as in 2.1.0 (no graphs without weights there).
+- OpenVINO is pinned to 2026.5.0b1 (the kit is built for exactly this release).
+
 ## v2.1.0 (2026-10-18)
 - **Three NF4 builds for Lunar Lake and later: Granite 4.2 3B / 8B and llm-jp-4.1-8B-thinking** (ryugyosoft/granite-4.2-3b-nf4-onw, granite-4.2-8b-nf4-onw, llm-jp-4.1-8b-thinking-nf4-onw). The weights are channel-wise NF4, except the attention inputs (q/k/v_proj) and down_proj, which are channel-wise INT8 (where 4 bits cost the most accuracy).
   - Panther Lake (Core Ultra X7 358H): Granite 4.2 8B decodes 5.4 -> ~9 tok/s (against the INT4 build on 2.0.3) and reads a ~900-token prompt at ~160 tok/s; llm-jp-4.1-8B ~9.9 tok/s / ~170 tok/s; Granite 4.2 3B ~14.3 tok/s / ~235 tok/s. Energy per token (whole package): ~1.3 J for the 8B, ~0.7 J for the 3B.

README.md

@@ -102,10 +102,10 @@ Python や git の知識は要りません。Python が入っていなければ
 | [ryugyosoft/NeoHorse-1-4B-onw](https://huggingface.co/ryugyosoft/NeoHorse-1-4B-onw) | `qwen3_5`（Qwen3.5-4B をエージェント・ツール呼び出し向けに追加学習した NeoHorse-1） | テキスト | 切替 | 6.4 tok/s | 2.4 GB | 約 6 GB |
 | [ryugyosoft/granite-4.2-3b-onw](https://huggingface.co/ryugyosoft/granite-4.2-3b-onw) | `granite`（IBM Granite 4.2：思考とツール呼び出しを強化した 3B。Llama 型） | テキスト | 切替 | 6.7 tok/s（チャット約 7.7） | 2.1 GB | 約 6 GB |
 | [ryugyosoft/granite-4.2-8b-onw](https://huggingface.co/ryugyosoft/granite-4.2-8b-onw) | `granite`（同上の 8B） | テキスト | 切替 | 3.6 tok/s（チャット約 3.9） | 5.0 GB | 約 13 GB |
-| [ryugyosoft/granite-4.2-3b-nf4-onw](https://huggingface.co/ryugyosoft/granite-4.2-3b-nf4-onw) | 3B の NF4 版（重みはチャネル単位 NF4、q/k/v・down_proj は INT8。**Lunar Lake 以降専用**） | テキスト | 切替 | —（Panther Lake で約 14.3 tok/s） | 2.6 GB | 約 5.3 GB |
-| [ryugyosoft/granite-4.2-8b-nf4-onw](https://huggingface.co/ryugyosoft/granite-4.2-8b-nf4-onw) | 8B の NF4 版（同上。**Lunar Lake 以降専用**） | テキスト | 切替 | —（Panther Lake で約 9 tok/s） | 6.2 GB | 約 12.5 GB |
+| [ryugyosoft/granite-4.2-3b-nf4-onw](https://huggingface.co/ryugyosoft/granite-4.2-3b-nf4-onw) | 3B の NF4 版（重みはチャネル単位 NF4、q/k/v・down_proj は INT8。**Lunar Lake 以降専用**） | テキスト | 切替 | —（Panther Lake で約 23 tok/s） | 2.6 GB | 約 4.8 GB |
+| [ryugyosoft/granite-4.2-8b-nf4-onw](https://huggingface.co/ryugyosoft/granite-4.2-8b-nf4-onw) | 8B の NF4 版（同上。**Lunar Lake 以降専用**） | テキスト | 切替 | —（Panther Lake で約 13 tok/s） | 6.2 GB | 約 9.8 GB |
 | [ryugyosoft/llm-jp-4.1-8b-thinking-onw](https://huggingface.co/ryugyosoft/llm-jp-4.1-8b-thinking-onw) | `llama`（NII LLM-jp の日本語中心の思考型 8B、harmony 形式） | テキスト | **常に** | 4.2〜4.7 tok/s | 4.6 GB | 約 10 GB |
-| [ryugyosoft/llm-jp-4.1-8b-thinking-nf4-onw](https://huggingface.co/ryugyosoft/llm-jp-4.1-8b-thinking-nf4-onw) | 同上、NF4 版（q/k/v・down_proj は INT8。**Lunar Lake 以降専用**） | テキスト | **常に** | —（Panther Lake で約 9.9 tok/s） | 6.1 GB | 約 11.6 GB |
+| [ryugyosoft/llm-jp-4.1-8b-thinking-nf4-onw](https://huggingface.co/ryugyosoft/llm-jp-4.1-8b-thinking-nf4-onw) | 同上、NF4 版（q/k/v・down_proj は INT8。**Lunar Lake 以降専用**） | テキスト | **常に** | —（Panther Lake で約 14 tok/s） | 6.1 GB | 約 9.7 GB |
 | [ryugyosoft/llm-jp-4.1-8b-thinking-int8-onw](https://huggingface.co/ryugyosoft/llm-jp-4.1-8b-thinking-int8-onw) | 同上、重みはチャネル単位 INT8（bf16 に近い精度） | テキスト | **常に** | 4.1 tok/s | 7.7 GB | 約 15 GB（節約モードで待機中 約 8 GB） |
 | [ryugyosoft/llm-jp-4.1-32b-a3b-thinking-onw](https://huggingface.co/ryugyosoft/llm-jp-4.1-32b-a3b-thinking-onw) | `qwen3_moe`（llm-jp の MoE：128 個から 8 個、32B のうち 1 トークン約 3B、harmony 形式） | テキスト | **常に** | 5.8 tok/s | 16 GB | 約 17 GB（SSD から読めば 16 GB の PC でも。Lunar Lake 16 GB で 3.4 tok/s） |
 | [ryugyosoft/llm-jp-4.1-32b-a3b-thinking-int8-onw](https://huggingface.co/ryugyosoft/llm-jp-4.1-32b-a3b-thinking-int8-onw) | 同上、エキスパート・アテンション INT8（bf16 に近い精度） | テキスト | **常に** | 4.6 tok/s | 30 GB | 約 32 GB |
@@ -200,7 +200,8 @@ onw list                                                      # 対応アーキ
 | **コンテキスト長**（`--context`、既定 32K、モデルの最大まで） | サーバーのオプション／設定画面 | 不要（KV キャッシュの上限が変わるだけ） |
 | 64 トークン処理（dense モデルのプロンプトを 64 トークンずつ。auto はオンで、16 トークン用のグラフの代わりに読み込む。off で 16 トークンずつ、`ONW_S64=1` で両方） | `ONW_S64=1/0`／設定画面 | 初回のみ |
 | MTP 先読み（Qwen3.5 系 NF4 版。auto は Lunar Lake 以降でオン） | `ONW_MTP=1/0` | 初回のみ |
-| 融合グラフ（試験的、既定オフ。dense モデルの区間を 1 回の呼び出しで。回答用、Lunar Lake 以降はプロンプト用も。1K・2K・4K 位置の 3 種類を裏でコンパイルし、キャッシュがモデルの約 6〜7 倍になる） | `ONW_FUSED=1`（有効）、`ONW_FUSED_PROMPT=0`（回答だけ） | 初回のみ |
+| 融合グラフ（dense モデルの区間を 1 回の呼び出しで。回答用と、Lunar Lake 以降はプロンプト用も。onw の NPU キットが使える Lunar Lake 以降の PC では既定でオン） | `ONW_FUSED=0`（無効）／`1`（キットなしでも有効。キャッシュがモデルの約 6〜7 倍になる）、`ONW_FUSED_PROMPT=0`（回答だけ） | 初回のみ |
+| onw の NPU キット（Windows・Linux。重みを持たないグラフと重みの共有、コンパイラーの修正） | `ONW_NPUKIT=0`（OpenVINO 自身の NPU プラグインを使う） | 起動時 |
 | エキスパートのメモリ（Qwen3.6 系。上限を超える分は SSD から読む） | `ONW_EXPERT_GB=5`／設定画面 | 不要 |
 | NPUW 重み共有（試験的。auto は Lunar Lake 以降でオン、MoE モデル（LFM2・Qwen3.6）と区間の多いモデル（Granite・llm-jp）は対象外：重みを 1 コピーにしてメモリとコンパイル時間を削減） | `ONW_NPUW=1/0`／設定画面 | 初回のみ |
 | メモリ節約モード（1 トークン用と 16 トークン用で重みを共有できない dense モデル：NPU 3720 ではすべて、Lunar Lake 以降は Granite・llm-jp と NPUW オフのとき。普段は 16 トークン用のグラフだけを持ち、回答を書く間だけ 1 トークン用と入れ替える。待機中のメモリ約 4〜6 割減、回答の出だし数秒は遅め、先読み検証なし） | `ONW_LAZY16=1`／設定画面（効くモデルがあるときだけ表示） | 初回のみ |
@@ -318,25 +319,15 @@ KV キャッシュは INT8 で、会話が実際に使った長さの分だけ
   （Windows 11、ドライバー 32.0.100.5540、2026 年 10 月 6 日。1 パーツは 4 トークン用グラフの最初のパーツ。命令の大きさは
   コンパイル結果の ELF セクションの大きさで、命令の数そのものではありません。）
 
-### 既知の問題（onw 2.1.0、2026 年 10 月 18 日）
-
-- **融合グラフ（試験的、`ONW_FUSED=1`）はキャッシュが大きい**：区間ごとに NPU を 1 回呼ぶだけにする融合グラフは、Panther Lake で
-  生成が約 1.2〜1.6 倍になりますが、1K・2K・4K 位置用の 3 種類 × ブロックサイズごとのグラフがそれぞれ重みの写しを持つので、
-  キャッシュがモデルの約 6〜7 倍になります（Granite 4.2 8B NF4 で約 42 GB、既定の経路では約 10.5 GB）。初回は裏でのコンパイルも長く、
-  会話が 1K・2K・4K 位置を越えるたびに数秒止まり、回答の言い回しが既定の経路と少し変わることがあります。重みを共有できるようになるまで既定ではオフです。
-- **Panther Lake で、dense モデルの初回のロードが長い**。2.0.3 で Granite 4.2 3B／8B と llm-jp-4.1-8B（INT4）は大きく短くなりました
-  （Panther Lake の小型機 Core Ultra X7 358H で、Granite 4.2 8B は約 2 時間 45 分 → 約 23 分、llm-jp-4.1-8B は約 50 分 → 約 19 分、Granite 4.2 3B は約 10 分）。
-  Qwen3.5-9B は INT4 で約 15 分、NF4 で約 8 分のままです（2 回目以降はキャッシュから数十秒）。ロードを途中で止めたり PC が
-  再起動したりしても、コンパイルの済んだ部分はキャッシュに残ります。
-  - 原因：OpenVINO 2026.5 に同梱の NPU コンパイラー（8.4。ナイトリー版の 8.5 も同じ）は、1 トークン用（生成用）のグラフの
-    コンパイル結果を書き出す段階が遅くなっています。1 トークン用のグラフでは、INT4 の行列積が量子化のグループごとの小さな
-    行列積に分解され（2 トークン以上なら分解されません）、その小さな定数を書き出す処理が 8.3 より約 2.7 倍遅くなっています。
-    Granite 4.2 8B の部品 1 つ（NPU 5010 向け、Core Ultra 9 285HX で計測）は、1 トークン用で 97 秒、2 トークン用で 4 秒でした。
-  - 2.0.3 の対処：NPU 4000 以降（Lunar Lake／Panther Lake。Arrow Lake などの NPU 3720 は変わりません）では、上の 3 モデルの生成に 2 トークン用のグラフを使います。
-    モデルの更新（数百バイトのファイル追加）が必要で、アプリが更新を案内します。生成は約 1 割遅くなります（Granite 4.2 8B で
-    6.0 → 5.4 tok/s）。環境変数 `ONW_DECODE2=0` で元の 1 トークン用に戻せます。
-  - 試験機能：環境変数 `ONW_NPU_COMPILER=driver` で、OpenVINO 同梱のコンパイラーではなくドライバー内のコンパイラーを使います。
-    Qwen3.5-9B（INT4）は正しく動きましたが、NF4 版の 4 トークン用グラフのコンパイルに失敗するので既定にはしていません。
+### 既知の問題（onw 3.0.0、2026 年 10 月 19 日）
+
+- **INT4 版のモデルと Meteor Lake・Arrow Lake（NPU 3720）は 2.1.0 と同じです**：INT4 の重みは重みを持たないグラフにできず、NPU 3720 は
+  それに対応していないので、キャッシュとメモリは 2.1.0 と同じで、融合グラフも使いません。Lunar Lake 以降では NF4 版を勧めます。
+- **長い会話では生成が遅くなります**：文脈が長いほど毎回読む KV キャッシュが増えるためです（Panther Lake の Granite 4.2 3B NF4：
+  約 7.3K トークン�� 13.6 tok/s、約 15K で 8.8 tok/s、約 22K で 4.9 tok/s。従来の経路の約 2〜3 倍です）。
+- **初回のロード（Panther Lake の小型機 Core Ultra X7 358H、3.0）**：Granite 4.2 8B NF4 約 12 分、llm-jp-4.1-8B NF4 約 10 分、
+  Granite 4.2 3B NF4 約 6.5 分、Qwen3.5-9B NF4 約 7.5 分。残りの融合グラフは、回答しながら裏でコンパイルします（4K 位置までで 7〜10 分、32K 位置まで全部で Granite 4.2 3B NF4 は約 28 分）。
+  2 回目以降はキャッシュから数十秒です。ロードを途中で止めたり PC が再起動したりしても、コンパイルの済んだ部分は残ります。
 - **Panther Lake の小型機では、MTP 先読みの生成が Lunar Lake より遅い**：Qwen3.5-9B（NF4）＋ MTP の生成は、Panther Lake の
   小型機で 13.6〜14.3 tok/s（Lunar Lake は約 19 tok/s）でした。MTP は CPU 側の処理が多く、電力の制限で定格まで落ちる小型機の
   CPU が効いていると見て調査中です。

README_en.md

@@ -89,10 +89,10 @@ troubleshooting), is the **[user guide](MANUAL_en.md)**.
 | [ryugyosoft/NeoHorse-1-4B-onw](https://huggingface.co/ryugyosoft/NeoHorse-1-4B-onw) | `qwen3_5` (NeoHorse-1: Qwen3.5-4B trained further for agents and tool calls) | text | switchable | 6.4 tok/s | 2.4 GB | ~6 GB |
 | [ryugyosoft/granite-4.2-3b-onw](https://huggingface.co/ryugyosoft/granite-4.2-3b-onw) | `granite` (IBM Granite 4.2: a 3B with thinking and stronger tool calls; Llama-type) | text | switchable | 6.7 tok/s (~7.7 in chat) | 2.1 GB | ~6 GB |
 | [ryugyosoft/granite-4.2-8b-onw](https://huggingface.co/ryugyosoft/granite-4.2-8b-onw) | `granite` (the same, 8B) | text | switchable | 3.6 tok/s (~3.9 in chat) | 5.0 GB | ~13 GB |
-| [ryugyosoft/granite-4.2-3b-nf4-onw](https://huggingface.co/ryugyosoft/granite-4.2-3b-nf4-onw) | the 3B as NF4 (channel-wise NF4, q/k/v and down_proj INT8; **Lunar Lake and newer only**) | text | switchable | — (~14.3 tok/s on Panther Lake) | 2.6 GB | ~5.3 GB |
-| [ryugyosoft/granite-4.2-8b-nf4-onw](https://huggingface.co/ryugyosoft/granite-4.2-8b-nf4-onw) | the 8B as NF4 (the same; **Lunar Lake and newer only**) | text | switchable | — (~9 tok/s on Panther Lake) | 6.2 GB | ~12.5 GB |
+| [ryugyosoft/granite-4.2-3b-nf4-onw](https://huggingface.co/ryugyosoft/granite-4.2-3b-nf4-onw) | the 3B as NF4 (channel-wise NF4, q/k/v and down_proj INT8; **Lunar Lake and newer only**) | text | switchable | — (~23 tok/s on Panther Lake) | 2.6 GB | ~4.8 GB |
+| [ryugyosoft/granite-4.2-8b-nf4-onw](https://huggingface.co/ryugyosoft/granite-4.2-8b-nf4-onw) | the 8B as NF4 (the same; **Lunar Lake and newer only**) | text | switchable | — (~13 tok/s on Panther Lake) | 6.2 GB | ~9.8 GB |
 | [ryugyosoft/llm-jp-4.1-8b-thinking-onw](https://huggingface.co/ryugyosoft/llm-jp-4.1-8b-thinking-onw) | `llama` (NII LLM-jp's Japanese-centred 8B thinking model, harmony format) | text | **always** | 4.2-4.7 tok/s | 4.6 GB | ~10 GB |
-| [ryugyosoft/llm-jp-4.1-8b-thinking-nf4-onw](https://huggingface.co/ryugyosoft/llm-jp-4.1-8b-thinking-nf4-onw) | the same as NF4 (q/k/v and down_proj INT8; **Lunar Lake and newer only**) | text | **always** | — (~9.9 tok/s on Panther Lake) | 6.1 GB | ~11.6 GB |
+| [ryugyosoft/llm-jp-4.1-8b-thinking-nf4-onw](https://huggingface.co/ryugyosoft/llm-jp-4.1-8b-thinking-nf4-onw) | the same as NF4 (q/k/v and down_proj INT8; **Lunar Lake and newer only**) | text | **always** | — (~14 tok/s on Panther Lake) | 6.1 GB | ~9.7 GB |
 | [ryugyosoft/llm-jp-4.1-8b-thinking-int8-onw](https://huggingface.co/ryugyosoft/llm-jp-4.1-8b-thinking-int8-onw) | the same with channel-wise INT8 weights (close to bf16) | text | **always** | 4.1 tok/s | 7.7 GB | ~15 GB (~8 GB idle in memory-saving mode) |
 | [ryugyosoft/llm-jp-4.1-32b-a3b-thinking-onw](https://huggingface.co/ryugyosoft/llm-jp-4.1-32b-a3b-thinking-onw) | `qwen3_moe` (llm-jp's MoE: 8 of 128 experts, ~3B of 32B per token, harmony format) | text | **always** | 5.8 tok/s | 16 GB | ~17 GB (16 GB PCs with the experts streamed from the SSD: 3.4 tok/s on a 16 GB Lunar Lake) |
 | [ryugyosoft/llm-jp-4.1-32b-a3b-thinking-int8-onw](https://huggingface.co/ryugyosoft/llm-jp-4.1-32b-a3b-thinking-int8-onw) | the same, INT8 experts and attention (close to bf16) | text | **always** | 4.6 tok/s | 30 GB | ~32 GB |
@@ -189,7 +189,8 @@ What needs what:
 | **context length** (`--context`, 32K by default, up to the model's maximum) | server option / settings | no (it only bounds the KV cache) |
 | 64-token prompt blocks (dense models read prompts 64 tokens at a time; auto = on, loaded in place of the 16-token graphs; off = 16 at a time, `ONW_S64=1` = both) | `ONW_S64=1/0` / settings | once, the first time |
 | MTP draft tokens (the NF4 Qwen3.5-family builds; auto = on from Lunar Lake) | `ONW_MTP=1/0` | once, the first time |
-| fused graphs (experimental, off by default: a dense model's segment in one call; answers, and prompts from Lunar Lake on; 1K / 2K / 4K-position variants compiled in the background, the cache grows to ~6-7x the model) | `ONW_FUSED=1` (on), `ONW_FUSED_PROMPT=0` (answers only) | once, the first time |
+| fused graphs (a dense model's segment in one call; answers, and prompts from Lunar Lake on; on by default on PCs from Lunar Lake on with onw's NPU kit) | `ONW_FUSED=0` (off) / `1` (on without the kit too - the cache then grows to ~6-7x the model), `ONW_FUSED_PROMPT=0` (answers only) | once, the first time |
+| onw's NPU kit (Windows and Linux: graphs without weights, shared weights, compiler fixes) | `ONW_NPUKIT=0` (use OpenVINO's own NPU plugin) | at start |
 | expert memory (Qwen3.6 models; the rest streams from the SSD) | `ONW_EXPERT_GB=5` / settings | no |
 | NPUW weight sharing (experimental; auto = on from Lunar Lake, not for the MoE models LFM2 / Qwen3.6 or models with many parts (Granite, llm-jp): one copy of the weights, less memory, much faster compile) | `ONW_NPUW=1/0` / settings | once, the first time |
 | Memory-saving mode (dense models whose 1-token and 16-token graphs cannot share weights: all of them on NPU 3720; from Lunar Lake on Granite / llm-jp, and any with NPUW off. Only the 16-token graphs are kept, swapped for the 1-token ones while an answer is written; 40-60% less memory when idle, the first seconds of an answer slower, no prompt lookup) | `ONW_LAZY16=1` / settings (shown only when a model it helps is there) | once, the first time |
@@ -306,27 +307,17 @@ How it was measured, and caveats:
   (Windows 11, driver 32.0.100.5540, 2026-10-06. One part = the first part of a 4-token graph. Code sizes are ELF section sizes of
   the compiled file, not instruction counts.)
 
-### Known issues (onw 2.1.0, 2026-10-18)
-
-- **The fused graphs (experimental, `ONW_FUSED=1`) take a lot of cache**: one NPU call per segment makes decoding ~1.2-1.6x faster
-  on Panther Lake, but the graphs for 1K, 2K and 4K positions x each block size all hold their own copy of the weights, so the
-  cache grows to ~6-7x the model (Granite 4.2 8B NF4: ~42 GB, ~10.5 GB on the default path). The first background compile is
-  long, a chat passing 1K / 2K / 4K positions pauses for a few seconds, and the wording can differ a little from the default path.
-  Off by default until the weights can be shared.
-- **On Panther Lake the first load of a dense model takes long**. 2.0.3 cuts it a lot for Granite 4.2 3B / 8B and
-  llm-jp-4.1-8B (INT4): on a small Panther Lake PC (Core Ultra X7 358H) Granite 4.2 8B went from ~2 h 45 min to ~23 min, llm-jp-4.1-8B from ~50 to ~19 min, and
-  Granite 4.2 3B takes ~10 min. Qwen3.5-9B still takes ~15 min (INT4) / ~8 min (NF4); later loads come from the cache in well
-  under a minute. If a load is stopped or the PC restarts midway, the parts already compiled stay in the cache.
-  - Cause: the NPU compiler that comes with OpenVINO 2026.5 (8.4; the nightly 8.5 is the same) is slow at writing out the
-    compiled 1-token (decode) graphs. In a 1-token graph each INT4 matmul is split into one small matmul per quantization group
-    (not with 2 or more tokens), and writing out those small constants is ~2.7x slower than in 8.3. One Granite 4.2 8B part
-    (for the NPU 5010, timed on a Core Ultra 9 285HX) takes 97 s as a 1-token graph and 4 s as a 2-token graph.
-  - The 2.0.3 fix: on the NPU 4000 and later (Lunar Lake / Panther Lake; the NPU 3720 of Arrow Lake etc. is unchanged) these three models decode on 2-token
-    graphs. This needs a model update (a few hundred bytes of new files); the app offers it. Decoding gets ~10% slower
-    (Granite 4.2 8B 6.0 -> 5.4 tok/s). `ONW_DECODE2=0` goes back to the 1-token graphs.
-  - Experimental: `ONW_NPU_COMPILER=driver` uses the compiler inside the NPU driver instead of the one that comes with
-    OpenVINO. Qwen3.5-9B (INT4) runs correctly with it, but it fails to compile the 4-token graphs of the NF4 builds, so it is
-    not the default.
+### Known issues (onw 3.0.0, 2026-10-19)
+
+- **The INT4 builds and Meteor Lake / Arrow Lake (NPU 3720) are as in 2.1.0**: INT4 weights cannot go into graphs without weights,
+  and the NPU 3720 has no such graphs, so cache and memory are as in 2.1.0 and no fused graphs are used. From Lunar Lake on, use
+  the NF4 builds.
+- **Answers slow down in long chats**: the longer the context, the more KV cache every step reads (Panther Lake, Granite 4.2 3B
+  NF4: 13.6 tok/s at ~7.3K tokens, 8.8 at ~15K, 4.9 at ~22K - about 2-3x the paged path).
+- **First loads (a small Panther Lake PC, Core Ultra X7 358H, 3.0)**: Granite 4.2 8B NF4 ~12 min, llm-jp-4.1-8B NF4 ~10 min,
+  Granite 4.2 3B NF4 ~6.5 min, Qwen3.5-9B NF4 ~7.5 min. The remaining fused graphs compile in the background while you chat
+  (~7-10 min up to 4K positions; all of them up to 32K positions ~28 min for Granite 4.2 3B NF4). Later loads come from the cache in well under a minute; if a load is stopped or the PC restarts midway, the
+  parts already compiled stay.
 - **On small Panther Lake PCs MTP decoding is slower than on Lunar Lake**: Qwen3.5-9B (NF4) + MTP decodes at 13.6-14.3 tok/s
   there (Lunar Lake ~19 tok/s). MTP does a lot of CPU-side work, and the small PC's CPU drops to its base clock under its power
   limit; being looked into.

TECHNICAL.md

@@ -412,6 +412,52 @@ dense モデルのプロンプトは 16 トークンのブロックで処理し
   - 種類の切り替え：会話が 1K・2K 位置を超えるとき、4K を超えて従来の経路に戻るとき、長い会話の後に短い会話に戻るとき、グラフを読み直します（Panther Lake の Granite 4.2 8B で 1 サイズ 6〜10 秒、NPU 3720 の 3B で約 3 秒）。この時間はプロンプトの処理時間に入ります。
 - **NF4 版（形式 C）**：Lunar Lake 以降向けの Granite 4.2 3B／8B、llm-jp-4.1-8B は、重みをチャネル単位の NF4 にし、q/k/v_proj と down_proj だけチャネル単位の INT8 にしました。いくつかの組み合わせ（全部 NF4、q/k/v だけ INT8、グループ 256 の NF4）を教師強制の精度で比べ、精度と速さの釣り合いでこれを選びました（512 トークンで 1 位の一致率／KL：Granite 4.2 8B 88.5%／0.097、INT4 版 85.5%／0.132）。グラフは 2 トークン用と 64 トークン用だけです（NPU 4000 以降は 2 トークン用で回答します）。
 
+## onw の NPU キット：重みを持たないグラフと重みの共有（onw 3.0、Windows・Linux）
+
+onw 3.0 は、Intel が公開している NPU コンパイラー（npu_compiler）と OpenVINO の NPU プラグインのソースに手を入れて作り直したものを
+`onw/npukit` に同梱しています（Apache 2.0、変更点は `onw/npukit/patches`）。プラグインは onw が入れる OpenVINO 2026.5.0b1
+（`2026.5.0-23160-6db412d0a27`）と同じコミットから作ってあり、onw は `ov.Core(onw/npukit/plugins.xml)` で「NPU」をこのプラグインに
+向けます。インストール済みの OpenVINO のファイルは変えません。版が違うときは使わず、OpenVINO 自身の NPU プラグインに戻ります。
+Linux 用（Ubuntu 24.04 で作った .so）は `onw/npukit/linux/npukit-linux.tar.xz`（32 MB）にまとめてあり、最初の起動で同じフォルダーに
+展開します。Linux の OpenVINO にはプラグイン側のコンパイラーが入っていないので、キットのない Linux では重みを持たないグラフを作れません
+（ドライバー内蔵のコンパイラーを使うため）。NPU 3720（Meteor Lake・Arrow Lake）では使いません（重みを持たないグラフに対応しておらず、
+OpenVINO 自身のコンパイラーのほうが約 7% 速いグラフを作るため。`ONW_NPUKIT=1` で強制）。
+
+- **重みを持たないグラフ**：`CACHE_MODE=OPTIMIZE_SIZE` と `ENABLE_WEIGHTLESS` でコンパイルすると、コンパイラーはグラフを
+  「重みを NPU 用の並びに組み立てる初期化スケジュール」と「本体」に分け、重みを含めずに書き出します。ロード時に同じ ov.Model を
+  `MODEL_PTR` で渡すと、モデルのファイル（.bin の写像）から重みを読み、NPU 上で組み立てて本体の入力につなぎます。組み立て済みの
+  重みを入力として渡しても、重みを含めたグラフと同じ速さで、計算結果もビット単位で同じでした。
+  - OpenVINO 2026.5 のコンパイラー（8.4）は NF4 の重みでこれに失敗します（「ui4 → quantile の変換は未対応」）。キットの
+    コンパイラーでは、NF4（quantile 型と、それを格納型にした量子化型）を格納型 ui4 のまま読み替えるようにし、4 ビット値の
+    クラスタ分割・並べ替えがバイト境界をまたがないようにしました（初期化スケジュールの中だけ）。
+  - 2 ビットの重み（Bonsai）は重みが分離されず（グラフの大きさがほぼ変わらない）、コンパイルが約 7 倍遅くなるので、使いません。
+  - INT4 の重みも分離されません（Granite 4.2 3B INT4 の 1 トークンの部品で 63.5 対 63.6 MB）。2 行以上のグラフでは、NPU 4000 以降の
+    コンパイラーが INT4 を表引き（4 ビットの値をそのまま表の番号にする）に変える処理と重ならず失敗し、直してもコンパイルが 20〜35 倍
+    遅くなるだけなので、INT4 のモデル（NPUW を使わない区間分割のもの）では使いません。融合グラフも、種類ごとに重みを持つことに
+    なるので既定でオフです（2.1.0 と同じ動作。Lunar Lake の 16 GB ではメモリーが足りなくなりました）。INT8 は分離されます
+    （llm-jp-4.1-8B INT8 の部品で 2.5 対 216 MB）。Lunar Lake 以降では NF4 版を勧めます。
+- **重みの共有**：重みを持たないグラフでも、グラフごとに NPU 上へ重みを組み立てるので、融合グラフの長さ違い（1K〜32K 位置）や
+  ブロックサイズの数だけ重みが増えます（8B NF4 で 1 種類あたり約 4.8 GB）。キットのプラグインは、初期化スケジュールを走らせた
+  あと、出力した重みのバッファを大きさ・標本ハッシュ・全バイト比較で既存のものと照合し、すべて一致したら本体をそちらにつないで
+  新しいバッファを捨てます（`ONW_WS_SHARE=0` で無効）。区切り方の違うグラフどうし（区間を分けた従来の経路と融合グラフ）でも共有
+  できるよう、初期化スケジュールの出力を重みごとに分けます（コンパイラーの `ONW_WS_NO_CONCAT`。OpenVINO は既定で 1 本にまとめます）。
+  - Panther Lake の llm-jp-4.1-8B NF4：融合グラフ 6 種類を同時に読み込んで +5.7 GB（共有なしなら約 29 GB）。
+  - 共有したまま、ある種類のグラフを捨てて別の種類を読み込むと、NPU ドライバーの中で解放済みの領域への書き込みが起きることが
+    あるので、3.0 は融合グラフの全種類と従来の経路のグラフを読み込んだままにします（`ONW_FUSED_KEEP`。省メモリモードでは従来どおり
+    使う長さのグラフだけ）。全種類を持つので、融合グラフは文脈の長さまで 8K・16K・32K 位置の種類も作ります（2.1.0 は 4K まで）。
+    回答もプロンプトも、そのときの位置を覆う一番短い種類で計算し、位置が進むと次の種類に切り替えます（長い種類ほど毎回読む
+    KV キャッシュの範囲が広く、遅くなるため）。
+- **コンパイラーのその他の修正**：
+  - Qwen3.5 系 NF4 の 4 トークンのグラフ（MTP 先読みの検証用）が `'async.await' has non 'return' consumer` で失敗していたのを修正
+    （関数の結果の手前に形だけ変える操作が残る場合）。これがないと MTP が使えません。
+  - 2 ビットのゼロ点付き重み（Bonsai）を 2 行以上で計算すると 0 になる不具合（2.x で報告済み）の修正。
+  - コンパイル時間：並列コンパイルが Windows のヒープのロックで直列になっていた（分散情報のマップが要素 1 つでも約 45 KB を確保）のを
+    直し、定数の畳み込みと縦方向の融合の探索の結果を使い回すようにしました。出力はビット単位で同じです（Qwen3.5-9B の 64 トークンの
+    部品 52.6 → 34 秒など）。
+
+結果（Panther Lake、2.1.0 の既定との比較）は CHANGELOG の v3.0.0 を見てください。キャッシュはモデル本体の 1/10 以下、専用メモリは
+数分の一になり、生成・プロンプトとも速くなります。
+
 ## 1 ステップのホスト側の処理（onw 1.4）
 
 MTP の 1 ブロックのうち、NPU 以外（Python・numpy）が約 47 ms ありました。生成結果を 1 ビットも変えずに、次のことで約 32 ms に

TECHNICAL_en.md

@@ -435,6 +435,52 @@ The paged KV cache (onw 0.10) cuts each segment at its attention layers. A dense
   - Switching variants: when a chat passes 1K / 2K positions, passes 4K (back to the previous path), or a short chat follows a long one, graphs are read again (Granite 4.2 8B on Panther Lake: 6-10 s per block size; Granite 4.2 3B on the NPU 3720 ~3 s). That time shows up in the prompt time.
 - **The NF4 builds (format C)**: Granite 4.2 3B / 8B and llm-jp-4.1-8B for Lunar Lake and later have channel-wise NF4 weights with only q/k/v_proj and down_proj as channel-wise INT8. Several mixes (all NF4, only q/k/v INT8, group-256 NF4) were compared on teacher-forced accuracy, and this one was the best balance of accuracy and speed (top-1 agreement / KL on 512 tokens: Granite 4.2 8B 88.5% / 0.097, the INT4 build 85.5% / 0.132). They carry 2- and 64-token graphs only (NPU 4000+ answers on the 2-token ones).
 
+## onw's NPU kit: graphs without weights and shared weights (onw 3.0, Windows and Linux)
+
+onw 3.0 ships in `onw/npukit` an NPU compiler and an NPU plugin rebuilt from Intel's open sources (npu_compiler and OpenVINO's NPU
+plugin; Apache 2.0, changes in `onw/npukit/patches`). The plugin is built from the very commit of the OpenVINO 2026.5.0b1 that onw
+installs (`2026.5.0-23160-6db412d0a27`); onw points the "NPU" device at it with `ov.Core(onw/npukit/plugins.xml)`, so no file of the
+installed OpenVINO changes. With another OpenVINO build it is not used and OpenVINO's own NPU plugin serves as before. The Linux
+build (.so files made on Ubuntu 24.04) comes packed as `onw/npukit/linux/npukit-linux.tar.xz` (32 MB) and is unpacked next to it at
+the first start; Linux OpenVINO has no plugin-side compiler, so without the kit no graphs without weights can be made there (the
+driver's own compiler is used). Not used on the NPU 3720 (Meteor / Arrow Lake): no graphs without weights there, and OpenVINO's own
+compiler makes ~7% faster graphs for it (`ONW_NPUKIT=1` forces it).
+
+- **Graphs without weights**: compiled with `CACHE_MODE=OPTIMIZE_SIZE` and `ENABLE_WEIGHTLESS`, a graph is split into an init
+  schedule (which lays the weights out for the NPU) and the main one, and written without its weights. At load, the same ov.Model
+  given as `MODEL_PTR` supplies the weights from the model's files (the mapped .bin); the NPU rebuilds them and they become the main
+  schedule's inputs. Passed as inputs, the rebuilt weights run as fast as weights inside the graph, with bit-identical results.
+  - OpenVINO 2026.5's compiler (8.4) fails this with NF4 weights ("ui4 -> quantile convert not supported"). The kit's compiler
+    reinterprets NF4 (the quantile type and quantized types stored as quantile) through its ui4 storage, and keeps cluster splits and
+    permutes of 4-bit values on byte boundaries (inside init schedules only).
+  - 2-bit weights (Bonsai) are not separated (the graph keeps its size) and compile ~7x slower, so they are left as they are.
+  - Nor are INT4 weights (a Granite 4.2 3B INT4 1-token part: 63.5 vs 63.6 MB). With 2+ rows the NPU 4000+ compiler's rewrite of INT4
+    into a lookup table (the 4-bit value used as the table index) clashes with it and fails; fixed, it only compiles 20-35x slower -
+    so INT4 models (the paged ones, without NPUW) do not use it, and the fused graphs stay off by default for them (each variant
+    would hold the weights again; the 16 GB Lunar Lake ran out of memory): the same as 2.1.0. INT8 is separated (an llm-jp-4.1-8B
+    INT8 part: 2.5 vs 216 MB). From Lunar Lake on, the NF4 models are the ones to use.
+- **Shared weights**: graphs without weights still rebuild their weights on the NPU each, so every fused-graph length (1K to 32K
+  positions) and block size adds a copy (~4.8 GB each for an 8B NF4). After an init schedule has run, the kit's plugin matches its
+  output buffers against the existing ones (size, a sampled hash, then all bytes); when all match, the main schedule is bound to
+  those and the new buffer is released (`ONW_WS_SHARE=0` turns it off). So that graphs cut differently (the paged parts and the fused
+  graphs) can share, the init outputs are kept per weight (the compiler's `ONW_WS_NO_CONCAT`; OpenVINO concatenates them into one).
+  - Panther Lake, llm-jp-4.1-8B NF4: six fused variants loaded at once take +5.7 GB (~29 GB without sharing).
+  - Dropping one variant and loading another while their weights are shared can make the NPU driver write into released memory, so
+    3.0 keeps every fused variant and the paged graphs loaded (`ONW_FUSED_KEEP`; the memory-saving mode keeps loading only the length
+    in use). With every variant kept, the fused graphs go on up to the context: 8K / 16K / 32K positions too (2.1.0: up to 4K). Answers
+    and prompts run in the shortest variant covering the current position and move to the next one as it grows (a longer variant
+    reads a longer stretch of the KV cache every step).
+- **Other compiler fixes**:
+  - The Qwen3.5 NF4 4-token graphs (MTP verification) failed with `'async.await' has non 'return' consumer` (a shape-only op left in
+    front of the function result) - without them no MTP.
+  - 2-bit weights with zero points (Bonsai) computing zeros with 2+ rows.
+  - Compile time: parallel compiles serialised on the Windows heap lock (a distribution map allocated ~45 KB even for one entry);
+    constant folding and the vertical-fusion search now reuse their results. Bit-identical output (a Qwen3.5-9B 64-token part
+    52.6 -> 34 s, ...).
+
+For the results (Panther Lake against 2.1.0's default) see v3.0.0 in the CHANGELOG: caches a tenth of the model or less, a fraction
+of the private memory, faster answers and prompts.
+
 ## Host-side work per step (onw 1.4)
 
 Of an MTP block, ~47 ms was Python / numpy. Without changing a single output bit, it is now ~32 ms (the same tokens as

install.ps1

@@ -25,6 +25,7 @@ $info = Get-WithRetry "https://huggingface.co/api/models/$repo" $null
 foreach ($s in $info.siblings) {
     $f = $s.rfilename
     if ($f -eq ".gitattributes") { continue }
+    if ($f.StartsWith("onw/npukit/linux/")) { continue }      # (the NPU kit for Linux)
     $out = Join-Path $dest ($f -replace "/", "\")
     New-Item -ItemType Directory -Force -Path (Split-Path $out) | Out-Null
     Get-WithRetry "https://huggingface.co/$repo/resolve/main/$f" $out

install.sh

@@ -28,6 +28,7 @@ def fetch(url, tries=4):                      # the hub redirects to a CDN; a re
 
 info = json.loads(fetch(f"https://huggingface.co/api/models/{repo}"))
 files = [s["rfilename"] for s in info["siblings"] if s["rfilename"] != ".gitattributes"]
+files = [f for f in files if not (f.startswith("onw/npukit/") and f.endswith(".dll"))]   # (the NPU kit for Windows)
 for f in files:
     out = os.path.join(dest, f)
     os.makedirs(os.path.dirname(out), exist_ok=True)

onw/__init__.py

@@ -2,4 +2,4 @@
 
 A small engine that runs LLMs entirely on the Intel NPU: static segment graphs + host-managed state.
 """
-__version__ = "2.1.0"
+__version__ = "3.0.0"

onw/app.py

@@ -32,15 +32,15 @@ CATALOG = [  # repo, title, takes images, download GB, decode speed (NPU 3720),
      "エージェント・ツール呼び出し向けの軽量モデル（Qwen3.5-4B ベース）"),
     ("ryugyosoft/granite-4.2-3b-onw", "Granite 4.2 3B", False, 2.1, "7.7 tok/s",
      "IBM の軽量モデル。思考とツール呼び出しに対応（日本語はやや弱め）"),
-    ("ryugyosoft/granite-4.2-3b-nf4-onw", "Granite 4.2 3B (NF4)", False, 2.6, "14 tok/s",
+    ("ryugyosoft/granite-4.2-3b-nf4-onw", "Granite 4.2 3B (NF4)", False, 2.6, "23 tok/s",
      "Lunar Lake 以降専用の高速版（重み NF4、アテンションの入力と down_proj は INT8。INT4 版より高精度）"),
     ("ryugyosoft/granite-4.2-8b-onw", "Granite 4.2 8B", False, 5.0, "3.9 tok/s",
      "IBM の 8B。思考とツール呼び出し・コーディング向け（メモリ 32 GB 以上推奨）"),
-    ("ryugyosoft/granite-4.2-8b-nf4-onw", "Granite 4.2 8B (NF4)", False, 6.2, "9 tok/s",
+    ("ryugyosoft/granite-4.2-8b-nf4-onw", "Granite 4.2 8B (NF4)", False, 6.2, "13 tok/s",
      "Lunar Lake 以降専用の高速版（重み NF4、アテンションの入力と down_proj は INT8。INT4 版より高精度）"),
     ("ryugyosoft/llm-jp-4.1-8b-thinking-onw", "llm-jp-4.1-8B-thinking", False, 4.6, "4.5 tok/s",
      "国立情報学研究所の日本語モデル。考えてから答えます（思考モードで思考を表示）"),
-    ("ryugyosoft/llm-jp-4.1-8b-thinking-nf4-onw", "llm-jp-4.1-8B-thinking (NF4)", False, 6.1, "9.9 tok/s",
+    ("ryugyosoft/llm-jp-4.1-8b-thinking-nf4-onw", "llm-jp-4.1-8B-thinking (NF4)", False, 6.1, "14 tok/s",
      "Lunar Lake 以降専用の高速版（重み NF4、アテンションの入力と down_proj は INT8。INT4 版より高精度）"),
     ("ryugyosoft/llm-jp-4.1-8b-thinking-int8-onw", "llm-jp-4.1-8B-thinking (INT8)", False, 7.7, "4.1 tok/s",
      "llm-jp の高精度版（重み INT8。メモリ 24 GB 以上向け、16 GB の PC はメモリ節約モードで）"),
@@ -358,11 +358,30 @@ def context_of(repo, cfg):
     return min(want, top) if top else want
 
 
+# [3.0] with onw's NPU kit (Windows and Linux, NPU 4000+): weightless graphs and one copy of the weights for every graph - peak
+# working set measured on Panther Lake with every fused variant loaded (onw 3.0)
+MEMORY_KIT = {
+    "ryugyosoft/granite-4.2-3b-nf4-onw": (4.8, 0.0, 80),
+    "ryugyosoft/granite-4.2-8b-nf4-onw": (9.8, 0.0, 80),
+    "ryugyosoft/llm-jp-4.1-8b-thinking-nf4-onw": (9.7, 0.0, 64),
+    "ryugyosoft/Qwen3.5-9B-nf4-onw": (6.0, 0.0, 16),
+    "ryugyosoft/Ornith-1.5-9B-nf4-onw": (6.0, 0.0, 16),
+}
+
+
+def _kit():
+    try:
+        from . import npukit
+        return npukit.active() and npu_arch() >= 4000
+    except Exception:
+        return False
+
+
 def memory_full(repo, cfg, ram):
     """GB this model needs with the whole expert bank in memory (context length, 64-token blocks), or None."""
     if repo not in MEMORY:
         return None
-    base, s64, kv = MEMORY[repo]
+    base, s64, kv = MEMORY_KIT[repo] if repo in MEMORY_KIT and _kit() else MEMORY[repo]
     return base + (s64 if big_blocks_on(cfg, ram) else 0.0) + kv * max(context_of(repo, cfg) - 1024, 0) / 1024 / 1024
 
 
@@ -423,8 +442,9 @@ def npu_arch():
     if NPU_ARCH is None:
         try:
             import openvino as ov
+            from . import npukit
             from .runtime import npu_arch as arch
-            NPU_ARCH = arch(ov.Core())
+            NPU_ARCH = arch(npukit.core())
         except Exception:
             NPU_ARCH = 0
     return NPU_ARCH

onw/chat.py

@@ -5,6 +5,7 @@ import re
 import hashlib, json, os, time
 import numpy as np
 import openvino as ov
+from . import npukit
 from transformers import AutoTokenizer
 
 from .runtime import Cancelled, SegmentedModel, all_sizes, grouped_wanted
@@ -112,7 +113,7 @@ class ChatEngine:
     _cancel = None                             # should_stop of the request whose prompt is being fed
 
     def __init__(self, engine_dir, device=None, pld=True, cache_dir=None, context=None):
-        core = ov.Core()
+        core = npukit.core()
         device = device or ("NPU" if npu_available(core) else "CPU")
         cfg = {"CACHE_DIR": cache_dir or os.path.join(engine_dir, "npu_cache")} if device == "NPU" else \
             {"INFERENCE_PRECISION_HINT": "f32"}
@@ -787,9 +788,14 @@ class ChatEngine:
             # [2.0.4] the fused graphs long enough for this prompt and its answer, set before the prompt runs (its last
             # block of 1-2 tokens goes to the answer graphs: a switch there would load them twice)
             sizes = self.model._fused_sizes
+            # ([3.0] every variant kept loaded: the shortest one for now - a longer one takes over at no cost once the
+            # prompt / the answer passes it (_step); a variant longer than needed reads its whole cache every block:
+            # a 21.8K-token prompt all in the 32K graphs ran at 30 tok/s)
+            keep = self.model.fused_keep_all()
             if len(sizes) > 1 and n - reuse > 4:
-                self.model.fused_prepare(n, max(sizes))
-            self.model.fused_prepare(n + min(max_new_tokens, 1024), min(sizes))
+                self.model.fused_prepare(reuse + max(sizes) if keep else n, max(sizes))
+            ahead = 64 if keep else min(max_new_tokens, 1024)
+            self.model.fused_prepare(n + ahead, min(sizes))
         try:
             if B > reuse:
                 self._prefill(lids, reuse, B, embeds, pos, need_logits=False)
@@ -816,8 +822,9 @@ class ChatEngine:
             self.model.swap_to(1, drop=16)
         elif swap:                                     # memory-saving mode: the 1-token graphs for the answer
             self.model.start_size(1, need_gb=self.swap_gb + 1.0, then_drop=16)
-        if not swap:                                   # [2.0.4] fused decode graphs long enough for most answers
-            self.model.fused_prepare(self.model.cur + min(max_new_tokens, 1024))
+        if not swap and getattr(self.model, "_fused_sizes", None):   # [2.0.4] fused decode graphs for most answers
+            ahead = 64 if self.model.fused_keep_all() else min(max_new_tokens, 1024)
+            self.model.fused_prepare(self.model.cur + ahead)
         # exact only for plain greedy; not while block sizes are swapped (the verification blocks go away mid-answer)
         use_pld = self.pld and plain_greedy and self.model.S > 1 and not swap and budget is None
         # MTP drafts (Qwen3.5 dense): every sampling setting (exact), the thinking budget included

onw/fused.py

@@ -196,7 +196,21 @@ def wanted(m):
     [2.1.0] off by default - every variant's blob holds its own copy of the weights (Granite 4.2 8B NF4: ~42 GB of
     cache against ~11 GB for the paged graphs); ONW_FUSED=1: on; ONW_FUSED_PROMPT=0: answers only)."""
     import os
-    if os.environ.get("ONW_FUSED", "0") != "1" or m._ctx["device"] != "NPU" or not m.paged or m.kv is None:
+    # [3.0] on by default with onw's NPU kit (weightless blobs: the variants cost no disk, their weights are shared)
+    # (not on the NPU 3720: no weightless blobs there - every variant would hold its own weights)
+    from . import npukit
+    from .runtime import npu_arch
+    if m._ctx["device"] != "NPU" or not m.paged or m.kv is None:
+        return None
+    # (nor for 2-bit weights: they stay inside the blobs, so the fused graphs would hold a second copy - Ternary Bonsai 2
+    # 9B: 7.1 GB private instead of ~3.5)
+    # (nor when this model's graphs keep their weights - INT4 (runtime.int4_weights), or a part the compiler could not
+    # build without them: every variant would carry the weights again; Lunar Lake 16 GB, Granite 4.2 3B INT4 with the
+    # fused graphs ran out of memory)
+    from .runtime import _WEIGHTLESS
+    on = (npukit.active() and npu_arch(m._ctx["core"]) >= 4000 and m.meta.get("needs") != "u2"
+          and not _WEIGHTLESS.get("skip_model") and _WEIGHTLESS["ok"] is not False and not _WEIGHTLESS.get("full"))
+    if os.environ.get("ONW_FUSED", "1" if on else "0") != "1":
         return None
     if m.E or m.predict_k or m.window or m.ring or not m.segs:
         return None
@@ -244,7 +258,8 @@ class FusedDecode:
         cfg = {k: v for k, v in c["config"].items() if k != "CACHE_DIR"}
         cache = c["config"].get("CACHE_DIR")
         npuw = cfg.get("NPU_USE_NPUW") == "YES"
-        self.plan, self.common, producer, bins = [], {}, {}, {}   # (bins: the mapped weight files, alive while building)
+        self.plan, self.common, producer, bins = [], {}, {}, {}
+        bp = bpw = None   # (bins: the mapped weight files, alive while building)
         self.mask_kv_t = mk(ov.Type.f16, [1, self.Ct])
         self.mask_self_t = mk(ov.Type.f16, [S, S])
         t0, built = time.time(), 0
@@ -286,24 +301,30 @@ class FusedDecode:
                 log(f"[fused] {name} ({self.Ct} positions, NPUW): ready in {time.time() - t:.0f}s", flush=True)
                 built += 1
             elif cache:
-                bp = blob_path(cache, f"fused-{name[:-4]}-B{B}", [VERSION, c["tried"].key, _signature(model),
-                                                                  sorted(cfg.items()), self.Cs])
+                key = [VERSION, c["tried"].key, _signature(model), sorted(cfg.items()), self.Cs]
+                bp = blob_path(cache, f"fused-{name[:-4]}-B{B}", key)
+                bpw = blob_path(cache, f"fused-{name[:-4]}-B{B}-wl", key)
             lk = _lock(bp) if bp and cm is None else None
             if lk is not None:
                 lk.acquire()                          # (the background may be compiling this graph: wait for it)
             try:
-                if bp and os.path.exists(bp):
-                    USED_BLOBS.add(bp)
+                # a weightless blob (the weights come from this model's constants at import) first, then a full one
+                for path, wl in ((bpw, True), (bp, False)) if bp else ():
+                    if not os.path.exists(path):
+                        continue
+                    USED_BLOBS.add(path)
                     if not compile_only:
                         try:
-                            cm = import_blob(core, bp, device, cfg)
+                            cm = import_blob(core, path, device, {**cfg, "MODEL_PTR": model} if wl else cfg)
                         except (RuntimeError, OSError):
                             cm = None
-                    if cm is None and compile_only:
-                        del model
-                        continue
+                    break
+                if cm is None and compile_only and bp and (os.path.exists(bpw) or os.path.exists(bp)):
+                    del model
+                    continue
                 if cm is None and not npuw:
-                    cm = self._compile(core, model, device, cfg, bp, f"{name} ({self.Ct} positions)", compile_only, log)
+                    cm = self._compile(core, model, device, cfg, bp, bpw, f"{name} ({self.Ct} positions)",
+                                       compile_only, log)
                     built += 1
             finally:
                 if lk is not None:
@@ -352,16 +373,39 @@ class FusedDecode:
             f"({built} compiled) in {time.time() - t0:.0f}s", flush=True)
         self._mask_pos = None
 
+    _weightless = None                                # (False once a weightless compile failed in this process)
+
     @staticmethod
-    def _compile(core, model, device, cfg, bp, name, compile_only, log):
+    def _compile(core, model, device, cfg, bp, bpw, name, compile_only, log):
         """Compile one fused graph; with a blob path: written there, and (unless compile_only) imported back as the
         driver's copy only. compile_only: no executor on the device - creating one next to running inference lost the
-        NPU on Panther Lake (ZE_RESULT_ERROR_DEVICE_LOST). Returns the compiled model, or None (compile_only)."""
+        NPU on Panther Lake (ZE_RESULT_ERROR_DEVICE_LOST). Returns the compiled model, or None (compile_only).
+        Weightless first (bpw): the blob keeps no weights (CACHE_MODE=OPTIMIZE_SIZE + ENABLE_WEIGHTLESS; the import
+        rebuilds them on the NPU from this model's constants, MODEL_PTR) - a fused graph holds a whole segment's
+        weights, ~1.2 GB each for an 8B model, so every variant would add a copy of the model to the cache otherwise
+        (~19 MB weightless). Needs a compiler that supports it for the weight format (onw's own build: NF4 too; pip
+        OpenVINO 2026.5: not NF4; NPU 3720: no) - else the full blob as before. ONW_FUSED_WEIGHTLESS=0: off."""
         import os, io, time
         from .runtime import import_blob, USED_BLOBS
+        base = {**cfg, "NPU_CREATE_EXECUTOR": "0"} if compile_only else cfg
+        cm, wl = None, False
         t = time.time()
-        cm = core.compile_model(model, device, {**cfg, "NPU_CREATE_EXECUTOR": "0"} if compile_only else cfg)
-        log(f"[fused] {name}: compiled in {time.time() - t:.0f}s{' (background)' if compile_only else ''}", flush=True)
+        from .runtime import weightless_wanted
+        if bpw and FusedDecode._weightless is not False and os.environ.get("ONW_FUSED_WEIGHTLESS", "1") != "0"                 and weightless_wanted():
+            try:
+                cm = core.compile_model(model, device, {**base, "CACHE_MODE": "OPTIMIZE_SIZE",
+                                                        "ENABLE_WEIGHTLESS": True, "NPU_COMPILER_TYPE": "PLUGIN"})
+                wl = FusedDecode._weightless = True
+            except RuntimeError as e:
+                FusedDecode._weightless = False
+                log(f"[fused] weightless blobs not supported here ({' '.join(str(e).split())[-120:]}): full blobs",
+                    flush=True)
+        if cm is None:
+            cm = core.compile_model(model, device, base)
+        log(f"[fused] {name}: compiled{' weightless' if wl else ''} in {time.time() - t:.0f}s"
+            f"{' (background)' if compile_only else ''}", flush=True)
+        if wl:
+            bp = bpw
         if not bp:
             return None if compile_only else cm
         os.makedirs(os.path.dirname(bp), exist_ok=True)
@@ -372,7 +416,7 @@ class FusedDecode:
         os.replace(f"{bp}.{os.getpid()}.tmp", bp)
         del buf, cm
         USED_BLOBS.add(bp)
-        return None if compile_only else import_blob(core, bp, device, cfg)
+        return None if compile_only else import_blob(core, bp, device, {**cfg, "MODEL_PTR": model} if wl else cfg)
 
     def step(self, ids, want_logits=True, embeds=None):
         m, S, n = self.m, self.S, len(ids)

onw/manager.py

@@ -1,685 +1,686 @@
-"""The onw manager: model list, downloads, server control and settings behind a small local HTTP API, used by the
-window (onw/web/manage.html in a webview or a browser). It runs inside the tray process, so downloads keep going when
-the window is closed; `onw window` without a tray runs its own copy.
-
-  GET  /manage.html            the page
-  GET  /api/state              everything the page shows (polled once a second)
-  POST /api/<action>           load / unload / restart / download / cancel_download / delete / clear_cache /
-                               folders / settings / open
-
-Bound to 127.0.0.1 only; every /api call needs the token from <config>/manager.json (X-Onw-Token header or ?token=).
-"""
-import json, os, secrets, shutil, subprocess, sys, threading, time, webbrowser
-from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
-from urllib.parse import parse_qs, urlparse
-
-from . import app, cachever, __version__
-from .cachever import rmtree
-from . import i18n
-from .i18n import T
-
-WEB = os.path.join(os.path.dirname(os.path.abspath(__file__)), "web")
-INFO = lambda: os.path.join(app.config_dir(), "manager.json")
-SPEED = {c[0]: c[4] for c in app.CATALOG}
-NOTE = {c[0]: c[5] for c in app.CATALOG}
-LAST_SEEN = [time.time()]            # when the page last asked for the state (a browser-only window keeps us alive)
-
-
-def _files(d):
-    """{path: (size, mtime)} under a model folder, without the compile cache."""
-    out = {}
-    for root, dirs, files in os.walk(d):
-        dirs[:] = [x for x in dirs if x != "npu_cache"]
-        for f in files:
-            p = os.path.join(root, f)
-            try:
-                st = os.stat(p)
-                out[p] = (st.st_size, st.st_mtime_ns)
-            except OSError:
-                pass
-    return out
-
-
-def _to_fetch(siblings, dest):
-    """Bytes a download into dest still has to fetch: files missing there, or older than the repo's (the hub's own
-    record of what it downloaded; without it, a size mismatch)."""
-    try:
-        from pathlib import Path
-        from huggingface_hub._local_folder import read_download_metadata
-    except ImportError:
-        read_download_metadata = None
-    n = 0
-    for s in siblings:
-        p = os.path.join(dest, s.rfilename)
-        if not os.path.exists(p):
-            n += s.size or 0
-            continue
-        want = s.lfs.sha256 if s.lfs else s.blob_id
-        try:
-            meta = read_download_metadata(Path(dest), s.rfilename) if read_download_metadata else None
-        except Exception:
-            meta = None
-        stale = (meta.etag != want) if meta is not None and want else os.path.getsize(p) != (s.size or 0)
-        n += (s.size or 0) if stale else 0
-    return n
-
-
-def _installed_version():
-    """The version of the onw files on disk now (this tray process may still run an older one: files replaced by the
-    installer or by hand while it ran - the window warns, as settings it does not know would be dropped)."""
-    import re
-    try:
-        s = open(os.path.join(os.path.dirname(os.path.abspath(__file__)), "__init__.py"), encoding="utf8").read()
-        return re.search(r'__version__\s*=\s*"([^"]+)"', s).group(1)
-    except (OSError, AttributeError):
-        return __version__
-
-
-def find_manager():
-    """{port, token} of a running manager (the tray), or None."""
-    try:
-        d = json.load(open(INFO()))
-        if app._alive(d["pid"], d.get("created")):        # (a file left by a crash may name a reused pid)
-            return d
-    except (OSError, ValueError, KeyError):
-        pass
-    return None
-
-
-class Manager:
-    def __init__(self):
-        self.lock = threading.Lock()
-        self.dl = None                  # {"repo", "proc", "dest", "total", "frac", "text", "error", "done", "t0"}
-        self.npu = None
-        self.ram = app.total_ram_gb()
-        self.busy = ""                  # a message about a long action (loading / deleting) for the page
-        self.busy_err = False
-        self.upd = {"latest": None, "tag": None, "notes": "", "checked": None, "state": "idle", "msg": "",
-                    "checking": False, "error": ""}
-        self.on_updated = None          # set by the tray: restart into the new version
-        self.on_update_found = None     # set by the tray: notification
-        threading.Thread(target=self._update_loop, daemon=True).start()
-        threading.Thread(target=self._npu_info, daemon=True).start()
-
-    def _npu_info(self):
-        try:
-            import openvino as ov
-            from .runtime import npu_info, npu_arch
-            core = ov.Core()
-            app.NPU_ARCH = npu_arch(core)
-            self.npu = npu_info(core)
-        except Exception:
-            self.npu = "NPU: unknown"
-
-    # ---------------------------------------------------------------- state
-    def state(self):
-        LAST_SEEN[0] = time.time()
-        cfg = app.load_config()
-        inst = app.installed(cfg)
-        app.note_max_contexts(inst)
-        info = app.server_info()
-        status = app.server_status(cfg, info=info)
-        save = app.models_dir(cfg)
-        self._poll_download()
-        dl = self.dl
-        models = []
-        for repo, title, image, size, _, _ in app.catalog():
-            path = inst.get(repo)
-            mem = app.memory_estimate(repo, cfg, self.ram)
-            stream = app.expert_budget(repo, cfg, self.ram)
-            m = {"key": repo, "stream_gb": stream, "title": title, "catalog": True, "path": path, "image": image, "size_gb": size,
-                 "mem_gb": mem, "fits": app.fits(mem, self.ram),
-                 "speed": SPEED.get(repo, "") if stream is None else T("遅め（SSD）"),
-                 "note": (T(NOTE.get(repo, "").split("（")[0]) + T("・エキスパートは SSD から読み込み（キャッシュ {0} GB）", f"{stream:g}"))
-                         if stream else T(NOTE.get(repo, "")), "in_save_dir": bool(path) and os.path.dirname(path) == save,
-                 "deletable": bool(path) and app.downloaded_by_onw(path), "cache_gb": self._cache_gb(path) if path else 0}
-            if app.mem_saver_on(repo, cfg):
-                m["note"] += T("・メモリ節約モード（回答の出だしは遅め）")
-            if path and cachever.stale(path):
-                m["note"] += T("・onw の更新で、次のロードでは NPU 向けの準備をやり直すことがあります（数分）")
-            m["think"] = app.think_kind(repo, path)
-            m["state"] = self._state_of(repo, path, info, status, dl)
-            if m["state"] == "ready" and app.update_interrupted(path):
-                m["state"], m["note"] = "outdated", T("更新が途中で止まっています（「更新」で残りを取り直します。それまでは正しく動かないことがあります）")
-            elif m["state"] == "ready" and app.outdated(repo, path):
-                m["state"], m["note"] = "outdated", T("新しい版があります（「更新」で変わったファイルだけ取り直します）")
-            if dl and dl["repo"] == repo and not dl.get("done"):
-                m["progress"] = dl["frac"]
-                m["note"] = T("ダウンロード中 {0}・ウィンドウを閉じても続きます", dl["text"])
-            elif dl and dl.get("done") == repo and dl.get("error"):
-                m["note"] = T("ダウンロードに失敗: {0}", dl["error"])
-            models.append(m)
-        for key, path in inst.items():
-            if key in app.CATALOG_REPOS:
-                continue
-            models.append({"key": key, "title": key, "catalog": False, "path": path, "image": None, "size_gb": None,
-                           "mem_gb": None, "fits": True, "speed": "", "note": T("自分で置いたモデル・{0}", path),
-                           "in_save_dir": os.path.dirname(path) == save, "deletable": app.downloaded_by_onw(path),
-                           "cache_gb": self._cache_gb(path), "state": self._state_of(key, path, info, status, None),
-                           "think": app.think_kind(key, path)})
-            if cachever.stale(path):
-                models[-1]["note"] += T("・onw の更新で、次のロードでは NPU 向けの準備をやり直すことがあります（数分）")
-            if app.update_interrupted(path):                   # (not in the catalog: no "update" to finish it with)
-                models[-1]["note"] += T("・ダウンロードが途中で止まっています（正しく動かないことがあります）")
-        downloading = dl["dest"] if dl and not dl.get("done") else None
-        for name, path in app.partial_downloads(cfg).items():  # stopped part-way: listed so they can be removed
-            if downloading and os.path.normcase(path) == os.path.normcase(downloading):
-                continue
-            models.append({"key": "partial:" + name, "title": name, "catalog": False, "path": path, "image": None,
-                           "size_gb": None, "mem_gb": None, "fits": True, "speed": "", "partial": True,
-                           "note": T("途中で止まったダウンロード・{0}", path), "in_save_dir": True, "deletable": True,
-                           "cache_gb": 0, "state": "partial"})
-        return {"version": __version__, "installed_version": _installed_version(), "npu": self.npu or "", "ram_gb": self.ram,
-                "config": cfg, "api_url": app.api_url(cfg, info),
-                "models_dir_missing": app.models_dir_missing(cfg),
-                # the memory-saving switch is shown only when a model it applies to is in the folders
-                "saver_models": [app.model_title(k) for k, p in inst.items() if app.saver_applies(p, cfg)],
-                "server": {"status": status, "model": info and info.get("model"), "path": info and info.get("path"),
-                           "progress": app.load_progress() if status == "loading" else None,
-                           "recompile": app.recompile_reason() if status == "loading" else None,
-                           "error": app.failure_reason() if status == "failed" else "",
-                           "title": app.model_title(info["model"]) if info else None, "port": info and info.get("port"),
-                           "speed": SPEED.get(info["model"], "") if info else "",
-                           # (the setting at most the model's longest: the setting itself applies from the next start)
-                           "context": app.context_of(info["model"], cfg) if info else None,
-                           "max_context": app.MAX_CONTEXT.get(info["model"]) if info else None,
-                           "mem_gb": app.memory_estimate(info["model"], cfg, self.ram) if info else None},
-                "save_dir": save, "folders": app.model_folders(cfg, existing=False)[1:], "models": models,
-                # (a start that failed from any path - this window, the tray, an update - until the next start / stop)
-                "busy": self.busy or (T("ロードできません: {0}", app.LAST_ERROR[0]) if app.LAST_ERROR[0] else ""),
-                "busy_err": self.busy_err or bool(not self.busy and app.LAST_ERROR[0]),
-                "log": os.path.join(app.config_dir(), "server.log"), "store_python": app.store_python(),
-                "deps_error": self._deps_error(), "usage": self._usage(info),
-                "update": {**self.upd, "can_update": self._can_update(), "current": __version__}}
-
-    @staticmethod
-    def _usage(info):
-        """Memory (onw's server, the whole machine) and the NPU's load for the server tab (onw.usage)."""
-        try:
-            from . import usage
-            return usage.sample(info["pid"] if info else None)
-        except Exception:
-            return None
-
-    @staticmethod
-    def _deps_error():
-        """The packages of this version could not be installed (pip failed in the update's restart): its error."""
-        from . import deps, update
-        d = update.install_dir()
-        return deps.failed(d) if d and not deps.current(d) else None
-
-    @staticmethod
-    def _state_of(key, path, info, status, dl):
-        if info and info.get("model") == key:
-            return "live" if status == "running" else ("loading" if status == "loading" else "failed")
-        if dl and dl["repo"] == key and not dl.get("done"):   # (a finished download is just there: ready)
-            return "downloading"
-        if path:
-            return "ready"
-        return "missing"
-
-    @staticmethod
-    def _cache_gb(path):
-        c = os.path.join(path, "npu_cache")
-        if not os.path.isdir(c):
-            return 0
-        n = 0
-        for r, _, fs in os.walk(c):
-            for f in fs:
-                try:
-                    n += os.path.getsize(os.path.join(r, f))
-                except OSError:
-                    pass
-        return round(n / 2**30, 1)
-
-    # ---------------------------------------------------------------- downloads (a child process per download)
-    def download(self, repo):
-        if repo not in app.CATALOG_REPOS:
-            return {"error": T("そのモデルはありません")}
-        if repo in app.NF4_ONLY and app.npu_arch() < 4000:    # (NF4 and 2-bit weights)
-            return {"error": T("このモデルは Lunar Lake 以降の NPU 専用です")}
-        info = app.server_info()
-        if info and info.get("model") == repo:            # updating it: its files are open in the server
-            return {"error": T("このモデルはロード中です。先にアンロードしてください。")}
-        if self.upd["state"] == "updating":
-            return {"error": T("onw の更新中です。終わってからダウンロードしてください")}
-        with self.lock:
-            if self.dl and self.dl["proc"].poll() is None:
-                return {"error": T("別のダウンロードが進行中です")}
-        dest = os.path.join(app.models_dir(app.load_config()), repo.split("/")[-1])
-        try:                                               # (the network call outside the lock: cancel stays instant)
-            from huggingface_hub import HfApi
-            total = _to_fetch(HfApi().model_info(repo, files_metadata=True).siblings, dest) or 1
-        except Exception as e:
-            return {"error": T("Hugging Face に接続できません: {0}", e)}
-        with self.lock:
-            if self.dl and self.dl["proc"].poll() is None:
-                return {"error": T("別のダウンロードが進行中です")}
-            log = os.path.join(app.config_dir(), "download.log")
-            before = _files(dest)                          # (before the child writes anything)
-            kw = {"creationflags": subprocess.CREATE_NO_WINDOW} if os.name == "nt" else {}
-            with open(log, "w", encoding="utf8") as out:   # a file, not a pipe: a chatty download never blocks on it
-                proc = subprocess.Popen([sys.executable, "-m", "onw.dl", repo, dest], stdout=out, stderr=subprocess.STDOUT,
-                                        env={**os.environ, "PYTHONUTF8": "1"}, **kw)
-            self.dl = {"repo": repo, "proc": proc, "dest": dest, "total": total, "frac": 0.0, "text": T("準備中…"),
-                       "error": None, "done": None, "t0": time.time(), "log": log, "before": before}
-            app.ACTIVE_DOWNLOAD[0] = dest                  # (an update: not loaded while its files are replaced)
-            return {}
-
-    def cancel_download(self):
-        with self.lock:
-            if self.dl and self.dl["proc"].poll() is None:
-                self.dl["proc"].terminate()
-                try:
-                    self.dl["proc"].wait(10)
-                except subprocess.TimeoutExpired:
-                    self.dl["proc"].kill()
-                    try:
-                        self.dl["proc"].wait(5)
-                    except subprocess.TimeoutExpired:
-                        pass
-                self._update_stopped(self.dl)
-            self.dl = None
-            app.ACTIVE_DOWNLOAD[0] = None
-        return {}
-
-    @staticmethod
-    def _update_stopped(dl):
-        """An update of a complete model cancelled or failed: when it replaced no file yet, the model is just as it
-        was - the marker goes (the hub's partial files stay in .cache for the next try). Otherwise the folder stays
-        marked: listed as an update to finish (app.update_interrupted), never as a partial download to delete."""
-        d, before = dl["dest"], dl["before"]
-        mark = os.path.join(d, ".downloading")
-        if not app.downloaded_by_onw(d) or mark in before:    # (a first download / one stopped before this try)
-            return
-        skip = (os.path.join(d, ".cache") + os.sep, mark)
-        mine = lambda files: {p: st for p, st in files.items() if not p.startswith(skip)}
-        if mine(_files(d)) == mine(before):
-            try:
-                os.remove(mark)
-            except OSError:
-                pass
-
-    def _poll_download(self):
-        dl = self.dl
-        if not dl or dl.get("done"):
-            return
-        before = dl["before"]                               # only what this download wrote (an update keeps the rest)
-        n = sum(st[0] for p, st in _files(dl["dest"]).items() if before.get(p) != st)
-        dl["frac"] = min(n / dl["total"], 0.99)
-        dl["text"] = f"{n / 2**30:.1f} / {dl['total'] / 2**30:.1f} GB ({dl['frac'] * 100:.0f}%)"
-        rc = dl["proc"].poll()
-        if rc is not None:
-            dl["done"] = dl["repo"]
-            app.ACTIVE_DOWNLOAD[0] = None
-            if rc != 0:
-                try:
-                    out = open(dl["log"], encoding="utf8", errors="replace").read()[-4000:]
-                except OSError:
-                    out = ""
-                dl["error"] = " ".join(out.split()[-30:]) or f"exit code {rc}"
-                self._update_stopped(dl)
-            else:
-                dl["frac"] = 1.0
-                with app._CFG_LOCK:
-                    if not app.load_config().get("model"):
-                        app.update_config(model=dl["repo"])
-
-    # ---------------------------------------------------------------- server
-    def load(self, key):
-        if key not in app.installed(app.load_config()):
-            return {"error": T("そのモデルはありません")}
-        with self.lock:                                    # (a second click while the first load is still starting)
-            if self.busy and not self.busy_err:
-                return {"error": T("ロードの準備中です。少しお待ちください")}
-            self.busy, self.busy_err = T("ロードしています…"), False
-        try:
-            cfg = app.update_config(model=key)
-        except Exception:
-            self.busy = ""
-            raise
-
-        def work():
-            try:
-                app.start_server(cfg)
-            except Exception:                          # (the reason: app.LAST_ERROR, shown by state())
-                pass
-            self.busy, self.busy_err = "", False
-        threading.Thread(target=work, daemon=True).start()
-        return {}
-
-    def unload(self):
-        threading.Thread(target=app.stop_server, daemon=True).start()
-        return {}
-
-    def restart(self):
-        info = app.server_info()
-        return self.load(info["model"]) if info else {}
-
-    # ---------------------------------------------------------------- models
-    def delete(self, key):
-        cfg = app.load_config()
-        if key.startswith("partial:"):                     # a download that stopped part-way
-            p = app.partial_downloads(cfg).get(key[8:])
-            if not p or (self.dl and not self.dl.get("done") and os.path.normcase(self.dl["dest"]) == os.path.normcase(p)):
-                return {"error": T("そのモデルはありません")}
-            rmtree(p)
-            return {}
-        inst = app.installed(cfg)
-        info = app.server_info()
-        if key not in inst:
-            return {"error": T("そのモデルはありません")}
-        if info and info.get("model") == key:
-            return {"error": T("このモデルはロ���ド中です。先にアンロードしてください。")}
-        if not app.downloaded_by_onw(inst[key]):
-            return {"error": T("onw がダウンロードしたモデルではないので、ここからは削除しません。")}
-        rmtree(inst[key])                                  # (the compile cache inside: read-only files)
-        return {}
-
-    def clear_cache(self, key):
-        cfg = app.load_config()
-        inst = app.installed(cfg)
-        info = app.server_info()
-        if key not in inst:
-            return {"error": T("そのモデルはありません")}
-        if info and info.get("model") == key:
-            return {"error": T("このモデルはロード中です。先にアンロードしてください。")}
-        rmtree(os.path.join(inst[key], "npu_cache"))
-        return {}
-
-    def folders(self, save_dir=None, add=None, remove=None):
-        with app._CFG_LOCK:
-            return self._folders(save_dir, add, remove)
-
-    def _folders(self, save_dir=None, add=None, remove=None):
-        cfg = app.load_config()
-        nc = os.path.normcase
-        if save_dir:
-            old, new = app.models_dir(cfg), os.path.abspath(save_dir)
-            try:
-                os.makedirs(new, exist_ok=True)
-            except OSError as e:
-                return {"error": T("そのフォルダは使えません: {0}", e)}
-            extra = [x for x in cfg.get("model_folders") or [] if nc(x) != nc(new)]
-            if nc(old) != nc(new) and app.installed({**cfg, "model_folders": []}):
-                extra.append(old)                          # keep finding the models already downloaded to the old place
-            cfg.update(models_dir=new, model_folders=extra)
-        if add:
-            d = os.path.abspath(add)
-            if not os.path.isdir(d):
-                return {"error": T("フォルダが見つかりません: {0}", d)}
-            if nc(d) not in map(nc, app.model_folders(cfg)):
-                cfg["model_folders"] = (cfg.get("model_folders") or []) + [d]
-        if remove:
-            cfg["model_folders"] = [x for x in cfg.get("model_folders") or [] if nc(x) != nc(os.path.abspath(remove))]
-        app.save_config(cfg)
-        return {}
-
-    def settings(self, new):
-        with app._CFG_LOCK:
-            return self._settings(new)
-
-    def model_prefs(self, key, prefs):
-        """A model's defaults for the API and the chat page (the server reads them per request: no restart):
-        think off | low | medium | high, sampling greedy | recommended | custom and the custom values."""
-        if not key:
-            return {"error": T("モデルが選ばれていません")}
-        p = {}
-        if prefs.get("think") in ("off", "low", "medium", "high"):
-            p["think"] = prefs["think"]
-        if prefs.get("sampling") in ("greedy", "recommended", "custom"):
-            p["sampling"] = prefs["sampling"]
-        try:
-            for k, cast, lo, hi in (("temperature", float, 0, 2), ("top_p", float, 0.01, 1), ("top_k", int, 0, 1000),
-                                    ("presence_penalty", float, 0, 2)):
-                if prefs.get(k) not in (None, ""):
-                    p[k] = min(max(cast(float(prefs[k])), lo), hi)
-        except (TypeError, ValueError):
-            return {"error": T("生成設定の値が不正です")}
-        with app._CFG_LOCK:
-            cfg = app.load_config()
-            allp = dict(cfg.get("model_prefs") or {})
-            allp[key] = p
-            cfg["model_prefs"] = allp
-            app.save_config(cfg)
-        return {"ok": True}
-
-    def _settings(self, new):
-        cfg = app.load_config()
-        try:
-            port, ctx = int(new.get("port", cfg["port"])), int(new.get("context", cfg["context"]))
-            if not (1 <= port <= 65535) or ctx < 256:
-                raise ValueError
-        except (TypeError, ValueError):
-            return {"error": T("ポートかコンテキスト長の値が不正です")}
-        from .i18n import LANGS
-        if new.get("lang", "auto") not in ("auto", *LANGS):
-            new = {k: v for k, v in new.items() if k != "lang"}
-        old = dict(cfg)
-        for k in ("lan", "api_key", "device", "pld", "npu_turbo", "big_blocks", "npuw", "expert_gb", "autostart_server", "login_start",
-                  "update_check", "lang", "npuw_moe", "npuw_async", "cache_size", "mem_saver", "mem_saver_strict", "think_default"):
-            if k in new:
-                cfg[k] = new[k]
-        if new.get("api_sampling") in ("greedy", "recommended", "custom"):
-            cfg["api_sampling"] = new["api_sampling"]
-        try:
-            for k, cast, lo, hi in (("api_temperature", float, 0, 2), ("api_top_p", float, 0.01, 1), ("api_top_k", int, 0, 1000),
-                                    ("api_presence", float, 0, 2)):
-                if k in new:
-                    cfg[k] = min(max(cast(float(new[k])), lo), hi)
-        except (TypeError, ValueError):
-            return {"error": T("生成設定の値が不正です")}
-        if "api_max_tokens" in new:                            # (0: up to the end of the context)
-            try:
-                cfg["api_max_tokens"] = max(0, int(new["api_max_tokens"]))
-            except (TypeError, ValueError):
-                return {"error": T("生成設定の値が不正です")}
-        if ctx != cfg["context"]:
-            cfg["context_chosen"] = True                       # (the old 1024 default moves up; a choice stays)
-        cfg["port"], cfg["context"] = port, ctx
-        server_keys = ("lan", "api_key", "device", "pld", "npu_turbo", "api_max_tokens", "big_blocks", "npuw", "expert_gb", "port", "context",
-                       "npuw_moe", "npuw_async", "cache_size", "mem_saver", "mem_saver_strict", "think_default",
-                       "api_sampling", "api_temperature", "api_top_p", "api_top_k", "api_presence")
-        app.save_config(cfg)
-        i18n.current(cfg)                                  # messages switch language right away
-        try:
-            app.set_login_start(bool(cfg["login_start"]))
-        except OSError:
-            pass
-        changed = any(str(old.get(k)) != str(cfg.get(k)) for k in server_keys)
-        return {"restart_needed": changed and app.server_info() is not None}
-
-    # ---------------------------------------------------------------- updates of onw itself
-    @staticmethod
-    def _can_update():
-        from . import update
-        return update.can_update()
-
-    def _update_loop(self):
-        time.sleep(30)                                     # let the tray and the server start first
-        while True:
-            if app.load_config().get("update_check", True):
-                self.check_update()
-            time.sleep(24 * 3600)
-
-    def start_check(self):
-        """The window's "Check now": checks in the background and answers at once; the page follows
-        upd["checking"] / upd["error"] (a slow connection must not leave the button without an answer)."""
-        if not self.upd["checking"]:
-            threading.Thread(target=self.check_update, daemon=True).start()
-        return {"checking": True}
-
-    def check_update(self):
-        self.upd.update(checking=True, error="")
-        try:
-            r = self._check_update()
-        except Exception as e:
-            r = {"error": T("更新を確認できませんでした: {0}", e)}
-        self.upd.update(checking=False, error=r.get("error", ""))
-        return r
-
-    def _check_update(self):
-        from . import update
-        found = update.latest()
-        self.upd["checked"] = time.time()
-        if found is None:
-            return {"error": T("更新を確認できませんでした（オフライン？）")}
-        v, tag = found
-        if update.newer(v):
-            fresh = self.upd["latest"] != v
-            self.upd.update(latest=v, tag=tag, notes=update.notes(lang=i18n.current()))
-            if fresh and self.on_update_found:
-                try:
-                    self.on_update_found(v)
-                except Exception:
-                    pass
-        else:
-            self.upd.update(latest=None, tag=None, notes="")
-        return {"latest": self.upd["latest"]}
-
-    def run_update(self):
-        from . import update
-        if not self.upd["tag"]:
-            return {"error": T("新しい版はありません")}
-        if not update.can_update():
-            return {"error": T("このフォルダの onw は 1 行インストーラーで入れたものではないので、自動更新できません")}
-        if self.upd["state"] == "updating":
-            return {}
-        if self.dl and self.dl["proc"].poll() is None:   # the restart would orphan the download
-            return {"error": T("ダウンロードが終わってから更新してください")}
-        self.upd.update(state="updating", msg=T("準備中…"))
-
-        def work():
-            try:
-                v = update.apply(self.upd["tag"], progress=lambda t: self.upd.update(msg=t))
-                self.upd.update(state="done", msg=T("v{0} に更新しました。再起動します…", v))
-                if self.on_updated:
-                    self.on_updated()
-                else:
-                    self.upd["msg"] = T("v{0} に更新しました。onw を起動し直してください", v)
-            except Exception as e:
-                self.upd.update(state="error", msg=T("更新できませんでした: {0}", e))
-        threading.Thread(target=work, daemon=True).start()
-        return {}
-
-    def open(self, what, key=None):
-        cfg = app.load_config()
-        if what == "log":
-            app.open_path(os.path.join(app.config_dir(), "server.log"))
-        elif what == "folder":
-            p = app.installed(cfg).get(key) if key else None
-            app.open_path(p or app.models_dir(cfg))
-        elif what in ("chat", "check"):
-            webbrowser.open(f"http://127.0.0.1:{app.server_port(cfg)}/{what}")
-        return {}
-
-
-class _Handler(BaseHTTPRequestHandler):
-    manager: Manager = None
-    token: str = ""
-
-    def log_message(self, *a):                             # quiet
-        pass
-
-    def _send(self, code, body, ctype="application/json"):
-        data = body if isinstance(body, bytes) else json.dumps(body, ensure_ascii=False).encode("utf8")
-        self.send_response(code)
-        self.send_header("Content-Type", ctype + ("; charset=utf-8" if ctype.startswith("text") else ""))
-        self.send_header("Content-Length", str(len(data)))
-        self.send_header("Cache-Control", "no-store")
-        self.end_headers()
-        self.wfile.write(data)
-
-    def _authed(self, q):
-        t = self.headers.get("X-Onw-Token") or (q.get("token") or [""])[0]
-        return secrets.compare_digest(t, self.token)
-
-    def do_GET(self):
-        u = urlparse(self.path)
-        q = parse_qs(u.query)
-        if u.path in ("/", "/manage.html"):
-            return self._send(200, i18n.page(os.path.join(WEB, "manage.html")), "text/html")
-        if u.path == "/api/state":
-            if not self._authed(q):
-                return self._send(403, {"error": T("この画面は onw のウィンドウかトレイから開いてください")})
-            return self._send(200, self.manager.state())
-        self._send(404, {"error": "not found"})
-
-    def do_POST(self):
-        u = urlparse(self.path)
-        if not self._authed(parse_qs(u.query)):
-            return self._send(403, {"error": T("この画面は onw のウィンドウかトレイから開いてください")})
-        m = self.manager
-        act = u.path[len("/api/"):]
-        try:
-            n = int(self.headers.get("Content-Length") or 0)
-            body = json.loads(self.rfile.read(n) or b"{}")
-            if act == "load":
-                r = m.load(body["key"])
-            elif act == "unload":
-                r = m.unload()
-            elif act == "restart":
-                r = m.restart()
-            elif act == "download":
-                r = m.download(body["repo"])
-            elif act == "cancel_download":
-                r = m.cancel_download()
-            elif act == "delete":
-                r = m.delete(body["key"])
-            elif act == "clear_cache":
-                r = m.clear_cache(body["key"])
-            elif act == "folders":
-                r = m.folders(body.get("save_dir"), body.get("add"), body.get("remove"))
-            elif act == "settings":
-                r = m.settings(body)
-            elif act == "model_prefs":
-                r = m.model_prefs(body.get("key"), body.get("prefs") or {})
-            elif act == "update_check":
-                r = m.start_check()
-            elif act == "update":
-                r = m.run_update()
-            elif act == "open":
-                r = m.open(body.get("what"), body.get("key"))
-            else:
-                return self._send(404, {"error": "not found"})
-        except Exception as e:
-            r = {"error": T("エラー: {0}", e)}
-        self._send(200, r)
-
-
-def _created():
-    try:
-        import psutil
-        return psutil.Process().create_time()
-    except Exception:
-        return None
-
-
-def forget(info):
-    """Remove manager.json if it still describes this process's manager (at exit)."""
-    try:
-        if json.load(open(INFO())).get("pid") == info.get("pid"):
-            os.remove(INFO())
-    except (OSError, ValueError):
-        pass
-
-
-def serve(manager=None):
-    """Start the manager's HTTP server in a daemon thread; returns {port, token}. Writes <config>/manager.json."""
-    manager = manager or Manager()
-    handler = type("H", (_Handler,), {"manager": manager, "token": secrets.token_hex(16)})
-    srv = ThreadingHTTPServer(("127.0.0.1", 0), handler)
-    threading.Thread(target=srv.serve_forever, daemon=True).start()
-    info = {"port": srv.server_address[1], "token": handler.token, "pid": os.getpid(), "created": _created()}
-    with open(INFO() + ".tmp", "w") as f:
-        json.dump(info, f)
-    try:
-        os.chmod(INFO() + ".tmp", 0o600)                   # (the token) before it is in place
-    except OSError:
-        pass
-    os.replace(INFO() + ".tmp", INFO())
-    return info
-
-
-def page_url(info, tab="server"):
-    return f"http://127.0.0.1:{info['port']}/manage.html?token={info['token']}#{tab}"
+"""The onw manager: model list, downloads, server control and settings behind a small local HTTP API, used by the
+window (onw/web/manage.html in a webview or a browser). It runs inside the tray process, so downloads keep going when
+the window is closed; `onw window` without a tray runs its own copy.
+
+  GET  /manage.html            the page
+  GET  /api/state              everything the page shows (polled once a second)
+  POST /api/<action>           load / unload / restart / download / cancel_download / delete / clear_cache /
+                               folders / settings / open
+
+Bound to 127.0.0.1 only; every /api call needs the token from <config>/manager.json (X-Onw-Token header or ?token=).
+"""
+import json, os, secrets, shutil, subprocess, sys, threading, time, webbrowser
+from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
+from urllib.parse import parse_qs, urlparse
+
+from . import app, cachever, __version__
+from .cachever import rmtree
+from . import i18n
+from .i18n import T
+
+WEB = os.path.join(os.path.dirname(os.path.abspath(__file__)), "web")
+INFO = lambda: os.path.join(app.config_dir(), "manager.json")
+SPEED = {c[0]: c[4] for c in app.CATALOG}
+NOTE = {c[0]: c[5] for c in app.CATALOG}
+LAST_SEEN = [time.time()]            # when the page last asked for the state (a browser-only window keeps us alive)
+
+
+def _files(d):
+    """{path: (size, mtime)} under a model folder, without the compile cache."""
+    out = {}
+    for root, dirs, files in os.walk(d):
+        dirs[:] = [x for x in dirs if x != "npu_cache"]
+        for f in files:
+            p = os.path.join(root, f)
+            try:
+                st = os.stat(p)
+                out[p] = (st.st_size, st.st_mtime_ns)
+            except OSError:
+                pass
+    return out
+
+
+def _to_fetch(siblings, dest):
+    """Bytes a download into dest still has to fetch: files missing there, or older than the repo's (the hub's own
+    record of what it downloaded; without it, a size mismatch)."""
+    try:
+        from pathlib import Path
+        from huggingface_hub._local_folder import read_download_metadata
+    except ImportError:
+        read_download_metadata = None
+    n = 0
+    for s in siblings:
+        p = os.path.join(dest, s.rfilename)
+        if not os.path.exists(p):
+            n += s.size or 0
+            continue
+        want = s.lfs.sha256 if s.lfs else s.blob_id
+        try:
+            meta = read_download_metadata(Path(dest), s.rfilename) if read_download_metadata else None
+        except Exception:
+            meta = None
+        stale = (meta.etag != want) if meta is not None and want else os.path.getsize(p) != (s.size or 0)
+        n += (s.size or 0) if stale else 0
+    return n
+
+
+def _installed_version():
+    """The version of the onw files on disk now (this tray process may still run an older one: files replaced by the
+    installer or by hand while it ran - the window warns, as settings it does not know would be dropped)."""
+    import re
+    try:
+        s = open(os.path.join(os.path.dirname(os.path.abspath(__file__)), "__init__.py"), encoding="utf8").read()
+        return re.search(r'__version__\s*=\s*"([^"]+)"', s).group(1)
+    except (OSError, AttributeError):
+        return __version__
+
+
+def find_manager():
+    """{port, token} of a running manager (the tray), or None."""
+    try:
+        d = json.load(open(INFO()))
+        if app._alive(d["pid"], d.get("created")):        # (a file left by a crash may name a reused pid)
+            return d
+    except (OSError, ValueError, KeyError):
+        pass
+    return None
+
+
+class Manager:
+    def __init__(self):
+        self.lock = threading.Lock()
+        self.dl = None                  # {"repo", "proc", "dest", "total", "frac", "text", "error", "done", "t0"}
+        self.npu = None
+        self.ram = app.total_ram_gb()
+        self.busy = ""                  # a message about a long action (loading / deleting) for the page
+        self.busy_err = False
+        self.upd = {"latest": None, "tag": None, "notes": "", "checked": None, "state": "idle", "msg": "",
+                    "checking": False, "error": ""}
+        self.on_updated = None          # set by the tray: restart into the new version
+        self.on_update_found = None     # set by the tray: notification
+        threading.Thread(target=self._update_loop, daemon=True).start()
+        threading.Thread(target=self._npu_info, daemon=True).start()
+
+    def _npu_info(self):
+        try:
+            import openvino as ov
+            from . import npukit
+            from .runtime import npu_info, npu_arch
+            core = npukit.core()
+            app.NPU_ARCH = npu_arch(core)
+            self.npu = npu_info(core)
+        except Exception:
+            self.npu = "NPU: unknown"
+
+    # ---------------------------------------------------------------- state
+    def state(self):
+        LAST_SEEN[0] = time.time()
+        cfg = app.load_config()
+        inst = app.installed(cfg)
+        app.note_max_contexts(inst)
+        info = app.server_info()
+        status = app.server_status(cfg, info=info)
+        save = app.models_dir(cfg)
+        self._poll_download()
+        dl = self.dl
+        models = []
+        for repo, title, image, size, _, _ in app.catalog():
+            path = inst.get(repo)
+            mem = app.memory_estimate(repo, cfg, self.ram)
+            stream = app.expert_budget(repo, cfg, self.ram)
+            m = {"key": repo, "stream_gb": stream, "title": title, "catalog": True, "path": path, "image": image, "size_gb": size,
+                 "mem_gb": mem, "fits": app.fits(mem, self.ram),
+                 "speed": SPEED.get(repo, "") if stream is None else T("遅め（SSD）"),
+                 "note": (T(NOTE.get(repo, "").split("（")[0]) + T("・エキスパートは SSD から読み込み（キャッシュ {0} GB）", f"{stream:g}"))
+                         if stream else T(NOTE.get(repo, "")), "in_save_dir": bool(path) and os.path.dirname(path) == save,
+                 "deletable": bool(path) and app.downloaded_by_onw(path), "cache_gb": self._cache_gb(path) if path else 0}
+            if app.mem_saver_on(repo, cfg):
+                m["note"] += T("・メモリ節約モード（回答の出だしは遅め）")
+            if path and cachever.stale(path):
+                m["note"] += T("・onw の更新で、次のロードでは NPU 向けの準備をやり直すことがあります（数分）")
+            m["think"] = app.think_kind(repo, path)
+            m["state"] = self._state_of(repo, path, info, status, dl)
+            if m["state"] == "ready" and app.update_interrupted(path):
+                m["state"], m["note"] = "outdated", T("更新が途中で止まっています（「更新」で残りを取り直します。それまでは正しく動かないことがあります）")
+            elif m["state"] == "ready" and app.outdated(repo, path):
+                m["state"], m["note"] = "outdated", T("新しい版があります（「更新」で変わったファイルだけ取り直します）")
+            if dl and dl["repo"] == repo and not dl.get("done"):
+                m["progress"] = dl["frac"]
+                m["note"] = T("ダウンロード中 {0}・ウィンドウを閉じても続きます", dl["text"])
+            elif dl and dl.get("done") == repo and dl.get("error"):
+                m["note"] = T("ダウンロードに失敗: {0}", dl["error"])
+            models.append(m)
+        for key, path in inst.items():
+            if key in app.CATALOG_REPOS:
+                continue
+            models.append({"key": key, "title": key, "catalog": False, "path": path, "image": None, "size_gb": None,
+                           "mem_gb": None, "fits": True, "speed": "", "note": T("自分で置いたモデル・{0}", path),
+                           "in_save_dir": os.path.dirname(path) == save, "deletable": app.downloaded_by_onw(path),
+                           "cache_gb": self._cache_gb(path), "state": self._state_of(key, path, info, status, None),
+                           "think": app.think_kind(key, path)})
+            if cachever.stale(path):
+                models[-1]["note"] += T("・onw の更新で、次のロードでは NPU 向けの準備をやり直すことがあります（数分）")
+            if app.update_interrupted(path):                   # (not in the catalog: no "update" to finish it with)
+                models[-1]["note"] += T("・ダウンロードが途中で止まっています（正しく動かないことがあります）")
+        downloading = dl["dest"] if dl and not dl.get("done") else None
+        for name, path in app.partial_downloads(cfg).items():  # stopped part-way: listed so they can be removed
+            if downloading and os.path.normcase(path) == os.path.normcase(downloading):
+                continue
+            models.append({"key": "partial:" + name, "title": name, "catalog": False, "path": path, "image": None,
+                           "size_gb": None, "mem_gb": None, "fits": True, "speed": "", "partial": True,
+                           "note": T("途中で止まったダウンロード・{0}", path), "in_save_dir": True, "deletable": True,
+                           "cache_gb": 0, "state": "partial"})
+        return {"version": __version__, "installed_version": _installed_version(), "npu": self.npu or "", "ram_gb": self.ram,
+                "config": cfg, "api_url": app.api_url(cfg, info),
+                "models_dir_missing": app.models_dir_missing(cfg),
+                # the memory-saving switch is shown only when a model it applies to is in the folders
+                "saver_models": [app.model_title(k) for k, p in inst.items() if app.saver_applies(p, cfg)],
+                "server": {"status": status, "model": info and info.get("model"), "path": info and info.get("path"),
+                           "progress": app.load_progress() if status == "loading" else None,
+                           "recompile": app.recompile_reason() if status == "loading" else None,
+                           "error": app.failure_reason() if status == "failed" else "",
+                           "title": app.model_title(info["model"]) if info else None, "port": info and info.get("port"),
+                           "speed": SPEED.get(info["model"], "") if info else "",
+                           # (the setting at most the model's longest: the setting itself applies from the next start)
+                           "context": app.context_of(info["model"], cfg) if info else None,
+                           "max_context": app.MAX_CONTEXT.get(info["model"]) if info else None,
+                           "mem_gb": app.memory_estimate(info["model"], cfg, self.ram) if info else None},
+                "save_dir": save, "folders": app.model_folders(cfg, existing=False)[1:], "models": models,
+                # (a start that failed from any path - this window, the tray, an update - until the next start / stop)
+                "busy": self.busy or (T("ロードできません: {0}", app.LAST_ERROR[0]) if app.LAST_ERROR[0] else ""),
+                "busy_err": self.busy_err or bool(not self.busy and app.LAST_ERROR[0]),
+                "log": os.path.join(app.config_dir(), "server.log"), "store_python": app.store_python(),
+                "deps_error": self._deps_error(), "usage": self._usage(info),
+                "update": {**self.upd, "can_update": self._can_update(), "current": __version__}}
+
+    @staticmethod
+    def _usage(info):
+        """Memory (onw's server, the whole machine) and the NPU's load for the server tab (onw.usage)."""
+        try:
+            from . import usage
+            return usage.sample(info["pid"] if info else None)
+        except Exception:
+            return None
+
+    @staticmethod
+    def _deps_error():
+        """The packages of this version could not be installed (pip failed in the update's restart): its error."""
+        from . import deps, update
+        d = update.install_dir()
+        return deps.failed(d) if d and not deps.current(d) else None
+
+    @staticmethod
+    def _state_of(key, path, info, status, dl):
+        if info and info.get("model") == key:
+            return "live" if status == "running" else ("loading" if status == "loading" else "failed")
+        if dl and dl["repo"] == key and not dl.get("done"):   # (a finished download is just there: ready)
+            return "downloading"
+        if path:
+            return "ready"
+        return "missing"
+
+    @staticmethod
+    def _cache_gb(path):
+        c = os.path.join(path, "npu_cache")
+        if not os.path.isdir(c):
+            return 0
+        n = 0
+        for r, _, fs in os.walk(c):
+            for f in fs:
+                try:
+                    n += os.path.getsize(os.path.join(r, f))
+                except OSError:
+                    pass
+        return round(n / 2**30, 1)
+
+    # ---------------------------------------------------------------- downloads (a child process per download)
+    def download(self, repo):
+        if repo not in app.CATALOG_REPOS:
+            return {"error": T("そのモデルはありません")}
+        if repo in app.NF4_ONLY and app.npu_arch() < 4000:    # (NF4 and 2-bit weights)
+            return {"error": T("このモデルは Lunar Lake 以降の NPU 専用です")}
+        info = app.server_info()
+        if info and info.get("model") == repo:            # updating it: its files are open in the server
+            return {"error": T("このモデルはロード中です。先にアンロードしてください。")}
+        if self.upd["state"] == "updating":
+            return {"error": T("onw の更新中です。終わってからダウンロードしてください")}
+        with self.lock:
+            if self.dl and self.dl["proc"].poll() is None:
+                return {"error": T("別のダウンロードが進行中です")}
+        dest = os.path.join(app.models_dir(app.load_config()), repo.split("/")[-1])
+        try:                                               # (the network call outside the lock: cancel stays instant)
+            from huggingface_hub import HfApi
+            total = _to_fetch(HfApi().model_info(repo, files_metadata=True).siblings, dest) or 1
+        except Exception as e:
+            return {"error": T("Hugging Face に接続できません: {0}", e)}
+        with self.lock:
+            if self.dl and self.dl["proc"].poll() is None:
+                return {"error": T("別のダウンロードが進行中です")}
+            log = os.path.join(app.config_dir(), "download.log")
+            before = _files(dest)                          # (before the child writes anything)
+            kw = {"creationflags": subprocess.CREATE_NO_WINDOW} if os.name == "nt" else {}
+            with open(log, "w", encoding="utf8") as out:   # a file, not a pipe: a chatty download never blocks on it
+                proc = subprocess.Popen([sys.executable, "-m", "onw.dl", repo, dest], stdout=out, stderr=subprocess.STDOUT,
+                                        env={**os.environ, "PYTHONUTF8": "1"}, **kw)
+            self.dl = {"repo": repo, "proc": proc, "dest": dest, "total": total, "frac": 0.0, "text": T("準備中…"),
+                       "error": None, "done": None, "t0": time.time(), "log": log, "before": before}
+            app.ACTIVE_DOWNLOAD[0] = dest                  # (an update: not loaded while its files are replaced)
+            return {}
+
+    def cancel_download(self):
+        with self.lock:
+            if self.dl and self.dl["proc"].poll() is None:
+                self.dl["proc"].terminate()
+                try:
+                    self.dl["proc"].wait(10)
+                except subprocess.TimeoutExpired:
+                    self.dl["proc"].kill()
+                    try:
+                        self.dl["proc"].wait(5)
+                    except subprocess.TimeoutExpired:
+                        pass
+                self._update_stopped(self.dl)
+            self.dl = None
+            app.ACTIVE_DOWNLOAD[0] = None
+        return {}
+
+    @staticmethod
+    def _update_stopped(dl):
+        """An update of a complete model cancelled or failed: when it replaced no file yet, the model is just as it
+        was - the marker goes (the hub's partial files stay in .cache for the next try). Otherwise the folder stays
+        marked: listed as an update to finish (app.update_interrupted), never as a partial download to delete."""
+        d, before = dl["dest"], dl["before"]
+        mark = os.path.join(d, ".downloading")
+        if not app.downloaded_by_onw(d) or mark in before:    # (a first download / one stopped before this try)
+            return
+        skip = (os.path.join(d, ".cache") + os.sep, mark)
+        mine = lambda files: {p: st for p, st in files.items() if not p.startswith(skip)}
+        if mine(_files(d)) == mine(before):
+            try:
+                os.remove(mark)
+            except OSError:
+                pass
+
+    def _poll_download(self):
+        dl = self.dl
+        if not dl or dl.get("done"):
+            return
+        before = dl["before"]                               # only what this download wrote (an update keeps the rest)
+        n = sum(st[0] for p, st in _files(dl["dest"]).items() if before.get(p) != st)
+        dl["frac"] = min(n / dl["total"], 0.99)
+        dl["text"] = f"{n / 2**30:.1f} / {dl['total'] / 2**30:.1f} GB ({dl['frac'] * 100:.0f}%)"
+        rc = dl["proc"].poll()
+        if rc is not None:
+            dl["done"] = dl["repo"]
+            app.ACTIVE_DOWNLOAD[0] = None
+            if rc != 0:
+                try:
+                    out = open(dl["log"], encoding="utf8", errors="replace").read()[-4000:]
+                except OSError:
+                    out = ""
+                dl["error"] = " ".join(out.split()[-30:]) or f"exit code {rc}"
+                self._update_stopped(dl)
+            else:
+                dl["frac"] = 1.0
+                with app._CFG_LOCK:
+                    if not app.load_config().get("model"):
+                        app.update_config(model=dl["repo"])
+
+    # ---------------------------------------------------------------- server
+    def load(self, key):
+        if key not in app.installed(app.load_config()):
+            return {"error": T("そのモデルはありません")}
+        with self.lock:                                    # (a second click while the first load is still starting)
+            if self.busy and not self.busy_err:
+                return {"error": T("ロードの準備中です。少しお待ちください")}
+            self.busy, self.busy_err = T("ロードしています…"), False
+        try:
+            cfg = app.update_config(model=key)
+        except Exception:
+            self.busy = ""
+            raise
+
+        def work():
+            try:
+                app.start_server(cfg)
+            except Exception:                          # (the reason: app.LAST_ERROR, shown by state())
+                pass
+            self.busy, self.busy_err = "", False
+        threading.Thread(target=work, daemon=True).start()
+        return {}
+
+    def unload(self):
+        threading.Thread(target=app.stop_server, daemon=True).start()
+        return {}
+
+    def restart(self):
+        info = app.server_info()
+        return self.load(info["model"]) if info else {}
+
+    # ---------------------------------------------------------------- models
+    def delete(self, key):
+        cfg = app.load_config()
+        if key.startswith("partial:"):                     # a download that stopped part-way
+            p = app.partial_downloads(cfg).get(key[8:])
+            if not p or (self.dl and not self.dl.get("done") and os.path.normcase(self.dl["dest"]) == os.path.normcase(p)):
+                return {"error": T("そのモデルはありません")}
+            rmtree(p)
+            return {}
+        inst = app.installed(cfg)
+        info = app.server_info()
+        if key not in inst:
+            return {"error": T("そのモデルはありません")}
+        if info and info.get("model") == key:
+            return {"error": T("このモデルはロード中です。先にアンロードしてください。")}
+        if not app.downloaded_by_onw(inst[key]):
+            return {"error": T("onw がダウンロードしたモデルではないので、ここからは削除しません。")}
+        rmtree(inst[key])                                  # (the compile cache inside: read-only files)
+        return {}
+
+    def clear_cache(self, key):
+        cfg = app.load_config()
+        inst = app.installed(cfg)
+        info = app.server_info()
+        if key not in inst:
+            return {"error": T("そのモデルはありません")}
+        if info and info.get("model") == key:
+            return {"error": T("このモデルはロード中です。先にアンロードしてください。")}
+        rmtree(os.path.join(inst[key], "npu_cache"))
+        return {}
+
+    def folders(self, save_dir=None, add=None, remove=None):
+        with app._CFG_LOCK:
+            return self._folders(save_dir, add, remove)
+
+    def _folders(self, save_dir=None, add=None, remove=None):
+        cfg = app.load_config()
+        nc = os.path.normcase
+        if save_dir:
+            old, new = app.models_dir(cfg), os.path.abspath(save_dir)
+            try:
+                os.makedirs(new, exist_ok=True)
+            except OSError as e:
+                return {"error": T("そのフォルダは使えません: {0}", e)}
+            extra = [x for x in cfg.get("model_folders") or [] if nc(x) != nc(new)]
+            if nc(old) != nc(new) and app.installed({**cfg, "model_folders": []}):
+                extra.append(old)                          # keep finding the models already downloaded to the old place
+            cfg.update(models_dir=new, model_folders=extra)
+        if add:
+            d = os.path.abspath(add)
+            if not os.path.isdir(d):
+                return {"error": T("フォルダが見つかりません: {0}", d)}
+            if nc(d) not in map(nc, app.model_folders(cfg)):
+                cfg["model_folders"] = (cfg.get("model_folders") or []) + [d]
+        if remove:
+            cfg["model_folders"] = [x for x in cfg.get("model_folders") or [] if nc(x) != nc(os.path.abspath(remove))]
+        app.save_config(cfg)
+        return {}
+
+    def settings(self, new):
+        with app._CFG_LOCK:
+            return self._settings(new)
+
+    def model_prefs(self, key, prefs):
+        """A model's defaults for the API and the chat page (the server reads them per request: no restart):
+        think off | low | medium | high, sampling greedy | recommended | custom and the custom values."""
+        if not key:
+            return {"error": T("モデルが選ばれていません")}
+        p = {}
+        if prefs.get("think") in ("off", "low", "medium", "high"):
+            p["think"] = prefs["think"]
+        if prefs.get("sampling") in ("greedy", "recommended", "custom"):
+            p["sampling"] = prefs["sampling"]
+        try:
+            for k, cast, lo, hi in (("temperature", float, 0, 2), ("top_p", float, 0.01, 1), ("top_k", int, 0, 1000),
+                                    ("presence_penalty", float, 0, 2)):
+                if prefs.get(k) not in (None, ""):
+                    p[k] = min(max(cast(float(prefs[k])), lo), hi)
+        except (TypeError, ValueError):
+            return {"error": T("生成設定の値が不正です")}
+        with app._CFG_LOCK:
+            cfg = app.load_config()
+            allp = dict(cfg.get("model_prefs") or {})
+            allp[key] = p
+            cfg["model_prefs"] = allp
+            app.save_config(cfg)
+        return {"ok": True}
+
+    def _settings(self, new):
+        cfg = app.load_config()
+        try:
+            port, ctx = int(new.get("port", cfg["port"])), int(new.get("context", cfg["context"]))
+            if not (1 <= port <= 65535) or ctx < 256:
+                raise ValueError
+        except (TypeError, ValueError):
+            return {"error": T("ポートかコンテキスト長の値が不正です")}
+        from .i18n import LANGS
+        if new.get("lang", "auto") not in ("auto", *LANGS):
+            new = {k: v for k, v in new.items() if k != "lang"}
+        old = dict(cfg)
+        for k in ("lan", "api_key", "device", "pld", "npu_turbo", "big_blocks", "npuw", "expert_gb", "autostart_server", "login_start",
+                  "update_check", "lang", "npuw_moe", "npuw_async", "cache_size", "mem_saver", "mem_saver_strict", "think_default"):
+            if k in new:
+                cfg[k] = new[k]
+        if new.get("api_sampling") in ("greedy", "recommended", "custom"):
+            cfg["api_sampling"] = new["api_sampling"]
+        try:
+            for k, cast, lo, hi in (("api_temperature", float, 0, 2), ("api_top_p", float, 0.01, 1), ("api_top_k", int, 0, 1000),
+                                    ("api_presence", float, 0, 2)):
+                if k in new:
+                    cfg[k] = min(max(cast(float(new[k])), lo), hi)
+        except (TypeError, ValueError):
+            return {"error": T("生成設定の値が不正です")}
+        if "api_max_tokens" in new:                            # (0: up to the end of the context)
+            try:
+                cfg["api_max_tokens"] = max(0, int(new["api_max_tokens"]))
+            except (TypeError, ValueError):
+                return {"error": T("生成設定の値が不正です")}
+        if ctx != cfg["context"]:
+            cfg["context_chosen"] = True                       # (the old 1024 default moves up; a choice stays)
+        cfg["port"], cfg["context"] = port, ctx
+        server_keys = ("lan", "api_key", "device", "pld", "npu_turbo", "api_max_tokens", "big_blocks", "npuw", "expert_gb", "port", "context",
+                       "npuw_moe", "npuw_async", "cache_size", "mem_saver", "mem_saver_strict", "think_default",
+                       "api_sampling", "api_temperature", "api_top_p", "api_top_k", "api_presence")
+        app.save_config(cfg)
+        i18n.current(cfg)                                  # messages switch language right away
+        try:
+            app.set_login_start(bool(cfg["login_start"]))
+        except OSError:
+            pass
+        changed = any(str(old.get(k)) != str(cfg.get(k)) for k in server_keys)
+        return {"restart_needed": changed and app.server_info() is not None}
+
+    # ---------------------------------------------------------------- updates of onw itself
+    @staticmethod
+    def _can_update():
+        from . import update
+        return update.can_update()
+
+    def _update_loop(self):
+        time.sleep(30)                                     # let the tray and the server start first
+        while True:
+            if app.load_config().get("update_check", True):
+                self.check_update()
+            time.sleep(24 * 3600)
+
+    def start_check(self):
+        """The window's "Check now": checks in the background and answers at once; the page follows
+        upd["checking"] / upd["error"] (a slow connection must not leave the button without an answer)."""
+        if not self.upd["checking"]:
+            threading.Thread(target=self.check_update, daemon=True).start()
+        return {"checking": True}
+
+    def check_update(self):
+        self.upd.update(checking=True, error="")
+        try:
+            r = self._check_update()
+        except Exception as e:
+            r = {"error": T("更新を確認できませんでした: {0}", e)}
+        self.upd.update(checking=False, error=r.get("error", ""))
+        return r
+
+    def _check_update(self):
+        from . import update
+        found = update.latest()
+        self.upd["checked"] = time.time()
+        if found is None:
+            return {"error": T("更新を確認できませんでした（オフライン？）")}
+        v, tag = found
+        if update.newer(v):
+            fresh = self.upd["latest"] != v
+            self.upd.update(latest=v, tag=tag, notes=update.notes(lang=i18n.current()))
+            if fresh and self.on_update_found:
+                try:
+                    self.on_update_found(v)
+                except Exception:
+                    pass
+        else:
+            self.upd.update(latest=None, tag=None, notes="")
+        return {"latest": self.upd["latest"]}
+
+    def run_update(self):
+        from . import update
+        if not self.upd["tag"]:
+            return {"error": T("新しい版はありません")}
+        if not update.can_update():
+            return {"error": T("このフォルダの onw は 1 行インストーラーで入れたものではないので、自動更新できません")}
+        if self.upd["state"] == "updating":
+            return {}
+        if self.dl and self.dl["proc"].poll() is None:   # the restart would orphan the download
+            return {"error": T("ダウンロードが終わってから更新してください")}
+        self.upd.update(state="updating", msg=T("準備中…"))
+
+        def work():
+            try:
+                v = update.apply(self.upd["tag"], progress=lambda t: self.upd.update(msg=t))
+                self.upd.update(state="done", msg=T("v{0} に更新しました。再起動します…", v))
+                if self.on_updated:
+                    self.on_updated()
+                else:
+                    self.upd["msg"] = T("v{0} に更新しました。onw を起動し直してください", v)
+            except Exception as e:
+                self.upd.update(state="error", msg=T("更新できませんでした: {0}", e))
+        threading.Thread(target=work, daemon=True).start()
+        return {}
+
+    def open(self, what, key=None):
+        cfg = app.load_config()
+        if what == "log":
+            app.open_path(os.path.join(app.config_dir(), "server.log"))
+        elif what == "folder":
+            p = app.installed(cfg).get(key) if key else None
+            app.open_path(p or app.models_dir(cfg))
+        elif what in ("chat", "check"):
+            webbrowser.open(f"http://127.0.0.1:{app.server_port(cfg)}/{what}")
+        return {}
+
+
+class _Handler(BaseHTTPRequestHandler):
+    manager: Manager = None
+    token: str = ""
+
+    def log_message(self, *a):                             # quiet
+        pass
+
+    def _send(self, code, body, ctype="application/json"):
+        data = body if isinstance(body, bytes) else json.dumps(body, ensure_ascii=False).encode("utf8")
+        self.send_response(code)
+        self.send_header("Content-Type", ctype + ("; charset=utf-8" if ctype.startswith("text") else ""))
+        self.send_header("Content-Length", str(len(data)))
+        self.send_header("Cache-Control", "no-store")
+        self.end_headers()
+        self.wfile.write(data)
+
+    def _authed(self, q):
+        t = self.headers.get("X-Onw-Token") or (q.get("token") or [""])[0]
+        return secrets.compare_digest(t, self.token)
+
+    def do_GET(self):
+        u = urlparse(self.path)
+        q = parse_qs(u.query)
+        if u.path in ("/", "/manage.html"):
+            return self._send(200, i18n.page(os.path.join(WEB, "manage.html")), "text/html")
+        if u.path == "/api/state":
+            if not self._authed(q):
+                return self._send(403, {"error": T("この画面は onw のウィンドウかトレイから開いてください")})
+            return self._send(200, self.manager.state())
+        self._send(404, {"error": "not found"})
+
+    def do_POST(self):
+        u = urlparse(self.path)
+        if not self._authed(parse_qs(u.query)):
+            return self._send(403, {"error": T("この画面は onw のウィンドウかトレイから開いてください")})
+        m = self.manager
+        act = u.path[len("/api/"):]
+        try:
+            n = int(self.headers.get("Content-Length") or 0)
+            body = json.loads(self.rfile.read(n) or b"{}")
+            if act == "load":
+                r = m.load(body["key"])
+            elif act == "unload":
+                r = m.unload()
+            elif act == "restart":
+                r = m.restart()
+            elif act == "download":
+                r = m.download(body["repo"])
+            elif act == "cancel_download":
+                r = m.cancel_download()
+            elif act == "delete":
+                r = m.delete(body["key"])
+            elif act == "clear_cache":
+                r = m.clear_cache(body["key"])
+            elif act == "folders":
+                r = m.folders(body.get("save_dir"), body.get("add"), body.get("remove"))
+            elif act == "settings":
+                r = m.settings(body)
+            elif act == "model_prefs":
+                r = m.model_prefs(body.get("key"), body.get("prefs") or {})
+            elif act == "update_check":
+                r = m.start_check()
+            elif act == "update":
+                r = m.run_update()
+            elif act == "open":
+                r = m.open(body.get("what"), body.get("key"))
+            else:
+                return self._send(404, {"error": "not found"})
+        except Exception as e:
+            r = {"error": T("エラー: {0}", e)}
+        self._send(200, r)
+
+
+def _created():
+    try:
+        import psutil
+        return psutil.Process().create_time()
+    except Exception:
+        return None
+
+
+def forget(info):
+    """Remove manager.json if it still describes this process's manager (at exit)."""
+    try:
+        if json.load(open(INFO())).get("pid") == info.get("pid"):
+            os.remove(INFO())
+    except (OSError, ValueError):
+        pass
+
+
+def serve(manager=None):
+    """Start the manager's HTTP server in a daemon thread; returns {port, token}. Writes <config>/manager.json."""
+    manager = manager or Manager()
+    handler = type("H", (_Handler,), {"manager": manager, "token": secrets.token_hex(16)})
+    srv = ThreadingHTTPServer(("127.0.0.1", 0), handler)
+    threading.Thread(target=srv.serve_forever, daemon=True).start()
+    info = {"port": srv.server_address[1], "token": handler.token, "pid": os.getpid(), "created": _created()}
+    with open(INFO() + ".tmp", "w") as f:
+        json.dump(info, f)
+    try:
+        os.chmod(INFO() + ".tmp", 0o600)                   # (the token) before it is in place
+    except OSError:
+        pass
+    os.replace(INFO() + ".tmp", INFO())
+    return info
+
+
+def page_url(info, tab="server"):
+    return f"http://127.0.0.1:{info['port']}/manage.html?token={info['token']}#{tab}"

onw/models/gemma4.py

@@ -1,135 +1,136 @@
-"""Gemma 4 (google/gemma-4-E4B-it) from the proven NPU graphs of the ryugyosoft/gemma-4-E4B-it-npu repo.
-
-Gemma 4 needs several NPU-specific graph rewrites (head_dim-512 attention split into 256-wide heads, host-built
-sliding / full masks, static vision tower); those graphs already exist and are validated, so this builder imports
-them instead of rebuilding from safetensors:
-  * decoder_S{1,16,64}_T1024.xml -> segment 0 of each block size (I/O renamed to the engine's conventions);
-  * a new segment 1 per block size: the LM head as a shared INT8 input (one copy for all sizes) + softcapping;
-  * the token embedding and the 2.7 GB per-layer embedding tables are pulled out of their OpenVINO IRs into
-    shared.bin and looked up on the host (they are pure table reads) - no NPU memory, no worker process;
-  * vision_split/vision_npu_280.xml is used as the vision tower.
-usage: onw convert <folder of ryugyosoft/gemma-4-E4B-it-npu> OUT_DIR
-"""
-import json, os, shutil
-import numpy as np
-import openvino as ov
-import openvino.opset13 as ops
-
-from .. import ir
-from ..ir import i64, f32c
-from . import common
-
-EMB = "models/gemma-4-E4B-it-int4-ov"
-
-
-def _table(core, xml):
-    """(int8 table, f16 per-row scale, output multiplier, gather-index subgraph output) from an embedding IR."""
-    m = core.read_model(xml)
-    tab = sc = mult = gather = None
-    for op in m.get_ordered_ops():
-        t = op.get_type_name()
-        if t == "Constant" and op.get_output_element_type(0) == ov.Type.i8:
-            tab = op.get_data().copy()
-        elif t == "Constant" and op.get_output_element_type(0) == ov.Type.f16 and len(op.get_output_shape(0)) == 2:
-            sc = op.get_data().reshape(-1).copy()
-        elif t == "Constant" and op.get_output_element_type(0) == ov.Type.f32 and list(op.get_output_shape(0)) == [1, 1, 1]:
-            mult = float(op.get_data().reshape(-1)[0])
-        elif t == "Gather":
-            gather = op
-    return m, tab, sc, mult, gather
-
-
-def _id_map(core, m, gather, V):
-    """The per-layer model remaps some token ids before its gather; evaluate that index for every id once."""
-    idx = gather.input(1).get_source_output()
-    sub = ov.Model([ops.result(idx)], m.get_parameters(), "idmap")
-    r = core.compile_model(sub, "CPU").create_infer_request()
-    return r.infer([np.arange(V, dtype=np.int64)[None]])[0].reshape(-1).astype(np.int32)
-
-
-def _rename(m):
-    for p in m.get_parameters():
-        n = p.get_output_tensor(0).get_any_name()
-        if n.startswith("past_key_values."):
-            _, l, kv = n.split(".")
-            p.get_output_tensor(0).set_names({f"past_{kv[0]}.{l}"})
-    for r in m.get_results():                          # names live on the result's tensor and on its source
-        tensors = [r.output(0).get_tensor(), r.input(0).get_source_output().get_tensor()]
-        n = tensors[0].get_any_name()
-        new = None
-        if n.startswith("present."):
-            _, l, kv = n.split(".")
-            new = f"new_{kv[0]}.{l}"
-        elif n == "hidden":
-            new = "next.h"
-        if new:
-            for t in tensors:
-                t.set_names({new})
-    return m
-
-
-def _head(S, V, H, cap):
-    """LM head + final logit softcapping cap * tanh(logits / cap). The 1/cap is applied to the hidden state before
-    the matmul: with it after the matmul the NPU compiler miscomputes multi-row blocks (NPU 3720, S=16: 99% error)."""
-    h = ops.parameter([1, S, H], ov.Type.f32, name="carry.h")
-    lg, p = ir.shared_head(ops.multiply(h, f32c(1.0 / cap)), V, H)
-    lg = ops.multiply(ops.tanh(lg), f32c(cap))
-    return ir.finish({"logits": lg}, [h] + p, f"gemma4_head_S{S}")
-
-
-def export(src, out, sizes=(1, 16, 64), T=1024, fmt=None, graphs_only=False):
-    from ..store import SharedWriter
-    os.makedirs(out, exist_ok=True)
-    core = ov.Core()
-    full = json.load(open(os.path.join(src, EMB, "config.json")))
-    tc = full["text_config"]
-    V, H = tc["vocab_size"], tc["hidden_size"]
-    metas = {}
-    for S in sizes:
-        xml = os.path.join(src, "models", "lm_static", f"decoder_S{S}_T{T}.xml")
-        if not os.path.exists(xml):
-            continue
-        ov.save_model(_rename(core.read_model(xml)), os.path.join(out, f"seg00_S{S}.xml"))
-        ov.save_model(_head(S, V, H, tc.get("final_logit_softcapping") or 30.0), os.path.join(out, f"seg01_S{S}.xml"))
-        for i in (0, 1):
-            metas[f"seg{i:02d}_S{S}.xml"] = {"index": i, "consume_moe": None, "router_layer": None, "S": S, "slots": 0,
-                                             "head": i == 1}
-        print(f"S={S}: segments saved", flush=True)
-    if not metas:
-        import glob
-        have = sorted(os.path.basename(p) for p in glob.glob(os.path.join(src, "models", "lm_static", "decoder_S*_T*.xml")))
-        raise SystemExit(f"no decoder graphs for T={T} and sizes {list(sizes)} in {src} (there are: {', '.join(have)}); "
-                         "Gemma 4 is imported from those pre-built graphs")
-    vx = os.path.join(src, "models", "vision_split", "vision_npu_280.xml")
-    shutil.copy(vx, os.path.join(out, "vision.xml"))
-    shutil.copy(vx[:-4] + ".bin", os.path.join(out, "vision.bin"))
-    pc = json.load(open(os.path.join(src, EMB, "processor_config.json")))
-    meta = {"arch": "gemma4", "T": T, "sizes": [S for S in sizes if f"seg00_S{S}.xml" in metas], "config": tc,
-            "segments": metas, "masks": {"sliding_window": tc["sliding_window"]},
-            "host_inputs": {"per_layer_inputs": {"table": "ple", "shape": [tc["num_hidden_layers"], tc["hidden_size_per_layer_input"]]}},
-            "vision": {"type": "gemma", "soft_tokens": 280, "image_processor": pc["image_processor"],
-                       "image_token_id": full["image_token_id"], "pad_token_id": tc.get("pad_token_id", 0)},
-            "npu_vision_fallback": {"NPU_USE_NPUW": "YES", "NPUW_DEVICES": "NPU", "NPUW_FOLD": "YES"}}
-    for f in ("tokenizer.json", "tokenizer_config.json", "chat_template.jinja", "config.json", "generation_config.json",
-              "processor_config.json", "preprocessor_config.json"):
-        if os.path.exists(os.path.join(src, EMB, f)):
-            shutil.copy(os.path.join(src, EMB, f), os.path.join(out, f))
-    if graphs_only:
-        common.write_engine(out, meta, graphs_only)
-        return
-    sw = SharedWriter(os.path.join(out, "shared.bin"))
-    m, tab, sc, mult, _ = _table(core, os.path.join(src, EMB, "openvino_text_embeddings_model.xml"))
-    sw.add("embed_q", tab, "i8", tab.shape)
-    sw.add("embed_s", (sc.astype(np.float32) * mult).astype(np.float16), "f16", sc.shape)
-    sw.alias("head_q", "embed_q")                                               # tied: same int8 rows,
-    sw.add("head_s", sc.astype(np.float16), "f16", sc.shape)                    # without the embedding scale
-    del tab
-    ple_xml = os.path.join(src, EMB, "openvino_text_embeddings_per_layer_model.xml")
-    m, tab, sc, mult, gather = _table(core, ple_xml)
-    sw.add("ple_q", tab, "i8", tab.shape)
-    sw.add("ple_s", (sc.astype(np.float32) * mult).astype(np.float16), "f16", sc.shape)
-    idmap = _id_map(core, m, gather, tab.shape[0])
-    sw.add("ple_map", idmap, "i32", idmap.shape)
-    sw.close()
-    print("shared tables saved", flush=True)
-    common.write_engine(out, meta, graphs_only)
+"""Gemma 4 (google/gemma-4-E4B-it) from the proven NPU graphs of the ryugyosoft/gemma-4-E4B-it-npu repo.
+
+Gemma 4 needs several NPU-specific graph rewrites (head_dim-512 attention split into 256-wide heads, host-built
+sliding / full masks, static vision tower); those graphs already exist and are validated, so this builder imports
+them instead of rebuilding from safetensors:
+  * decoder_S{1,16,64}_T1024.xml -> segment 0 of each block size (I/O renamed to the engine's conventions);
+  * a new segment 1 per block size: the LM head as a shared INT8 input (one copy for all sizes) + softcapping;
+  * the token embedding and the 2.7 GB per-layer embedding tables are pulled out of their OpenVINO IRs into
+    shared.bin and looked up on the host (they are pure table reads) - no NPU memory, no worker process;
+  * vision_split/vision_npu_280.xml is used as the vision tower.
+usage: onw convert <folder of ryugyosoft/gemma-4-E4B-it-npu> OUT_DIR
+"""
+import json, os, shutil
+import numpy as np
+import openvino as ov
+from .. import npukit
+import openvino.opset13 as ops
+
+from .. import ir
+from ..ir import i64, f32c
+from . import common
+
+EMB = "models/gemma-4-E4B-it-int4-ov"
+
+
+def _table(core, xml):
+    """(int8 table, f16 per-row scale, output multiplier, gather-index subgraph output) from an embedding IR."""
+    m = core.read_model(xml)
+    tab = sc = mult = gather = None
+    for op in m.get_ordered_ops():
+        t = op.get_type_name()
+        if t == "Constant" and op.get_output_element_type(0) == ov.Type.i8:
+            tab = op.get_data().copy()
+        elif t == "Constant" and op.get_output_element_type(0) == ov.Type.f16 and len(op.get_output_shape(0)) == 2:
+            sc = op.get_data().reshape(-1).copy()
+        elif t == "Constant" and op.get_output_element_type(0) == ov.Type.f32 and list(op.get_output_shape(0)) == [1, 1, 1]:
+            mult = float(op.get_data().reshape(-1)[0])
+        elif t == "Gather":
+            gather = op
+    return m, tab, sc, mult, gather
+
+
+def _id_map(core, m, gather, V):
+    """The per-layer model remaps some token ids before its gather; evaluate that index for every id once."""
+    idx = gather.input(1).get_source_output()
+    sub = ov.Model([ops.result(idx)], m.get_parameters(), "idmap")
+    r = core.compile_model(sub, "CPU").create_infer_request()
+    return r.infer([np.arange(V, dtype=np.int64)[None]])[0].reshape(-1).astype(np.int32)
+
+
+def _rename(m):
+    for p in m.get_parameters():
+        n = p.get_output_tensor(0).get_any_name()
+        if n.startswith("past_key_values."):
+            _, l, kv = n.split(".")
+            p.get_output_tensor(0).set_names({f"past_{kv[0]}.{l}"})
+    for r in m.get_results():                          # names live on the result's tensor and on its source
+        tensors = [r.output(0).get_tensor(), r.input(0).get_source_output().get_tensor()]
+        n = tensors[0].get_any_name()
+        new = None
+        if n.startswith("present."):
+            _, l, kv = n.split(".")
+            new = f"new_{kv[0]}.{l}"
+        elif n == "hidden":
+            new = "next.h"
+        if new:
+            for t in tensors:
+                t.set_names({new})
+    return m
+
+
+def _head(S, V, H, cap):
+    """LM head + final logit softcapping cap * tanh(logits / cap). The 1/cap is applied to the hidden state before
+    the matmul: with it after the matmul the NPU compiler miscomputes multi-row blocks (NPU 3720, S=16: 99% error)."""
+    h = ops.parameter([1, S, H], ov.Type.f32, name="carry.h")
+    lg, p = ir.shared_head(ops.multiply(h, f32c(1.0 / cap)), V, H)
+    lg = ops.multiply(ops.tanh(lg), f32c(cap))
+    return ir.finish({"logits": lg}, [h] + p, f"gemma4_head_S{S}")
+
+
+def export(src, out, sizes=(1, 16, 64), T=1024, fmt=None, graphs_only=False):
+    from ..store import SharedWriter
+    os.makedirs(out, exist_ok=True)
+    core = npukit.core()
+    full = json.load(open(os.path.join(src, EMB, "config.json")))
+    tc = full["text_config"]
+    V, H = tc["vocab_size"], tc["hidden_size"]
+    metas = {}
+    for S in sizes:
+        xml = os.path.join(src, "models", "lm_static", f"decoder_S{S}_T{T}.xml")
+        if not os.path.exists(xml):
+            continue
+        ov.save_model(_rename(core.read_model(xml)), os.path.join(out, f"seg00_S{S}.xml"))
+        ov.save_model(_head(S, V, H, tc.get("final_logit_softcapping") or 30.0), os.path.join(out, f"seg01_S{S}.xml"))
+        for i in (0, 1):
+            metas[f"seg{i:02d}_S{S}.xml"] = {"index": i, "consume_moe": None, "router_layer": None, "S": S, "slots": 0,
+                                             "head": i == 1}
+        print(f"S={S}: segments saved", flush=True)
+    if not metas:
+        import glob
+        have = sorted(os.path.basename(p) for p in glob.glob(os.path.join(src, "models", "lm_static", "decoder_S*_T*.xml")))
+        raise SystemExit(f"no decoder graphs for T={T} and sizes {list(sizes)} in {src} (there are: {', '.join(have)}); "
+                         "Gemma 4 is imported from those pre-built graphs")
+    vx = os.path.join(src, "models", "vision_split", "vision_npu_280.xml")
+    shutil.copy(vx, os.path.join(out, "vision.xml"))
+    shutil.copy(vx[:-4] + ".bin", os.path.join(out, "vision.bin"))
+    pc = json.load(open(os.path.join(src, EMB, "processor_config.json")))
+    meta = {"arch": "gemma4", "T": T, "sizes": [S for S in sizes if f"seg00_S{S}.xml" in metas], "config": tc,
+            "segments": metas, "masks": {"sliding_window": tc["sliding_window"]},
+            "host_inputs": {"per_layer_inputs": {"table": "ple", "shape": [tc["num_hidden_layers"], tc["hidden_size_per_layer_input"]]}},
+            "vision": {"type": "gemma", "soft_tokens": 280, "image_processor": pc["image_processor"],
+                       "image_token_id": full["image_token_id"], "pad_token_id": tc.get("pad_token_id", 0)},
+            "npu_vision_fallback": {"NPU_USE_NPUW": "YES", "NPUW_DEVICES": "NPU", "NPUW_FOLD": "YES"}}
+    for f in ("tokenizer.json", "tokenizer_config.json", "chat_template.jinja", "config.json", "generation_config.json",
+              "processor_config.json", "preprocessor_config.json"):
+        if os.path.exists(os.path.join(src, EMB, f)):
+            shutil.copy(os.path.join(src, EMB, f), os.path.join(out, f))
+    if graphs_only:
+        common.write_engine(out, meta, graphs_only)
+        return
+    sw = SharedWriter(os.path.join(out, "shared.bin"))
+    m, tab, sc, mult, _ = _table(core, os.path.join(src, EMB, "openvino_text_embeddings_model.xml"))
+    sw.add("embed_q", tab, "i8", tab.shape)
+    sw.add("embed_s", (sc.astype(np.float32) * mult).astype(np.float16), "f16", sc.shape)
+    sw.alias("head_q", "embed_q")                                               # tied: same int8 rows,
+    sw.add("head_s", sc.astype(np.float16), "f16", sc.shape)                    # without the embedding scale
+    del tab
+    ple_xml = os.path.join(src, EMB, "openvino_text_embeddings_per_layer_model.xml")
+    m, tab, sc, mult, gather = _table(core, ple_xml)
+    sw.add("ple_q", tab, "i8", tab.shape)
+    sw.add("ple_s", (sc.astype(np.float32) * mult).astype(np.float16), "f16", sc.shape)
+    idmap = _id_map(core, m, gather, tab.shape[0])
+    sw.add("ple_map", idmap, "i32", idmap.shape)
+    sw.close()
+    print("shared tables saved", flush=True)
+    common.write_engine(out, meta, graphs_only)

onw/npukit.py

@@ -0,0 +1,122 @@
+"""[3.0] onw's own NPU kit: an NPU compiler and an NPU plugin built for onw from Intel's open sources, for exactly the
+OpenVINO release onw installs (Windows and Linux). The kit is used through a plugins file given to ov.Core - the "NPU" device is
+then served by the kit's plugin, which loads the kit's compiler next to it; nothing of the installed OpenVINO is
+changed, and with any other OpenVINO build onw uses OpenVINO's own NPU plugin as before.
+
+What the kit adds (see TECHNICAL.md):
+- weightless blobs (CACHE_MODE=OPTIMIZE_SIZE + ENABLE_WEIGHTLESS): the compiled graphs keep no weights, the cache holds
+  the graphs only (NF4 too - OpenVINO 2026.5's compiler cannot); the weights are rebuilt on the NPU at load from the
+  model's own files;
+- one copy of identical weights for every graph of a model (paged parts, fused graphs of every length);
+- faster compiles (the strategy search no longer serialises on the Windows heap, constant folding memoised) and
+  compiler fixes: 2-bit weights on 2+ rows (Bonsai), 4-token MTP verification blocks (Qwen3.5 NF4).
+ONW_NPUKIT=0 turns it off."""
+import json
+import os
+import tarfile
+
+import openvino as ov
+
+KIT = os.path.join(os.path.dirname(os.path.abspath(__file__)), "npukit")
+# Linux: the kit comes packed (npukit/linux/npukit-linux.tar.xz, ~1/4 of its size - installs that update from an older
+# onw download every file once) and is unpacked next to it at the first start
+LINUX = os.path.join(KIT, "linux")
+_STATE = {"ok": None, "why": ""}
+
+
+def _info():
+    try:
+        with open(os.path.join(KIT, "kit.json"), encoding="utf8") as f:
+            return json.load(f)
+    except (OSError, ValueError):
+        return None
+
+
+def active():
+    """True when the kit serves the NPU in this process (checked once)."""
+    if _STATE["ok"] is None:
+        for n in os.listdir(KIT) if os.path.isdir(KIT) else []:
+            if ".dll.old-" in n:                      # (set aside by an update while a process had them loaded)
+                try:
+                    os.remove(os.path.join(KIT, n))
+                except OSError:
+                    pass
+        info = _info()
+        if os.environ.get("ONW_NPUKIT", "1") == "0":
+            _STATE.update(ok=False, why="off (ONW_NPUKIT=0)")
+        elif info is None:
+            _STATE.update(ok=False, why="not installed")
+        elif ov.__version__ != info.get("openvino"):
+            _STATE.update(ok=False, why=f"built for OpenVINO {info.get('openvino')}, not {ov.__version__}")
+        elif os.environ.get("ONW_NPUKIT") != "1" and _arch() < 4000:
+            # NPU 3720 (Meteor / Arrow Lake): nothing to gain - no graphs without weights there, and OpenVINO's own
+            # compiler builds ~7% faster graphs for it (Granite 4.2 3B INT4: 6.4 vs 6.0 tok/s, same answers)
+            _STATE.update(ok=False, why="not used on this NPU (ONW_NPUKIT=1 forces it)")
+        elif not _unpacked(info):
+            _STATE.update(ok=False, why="not installed")
+        else:
+            if os.name == "nt":
+                # the kit's plugin needs OpenVINO's own libraries (openvino.dll, TBB) from the installed package
+                os.add_dll_directory(os.path.join(os.path.dirname(ov.__file__), "libs"))
+            _STATE.update(ok=True, why=info.get("version", ""))
+    return _STATE["ok"]
+
+
+def _dir():
+    return KIT if os.name == "nt" else LINUX
+
+
+def _unpacked(info):
+    """All of the kit's files are there (Linux: unpacked from its archive when needed)."""
+    files = info.get("files" if os.name == "nt" else "files_linux", [])
+    if not files:
+        return False
+    arc = os.path.join(LINUX, info.get("archive_linux", "-"))
+    stamp = os.path.join(LINUX, "unpacked.txt")
+    if os.name != "nt" and os.path.isfile(arc):
+        mark = f"{os.path.getsize(arc)} {info.get('version', '')}"
+        try:
+            with open(stamp, encoding="utf8") as f:
+                done = f.read().strip() == mark
+        except OSError:
+            done = False
+        if done and all(os.path.exists(os.path.join(LINUX, f)) for f in files):
+            return True
+        try:
+            with tarfile.open(arc) as t:
+                for m in t.getmembers():
+                    if m.isfile() and os.path.basename(m.name) == m.name and m.name in files:
+                        tmp = os.path.join(LINUX, m.name + ".part")
+                        with t.extractfile(m) as src, open(tmp, "wb") as dst:
+                            while True:
+                                b = src.read(1 << 22)
+                                if not b:
+                                    break
+                                dst.write(b)
+                        os.replace(tmp, os.path.join(LINUX, m.name))
+            with open(stamp, "w", encoding="utf8") as f:
+                f.write(mark + "\n")
+        except (OSError, tarfile.TarError):
+            return False
+    return all(os.path.exists(os.path.join(_dir(), f)) for f in files)
+
+
+def _arch():
+    """NPU generation from OpenVINO's own plugin (3720, 4000, 5010, ...), 0 when there is no NPU."""
+    try:
+        a = str(ov.Core().get_property("NPU", "DEVICE_ARCHITECTURE"))
+        return int("".join(c for c in a if c.isdigit()) or 0)
+    except Exception:
+        return 0
+
+
+def why():
+    active()
+    return _STATE["why"]
+
+
+def core():
+    """An ov.Core whose NPU is the kit's when the kit is usable (else OpenVINO's own)."""
+    if active():
+        return ov.Core(os.path.join(_dir(), "plugins.xml"))
+    return ov.Core()

onw/npukit/LICENSE

@@ -0,0 +1,201 @@
+                                 Apache License
+                           Version 2.0, January 2004
+                        http://www.apache.org/licenses/
+
+   TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
+
+   1. Definitions.
+
+      "License" shall mean the terms and conditions for use, reproduction,
+      and distribution as defined by Sections 1 through 9 of this document.
+
+      "Licensor" shall mean the copyright owner or entity authorized by
+      the copyright owner that is granting the License.
+
+      "Legal Entity" shall mean the union of the acting entity and all
+      other entities that control, are controlled by, or are under common
+      control with that entity. For the purposes of this definition,
+      "control" means (i) the power, direct or indirect, to cause the
+      direction or management of such entity, whether by contract or
+      otherwise, or (ii) ownership of fifty percent (50%) or more of the
+      outstanding shares, or (iii) beneficial ownership of such entity.
+
+      "You" (or "Your") shall mean an individual or Legal Entity
+      exercising permissions granted by this License.
+
+      "Source" form shall mean the preferred form for making modifications,
+      including but not limited to software source code, documentation
+      source, and configuration files.
+
+      "Object" form shall mean any form resulting from mechanical
+      transformation or translation of a Source form, including but
+      not limited to compiled object code, generated documentation,
+      and conversions to other media types.
+
+      "Work" shall mean the work of authorship, whether in Source or
+      Object form, made available under the License, as indicated by a
+      copyright notice that is included in or attached to the work
+      (an example is provided in the Appendix below).
+
+      "Derivative Works" shall mean any work, whether in Source or Object
+      form, that is based on (or derived from) the Work and for which the
+      editorial revisions, annotations, elaborations, or other modifications
+      represent, as a whole, an original work of authorship. For the purposes
+      of this License, Derivative Works shall not include works that remain
+      separable from, or merely link (or bind by name) to the interfaces of,
+      the Work and Derivative Works thereof.
+
+      "Contribution" shall mean any work of authorship, including
+      the original version of the Work and any modifications or additions
+      to that Work or Derivative Works thereof, that is intentionally
+      submitted to Licensor for inclusion in the Work by the copyright owner
+      or by an individual or Legal Entity authorized to submit on behalf of
+      the copyright owner. For the purposes of this definition, "submitted"
+      means any form of electronic, verbal, or written communication sent
+      to the Licensor or its representatives, including but not limited to
+      communication on electronic mailing lists, source code control systems,
+      and issue tracking systems that are managed by, or on behalf of, the
+      Licensor for the purpose of discussing and improving the Work, but
+      excluding communication that is conspicuously marked or otherwise
+      designated in writing by the copyright owner as "Not a Contribution."
+
+      "Contributor" shall mean Licensor and any individual or Legal Entity
+      on behalf of whom a Contribution has been received by Licensor and
+      subsequently incorporated within the Work.
+
+   2. Grant of Copyright License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      copyright license to reproduce, prepare Derivative Works of,
+      publicly display, publicly perform, sublicense, and distribute the
+      Work and such Derivative Works in Source or Object form.
+
+   3. Grant of Patent License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      (except as stated in this section) patent license to make, have made,
+      use, offer to sell, sell, import, and otherwise transfer the Work,
+      where such license applies only to those patent claims licensable
+      by such Contributor that are necessarily infringed by their
+      Contribution(s) alone or by combination of their Contribution(s)
+      with the Work to which such Contribution(s) was submitted. If You
+      institute patent litigation against any entity (including a
+      cross-claim or counterclaim in a lawsuit) alleging that the Work
+      or a Contribution incorporated within the Work constitutes direct
+      or contributory patent infringement, then any patent licenses
+      granted to You under this License for that Work shall terminate
+      as of the date such litigation is filed.
+
+   4. Redistribution. You may reproduce and distribute copies of the
+      Work or Derivative Works thereof in any medium, with or without
+      modifications, and in Source or Object form, provided that You
+      meet the following conditions:
+
+      (a) You must give any other recipients of the Work or
+          Derivative Works a copy of this License; and
+
+      (b) You must cause any modified files to carry prominent notices
+          stating that You changed the files; and
+
+      (c) You must retain, in the Source form of any Derivative Works
+          that You distribute, all copyright, patent, trademark, and
+          attribution notices from the Source form of the Work,
+          excluding those notices that do not pertain to any part of
+          the Derivative Works; and
+
+      (d) If the Work includes a "NOTICE" text file as part of its
+          distribution, then any Derivative Works that You distribute must
+          include a readable copy of the attribution notices contained
+          within such NOTICE file, excluding those notices that do not
+          pertain to any part of the Derivative Works, in at least one
+          of the following places: within a NOTICE text file distributed
+          as part of the Derivative Works; within the Source form or
+          documentation, if provided along with the Derivative Works; or,
+          within a display generated by the Derivative Works, if and
+          wherever such third-party notices normally appear. The contents
+          of the NOTICE file are for informational purposes only and
+          do not modify the License. You may add Your own attribution
+          notices within Derivative Works that You distribute, alongside
+          or as an addendum to the NOTICE text from the Work, provided
+          that such additional attribution notices cannot be construed
+          as modifying the License.
+
+      You may add Your own copyright statement to Your modifications and
+      may provide additional or different license terms and conditions
+      for use, reproduction, or distribution of Your modifications, or
+      for any such Derivative Works as a whole, provided Your use,
+      reproduction, and distribution of the Work otherwise complies with
+      the conditions stated in this License.
+
+   5. Submission of Contributions. Unless You explicitly state otherwise,
+      any Contribution intentionally submitted for inclusion in the Work
+      by You to the Licensor shall be under the terms and conditions of
+      this License, without any additional terms or conditions.
+      Notwithstanding the above, nothing herein shall supersede or modify
+      the terms of any separate license agreement you may have executed
+      with Licensor regarding such Contributions.
+
+   6. Trademarks. This License does not grant permission to use the trade
+      names, trademarks, service marks, or product names of the Licensor,
+      except as required for reasonable and customary use in describing the
+      origin of the Work and reproducing the content of the NOTICE file.
+
+   7. Disclaimer of Warranty. Unless required by applicable law or
+      agreed to in writing, Licensor provides the Work (and each
+      Contributor provides its Contributions) on an "AS IS" BASIS,
+      WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+      implied, including, without limitation, any warranties or conditions
+      of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
+      PARTICULAR PURPOSE. You are solely responsible for determining the
+      appropriateness of using or redistributing the Work and assume any
+      risks associated with Your exercise of permissions under this License.
+
+   8. Limitation of Liability. In no event and under no legal theory,
+      whether in tort (including negligence), contract, or otherwise,
+      unless required by applicable law (such as deliberate and grossly
+      negligent acts) or agreed to in writing, shall any Contributor be
+      liable to You for damages, including any direct, indirect, special,
+      incidental, or consequential damages of any character arising as a
+      result of this License or out of the use or inability to use the
+      Work (including but not limited to damages for loss of goodwill,
+      work stoppage, computer failure or malfunction, or any and all
+      other commercial damages or losses), even if such Contributor
+      has been advised of the possibility of such damages.
+
+   9. Accepting Warranty or Additional Liability. While redistributing
+      the Work or Derivative Works thereof, You may choose to offer,
+      and charge a fee for, acceptance of support, warranty, indemnity,
+      or other liability obligations and/or rights consistent with this
+      License. However, in accepting such obligations, You may act only
+      on Your own behalf and on Your sole responsibility, not on behalf
+      of any other Contributor, and only if You agree to indemnify,
+      defend, and hold each Contributor harmless for any liability
+      incurred by, or claims asserted against, such Contributor by reason
+      of your accepting any such warranty or additional liability.
+
+   END OF TERMS AND CONDITIONS
+
+   APPENDIX: How to apply the Apache License to your work.
+
+      To apply the Apache License to your work, attach the following
+      boilerplate notice, with the fields enclosed by brackets "[]"
+      replaced with your own identifying information. (Don't include
+      the brackets!)  The text should be enclosed in the appropriate
+      comment syntax for the file format. We also recommend that a
+      file or class name and description of purpose be included on the
+      same "printed page" as the copyright notice for easier
+      identification within third-party archives.
+
+   Copyright [yyyy] [name of copyright owner]
+
+   Licensed under the Apache License, Version 2.0 (the "License");
+   you may not use this file except in compliance with the License.
+   You may obtain a copy of the License at
+
+       http://www.apache.org/licenses/LICENSE-2.0
+
+   Unless required by applicable law or agreed to in writing, software
+   distributed under the License is distributed on an "AS IS" BASIS,
+   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+   See the License for the specific language governing permissions and
+   limitations under the License.

onw/npukit/NOTICE.txt

@@ -0,0 +1,16 @@
+onw NPU kit (onw 3.0)
+
+This folder holds binaries built by the onw project from open-source code of Intel Corporation, licensed under the
+Apache License 2.0 (LICENSE). They are modified versions; the modifications are in patches/.
+
+- openvino_intel_npu_compiler.dll: Intel NPU compiler (https://github.com/openvinotoolkit/npu_compiler) at commit
+  20e7912fbc230725e1ca7394762348d4f480aa65 (submodule thirdparty/vpucostmodel at 4b5da8484d7a5c8cb097486c1248c4d2a753ae4b),
+  built as the compiler-in-driver library (BUILD_COMPILER_FOR_DRIVER=ON, Release) against OpenVINO
+  d8047fb380b27a9d5827cb3f22aec7781ae5ac82. Changes: patches/npu_compiler-20e7912.patch, patches/vpucostmodel.patch.
+- openvino_intel_npu_plugin.dll: the NPU plugin of OpenVINO (https://github.com/openvinotoolkit/openvino) at commit
+  6db412d0a2766d88936477bb97bd37acf2d9d767 - the commit of the OpenVINO 2026.5.0 release onw installs.
+  Changes: patches/openvino-6db412d-npu_plugin.patch.
+- openvino_intel_npu_compiler_loader.dll, openvino_intel_npu_vm_runtime.dll: unchanged copies from the OpenVINO 2026.5.0
+  wheel (https://pypi.org/project/openvino/), needed next to the plugin.
+
+Copyright (C) 2018-2026 Intel Corporation (the original code). The binaries are provided as is, without warranty.

onw/npukit/kit.json

@@ -0,0 +1,19 @@
+{
+  "version": "3.0.0 (npu_compiler 8.3 + onw fixes, NPU plugin 2026.5.0 6db412d + weight sharing)",
+  "openvino": "2026.5.0-23160-6db412d0a27",
+  "files": [
+    "openvino_intel_npu_plugin.dll",
+    "openvino_intel_npu_compiler.dll",
+    "openvino_intel_npu_compiler_loader.dll",
+    "openvino_intel_npu_vm_runtime.dll",
+    "plugins.xml"
+  ],
+  "files_linux": [
+    "libopenvino_intel_npu_plugin.so",
+    "libopenvino_intel_npu_compiler.so",
+    "libopenvino_intel_npu_compiler_loader.so",
+    "libopenvino_intel_npu_vm_runtime.so",
+    "plugins.xml"
+  ],
+  "archive_linux": "npukit-linux.tar.xz"
+}

onw/npukit/linux/npukit-linux.tar.xz

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:46fa6b160305b720e56d0d4a44d76a344b06cf7ca886b2a26a447bfb338ba857
+size 32282368

onw/npukit/openvino_intel_npu_compiler.dll

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ce1113abf245d6ada1fff920eb1373dc136e1190dc0558bc97d0b6c9ae27ed22
+size 83966464

onw/npukit/openvino_intel_npu_compiler_loader.dll

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:15494682f53e465e38c7d655a108f04681fb2af7a1dce1738d2e9212aafe5a14
+size 529680

onw/npukit/openvino_intel_npu_plugin.dll

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:91cf9ebe2bd9ef7a3e3e451b63e7497abe465bfa2396edb52b35e8eb1d4db2ce
+size 7992832

onw/npukit/openvino_intel_npu_vm_runtime.dll

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:39ece78739633565efd60858f63f2a307397b6ef168019c8abe4a24f7b345c88
+size 559376

onw/npukit/patches/npu_compiler-20e7912.patch

@@ -0,0 +1,1455 @@
+diff --git a/src/vpux_compiler/include/vpux/compiler/core/dma_cost_model_utils.hpp b/src/vpux_compiler/include/vpux/compiler/core/dma_cost_model_utils.hpp
+index 3bbd048..c7d8799 100644
+--- a/src/vpux_compiler/include/vpux/compiler/core/dma_cost_model_utils.hpp
++++ b/src/vpux_compiler/include/vpux/compiler/core/dma_cost_model_utils.hpp
+@@ -34,10 +34,10 @@ double getStrideDMACorrectionThresholdByArch(config::ArchKind arch);
+ bool applyStrideDMACorrectionForTile(vpux::NDTypeInterface tileType, bool isStridedDMA, uint32_t& cost,
+                                      config::ArchKind arch, bool isFullSearchVersion = false);
+ bool correctStrideDMACostOnAllTiles(
+-        ArrayRef<std::vector<std::pair<vpux::NDTypeInterface, llvm::DenseMap<mlir::Type, VPU::DistributionInfo>>>>
++        ArrayRef<std::vector<std::pair<vpux::NDTypeInterface, llvm::SmallDenseMap<mlir::Type, VPU::DistributionInfo, 4>>>>
+                 tilesTypes,
+         const std::function<vpux::NDTypeInterface(
+-                ArrayRef<std::pair<vpux::NDTypeInterface, llvm::DenseMap<mlir::Type, VPU::DistributionInfo>>>)>&
++                ArrayRef<std::pair<vpux::NDTypeInterface, llvm::SmallDenseMap<mlir::Type, VPU::DistributionInfo, 4>>>)>&
+                 tileTypeGetter,
+         SmallVector<uint32_t>& dmaCost, bool isStridedDMA, config::ArchKind arch);
+ 
+diff --git a/src/vpux_compiler/include/vpux/compiler/dialect/VPU/utils/distributed_tensor_utils.hpp b/src/vpux_compiler/include/vpux/compiler/dialect/VPU/utils/distributed_tensor_utils.hpp
+index 3943fd7..c15b6aa 100644
+--- a/src/vpux_compiler/include/vpux/compiler/dialect/VPU/utils/distributed_tensor_utils.hpp
++++ b/src/vpux_compiler/include/vpux/compiler/dialect/VPU/utils/distributed_tensor_utils.hpp
+@@ -46,7 +46,30 @@ const SmallVector<int64_t> DISTRIBUTED_N_ALIGNMENT = SmallVector<int64_t>{16, 1,
+ const SmallVector<int64_t> DISTRIBUTED_DW_ACT_C_ALIGNMENT = SmallVector<int64_t>{1, 32, 1, 1};
+ const SmallVector<int64_t> DISTRIBUTED_DW_WT_C_ALIGNMENT = SmallVector<int64_t>{32, 1, 1, 1};
+ 
+-using TensorDistributionMap = llvm::DenseMap<mlir::Type, VPU::DistributionInfo>;
++// onw: a plain DenseMap allocates 64 buckets of DistributionInfo (~45 KB) even for one entry - above the Windows
++// low-fragmentation-heap limit, so every map went through the locked heap path and the parallel compile threads
++// serialised on it (strategy assignment ~3x slower than single-threaded). Up to 3 entries now live inline.
++using TensorDistributionMap = llvm::SmallDenseMap<mlir::Type, VPU::DistributionInfo, 4>;
++
++}  // namespace VPU
++}  // namespace vpux
++
++// onw: the inline buckets make these element types too big for SmallVector's default inline count; keep one inline.
++template <>
++struct llvm::CalculateSmallVectorDefaultInlinedElements<vpux::VPU::TensorDistributionMap> {
++    static constexpr size_t value = 1;
++};
++template <>
++struct llvm::CalculateSmallVectorDefaultInlinedElements<std::pair<vpux::NDTypeInterface, vpux::VPU::TensorDistributionMap>> {
++    static constexpr size_t value = 1;
++};
++template <>
++struct llvm::CalculateSmallVectorDefaultInlinedElements<std::pair<mlir::Type, vpux::VPU::TensorDistributionMap>> {
++    static constexpr size_t value = 1;
++};
++
++namespace vpux {
++namespace VPU {
+ 
+ VPU::DistributionInfoAttr updateSliceLikeOpsAlignment(mlir::MLIRContext* ctx, vpux::ShapeRef inShape,
+                                                       vpux::ShapeRef sliceShape,
+diff --git a/src/vpux_compiler/include/vpux/compiler/dialect/VPU/utils/multi_cluster_strategy_utils.hpp b/src/vpux_compiler/include/vpux/compiler/dialect/VPU/utils/multi_cluster_strategy_utils.hpp
+index 8895bf9..1e0b4d3 100644
+--- a/src/vpux_compiler/include/vpux/compiler/dialect/VPU/utils/multi_cluster_strategy_utils.hpp
++++ b/src/vpux_compiler/include/vpux/compiler/dialect/VPU/utils/multi_cluster_strategy_utils.hpp
+@@ -148,6 +148,9 @@ private:
+     std::pair<std::pair<mlir::Type, TensorDistributionMap>, std::pair<mlir::Type, TensorDistributionMap>>
+     getDistributionsWithStrategy(VPU::ClusteredOpInterface parentOp, VPU::MultiClusterStrategy parentStrategy,
+                                  VPU::ClusteredOpInterface userOp, VPU::MultiClusterStrategy userStrategy) const;
++    std::pair<std::pair<mlir::Type, TensorDistributionMap>, std::pair<mlir::Type, TensorDistributionMap>>
++    getDistributionsWithStrategyImpl(VPU::ClusteredOpInterface parentOp, VPU::MultiClusterStrategy parentStrategy,
++                                     VPU::ClusteredOpInterface userOp, VPU::MultiClusterStrategy userStrategy) const;
+ 
+     /* DMA cost calculation helpers */
+     double getDMACostOfType(vpux::NDTypeInterface srcTensorType) const;
+@@ -184,6 +187,8 @@ private:
+     SiblingOpsAnalysis& _siblingsOpsAnalysis;
+     bool _underSubgraphOpt = false;
+     bool _enableTilingFullSearchSpace = false;
++    // onw: memo of getDistributionsWithStrategy results (type-erased, see multi_cluster_strategy_utils.cpp)
++    mutable std::shared_ptr<void> _onwDistCache;
+ };
+ vpux::VPU::StrategyCost correctSwOpCost(VPU::SWOpInterface swOp, ArrayRef<vpux::NDTypeInterface> tiledInputTypes,
+                                         vpux::VPU::StrategyCost cost);
+diff --git a/src/vpux_compiler/include/vpux/compiler/dialect/VPU/utils/vertical_fusion/v2/merge_vf_region_rewriter.hpp b/src/vpux_compiler/include/vpux/compiler/dialect/VPU/utils/vertical_fusion/v2/merge_vf_region_rewriter.hpp
+index fe48940..c819038 100644
+--- a/src/vpux_compiler/include/vpux/compiler/dialect/VPU/utils/vertical_fusion/v2/merge_vf_region_rewriter.hpp
++++ b/src/vpux_compiler/include/vpux/compiler/dialect/VPU/utils/vertical_fusion/v2/merge_vf_region_rewriter.hpp
+@@ -40,6 +40,8 @@ protected:
+     std::shared_ptr<IVFScheduling<VFConfig>> detectScenario(VFConfig& vfConfig) const override;
+     std::optional<VFCase> findVFTiling(VPU::VerticalFusionOp mergedOp, VPU::VerticalFusionOp prevOp,
+                                        VPU::VerticalFusionOp currentOp, const MergeTilingPolicy& tilingPolicy) const;
++    std::optional<VFCase> findVFTilingImpl(VPU::VerticalFusionOp mergedOp, VPU::VerticalFusionOp prevOp,
++                                       VPU::VerticalFusionOp currentOp, const MergeTilingPolicy& tilingPolicy) const;
+     StrategyCost getOriginalVFCost(VPU::VerticalFusionOp vfOp) const;
+ 
+     VFCacheAnalysis& _cache;
+diff --git a/src/vpux_compiler/src/compiler.cpp b/src/vpux_compiler/src/compiler.cpp
+index 263a325..acf446c 100644
+--- a/src/vpux_compiler/src/compiler.cpp
++++ b/src/vpux_compiler/src/compiler.cpp
+@@ -1087,7 +1087,11 @@ void compileIEtoVPU(mlir::OwningOpRef<mlir::ModuleOp>& moduleOp,
+                         wsExtractionMode.value());
+         ieToVPUpm.addPass(VPU::createConstructWsAnalysisPass(log));
+         ieToVPUpm.addPass(VPU::createIntroduceInitFunctionPass(wsExtractionMode.value(), initPart, memLimit, log));
+-        ieToVPUpm.addPass(VPU::createConcatInitResultsPass(wsExtractionMode.value(), initPart, memLimit, log));
++        // onw: ONW_WS_NO_CONCAT keeps one init output per weight (no concatenation into one obfuscated buffer), so the
++        // plugin can share identical weights between graphs of different structure
++        if (std::getenv("ONW_WS_NO_CONCAT") == nullptr) {
++            ieToVPUpm.addPass(VPU::createConcatInitResultsPass(wsExtractionMode.value(), initPart, memLimit, log));
++        }
+         ieToVPUpm.addPass(VPU::createDestructWsAnalysisPass(log));
+ 
+         const auto grc = getDefaultGreedyRewriteConfig();
+@@ -1121,8 +1125,10 @@ void compileVPUIP(mlir::OwningOpRef<mlir::ModuleOp>& vpuModule,
+         vpuipPM.addPass(VPU::createConstructWsAnalysisPass(log));
+         vpuipPM.addPass(VPU::createIntroduceInitFunctionPass(wsExtractionMode.value(), /* initPart = */ std::nullopt,
+                                                              memLimit, log));
+-        vpuipPM.addPass(VPU::createConcatInitResultsPass(wsExtractionMode.value(), /* initPart = */ std::nullopt,
+-                                                         memLimit, log));
++        if (std::getenv("ONW_WS_NO_CONCAT") == nullptr) {
++            vpuipPM.addPass(VPU::createConcatInitResultsPass(wsExtractionMode.value(), /* initPart = */ std::nullopt,
++                                                             memLimit, log));
++        }
+         vpuipPM.addPass(VPU::createDestructWsAnalysisPass(log));
+     }
+ 
+diff --git a/src/vpux_compiler/src/conversion/passes/IE2VPU/convert_layers_to_VPU.cpp b/src/vpux_compiler/src/conversion/passes/IE2VPU/convert_layers_to_VPU.cpp
+index f80f56e..3cfef97 100644
+--- a/src/vpux_compiler/src/conversion/passes/IE2VPU/convert_layers_to_VPU.cpp
++++ b/src/vpux_compiler/src/conversion/passes/IE2VPU/convert_layers_to_VPU.cpp
+@@ -10,6 +10,8 @@
+ #include <mlir/Support/LLVM.h>
+ #include "vpux/compiler/conversion.hpp"
+ #include "vpux/compiler/dialect/IE/IR/dialect.hpp"
++#include "vpux/compiler/dialect/IE/IR/ops/data_movement.hpp"
++#include "vpux/compiler/dialect/IE/IR/ops/data_type.hpp"
+ #include "vpux/compiler/dialect/IE/IR/ops/activation.hpp"
+ #include "vpux/compiler/dialect/IE/IR/ops/arithmetic.hpp"
+ #include "vpux/compiler/dialect/IE/IR/ops/bitwise.hpp"
+@@ -820,9 +822,72 @@ private:
+     void safeRunOnFunc() final;
+ };
+ 
++// onw: the weights-separation @init schedule permutes sub-byte (2/4-bit) weights as data; a permute of 4-bit elements
++// becomes a PermuteDMA with 4-bit element strides, which the DMA cannot do ("Non byte aligned stride value"). In init
++// functions only, do such permutes in f16 (exact for these integers) between two Converts.
++static void onwWidenSubBytePermutes(mlir::func::FuncOp func) {
++    if (std::getenv("ONW_WS_NO_WIDEN") != nullptr || func.getName().find("init") == llvm::StringRef::npos ||
++        config::getArch(func) <= config::ArchKind::NPU37XX) {
++        return;
++    }
++    SmallVector<mlir::Operation*> todo;
++    func.walk([&](mlir::Operation* op) {
++        if (!mlir::isa<IE::MemPermuteOp, IE::ReorderOp, IE::TransposeOp>(op)) {
++            return;
++        }
++        auto elemType = mlir::cast<NDTypeInterface>(op->getOperand(0).getType()).getElementType();
++        mlir::Type storage = elemType;
++        if (auto q = mlir::dyn_cast<mlir::quant::QuantizedType>(elemType)) {
++            storage = q.getStorageType();
++        }
++        auto intType = mlir::dyn_cast<mlir::IntegerType>(storage);
++        if (intType != nullptr && intType.getWidth() < 8) {
++            todo.push_back(op);
++        }
++    });
++    for (auto* op : todo) {
++        mlir::OpBuilder builder(op);
++        const auto loc = op->getLoc();
++        const auto inType = mlir::cast<NDTypeInterface>(op->getOperand(0).getType());
++        const auto outType = mlir::cast<NDTypeInterface>(op->getResult(0).getType());
++        const auto inElem = inType.getElementType();
++        const auto outElem = outType.getElementType();
++        auto toStorage = [](mlir::Type t) -> mlir::Type {
++            if (auto q = mlir::dyn_cast<mlir::quant::QuantizedType>(t)) {
++                return q.getStorageType();
++            }
++            return t;
++        };
++        mlir::Value x = op->getOperand(0);
++        if (mlir::isa<mlir::quant::QuantizedType>(inElem)) {
++            x = builder.create<IE::QuantizeCastOp>(appendLoc(loc, "onw_storage_in"), x, toStorage(inElem));
++        }
++        auto f16 = mlir::Float16Type::get(op->getContext());
++        x = builder.create<IE::ConvertOp>(appendLoc(loc, "onw_to_f16"), x, f16);
++        auto* newOp = builder.clone(*op);
++        newOp->setOperand(0, x);
++        newOp->getResult(0).setType(outType.changeElemType(f16));
++        mlir::Value y = builder.create<IE::ConvertOp>(appendLoc(loc, "onw_from_f16"), newOp->getResult(0),
++                                                       toStorage(outElem));
++        if (mlir::isa<mlir::quant::QuantizedType>(outElem)) {
++            y = builder.create<IE::QuantizeCastOp>(appendLoc(loc, "onw_storage_out"), y, outElem);
++        }
++        op->getResult(0).replaceAllUsesWith(y);
++        op->erase();
++    }
++}
++
+ void ConvertLayers2VPUPass::safeRunOnFunc() {
+     auto& ctx = getContext();
+     auto func = getOperation();
++    if (const char* onwDump = std::getenv("ONW_DUMP_IE")) {
++        std::error_code ec;
++        llvm::raw_fd_ostream fo((std::string(onwDump) + "_" + func.getName().str() + ".mlir"), ec);
++        if (!ec) {
++            func->print(fo, mlir::OpPrintingFlags().enableDebugInfo(true, false));
++        }
++    }
++    onwWidenSubBytePermutes(func);
+ 
+     mlir::ConversionTarget target(ctx);
+     target.addIllegalDialect<IE::IEDialect>();
+diff --git a/src/vpux_compiler/src/dialect/IE/transforms/passes/swap_convert_with_reshape_kind_ops.cpp b/src/vpux_compiler/src/dialect/IE/transforms/passes/swap_convert_with_reshape_kind_ops.cpp
+index d16690c..93f292f 100644
+--- a/src/vpux_compiler/src/dialect/IE/transforms/passes/swap_convert_with_reshape_kind_ops.cpp
++++ b/src/vpux_compiler/src/dialect/IE/transforms/passes/swap_convert_with_reshape_kind_ops.cpp
+@@ -3,6 +3,7 @@
+ // SPDX-License-Identifier: Apache-2.0
+ //
+ 
++#include "vpux/compiler/core/types/quantile_float/types.hpp"
+ #include "vpux/compiler/dialect/IE/IR/dialect.hpp"
+ #include "vpux/compiler/dialect/IE/IR/ops/data_movement.hpp"
+ #include "vpux/compiler/dialect/IE/IR/ops/data_type.hpp"
+@@ -212,6 +213,27 @@ PropagationDecision SwapConvertWithReshapeKindOps::PropagateConvertToFuseConvert
+     if (!firstInputSize || !middleTypeSize || !lastOutputSize) {
+         return PropagationDecision::NoPropagate;
+     }
++    // onw: in the weights-separation @init schedule, Converts around a QuantizeCast to/from a quantile (NF4) or
++    // quantized type carry raw table indices; fusing them would create an unsupported "Convert to quantile".
++    {
++        auto isQuantLike = [](mlir::Type t) {
++            return mlir::isa<mlir::quant::QuantizedType, vpux::type::QuantileType>(t);
++        };
++        auto func = firstConvert->getParentOfType<mlir::func::FuncOp>();
++        if (func != nullptr && func.getName().find("init") != llvm::StringRef::npos &&
++            (isQuantLike(firstInputType) || isQuantLike(firstOutputType) || isQuantLike(lastOutputType))) {
++            return PropagationDecision::NoPropagate;
++        }
++        if (func != nullptr && func.getName().find("init") != llvm::StringRef::npos) {
++            for (mlir::Operation* op = firstConvert.getOperation(); op != nullptr && op != lastConvert.getOperation();
++                 op = op->hasOneUse() ? *op->getUsers().begin() : nullptr) {
++                if (mlir::isa<IE::QuantizeCastOp>(op) ||
++                    isQuantLike(mlir::cast<vpux::NDTypeInterface>(op->getResult(0).getType()).getElementType())) {
++                    return PropagationDecision::NoPropagate;
++                }
++            }
++        }
++    }
+ 
+     int64_t minTypeSize = std::min({firstInputSize, middleTypeSize, lastOutputSize});
+     if (minTypeSize == lastOutputSize) {
+diff --git a/src/vpux_compiler/src/dialect/IE/transforms/passes/weights_dequantize_to_dynamic_dequantize.cpp b/src/vpux_compiler/src/dialect/IE/transforms/passes/weights_dequantize_to_dynamic_dequantize.cpp
+index ac46b4a..5ec32cc 100644
+--- a/src/vpux_compiler/src/dialect/IE/transforms/passes/weights_dequantize_to_dynamic_dequantize.cpp
++++ b/src/vpux_compiler/src/dialect/IE/transforms/passes/weights_dequantize_to_dynamic_dequantize.cpp
+@@ -113,7 +113,11 @@ mlir::LogicalResult finishDynamicDequantizeRewrite(mlir::PatternRewriter& rewrit
+             rewriter.create<IE::DynamicDequantizeOp>(appendLoc(loc, "ddq"), rawWeights, scaleValue, zpValue, dstType);
+ 
+     // Mark the op for the DynDeq -> FakeQuantize bridge pass.
+-    if (wdInfo.isKVcachedPattern() || !zpElemType.isInteger(2)) {
++    // [onw] 2-bit zero points too outside the KV-cached (one-row) pattern: unmarked, a multi-row FullyConnected
++    // dequantizes 2-bit weights with the DynamicDequantize SHAVE kernel, which returns zeros for them; marked, they go
++    // FakeQuantize -> group unroll -> palletized LUT on the DPU as the one-row graphs do (2-bit weights kept)
++    static const bool onwMarkAll = std::getenv("ONW_U2_MARK") == nullptr || std::string(std::getenv("ONW_U2_MARK")) != "0";
++    if (wdInfo.isKVcachedPattern() || !zpElemType.isInteger(2) || onwMarkAll) {
+         dynamicDequant->setAttr(IE::WEIGHTS_IMPORT_DYN_DEQUANT_ATTR, mlir::UnitAttr::get(rewriter.getContext()));
+     }
+ 
+diff --git a/src/vpux_compiler/src/dialect/VPU/IR/ops/convert.cpp b/src/vpux_compiler/src/dialect/VPU/IR/ops/convert.cpp
+index 7390492..4dd29e4 100644
+--- a/src/vpux_compiler/src/dialect/VPU/IR/ops/convert.cpp
++++ b/src/vpux_compiler/src/dialect/VPU/IR/ops/convert.cpp
+@@ -55,11 +55,37 @@ bool vpux::VPU::ConvertOp::areCastCompatible(mlir::TypeRange inputs, mlir::TypeR
+     return input && output && input.getShape() == output.getShape();
+ }
+ 
+-bool vpux::VPU::ConvertOp::checkStrategyCompatibility(vpux::VPU::MultiClusterStrategy strategy, size_t) {
++bool vpux::VPU::ConvertOp::checkStrategyCompatibility(vpux::VPU::MultiClusterStrategy strategy, size_t numTiles) {
+     bool isStrategyCompatible = false;
+     constexpr int64_t MIN_DIM_SIZE_FOR_TILING = 4;
+     auto inputShape = getBoundedShape(getInput());
+ 
++    // onw: a sub-byte output split along its innermost memory dim must give byte-aligned cluster segments, otherwise
++    // the cluster DMAs start mid-byte ("Can't convert 172 Bit to Byte" in UnrollDistributedOps; seen in the init
++    // schedule of weightless blobs: f16 -> si4, 128 rows over 3 clusters = 43 x 4 bit).
++    const auto splitsMidByteFor = [&](mlir::Value v, Dim dim) {
++        const auto type = mlir::cast<NDTypeInterface>(v.getType());
++        const auto bits = type.getElemTypeSize().count();
++        if (bits >= 8 || numTiles == 0 || type.getRank() != 4 || type.getDimsOrder().dimAt(type.getRank() - 1) != dim) {
++            return false;
++        }
++        const auto shape = getBoundedShape(v);
++        const auto chunk = divUp(shape[dim], static_cast<int64_t>(numTiles));
++        return (chunk * bits) % 8 != 0;
++    };
++    const auto splitsMidByte = [&](Dim dim) {
++        static const bool onwOff = std::getenv("ONW_NO_SUBBYTE_SPLIT_FIX") != nullptr;
++        return !onwOff && (splitsMidByteFor(getInput(), dim) || splitsMidByteFor(getOutput(), dim));
++    };
++    if ((strategy == VPU::MultiClusterStrategy::SplitOverHeight ||
++         strategy == VPU::MultiClusterStrategy::SplitOverHeightOverlapped) &&
++        splitsMidByte(Dims4D::Act::H)) {
++        return false;
++    }
++    if (strategy == VPU::MultiClusterStrategy::SplitOverKernel && splitsMidByte(Dims4D::Act::C)) {
++        return false;
++    }
++
+     switch (strategy) {
+     case VPU::MultiClusterStrategy::Clustering:
+         isStrategyCompatible = true;
+diff --git a/src/vpux_compiler/src/dialect/VPU/transforms/passes/multi_cluster_strategy_assignment.cpp b/src/vpux_compiler/src/dialect/VPU/transforms/passes/multi_cluster_strategy_assignment.cpp
+index adaca74..fab49de 100644
+--- a/src/vpux_compiler/src/dialect/VPU/transforms/passes/multi_cluster_strategy_assignment.cpp
++++ b/src/vpux_compiler/src/dialect/VPU/transforms/passes/multi_cluster_strategy_assignment.cpp
+@@ -5,6 +5,10 @@
+ 
+ #include "vpux/compiler/dialect/VPU/IR/dialect.hpp"
+ #include "vpux/compiler/dialect/VPU/transforms/passes.hpp"
++
++#include <chrono>
++#include <cstdio>
++#include <cstdlib>
+ #include "vpux/compiler/dialect/VPU/utils/cost_model/layer_vpunn_cost.hpp"
+ #include "vpux/compiler/dialect/VPU/utils/manual_strategy_utils.hpp"
+ #include "vpux/compiler/dialect/VPU/utils/multi_cluster_strategy_utils.hpp"
+@@ -121,10 +125,21 @@ void MultiClusterStrategyAssignmentPass::safeRunOnFunc() {
+     _log.debug("Greedy Strategy Assignment");
+     auto enableMultiClusterForSWLayer =
+             config::getAvailableExecutor(module, config::ExecutorKind::SHAVE_ACT) != nullptr;
++    const auto onwT0 = std::chrono::steady_clock::now();
+     strategyManager.assignMultiClusterStrategy(enableMultiClusterForSWLayer);
++    const auto onwT1 = std::chrono::steady_clock::now();
+ 
+     _log.debug("Execute Subgraph Optimization");
+     strategyManager.optimizeMulticlusterStrategy();
++    const auto onwT2 = std::chrono::steady_clock::now();
++    if (const char* onwP = std::getenv("ONW_PROF_FILE")) {
++        if (FILE* f = std::fopen(onwP, "a")) {
++            std::fprintf(f, "mcsa ops %lld assign %.2f s optimize %.2f s%c", (long long)clusteredOpCount,
++                         std::chrono::duration<double>(onwT1 - onwT0).count(),
++                         std::chrono::duration<double>(onwT2 - onwT1).count(), 10);
++            std::fclose(f);
++        }
++    }
+ 
+     _log.trace("Remove Temporary Strategy");
+     strategyManager.removeTemporaryMulticlusterStrategy();
+diff --git a/src/vpux_compiler/src/dialect/VPU/utils/strategy_manager/multi_cluster_strategy_utils.cpp b/src/vpux_compiler/src/dialect/VPU/utils/strategy_manager/multi_cluster_strategy_utils.cpp
+index 4bfec49..26e3194 100644
+--- a/src/vpux_compiler/src/dialect/VPU/utils/strategy_manager/multi_cluster_strategy_utils.cpp
++++ b/src/vpux_compiler/src/dialect/VPU/utils/strategy_manager/multi_cluster_strategy_utils.cpp
+@@ -29,6 +29,11 @@
+ #include "vpux/compiler/dialect/VPU/utils/op_tiling_cache.hpp"
+ #include "vpux/compiler/dialect/VPU/utils/scheduling/temporal_tiling_driver.hpp"
+ #include "vpux/compiler/dialect/VPU/utils/singleton_cache.hpp"
++#include <atomic>
++#include <cstdio>
++#include <cstdlib>
++#include <map>
++#include <tuple>
+ #include "vpux/compiler/dialect/VPU/utils/sparsity_utils.hpp"
+ #include "vpux/compiler/dialect/VPU/utils/tile_utils.hpp"
+ #include "vpux/compiler/dialect/VPUIP/utils/convert_to_dma_utils.hpp"
+@@ -1081,8 +1086,90 @@ bool LayerCostModel::hasSpilling(VPU::ClusteredOpInterface op, vpux::NDTypeInter
+     return hasSpilling(op, srcTensorDistributionMap, dstTensorDistributionMap);
+ }
+ 
++namespace {
++using OnwDistPair = std::pair<std::pair<mlir::Type, TensorDistributionMap>, std::pair<mlir::Type, TensorDistributionMap>>;
++struct OnwDistCache {
++    std::map<std::tuple<void*, int, void*, int>, OnwDistPair> m;
++};
++std::atomic<long long> onwDistCalls{0}, onwDistHits{0}, onwDistBad{0};
++// ONW_DIST_CACHE: unset/0 = off, "check" = compute every time and compare with the memo, anything else = use the memo
++int onwDistMode() {
++    static const int mode = [] {
++        const char* e = std::getenv("ONW_DIST_CACHE");
++        if (e == nullptr || std::string(e) == "0" || std::string(e).empty()) {
++            return 0;
++        }
++        return std::string(e) == "check" ? 2 : 1;
++    }();
++    return mode;
++}
++bool onwSameMap(const TensorDistributionMap& a, const TensorDistributionMap& b) {
++    if (a.size() != b.size()) {
++        return false;
++    }
++    for (const auto& kv : a) {
++        auto it = b.find(kv.first);
++        if (it == b.end() || !(it->second == kv.second)) {
++            return false;
++        }
++    }
++    return true;
++}
++bool onwSame(const OnwDistPair& a, const OnwDistPair& b) {
++    return a.first.first == b.first.first && a.second.first == b.second.first &&
++           onwSameMap(a.first.second, b.first.second) && onwSameMap(a.second.second, b.second.second);
++}
++struct OnwDistReport {
++    ~OnwDistReport() {
++        if (onwDistCalls.load() == 0) {
++            return;
++        }
++        if (const char* p = std::getenv("ONW_PROF_FILE")) {
++            if (FILE* f = std::fopen(p, "a")) {
++                std::fprintf(f, "distcache calls %lld hits %lld mismatches %lld%c", onwDistCalls.load(),
++                             onwDistHits.load(), onwDistBad.load(), 10);
++                std::fclose(f);
++            }
++        }
++    }
++} onwDistReport;
++}  // namespace
++
++OnwDistPair LayerCostModel::getDistributionsWithStrategy(VPU::ClusteredOpInterface parentOp,
++                                                         VPU::MultiClusterStrategy parentStrategy,
++                                                         VPU::ClusteredOpInterface userOp,
++                                                         VPU::MultiClusterStrategy userStrategy) const {
++    const int mode = onwDistMode();
++    if (mode == 0) {
++        return getDistributionsWithStrategyImpl(parentOp, parentStrategy, userOp, userStrategy);
++    }
++    if (_onwDistCache == nullptr) {
++        _onwDistCache = std::make_shared<OnwDistCache>();
++    }
++    auto& cache = static_cast<OnwDistCache*>(_onwDistCache.get())->m;
++    ++onwDistCalls;
++    const auto key = std::make_tuple(static_cast<void*>(parentOp.getOperation()), static_cast<int>(parentStrategy),
++                                     static_cast<void*>(userOp.getOperation()), static_cast<int>(userStrategy));
++    auto it = cache.find(key);
++    if (it != cache.end()) {
++        ++onwDistHits;
++        if (mode == 1) {
++            return it->second;
++        }
++        auto fresh = getDistributionsWithStrategyImpl(parentOp, parentStrategy, userOp, userStrategy);
++        if (!onwSame(fresh, it->second)) {
++            ++onwDistBad;
++            it->second = fresh;
++        }
++        return fresh;
++    }
++    auto fresh = getDistributionsWithStrategyImpl(parentOp, parentStrategy, userOp, userStrategy);
++    cache.emplace(key, fresh);
++    return fresh;
++}
++
+ std::pair<std::pair<mlir::Type, TensorDistributionMap>, std::pair<mlir::Type, TensorDistributionMap>>
+-LayerCostModel::getDistributionsWithStrategy(VPU::ClusteredOpInterface parentOp,
++LayerCostModel::getDistributionsWithStrategyImpl(VPU::ClusteredOpInterface parentOp,
+                                              VPU::MultiClusterStrategy parentStrategy, VPU::ClusteredOpInterface userOp,
+                                              VPU::MultiClusterStrategy userStrategy) const {
+     // Set the custom strategy to the op to get the accurate distributed type
+@@ -1090,14 +1177,26 @@ LayerCostModel::getDistributionsWithStrategy(VPU::ClusteredOpInterface parentOp,
+     // e.g., Conv (SOK) -> SW (SOK), the output of the Conv would be SEGMENTED
+     // Conv (SOK) -> SW (Clustering), the output of the Conv would be DUPLICATED|SEGMENTED
+     // The DistributedType is decided by the ops multiCluster strategy attributes
++    // onw: setting an op's strategy to the value it already has is a no-op (same uniqued attribute), so skip the
++    // temporary set/restore pair then - each one rebuilds the op's attribute dictionary in the shared context.
+     auto greedyStrategyParentOp = getMultiClusterStrategyValue(parentOp);
+     auto greedyStrategyUserOp = getMultiClusterStrategyValue(userOp);
+-    parentOp.setMultiClusterStrategy(parentStrategy);
+-    userOp.setMultiClusterStrategy(userStrategy);
++    const bool setParent = parentStrategy != greedyStrategyParentOp;
++    const bool setUser = userStrategy != greedyStrategyUserOp;
++    if (setParent) {
++        parentOp.setMultiClusterStrategy(parentStrategy);
++    }
++    if (setUser) {
++        userOp.setMultiClusterStrategy(userStrategy);
++    }
+     auto targetOutput = getOutputWithDistribution(parentOp, parentStrategy);
+     auto targetInput = getInputWithDistribution(userOp, parentOp, userStrategy);
+-    parentOp.setMultiClusterStrategy(greedyStrategyParentOp);
+-    userOp.setMultiClusterStrategy(greedyStrategyUserOp);
++    if (setParent) {
++        parentOp.setMultiClusterStrategy(greedyStrategyParentOp);
++    }
++    if (setUser) {
++        userOp.setMultiClusterStrategy(greedyStrategyUserOp);
++    }
+ 
+     VPU::DistributionInfo outputDistribution{};
+     auto outputType = mlir::isa<VPU::SparseTensorType>(targetOutput.first)
+diff --git a/src/vpux_compiler/src/dialect/VPU/utils/strategy_manager/subgraph_optimizer.cpp b/src/vpux_compiler/src/dialect/VPU/utils/strategy_manager/subgraph_optimizer.cpp
+index 610fe6c..8cac379 100644
+--- a/src/vpux_compiler/src/dialect/VPU/utils/strategy_manager/subgraph_optimizer.cpp
++++ b/src/vpux_compiler/src/dialect/VPU/utils/strategy_manager/subgraph_optimizer.cpp
+@@ -3,6 +3,10 @@
+ // SPDX-License-Identifier: Apache-2.0
+ //
+ 
++#include <atomic>
++#include <chrono>
++#include <cstdio>
++#include <cstdlib>
+ #include "vpux/compiler/dialect/VPU/utils/strategy_manager/subgraph_optimizer.hpp"
+ #include "vpux/compiler/dialect/VPU/IR/ops/data_movement.hpp"
+ #include "vpux/compiler/dialect/VPU/IR/ops/internal.hpp"
+@@ -17,6 +21,19 @@
+ #include <llvm/ADT/STLExtras.h>
+ #include <llvm/ADT/TypeSwitch.h>
+ #include "mlir/IR/Iterators.h"
++#include <atomic>
++#include <chrono>
++#include <optional>
++
++namespace {
++std::atomic<long long> onwSoNs[16];
++std::atomic<long long> onwSoN[16];
++struct OnwSoT {
++    int i; std::chrono::steady_clock::time_point t0;
++    explicit OnwSoT(int ii): i(ii), t0(std::chrono::steady_clock::now()) {}
++    ~OnwSoT() { onwSoNs[i] += std::chrono::duration_cast<std::chrono::nanoseconds>(std::chrono::steady_clock::now() - t0).count(); ++onwSoN[i]; }
++};
++}  // namespace
+ 
+ using namespace vpux;
+ using namespace VPU;
+@@ -55,6 +72,7 @@ bool SubgraphOptimizer::isStrategySOHLike(VPU::ClusteredOpInterface op) {
+ }
+ 
+ bool SubgraphOptimizer::isValidStrategy(VPU::ClusteredOpInterface clusteredOp, VPU::MultiClusterStrategy strategy) {
++    OnwSoT onwT8(8);
+     auto moduleOp = clusteredOp->getParentOfType<mlir::ModuleOp>();
+     auto tileOp = config::getTileExecutor(moduleOp);
+     const auto numTiles = tileOp.getCount();
+@@ -249,6 +267,7 @@ VPU::MultiClusterStrategy SubgraphOptimizer::getBestInSOHLikeStrategies(VPU::Clu
+ double SubgraphOptimizer::getInputSpillingCostToMultiClusterLayer(VPU::ClusteredOpInterface clusteredOp,
+                                                                   VPU::MultiClusterStrategy strategy,
+                                                                   const SubgraphOptConfig& config) {
++    OnwSoT onwT9(9);
+     return llvm::TypeSwitch<mlir::Operation*, double>(clusteredOp.getOperation())
+             .Case<NCEMaxPoolOp>([&](NCEMaxPoolOp origOp) {
+                 return getInputSpillingCostToMultiClusterLayer(origOp, origOp.getInput(), strategy, config);
+@@ -351,6 +370,7 @@ double SubgraphOptimizer::getInputSpillingCostToMultiClusterLayer(VPU::Clustered
+ double SubgraphOptimizer::getOutputSpillingCostToMultiClusterLayer(VPU::ClusteredOpInterface clusteredOp,
+                                                                    VPU::MultiClusterStrategy strategy,
+                                                                    const SubgraphOptConfig& config) {
++    OnwSoT onwT10(10);
+     bool hasCalculatedSpillingWriteCost = false;
+     double totalSpillingCost = 0.0;
+     // Iterate users in stable program order: the write cost is accumulated only for the first user
+@@ -380,8 +400,10 @@ double SubgraphOptimizer::getOutputSpillingCostToMultiClusterLayer(VPU::Clustere
+             auto userStrategy = config.useRollbackStrategy
+                                         ? getRollbackStrategy(userClusteredOp)
+                                         : _layerCostModel.getMultiClusterStrategyValue(userClusteredOp);
++            std::optional<OnwSoT> onwCs(std::in_place, 13);
+             auto spillingCost =
+                     _layerCostModel.calculateSpillingCost(clusteredOp, userClusteredOp, strategy, userStrategy);
++            onwCs.reset();
+             if (!hasCalculatedSpillingWriteCost) {
+                 totalSpillingCost += spillingCost.writeCost;
+                 hasCalculatedSpillingWriteCost = true;
+@@ -802,6 +824,7 @@ bool SubgraphOptimizer::hasInputSpillingToMultiClusterLayer(VPU::ClusteredOpInte
+ bool SubgraphOptimizer::hasInputSpillingToMultiClusterLayer(VPU::ClusteredOpInterface origClusteredOp,
+                                                             mlir::Value input, VPU::MultiClusterStrategy origStrategy,
+                                                             const SubgraphOptConfig& config) {
++    OnwSoT onwT12(12);
+     if (mlir::isa<mlir::BlockArgument>(input)) {
+         return false;
+     }
+@@ -891,6 +914,7 @@ bool SubgraphOptimizer::hasOutputSpillingToMultiClusterLayer(VPU::ClusteredOpInt
+                                                              mlir::Operation* userOp,
+                                                              VPU::MultiClusterStrategy origStrategy,
+                                                              const SubgraphOptConfig& config) {
++    OnwSoT onwT11(11);
+     auto propagateShapeCast = false;
+ 
+     if (mlir::isa<VPU::QuantizeCastOp, VPU::GroupSparseTensorOp>(userOp)) {
+@@ -1010,6 +1034,8 @@ bool SubgraphOptimizer::hasOutputSpillingToMultiClusterLayer(VPU::ClusteredOpInt
+ /// 4. {SOK -> SOH -> SOK} convert to {SOH -> HK -> SOK}
+ /// @todo This algorithm is a local strategy optimization for subgraphs. The final strategy maybe not global optimized.
+ /// For example, {SOK -> SOK -> SOH} maybe better converting to {SOK -> SOK -> SOK}
++
++
+ void SubgraphOptimizer::optimizeStrategyAvoidSpillingOnSubgraph(VPU::ClusteredOpInterface clusteredOp) {
+     // #E112083 performance regression to be investigated in order to remove this filter
+     // TODO: [E-126102] Cost model for Grouped MatMul
+@@ -1021,6 +1047,8 @@ void SubgraphOptimizer::optimizeStrategyAvoidSpillingOnSubgraph(VPU::ClusteredOp
+     // As a result we specifically handle it for Runtime Dequant cases. Details : E#162100
+     // Non-VF cost calculated here chooses misaligned strategy. This workaround can be removed when VF can
+     // align MC strategies itself. E#148750
++    OnwSoT onwAll(0);
++    std::optional<OnwSoT> onwPre(std::in_place, 1);
+     if (alignStrategyWithParentRuntimeDequant(clusteredOp, _layerCostModel)) {
+         _log.debug("[Subgraph Optimizer] Set SOK to Runtime Dequant Convolution to fix weights spilling");
+     }
+@@ -1042,6 +1070,8 @@ void SubgraphOptimizer::optimizeStrategyAvoidSpillingOnSubgraph(VPU::ClusteredOp
+         return;
+     }
+ 
++    onwPre.reset();
++    OnwSoT onwBfs(2);
+     _log.debug("[Subgraph Optimizer] Begin Op {0}", clusteredOp->getLoc());
+ 
+     layersNeedRollback.clear();
+@@ -1089,8 +1119,12 @@ void SubgraphOptimizer::optimizeStrategyAvoidSpillingOnSubgraph(VPU::ClusteredOp
+         currentOp = layersNeedRollback.front();
+         auto clusteredCurrentOp = mlir::dyn_cast<ClusteredOpInterface>(currentOp.getOperation());
+         layersNeedRollback.pop_front();
+-        rollbackStrategy =
+-                rollbackToSOH ? getBestInSOHLikeStrategies(currentOp) : getBestInSOKLikeStrategies(currentOp);
++        ++onwSoN[3];
++        {
++            OnwSoT onwB(4);
++            rollbackStrategy =
++                    rollbackToSOH ? getBestInSOHLikeStrategies(currentOp) : getBestInSOKLikeStrategies(currentOp);
++        }
+         // Sometimes a layer triggers rollback several times in a subgraph because we allows rollback strategy be
+         // changed during optimization. It makes sense when we find rollback strategy A first but later a better
+         // rollback strategy B is found. However if B is equal to A, it means the rollback strategy is converged. We
+@@ -1109,6 +1143,7 @@ void SubgraphOptimizer::optimizeStrategyAvoidSpillingOnSubgraph(VPU::ClusteredOp
+         // Check if the child layer needs rollback strategy.
+         // Iterate users in stable program order so the rollback BFS exploration (and the resulting
+         // strategies) does not depend on use-list ordering.
++        std::optional<OnwSoT> onwCh(std::in_place, 6);
+         for (auto* directChild : usersInProgramOrder(currentOp.getOperation())) {
+             auto executeChild = directChild;
+             while (mlir::isa_and_nonnull<VPU::GroupSparseTensorOp>(executeChild) ||
+@@ -1152,6 +1187,8 @@ void SubgraphOptimizer::optimizeStrategyAvoidSpillingOnSubgraph(VPU::ClusteredOp
+             }
+         }
+ 
++        onwCh.reset();
++        OnwSoT onwPa(7);
+         // Check if the parent layer needs rollback strategy
+         SmallVector<mlir::Value> layerInputs;
+         if (auto eltwiseOp = mlir::dyn_cast<VPU::NCEEltwiseOp>(currentOp.getOperation())) {
+@@ -1210,6 +1247,7 @@ void SubgraphOptimizer::optimizeStrategyAvoidSpillingOnSubgraph(VPU::ClusteredOp
+         }
+     }
+ 
++    OnwSoT onwAfter(5);
+     // Calculate original cost and rollback cost
+     double originalCost = 0;
+     double rollbackCost = 0;
+@@ -1447,7 +1485,9 @@ void SubgraphOptimizer::optimizeStrategyAvoidSpillingOnModel() {
+     // Assure Layer cost only include layer DPU cost under this phase. Exclude layer activation DMAs
+     _layerCostModel.setUnderSubgraphOpt(true);
+     // detect shortcuts in model like resnet
++    const auto onwS0 = std::chrono::steady_clock::now();
+     detectShortcuts();
++    const auto onwS1 = std::chrono::steady_clock::now();
+ 
+     const auto callback = [this](VPU::ClusteredOpInterface clusteredOp) {
+         if (auto swOp = mlir::dyn_cast_or_null<VPU::SWOpInterface>(clusteredOp.getOperation())) {
+@@ -1465,6 +1505,21 @@ void SubgraphOptimizer::optimizeStrategyAvoidSpillingOnModel() {
+ 
+     /// @brief Traversing nodes with preOrder to execute subgraph optimization
+     _func.walk(callback);
++    const auto onwS2 = std::chrono::steady_clock::now();
++    if (const char* onwP = std::getenv("ONW_PROF_FILE")) {
++        if (FILE* f = std::fopen(onwP, "a")) {
++            std::fprintf(f, "subgraph detectShortcuts %.2f s avoidSpilling walk %.2f s | per-op all %lld %.2f s, pre-checks %lld %.2f s, rollback search %lld %.2f s, bfs visits %lld, best-strategy %lld %.2f s, after-bfs %lld %.2f s, children %.2f s, parents %.2f s, isValid %lld %.2f s, inSpill %lld %.2f s, outSpill %lld %.2f s, hasOut %lld %.2f s, hasIn %lld %.2f s, calcSpill(out) %lld %.2f s%c",
++                         std::chrono::duration<double>(onwS1 - onwS0).count(),
++                         std::chrono::duration<double>(onwS2 - onwS1).count(),
++                         (long long)onwSoN[0].load(), onwSoNs[0].load() / 1e9, (long long)onwSoN[1].load(), onwSoNs[1].load() / 1e9,
++                         (long long)onwSoN[2].load(), onwSoNs[2].load() / 1e9, (long long)onwSoN[3].load(),
++                         (long long)onwSoN[4].load(), onwSoNs[4].load() / 1e9, (long long)onwSoN[5].load(), onwSoNs[5].load() / 1e9, onwSoNs[6].load() / 1e9, onwSoNs[7].load() / 1e9,
++                         (long long)onwSoN[8].load(), onwSoNs[8].load() / 1e9, (long long)onwSoN[9].load(), onwSoNs[9].load() / 1e9,
++                         (long long)onwSoN[10].load(), onwSoNs[10].load() / 1e9, (long long)onwSoN[11].load(), onwSoNs[11].load() / 1e9,
++                         (long long)onwSoN[12].load(), onwSoNs[12].load() / 1e9, (long long)onwSoN[13].load(), onwSoNs[13].load() / 1e9, 10);
++            std::fclose(f);
++        }
++    }
+ 
+     const auto callbackForHK = [this](VPU::ClusteredOpInterface clusteredOp) {
+         auto currentStrategyAttr = clusteredOp.getMultiClusterStrategy();
+diff --git a/src/vpux_compiler/src/dialect/VPU/utils/vertical_fusion/v1/merge_vf_region_rewriter.cpp b/src/vpux_compiler/src/dialect/VPU/utils/vertical_fusion/v1/merge_vf_region_rewriter.cpp
+index 4207b33..7c9f3d1 100644
+--- a/src/vpux_compiler/src/dialect/VPU/utils/vertical_fusion/v1/merge_vf_region_rewriter.cpp
++++ b/src/vpux_compiler/src/dialect/VPU/utils/vertical_fusion/v1/merge_vf_region_rewriter.cpp
+@@ -17,6 +17,40 @@
+ 
+ #include <deque>
+ 
++#include <atomic>
++#include <chrono>
++#include <cstdio>
++#include <cstdlib>
++
++namespace {
++// [onw] profiling of the merge attempts (ONW_PROF_FILE=path: totals appended at exit)
++struct OnwVfProf {
++    std::atomic<long long> ns[5]{};
++    std::atomic<long long> n[5]{};
++    const char* tag;
++    explicit OnwVfProf(const char* t): tag(t) {}
++    ~OnwVfProf() {
++        const char* p = std::getenv("ONW_PROF_FILE");
++        if (p == nullptr) return;
++        if (FILE* f = std::fopen(p, "a")) {
++            const char* names[5] = {"match", "fuseOpsInBlock", "findVFCase", "checkVFCost", "fuseBlocks"};
++            for (int i = 0; i < 5; ++i)
++                std::fprintf(f, "%s %s: %lld calls %.2f s%c", tag, names[i], (long long)n[i].load(), ns[i].load() / 1e9, 10);
++            std::fclose(f);
++        }
++    }
++};
++struct OnwVfTimer {
++    OnwVfProf& p; int i; std::chrono::steady_clock::time_point t0;
++    OnwVfTimer(OnwVfProf& pp, int ii): p(pp), i(ii), t0(std::chrono::steady_clock::now()) {}
++    ~OnwVfTimer() {
++        p.ns[i] += std::chrono::duration_cast<std::chrono::nanoseconds>(std::chrono::steady_clock::now() - t0).count();
++        ++p.n[i];
++    }
++};
++OnwVfProf onwVfProf("v1");
++}  // namespace
++
+ namespace vpux::VPU::VF::v1 {
+ 
+ namespace {
+@@ -674,6 +708,7 @@ bool MergeVFRegionRewriter::checkVFCostFunction(VPU::VerticalFusionOp prevOp, VP
+ 
+ mlir::LogicalResult MergeVFRegionRewriter::matchAndRewrite(VPU::VerticalFusionOp vfOp,
+                                                            mlir::PatternRewriter& rewriter) const {
++    OnwVfTimer onwT0(onwVfProf, 0);
+     _log.trace("Starting vertical fusion for region with VerticalFusionOp {0} at location {1}", vfOp, vfOp->getLoc());
+     if (vfOp.getIsManualConfigured()) {
+         _log.trace("Skipping vertical fusion for manually configured VerticalFusionOp at location {0}", vfOp->getLoc());
+@@ -709,7 +744,10 @@ mlir::LogicalResult MergeVFRegionRewriter::matchAndRewrite(VPU::VerticalFusionOp
+             continue;
+         }
+ 
+-        vfBlock = fuseOpsInBlock(rewriter, vfOp, parentVFOp.getOperation());
++        {
++            OnwVfTimer onwT(onwVfProf, 1);
++            vfBlock = fuseOpsInBlock(rewriter, vfOp, parentVFOp.getOperation());
++        }
+         // `findVFCase` internally compares operations using IR order which is calculated "on the fly" in e.g.
+         // `isBeforeInBlock()`
+         //  - order recalculation modifies the state and, due to multi-threaded environment, results in a data race
+@@ -718,10 +756,25 @@ mlir::LogicalResult MergeVFRegionRewriter::matchAndRewrite(VPU::VerticalFusionOp
+         // As `findVFCase` is expected to not modify the IR, what can be done is an eager recalculation
+         // of the operation orders that would ensure later calls to e.g. `isBeforeInBlock()` become concurrent
+         // reads instead of concurrent writes (which is no longer a data race).
+-        vfBlock->getBlock()->recomputeOpOrder();
+-        vfBlock.getBody()->recomputeOpOrder();
+-        auto vfCase = findVFCase(parentVFOp, vfOp, vfBlock);
+-        if (!vfCase.has_value() || !checkVFCostFunction(parentVFOp, vfOp, vfCase.value())) {
++        // [onw] only when the order is invalid (see v2): renumbering the whole function block on every candidate made the
++        // pass quadratic in the op count
++        if (!vfBlock->getBlock()->isOpOrderValid()) {
++            vfBlock->getBlock()->recomputeOpOrder();
++        }
++        if (!vfBlock.getBody()->isOpOrderValid()) {
++            vfBlock.getBody()->recomputeOpOrder();
++        }
++        std::optional<VFCase> vfCase;
++        {
++            OnwVfTimer onwT(onwVfProf, 2);
++            vfCase = findVFCase(parentVFOp, vfOp, vfBlock);
++        }
++        bool onwCostOk = false;
++        if (vfCase.has_value()) {
++            OnwVfTimer onwT(onwVfProf, 3);
++            onwCostOk = checkVFCostFunction(parentVFOp, vfOp, vfCase.value());
++        }
++        if (!vfCase.has_value() || !onwCostOk) {
+             // Drop all references to vfBlock to avoid it being added back to the rewriter
+             // worklist.
+             vfBlock->dropAllReferences();
+@@ -743,7 +796,10 @@ mlir::LogicalResult MergeVFRegionRewriter::matchAndRewrite(VPU::VerticalFusionOp
+     }
+ 
+     _log.trace("Merged subgraph {0}", vfBlock);
+-    fuseBlocks(rewriter, vfOp, vfBlock);
++    {
++        OnwVfTimer onwT(onwVfProf, 4);
++        fuseBlocks(rewriter, vfOp, vfBlock);
++    }
+ 
+     return mlir::success();
+ }
+diff --git a/src/vpux_compiler/src/dialect/VPU/utils/vertical_fusion/v2/merge_vf_region_rewriter.cpp b/src/vpux_compiler/src/dialect/VPU/utils/vertical_fusion/v2/merge_vf_region_rewriter.cpp
+index f3d8660..2e84289 100644
+--- a/src/vpux_compiler/src/dialect/VPU/utils/vertical_fusion/v2/merge_vf_region_rewriter.cpp
++++ b/src/vpux_compiler/src/dialect/VPU/utils/vertical_fusion/v2/merge_vf_region_rewriter.cpp
+@@ -28,6 +28,43 @@
+ #include <mlir/IR/IRMapping.h>
+ #include <mlir/IR/Value.h>
+ 
++#include <atomic>
++#include <string>
++#include <unordered_set>
++#include <mutex>
++#include <chrono>
++#include <cstdio>
++#include <cstdlib>
++
++namespace {
++// [onw] profiling of the merge attempts (ONW_PROF_FILE=path: totals appended at exit)
++struct OnwVfProf {
++    std::atomic<long long> ns[20]{};
++    std::atomic<long long> n[20]{};
++    const char* tag;
++    explicit OnwVfProf(const char* t): tag(t) {}
++    ~OnwVfProf() {
++        const char* p = std::getenv("ONW_PROF_FILE");
++        if (p == nullptr) return;
++        if (FILE* f = std::fopen(p, "a")) {
++            const char* names[20] = {"match", "fuseOpsInBlock", "findVFCase", "checkVFCost", "fuseBlocks", "isLegalFusion", "isMCAligned", "VFConfig x2", "policy.match", "adjustMC", "findVFTiling", "snapshot/restore", "tiling none fail", "tiling none ok", "tiling one fail", "tiling one ok", "tiling both fail", "tiling both ok", "skipMerge", "negCacheHit"};
++            for (int i = 0; i < 20; ++i)
++                std::fprintf(f, "%s %s: %lld calls %.2f s%c", tag, names[i], (long long)n[i].load(), ns[i].load() / 1e9, 10);
++            std::fclose(f);
++        }
++    }
++};
++struct OnwVfTimer {
++    OnwVfProf& p; int i; std::chrono::steady_clock::time_point t0;
++    OnwVfTimer(OnwVfProf& pp, int ii): p(pp), i(ii), t0(std::chrono::steady_clock::now()) {}
++    ~OnwVfTimer() {
++        p.ns[i] += std::chrono::duration_cast<std::chrono::nanoseconds>(std::chrono::steady_clock::now() - t0).count();
++        ++p.n[i];
++    }
++};
++OnwVfProf onwVfProf("v2");
++}  // namespace
++
+ namespace vpux::VPU::VF::v2 {
+ 
+ namespace {
+@@ -241,11 +278,20 @@ bool MergeVFRegionRewriter::isMCStrategyAligned(VPU::VerticalFusionOp currentOp,
+ 
+ std::optional<VFCase> MergeVFRegionRewriter::findVFCase(VPU::VerticalFusionOp prevOp, VPU::VerticalFusionOp currentOp,
+                                                         VPU::VerticalFusionOp mergedVFOp) const {
+-    if (!isLegalFusion(currentOp, prevOp)) {
++    bool onwLegal;
++    {
++        OnwVfTimer onwT(onwVfProf, 5);
++        onwLegal = isLegalFusion(currentOp, prevOp);
++    }
++    if (!onwLegal) {
+         _log.trace("Illegal fusion between {0} and {1}", currentOp->getLoc(), prevOp->getLoc());
+         return std::nullopt;
+     }
+-    const auto isMCStrategyAlreadyAligned = isMCStrategyAligned(currentOp, prevOp);
++    bool isMCStrategyAlreadyAligned;
++    {
++        OnwVfTimer onwT(onwVfProf, 6);
++        isMCStrategyAlreadyAligned = isMCStrategyAligned(currentOp, prevOp);
++    }
+     const auto supportedPolicyTypes = getSupportedMergeTilingPolicyTypes();
+ 
+     for (const auto& tilingPolicy : _tilingPolicies) {
+@@ -253,20 +299,47 @@ std::optional<VFCase> MergeVFRegionRewriter::findVFCase(VPU::VerticalFusionOp pr
+             continue;
+         }
+ 
+-        VFConfig prevConfig(prevOp, _cache, _enableVerticalFusionPipelining);
+-        VFConfig currentConfig(currentOp, _cache, _enableVerticalFusionPipelining);
+-        if (!tilingPolicy->match(prevConfig, currentConfig)) {
++        std::optional<VFConfig> prevConfigO, currentConfigO;
++        {
++            OnwVfTimer onwT(onwVfProf, 7);
++            prevConfigO.emplace(prevOp, _cache, _enableVerticalFusionPipelining);
++            currentConfigO.emplace(currentOp, _cache, _enableVerticalFusionPipelining);
++        }
++        auto& prevConfig = *prevConfigO;
++        auto& currentConfig = *currentConfigO;
++        bool onwMatch;
++        {
++            OnwVfTimer onwT(onwVfProf, 8);
++            onwMatch = tilingPolicy->match(prevConfig, currentConfig);
++        }
++        if (!onwMatch) {
+             continue;
+         }
+ 
+-        auto mcStrategyAttrSnapshot = snapshotMCStrategyAttrs(mergedVFOp);
+-        if (isMCStrategyAlreadyAligned || tilingPolicy->adjustMCStrategyInMergedVF(currentOp, prevOp, mergedVFOp)) {
+-            auto mergedCase = findVFTiling(mergedVFOp, prevOp, currentOp, *tilingPolicy);
++        std::optional<decltype(snapshotMCStrategyAttrs(mergedVFOp))> snapO;
++        {
++            OnwVfTimer onwT(onwVfProf, 11);
++            snapO.emplace(snapshotMCStrategyAttrs(mergedVFOp));
++        }
++        bool onwAdj = isMCStrategyAlreadyAligned;
++        if (!onwAdj) {
++            OnwVfTimer onwT(onwVfProf, 9);
++            onwAdj = tilingPolicy->adjustMCStrategyInMergedVF(currentOp, prevOp, mergedVFOp);
++        }
++        if (onwAdj) {
++            std::optional<VFCase> mergedCase;
++            {
++                OnwVfTimer onwT(onwVfProf, 10);
++                mergedCase = findVFTiling(mergedVFOp, prevOp, currentOp, *tilingPolicy);
++            }
+             if (mergedCase.has_value()) {
+                 return mergedCase;
+             }
+         }
+-        restoreMCStrategyAttrs(mcStrategyAttrSnapshot);
++        {
++            OnwVfTimer onwT(onwVfProf, 11);
++            restoreMCStrategyAttrs(*snapO);
++        }
+     }
+ 
+     if (!isMCStrategyAlreadyAligned) {
+@@ -325,6 +398,19 @@ MergeVFRegionRewriter::IVFSchedulingPtr MergeVFRegionRewriter::detectScenario(VF
+ std::optional<VFCase> MergeVFRegionRewriter::findVFTiling(VPU::VerticalFusionOp mergedOp, VPU::VerticalFusionOp prevOp,
+                                                           VPU::VerticalFusionOp currentOp,
+                                                           const MergeTilingPolicy& tilingPolicy) const {
++    const auto t0 = std::chrono::steady_clock::now();
++    const int cls = (hasTiling(parseIntArrayAttr<int64_t>(currentOp.getTilingStrategy())) ? 1 : 0) +
++                    (hasTiling(parseIntArrayAttr<int64_t>(prevOp.getTilingStrategy())) ? 1 : 0);
++    auto r = findVFTilingImpl(mergedOp, prevOp, currentOp, tilingPolicy);
++    const int slot = 12 + cls * 2 + (r.has_value() ? 1 : 0);
++    onwVfProf.ns[slot] += std::chrono::duration_cast<std::chrono::nanoseconds>(std::chrono::steady_clock::now() - t0).count();
++    ++onwVfProf.n[slot];
++    return r;
++}
++
++std::optional<VFCase> MergeVFRegionRewriter::findVFTilingImpl(VPU::VerticalFusionOp mergedOp, VPU::VerticalFusionOp prevOp,
++                                                          VPU::VerticalFusionOp currentOp,
++                                                          const MergeTilingPolicy& tilingPolicy) const {
+     const auto currentTiling = parseIntArrayAttr<int64_t>(currentOp.getTilingStrategy());
+     const auto prevTiling = parseIntArrayAttr<int64_t>(prevOp.getTilingStrategy());
+ 
+@@ -345,6 +431,7 @@ std::optional<VFCase> MergeVFRegionRewriter::findVFTiling(VPU::VerticalFusionOp
+ 
+     bool opsNeedTiling = prevHasTiling || curHasTiling;
+     if (canSkipMergeVF(vfConfig, opsNeedTiling)) {
++        ++onwVfProf.n[18];
+         return std::nullopt;
+     }
+ 
+@@ -543,8 +630,61 @@ bool MergeVFRegionRewriter::checkVFCostFunction(VPU::VerticalFusionOp prevOp, VP
+     return true;
+ }
+ 
++
++namespace {
++// [onw] Structural signature of a VF candidate pair: the ops of both regions (names, attributes, operand / result
++// types with quantized element types reduced to their storage type) and the link between them. Group-unrolled
++// MatMuls (2-bit / 4-bit group-quantized weights) produce long runs of identically shaped VF regions; the same merge
++// was searched (tiling + VPUNN cost, ~15 ms) and rejected thousands of times. Rejections are remembered per
++// signature (ONW_VF_NEG_CACHE=0: off).
++llvm::hash_code onwHashType(mlir::Type t) {
++    if (auto nd = mlir::dyn_cast<vpux::NDTypeInterface>(t)) {
++        auto elem = nd.getElementType();
++        if (auto q = mlir::dyn_cast<mlir::quant::QuantizedType>(elem)) {
++            elem = q.getStorageType();
++        }
++        const auto shape = nd.getShape().raw();
++        return llvm::hash_combine(llvm::hash_combine_range(shape.begin(), shape.end()), elem.getAsOpaquePointer(),
++                                  nd.getDimsOrder().code(), nd.getMemSpace().getAsOpaquePointer(),
++                                  t.getTypeID().getAsOpaquePointer());
++    }
++    return llvm::hash_value(t.getAsOpaquePointer());
++}
++
++llvm::hash_code onwHashVF(VPU::VerticalFusionOp op) {
++    llvm::hash_code h = llvm::hash_value(op->getAttrDictionary().getAsOpaquePointer());
++    for (auto v : op->getOperands()) {
++        h = llvm::hash_combine(h, onwHashType(v.getType()));
++    }
++    op.getBody()->walk([&](mlir::Operation* o) {
++        h = llvm::hash_combine(h, o->getName().getAsOpaquePointer(), o->getAttrDictionary().getAsOpaquePointer());
++        for (auto v : o->getOperands()) {
++            h = llvm::hash_combine(h, onwHashType(v.getType()));
++        }
++        for (auto t : o->getResultTypes()) {
++            h = llvm::hash_combine(h, onwHashType(t));
++        }
++    });
++    return h;
++}
++
++struct OnwVfNegCache {
++    std::mutex mutex;
++    std::unordered_set<size_t> keys;
++};
++OnwVfNegCache& onwVfNegCache() {
++    static OnwVfNegCache c;
++    return c;
++}
++bool onwNegCacheOn() {
++    static const bool on = std::getenv("ONW_VF_NEG_CACHE") == nullptr || std::string(std::getenv("ONW_VF_NEG_CACHE")) != "0";
++    return on;
++}
++}  // namespace
++
+ mlir::LogicalResult MergeVFRegionRewriter::matchAndRewrite(VPU::VerticalFusionOp vfOp,
+                                                            mlir::PatternRewriter& rewriter) const {
++    OnwVfTimer onwT0(onwVfProf, 0);
+     _log.trace("Starting vertical fusion for region with VerticalFusionOp {0} at location {1}", vfOp, vfOp->getLoc());
+     if (vfOp.getIsManualConfigured()) {
+         _log.trace("Skipping vertical fusion for manually configured VerticalFusionOp at location {0}", vfOp->getLoc());
+@@ -553,13 +693,27 @@ mlir::LogicalResult MergeVFRegionRewriter::matchAndRewrite(VPU::VerticalFusionOp
+ 
+     VPU::VerticalFusionOp vfBlock = nullptr;
+     VPU::VerticalFusionOp parentVFOp = nullptr;
++    unsigned onwIdx = 0;
+     for (auto operand : vfOp->getOperands()) {
++        const unsigned onwOperandIdx = onwIdx++;
+         parentVFOp = operand.getDefiningOp<VPU::VerticalFusionOp>();
+         vfBlock = nullptr;
+ 
+         if (parentVFOp == nullptr || parentVFOp.getIsManualConfigured()) {
+             continue;
+         }
++        size_t onwKey = 0;
++        if (onwNegCacheOn()) {
++            onwKey = llvm::hash_combine(onwHashVF(parentVFOp), onwHashVF(vfOp), onwOperandIdx);
++            std::lock_guard<std::mutex> lock(onwVfNegCache().mutex);
++            if (onwVfNegCache().keys.count(onwKey) != 0) {
++                ++onwVfProf.n[19];
++                if (checkOtherVFInput(vfOp, parentVFOp)) {
++                    continue;
++                }
++                return mlir::failure();
++            }
++        }
+ 
+         _log.trace("Analyzing vertical fusion region with parent VerticalFusionOp {0} at location {1}", parentVFOp,
+                    parentVFOp->getLoc());
+@@ -580,7 +734,10 @@ mlir::LogicalResult MergeVFRegionRewriter::matchAndRewrite(VPU::VerticalFusionOp
+             continue;
+         }
+ 
+-        vfBlock = fuseOpsInBlock(rewriter, vfOp, parentVFOp.getOperation());
++        {
++            OnwVfTimer onwT(onwVfProf, 1);
++            vfBlock = fuseOpsInBlock(rewriter, vfOp, parentVFOp.getOperation());
++        }
+         _log.trace("Merged subgraph candidate {0}", vfBlock.getLoc());
+         // `findVFCase` internally compares operations using IR order which is calculated "on the fly" in e.g.
+         // `isBeforeInBlock()`
+@@ -590,15 +747,34 @@ mlir::LogicalResult MergeVFRegionRewriter::matchAndRewrite(VPU::VerticalFusionOp
+         // As `findVFCase` is expected to not modify the IR, what can be done is an eager recalculation
+         // of the operation orders that would ensure later calls to e.g. `isBeforeInBlock()` become concurrent
+         // reads instead of concurrent writes (which is no longer a data race).
+-        vfBlock->getBlock()->recomputeOpOrder();
+-        vfBlock.getBody()->recomputeOpOrder();
+-        auto vfCase = findVFCase(parentVFOp, vfOp, vfBlock);
+-        if (!vfCase.has_value() || !checkVFCostFunction(parentVFOp, vfOp, vfCase.value())) {
++        // [onw] only when the order is invalid: renumbering the whole function block on every candidate made the pass
++        // quadratic in the op count (Bonsai 2-bit group-unrolled graphs: ~60% of the compile time)
++        if (!vfBlock->getBlock()->isOpOrderValid()) {
++            vfBlock->getBlock()->recomputeOpOrder();
++        }
++        if (!vfBlock.getBody()->isOpOrderValid()) {
++            vfBlock.getBody()->recomputeOpOrder();
++        }
++        std::optional<VFCase> vfCase;
++        {
++            OnwVfTimer onwT(onwVfProf, 2);
++            vfCase = findVFCase(parentVFOp, vfOp, vfBlock);
++        }
++        bool onwCostOk = false;
++        if (vfCase.has_value()) {
++            OnwVfTimer onwT(onwVfProf, 3);
++            onwCostOk = checkVFCostFunction(parentVFOp, vfOp, vfCase.value());
++        }
++        if (!vfCase.has_value() || !onwCostOk) {
+             // Drop all references to vfBlock to avoid it being added back to the rewriter
+             // worklist.
+             vfBlock->dropAllReferences();
+             rewriter.eraseOp(vfBlock);
+             vfBlock = nullptr;
++            if (onwNegCacheOn()) {
++                std::lock_guard<std::mutex> lock(onwVfNegCache().mutex);
++                onwVfNegCache().keys.insert(onwKey);
++            }
+             if (checkOtherVFInput(vfOp, parentVFOp)) {
+                 continue;
+             }
+@@ -614,7 +790,10 @@ mlir::LogicalResult MergeVFRegionRewriter::matchAndRewrite(VPU::VerticalFusionOp
+     }
+ 
+     _log.trace("Merged subgraph {0}", vfBlock->getLoc());
+-    fuseBlocks(rewriter, vfOp, vfBlock);
++    {
++        OnwVfTimer onwT(onwVfProf, 4);
++        fuseBlocks(rewriter, vfOp, vfBlock);
++    }
+ 
+     return mlir::success();
+ }
+diff --git a/src/vpux_compiler/src/dialect/VPU/utils/weights_separation.cpp b/src/vpux_compiler/src/dialect/VPU/utils/weights_separation.cpp
+index 35a6c00..d48e08c 100644
+--- a/src/vpux_compiler/src/dialect/VPU/utils/weights_separation.cpp
++++ b/src/vpux_compiler/src/dialect/VPU/utils/weights_separation.cpp
+@@ -4,6 +4,7 @@
+ //
+ 
+ #include "vpux/compiler/dialect/VPU/utils/weights_separation.hpp"
++#include "vpux/compiler/core/types/quantile_float/types.hpp"
+ #include "vpux/compiler/core/force_link_macros.hpp"
+ #include "vpux/compiler/dialect/IE/IR/ops/data_movement.hpp"
+ #include "vpux/compiler/dialect/IE/IR/ops/data_type.hpp"
+@@ -345,6 +346,9 @@ mlir::Value createAvgPoolForInterQuantizedConvert(mlir::OpBuilder& builder, mlir
+ /// Returns an IE operation for the given constant transformation.
+ mlir::Value createMatchingIeOperation(mlir::OpBuilder& builder, mlir::Location loc, mlir::Value input,
+                                       Const::TransformAttrInterface t) {
++    if (std::getenv("ONW_DEBUG_WS") != nullptr) {
++        llvm::errs() << "ONW ws transform: in " << input.getType() << " t " << t << char(10);
++    }
+     return llvm::TypeSwitch<Const::TransformAttrInterface, mlir::Value>(t)
+             .Case<Const::AddAttr>([&](Const::AddAttr add) {
+                 const auto biasValue = checked_cast<float>(add.getBias().getValueAsDouble());
+@@ -398,15 +402,94 @@ mlir::Value createMatchingIeOperation(mlir::OpBuilder& builder, mlir::Location l
+                 const auto inElemType = mlir::cast<NDTypeInterface>(input.getType()).getElementType();
+                 const auto outElemType = cast.getElemType();
+ 
+-                const bool inputQuantized = mlir::isa<mlir::quant::QuantizedType>(inElemType);
+-                const bool outputQuantized = mlir::isa<mlir::quant::QuantizedType>(outElemType);
++                // onw: quantile (NF4) element types are a storage reinterpretation too, like quantized ones
++                const bool inputQuantized = mlir::isa<mlir::quant::QuantizedType, vpux::type::QuantileType>(inElemType);
++                const bool outputQuantized =
++                        mlir::isa<mlir::quant::QuantizedType, vpux::type::QuantileType>(outElemType);
+ 
++                const auto hasQuantileStorage = [](mlir::Type t) {
++                    if (auto q = mlir::dyn_cast<mlir::quant::QuantizedType>(t)) {
++                        t = q.getStorageType();
++                    }
++                    return mlir::isa<vpux::type::QuantileType>(t);
++                };
++                if (hasQuantileStorage(inElemType) || hasQuantileStorage(outElemType)) {
++                    // onw: CastElemType keeps the stored values (NF4 table indices); go through the storage type:
++                    // quantile -> storage int (reinterpret), int/float conversions of the index values (via f16,
++                    // exact for 0..15), storage int -> quantile (reinterpret)
++                    auto storageOf = [](mlir::Type t) -> mlir::Type {
++                        // quant.uniform<quantile<ui4>> -> quantile<ui4> -> ui4
++                        for (int k = 0; k < 2; ++k) {
++                            if (auto q = mlir::dyn_cast<vpux::type::QuantileType>(t)) {
++                                t = q.getStorageType();
++                            } else if (auto q = mlir::dyn_cast<mlir::quant::QuantizedType>(t)) {
++                                t = q.getStorageType();
++                            }
++                        }
++                        return t;
++                    };
++                    // Index values only pass through value-preserving casts here (integer widening, int <-> f16,
++                    // storage reinterpretation). Walk back to the earliest value of this chain whose type is the
++                    // target or its storage type and reinterpret that directly - the greedy canonicalizer would
++                    // otherwise fold a Convert+QuantizeCast pair into an unsupported "Convert to quantile".
++                    {
++                        const auto targetStorage = storageOf(outElemType);
++                        mlir::Value origin = nullptr;
++                        for (mlir::Value v = input; v != nullptr;) {
++                            const auto vt = mlir::cast<NDTypeInterface>(v.getType()).getElementType();
++                            if (vt == outElemType || vt == targetStorage) {
++                                origin = v;
++                            }
++                            auto* def = v.getDefiningOp();
++                            if (!mlir::isa_and_nonnull<IE::ConvertOp, IE::QuantizeCastOp>(def)) {
++                                break;
++                            }
++                            v = def->getOperand(0);
++                        }
++                        if (origin != nullptr && mlir::cast<NDTypeInterface>(origin.getType()).getShape() ==
++                                                         mlir::cast<NDTypeInterface>(input.getType()).getShape()) {
++                            const auto ot = mlir::cast<NDTypeInterface>(origin.getType()).getElementType();
++                            return ot == outElemType ? origin
++                                                     : builder.create<IE::QuantizeCastOp>(loc, origin, outElemType)
++                                                               .getOutput();
++                        }
++                    }
++                    const auto f16 = mlir::Float16Type::get(builder.getContext());
++                    mlir::Value x = input;
++                    mlir::Type cur = inElemType;
++                    if (storageOf(cur) != cur) {
++                        cur = storageOf(cur);
++                        x = builder.create<IE::QuantizeCastOp>(appendLoc(loc, "onw_q_in"), x, cur);
++                    }
++                    const auto target = storageOf(outElemType);
++                    if (cur != target) {
++                        if (cur != f16 && target != f16 && !mlir::isa<mlir::FloatType>(cur)) {
++                            x = builder.create<IE::ConvertOp>(appendLoc(loc, "onw_q_f16"), x, f16);
++                        }
++                        x = builder.create<IE::ConvertOp>(appendLoc(loc, "onw_q_cvt"), x, target);
++                    }
++                    if (target != outElemType) {
++                        x = builder.create<IE::QuantizeCastOp>(loc, x, outElemType);
++                    }
++                    return x;
++                }
+                 if (inputQuantized || outputQuantized) {
+                     return createTypeCorrectedQuantizedCast(builder, input, loc, inElemType, outElemType);
+                 }
+                 return builder.create<IE::ConvertOp>(loc, input, outElemType);
+             })
+             .Case<Const::ConvertElemTypeAttr>([&](Const::ConvertElemTypeAttr convert) -> mlir::Value {
++                {
++                    // onw: storage integer <-> quantile (NF4) of the same width is a reinterpretation
++                    const auto inT = mlir::cast<NDTypeInterface>(input.getType()).getElementType();
++                    const auto outT = convert.getElemType();
++                    const auto inQ = mlir::dyn_cast<vpux::type::QuantileType>(inT);
++                    const auto outQ = mlir::dyn_cast<vpux::type::QuantileType>(outT);
++                    if ((outQ != nullptr && outQ.getStorageType() == inT) ||
++                        (inQ != nullptr && inQ.getStorageType() == outT)) {
++                        return builder.create<IE::QuantizeCastOp>(loc, input, outT);
++                    }
++                }
+                 const auto inElemType = mlir::dyn_cast_or_null<mlir::quant::QuantizedType>(
+                         mlir::cast<NDTypeInterface>(input.getType()).getElementType());
+                 const auto outElemType = mlir::dyn_cast_or_null<mlir::quant::QuantizedType>(convert.getElemType());
+diff --git a/src/vpux_compiler/src/dialect/VPUIP/transforms/passes/convert_async_ops_to_tasks.cpp b/src/vpux_compiler/src/dialect/VPUIP/transforms/passes/convert_async_ops_to_tasks.cpp
+index f637bad..6c3db25 100644
+--- a/src/vpux_compiler/src/dialect/VPUIP/transforms/passes/convert_async_ops_to_tasks.cpp
++++ b/src/vpux_compiler/src/dialect/VPUIP/transforms/passes/convert_async_ops_to_tasks.cpp
+@@ -164,7 +164,40 @@ mlir::LogicalResult RemoveWait::matchAndRewrite(mlir::async::AwaitOp waitOp, OpA
+     // If you faced with "'async.await' has more than one consumer" make sure you add your layer
+     // into src/vpux_compiler/src/dialect/VPUIP/ops.cpp `redirectOpInterfacesForIE(...)`
+     VPUX_THROW_UNLESS(waitOp.getResult().hasOneUse(), "'async.await' doesn't have only one consumer");
+-    VPUX_THROW_UNLESS(mlir::isa<mlir::func::ReturnOp>(*waitOp.getResult().user_begin()),
++    if (!mlir::isa<mlir::func::ReturnOp>(*waitOp.getResult().user_begin())) {
++        if (const char* onwDump = std::getenv("ONW_DEBUG_ASYNC")) {
++            std::error_code ec;
++            llvm::raw_fd_ostream fo(onwDump, ec);
++            if (!ec) {
++                fo << "=== await user: " << **waitOp.getResult().user_begin() << char(10);
++                fo << "=== await: " << waitOp << char(10);
++                waitOp->getParentOfType<mlir::func::FuncOp>()->print(fo, mlir::OpPrintingFlags().enableDebugInfo(true, false));
++                fo << char(10);
++            }
++        }
++    }
++    // onw: a pure view (ShapeCast, PermuteCast, ...: no data movement) between the await and the return is fine as
++    // well - the 8.3 pipeline leaves such views of a function's result outside the async regions (Qwen3.5 4-token
++    // blocks, Bonsai 4/8-token blocks: "'async.await' has non 'return' consumer")
++    const auto onlyViewsToReturn = [](mlir::Value v) {
++        while (true) {
++            if (!v.hasOneUse()) {
++                return false;
++            }
++            auto* user = *v.user_begin();
++            if (mlir::isa<mlir::func::ReturnOp>(user)) {
++                return true;
++            }
++            if (!mlir::isa<VPUIP::ShapeCastOp, VPUIP::PermuteCastOp, VPUIP::GenericReshapeOp, VPUIP::QuantizeCastOp,
++                           VPUIP::ViewOp>(user) ||
++                user->getNumResults() != 1) {
++                return false;
++            }
++            v = user->getResult(0);
++        }
++    };
++    VPUX_THROW_UNLESS(mlir::isa<mlir::func::ReturnOp>(*waitOp.getResult().user_begin()) ||
++                              onlyViewsToReturn(waitOp.getResult()),
+                       "'async.await' has non 'return' consumer");
+ 
+     rewriter.replaceOp(waitOp, newArgs.getOperand());
+diff --git a/src/vpux_compiler/src/dialect/VPUIP/transforms/passes/unroll_distributed_ops.cpp b/src/vpux_compiler/src/dialect/VPUIP/transforms/passes/unroll_distributed_ops.cpp
+index 23fe4fc..b93a938 100644
+--- a/src/vpux_compiler/src/dialect/VPUIP/transforms/passes/unroll_distributed_ops.cpp
++++ b/src/vpux_compiler/src/dialect/VPUIP/transforms/passes/unroll_distributed_ops.cpp
+@@ -1732,6 +1732,13 @@ void VPUIP::unrollDistributedOpsCommon40XXPlus(mlir::func::FuncOp func,
+     const VPUIP::ClusterNCERewriter nceRewriter(ctx, log);
+     const VPUIP::ClusterConvertDMARewriter convertDMARewriter(ctx, dmaPortCount, log);
+ 
++    if (const char* onwDump = std::getenv("ONW_DUMP_UNROLL")) {
++        std::error_code ec;
++        llvm::raw_fd_ostream fo((std::string(onwDump) + "_" + func.getName().str() + ".mlir"), ec);
++        if (!ec) {
++            func->print(fo, mlir::OpPrintingFlags().enableDebugInfo(true, false));
++        }
++    }
+     uint64_t unrollIdx = 0;
+     func.walk<mlir::WalkOrder::PostOrder>([&](VPURT::TaskOp vpurtTask) {
+         auto op = vpurtTask.getInnerTaskOp();
+@@ -1740,6 +1747,7 @@ void VPUIP::unrollDistributedOpsCommon40XXPlus(mlir::func::FuncOp func,
+         }
+ 
+         mlir::OpBuilder builder(op);
++        try {
+         if (auto nndmaOp = mlir::dyn_cast<VPUIP::NNDMAOp>(op)) {
+             dmaRewriter.matchAndRewrite(nndmaOp, builder, unrollIdx, /*isDataOverlapped*/ true);
+             unrollIdx++;
+@@ -1748,6 +1756,22 @@ void VPUIP::unrollDistributedOpsCommon40XXPlus(mlir::func::FuncOp func,
+         } else if (auto dmaOp = mlir::dyn_cast<VPUIP::DMATypeOpInterface>(op)) {
+             convertDMARewriter.matchAndRewrite(dmaOp, builder);
+         }
++        } catch (...) {
++            if (std::getenv("ONW_DEBUG_UNROLL") != nullptr) {
++                llvm::errs() << "ONW unroll failed on: " << *op << char(10);
++                {
++                    std::error_code ec;
++                    llvm::raw_fd_ostream fo(std::getenv("ONW_DEBUG_UNROLL"), ec);
++                    if (!ec) {
++                        func->print(fo);
++                    }
++                }
++                for (auto operand : op->getOperands()) {
++                    llvm::errs() << "  operand type: " << operand.getType() << char(10); if (auto d = operand.getDefiningOp()) { llvm::errs() << "  def: " << *d << char(10); for (auto u : d->getOperands()) if (auto dd = u.getDefiningOp()) llvm::errs() << "    def-def: " << *dd << char(10); }
++                }
++            }
++            throw;
++        }
+     });
+ }
+ 
+diff --git a/src/vpux_compiler/src/dialect/const/attributes/content.cpp b/src/vpux_compiler/src/dialect/const/attributes/content.cpp
+index 3f4f862..f5d66fe 100644
+--- a/src/vpux_compiler/src/dialect/const/attributes/content.cpp
++++ b/src/vpux_compiler/src/dialect/const/attributes/content.cpp
+@@ -7,6 +7,14 @@
+ #include "vpux/compiler/dialect/const/attributes/stable_hash_storage.hpp"
+ #include "vpux/compiler/dialect/const/constant_call_stack.hpp"
+ #include "vpux/compiler/dialect/const/constant_transformations_control.hpp"
++
++#include <atomic>
++#include <cstdio>
++#include <cstdlib>
++#include <list>
++#include <memory>
++#include <mutex>
++#include <vector>
+ #include "vpux/compiler/dialect/const/dialect.hpp"
+ #include "vpux/compiler/dialect/const/utils/const_logger.hpp"
+ #include "vpux/compiler/utils/stable_hash.hpp"
+@@ -468,11 +476,117 @@ mlir::DenseElementsAttr Const::detail::createConstContentWithConversion(mlir::Sh
+ // ContentAttr::fold
+ //
+ 
++namespace {
++// [onw] Folding memo for "... Concat ... SubView ..." chains: a large weight is concatenated from many pieces (the
++// unrolled groups of a group-quantized MatMul, Q/K/V merged), reshaped, then cut into DPU tiles with a SubView each.
++// When the SubView cannot be moved in front of the reshape (it slices the inner factor of a flattened dim), every tile
++// folds the whole concatenation again: O(tiles x weight) - most of the ELF export time of one-row (decode) graphs
++// (Granite 4.2 8B part on NPU 5010: ~2000 tiles x 0.16 s). The result of the prefix before the SubView is the same for
++// all tiles of a weight (attributes are uniqued: pointers identify it), so it is kept and copied for each tile.
++struct FoldMemo {
++    std::mutex mutex;
++    struct Entry {
++        std::vector<const void*> key;
++        std::shared_ptr<std::vector<char>> data;
++        vpux::NDTypeInterface type;
++        mlir::Type storageElemType;
++        bool isSplat;
++    };
++    std::list<Entry> entries;   // most recent first
++    size_t bytes = 0;
++    static constexpr size_t LIMIT = size_t(512) << 20;
++};
++
++FoldMemo& foldMemo() {
++    static FoldMemo memo;
++    return memo;
++}
++}  // namespace
++
+ Const::Content vpux::Const::ContentAttr::fold() const {
+     auto baseContent = getBaseContent();
+-    auto res = wrapBaseContent(baseContent);
+-    for (const auto& attr : getTransformations()) {
+-        res = attr.transform(res);
++    const auto transformations = getTransformations();
++    // the last SubView that follows a Concat: the prefix before it is memoized
++    int64_t cut = -1;
++    bool sawConcat = false;
++    for (size_t i = 0; i < transformations.size(); ++i) {
++        if (mlir::isa<Const::ConcatAttr>(transformations[i])) {
++            sawConcat = true;
++        } else if (sawConcat && mlir::isa<Const::SubViewAttr>(transformations[i])) {
++            cut = static_cast<int64_t>(i);
++        }
++    }
++    static std::atomic<int64_t> nHit{0}, nMiss{0}, nPlain{0};
++    if (cut < 0 || std::getenv("ONW_NO_FOLD_MEMO") != nullptr) {
++        ++nPlain;
++        auto res = wrapBaseContent(baseContent);
++        for (const auto& attr : transformations) {
++            res = attr.transform(res);
++        }
++        return res;
++    }
++    std::vector<const void*> key;
++    key.reserve(cut + 1);
++    key.push_back(baseContent.getAsOpaquePointer());
++    for (int64_t i = 0; i < cut; ++i) {
++        key.push_back(transformations[i].getAsOpaquePointer());
++    }
++    Const::Content res;
++    bool hit = false;
++    std::shared_ptr<std::vector<char>> held;      // (alive while the SubView reads it, whatever the memo evicts)
++    vpux::NDTypeInterface heldType;
++    mlir::Type heldStorage;
++    bool heldSplat = false;
++    {
++        auto& memo = foldMemo();
++        std::lock_guard<std::mutex> lock(memo.mutex);
++        for (auto it = memo.entries.begin(); it != memo.entries.end(); ++it) {
++            if (it->key == key) {
++                held = it->data;
++                heldType = it->type;
++                heldStorage = it->storageElemType;
++                heldSplat = it->isSplat;
++                memo.entries.splice(memo.entries.begin(), memo.entries, it);
++                hit = true;
++                ++nHit;
++                break;
++            }
++        }
++    }
++    if (hit) {
++        // no copy: the SubView only reads its input and writes a buffer of its own (SubViewAttr::transform)
++        auto view = Const::Content::fromRawBuffer(heldType, ArrayRef<char>(held->data(), held->size()), heldStorage,
++                                                  heldSplat);
++        res = transformations[cut].transform(view);
++        for (size_t i = static_cast<size_t>(cut) + 1; i < transformations.size(); ++i) {
++            res = transformations[i].transform(res);
++        }
++        return res;
++    }
++    if (!hit) {
++        ++nMiss;
++        res = wrapBaseContent(baseContent);
++        for (int64_t i = 0; i < cut; ++i) {
++            res = transformations[i].transform(res);
++        }
++        FoldMemo::Entry e;
++        e.key = key;
++        e.type = res.getType();
++        e.storageElemType = res.getStorageElemType();
++        e.isSplat = res.isSplat();
++        const auto raw = res.getRawStorageBuf();
++        e.data = std::make_shared<std::vector<char>>(raw.begin(), raw.end());
++        auto& memo = foldMemo();
++        std::lock_guard<std::mutex> lock(memo.mutex);
++        memo.bytes += e.data->size();
++        memo.entries.push_front(std::move(e));
++        while (memo.bytes > FoldMemo::LIMIT && memo.entries.size() > 1) {
++            memo.bytes -= memo.entries.back().data->size();
++            memo.entries.pop_back();
++        }
++    }
++    for (size_t i = static_cast<size_t>(cut); i < transformations.size(); ++i) {
++        res = transformations[i].transform(res);
+     }
+     return res;
+ }
+diff --git a/src/vpux_compiler/src/utils/dma_transaction_utils.cpp b/src/vpux_compiler/src/utils/dma_transaction_utils.cpp
+index 10cb1f4..45e02e8 100644
+--- a/src/vpux_compiler/src/utils/dma_transaction_utils.cpp
++++ b/src/vpux_compiler/src/utils/dma_transaction_utils.cpp
+@@ -181,6 +181,16 @@ DMAPattern reduceDimsForDma(SmallVector<int64_t> memShape, SmallVector<vpux::Mem
+     // Innermost dim is special
+     reducedDims.front() = Bit(reducedDims.front() * elemSize).to<Byte>().count();
+ 
++    if (std::getenv("ONW_DEBUG_UNROLL") != nullptr &&
++        std::any_of(std::begin(reducedBitStrides), std::prev(std::end(reducedBitStrides)), [](auto bitStride) {
++            return bitStride.count() % CHAR_BIT != 0;
++        })) {
++        llvm::errs() << "ONW dma reduce: elem " << elemSize << " shape";
++        for (auto v : memShape) llvm::errs() << " " << v;
++        llvm::errs() << " strides";
++        for (auto v : memStrides) llvm::errs() << " " << v.count();
++        llvm::errs() << char(10);
++    }
+     VPUX_THROW_WHEN(std::any_of(std::begin(reducedBitStrides), std::prev(std::end(reducedBitStrides)),
+                                 [](auto bitStride) {
+                                     return bitStride.count() % CHAR_BIT != 0;

onw/npukit/patches/openvino-6db412d-npu_plugin.patch

@@ -0,0 +1,186 @@
+diff --git a/src/plugins/intel_npu/src/compiler_adapter/include/weightless_graph.hpp b/src/plugins/intel_npu/src/compiler_adapter/include/weightless_graph.hpp
+index c5f05a69..e976d53b 100644
+--- a/src/plugins/intel_npu/src/compiler_adapter/include/weightless_graph.hpp
++++ b/src/plugins/intel_npu/src/compiler_adapter/include/weightless_graph.hpp
+@@ -133,6 +133,9 @@ private:
+      * per init compiled model and not per init schedule output for performance reasons.
+      */
+     mutable std::vector<std::shared_ptr<ZeroTensor>> _mainInputsAllocatedTensors;
++    // onw: init outputs found identical to an earlier graph's - released only after the init schedules' graphs and
++    // command lists are destroyed (freed earlier, the driver later released their reused address range)
++    mutable std::vector<std::shared_ptr<ZeroTensor>> _onwDeferredRelease;
+     /**
+      * @brief Tensors pointing towards the buffers found in "_mainInputsAllocatedTensors".
+      * @details Each map entry corresponds to one input of the main schedule.
+diff --git a/src/plugins/intel_npu/src/compiler_adapter/src/weightless_graph.cpp b/src/plugins/intel_npu/src/compiler_adapter/src/weightless_graph.cpp
+index b56d95b5..24790c3c 100644
+--- a/src/plugins/intel_npu/src/compiler_adapter/src/weightless_graph.cpp
++++ b/src/plugins/intel_npu/src/compiler_adapter/src/weightless_graph.cpp
+@@ -5,6 +5,11 @@
+ #include "weightless_graph.hpp"
+ 
+ #include <cinttypes>
++#include <unordered_map>
++#include <optional>
++#include <cstring>
++#include <cstdlib>
++#include <cstdio>
+ #include <condition_variable>
+ #include <iterator>
+ #include <mutex>
+@@ -30,6 +35,102 @@ namespace intel_npu {
+ 
+ namespace {
+ 
++// onw: share identical initialized weights between weightless graphs. Every weightless graph runs its init schedules
++// into its own device buffer - a fused graph of an 8B model holds ~4.8 GB, so each extra graph variant (more positions,
++// another block size) of the same weights costs that much again. After an init schedule ran, its output buffers are
++// looked up by size + a sampled hash and confirmed byte by byte; when every output of the schedule already exists in
++// a live buffer of an earlier graph, the main schedule is bound to those and the new buffer is released.
++// ONW_WS_SHARE=0 turns it off; ONW_WS_SHARE_LOG=1 prints what was shared.
++struct OnwWsEntry {
++    std::weak_ptr<ZeroTensor> owner;
++    const unsigned char* data;
++    size_t size;
++};
++std::mutex onwWsMutex;
++std::unordered_multimap<uint64_t, OnwWsEntry> onwWsRegistry;
++
++uint64_t onwSampledHash(const unsigned char* p, size_t n) {
++    uint64_t h = 1469598103934665603ULL ^ n;
++    auto mix = [&](const unsigned char* q, size_t len) {
++        for (size_t i = 0; i < len; ++i) {
++            h = (h ^ q[i]) * 1099511628211ULL;
++        }
++    };
++    for (size_t off = 0; off < n; off += 4096) {
++        mix(p + off, std::min<size_t>(64, n - off));
++    }
++    if (n > 64) {
++        mix(p + n - 64, 64);
++    }
++    return h;
++}
++
++bool onwWsShareEnabled() {
++    const char* e = std::getenv("ONW_WS_SHARE");
++    return e == nullptr || std::string(e) != "0";
++}
++
++// returns the owners to keep alive when every output was found (and rewrites the map to point at them), else registers
++// this buffer's outputs and returns nothing
++std::optional<std::vector<std::shared_ptr<ZeroTensor>>> onwWsShare(
++    const std::shared_ptr<ZeroTensor>& hostTensor,
++    std::unordered_map<std::string, std::shared_ptr<ov::ITensor>>& tensorsMap) {
++    if (!onwWsShareEnabled() || hostTensor == nullptr) {
++        return std::nullopt;
++    }
++    std::lock_guard<std::mutex> lock(onwWsMutex);
++    std::vector<std::pair<std::string, std::pair<std::shared_ptr<ZeroTensor>, const unsigned char*>>> found;
++    size_t bytes = 0;
++    for (const auto& [name, t] : tensorsMap) {
++        const auto* p = static_cast<const unsigned char*>(t->data());
++        const size_t n = t->get_byte_size();
++        const uint64_t h = onwSampledHash(p, n);
++        bool hit = false;
++        auto range = onwWsRegistry.equal_range(h);
++        for (auto it = range.first; it != range.second; ++it) {
++            auto owner = it->second.owner.lock();
++            if (owner == nullptr || owner == hostTensor || it->second.size != n) {
++                continue;
++            }
++            if (std::memcmp(it->second.data, p, n) == 0) {
++                found.push_back({name, {owner, it->second.data}});
++                bytes += n;
++                hit = true;
++                break;
++            }
++        }
++        if (!hit) {
++            break;
++        }
++    }
++    const bool log = std::getenv("ONW_WS_SHARE_LOG") != nullptr;
++    if (found.size() != tensorsMap.size()) {
++        for (const auto& [name, t] : tensorsMap) {
++            const auto* p = static_cast<const unsigned char*>(t->data());
++            const size_t n = t->get_byte_size();
++            onwWsRegistry.emplace(onwSampledHash(p, n), OnwWsEntry{hostTensor, p, n});
++        }
++        if (log) {
++            std::fprintf(stderr, "[onw ws-share] new weights: %zu outputs, %.1f MB%c", tensorsMap.size(),
++                         hostTensor->get_byte_size() / 1e6, 10);
++        }
++        return std::nullopt;
++    }
++    std::vector<std::shared_ptr<ZeroTensor>> owners;
++    for (auto& [name, src] : found) {
++        const auto& old = tensorsMap.at(name);
++        tensorsMap[name] = ov::make_tensor(old->get_element_type(), old->get_shape(),
++                                           const_cast<unsigned char*>(src.second));
++        owners.push_back(src.first);
++    }
++    if (log) {
++        std::fprintf(stderr, "[onw ws-share] shared: %zu outputs, %.1f MB released%c", found.size(),
++                     hostTensor->get_byte_size() / 1e6, 10);
++    }
++    return owners;
++}
++
++
+ constexpr uint8_t MAIN_SCHEDULE_INDEX = 0;
+ constexpr std::string_view WEIGHTS_IR_EXTENSION = ".bin";
+ constexpr std::string_view WEIGHTS_ONNX_EXTENSION = ".data";
+@@ -423,6 +524,7 @@ void WeightlessGraph::initialize_impl(const FilteredConfig& config) {
+     _initsFences.clear();
+     _initsMetadata.clear();
+     _initsCommandQueue.reset();
++    _onwDeferredRelease.clear();
+ 
+     // The main schedule is initialized after the weights initialization ones in order to save some memory
+     Graph::initialize_impl(config);
+@@ -502,7 +604,13 @@ WeightlessGraph::InputData WeightlessGraph::allocate_inputs(
+         // model appears in exactly one schedule. Thus, one can delete
+         // the handle to the constant memory early.
+         if (auto it = constants.find(id); it != constants.end()) {
+-            ov::wsh::Extension::hint_evict(*it->second);
++            // onw: several graphs (paged parts, fused graphs, every block size) are built from the same mapped weight
++            // files and imported one after another; evicting a constant's source here left later imports reading
++            // released memory (access violation in this function). The mapping is file-backed, so the OS reclaims it
++            // anyway. ONW_WS_EVICT=1 restores the eviction.
++            if (std::getenv("ONW_WS_EVICT") != nullptr) {
++                ov::wsh::Extension::hint_evict(*it->second);
++            }
+             constants.erase(it);
+         }
+     }
+@@ -557,6 +665,14 @@ void WeightlessGraph::run_init_single_threaded() {
+         create_pipeline(initIndex, initInputsViewTensors, initOutputsViewTensors);
+         run_pipeline(initIndex);
+ 
++        if (auto owners = onwWsShare(initOutputsAllocatedTensor, initOutputsViewTensorsMap)) {
++            merge_two_maps(_mainInputsViewTensors, initOutputsViewTensorsMap);
++            for (auto& o : *owners) {
++                _mainInputsAllocatedTensors.push_back(o);
++            }
++            _onwDeferredRelease.push_back(std::move(initOutputsAllocatedTensor));
++            continue;
++        }
+         merge_two_maps(_mainInputsViewTensors, initOutputsViewTensorsMap);
+         _mainInputsAllocatedTensors.push_back(std::move(initOutputsAllocatedTensor));
+     }
+@@ -600,6 +716,14 @@ void WeightlessGraph::run_init_multi_threaded() {
+             run_pipeline(data.initIndex);
+ 
+             // TODO: pre-allocate those well in advance? (outside of this loop)
++            if (auto owners = onwWsShare(data.outputs.hostTensor, data.outputs.tensorsMap)) {
++                merge_two_maps(_mainInputsViewTensors, data.outputs.tensorsMap);
++                for (auto& o : *owners) {
++                    _mainInputsAllocatedTensors.push_back(o);
++                }
++                _onwDeferredRelease.push_back(std::move(data.outputs.hostTensor));
++                return;
++            }
+             merge_two_maps(_mainInputsViewTensors, data.outputs.tensorsMap);
+             _mainInputsAllocatedTensors.push_back(data.outputs.hostTensor);
+         },

onw/npukit/patches/vpucostmodel.patch

@@ -0,0 +1,17 @@
+diff --git a/include/core/cache.h b/include/core/cache.h
+index 0340f76..20dca16 100644
+--- a/include/core/cache.h
++++ b/include/core/cache.h
+@@ -215,6 +215,12 @@ public:
+             if (map_it == m_table.cend()) {
+                 return std::nullopt;
+             }
++            // onw: below the size limit nothing is ever evicted, so the recency order does not matter - return
++            // under the shared lock instead of taking the exclusive one (every hit serialised all compile threads)
++            if (m_table.size() < max_size) {
++                if (source) *source = "dyn_cache";
++                return (map_it->second->second);
++            }
+         }
+ 
+         // If found, acquire a unique lock and do the mutation

onw/npukit/plugins.xml

@@ -0,0 +1,6 @@
+<ie>
+    <plugins>
+        <plugin name="NPU" location="openvino_intel_npu_plugin.dll">
+        </plugin>
+    </plugins>
+</ie>

onw/precompile.py

@@ -1,19 +1,20 @@
-"""A first load's precompile child (runtime.compile_jobs): build the segments named in ONW_PRECOMPILE_ONLY exactly as
-the parent would, so they land in the compile cache (marked as the parent's run, ONW_NPUW_SESSION), then exit.
-usage (internal): python -m onw.precompile '[MODEL_DIR, DEVICE, CONFIG, SIZES, CONTEXT]'"""
-import json, sys
-import openvino as ov
-
-
-def main():
-    d, device, config, sizes, context = json.loads(sys.argv[1])
-    from .runtime import Precompiled, SegmentedModel
-    try:
-        SegmentedModel(ov.Core(), d, device, config, sizes=sizes, context=context)
-    except Precompiled:
-        return
-    sys.exit(3)                                           # (not reached: the model raises once compiled)
-
-
-if __name__ == "__main__":
-    main()
+"""A first load's precompile child (runtime.compile_jobs): build the segments named in ONW_PRECOMPILE_ONLY exactly as
+the parent would, so they land in the compile cache (marked as the parent's run, ONW_NPUW_SESSION), then exit.
+usage (internal): python -m onw.precompile '[MODEL_DIR, DEVICE, CONFIG, SIZES, CONTEXT]'"""
+import json, sys
+import openvino as ov
+from . import npukit
+
+
+def main():
+    d, device, config, sizes, context = json.loads(sys.argv[1])
+    from .runtime import Precompiled, SegmentedModel
+    try:
+        SegmentedModel(npukit.core(), d, device, config, sizes=sizes, context=context)
+    except Precompiled:
+        return
+    sys.exit(3)                                           # (not reached: the model raises once compiled)
+
+
+if __name__ == "__main__":
+    main()

onw/probe.py

@@ -1,59 +1,60 @@
-"""Compile one segment in a separate process, so a crash inside the NPU compiler (a driver bug: the process dies
-with no Python exception) does not take the server down. With EXPORT_BLOB the compiled blob is written there for the
-parent to import (onw's own blob store, see runtime.import_blob); otherwise the result lands in the OpenVINO cache dir
-given in the config and the parent's compile of the same segment is a cache hit.
-usage (internal): python -m onw.probe MODEL_DIR SEG_XML CONFIG_JSON NPUW(0/1) T [EXPORT_BLOB]
-ONW_RING (env): the sliding-window ring length the parent reshaped the caches to (paged mode).
-exit 0: compiled; 1: the compiler rejected it (reason on stdout); 2: something else went wrong (disk full, a file
-missing - not the compiler's verdict, so the parent does not remember it); anything else: crashed"""
-import io, json, os, sys
-import openvino as ov
-
-
-def main():
-    d, name, cfg, npuw, T = sys.argv[1], sys.argv[2], json.loads(sys.argv[3]), sys.argv[4] == "1", int(sys.argv[5])
-    from .runtime import read_segment
-    core = ov.Core()
-    from .runtime import enable_compiler
-    enable_compiler(core, "NPU", {}, quiet=True)  # [2.0.3] the same NPU compiler as the parent (see npu_compiler)
-    meta = json.load(open(f"{d}/engine.json"))
-    bins = {}                                             # keeps the .bin buffer alive while compiling
-    try:
-        segs = {**meta["segments"], **(meta.get("mtp") or {}).get("segments", {})}     # (the MTP graphs too)
-        model = read_segment(core, d, name, segs[name], npuw, bins, T, meta["T"])
-        if os.environ.get("ONW_KV", "paged") != "legacy":  # the same graph the parent compiles (host-side rope)
-            from . import paged
-            paged.host_rope(model)
-        if os.environ.get("ONW_RING"):                    # paged mode: sliding-window caches live in a ring
-            from .runtime import ring_shapes
-            model.reshape(ring_shapes(model, int(os.environ["ONW_RING"]), segs[name]["S"]))
-    except Exception as e:
-        print(f"{type(e).__name__}: {e}", flush=True)
-        sys.exit(2)
-    if os.environ.get("ONW_TEST_CRASH_KEY") in cfg:      # test hook: behave like a compiler crash on this route
-        os.abort()
-    try:
-        compiled = core.compile_model(model, "NPU", cfg)
-    except MemoryError as e:                              # (out of memory: not the compiler's verdict)
-        print(f"MemoryError: {e}", flush=True)
-        sys.exit(2)
-    except RuntimeError as e:
-        why = str(e)
-        print(why[why.find("While validating"):] if "While validating" in why else why[-300:], flush=True)
-        sys.exit(1)
-    if len(sys.argv) > 6:                                 # write the blob for the parent to import
-        out = sys.argv[6]
-        try:
-            os.makedirs(os.path.dirname(out), exist_ok=True)
-            buf = io.BytesIO()
-            compiled.export_model(buf)
-            with open(out + ".tmp", "wb") as f:
-                f.write(buf.getbuffer())
-            os.replace(out + ".tmp", out)
-        except Exception as e:
-            print(f"{type(e).__name__}: {e}", flush=True)
-            sys.exit(2)
-
-
-if __name__ == "__main__":
-    main()
+"""Compile one segment in a separate process, so a crash inside the NPU compiler (a driver bug: the process dies
+with no Python exception) does not take the server down. With EXPORT_BLOB the compiled blob is written there for the
+parent to import (onw's own blob store, see runtime.import_blob); otherwise the result lands in the OpenVINO cache dir
+given in the config and the parent's compile of the same segment is a cache hit.
+usage (internal): python -m onw.probe MODEL_DIR SEG_XML CONFIG_JSON NPUW(0/1) T [EXPORT_BLOB]
+ONW_RING (env): the sliding-window ring length the parent reshaped the caches to (paged mode).
+exit 0: compiled; 1: the compiler rejected it (reason on stdout); 2: something else went wrong (disk full, a file
+missing - not the compiler's verdict, so the parent does not remember it); anything else: crashed"""
+import io, json, os, sys
+import openvino as ov
+from . import npukit
+
+
+def main():
+    d, name, cfg, npuw, T = sys.argv[1], sys.argv[2], json.loads(sys.argv[3]), sys.argv[4] == "1", int(sys.argv[5])
+    from .runtime import read_segment
+    core = npukit.core()
+    from .runtime import enable_compiler
+    enable_compiler(core, "NPU", {}, quiet=True)  # [2.0.3] the same NPU compiler as the parent (see npu_compiler)
+    meta = json.load(open(f"{d}/engine.json"))
+    bins = {}                                             # keeps the .bin buffer alive while compiling
+    try:
+        segs = {**meta["segments"], **(meta.get("mtp") or {}).get("segments", {})}     # (the MTP graphs too)
+        model = read_segment(core, d, name, segs[name], npuw, bins, T, meta["T"])
+        if os.environ.get("ONW_KV", "paged") != "legacy":  # the same graph the parent compiles (host-side rope)
+            from . import paged
+            paged.host_rope(model)
+        if os.environ.get("ONW_RING"):                    # paged mode: sliding-window caches live in a ring
+            from .runtime import ring_shapes
+            model.reshape(ring_shapes(model, int(os.environ["ONW_RING"]), segs[name]["S"]))
+    except Exception as e:
+        print(f"{type(e).__name__}: {e}", flush=True)
+        sys.exit(2)
+    if os.environ.get("ONW_TEST_CRASH_KEY") in cfg:      # test hook: behave like a compiler crash on this route
+        os.abort()
+    try:
+        compiled = core.compile_model(model, "NPU", cfg)
+    except MemoryError as e:                              # (out of memory: not the compiler's verdict)
+        print(f"MemoryError: {e}", flush=True)
+        sys.exit(2)
+    except RuntimeError as e:
+        why = str(e)
+        print(why[why.find("While validating"):] if "While validating" in why else why[-300:], flush=True)
+        sys.exit(1)
+    if len(sys.argv) > 6:                                 # write the blob for the parent to import
+        out = sys.argv[6]
+        try:
+            os.makedirs(os.path.dirname(out), exist_ok=True)
+            buf = io.BytesIO()
+            compiled.export_model(buf)
+            with open(out + ".tmp", "wb") as f:
+                f.write(buf.getbuffer())
+            os.replace(out + ".tmp", out)
+        except Exception as e:
+            print(f"{type(e).__name__}: {e}", flush=True)
+            sys.exit(2)
+
+
+if __name__ == "__main__":
+    main()

onw/runtime.py

@@ -692,6 +692,46 @@ def import_blob(core, path, device, cfg):
         return core.import_model(io.BytesIO(f.read()), device, cfg)
 
 
+# [3.0] Weightless blobs: the compiled blob keeps no weights (CACHE_MODE=OPTIMIZE_SIZE + ENABLE_WEIGHTLESS); importing
+# it with the same ov.Model (MODEL_PTR) runs the blob's init schedules, which rebuild the weights on the NPU from the
+# model's constants (the mapped .bin files). The disk cache then holds the graphs only, and onw's own NPU plugin shares
+# identical rebuilt weights between graphs (paged parts, fused graphs, every block size: the model's weights once in
+# memory) - with one init output per weight (ONW_WS_NO_CONCAT, read by onw's own compiler). Needs a compiler that
+# supports it for the weight format (onw's own build; pip OpenVINO 2026.5: not NF4; NPU 3720: no): after the first
+# failure in a process, full blobs as before. ONW_WEIGHTLESS=0 turns it off.
+WEIGHTLESS_CFG = {"CACHE_MODE": "OPTIMIZE_SIZE", "ENABLE_WEIGHTLESS": True, "NPU_COMPILER_TYPE": "PLUGIN"}
+_WEIGHTLESS = {"ok": None}
+os.environ.setdefault("ONW_WS_NO_CONCAT", "1")
+
+
+def weightless_wanted():
+    from . import npukit                             # (default: with onw's NPU kit only - OpenVINO's compiler fails NF4)
+    return (os.environ.get("ONW_WEIGHTLESS", "1" if npukit.active() else "0") != "0" and _WEIGHTLESS["ok"] is not False
+            and not _WEIGHTLESS.get("skip_model"))
+
+
+def int4_weights(d, meta):
+    """[3.0] The model's weights are INT4 (i4 / u4 constants, no NF4) - read from its first segment's IR."""
+    names = sorted(n for n in meta.get("segments", {}) if n.endswith(".xml"))
+    if not names:
+        return False
+    try:
+        with open(os.path.join(d, names[0]), encoding="utf8", errors="replace") as f:
+            xml = f.read()
+    except OSError:
+        return False
+    return ('element_type="nf4"' not in xml and
+            ('element_type="i4"' in xml or 'element_type="u4"' in xml))
+
+
+def weightless_failed(e, log=print):
+    if _WEIGHTLESS["ok"] is not False:
+        log(f"[compile] weightless blobs not supported here ({' '.join(str(e).split())[-140:]}): full blobs",
+            flush=True)
+    _WEIGHTLESS["ok"] = False
+    _WEIGHTLESS["full"] = True
+
+
 def _signature(model):
     """A cheap fingerprint of a graph's structure (op types, shapes, input / output names): a part's compiled blob is
     only reused for exactly the same part, whatever changes in how segments are split."""
@@ -1041,6 +1081,18 @@ class SegmentedModel:
         # MoE models already hold their experts once (weights as inputs), so NPUW is off for them unless ONW_NPUW_MOE=1:
         # it stopped mid-answer on LFM2 (NPU 4000), but that was with a Linux driver < 1.38 (see the conv fix there)
         npuw = device == "NPU" and use_npuw(core) and (not self.bank or os.environ.get("ONW_NPUW_MOE") == "1")
+        # [3.0] 2-bit weights stay inside the blobs: the weights separation leaves them there anyway (Bonsai 2 9B part:
+        # 332.6 -> 331.8 MB) and compiles 7x slower (19 -> 142 s)
+        # INT4 weights of the plain paged models as well: never separated by the compiler (Granite 4.2 3B INT4 part:
+        # 63.5 MB weightless against 63.6 MB, multi-row parts 20-35x slower to compile) - INT8 and NF4 are (llm-jp INT8
+        # part: 2.5 against 216 MB). (NPUW and MoE models as they were checked: gemma-4, Qwen3.6)
+        _WEIGHTLESS["skip_model"] = self.meta.get("needs") == "u2"
+        _WEIGHTLESS["full"] = False                    # (this model: a part with its weights inside, see fused.wanted)
+        if device == "NPU" and not _WEIGHTLESS.get("told"):
+            from . import npukit
+            _WEIGHTLESS["told"] = True
+            print(f"[npukit] {'onw NPU kit ' + npukit.why() if npukit.active() else 'OpenVINO NPU plugin (onw kit: ' + npukit.why() + ')'}",
+                  flush=True)
         if npuw and self.meta.get("needs") == "u2" and os.environ.get("ONW_NPUW") != "1":
             # 2-bit weights stay 2-bit compiled plain only: NPUW expands them (Lunar Lake, Ternary Bonsai 2 27B: 12.6 GB
             # of NPU memory instead of ~7, the 16 GB machine out of memory, 0.04 tok/s against 4 tok/s plain)
@@ -1052,6 +1104,8 @@ class SegmentedModel:
             print(f"[npuw] off: ~{parts} compiled parts (each attention layer cuts a segment) - more than the NPU driver "
                   f"holds with NPUW (~30); ONW_NPUW=1 forces it", flush=True)
             npuw = False
+        if not npuw and not self.bank and int4_weights(d, self.meta):
+            _WEIGHTLESS["skip_model"] = True           # (decided here: NPUW may have been turned off just above)
         if npuw:
             # experimental: NPUW pulls the weights out of every compiled graph into one weights bank; constants match
             # by data pointer, so every block size reads its IR against the same in-memory copy of the .bin
@@ -1340,6 +1394,12 @@ class SegmentedModel:
     # ---- [2.0.4] fused graphs (onw/fused.py): one NPU call per segment, for answers and for prompts ----
     FUSED_B = (1, 2, 3)                                # the variants: the first 1 / 2 / 3 paged blocks (1K / 2K / 4K)
 
+    def fused_keep_all(self):
+        """[3.0] every fused variant stays loaded (onw's NPU kit, NPU 4000+, not the memory-saving mode)."""
+        from . import npukit
+        return os.environ.get("ONW_FUSED_KEEP", "1" if npukit.active() and npu_arch(self._ctx["core"]) >= 4000
+                              and not mem_saver() else "0") == "1"
+
     def fused_prepare(self, need, S=None):
         """Make the fused S-token graphs (default: the answer size) covering `need` positions the ones in use for that
         size (loaded from onw's blob store, or compiled the first time); the other variants are compiled in the
@@ -1352,9 +1412,38 @@ class SegmentedModel:
         if S not in sizes:
             return None
         from . import fused as _f
-        caps = [(B, sum(_f.blocks_for(self.kv.capacity, B))) for B in self.FUSED_B]
+        keep_all = self.fused_keep_all()
+        # [3.0] with every variant kept (shared weights) the variants go on up to the context: 8K / 16K / 32K positions
+        # (the first 4 / 5 / 6 paged blocks) - past 4K the paged graphs answered at about half the speed
+        nb = len(_f.blocks_for(self.kv.capacity, 99))
+        bs = tuple(range(1, min(nb, int(os.environ.get("ONW_FUSED_BLOCKS", "6"))) + 1)) if keep_all else self.FUSED_B
+        caps = [(B, sum(_f.blocks_for(self.kv.capacity, B))) for B in bs]
         want = next((B for B, c in caps if c >= min(need, self.T)), None)
         F = self._fused.get(S)
+        if keep_all:
+            # [3.0] every variant stays loaded next to the paged graphs (with onw's own NPU plugin they share one copy
+            # of the weights): no graph is dropped or reloaded while chatting - no pause at 1K / 2K / 4K positions, and
+            # the same graphs for the same positions whatever came before
+            keep = self.__dict__.setdefault("_fused_keep", {})
+            if want is None:
+                self._fused.pop(S, None)
+                if S not in self.segs:
+                    self.swap_to(S)
+                self.S = max([*self.segs, *self._fused])
+                return None
+            F = next((keep[(S, B)] for B in bs if B >= want and (S, B) in keep), None)
+            if F is None:
+                with self._mutex:
+                    try:
+                        F = keep[(S, want)] = _f.FusedDecode(self, S, want)
+                    except Exception as e:
+                        print(f"[fused] {S}-token graphs off: {' '.join(str(e).split())[-400:]}", flush=True)
+                        self._fused_sizes = [x for x in sizes if x != S]
+                        return None
+            self._fused[S] = F
+            self.S = max([*self.segs, *self._fused])
+            self._fused_background(caps)
+            return F
         if want is None:                               # longer than the largest variant: the paged graphs
             if F is not None:
                 F = None                               # [2.1.0] (no reference left: freed before the paged graphs load)
@@ -1387,6 +1476,10 @@ class SegmentedModel:
                     self.swap_to(S)
                 return None
             self.S = max([*self.segs, *self._fused])   # (prompts are still fed in blocks of the largest size)
+        self._fused_background(caps)
+        return self._fused[S]
+
+    def _fused_background(self, caps):
         npuw_ = (self._ctx["config"] or {}).get("NPU_USE_NPUW") == "YES"   # (NPUW: compiled when needed, no background)
         if os.environ.get("ONW_FUSED_BG", "1") != "0" and not self._fused_bg and not npuw_:
             # every variant of every fused size, in one background thread, the short ones first (compiled once; later
@@ -1396,7 +1489,6 @@ class SegmentedModel:
             jobs = [(x, B) for B, _ in caps for x in self._fused_sizes]
             self._fused_thread = threading.Thread(target=self._fused_precompile, args=(jobs,), daemon=True)
             self._fused_thread.start()
-        return self._fused[S]
 
     def _fused_precompile(self, jobs):
         from . import fused as _f
@@ -1842,10 +1934,33 @@ class SegmentedModel:
                     os.remove(bp)
                 except OSError:
                     pass
+            bpw = blob_path(cache, gname, src + [key, "weightless"]) if own and device == "NPU" else None
+            if bpw and os.path.exists(bpw) and stale(binf, bpw):
+                try:
+                    os.remove(bpw)
+                except OSError:
+                    pass
+            if bpw and os.path.exists(bpw) and weightless_wanted():
+                try:
+                    req = import_blob(core, bpw, device, {**nocache, "MODEL_PTR": model}).create_infer_request()
+                    USED_BLOBS.add(bpw)
+                    tried.set(gname, how)
+                    return req
+                except (RuntimeError, OSError) as e:
+                    if _transient(str(e)):
+                        raise RuntimeError(TRANSIENT + str(e)) from e
+                    print(f"[compile] {gname}: stored weightless blob unusable ({' '.join(str(e).split())[-120:]})",
+                          flush=True)
+                    try:
+                        os.remove(bpw)
+                    except OSError:
+                        pass
             if own and os.path.exists(bp):
                 try:
                     req = import_blob(core, bp, device, nocache).create_infer_request()
                     USED_BLOBS.add(bp)
+                    if bpw and weightless_wanted():
+                        _WEIGHTLESS["full"] = True     # (compiled with its weights: weightless failed for it before)
                     tried.set(gname, how)
                     return req
                 except (RuntimeError, OSError) as e:      # (OSError: removed meanwhile by another process)
@@ -1858,6 +1973,47 @@ class SegmentedModel:
                     except OSError:
                         pass
             fresh = how != known and device == "NPU"
+            if bpw and weightless_wanted():
+                try:
+                    if fresh:
+                        tried.trying(gname, how)
+                    if part is None:               # (whole segments: out of process, as below)
+                        r = _probe(d, name, {**nocache, **WEIGHTLESS_CFG}, npuw, T, export=bpw, ring=ring)
+                        if r is not True:
+                            raise RuntimeError(r if isinstance(r, str) else "rejected")
+                    else:
+                        cm = core.compile_model(model, device, {**nocache, **WEIGHTLESS_CFG})
+                        import io
+                        buf = io.BytesIO()
+                        cm.export_model(buf)
+                        del cm
+                        os.makedirs(os.path.dirname(bpw), exist_ok=True)
+                        with open(f"{bpw}.{os.getpid()}.tmp", "wb") as f:
+                            f.write(buf.getbuffer())
+                        del buf
+                        os.replace(f"{bpw}.{os.getpid()}.tmp", bpw)
+                    req = import_blob(core, bpw, device, {**nocache, "MODEL_PTR": model}).create_infer_request()
+                    USED_BLOBS.add(bpw)
+                    _WEIGHTLESS["ok"] = True
+                    if fresh:
+                        print(f"[compile] {gname}: {how} (weightless)", flush=True)
+                    if provisional:
+                        tried.untry(gname, how, known)
+                    else:
+                        tried.set(gname, how)
+                    tried.save()
+                    return req
+                except RuntimeError as e:
+                    if _transient(str(e)):
+                        tried.untry(gname, how, known)
+                        raise RuntimeError(TRANSIENT + str(e)) from e
+                    weightless_failed(e)
+                    if fresh:
+                        tried.untry(gname, how, known)
+                    try:
+                        os.remove(bpw)
+                    except OSError:
+                        pass
             try:
                 if fresh:
                     tried.trying(gname, how)

onw/server.py

@@ -1,748 +1,749 @@
-"""OpenAI-compatible API server for an onw model on the Intel NPU (point other apps at http://HOST:PORT/v1).
-
-  GET  /                      what this server is: model, OpenAI base URL, endpoints (JSON)
-  GET  /v1/models
-  GET  /v1/models/{id}
-  POST /v1/chat/completions   (stream true/false; text and image_url parts: data: URLs or http(s)); responses
-                              also carry "onw_stats" (prefill / decode speed, reused tokens) - extra field
-  GET  /health                200 when the model is loaded
-  GET  /chat                  browser chat UI for trying the model
-  GET  /check                 verification page: speed, image, multi-turn reuse
-  GET  /api/info              model and device details
-
-usage: onw serve MODEL [--port 8000] [--host 0.0.0.0] [--api-key KEY] [--device NPU|CPU] [--open]
-CORS is open, so browser-based clients work too; with --api-key every /v1 request needs "Authorization: Bearer KEY"
-(except from this PC's own programs and pages: the chat and check pages send no key).
-Thinking mode: "chat_template_kwargs": {"enable_thinking": true}, or "reasoning_effort": "low" | "medium" | "high"
-(gpt-oss uses the level; "none" / "minimal" turn thinking off). Whatever the model thought comes back as
-message.reasoning_content (delta.reasoning_content when streaming; also for models that think with thinking off, as vLLM's
-reasoning parsers do), the answer as content, and the thinking's token count as OpenAI's
-usage.completion_tokens_details.reasoning_tokens.
-Tool calls: "tools" (OpenAI format) -> message.tool_calls / finish_reason "tool_calls"; results as role "tool".
-tool_choice "none" hides the tools; a named function narrows them to that one ("required" cannot be enforced).
-response_format json_object / json_schema is best effort (an instruction is added; the output is not constrained).
-Not supported: n > 1 (400), logprobs (null).
-One NPU pipeline serves requests one at a time (a lock serializes them).
-"""
-import argparse, asyncio, base64, io, ipaddress, json, os, re, socket, threading, time, uuid, urllib.parse, urllib.request
-
-import uvicorn
-from fastapi import FastAPI, HTTPException, Request
-from fastapi.exceptions import RequestValidationError
-from fastapi.responses import HTMLResponse, JSONResponse, StreamingResponse
-from PIL import Image
-from onw import __version__, i18n
-from onw import tools as tool_calls
-
-MODEL_ID = "npu-model"      # set from the engine directory in main()
-app = FastAPI(title="LLM on Intel NPU")
-pipe = None                 # onw.chat.ChatEngine, created in main()
-info = {}
-lock = threading.Lock()     # one generation at a time on the NPU
-WEB = os.path.join(os.path.dirname(os.path.abspath(__file__)), "web")
-API_KEY = None              # --api-key: required as a Bearer token on /v1
-API_SAMPLING = None         # sampling for requests that send none (ONW_API_SAMPLING: "recommended" or a JSON of values)
-TOO_LONG = re.compile(r"the prompt is (\d+) tokens; the context holds (\d+)")   # (ChatEngine.stream_chat's ValueError)
-
-
-def oai_error(status, message, type_="invalid_request_error", code=None):
-    """An error in OpenAI's shape: {"error": {"message", "type", "code"}} (a prompt over the context: code
-    "context_length_exceeded", as OpenAI's)."""
-    code = code or ("context_length_exceeded" if TOO_LONG.match(message) else None)
-    return JSONResponse({"error": {"message": message, "type": type_, "param": None, "code": code}}, status_code=status)
-
-
-def anthropic_message(message):
-    """Anthropic's wording for a prompt over the context ("prompt is too long: N tokens > M maximum" - Claude Code
-    compacts the conversation on it); other messages as they are."""
-    m = TOO_LONG.match(message)
-    return f"prompt is too long: {m[1]} tokens > {m[2]} maximum" if m else message
-
-
-def api_error(request, status, message, type_="invalid_request_error", code=None):
-    """oai_error, or on the Anthropic Messages API (/v1/messages...) Anthropic's shape:
-    {"type": "error", "error": {"type", "message"}}."""
-    if request.url.path.startswith("/v1/messages"):
-        typ = {401: "authentication_error", 403: "permission_error", 404: "not_found_error",
-               413: "request_too_large"}.get(status, "api_error" if status >= 500 else "invalid_request_error")
-        return JSONResponse({"type": "error", "error": {"type": typ, "message": anthropic_message(message)}},
-                            status_code=status)
-    return oai_error(status, message, type_, code)
-
-
-@app.exception_handler(HTTPException)
-async def _http_error(request, e):
-    return api_error(request, e.status_code, str(e.detail),
-                     "invalid_request_error" if e.status_code < 500 else "server_error")
-
-
-@app.exception_handler(RequestValidationError)
-async def _validation_error(request, e):
-    return api_error(request, 400, str(e))
-
-
-@app.exception_handler(Exception)
-async def _server_error(request, e):
-    return api_error(request, 500, f"{type(e).__name__}: {e}", "server_error")
-
-
-CORS = [(b"access-control-allow-origin", b"*"), (b"access-control-allow-headers", b"*"),
-        (b"access-control-allow-methods", b"GET, POST, OPTIONS")]
-
-
-class CorsAndKey:
-    """Open CORS (browser-based OpenAI clients) and the optional API key on /v1 (not asked of this PC's own programs and
-    pages - the chat and check pages send none; the key keeps the LAN out). A plain ASGI middleware, not
-    @app.middleware("http"): behind that one (Starlette's BaseHTTPMiddleware) request.is_disconnected() never turns
-    true, and a client that left a whole (non-streamed) answer kept the NPU busy to max_tokens."""
-    def __init__(self, inner):
-        self.inner = inner
-
-    async def __call__(self, scope, receive, send):
-        if scope["type"] != "http":
-            return await self.inner(scope, receive, send)
-        request = Request(scope)                       # (headers and client only: the body stays with the app)
-        path = scope["path"]
-        if path.startswith("/api/chats"):              # saved conversations: this PC's own chat page only
-            why = _chats_refused(request)
-            if why:
-                return await JSONResponse({"error": why}, status_code=403)(scope, receive, send)   # (no CORS)
-            return await self.inner(scope, receive, send)
-
-        async def with_cors(msg):
-            if msg["type"] == "http.response.start":
-                msg = {**msg, "headers": [h for h in msg.get("headers", []) if h[0].lower() not in dict(CORS)] + CORS}
-            await send(msg)
-        if scope["method"] == "OPTIONS":
-            return await JSONResponse({})(scope, receive, with_cors)
-        if API_KEY and path.startswith("/v1") and not _local(request) and \
-                request.headers.get("authorization", "") != f"Bearer {API_KEY}" and \
-                request.headers.get("x-api-key", "") != API_KEY:            # (Anthropic clients send x-api-key)
-            return await api_error(request, 401, "invalid API key", code="invalid_api_key")(scope, receive, with_cors)
-        await self.inner(scope, receive, with_cors)
-
-
-app.add_middleware(CorsAndKey)
-
-
-# ---------------------------------------------------------------- saved conversations (the chat page's history)
-LOCAL_HOSTS = ("localhost", "127.0.0.1", "[::1]", "::1")
-CHAT_ID = __import__("re").compile(r"^[a-z0-9-]{8,48}$")
-CHAT_MAX = 32 << 20                                     # bytes per conversation (images are kept small by the page)
-
-
-def _chats_refused(request):
-    """Why a request for the saved conversations is refused, or None: only from this PC (also when the server listens on
-    the LAN), only under a localhost name (a rebound DNS name of some web site would otherwise pass as same-origin) and
-    only from the chat page itself (no cross-site reads or writes: these responses carry no CORS headers)."""
-    client = request.client.host if request.client else ""
-    try:
-        if not ipaddress.ip_address(client).is_loopback:
-            return "only from this PC"
-    except ValueError:
-        return "only from this PC"
-    host = urllib.parse.urlsplit("//" + request.headers.get("host", "")).hostname or ""
-    if host not in LOCAL_HOSTS:
-        return "only under localhost"
-    origin = request.headers.get("origin")
-    if origin and (urllib.parse.urlsplit(origin).hostname or "") not in LOCAL_HOSTS:
-        return "not from another site"
-    return None
-
-
-def _local(request):
-    """True for a request from this PC itself as _chats_refused judges it (this PC's address, a localhost name, no other
-    site's page) and not one a proxy forwards (with X-Forwarded-For from a proxy on this PC, uvicorn already puts the
-    real client's address in request.client; a bare "Forwarded" it does not read)."""
-    return _chats_refused(request) is None and "forwarded" not in request.headers and \
-        "x-forwarded-for" not in request.headers
-
-
-def chats_dir():
-    from onw import app as _app
-    d = os.path.join(_app.config_dir(), "chats")
-    os.makedirs(d, exist_ok=True)
-    return d
-
-
-def _chat_path(cid):
-    if not CHAT_ID.match(cid or ""):
-        raise HTTPException(400, "bad conversation id")
-    return os.path.join(chats_dir(), cid + ".json")
-
-
-@app.get("/api/chats")
-def chats_list():
-    out = []
-    for f in os.listdir(chats_dir()):
-        if not f.endswith(".json"):
-            continue
-        try:
-            c = json.load(open(os.path.join(chats_dir(), f), encoding="utf8"))
-            out.append({k: c.get(k) for k in ("id", "title", "model", "created", "updated")} | {"n": len(c.get("messages", []))})
-        except (OSError, ValueError):
-            continue
-    return sorted(out, key=lambda c: c.get("updated") or 0, reverse=True)
-
-
-@app.get("/api/chats/{cid}")
-def chat_get(cid: str):
-    try:
-        return json.load(open(_chat_path(cid), encoding="utf8"))
-    except FileNotFoundError:
-        raise HTTPException(404, "no such conversation")
-
-
-@app.put("/api/chats/{cid}")
-async def chat_put(cid: str, request: Request):
-    p = _chat_path(cid)
-    raw = await request.body()
-    if len(raw) > CHAT_MAX:
-        raise HTTPException(413, "conversation too large")
-    try:
-        c = json.loads(raw)
-        assert isinstance(c, dict) and isinstance(c.get("messages"), list)
-    except (ValueError, AssertionError):
-        raise HTTPException(400, "not a conversation")
-    c["id"] = cid
-    with open(p + ".tmp", "w", encoding="utf8") as f:
-        json.dump(c, f, ensure_ascii=False)
-    os.replace(p + ".tmp", p)
-    return {"ok": True}
-
-
-@app.delete("/api/chats/{cid}")
-def chat_delete(cid: str):
-    try:
-        os.remove(_chat_path(cid))
-    except FileNotFoundError:
-        pass
-    return {"ok": True}
-
-
-@app.delete("/api/chats")
-def chats_delete_all():
-    n = 0
-    for f in os.listdir(chats_dir()):
-        if f.endswith(".json"):
-            try:
-                os.remove(os.path.join(chats_dir(), f))
-                n += 1
-            except OSError:
-                pass
-    return {"ok": True, "deleted": n}
-
-
-def start_generation(msgs, kw):
-    """Run stream_chat in a worker thread; returns (asyncio queue of deltas/stats/exceptions, None at the end; stop
-    event). Called on the event loop: the worker hands its items over to it, so no thread waits per queued request."""
-    loop, q, stop = asyncio.get_running_loop(), asyncio.Queue(), threading.Event()
-
-    def put(x):
-        try:
-            loop.call_soon_threadsafe(q.put_nowait, x)
-        except RuntimeError:                        # the event loop is gone (the server is stopping)
-            stop.set()
-
-    def work():
-        try:
-            with lock:
-                if stop.is_set():                   # the client left while this request waited for the NPU
-                    return
-                set_priority(True)
-                try:
-                    for d in pipe.stream_chat(msgs, should_stop=stop.is_set, split_reasoning=True, **kw):
-                        put(d)
-                finally:
-                    set_priority(False)
-        except Exception as e:                      # surfaced to the client
-            put(e)
-        finally:
-            put(None)
-    threading.Thread(target=work, daemon=True).start()
-    return q, stop
-
-
-def reader(q, stop, request=None):
-    """-> async function returning start_generation's next item. With request (a whole response, not a stream): a
-    client that leaves meanwhile stops the generation (checked about once a second; a stream stops when it closes)."""
-    last = [time.monotonic()]
-
-    async def next_item():
-        while request is not None and not stop.is_set():
-            wait = last[0] + 1.0 - time.monotonic()
-            if wait <= 0:
-                last[0] = time.monotonic()
-                if await request.is_disconnected():
-                    stop.set()
-                continue
-            try:
-                return await asyncio.wait_for(q.get(), wait)
-            except asyncio.TimeoutError:
-                pass
-        return await q.get()
-    return next_item
-
-
-MAX_IMAGE = 20 << 20                                    # bytes
-
-
-def _public_host(url):
-    """True when every address the URL's host resolves to is a public one: the server does not fetch from the local
-    network on a client's behalf (ONW_ALLOW_LOCAL_IMAGES=1 allows it)."""
-    if os.environ.get("ONW_ALLOW_LOCAL_IMAGES") == "1":
-        return True
-    host = urllib.parse.urlsplit(url).hostname
-    try:
-        addrs = {a[4][0] for a in socket.getaddrinfo(host, None)}
-    except (socket.gaierror, UnicodeError, TypeError):
-        return False
-    return bool(addrs) and all(ipaddress.ip_address(a.split("%")[0]).is_global for a in addrs)
-
-
-def load_image(url):
-    try:
-        if url.startswith("data:"):
-            data = base64.b64decode(url.split(",", 1)[1])
-        elif url.startswith(("http://", "https://")):
-            if not _public_host(url):
-                raise HTTPException(400, "image_url: only public http(s) addresses are fetched (send a data: URL instead)")
-            req = urllib.request.Request(url, headers={"User-Agent": f"onw/{__version__}"})
-            with urllib.request.urlopen(req, timeout=20) as r:
-                data = r.read(MAX_IMAGE + 1)
-            if len(data) > MAX_IMAGE:
-                raise HTTPException(400, "image_url: the image is larger than 20 MB")
-        else:
-            raise HTTPException(400, "image_url must be a data: URL or http(s) URL")
-        return Image.open(io.BytesIO(data)).convert("RGB")
-    except HTTPException:
-        raise
-    except Exception as e:                                # bad base64, not an image, unreachable URL
-        raise HTTPException(400, f"image_url: {type(e).__name__}: {e}")
-
-
-def to_hf_messages(messages):
-    """OpenAI chat messages -> HF chat format with PIL images."""
-    out = []
-    for m in tool_calls.history(messages):
-        keep = {k: m[k] for k in ("tool_calls", "tool_call_id", "name", "reasoning_content") if m.get(k) is not None}
-        c = m.get("content")
-        if c is None or isinstance(c, str):
-            out.append({"role": m["role"], "content": c or "", **keep})
-            continue
-        parts = []
-        for p in c:
-            if p.get("type") == "text":
-                parts.append({"type": "text", "text": p.get("text") or ""})
-            elif p.get("type") == "image_url":
-                iu = p.get("image_url") or {}
-                url = iu.get("url") if isinstance(iu, dict) else iu
-                if not isinstance(url, str):
-                    raise HTTPException(400, "image_url part without a url")
-                parts.append({"type": "image", "image": load_image(url)})
-            else:
-                raise HTTPException(400, f"unsupported content part type: {p.get('type')}")
-        out.append({"role": m["role"], "content": parts, **keep})
-    return out
-
-
-def _num(body, key, default, cast=float):
-    v = body.get(key)
-    if v is None:
-        return default
-    try:
-        return cast(v)
-    except (TypeError, ValueError):
-        raise HTTPException(400, f"{key} must be a number")
-
-
-LEVELS = ("off", "low", "medium", "high")   # the thinking levels of the window and the chat page (reasoning_effort)
-_PREFS = [None, None]                       # (config mtime, this model's defaults)
-
-
-def model_defaults():
-    """This model's defaults from the onw window (config model_prefs, read again when the file changes - no restart):
-    {"think": level, "sampling": "greedy" | "recommended" | "custom", values...}; {} when none were saved. Settings
-    of earlier versions (ONW_THINK_DEFAULT, ONW_API_SAMPLING) apply where these say nothing."""
-    key = os.environ.get("ONW_MODEL_KEY")
-    if not key:
-        return {}
-    try:
-        from onw import app as _app
-        p = os.path.join(_app.config_dir(), "config.json")
-        mt = os.path.getmtime(p)
-        if _PREFS[0] != mt:
-            _PREFS[:] = [mt, (_app.load_config().get("model_prefs") or {}).get(key) or {}]
-        return _PREFS[1]
-    except (OSError, ValueError):
-        return {}
-
-
-def default_think():
-    """Thinking for a request that says nothing about it: the model's saved level, else the old switch."""
-    lvl = model_defaults().get("think")
-    if lvl == "off":
-        return False
-    if lvl in LEVELS:
-        return lvl
-    return True if os.environ.get("ONW_THINK_DEFAULT") == "1" else None
-
-
-def default_sampling():
-    """Sampling for a request that sends none: the model's saved choice, else the old setting (API_SAMPLING)."""
-    d = model_defaults()
-    mode = d.get("sampling")
-    if mode == "greedy":
-        return None
-    if mode == "recommended":
-        return {k: v for k, v in (info.get("sampling") or {}).items() if k != "source"} or None
-    if mode == "custom":
-        return {k: d[k] for k in ("temperature", "top_p", "top_k", "presence_penalty") if k in d}
-    return API_SAMPLING
-
-
-def gen_args(body):
-    """OpenAI request fields -> ChatEngine.stream_chat arguments."""
-    if _num(body, "n", 1, int) != 1:
-        raise HTTPException(400, "n > 1 is not supported (one NPU pipeline)")
-    stop = body.get("stop") or []
-    mx = body.get("max_completion_tokens", body.get("max_tokens"))
-    # a request without max_tokens: the setting (ONW_MAX_TOKENS; 0 = up to the end of the context - the engine stops
-    # there anyway); agents send none and write whole files in one answer
-    if mx is None:
-        mx = int(os.environ.get("ONW_MAX_TOKENS", "8192") or 0) or (1 << 30)
-    else:
-        mx = _num({"max_tokens": mx}, "max_tokens", 512, int)
-    if mx < 1:
-        raise HTTPException(400, "max_tokens must be at least 1")
-    think = (body.get("chat_template_kwargs") or {}).get("enable_thinking")
-    effort = body.get("reasoning_effort")
-    if isinstance(effort, str):                            # a level: low / medium / high (gpt-oss, llm-jp: trained;
-        think = False if effort in ("none", "minimal", "off") else effort   # others: a thinking budget)
-    if think is None:                                      # nothing said: the window's default for this model
-        think = default_think()
-    tools, choice = body.get("tools") or None, body.get("tool_choice")
-    if choice == "none":
-        tools = None
-    elif isinstance(choice, dict) and tools:              # a named function: offer just that one
-        name = (choice.get("function") or {}).get("name")
-        tools = [t for t in tools if (t.get("function") or {}).get("name") == name] or tools
-    # what the request leaves out: greedy, or the settings' API defaults (API_SAMPLING; thinking has its own set)
-    dflt = dict(default_sampling() or {})
-    if think and isinstance(dflt.get("think"), dict):
-        dflt.update(dflt["think"])
-    return dict(max_new_tokens=mx, temperature=_num(body, "temperature", dflt.get("temperature", 0.0)),
-                top_p=_num(body, "top_p", dflt.get("top_p", 1.0)),
-                stop=[stop] if isinstance(stop, str) else list(stop), seed=body.get("seed"), think=think,
-                top_k=_num(body, "top_k", dflt.get("top_k", 0), int), repetition_penalty=_num(body, "repetition_penalty", 1.0),
-                presence_penalty=_num(body, "presence_penalty", dflt.get("presence_penalty", 0.0)),
-                frequency_penalty=_num(body, "frequency_penalty", 0.0), tools=tools)
-
-
-def with_format(msgs, body):
-    """response_format json_object / json_schema: best effort - an instruction as a system message (the output is not
-    constrained)."""
-    rf = body.get("response_format") or {}
-    kind = rf.get("type") if isinstance(rf, dict) else None
-    if kind not in ("json_object", "json_schema"):
-        return msgs
-    note = "Respond with a single JSON object only, no other text."
-    if kind == "json_schema":
-        note += " It must follow this JSON schema: " + json.dumps((rf.get("json_schema") or {}).get("schema") or {},
-                                                                   ensure_ascii=False)
-    if msgs and msgs[0]["role"] == "system" and isinstance(msgs[0]["content"], str):
-        return [dict(msgs[0], content=msgs[0]["content"] + "\n\n" + note)] + msgs[1:]
-    return [{"role": "system", "content": note}] + msgs
-
-
-@app.get("/")
-def index(request: Request):
-    base = str(request.base_url).rstrip("/")
-    return {"object": "onw.server", "name": "onw - 俺のNPUがこんなに動くわけない", "version": __version__, "model": MODEL_ID,
-            "openai_base_url": base + "/v1", "api_key_required": bool(API_KEY),
-            "endpoints": {"models": "/v1/models", "chat_completions": "/v1/chat/completions", "health": "/health",
-                          "chat_ui": "/chat", "check_page": "/check", "info": "/api/info"},
-            "context": info.get("max_context"), "vision": info.get("vision")}
-
-
-@app.get("/health")
-def health():
-    return {"status": "ok", "model": MODEL_ID}
-
-
-@app.get("/chat")
-def chat_ui():
-    return HTMLResponse(i18n.page(os.path.join(WEB, "index.html")))
-
-
-@app.get("/check")
-def check_page():
-    return HTMLResponse(i18n.page(os.path.join(WEB, "check.html")))
-
-
-@app.get("/api/info")
-def api_info():
-    # the window's defaults for this model, read now (they change without a restart)
-    d, lvl = model_defaults(), default_think()
-    samp = default_sampling()
-    mode = d.get("sampling") or ("custom" if API_SAMPLING and os.environ.get("ONW_API_SAMPLING") != "recommended"
-                                 else "recommended" if API_SAMPLING else "greedy")
-    return {**info, "api_sampling": samp, "api_sampling_mode": mode,
-            "think_default": "off" if lvl is False else ("medium" if lvl is True else lvl)}
-
-
-STARTED = int(time.time())
-
-
-def _model_card():
-    return {"id": MODEL_ID, "object": "model", "created": STARTED, "owned_by": "local-npu"}
-
-
-@app.get("/v1/models")
-def models():
-    return {"object": "list", "data": [_model_card()]}
-
-
-@app.get("/v1/models/{model_id:path}")
-def model(model_id: str):
-    if model_id != MODEL_ID:
-        raise HTTPException(404, f"model {model_id!r} not found (this server runs {MODEL_ID!r})")
-    return _model_card()
-
-
-def finalize(text, tools, finish):
-    """The answer's end: tool calls parsed out of the text (OpenAI tool_calls), finish_reason "tool_calls" when there
-    are any. -> (text, calls, finish_reason)."""
-    if not tools:
-        return text, [], finish
-    text, calls = tool_calls.parse(text, tools)
-    return text, calls, ("tool_calls" if calls else finish)
-
-
-def usage_of(st):
-    return {"prompt_tokens": st["prompt_tokens"], "completion_tokens": st["completion_tokens"],
-            "total_tokens": st["prompt_tokens"] + st["completion_tokens"],
-            "prompt_tokens_details": {"cached_tokens": st.get("cached_tokens", 0)},
-            # (OpenAI's field: how many of the completion tokens were thinking)
-            "completion_tokens_details": {"reasoning_tokens": st.get("reasoning_tokens", 0)}}
-
-
-def stats_of(st):
-    return {k: (round(v, 2) if isinstance(v, float) else v) for k, v in st.items() if k != "finish_reason"}
-
-
-def worker_error(e):
-    """A generation error -> (HTTP status, OpenAI error type): the request's fault (a prompt too long, an image for a
-    text-only model) or the server's (the NPU, the driver)."""
-    return (400, "invalid_request_error") if isinstance(e, (ValueError, KeyError, TypeError)) else (500, "server_error")
-
-
-@app.post("/v1/chat/completions")
-async def chat(request: Request):
-    try:
-        body = await request.json()
-    except ValueError:
-        raise HTTPException(400, "the request body is not valid JSON")
-    if not isinstance(body, dict) or not isinstance(body.get("messages"), list):
-        raise HTTPException(400, "messages (a list) is required")
-    kw = gen_args(body)
-    loop = asyncio.get_running_loop()
-    try:                                                  # images may be downloaded: not on the event loop
-        msgs = await loop.run_in_executor(None, to_hf_messages, body["messages"])
-    except (KeyError, TypeError, AttributeError, ValueError) as e:
-        raise HTTPException(400, f"malformed messages: {e}")
-    msgs = with_format(msgs, body)
-    cid, created = f"chatcmpl-{uuid.uuid4().hex[:24]}", int(time.time())
-    tools = kw["tools"]
-    if body.get("stream"):
-        def chunk(delta, finish=None, **extra):
-            d = {"id": cid, "object": "chat.completion.chunk", "created": created, "model": MODEL_ID,
-                 "choices": [{"index": 0, "delta": delta, "logprobs": None, "finish_reason": finish}], **extra}
-            return f"data: {json.dumps(d, ensure_ascii=False)}\n\n"
-
-        async def events():
-            full, sent = "", 0                             # with tools: text is held back once a call may start
-            # (started here, not before the response: one whose stream never began would generate on to max_tokens)
-            q, stop = start_generation(msgs, kw)
-            next_item = reader(q, stop)
-            try:
-                yield chunk({"role": "assistant", "content": ""})
-                while True:
-                    d = await next_item()
-                    if d is None:
-                        break
-                    if isinstance(d, Exception):
-                        status, typ = worker_error(d)
-                        err = {"error": {"message": str(d), "type": typ, "param": None,
-                                         "code": "context_length_exceeded" if TOO_LONG.match(str(d)) else None}}
-                        yield f"data: {json.dumps(err, ensure_ascii=False)}\n\n"
-                    elif isinstance(d, dict) and "reasoning" in d:
-                        yield chunk({"reasoning_content": d["reasoning"]})
-                    elif isinstance(d, dict):
-                        rest, calls, finish = finalize(full, tools, d["finish_reason"])
-                        if tools and rest.startswith(full[:sent]) and rest[sent:].strip():   # text after the calls
-                            yield chunk({"content": rest[sent:]})
-                        for i, tc in enumerate(calls):
-                            yield chunk({"tool_calls": [dict(tc, index=i)]})
-                        yield chunk({}, finish, onw_stats=stats_of(d))
-                        if (body.get("stream_options") or {}).get("include_usage"):   # OpenAI: its own last chunk
-                            yield f"data: {json.dumps({'id': cid, 'object': 'chat.completion.chunk', 'created': created, 'model': MODEL_ID, 'choices': [], 'usage': usage_of(d)})}\n\n"
-                    elif tools:
-                        full += d
-                        upto = tool_calls.safe_upto(full)
-                        if upto > sent:
-                            yield chunk({"content": full[sent:upto]})
-                            sent = upto
-                    else:
-                        yield chunk({"content": d})
-                yield "data: [DONE]\n\n"
-            finally:  # client disconnected or finished: stop the NPU loop so the lock is released
-                stop.set()
-        return StreamingResponse(events(), media_type="text/event-stream",
-                                 headers={"Cache-Control": "no-cache", "X-Accel-Buffering": "no"})
-
-    parts, thought, st = [], [], None
-    q, stop = start_generation(msgs, kw)
-    next_item = reader(q, stop, request)                  # (a client that leaves stops the generation)
-    try:
-        while True:
-            d = await next_item()
-            if d is None:
-                break
-            if isinstance(d, Exception):
-                status, typ = worker_error(d)
-                return oai_error(status, str(d), typ)
-            if isinstance(d, dict) and "reasoning" in d:
-                thought.append(d["reasoning"])
-            elif isinstance(d, dict):
-                st = d
-            else:
-                parts.append(d)
-    finally:
-        stop.set()
-    if st is None:
-        return oai_error(500, "the generation ended without a result", "server_error")
-    text, calls, finish = finalize("".join(parts), tools, st["finish_reason"])
-    msg = {"role": "assistant", "content": (text.strip() or None) if tools else text}
-    if thought:
-        msg["reasoning_content"] = "".join(thought)
-    if calls:
-        msg["tool_calls"] = calls
-    return JSONResponse({
-        "id": cid, "object": "chat.completion", "created": created, "model": MODEL_ID,
-        "choices": [{"index": 0, "message": msg, "logprobs": None, "finish_reason": finish}],
-        "usage": usage_of(st),
-        "onw_stats": stats_of(st),
-    })
-
-
-def recommended_sampling(arch, engine_dir=None):
-    """The chat page's starting sampling values ({temperature, top_p, top_k, presence_penalty?, think?, source}) or
-    None (greedy): the model's own generation_config.json first (Gemma 4, Granite 4.2), else onw/sampling.json by
-    model_type prefix. The API itself stays greedy unless asked."""
-    keys = ("temperature", "top_p", "top_k")
-    try:
-        with open(os.path.join(engine_dir, "generation_config.json"), encoding="utf8") as f:
-            g = json.load(f)
-        if g.get("do_sample") and any(k in g for k in keys):
-            return {"source": "model", "temperature": g.get("temperature", 1.0), "top_p": g.get("top_p", 1.0),
-                    "top_k": g.get("top_k", 0)}
-    except (OSError, ValueError, TypeError):
-        pass
-    try:
-        with open(os.path.join(WEB, "..", "sampling.json"), encoding="utf8") as f:
-            table = {k: v for k, v in json.load(f).items() if not k.startswith("_")}
-    except (OSError, ValueError):
-        return None
-    hit = max((k for k in table if arch.startswith(k)), key=len, default=None)
-    if hit is None:
-        return None
-    return {k: v for k, v in table[hit].items() if k != "note"}
-
-
-from onw import compat  # noqa: E402,F401  (/v1/messages and /v1/responses on this app)
-
-
-def set_priority(busy):
-    """Windows: the server's priority class - normal while loading and waiting, above normal while it generates
-    (busy). Windows puts a below-normal process on the low-power cores at low clocks (onw 1.4.5 and earlier started
-    the server below normal: Lunar Lake, Qwen3.5-9B NF4 with MTP ~12 tok/s, 16 above normal); generating takes one or two
-    cores, so the rest of the PC hardly notices. ONW_PRIORITY: the class while generating, normal | above (default) |
-    off (leave it as started)."""
-    want = os.environ.get("ONW_PRIORITY", "above")
-    if os.name != "nt" or want not in ("normal", "above"):
-        return
-    import ctypes
-    k32 = ctypes.WinDLL("kernel32")
-    k32.GetCurrentProcess.restype = ctypes.c_void_p
-    k32.SetPriorityClass.argtypes = [ctypes.c_void_p, ctypes.c_uint32]
-    k32.SetPriorityClass(k32.GetCurrentProcess(), 0x8000 if busy and want == "above" else 0x20)
-
-
-def main(argv=None):
-    global pipe, MODEL_ID, API_KEY, API_SAMPLING
-    ap = argparse.ArgumentParser(prog="onw serve", description="chat UI + OpenAI-compatible API on the Intel NPU")
-    ap.add_argument("model", help="onw model folder, or a Hugging Face repo id (downloaded into ./models)")
-    ap.add_argument("--host", default="127.0.0.1")
-    ap.add_argument("--port", type=int, default=8000)
-    ap.add_argument("--device", default="NPU", help="run every component on this device (CPU works without an NPU driver)")
-    ap.add_argument("--name", help="model id / title (default: engine directory name)")
-    ap.add_argument("--context", type=int, default=None,
-                    help="context length in tokens (default 32768, at most the model's maximum); no recompile")
-    ap.add_argument("--open", action="store_true", help="open the chat UI in the default browser once loaded")
-    ap.add_argument("--no-pld", action="store_true", help="disable prompt lookup decoding")
-    ap.add_argument("--api-key", help="require this key as a Bearer token on /v1 from other machines "
-                                         "(e.g. with --host 0.0.0.0)")
-    a = ap.parse_args(argv)
-    API_KEY = a.api_key or os.environ.get("ONW_API_KEY") or None
-    from onw.hub import resolve
-    a.engine = resolve(a.model)
-    import openvino as ov
-    from onw.runtime import npu_info
-    print(f"[onw] {__version__} | OpenVINO {ov.__version__} | {npu_info(ov.Core())}", flush=True)
-    if a.device == "NPU" and "NPU" not in ov.Core().available_devices:
-        print("WARNING: no NPU found (is the NPU driver installed?) - running everything on the CPU instead.", flush=True)
-        a.device = "CPU"
-    from onw.chat import ChatEngine
-    pipe = ChatEngine(a.engine, a.device, context=a.context, pld=not a.no_pld)
-    set_priority(False)
-    MODEL_ID = a.name or os.path.basename(os.path.normpath(a.engine))
-    v = pipe.vision
-    info.update(model=MODEL_ID, title=MODEL_ID, devices={"lm": a.device}, fast=False, vision=bool(v),
-                image_px=v.get("image_px") if v else None,
-                think=pipe.think_style is not None, max_context=pipe.max_context,
-                always_thinks=pipe.think_style == "harmony",   # (gpt-oss, llm-jp: the thinking counts against max_tokens)
-                note=f"文脈は約{pipe.max_context}トークン", context=pipe.max_context, pld=pipe.pld,
-                mtp=getattr(pipe.model, "mtp", None) is not None)
-    try:
-        with open(os.path.join(a.engine, "config.json"), encoding="utf8") as f:
-            arch = json.load(f).get("model_type") or ""
-    except (OSError, ValueError):
-        arch = ""
-    # greedy Gemma on the NPU can fall into repeating one word (fp16 on the NPU tips a near tie; the same happens with
-    # any onw version): the pages suggest a repetition penalty, the API default stays 1.0 (OpenAI behaviour)
-    info.update(arch=arch, rep_hint=1.2 if arch.startswith("gemma") else None, sampling=recommended_sampling(arch, a.engine))
-    want = os.environ.get("ONW_API_SAMPLING")
-    if want == "recommended":
-        API_SAMPLING = {k: v for k, v in (info["sampling"] or {}).items() if k != "source"} or None
-    elif want:
-        try:
-            API_SAMPLING = json.loads(want)
-        except ValueError:
-            API_SAMPLING = None
-    info.update(api_sampling=API_SAMPLING, api_sampling_mode="recommended" if want == "recommended" else
-                "custom" if API_SAMPLING else "greedy")
-    if API_SAMPLING:
-        print(f"[api] requests without sampling values use {API_SAMPLING}", flush=True)
-    if a.open:
-        import webbrowser
-        threading.Timer(1.5, webbrowser.open, [f"http://{'localhost' if a.host in ('0.0.0.0', '127.0.0.1') else a.host}:{a.port}/chat"]).start()
-    print(f"\n  OpenAI API:  http://localhost:{a.port}/v1   (model id: {MODEL_ID})"
-          f"\n  Chat UI:     http://localhost:{a.port}/chat\n  Check page:  http://localhost:{a.port}/check\n", flush=True)
-    uvicorn.run(app, host=a.host, port=a.port, log_level="warning")
-
-
-if __name__ == "__main__":
-    # (python -m onw.server: run the module that onw.compat put its routes on - this one is a second copy of it)
-    from onw.server import main as _main
-    _main()
+"""OpenAI-compatible API server for an onw model on the Intel NPU (point other apps at http://HOST:PORT/v1).
+
+  GET  /                      what this server is: model, OpenAI base URL, endpoints (JSON)
+  GET  /v1/models
+  GET  /v1/models/{id}
+  POST /v1/chat/completions   (stream true/false; text and image_url parts: data: URLs or http(s)); responses
+                              also carry "onw_stats" (prefill / decode speed, reused tokens) - extra field
+  GET  /health                200 when the model is loaded
+  GET  /chat                  browser chat UI for trying the model
+  GET  /check                 verification page: speed, image, multi-turn reuse
+  GET  /api/info              model and device details
+
+usage: onw serve MODEL [--port 8000] [--host 0.0.0.0] [--api-key KEY] [--device NPU|CPU] [--open]
+CORS is open, so browser-based clients work too; with --api-key every /v1 request needs "Authorization: Bearer KEY"
+(except from this PC's own programs and pages: the chat and check pages send no key).
+Thinking mode: "chat_template_kwargs": {"enable_thinking": true}, or "reasoning_effort": "low" | "medium" | "high"
+(gpt-oss uses the level; "none" / "minimal" turn thinking off). Whatever the model thought comes back as
+message.reasoning_content (delta.reasoning_content when streaming; also for models that think with thinking off, as vLLM's
+reasoning parsers do), the answer as content, and the thinking's token count as OpenAI's
+usage.completion_tokens_details.reasoning_tokens.
+Tool calls: "tools" (OpenAI format) -> message.tool_calls / finish_reason "tool_calls"; results as role "tool".
+tool_choice "none" hides the tools; a named function narrows them to that one ("required" cannot be enforced).
+response_format json_object / json_schema is best effort (an instruction is added; the output is not constrained).
+Not supported: n > 1 (400), logprobs (null).
+One NPU pipeline serves requests one at a time (a lock serializes them).
+"""
+import argparse, asyncio, base64, io, ipaddress, json, os, re, socket, threading, time, uuid, urllib.parse, urllib.request
+
+import uvicorn
+from fastapi import FastAPI, HTTPException, Request
+from fastapi.exceptions import RequestValidationError
+from fastapi.responses import HTMLResponse, JSONResponse, StreamingResponse
+from PIL import Image
+from onw import __version__, i18n
+from onw import tools as tool_calls
+
+MODEL_ID = "npu-model"      # set from the engine directory in main()
+app = FastAPI(title="LLM on Intel NPU")
+pipe = None                 # onw.chat.ChatEngine, created in main()
+info = {}
+lock = threading.Lock()     # one generation at a time on the NPU
+WEB = os.path.join(os.path.dirname(os.path.abspath(__file__)), "web")
+API_KEY = None              # --api-key: required as a Bearer token on /v1
+API_SAMPLING = None         # sampling for requests that send none (ONW_API_SAMPLING: "recommended" or a JSON of values)
+TOO_LONG = re.compile(r"the prompt is (\d+) tokens; the context holds (\d+)")   # (ChatEngine.stream_chat's ValueError)
+
+
+def oai_error(status, message, type_="invalid_request_error", code=None):
+    """An error in OpenAI's shape: {"error": {"message", "type", "code"}} (a prompt over the context: code
+    "context_length_exceeded", as OpenAI's)."""
+    code = code or ("context_length_exceeded" if TOO_LONG.match(message) else None)
+    return JSONResponse({"error": {"message": message, "type": type_, "param": None, "code": code}}, status_code=status)
+
+
+def anthropic_message(message):
+    """Anthropic's wording for a prompt over the context ("prompt is too long: N tokens > M maximum" - Claude Code
+    compacts the conversation on it); other messages as they are."""
+    m = TOO_LONG.match(message)
+    return f"prompt is too long: {m[1]} tokens > {m[2]} maximum" if m else message
+
+
+def api_error(request, status, message, type_="invalid_request_error", code=None):
+    """oai_error, or on the Anthropic Messages API (/v1/messages...) Anthropic's shape:
+    {"type": "error", "error": {"type", "message"}}."""
+    if request.url.path.startswith("/v1/messages"):
+        typ = {401: "authentication_error", 403: "permission_error", 404: "not_found_error",
+               413: "request_too_large"}.get(status, "api_error" if status >= 500 else "invalid_request_error")
+        return JSONResponse({"type": "error", "error": {"type": typ, "message": anthropic_message(message)}},
+                            status_code=status)
+    return oai_error(status, message, type_, code)
+
+
+@app.exception_handler(HTTPException)
+async def _http_error(request, e):
+    return api_error(request, e.status_code, str(e.detail),
+                     "invalid_request_error" if e.status_code < 500 else "server_error")
+
+
+@app.exception_handler(RequestValidationError)
+async def _validation_error(request, e):
+    return api_error(request, 400, str(e))
+
+
+@app.exception_handler(Exception)
+async def _server_error(request, e):
+    return api_error(request, 500, f"{type(e).__name__}: {e}", "server_error")
+
+
+CORS = [(b"access-control-allow-origin", b"*"), (b"access-control-allow-headers", b"*"),
+        (b"access-control-allow-methods", b"GET, POST, OPTIONS")]
+
+
+class CorsAndKey:
+    """Open CORS (browser-based OpenAI clients) and the optional API key on /v1 (not asked of this PC's own programs and
+    pages - the chat and check pages send none; the key keeps the LAN out). A plain ASGI middleware, not
+    @app.middleware("http"): behind that one (Starlette's BaseHTTPMiddleware) request.is_disconnected() never turns
+    true, and a client that left a whole (non-streamed) answer kept the NPU busy to max_tokens."""
+    def __init__(self, inner):
+        self.inner = inner
+
+    async def __call__(self, scope, receive, send):
+        if scope["type"] != "http":
+            return await self.inner(scope, receive, send)
+        request = Request(scope)                       # (headers and client only: the body stays with the app)
+        path = scope["path"]
+        if path.startswith("/api/chats"):              # saved conversations: this PC's own chat page only
+            why = _chats_refused(request)
+            if why:
+                return await JSONResponse({"error": why}, status_code=403)(scope, receive, send)   # (no CORS)
+            return await self.inner(scope, receive, send)
+
+        async def with_cors(msg):
+            if msg["type"] == "http.response.start":
+                msg = {**msg, "headers": [h for h in msg.get("headers", []) if h[0].lower() not in dict(CORS)] + CORS}
+            await send(msg)
+        if scope["method"] == "OPTIONS":
+            return await JSONResponse({})(scope, receive, with_cors)
+        if API_KEY and path.startswith("/v1") and not _local(request) and \
+                request.headers.get("authorization", "") != f"Bearer {API_KEY}" and \
+                request.headers.get("x-api-key", "") != API_KEY:            # (Anthropic clients send x-api-key)
+            return await api_error(request, 401, "invalid API key", code="invalid_api_key")(scope, receive, with_cors)
+        await self.inner(scope, receive, with_cors)
+
+
+app.add_middleware(CorsAndKey)
+
+
+# ---------------------------------------------------------------- saved conversations (the chat page's history)
+LOCAL_HOSTS = ("localhost", "127.0.0.1", "[::1]", "::1")
+CHAT_ID = __import__("re").compile(r"^[a-z0-9-]{8,48}$")
+CHAT_MAX = 32 << 20                                     # bytes per conversation (images are kept small by the page)
+
+
+def _chats_refused(request):
+    """Why a request for the saved conversations is refused, or None: only from this PC (also when the server listens on
+    the LAN), only under a localhost name (a rebound DNS name of some web site would otherwise pass as same-origin) and
+    only from the chat page itself (no cross-site reads or writes: these responses carry no CORS headers)."""
+    client = request.client.host if request.client else ""
+    try:
+        if not ipaddress.ip_address(client).is_loopback:
+            return "only from this PC"
+    except ValueError:
+        return "only from this PC"
+    host = urllib.parse.urlsplit("//" + request.headers.get("host", "")).hostname or ""
+    if host not in LOCAL_HOSTS:
+        return "only under localhost"
+    origin = request.headers.get("origin")
+    if origin and (urllib.parse.urlsplit(origin).hostname or "") not in LOCAL_HOSTS:
+        return "not from another site"
+    return None
+
+
+def _local(request):
+    """True for a request from this PC itself as _chats_refused judges it (this PC's address, a localhost name, no other
+    site's page) and not one a proxy forwards (with X-Forwarded-For from a proxy on this PC, uvicorn already puts the
+    real client's address in request.client; a bare "Forwarded" it does not read)."""
+    return _chats_refused(request) is None and "forwarded" not in request.headers and \
+        "x-forwarded-for" not in request.headers
+
+
+def chats_dir():
+    from onw import app as _app
+    d = os.path.join(_app.config_dir(), "chats")
+    os.makedirs(d, exist_ok=True)
+    return d
+
+
+def _chat_path(cid):
+    if not CHAT_ID.match(cid or ""):
+        raise HTTPException(400, "bad conversation id")
+    return os.path.join(chats_dir(), cid + ".json")
+
+
+@app.get("/api/chats")
+def chats_list():
+    out = []
+    for f in os.listdir(chats_dir()):
+        if not f.endswith(".json"):
+            continue
+        try:
+            c = json.load(open(os.path.join(chats_dir(), f), encoding="utf8"))
+            out.append({k: c.get(k) for k in ("id", "title", "model", "created", "updated")} | {"n": len(c.get("messages", []))})
+        except (OSError, ValueError):
+            continue
+    return sorted(out, key=lambda c: c.get("updated") or 0, reverse=True)
+
+
+@app.get("/api/chats/{cid}")
+def chat_get(cid: str):
+    try:
+        return json.load(open(_chat_path(cid), encoding="utf8"))
+    except FileNotFoundError:
+        raise HTTPException(404, "no such conversation")
+
+
+@app.put("/api/chats/{cid}")
+async def chat_put(cid: str, request: Request):
+    p = _chat_path(cid)
+    raw = await request.body()
+    if len(raw) > CHAT_MAX:
+        raise HTTPException(413, "conversation too large")
+    try:
+        c = json.loads(raw)
+        assert isinstance(c, dict) and isinstance(c.get("messages"), list)
+    except (ValueError, AssertionError):
+        raise HTTPException(400, "not a conversation")
+    c["id"] = cid
+    with open(p + ".tmp", "w", encoding="utf8") as f:
+        json.dump(c, f, ensure_ascii=False)
+    os.replace(p + ".tmp", p)
+    return {"ok": True}
+
+
+@app.delete("/api/chats/{cid}")
+def chat_delete(cid: str):
+    try:
+        os.remove(_chat_path(cid))
+    except FileNotFoundError:
+        pass
+    return {"ok": True}
+
+
+@app.delete("/api/chats")
+def chats_delete_all():
+    n = 0
+    for f in os.listdir(chats_dir()):
+        if f.endswith(".json"):
+            try:
+                os.remove(os.path.join(chats_dir(), f))
+                n += 1
+            except OSError:
+                pass
+    return {"ok": True, "deleted": n}
+
+
+def start_generation(msgs, kw):
+    """Run stream_chat in a worker thread; returns (asyncio queue of deltas/stats/exceptions, None at the end; stop
+    event). Called on the event loop: the worker hands its items over to it, so no thread waits per queued request."""
+    loop, q, stop = asyncio.get_running_loop(), asyncio.Queue(), threading.Event()
+
+    def put(x):
+        try:
+            loop.call_soon_threadsafe(q.put_nowait, x)
+        except RuntimeError:                        # the event loop is gone (the server is stopping)
+            stop.set()
+
+    def work():
+        try:
+            with lock:
+                if stop.is_set():                   # the client left while this request waited for the NPU
+                    return
+                set_priority(True)
+                try:
+                    for d in pipe.stream_chat(msgs, should_stop=stop.is_set, split_reasoning=True, **kw):
+                        put(d)
+                finally:
+                    set_priority(False)
+        except Exception as e:                      # surfaced to the client
+            put(e)
+        finally:
+            put(None)
+    threading.Thread(target=work, daemon=True).start()
+    return q, stop
+
+
+def reader(q, stop, request=None):
+    """-> async function returning start_generation's next item. With request (a whole response, not a stream): a
+    client that leaves meanwhile stops the generation (checked about once a second; a stream stops when it closes)."""
+    last = [time.monotonic()]
+
+    async def next_item():
+        while request is not None and not stop.is_set():
+            wait = last[0] + 1.0 - time.monotonic()
+            if wait <= 0:
+                last[0] = time.monotonic()
+                if await request.is_disconnected():
+                    stop.set()
+                continue
+            try:
+                return await asyncio.wait_for(q.get(), wait)
+            except asyncio.TimeoutError:
+                pass
+        return await q.get()
+    return next_item
+
+
+MAX_IMAGE = 20 << 20                                    # bytes
+
+
+def _public_host(url):
+    """True when every address the URL's host resolves to is a public one: the server does not fetch from the local
+    network on a client's behalf (ONW_ALLOW_LOCAL_IMAGES=1 allows it)."""
+    if os.environ.get("ONW_ALLOW_LOCAL_IMAGES") == "1":
+        return True
+    host = urllib.parse.urlsplit(url).hostname
+    try:
+        addrs = {a[4][0] for a in socket.getaddrinfo(host, None)}
+    except (socket.gaierror, UnicodeError, TypeError):
+        return False
+    return bool(addrs) and all(ipaddress.ip_address(a.split("%")[0]).is_global for a in addrs)
+
+
+def load_image(url):
+    try:
+        if url.startswith("data:"):
+            data = base64.b64decode(url.split(",", 1)[1])
+        elif url.startswith(("http://", "https://")):
+            if not _public_host(url):
+                raise HTTPException(400, "image_url: only public http(s) addresses are fetched (send a data: URL instead)")
+            req = urllib.request.Request(url, headers={"User-Agent": f"onw/{__version__}"})
+            with urllib.request.urlopen(req, timeout=20) as r:
+                data = r.read(MAX_IMAGE + 1)
+            if len(data) > MAX_IMAGE:
+                raise HTTPException(400, "image_url: the image is larger than 20 MB")
+        else:
+            raise HTTPException(400, "image_url must be a data: URL or http(s) URL")
+        return Image.open(io.BytesIO(data)).convert("RGB")
+    except HTTPException:
+        raise
+    except Exception as e:                                # bad base64, not an image, unreachable URL
+        raise HTTPException(400, f"image_url: {type(e).__name__}: {e}")
+
+
+def to_hf_messages(messages):
+    """OpenAI chat messages -> HF chat format with PIL images."""
+    out = []
+    for m in tool_calls.history(messages):
+        keep = {k: m[k] for k in ("tool_calls", "tool_call_id", "name", "reasoning_content") if m.get(k) is not None}
+        c = m.get("content")
+        if c is None or isinstance(c, str):
+            out.append({"role": m["role"], "content": c or "", **keep})
+            continue
+        parts = []
+        for p in c:
+            if p.get("type") == "text":
+                parts.append({"type": "text", "text": p.get("text") or ""})
+            elif p.get("type") == "image_url":
+                iu = p.get("image_url") or {}
+                url = iu.get("url") if isinstance(iu, dict) else iu
+                if not isinstance(url, str):
+                    raise HTTPException(400, "image_url part without a url")
+                parts.append({"type": "image", "image": load_image(url)})
+            else:
+                raise HTTPException(400, f"unsupported content part type: {p.get('type')}")
+        out.append({"role": m["role"], "content": parts, **keep})
+    return out
+
+
+def _num(body, key, default, cast=float):
+    v = body.get(key)
+    if v is None:
+        return default
+    try:
+        return cast(v)
+    except (TypeError, ValueError):
+        raise HTTPException(400, f"{key} must be a number")
+
+
+LEVELS = ("off", "low", "medium", "high")   # the thinking levels of the window and the chat page (reasoning_effort)
+_PREFS = [None, None]                       # (config mtime, this model's defaults)
+
+
+def model_defaults():
+    """This model's defaults from the onw window (config model_prefs, read again when the file changes - no restart):
+    {"think": level, "sampling": "greedy" | "recommended" | "custom", values...}; {} when none were saved. Settings
+    of earlier versions (ONW_THINK_DEFAULT, ONW_API_SAMPLING) apply where these say nothing."""
+    key = os.environ.get("ONW_MODEL_KEY")
+    if not key:
+        return {}
+    try:
+        from onw import app as _app
+        p = os.path.join(_app.config_dir(), "config.json")
+        mt = os.path.getmtime(p)
+        if _PREFS[0] != mt:
+            _PREFS[:] = [mt, (_app.load_config().get("model_prefs") or {}).get(key) or {}]
+        return _PREFS[1]
+    except (OSError, ValueError):
+        return {}
+
+
+def default_think():
+    """Thinking for a request that says nothing about it: the model's saved level, else the old switch."""
+    lvl = model_defaults().get("think")
+    if lvl == "off":
+        return False
+    if lvl in LEVELS:
+        return lvl
+    return True if os.environ.get("ONW_THINK_DEFAULT") == "1" else None
+
+
+def default_sampling():
+    """Sampling for a request that sends none: the model's saved choice, else the old setting (API_SAMPLING)."""
+    d = model_defaults()
+    mode = d.get("sampling")
+    if mode == "greedy":
+        return None
+    if mode == "recommended":
+        return {k: v for k, v in (info.get("sampling") or {}).items() if k != "source"} or None
+    if mode == "custom":
+        return {k: d[k] for k in ("temperature", "top_p", "top_k", "presence_penalty") if k in d}
+    return API_SAMPLING
+
+
+def gen_args(body):
+    """OpenAI request fields -> ChatEngine.stream_chat arguments."""
+    if _num(body, "n", 1, int) != 1:
+        raise HTTPException(400, "n > 1 is not supported (one NPU pipeline)")
+    stop = body.get("stop") or []
+    mx = body.get("max_completion_tokens", body.get("max_tokens"))
+    # a request without max_tokens: the setting (ONW_MAX_TOKENS; 0 = up to the end of the context - the engine stops
+    # there anyway); agents send none and write whole files in one answer
+    if mx is None:
+        mx = int(os.environ.get("ONW_MAX_TOKENS", "8192") or 0) or (1 << 30)
+    else:
+        mx = _num({"max_tokens": mx}, "max_tokens", 512, int)
+    if mx < 1:
+        raise HTTPException(400, "max_tokens must be at least 1")
+    think = (body.get("chat_template_kwargs") or {}).get("enable_thinking")
+    effort = body.get("reasoning_effort")
+    if isinstance(effort, str):                            # a level: low / medium / high (gpt-oss, llm-jp: trained;
+        think = False if effort in ("none", "minimal", "off") else effort   # others: a thinking budget)
+    if think is None:                                      # nothing said: the window's default for this model
+        think = default_think()
+    tools, choice = body.get("tools") or None, body.get("tool_choice")
+    if choice == "none":
+        tools = None
+    elif isinstance(choice, dict) and tools:              # a named function: offer just that one
+        name = (choice.get("function") or {}).get("name")
+        tools = [t for t in tools if (t.get("function") or {}).get("name") == name] or tools
+    # what the request leaves out: greedy, or the settings' API defaults (API_SAMPLING; thinking has its own set)
+    dflt = dict(default_sampling() or {})
+    if think and isinstance(dflt.get("think"), dict):
+        dflt.update(dflt["think"])
+    return dict(max_new_tokens=mx, temperature=_num(body, "temperature", dflt.get("temperature", 0.0)),
+                top_p=_num(body, "top_p", dflt.get("top_p", 1.0)),
+                stop=[stop] if isinstance(stop, str) else list(stop), seed=body.get("seed"), think=think,
+                top_k=_num(body, "top_k", dflt.get("top_k", 0), int), repetition_penalty=_num(body, "repetition_penalty", 1.0),
+                presence_penalty=_num(body, "presence_penalty", dflt.get("presence_penalty", 0.0)),
+                frequency_penalty=_num(body, "frequency_penalty", 0.0), tools=tools)
+
+
+def with_format(msgs, body):
+    """response_format json_object / json_schema: best effort - an instruction as a system message (the output is not
+    constrained)."""
+    rf = body.get("response_format") or {}
+    kind = rf.get("type") if isinstance(rf, dict) else None
+    if kind not in ("json_object", "json_schema"):
+        return msgs
+    note = "Respond with a single JSON object only, no other text."
+    if kind == "json_schema":
+        note += " It must follow this JSON schema: " + json.dumps((rf.get("json_schema") or {}).get("schema") or {},
+                                                                   ensure_ascii=False)
+    if msgs and msgs[0]["role"] == "system" and isinstance(msgs[0]["content"], str):
+        return [dict(msgs[0], content=msgs[0]["content"] + "\n\n" + note)] + msgs[1:]
+    return [{"role": "system", "content": note}] + msgs
+
+
+@app.get("/")
+def index(request: Request):
+    base = str(request.base_url).rstrip("/")
+    return {"object": "onw.server", "name": "onw - 俺のNPUがこんなに動くわけない", "version": __version__, "model": MODEL_ID,
+            "openai_base_url": base + "/v1", "api_key_required": bool(API_KEY),
+            "endpoints": {"models": "/v1/models", "chat_completions": "/v1/chat/completions", "health": "/health",
+                          "chat_ui": "/chat", "check_page": "/check", "info": "/api/info"},
+            "context": info.get("max_context"), "vision": info.get("vision")}
+
+
+@app.get("/health")
+def health():
+    return {"status": "ok", "model": MODEL_ID}
+
+
+@app.get("/chat")
+def chat_ui():
+    return HTMLResponse(i18n.page(os.path.join(WEB, "index.html")))
+
+
+@app.get("/check")
+def check_page():
+    return HTMLResponse(i18n.page(os.path.join(WEB, "check.html")))
+
+
+@app.get("/api/info")
+def api_info():
+    # the window's defaults for this model, read now (they change without a restart)
+    d, lvl = model_defaults(), default_think()
+    samp = default_sampling()
+    mode = d.get("sampling") or ("custom" if API_SAMPLING and os.environ.get("ONW_API_SAMPLING") != "recommended"
+                                 else "recommended" if API_SAMPLING else "greedy")
+    return {**info, "api_sampling": samp, "api_sampling_mode": mode,
+            "think_default": "off" if lvl is False else ("medium" if lvl is True else lvl)}
+
+
+STARTED = int(time.time())
+
+
+def _model_card():
+    return {"id": MODEL_ID, "object": "model", "created": STARTED, "owned_by": "local-npu"}
+
+
+@app.get("/v1/models")
+def models():
+    return {"object": "list", "data": [_model_card()]}
+
+
+@app.get("/v1/models/{model_id:path}")
+def model(model_id: str):
+    if model_id != MODEL_ID:
+        raise HTTPException(404, f"model {model_id!r} not found (this server runs {MODEL_ID!r})")
+    return _model_card()
+
+
+def finalize(text, tools, finish):
+    """The answer's end: tool calls parsed out of the text (OpenAI tool_calls), finish_reason "tool_calls" when there
+    are any. -> (text, calls, finish_reason)."""
+    if not tools:
+        return text, [], finish
+    text, calls = tool_calls.parse(text, tools)
+    return text, calls, ("tool_calls" if calls else finish)
+
+
+def usage_of(st):
+    return {"prompt_tokens": st["prompt_tokens"], "completion_tokens": st["completion_tokens"],
+            "total_tokens": st["prompt_tokens"] + st["completion_tokens"],
+            "prompt_tokens_details": {"cached_tokens": st.get("cached_tokens", 0)},
+            # (OpenAI's field: how many of the completion tokens were thinking)
+            "completion_tokens_details": {"reasoning_tokens": st.get("reasoning_tokens", 0)}}
+
+
+def stats_of(st):
+    return {k: (round(v, 2) if isinstance(v, float) else v) for k, v in st.items() if k != "finish_reason"}
+
+
+def worker_error(e):
+    """A generation error -> (HTTP status, OpenAI error type): the request's fault (a prompt too long, an image for a
+    text-only model) or the server's (the NPU, the driver)."""
+    return (400, "invalid_request_error") if isinstance(e, (ValueError, KeyError, TypeError)) else (500, "server_error")
+
+
+@app.post("/v1/chat/completions")
+async def chat(request: Request):
+    try:
+        body = await request.json()
+    except ValueError:
+        raise HTTPException(400, "the request body is not valid JSON")
+    if not isinstance(body, dict) or not isinstance(body.get("messages"), list):
+        raise HTTPException(400, "messages (a list) is required")
+    kw = gen_args(body)
+    loop = asyncio.get_running_loop()
+    try:                                                  # images may be downloaded: not on the event loop
+        msgs = await loop.run_in_executor(None, to_hf_messages, body["messages"])
+    except (KeyError, TypeError, AttributeError, ValueError) as e:
+        raise HTTPException(400, f"malformed messages: {e}")
+    msgs = with_format(msgs, body)
+    cid, created = f"chatcmpl-{uuid.uuid4().hex[:24]}", int(time.time())
+    tools = kw["tools"]
+    if body.get("stream"):
+        def chunk(delta, finish=None, **extra):
+            d = {"id": cid, "object": "chat.completion.chunk", "created": created, "model": MODEL_ID,
+                 "choices": [{"index": 0, "delta": delta, "logprobs": None, "finish_reason": finish}], **extra}
+            return f"data: {json.dumps(d, ensure_ascii=False)}\n\n"
+
+        async def events():
+            full, sent = "", 0                             # with tools: text is held back once a call may start
+            # (started here, not before the response: one whose stream never began would generate on to max_tokens)
+            q, stop = start_generation(msgs, kw)
+            next_item = reader(q, stop)
+            try:
+                yield chunk({"role": "assistant", "content": ""})
+                while True:
+                    d = await next_item()
+                    if d is None:
+                        break
+                    if isinstance(d, Exception):
+                        status, typ = worker_error(d)
+                        err = {"error": {"message": str(d), "type": typ, "param": None,
+                                         "code": "context_length_exceeded" if TOO_LONG.match(str(d)) else None}}
+                        yield f"data: {json.dumps(err, ensure_ascii=False)}\n\n"
+                    elif isinstance(d, dict) and "reasoning" in d:
+                        yield chunk({"reasoning_content": d["reasoning"]})
+                    elif isinstance(d, dict):
+                        rest, calls, finish = finalize(full, tools, d["finish_reason"])
+                        if tools and rest.startswith(full[:sent]) and rest[sent:].strip():   # text after the calls
+                            yield chunk({"content": rest[sent:]})
+                        for i, tc in enumerate(calls):
+                            yield chunk({"tool_calls": [dict(tc, index=i)]})
+                        yield chunk({}, finish, onw_stats=stats_of(d))
+                        if (body.get("stream_options") or {}).get("include_usage"):   # OpenAI: its own last chunk
+                            yield f"data: {json.dumps({'id': cid, 'object': 'chat.completion.chunk', 'created': created, 'model': MODEL_ID, 'choices': [], 'usage': usage_of(d)})}\n\n"
+                    elif tools:
+                        full += d
+                        upto = tool_calls.safe_upto(full)
+                        if upto > sent:
+                            yield chunk({"content": full[sent:upto]})
+                            sent = upto
+                    else:
+                        yield chunk({"content": d})
+                yield "data: [DONE]\n\n"
+            finally:  # client disconnected or finished: stop the NPU loop so the lock is released
+                stop.set()
+        return StreamingResponse(events(), media_type="text/event-stream",
+                                 headers={"Cache-Control": "no-cache", "X-Accel-Buffering": "no"})
+
+    parts, thought, st = [], [], None
+    q, stop = start_generation(msgs, kw)
+    next_item = reader(q, stop, request)                  # (a client that leaves stops the generation)
+    try:
+        while True:
+            d = await next_item()
+            if d is None:
+                break
+            if isinstance(d, Exception):
+                status, typ = worker_error(d)
+                return oai_error(status, str(d), typ)
+            if isinstance(d, dict) and "reasoning" in d:
+                thought.append(d["reasoning"])
+            elif isinstance(d, dict):
+                st = d
+            else:
+                parts.append(d)
+    finally:
+        stop.set()
+    if st is None:
+        return oai_error(500, "the generation ended without a result", "server_error")
+    text, calls, finish = finalize("".join(parts), tools, st["finish_reason"])
+    msg = {"role": "assistant", "content": (text.strip() or None) if tools else text}
+    if thought:
+        msg["reasoning_content"] = "".join(thought)
+    if calls:
+        msg["tool_calls"] = calls
+    return JSONResponse({
+        "id": cid, "object": "chat.completion", "created": created, "model": MODEL_ID,
+        "choices": [{"index": 0, "message": msg, "logprobs": None, "finish_reason": finish}],
+        "usage": usage_of(st),
+        "onw_stats": stats_of(st),
+    })
+
+
+def recommended_sampling(arch, engine_dir=None):
+    """The chat page's starting sampling values ({temperature, top_p, top_k, presence_penalty?, think?, source}) or
+    None (greedy): the model's own generation_config.json first (Gemma 4, Granite 4.2), else onw/sampling.json by
+    model_type prefix. The API itself stays greedy unless asked."""
+    keys = ("temperature", "top_p", "top_k")
+    try:
+        with open(os.path.join(engine_dir, "generation_config.json"), encoding="utf8") as f:
+            g = json.load(f)
+        if g.get("do_sample") and any(k in g for k in keys):
+            return {"source": "model", "temperature": g.get("temperature", 1.0), "top_p": g.get("top_p", 1.0),
+                    "top_k": g.get("top_k", 0)}
+    except (OSError, ValueError, TypeError):
+        pass
+    try:
+        with open(os.path.join(WEB, "..", "sampling.json"), encoding="utf8") as f:
+            table = {k: v for k, v in json.load(f).items() if not k.startswith("_")}
+    except (OSError, ValueError):
+        return None
+    hit = max((k for k in table if arch.startswith(k)), key=len, default=None)
+    if hit is None:
+        return None
+    return {k: v for k, v in table[hit].items() if k != "note"}
+
+
+from onw import compat  # noqa: E402,F401  (/v1/messages and /v1/responses on this app)
+
+
+def set_priority(busy):
+    """Windows: the server's priority class - normal while loading and waiting, above normal while it generates
+    (busy). Windows puts a below-normal process on the low-power cores at low clocks (onw 1.4.5 and earlier started
+    the server below normal: Lunar Lake, Qwen3.5-9B NF4 with MTP ~12 tok/s, 16 above normal); generating takes one or two
+    cores, so the rest of the PC hardly notices. ONW_PRIORITY: the class while generating, normal | above (default) |
+    off (leave it as started)."""
+    want = os.environ.get("ONW_PRIORITY", "above")
+    if os.name != "nt" or want not in ("normal", "above"):
+        return
+    import ctypes
+    k32 = ctypes.WinDLL("kernel32")
+    k32.GetCurrentProcess.restype = ctypes.c_void_p
+    k32.SetPriorityClass.argtypes = [ctypes.c_void_p, ctypes.c_uint32]
+    k32.SetPriorityClass(k32.GetCurrentProcess(), 0x8000 if busy and want == "above" else 0x20)
+
+
+def main(argv=None):
+    global pipe, MODEL_ID, API_KEY, API_SAMPLING
+    ap = argparse.ArgumentParser(prog="onw serve", description="chat UI + OpenAI-compatible API on the Intel NPU")
+    ap.add_argument("model", help="onw model folder, or a Hugging Face repo id (downloaded into ./models)")
+    ap.add_argument("--host", default="127.0.0.1")
+    ap.add_argument("--port", type=int, default=8000)
+    ap.add_argument("--device", default="NPU", help="run every component on this device (CPU works without an NPU driver)")
+    ap.add_argument("--name", help="model id / title (default: engine directory name)")
+    ap.add_argument("--context", type=int, default=None,
+                    help="context length in tokens (default 32768, at most the model's maximum); no recompile")
+    ap.add_argument("--open", action="store_true", help="open the chat UI in the default browser once loaded")
+    ap.add_argument("--no-pld", action="store_true", help="disable prompt lookup decoding")
+    ap.add_argument("--api-key", help="require this key as a Bearer token on /v1 from other machines "
+                                         "(e.g. with --host 0.0.0.0)")
+    a = ap.parse_args(argv)
+    API_KEY = a.api_key or os.environ.get("ONW_API_KEY") or None
+    from onw.hub import resolve
+    a.engine = resolve(a.model)
+    import openvino as ov
+    from . import npukit
+    from onw.runtime import npu_info
+    print(f"[onw] {__version__} | OpenVINO {ov.__version__} | {npu_info(npukit.core())}", flush=True)
+    if a.device == "NPU" and "NPU" not in npukit.core().available_devices:
+        print("WARNING: no NPU found (is the NPU driver installed?) - running everything on the CPU instead.", flush=True)
+        a.device = "CPU"
+    from onw.chat import ChatEngine
+    pipe = ChatEngine(a.engine, a.device, context=a.context, pld=not a.no_pld)
+    set_priority(False)
+    MODEL_ID = a.name or os.path.basename(os.path.normpath(a.engine))
+    v = pipe.vision
+    info.update(model=MODEL_ID, title=MODEL_ID, devices={"lm": a.device}, fast=False, vision=bool(v),
+                image_px=v.get("image_px") if v else None,
+                think=pipe.think_style is not None, max_context=pipe.max_context,
+                always_thinks=pipe.think_style == "harmony",   # (gpt-oss, llm-jp: the thinking counts against max_tokens)
+                note=f"文脈は約{pipe.max_context}トークン", context=pipe.max_context, pld=pipe.pld,
+                mtp=getattr(pipe.model, "mtp", None) is not None)
+    try:
+        with open(os.path.join(a.engine, "config.json"), encoding="utf8") as f:
+            arch = json.load(f).get("model_type") or ""
+    except (OSError, ValueError):
+        arch = ""
+    # greedy Gemma on the NPU can fall into repeating one word (fp16 on the NPU tips a near tie; the same happens with
+    # any onw version): the pages suggest a repetition penalty, the API default stays 1.0 (OpenAI behaviour)
+    info.update(arch=arch, rep_hint=1.2 if arch.startswith("gemma") else None, sampling=recommended_sampling(arch, a.engine))
+    want = os.environ.get("ONW_API_SAMPLING")
+    if want == "recommended":
+        API_SAMPLING = {k: v for k, v in (info["sampling"] or {}).items() if k != "source"} or None
+    elif want:
+        try:
+            API_SAMPLING = json.loads(want)
+        except ValueError:
+            API_SAMPLING = None
+    info.update(api_sampling=API_SAMPLING, api_sampling_mode="recommended" if want == "recommended" else
+                "custom" if API_SAMPLING else "greedy")
+    if API_SAMPLING:
+        print(f"[api] requests without sampling values use {API_SAMPLING}", flush=True)
+    if a.open:
+        import webbrowser
+        threading.Timer(1.5, webbrowser.open, [f"http://{'localhost' if a.host in ('0.0.0.0', '127.0.0.1') else a.host}:{a.port}/chat"]).start()
+    print(f"\n  OpenAI API:  http://localhost:{a.port}/v1   (model id: {MODEL_ID})"
+          f"\n  Chat UI:     http://localhost:{a.port}/chat\n  Check page:  http://localhost:{a.port}/check\n", flush=True)
+    uvicorn.run(app, host=a.host, port=a.port, log_level="warning")
+
+
+if __name__ == "__main__":
+    # (python -m onw.server: run the module that onw.compat put its routes on - this one is a second copy of it)
+    from onw.server import main as _main
+    _main()

onw/update.py

@@ -86,8 +86,27 @@ def apply(tag, progress=lambda text: None):
     d = install_dir()
     if d is None:
         raise RuntimeError("this onw is not an installer-made install (update it with git / hf instead)")
-    info = json.loads(_get(f"https://huggingface.co/api/models/{REPO}/revision/{tag}"))
+    info = json.loads(_get(f"https://huggingface.co/api/models/{REPO}/revision/{tag}?blobs=true"))
     files = [s["rfilename"] for s in info["siblings"] if s["rfilename"] != ".gitattributes"]
+    # [3.0] the NPU kit of the other system is not downloaded (Windows: the DLLs, Linux: npukit/linux)
+    if os.name == "nt":
+        files = [f for f in files if not f.startswith("onw/npukit/linux/")]
+    else:
+        files = [f for f in files if not (f.startswith("onw/npukit/") and f.endswith(".dll"))]
+    # [3.0] large (LFS) files that are already here unchanged - the NPU kit's ~92 MB - are not downloaded again
+    lfs = {s["rfilename"]: s["lfs"] for s in info["siblings"] if s.get("lfs")}
+
+    def same(f):
+        meta, dst = lfs.get(f), os.path.join(d, f)
+        if not meta or not os.path.exists(dst) or os.path.getsize(dst) != meta.get("size"):
+            return False
+        import hashlib
+        h = hashlib.sha256()
+        with open(dst, "rb") as fh:
+            for chunk in iter(lambda: fh.read(1 << 22), b""):
+                h.update(chunk)
+        return h.hexdigest() == meta.get("sha256")
+    files = [f for f in files if not same(f)]
     stage = os.path.join(d, ".update")
     shutil.rmtree(stage, ignore_errors=True)
 
@@ -118,6 +137,12 @@ def apply(tag, progress=lambda text: None):
                 break
             except PermissionError:
                 if i == 9:
+                    if f.lower().endswith(".dll") and os.path.exists(dst):
+                        # a DLL loaded by a running onw process (the NPU kit): it can be renamed, not replaced -
+                        # moved aside (removed at the next start, see npukit) and the new one put in its place
+                        os.replace(dst, f"{dst}.old-{os.getpid()}-{int(time.time())}")
+                        os.replace(os.path.join(stage, f), dst)
+                        break
                     raise
                 time.sleep(0.5)
     # modules the new version no longer has: stale code must not be imported by accident

pyproject.toml

@@ -4,13 +4,14 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "onw"
-version = "1.3.0"
+version = "3.0.0"
 description = "Ore no NPU ga konna ni ugoku wake nai - LLMs entirely on the Intel NPU"
 readme = "README.md"
 license = {text = "Apache-2.0"}
 requires-python = ">=3.10"
 # openvino: from https://storage.openvinotoolkit.org/simple/wheels/pre-release until 2026.5 is on PyPI
-dependencies = ["openvino>=2026.5.0b1,<2026.6", "transformers>=5.5.4,<6", "numpy>=1.26", "pillow", "fastapi", "uvicorn",
+# [3.0] exactly the OpenVINO build onw's NPU kit (onw/npukit) is made for (2026.5.0b1 = 2026.5.0-23160-6db412d0a27)
+dependencies = ["openvino==2026.5.0b1", "transformers>=5.5.4,<6", "numpy>=1.26", "pillow", "fastapi", "uvicorn",
                 "huggingface_hub>=1.0,<2", "jinja2"]
 
 [project.optional-dependencies]
@@ -24,4 +25,4 @@ onw = "onw.__main__:main"
 include = ["onw*"]
 
 [tool.setuptools.package-data]
-onw = ["web/*"]
+onw = ["web/*", "npukit/*", "npukit/patches/*", "npukit/linux/*"]

requirements.txt

@@ -1,7 +1,7 @@
 # Same dependencies as pyproject.toml (pip install --pre -e . is what start.bat / start.sh do).
 # OpenVINO is a pre-release build (2026.5.0b1, from OpenVINO's pre-release index):
 #   pip install -r requirements.txt --extra-index-url https://storage.openvinotoolkit.org/simple/wheels/pre-release
-openvino>=2026.5.0b1,<2026.6
+openvino==2026.5.0b1
 transformers>=5.5.4,<6
 numpy>=1.26
 pillow