onw 3.2.0: Ternary Bonsai 2 4B / 9B 64-token graphs (one copy of the 2-bit weights), parallel compile in processes, faster kit compiler, paged graphs released under fused ones, vision encoder in the background

CHANGELOG.md

@@ -5,6 +5,22 @@
 日付は日本時間です。各版は Hugging Face の `ryugyosoft/onw` にタグ（`v0.4.2` 以降）付きで残っています。
 onw ウィンドウは、新しい版があるとこのファイルから変更点を表示します（`## v…` の見出しで区切っています）。
 
+## v3.2.0 (2026-10-20)
+- **Ternary Bonsai 2 4B／9B のプロンプト処理を 10 倍以上に（64 トークン用グラフ、2 ビットの重みは 1 組のまま）**：これまで 2 ビットの重みは 1 トークンずつしか正しく計算できず、プロンプトも 1 トークンずつでした。onw の NPU キットのコンパイラーで 64 トークンのブロックを正しく速く計算できるようにし、NPU 上の 2 ビットの重みを 1 トークン用と 64 トークン用のグラフで共有します（重みを 2 組持たない）。モデルの更新（64 トークン用のグラフ入り）が必要です（モデルの一覧に「更新」が出ます。新しいグラフは onw 3.2 以降だけが使い、3.1.0 以前はこれまでどおり 1 トークンずつ動きます）。Panther Lake（Core Ultra X7 358H）、3.1.0 との比較：
+  - Bonsai 2 4B：約 1,200 トークンのプロンプト 25 → 約 300 tok/s、生成 20〜21.6 → 約 22 tok/s、専用メモリ 2.96 → 約 2.1 GB、初回のロード 約 3.6 → 約 6 分（2 回目以降 約 20 秒）。
+  - Bonsai 2 9B：プロンプト 約 190 tok/s、生成 約 14.5 tok/s、専用メモリ 約 3.2 GB、初回のロード 約 12 分（2 回目以降 約 35 秒）。
+  - 画像の質問も、64 トークンのブロックで読み込みます。
+  - **正直に書くと、NPU のメモリは 2 ビットの重みよりずっと大きく、ここは失敗です**：重みとスケール 約 0.9 / 2.1 GB に対してNPU 約 2.6 / 4.5 GB（4B / 9B）。グラフごとに要る重みテーブルが、1 トークン用と 64 トークン用の 2 組で重みと同じくらいあるためです。Intel に改善を要望しています。
+- **Ternary Bonsai 2 27B は 1 トークン用のグラフのままです（64 トークン用は試作したが失敗）**：作って動かすことはできました（プロンプト 約 60 tok/s）が、NPU のメモリを 13.8 GB 使い、16 GB の PC に載りません。2 ビットの重みを NPU で計算するには、グループ（128 個の重み）ごとに 16 バイトの「重みテーブル」がグラフごとに必要で、27B ではグラフ 1 組あたり 3.4 GB になるためです。初回のコンパイルも Panther Lake で 36 分かかりました。Intel に改善を要望しています。
+- **初回のロードを速く：コンパイルを複数のプロセスで同時に**：NPU のコンパイラーはグラフ 1 つに CPU を約 1 コアしか使わないので、空きメモリとコア数に応じて最大 6 つの子プロセスで同時にコンパイルします（省メモリモードでは最大 2）。子プロセスはコンパイルして保存するだけで、NPU には読み込みません（重みが二重にならない）。Panther Lake で Bonsai 2 4B の 1 トークン用＋64 トークン用のグラフ：819 → 約 180 秒。融合グラフの裏でのコンパイルも、既定で同時に行います。
+- **NPU コンパイラーの高速化（キット）**：戦略探索で同じ計算を繰り返していた所を直し、2 ビットのモデルの 64 トークン用グラフ 1 つのコンパイルが 335 → 約 20 秒になりました（結果のバイナリーは同じ）。
+- **融合グラフを使う大きさでは、従来のグラフをメモリから外す**：融合グラフが動く範囲では従来のグラフは使わないので、読み込んだあとで外します（重みテーブルの分だけ省メモリ。Bonsai 2 4B：NPU 3.9 → 2.6 GB）。画像の質問や、融合グラフより長い文脈のときだけ、その場で読み込み直します（数秒）。
+- **画像エンコーダーを裏でコンパイル**：画像の付いたモデルは、初回のロードのあと裏で画像エンコーダーをコンパイルしておき、最初の画像の質問で数分待たないようにしました。
+- **Linux 版のキットも更新**しました（上の修正を含む。Bonsai 2 の 64 トークン用グラフも、Ubuntu の Lunar Lake で確かめました）。
+- 2 ビットのモデルでは、融合グラフの長さ違いの種類を全部は常駐させず、1 つずつ入れ替えます（種類ごとに重みテーブルが要るため）。
+- 融合グラフでも、最後の正規化の fp16 のあふれで 0 になった行を、縮めて計算し直します（従来のグラフと同じ処理。Bonsai 2 9B の 64 トークンのブロックで、Lunar Lake の 256 行中 20 行が一様分布になっていた）。
+- 2 ビットのモデルでは、先読み検証（PLD）を使いません（下書きの確認に 64 トークン用のグラフを使うこ��になり、Lunar Lake で生成が 23 → 13〜17 tok/s に落ちていた）。
+
 ## v3.1.0 (2026-10-19)
 - **Qwen 系を速く（設定「Qwen 系を高速化する」、既定オン）**：Qwen3.5・Ornith など、これまで NPUW で動いていたモデルを、Lunar Lake 以降の PC（onw の NPU キット）では区間ごとの融合グラフと MTP 先読みで動かします。重みは 1 組のまま（キットの共有）で、同じ質問には 3.0.0 と同じ答えが返ります。
   - Panther Lake（Core Ultra X7 358H）の Qwen3.5-9B NF4、3.0.0 との比較：約 3.5K トークンのプロンプト 205 → 299 tok/s、約 900 トークン 242 → 280 tok/s、生成 17.9 → 19.3 tok/s。

CHANGELOG_en.md

@@ -4,6 +4,37 @@
 
 Dates are JST. Every version is on Hugging Face `ryugyosoft/onw`, tagged from `v0.4.2` on.
 
+## v3.2.0 (2026-10-20)
+- **Ternary Bonsai 2 4B / 9B: prompts more than 10x faster (64-token graphs, the 2-bit weights still held once)**:
+  until now 2-bit weights were only computed right one token at a time, so prompts went one token at a time too. The
+  compiler of onw's NPU kit now computes 64-token blocks right and fast, and the 1-token and the 64-token graphs share
+  the 2-bit weights on the NPU (no second copy). Needs the updated models (with 64-token graphs; the model list offers the update. Only onw 3.2 and later use the new graphs - 3.1.0 and earlier keep going token by token). Panther Lake
+  (Core Ultra X7 358H), against 3.1.0:
+  - Bonsai 2 4B: ~1,200-token prompt 25 -> ~300 tok/s, answers 20-21.6 -> ~22 tok/s, private memory 2.96 -> ~2.1 GB,
+    first load ~3.6 -> ~6 min (later loads ~20 s).
+  - Bonsai 2 9B: prompts ~190 tok/s, answers ~14.5 tok/s, private memory ~3.2 GB, first load ~12 min (later ~35 s).
+  - Image questions are read in 64-token blocks too.
+  - **To be frank, the NPU memory is far above the 2-bit weights - a failure on that count**: weights + scales ~0.9 / 2.1 GB, NPU ~2.6 / 4.5 GB (4B / 9B). The weight tables every graph needs are, for the 1-token and the 64-token set, about as large as the weights. Requested from Intel.
+- **Ternary Bonsai 2 27B keeps its 1-token graphs (64-token graphs prototyped - and failed)**: they built and ran
+  (prompts ~60 tok/s), but used 13.8 GB of NPU memory - no 16 GB PC holds that. Computing 2-bit weights on the NPU needs
+  a 16-byte "weight table" entry per group of 128 weights in every graph, 3.4 GB per set of graphs for the 27B. The first
+  compile also took 36 minutes on Panther Lake. Reported to Intel with a request.
+- **Faster first loads: graphs compile in parallel processes**: the NPU compiler uses about one CPU core per graph, so up
+  to 6 child processes compile at the same time, as the free memory and the cores allow (2 at most in the memory-saving
+  mode). The children only compile and save; nothing is loaded on the NPU twice. Panther Lake, Bonsai 2 4B 1-token +
+  64-token graphs: 819 -> ~180 s. The fused graphs' background compile runs in parallel by default as well.
+- **Faster NPU compiler (kit)**: the strategy search no longer repeats identical work; one 64-token graph of a 2-bit model
+  compiles in ~20 instead of 335 s (same binary).
+- **The paged graphs leave memory where the fused graphs run**: they are not used for the positions the fused graphs
+  cover, so they are released after loading (their weight tables: Bonsai 2 4B NPU memory 3.9 -> 2.6 GB). Image questions
+  and contexts longer than the fused graphs load them again on the spot (a few seconds).
+- **The vision encoder compiles in the background**: models that read images compile their encoder right after the
+  first load, so the first image question does not wait minutes.
+- **The Linux kit is updated** too (with the fixes above; Bonsai 2's 64-token graphs checked on Lunar Lake under Ubuntu too).
+- 2-bit models keep one fused variant at a time instead of all lengths (each variant needs its own weight tables).
+- The fused graphs recompute rows the final norm's fp16 overflow zeroed, scaled down (as the paged graphs do; Bonsai 2 9B 64-token blocks on Lunar Lake: 20 of 256 rows came out uniform).
+- 2-bit models do not use prompt-lookup decoding (a draft check would run the 64-token graphs: answers fell from 23 to 13-17 tok/s on Lunar Lake).
+
 ## v3.1.0 (2026-10-19)
 - **The Qwen family faster (setting "Speed up the Qwen family", on by default)**: Qwen3.5, Ornith and the other models that ran through NPUW run fused segment graphs with MTP draft tokens on PCs from Lunar Lake on (onw's NPU kit). One copy of the weights as before (the kit shares them), and the same question gets the same answer as with 3.0.0.
   - Panther Lake (Core Ultra X7 358H), Qwen3.5-9B NF4 against 3.0.0: a ~3.5K-token prompt 205 -> 299 tok/s, ~900 tokens 242 -> 280 tok/s, answers 17.9 -> 19.3 tok/s.

README.md

@@ -105,8 +105,8 @@ Python や git の知識は要りません。Python が入っていなければ
 | [ryugyosoft/Ornith-1.5-9B-onw](https://huggingface.co/ryugyosoft/Ornith-1.5-9B-onw) | `qwen3_5`（Qwen3.5-9B をエージェント・ツール呼び出し向けに鍛えた Ornith-1.5） | テキスト＋画像 | 切替 | 4.9 tok/s | 5.3 GB | 約 11 GB |
 | [ryugyosoft/Ornith-1.5-9B-nf4-onw](https://huggingface.co/ryugyosoft/Ornith-1.5-9B-nf4-onw) | 同上、重みはチャネル単位 NF4、MTP 先読み付き（**Lunar Lake 以降専用**） | テキスト＋画像 | 切替 | —（Lunar Lake で約 15 tok/s、通常版の約 2 倍） | 5.6 GB | 通常版の約 85% |
 | [ryugyosoft/Ternary-Bonsai-2-27B-onw](https://huggingface.co/ryugyosoft/Ternary-Bonsai-2-27B-onw) | `qwen3_5`（Qwen3.8-27B を三値の重みで学習し直した Ternary Bonsai 2。重みは 2 ビットのまま。**Lunar Lake 以降専用**） | テキスト | 切替（既定オフ） | —（Lunar Lake で約 4 tok/s。プロンプトは 1 トークンずつ） | 7.6 GB | 約 10 GB |
-| [ryugyosoft/Ternary-Bonsai-2-9B-onw](https://huggingface.co/ryugyosoft/Ternary-Bonsai-2-9B-onw) | `qwen3_5`（Qwen3.5-9B を三値の重みで学習し直した Ternary Bonsai 2。重みは 2 ビットのまま。**Lunar Lake 以降専用**） | テキスト＋画像 | 切替 | —（Lunar Lake で約 13 tok/s。プロンプトは 1 トークンずつ） | 3.0 GB | 約 3.5 GB |
-| [ryugyosoft/Ternary-Bonsai-2-4B-onw](https://huggingface.co/ryugyosoft/Ternary-Bonsai-2-4B-onw) | `qwen3_5`（同じく Qwen3.5-4B の三値版。**Lunar Lake 以降専用**） | テキスト＋画像 | 切替 | —（Lunar Lake で約 21 tok/s。プロンプトは 1 トークンずつ） | 1.8 GB | 約 2 GB |
+| [ryugyosoft/Ternary-Bonsai-2-9B-onw](https://huggingface.co/ryugyosoft/Ternary-Bonsai-2-9B-onw) | `qwen3_5`（Qwen3.5-9B を三値の重みで学習し直した Ternary Bonsai 2。重みは 2 ビットのまま。**Lunar Lake 以降専用**） | テキスト＋画像 | 切替 | —（Lunar Lake で約 13 tok/s。3.2.0 からプロンプトは 64 トークンずつ） | 3.0 GB | 約 3.5 GB |
+| [ryugyosoft/Ternary-Bonsai-2-4B-onw](https://huggingface.co/ryugyosoft/Ternary-Bonsai-2-4B-onw) | `qwen3_5`（同じく Qwen3.5-4B の三値版。**Lunar Lake 以降専用**） | テキスト＋画像 | 切替 | —（Lunar Lake で約 21 tok/s。3.2.0 からプロンプトは 64 トークンずつ） | 1.8 GB | 約 2 GB |
 | [ryugyosoft/NeoHorse-1-4B-onw](https://huggingface.co/ryugyosoft/NeoHorse-1-4B-onw) | `qwen3_5`（Qwen3.5-4B をエージェント・ツール呼び出し向けに追加学習した NeoHorse-1） | テキスト | 切替 | 6.4 tok/s | 2.4 GB | 約 6 GB |
 | [ryugyosoft/granite-4.2-3b-onw](https://huggingface.co/ryugyosoft/granite-4.2-3b-onw) | `granite`（IBM Granite 4.2：思考とツール呼び出しを強化した 3B。Llama 型） | テキスト | 切替 | 6.7 tok/s（チャット約 7.7） | 2.1 GB | 約 6 GB |
 | [ryugyosoft/granite-4.2-8b-onw](https://huggingface.co/ryugyosoft/granite-4.2-8b-onw) | `granite`（同上の 8B） | テキスト | 切替 | 3.6 tok/s（チャット約 3.9） | 5.0 GB | 約 13 GB |
@@ -128,7 +128,7 @@ down_proj だけチャネル単位 INT8。INT4 版より元のモデルに近く
 
 **2 ビット版（onw 1.4.9、Lunar Lake 以降専用）**：Lunar Lake 以降の NPU は 2 ビットの重みも計算できます。Ternary Bonsai 2 27B は三値（−1・0・+1）で
 学習されたモデルで、重みを 2 ビットのまま NPU に載せると 27B がメモリ約 10 GB で動きます。ただし今の NPU コンパイラーは 2 ビットの重みを複数トークンの
-ブロックで計算すると間違えるため、プロンプトは 1 トークンずつ（約 4 tok/s）処理します。実用というより「NPU で 27B が動く」ロマン向けです（Intel に報告済み）。onw 1.4.13 では同じ三値の Ternary Bonsai 2 9B / 4B も加えました。こちらは画像も読め、メモリ約 3.5 GB / 約 2 GB と軽く、Lunar Lake で生成約 13 / 21 tok/s です（プロンプトは同じく 1 トークンずつ）。
+ブロックで計算すると間違えるため、プロンプトは 1 トークンずつ（約 4 tok/s）処理します。実用というより「NPU で 27B が動く」ロマン向けです（Intel に報告済み）。onw 1.4.13 では同じ三値の Ternary Bonsai 2 9B / 4B も加えました。こちらは画像も読め、メモリ約 3.5 GB / 約 2 GB と軽く、Lunar Lake で生成約 13 / 21 tok/s です。onw 3.2.0 では、NPU キットのコンパイラーでこの不具合を直し、9B / 4B に 64 トークン用のグラフを加えました（2 ビットの重みは 1 組のまま共有。Panther Lake でプロンプト約 190 / 300 tok/s）。27B も試作しましたが、NPU のメモリを 13.8 GB 使うため出していません（1 トークンずつのまま）。
 
 **MTP 先読み（onw 1.4.0、Lunar Lake 以降）**：Qwen3.5 系のモデルには、次の次のト��クンを予想する小さな層（MTP）が付いています。
 onw はこれで 2 トークン先まで下書きし、本体の 4 トークン用のグラフで一度に確かめます（外れた下書きの分だけ正確に巻き戻します）。
@@ -328,7 +328,10 @@ KV キャッシュは INT8 で、会話が実際に使った長さの分だけ
   （Windows 11、ドライバー 32.0.100.5540、2026 年 10 月 6 日。1 パーツは 4 トークン用グラフの最初のパーツ。命令の大きさは
   コンパイル結果の ELF セクションの大きさで、命令の数そのものではありません。）
 
-### 既知の問題（onw 3.1.0、2026 年 10 月 19 日）
+### 既知の問題（onw 3.2.0、2026 年 10 月 20 日）
+
+- **Ternary Bonsai 2 27B の 64 トークン用グラフは試作したが失敗**：NPU のメモリを 13.8 GB 使い（グラフごとに要る 2 ビットの重みテーブルが 1 組 3.4 GB）、16 GB の PC に載らないため出していません。27B のプロンプトは 1 トークンずつのままです。Intel に要望しています。
+- **Ternary Bonsai 2 の 64 トークン版の初回のロードは長め**：Panther Lake で 4B 約 6 分、9B 約 12 分（2 回目以降は 20〜35 秒）。
 
 - **Qwen 系の高速化をオンにした最初のロードは長め**：融合グラフのコンパイルが加わります（Panther Lake の小型機の Qwen3.5-9B NF4：使い始めるまで約 13 分、残りの種類は回答しながら裏で約 25 分）。2 回目以降はキャッシュから数十秒です。
 - **INT4 版のモデルと Meteor Lake・Arrow Lake（NPU 3720）は 2.1.0 と同じです**：INT4 の重みは重みを持たないグラフにできず、NPU 3720 は

README_en.md

@@ -92,8 +92,8 @@ troubleshooting), is the **[user guide](MANUAL_en.md)**.
 | [ryugyosoft/Ornith-1.5-9B-onw](https://huggingface.co/ryugyosoft/Ornith-1.5-9B-onw) | `qwen3_5` (Ornith-1.5: Qwen3.5-9B trained further for agents and tool calls) | text + image | switchable | 4.9 tok/s | 5.3 GB | ~11 GB |
 | [ryugyosoft/Ornith-1.5-9B-nf4-onw](https://huggingface.co/ryugyosoft/Ornith-1.5-9B-nf4-onw) | the same, channel-wise NF4 weights, MTP draft tokens (**Lunar Lake and newer only**) | text + image | switchable | - (~15 tok/s on Lunar Lake, ~2x the regular build) | 5.6 GB | ~85% of the regular build |
 | [ryugyosoft/Ternary-Bonsai-2-27B-onw](https://huggingface.co/ryugyosoft/Ternary-Bonsai-2-27B-onw) | `qwen3_5` (Ternary Bonsai 2: Qwen3.8-27B retrained with ternary weights, kept 2-bit; **Lunar Lake and newer only**) | text | switchable (off by default) | - (~4 tok/s on Lunar Lake; prompts token by token) | 7.6 GB | ~10 GB |
-| [ryugyosoft/Ternary-Bonsai-2-9B-onw](https://huggingface.co/ryugyosoft/Ternary-Bonsai-2-9B-onw) | `qwen3_5` (Ternary Bonsai 2: Qwen3.5-9B retrained with ternary weights, kept 2-bit; **Lunar Lake and newer only**) | text + image | switchable | - (~13 tok/s on Lunar Lake; prompts token by token) | 3.0 GB | ~3.5 GB |
-| [ryugyosoft/Ternary-Bonsai-2-4B-onw](https://huggingface.co/ryugyosoft/Ternary-Bonsai-2-4B-onw) | `qwen3_5` (the same for Qwen3.5-4B; **Lunar Lake and newer only**) | text + image | switchable | - (~21 tok/s on Lunar Lake; prompts token by token) | 1.8 GB | ~2 GB |
+| [ryugyosoft/Ternary-Bonsai-2-9B-onw](https://huggingface.co/ryugyosoft/Ternary-Bonsai-2-9B-onw) | `qwen3_5` (Ternary Bonsai 2: Qwen3.5-9B retrained with ternary weights, kept 2-bit; **Lunar Lake and newer only**) | text + image | switchable | - (~13 tok/s on Lunar Lake; prompts 64 tokens at a time from 3.2.0) | 3.0 GB | ~3.5 GB |
+| [ryugyosoft/Ternary-Bonsai-2-4B-onw](https://huggingface.co/ryugyosoft/Ternary-Bonsai-2-4B-onw) | `qwen3_5` (the same for Qwen3.5-4B; **Lunar Lake and newer only**) | text + image | switchable | - (~21 tok/s on Lunar Lake; prompts 64 tokens at a time from 3.2.0) | 1.8 GB | ~2 GB |
 | [ryugyosoft/NeoHorse-1-4B-onw](https://huggingface.co/ryugyosoft/NeoHorse-1-4B-onw) | `qwen3_5` (NeoHorse-1: Qwen3.5-4B trained further for agents and tool calls) | text | switchable | 6.4 tok/s | 2.4 GB | ~6 GB |
 | [ryugyosoft/granite-4.2-3b-onw](https://huggingface.co/ryugyosoft/granite-4.2-3b-onw) | `granite` (IBM Granite 4.2: a 3B with thinking and stronger tool calls; Llama-type) | text | switchable | 6.7 tok/s (~7.7 in chat) | 2.1 GB | ~6 GB |
 | [ryugyosoft/granite-4.2-8b-onw](https://huggingface.co/ryugyosoft/granite-4.2-8b-onw) | `granite` (the same, 8B) | text | switchable | 3.6 tok/s (~3.9 in chat) | 5.0 GB | ~13 GB |
@@ -117,7 +117,7 @@ original than the INT4 builds; on Panther Lake Granite 4.2 8B decodes ~1.7x as f
 **2-bit build (onw 1.4.9, Lunar Lake and newer only)**: NPUs from Lunar Lake on compute 2-bit weights too. Ternary Bonsai 2 27B was
 trained with ternary weights (-1, 0, +1); kept 2-bit on the NPU, the 27B runs in about 10 GB of memory. Today's NPU compilers compute
 2-bit weights wrong in blocks of more than one token, so prompts are processed token by token (~4 tok/s): a "27B on the NPU" showpiece
-rather than a daily driver (reported to Intel). onw 1.4.13 adds the ternary Ternary Bonsai 2 9B / 4B as well: they read images, need only ~3.5 / ~2 GB of memory and generate ~13 / ~21 tok/s on Lunar Lake (prompts token by token too).
+rather than a daily driver (reported to Intel). onw 1.4.13 adds the ternary Ternary Bonsai 2 9B / 4B as well: they read images, need only ~3.5 / ~2 GB of memory and generate ~13 / ~21 tok/s on Lunar Lake. onw 3.2.0 fixes the compiler bug in the NPU kit and adds 64-token graphs to the 9B / 4B (sharing one copy of the 2-bit weights; prompts ~190 / ~300 tok/s on Panther Lake). We prototyped them for the 27B too, but it took 13.8 GB of NPU memory, so the 27B stays token by token.
 
 **MTP draft tokens (onw 1.4.0, Lunar Lake and newer)**: Qwen3.5-family models carry a small layer (MTP) that predicts the
 token after next. onw uses it to draft two tokens ahead and checks them in one pass on the model's 4-token graphs
@@ -316,7 +316,10 @@ How it was measured, and caveats:
   (Windows 11, driver 32.0.100.5540, 2026-10-06. One part = the first part of a 4-token graph. Code sizes are ELF section sizes of
   the compiled file, not instruction counts.)
 
-### Known issues (onw 3.1.0, 2026-10-19)
+### Known issues (onw 3.2.0, 2026-10-20)
+
+- **64-token graphs for Ternary Bonsai 2 27B: prototyped, and failed**: they took 13.8 GB of NPU memory (the 2-bit weight tables every graph needs are 3.4 GB per set), which no 16 GB PC holds, so they are not shipped. The 27B still reads prompts token by token. Requested from Intel.
+- **The first load of the 64-token Ternary Bonsai 2 builds takes a while**: on Panther Lake ~6 min for the 4B, ~12 min for the 9B (later loads 20-35 s).
 
 - **The first load with the Qwen family sped up takes longer**: the fused graphs compile as well (a small Panther Lake PC, Qwen3.5-9B NF4: ~13 min until usable, the other variants ~25 min in the background while you chat). Later loads come from the cache in well under a minute.
 - **The INT4 builds and Meteor Lake / Arrow Lake (NPU 3720) are as in 2.1.0**: INT4 weights cannot go into graphs without weights,

TECHNICAL.md

@@ -59,7 +59,7 @@ OpenVINO 2026.5 のコンパイラーは NF4 の重みでこれに失敗し、
   63.5 対 63.6 MB）。INT4 は 2 行以上のグラフで、NPU 4000 以降のコンパイラーが INT4 を表引き（4 ビットの値をそのまま表の番号に
   する）に変える処理と重なって失敗し、直してもコンパイルが 20〜35 倍遅くなるだけでした。このため 2 ビットと INT4 のモデル
   （NPUW を使わないもの）では使わず、融合グラフも既定でオフです（2.1.0 と同じ動作）。INT8 は分離されます
-  （llm-jp-4.1-8B INT8 の部品で 2.5 対 216 MB）。Lunar Lake 以降では NF4 版を勧めます。
+  （llm-jp-4.1-8B INT8 の部品で 2.5 対 216 MB）。Lunar Lake 以降では NF4 版を勧めます。2 ビットは 3.2 で分離できるようにしました（下の「onw 3.2」）。
 
 ### コンパイラーの修正（npu_compiler 8.3 をもとに）
 
@@ -139,6 +139,40 @@ OpenVINO の仕組み）で動かしていました。キットで重みを共
 
 結果は上のグラフと CHANGELOG の v3.0.0・v3.1.0 を見てください。
 
+### onw 3.2：2 ビット（Bonsai）を 64 トークンずつ、重みは 1 組のまま
+
+- **重みを持たないグラフを 2 ビットでも**：Bonsai の 2 ビットの重みは 128 個ずつのグループに分けて、グループごとのスケールと
+  ゼロ点で量子化されています。コンパイラーは隣り合うグループの切り出しを「仮の土台の上の Concat」にまとめてしまい、重みを
+  分離できませんでした。キットのコンパイラーは、同じ OV 定数の連続したグループの切り出しを「先頭＋k グループの切り出し＋残り」
+  に書き直し、[k, N, K] の切り出しはグループごとの Slice → Reshape → Concat で組み立てます（ui2 の Transpose は初期化スケジュール
+  の中では使えないため）。`MergeFullyConnected` のまとめはそのまま使えます。
+- **重みの部分一致の共有（プラグイン）**：1 トークン用と 64 トークン用のグラフは、重みの区切り方が違います（片方の 1 本が、もう
+  片方の数本をつないだもの）。キットのプラグインは、64 KB 以上の出力を登録しておき、32 KB 以上の新しい出力を既存のバッファの
+  中の同じ大きさの倍数の位置と照合して、一致すればその一部を指すようにします（`ONW_WS_SUBRANGE=0` で無効）。2 ビットの重みは
+  NPU 上に 1 組だけです。
+- **重みテーブルは共有できない**：NPU（5010 で確認）は、2 ビットのグループのスケールを、出力チャネルごと・グループごとに 16 バイト
+  の重みテーブルの行から読みます。テーブルはグラフの中で並びが決まるので、グラフごとに別に要り、重み 1 個あたり 1 ビット
+  （2 ビットの重みの半分）になります。新しい形式のテーブル（`useNewWeightTableFormat`）は 5010 の PPE では使えませんでした。
+  27B では 1 組 3.4 GB になり、1 トークン用＋64 トークン用で NPU 13.8 GB を使ったので、27B には入れていません。
+- **コンパイル時間**：戦略探索（MCSA・兄弟の操作の解析）が、同じ重なり方の分散情報や兄弟のリストを何度も作り直していたのを
+  覚えておくようにし、定数の畳み込みのメモを転置・並べ替えにも広げました。定数を左から掛ける行列積がバッチごとに展開される
+  所も、右から掛ける形に直します（`ONW_LEFTMM=0` で無効）。2 ビットの 64 トークンの部品 1 つが 335 → 約 20 秒（バイナリーは同じ）。
+- **プロセスでの並列コンパイル**：OpenVINO は 1 プロセスの中の `compile_model` を同時に呼んでも順番に処理し、コンパイラーも
+  グラフ 1 つにほぼ 1 コアしか使いません。初回のロードでは、コンパイルして `export_model` で保存するだけの子プロセスを、空き
+  メモリ（1 つあたり 2.5 GB）とコア数（半分）に応じて最大 6 つ走らせ、親はできたファイルを読み込みます。子は NPU に重みを
+  載せないので、重みは二重になりません（Bonsai 2 4B：819 → 約 180 秒）。子は MoE のエキスパートの束も共有の重みも NPU から
+  見えるメモリに読み込みません（最初は読み込んでいて、Qwen3.6-35B-A3B の子 6 つが Panther Lake の NPU ドライバーの中で止まり、
+  再起動が要りました。直したあとは 84 部品を 185 秒、初回のロード 231 秒）。
+- **従来のグラフを外す**：融合グラフが覆う長さでは、従来のグラフ（とその重みテーブル）を読み込んだあとで外します。画像の質問や、
+  融合グラフより長い文脈になったときだけ読み込み直します（`ONW_FUSED_DROP_PAGED=0` で外さない）。外したグラフの状態のテンソルは
+  onw 側で持ちます。
+- **融合グラフの 0 の行**：複数トークンのブロックでは、最後の RMSNorm の二乗和が fp16 であふれて行全体が 0 になることがあります
+  （NPU 4000 と 3720 で見ていた現象）。従来のグラフは頭の部分だけ縮めた入力で計算し直していましたが、融合グラフにはそれが
+  なく、Bonsai 2 9B の 64 トークンのブロックで Lunar Lake の 256 行中 20 行が一様分布になっていました（KL 0.86）。同じ処理を
+  入れて KL 0.02 です。
+- **2 ビットのモデルは先読み検証（PLD）を使いません**：下書きの確認が融合 64 トークンのグラフになり、Lunar Lake で生成が
+  23 → 13〜17 tok/s に落ちていました（確認の重さは従来のグラフで測っていたので、オンと判断していました）。
+
 ## NPU で初めて動かした 4 つ
 
 **私たちの知る限り、次の 4 つを NPU で動かして公開している実装は onw が初めてです**（2026 年 9 月、Fable 5.1 調べ）。

TECHNICAL_en.md

@@ -58,7 +58,7 @@ and compiles were slow. Instead of rewriting the graphs, we fixed the compiler's
   With 2+ rows, INT4 clashes with the NPU 4000+ compiler's rewrite of INT4 into a lookup table (the 4-bit value used as the table
   index) and fails; fixed, it only compiled 20-35x slower. So 2-bit and INT4 models (those without NPUW) do not use it, and their
   fused graphs stay off by default (as in 2.1.0). INT8 is separated (an llm-jp-4.1-8B INT8 part: 2.5 vs 216 MB). From Lunar Lake
-  on, use the NF4 builds.
+  on, use the NF4 builds. 3.2 separates 2-bit weights (see "onw 3.2" below).
 
 ### Compiler fixes (based on npu_compiler 8.3)
 
@@ -142,6 +142,40 @@ one copy of the weights). With the kit sharing the weights, 3.1 turns NPUW off f
 
 For the results see the chart above and v3.0.0 / v3.1.0 in the CHANGELOG.
 
+### onw 3.2: 2-bit (Bonsai) 64 tokens at a time, one copy of the weights
+
+- **Graphs without weights for 2-bit too**: Bonsai's 2-bit weights are quantized in groups of 128 with a scale and a zero
+  point per group. The compiler folded slices of neighbouring groups into a Concat on a dummy base, which cannot be
+  separated. The kit's compiler rewrites consecutive group slices of one OV constant into "prefix + k-group slice + rest",
+  and builds a [k, N, K] slice from per-group Slice -> Reshape -> Concat (a ui2 Transpose is not available in the init
+  schedule). `MergeFullyConnected` still merges.
+- **Sharing parts of weight buffers (plugin)**: the 1-token and the 64-token graphs cut their weights differently (one
+  output of one is several outputs of the other joined). The kit's plugin registers outputs of 64 KB and more and matches a
+  new output of 32 KB or more against same-size-aligned offsets inside them; a match points into the existing buffer
+  (`ONW_WS_SUBRANGE=0` turns it off). The NPU holds the 2-bit weights once.
+- **Weight tables cannot be shared**: the NPU (seen on 5010) reads a 2-bit group's scale from a 16-byte weight-table row
+  per output channel and group. The table layout is fixed per graph, so every graph needs its own: 1 bit per weight, half of
+  the 2-bit weights. The newer table format (`useNewWeightTableFormat`) did not work on the 5010's PPE. For the 27B that is
+  3.4 GB per set; the 1-token + 64-token sets took 13.8 GB of NPU memory, so the 27B does not get them.
+- **Compile time**: the strategy search (MCSA, sibling-op analysis) rebuilt the same overlapped distributions and sibling
+  lists again and again; they are remembered now, and the constant-folding memo covers transposes / permutes too. MatMuls
+  with a constant on the left that were unrolled per batch are rewritten with the constant on the right (`ONW_LEFTMM=0`
+  turns it off). One 64-token part of a 2-bit model: 335 -> ~20 s (same binary).
+- **Parallel compile in processes**: OpenVINO serializes concurrent `compile_model` calls in one process, and the compiler
+  uses about one core per graph. At the first load, up to 6 child processes (2.5 GB of free memory and half a core count
+  each) only compile and `export_model`; the parent imports the files. The children never put weights on the NPU, so
+  nothing is held twice (Bonsai 2 4B: 819 -> ~180 s). The children load neither a MoE's expert bank nor the shared
+  weights into NPU-visible memory (at first they did: six children of Qwen3.6-35B-A3B hung inside Panther Lake's NPU
+  driver, a reboot was needed; fixed, 84 parts in 185 s, the first load 231 s).
+- **Dropping the paged graphs**: at lengths the fused graphs cover, the paged graphs (and their weight tables) are released
+  after loading; image questions and contexts longer than the fused graphs load them again (`ONW_FUSED_DROP_PAGED=0` keeps
+  them). onw keeps the state tensors of the dropped graphs itself.
+- **Zero rows in the fused graphs**: in multi-token blocks the final RMSNorm's sum of squares can overflow fp16 and zero a
+  whole row (seen on NPU 4000 and 3720). The paged graphs ran their head again on scaled inputs; the fused ones did not,
+  and Bonsai 2 9B's 64-token blocks came out uniform on 20 of 256 rows on Lunar Lake (KL 0.86). With the same fix: 0.02.
+- **2-bit models do not use prompt-lookup decoding**: a draft check would run the fused 64-token graphs (answers 23 ->
+  13-17 tok/s on Lunar Lake; the check's cost had been timed on the paged graphs, so it was switched on).
+
 ## Four firsts on the NPU
 
 **To our knowledge onw is the first public implementation of these four on the NPU** (survey by Fable 5.1,

onw/__init__.py

@@ -2,4 +2,4 @@
 
 A small engine that runs LLMs entirely on the Intel NPU: static segment graphs + host-managed state.
 """
-__version__ = "3.1.0"
+__version__ = "3.2.0"

onw/app.py

@@ -25,9 +25,9 @@ CATALOG = [  # repo, title, takes images, download GB, decode speed (NPU 3720),
     ("ryugyosoft/Ternary-Bonsai-2-27B-onw", "Ternary Bonsai 2 27B (2-bit)", False, 7.6, "4 tok/s",
      "27B を三値（2 ビット）の重みで（Qwen3.8-27B ベースの Ternary Bonsai 2、Lunar Lake 以降専用）。メモリ約 10 GB で 27B が動きますが、プロンプトの処理は 1 トークンずつで遅い（約 4 tok/s）ので、短い質問向け"),
     ("ryugyosoft/Ternary-Bonsai-2-9B-onw", "Ternary Bonsai 2 9B (2-bit)", True, 3.0, "13 tok/s",
-     "三値（2 ビット）の 9B（Qwen3.5-9B ベースの Ternary Bonsai 2、Lunar Lake 以降専用）。メモリ約 3.5 GB と軽く、画像も読めます。プロンプトの処理は 1 トークンずつ（約 13 tok/s）なので、短めの質問向け"),
+     "三値（2 ビット）の 9B（Qwen3.5-9B ベースの Ternary Bonsai 2、Lunar Lake 以降専用）。メモリ約 3.5 GB と軽く、画像も読めます。onw 3.2 からプロンプトは 64 トークンずつ（Panther Lake で約 190 tok/s）"),
     ("ryugyosoft/Ternary-Bonsai-2-4B-onw", "Ternary Bonsai 2 4B (2-bit)", True, 1.8, "21 tok/s",
-     "三値（2 ビット）の軽い 4B（Qwen3.5-4B ベースの Ternary Bonsai 2、Lunar Lake 以降専用）。メモリ約 2 GB で約 21 tok/s、画像も読めます。プロンプトの処理は 1 トークンずつ（約 21 tok/s）なので、短めの質問向け"),
+     "三値（2 ビット）の軽い 4B（Qwen3.5-4B ベースの Ternary Bonsai 2、Lunar Lake 以降専用）。メモリ約 2 GB で約 21 tok/s、画像も読めます。onw 3.2 からプロンプトは 64 トークンずつ（Panther Lake で約 300 tok/s）"),
     ("ryugyosoft/NeoHorse-1-4B-onw", "NeoHorse-1-4B", False, 2.4, "6.4 tok/s",
      "エージェント・ツール呼び出し向けの軽量モデル（Qwen3.5-4B ベース）"),
     ("ryugyosoft/granite-4.2-3b-onw", "Granite 4.2 3B", False, 2.1, "7.7 tok/s",
@@ -75,6 +75,9 @@ DENSE_PROMPT = {"ryugyosoft/Qwen3.5-9B-onw", "ryugyosoft/Ornith-1.5-9B-onw", "ry
                 "ryugyosoft/llm-jp-4.1-8b-thinking-onw", "ryugyosoft/llm-jp-4.1-8b-thinking-int8-onw"}
 # [2.0.3] republished with 2-token graphs: NPU 4000+ answers on them instead of the 1-token ones (chat.block_sizes), whose
 # compile takes minutes a part there (Granite 4.2 8B first load on Panther Lake 2 h 46 min -> 23 min)
+# [3.2] republished with 64-token graphs for onw's kit (engine.json "kit_sizes"): a copy without them is an old download
+# where the kit serves a 2-bit-capable NPU
+U2_PROMPT = {"ryugyosoft/Ternary-Bonsai-2-4B-onw", "ryugyosoft/Ternary-Bonsai-2-9B-onw"}
 DECODE2 = {"ryugyosoft/granite-4.2-3b-onw", "ryugyosoft/granite-4.2-8b-onw", "ryugyosoft/llm-jp-4.1-8b-thinking-onw"}
 
 
@@ -619,8 +622,8 @@ def outdated(repo, p):
     the INT8 one is noticeably more accurate; the NF4 Qwen3.5 models published before the MTP layer and the 64-token
     blocks (onw 1.4.0: ~1.3x the answer and 2x the prompt speed on Lunar Lake)."""
     gpt, nf4, mix = repo.startswith("ryugyosoft/gpt-oss-20b"), repo in NF4_MTP, repo in MOE_MIXER
-    dense = repo in DENSE_PROMPT
-    if not (gpt or nf4 or mix or dense) or not downloaded_by_onw(p):
+    dense, u2 = repo in DENSE_PROMPT, repo in U2_PROMPT
+    if not (gpt or nf4 or mix or dense or u2) or not downloaded_by_onw(p):
         return False
     try:
         meta = json.load(open(os.path.join(p, "engine.json")))
@@ -628,6 +631,8 @@ def outdated(repo, p):
         return False
     if mix:
         return "mixer" not in meta
+    if u2:
+        return "kit_sizes" not in meta and _kit()                         # (chat.u2_blocks)
     if dense:
         if repo in DECODE2 and npu_arch() >= 4000 and 2 not in (meta.get("prompt_sizes") or []):
             return True

onw/chat.py

@@ -49,9 +49,9 @@ def kit_shares(core, device, engine_dir, meta):
     (2-bit, INT4: runtime.int4_weights) nor MoE models (experts.bin: their own path)."""
     from .runtime import npu_arch, int4_weights
     return (device == "NPU" and npukit.active() and npu_arch(core) >= 4000
-            and (meta.get("needs") != "u2" or os.environ.get("ONW_U2_WEIGHTLESS") == "1")
+            and (meta.get("needs") != "u2" or os.environ.get("ONW_U2_WEIGHTLESS", "1") != "0")   # [3.2] on
             and not os.path.exists(os.path.join(engine_dir, "experts.bin"))
-            and (not int4_weights(engine_dir, meta) or os.environ.get("ONW_U2_WEIGHTLESS") == "1")
+            and (not int4_weights(engine_dir, meta) or os.environ.get("ONW_INT4_WEIGHTLESS") == "1")
             and os.environ.get("ONW_WEIGHTLESS", "1") != "0")
 
 
@@ -63,6 +63,13 @@ def use_mtp(meta, shared):
     return bool(meta.get("mtp")) and (env == "1" if env in ("0", "1") else shared)
 
 
+def u2_blocks():
+    """[3.2] 2-bit (Bonsai) blocks of 2+ tokens: with onw's kit (Windows and Linux: Bonsai 2 4B / 9B checked on Panther
+    Lake and on Lunar Lake under Ubuntu). ONW_U2_BLOCKS=1 / 0 forces them on / off."""
+    env = os.environ.get("ONW_U2_BLOCKS")
+    return env == "1" if env in ("0", "1") else npukit.active()
+
+
 def block_sizes(meta, shared=False, mtp=False, arch=None):
     """Block sizes to load: the 1-token graphs for answers and ONE prompt size, the largest the model has ([2.0]: in
     the 16-token graphs' place - with the weights shared between the sizes (NPUW) every size still adds its NPU
@@ -71,6 +78,10 @@ def block_sizes(meta, shared=False, mtp=False, arch=None):
     1-token graphs. ONW_S64=0: the 16-token graphs as before; 1: every size. arch: the NPU's (runtime.npu_arch) -
     NPU 3720 takes blocks up to 64 tokens (128 are slower there: its INT4 matmuls stop scaling); ONW_PROMPT_MAX: a cap."""
     sizes = all_sizes(meta)
+    if meta.get("needs") == "u2" and not u2_blocks():
+        # [3.2] 2-bit weights in blocks of 2+ rows need onw's kit (its compiler has the fix of npu_compiler #355: the
+        # stock compilers return zeros for them) - the 1-token graphs only without it
+        sizes = [S for S in sizes if S == 1] or sizes
     only = os.environ.get("ONW_SIZES")                  # e.g. "16": decode with the 16-token graphs only
     if not only and 1 in sizes and grouped_wanted(meta):
         # [2.0] a MoE model with mixer graphs: every prompt goes the grouped-MoE path (onw/grouped.py), so only the
@@ -115,6 +126,34 @@ def block_sizes(meta, shared=False, mtp=False, arch=None):
 THINK_BUDGET = {"low": 512, "medium": 2048, "high": None}
 
 
+def vision_blob(d, c):
+    """[3.2] onw's blob store path of the vision encoder for config c (the plain route), or None."""
+    from .runtime import blob_path, NO_KEY
+    if not c.get("CACHE_DIR") or any(k.startswith("NPU_USE_NPUW") for k in c) or os.environ.get("ONW_OWN_BLOBS") == "0":
+        return None
+    xml = os.path.join(d, "vision.xml")
+    nocache = {k: v for k, v in c.items() if k != "CACHE_DIR"}
+    return blob_path(c["CACHE_DIR"], "vision.xml", [open(xml, "rb").read(), os.path.getsize(xml[:-4] + ".bin"),
+                                                    ov.__version__, json.dumps({k: v for k, v in nocache.items()
+                                                                               if k not in NO_KEY}, sort_keys=True)])
+
+
+def export_vision_blob(core, d, device, c):
+    """[3.2] Compile the vision encoder and write its blob (a background child at a first load: the first image
+    then only imports it - Panther Lake, Bonsai 2 4B: 144 s at the first image before)."""
+    import io
+    bp = vision_blob(d, c)
+    if bp is None or os.path.exists(bp) or os.path.exists(bp + ".bad"):
+        return
+    first = core.compile_model(os.path.join(d, "vision.xml"), device, c)
+    buf = io.BytesIO()
+    first.export_model(buf)
+    os.makedirs(os.path.dirname(bp), exist_ok=True)
+    with open(f"{bp}.{os.getpid()}.tmp", "wb") as f:
+        f.write(buf.getbuffer())
+    os.replace(f"{bp}.{os.getpid()}.tmp", bp)
+
+
 class ChatEngine:
     _think_marks = None                        # the thought markers when they are not <think> ... </think> (Gemma 4)
     swap_gb = None                             # memory-saving mode: GB of the 1-token graphs swapped in and out
@@ -248,8 +287,26 @@ class ChatEngine:
             # [2.1.0] the answer graphs as stream_chat sets them for an answer of the usual length (n + up to 1024 new
             # tokens: the 2K variant), so the first question does not switch variants (Panther Lake, Granite 4.2 8B
             # NF4: 6 s added to the first prompt)
-            for S in sorted(m._fused_sizes, reverse=True):
+            # [3.2] memory first: the paged graphs of those sizes go before the fused ones load, and the 1-token ones
+            # load first - their merged 2-bit group weights are the buffers the 64-token graphs then share (a graph
+            # must not share a buffer of a graph that is dropped later: Bonsai 2 9B crashed in the NPU driver)
+            for S in sorted(m._fused_sizes):
+                if hasattr(m, "_drop_paged_for_fused"):
+                    m._drop_paged_for_fused(S, before=True)
+            for S in sorted(m._fused_sizes):
                 m.fused_prepare(min(1024 if S > min(m._fused_sizes) else 2048, m.T), S)
+        self._venc_job = None
+        if self.vision and device == "NPU" and os.environ.get("ONW_VISION_PRECOMPILE", "1") != "0":
+            vb = vision_blob(engine_dir, self._venc_args[3])
+            if vb is not None and not os.path.exists(vb) and not os.path.exists(vb + ".bad"):
+                import subprocess, sys
+                flags = 0x00004000 if os.name == "nt" else 0      # (BELOW_NORMAL_PRIORITY_CLASS)
+                self._venc_job = subprocess.Popen(
+                    [sys.executable, "-m", "onw.visionjob", json.dumps([engine_dir, device, self._venc_args[3]],
+                                                                       default=str)],
+                    env={**os.environ, "PYTHONUTF8": "1"}, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL,
+                    creationflags=flags)
+                print("[vision] encoder compiling in the background", flush=True)
         pld_state = "off (memory-saving mode)" if self.swap_gb is not None else "on" if self.pld else "off"
         print(f"[load] {time.time()-t0:.1f}s on {device}; verify block = {self.block_cost or 0:.1f} single steps, "
               f"prompt lookup decoding {pld_state}", flush=True)
@@ -258,6 +315,10 @@ class ChatEngine:
     def venc(self):
         if self._venc is None:
             t0 = time.time()
+            job = getattr(self, "_venc_job", None)
+            if job is not None:                        # [3.2] (being compiled in the background: wait for its blob)
+                job.wait()
+                self._venc_job = None
             self._venc = self._compile_vision(*self._venc_args)
             print(f"[vision] encoder loaded in {time.time() - t0:.1f}s", flush=True)
         return self._venc
@@ -854,6 +915,12 @@ class ChatEngine:
             self.model.fused_prepare(self.model.cur + ahead)
         # exact only for plain greedy; not while block sizes are swapped (the verification blocks go away mid-answer)
         use_pld = self.pld and plain_greedy and self.model.S > 1 and not swap and budget is None
+        fz = sorted(getattr(self.model, "_fused", None) or [])
+        if use_pld and fz[:1] == [1] and not any(1 < S <= 16 for S in fz) and os.environ.get("ONW_PLD_BIG") != "1":
+            # [3.2] answers on the fused 1-token graphs with no small block beside them (2-bit models: [1, 64]): a draft
+            # check would take the fused 64-token graphs, many fused steps (block_cost was timed on the paged graphs,
+            # whose single steps are slower - Bonsai 2 4B on Lunar Lake: answers 23 -> 13-17 tok/s with PLD)
+            use_pld = False
         # MTP drafts (Qwen3.5 dense): every sampling setting (exact), the thinking budget included
         mtp = getattr(self.model, "mtp", None) is not None and not swap and             any(S >= 4 for S in [*self.model.segs, *(getattr(self.model, "_fused", None) or {})])
         use_pld = use_pld and not mtp

onw/fused.py

@@ -209,7 +209,7 @@ def wanted(m):
     # fused graphs ran out of memory)
     from .runtime import _WEIGHTLESS
     on = (npukit.active() and npu_arch(m._ctx["core"]) >= 4000
-          and (m.meta.get("needs") != "u2" or os.environ.get("ONW_U2_WEIGHTLESS") == "1")
+          and (m.meta.get("needs") != "u2" or os.environ.get("ONW_U2_WEIGHTLESS", "1") != "0")   # [3.2] on
           and not _WEIGHTLESS.get("skip_model") and _WEIGHTLESS["ok"] is not False and not _WEIGHTLESS.get("full"))
     if os.environ.get("ONW_FUSED", "1" if on else "0") != "1":
         return None
@@ -264,8 +264,9 @@ class FusedDecode:
         self.mask_kv_t = mk(ov.Type.f16, [1, self.Ct])
         self.mask_self_t = mk(ov.Type.f16, [S, S])
         t0, built = time.time(), 0
+        only = os.environ.get("ONW_FUSED_SEG") if compile_only else None   # [3.2] (a child: this segment only)
         for name, sm in sorted(m.meta["segments"].items()):
-            if sm["S"] != S:
+            if sm["S"] != S or (only and name != only):
                 continue
             # (NPUW: the model's shared mapping of the weight files and NPUW's reading, so its weights bank sees the
             # same constants as for the other block size - one copy of the weights)
@@ -441,7 +442,12 @@ class FusedDecode:
                    "prev": {k: v.copy() for k, v in m.state.items() if k not in m._pp} if n > 1 else None,
                    "pp": {k: e[1] for k, e in m._pp.items()} if n > 1 else None}
         logits, pend = None, None
-        for st in self.plan:
+        import os, time
+        prof = None
+        if os.environ.get("ONW_PROFILE") == "1":               # [3.2] per-segment NPU times, every 64 steps
+            prof = self.__dict__.setdefault("_prof", {"n": 0, "seg": [0.0] * len(self.plan), "step": 0.0})
+            t_step = time.perf_counter()
+        for si, st in enumerate(self.plan):
             req = st["req"]
             if st["head"] and m.mtp is not None and st["h_in"] is not None:
                 m.last_h = np.asarray(st["h_in"].data[0, :n], np.float32)    # (what the MTP layer drafts from)
@@ -451,11 +457,15 @@ class FusedDecode:
                 bufs, c = m._pp[k]
                 req.set_tensor(k, bufs[c])
                 req.set_tensor("ssm_new." + k[4:], bufs[1 - c])
+            if prof is not None:
+                t0 = time.perf_counter()
             req.start_async()
             if pend is not None:                      # the previous segment's new rows while this one runs
                 pend()
                 pend = None
             req.wait()
+            if prof is not None:
+                prof["seg"][si] += time.perf_counter() - t0
 
             if n > 1:
                 for o in st["rows"]:                  # (the request's own buffer: read by truncate() before the
@@ -479,9 +489,31 @@ class FusedDecode:
             pend = write
             if st["head"]:
                 logits = np.array(req.get_tensor("logits").data[0, :n])
+                dead = ~logits.any(axis=1)
+                if dead.any() and st["h_in"] is not None:
+                    # [3.2] rows of all zeros: the final RMSNorm's sum of squares overflowed fp16 (large activations in
+                    # multi-token blocks - Bonsai 2 9B 64-token blocks on Lunar Lake, 20 of 256 rows), as
+                    # SegmentedModel._head_again: the norm does not care about scale, so the head runs again on the
+                    # hidden states divided by their max |h| (written in place: the next step overwrites them anyway)
+                    h = st["h_in"].data
+                    x = np.asarray(h[0, :n], np.float32)
+                    h[0, :n] = (x / np.maximum(np.abs(x).max(-1, keepdims=True), 1e-2)).astype(h.dtype)
+                    req.infer()
+                    logits[dead] = np.array(req.get_tensor("logits").data[0, :n])[dead]
+                    if not getattr(m, "_dead_note", False):
+                        print("[head] logit rows of all zeros (fp16 overflow in the final norm) computed again, scaled",
+                              flush=True)
+                        m._dead_note = True
         if pend is not None:
             pend()
         m.cur += n
+        if prof is not None:
+            prof["step"] += time.perf_counter() - t_step
+            prof["n"] += 1
+            if prof["n"] % 64 == 0:
+                k = prof["n"]
+                print(f"[profile] fused {S}-token, {k} steps: {prof['step'] / k * 1e3:.2f} ms a step, segments "
+                      + " ".join(f"{v / k * 1e3:.2f}" for v in prof["seg"]), flush=True)
         if logits is not None:
             if not np.isfinite(logits).all():
                 raise FloatingPointError(f"the fused {S}-token graphs returned NaN/inf logits")

onw/i18n.json

@@ -355,8 +355,8 @@
 "27B を三値（2 ビット）の重みで（Qwen3.8-27B ベースの Ternary Bonsai 2、Lunar Lake 以降専用）。メモリ約 10 GB で 27B が動きますが、プロンプトの処理は 1 トークンずつで遅い（約 4 tok/s）ので、短い質問向け": "27B with ternary (2-bit) weights (Ternary Bonsai 2, from Qwen3.8-27B; Lunar Lake and newer only). A 27B in about 10 GB of memory - but prompts are processed token by token, slowly (~4 tok/s): for short questions",
 "NPU を性能優先で動かす（ターボ）": "Run the NPU for speed (turbo)",
 "NPU のクロックを上げて、回答とプロンプトの読み込みを速くします（Lunar Lake で 2〜4 割、Panther Lake で 2 割ほど）。そのぶん消費電力が増えます（Lunar Lake で約 6 W）。電池で長く使いたいときはオフに。": "Raises the NPU's clocks: answers and prompt reading get faster (20-40% on Lunar Lake, ~20% on Panther Lake), at the cost of more power (~6 W on Lunar Lake). Turn it off for long battery use.",
-"三値（2 ビット）の 9B（Qwen3.5-9B ベースの Ternary Bonsai 2、Lunar Lake 以降専用）。メモリ約 3.5 GB と軽く、画像も読めます。プロンプトの処理は 1 トークンずつ（約 13 tok/s）なので、短めの質問向け": "Ternary (2-bit) 9B (Ternary Bonsai 2, based on Qwen3.5-9B; Lunar Lake and later only). Light at about 3.5 GB of memory and reads images. Prompts are processed one token at a time (about 13 tok/s): best for shorter questions",
-"三値（2 ビット）の軽い 4B（Qwen3.5-4B ベースの Ternary Bonsai 2、Lunar Lake 以降専用）。メモリ約 2 GB で約 21 tok/s、画像も読めます。プロンプトの処理は 1 トークンずつ（約 21 tok/s）なので、短めの質問向け": "A light ternary (2-bit) 4B (Ternary Bonsai 2, based on Qwen3.5-4B; Lunar Lake and later only). About 2 GB of memory, about 21 tok/s, reads images. Prompts are processed one token at a time (about 21 tok/s): best for shorter questions",
+"三値（2 ビット）の 9B（Qwen3.5-9B ベースの Ternary Bonsai 2、Lunar Lake 以降専用）。メモリ約 3.5 GB と軽く、画像も読めます。onw 3.2 からプロンプトは 64 トークンずつ（Panther Lake で約 190 tok/s）": "Ternary (2-bit) 9B (Ternary Bonsai 2, based on Qwen3.5-9B; Lunar Lake and later only). Light at about 3.5 GB of memory and reads images. From onw 3.2 prompts go 64 tokens at a time (~190 tok/s on Panther Lake)",
+"三値（2 ビット）の軽い 4B（Qwen3.5-4B ベースの Ternary Bonsai 2、Lunar Lake 以降専用）。メモリ約 2 GB で約 21 tok/s、画像も読めます。onw 3.2 からプロンプトは 64 トークンずつ（Panther Lake で約 300 tok/s）": "A light ternary (2-bit) 4B (Ternary Bonsai 2, based on Qwen3.5-4B; Lunar Lake and later only). About 2 GB of memory, about 21 tok/s, reads images. From onw 3.2 prompts go 64 tokens at a time (~300 tok/s on Panther Lake)",
 "Qwen 系を高速化する": "Speed up the Qwen family",
 "Qwen3.5・Ornith などで、区間ごとの融合グラフと MTP 先読みを使います。プロンプトの処理が 2〜4 割、回答が 1 割ほど速くなり、メモリを約 1 GB 多く使います。Lunar Lake 以降で、メモリ節約モードがオフのときに働きます。": "Qwen3.5, Ornith and similar models run fused segment graphs with MTP draft tokens: prompts 20-40% and answers about 10% faster, about 1 GB more memory. Works from Lunar Lake on, while the memory-saving mode is off.",
 "モデルを検索": "Search models",
@@ -726,8 +726,8 @@
 "27B を三値（2 ビット）の重みで（Qwen3.8-27B ベースの Ternary Bonsai 2、Lunar Lake 以降専用）。メモリ約 10 GB で 27B が動きますが、プロンプトの処理は 1 トークンずつで遅い（約 4 tok/s）ので、短い質問向け": "三值（2 位）权重的 27B（基于 Qwen3.8-27B 的 Ternary Bonsai 2，仅限 Lunar Lake 及更新的 NPU）。约 10 GB 内存即可运行 27B，但提示词逐个 token 处理、较慢（约 4 tok/s），适合简短提问",
 "NPU を性能優先で動かす（ターボ）": "NPU 性能优先运行（加速模式）",
 "NPU のクロックを上げて、回答とプロンプトの読み込みを速くします（Lunar Lake で 2〜4 割、Panther Lake で 2 割ほど）。そのぶん消費電力が増えます（Lunar Lake で約 6 W）。電池で長く使いたいときはオフに。": "提高 NPU 频率，回答和提示词读取更快（Lunar Lake 约快 20~40%，Panther Lake 约 20%），但功耗增加（Lunar Lake 约 6 W）。需要长时间使用电池时请关闭。",
-"三値（2 ビット）の 9B（Qwen3.5-9B ベースの Ternary Bonsai 2、Lunar Lake 以降専用）。メモリ約 3.5 GB と軽く、画像も読めます。プロンプトの処理は 1 トークンずつ（約 13 tok/s）なので、短めの質問向け": "三值（2 位）9B（基于 Qwen3.5-9B 的 Ternary Bonsai 2，仅限 Lunar Lake 及以后）。内存约 3.5 GB，很轻，也能读图。提示词逐个词元处理（约 13 tok/s），适合较短的问题",
-"三値（2 ビット）の軽い 4B（Qwen3.5-4B ベースの Ternary Bonsai 2、Lunar Lake 以降専用）。メモリ約 2 GB で約 21 tok/s、画像も読めます。プロンプトの処理は 1 トークンずつ（約 21 tok/s）なので、短めの質問向け": "轻量的三值（2 位）4B（基于 Qwen3.5-4B 的 Ternary Bonsai 2，仅限 Lunar Lake 及以后）。内存约 2 GB，约 21 tok/s，也能读图。提示词逐个词元处理（约 21 tok/s），适合较短的问题",
+"三値（2 ビット）の 9B（Qwen3.5-9B ベースの Ternary Bonsai 2、Lunar Lake 以降専用）。メモリ約 3.5 GB と軽く、画像も読めます。onw 3.2 からプロンプトは 64 トークンずつ（Panther Lake で約 190 tok/s）": "三值（2 位）9B（基于 Qwen3.5-9B 的 Ternary Bonsai 2，仅限 Lunar Lake 及以后）。内存约 3.5 GB，很轻，也能读图。自 onw 3.2 起提示词每次处理 64 个词元（Panther Lake 上约 190 tok/s）",
+"三値（2 ビット）の軽い 4B（Qwen3.5-4B ベースの Ternary Bonsai 2、Lunar Lake 以降専用）。メモリ約 2 GB で約 21 tok/s、画像も読めます。onw 3.2 からプロンプトは 64 トークンずつ（Panther Lake で約 300 tok/s）": "轻量的三值（2 位）4B（基于 Qwen3.5-4B 的 Ternary Bonsai 2，仅限 Lunar Lake 及以后）。内存约 2 GB，约 21 tok/s，也能读图。自 onw 3.2 起提示词每次处理 64 个词元（Panther Lake 上约 300 tok/s）",
 "Qwen 系を高速化する": "加速 Qwen 系列",
 "Qwen3.5・Ornith などで、区間ごとの融合グラフと MTP 先読みを使います。プロンプトの処理が 2〜4 割、回答が 1 割ほど速くなり、メモリを約 1 GB 多く使います。Lunar Lake 以降で、メモリ節約モードがオフのときに働きます。": "Qwen3.5、Ornith 等模型使用按区段融合的图和 MTP 预测：提示处理快 2〜4 成，回答快约 1 成，内存多用约 1 GB。适用于 Lunar Lake 及之后的机型，且内存节省模式关闭时。",
 "モデルを検索": "搜索模型",

onw/ir.py

@@ -9,6 +9,8 @@ Conventions shared by every model builder:
     the output side. Those are the weights-as-inputs forms the NPU runs at full speed (group-wise scales on
     runtime weights turn into slow element-wise work), and they let the host bind just the selected experts.
 """
+import os
+
 import numpy as np
 import openvino as ov
 import openvino.opset13 as ops
@@ -197,6 +199,12 @@ def ternary_weight(w):
     G = i // w.group
     t = ov.Tensor(ov.Type.u2, [o, G, w.group])
     np.copyto(t.data, np.asarray(w.packed).view(np.uint8).reshape(t.data.shape))
+    if os.environ.get("ONW_TERNARY_F32") == "1":
+        # (test) the chain in the f32 form NPUW's asymmetric-weight pattern (npuw/partitioning/patterns/dcoff.cpp,
+        # AsymmZP::CWAI) recognizes: only the 2-bit codes go to its shared weights bank, zero point and scale stay
+        d = ops.multiply(ops.subtract(ops.convert(ops.constant(t), F32), ops.constant(np.array(1.0, np.float32))),
+                         ops.constant(np.asarray(w.scales, np.float32).reshape(o, G, 1)))
+        return ops.reshape(d, i64([o, i]), False)
     d = ops.multiply(ops.subtract(ops.convert(ops.constant(t), ov.Type.f16), ops.constant(np.array(1.0, np.float16))),
                      ops.constant(np.asarray(w.scales, np.float16).reshape(o, G, 1)))
     return ops.convert(ops.reshape(d, i64([o, i]), False), F32)

onw/npukit/NOTICE.txt

@@ -1,4 +1,4 @@
-onw NPU kit (onw 3.0)
+onw NPU kit (onw 3.2)
 
 This folder holds binaries built by the onw project from open-source code of Intel Corporation, licensed under the
 Apache License 2.0 (LICENSE). They are modified versions; the modifications are in patches/.

onw/npukit/kit.json

@@ -1,5 +1,5 @@
 {
-  "version": "3.0.0 (npu_compiler 8.3 + onw fixes, NPU plugin 2026.5.0 6db412d + weight sharing)",
+  "version": "3.2.0 (npu_compiler 8.3 + onw fixes, NPU plugin 2026.5.0 6db412d + weight sharing)",
   "openvino": "2026.5.0-23160-6db412d0a27",
   "files": [
     "openvino_intel_npu_plugin.dll",

onw/npukit/linux/npukit-linux.tar.xz

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:46fa6b160305b720e56d0d4a44d76a344b06cf7ca886b2a26a447bfb338ba857
-size 32282368
+oid sha256:2eeec38edc79a59b39e52f4f65b3bdb1b6cba5f944f189aa2a2343057ef5efa8
+size 33102668

onw/npukit/openvino_intel_npu_compiler.dll

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:ce1113abf245d6ada1fff920eb1373dc136e1190dc0558bc97d0b6c9ae27ed22
-size 83966464
+oid sha256:609f80dad80b510eab914352b02dedf61c8744e0b5fc6da59c6816ba7ae2f105
+size 84021760

onw/npukit/openvino_intel_npu_plugin.dll

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:91cf9ebe2bd9ef7a3e3e451b63e7497abe465bfa2396edb52b35e8eb1d4db2ce
-size 7992832
+oid sha256:fe7fd8fa7f0a5a69b7ad627a9d5568d5a0206cdedc0fd01d92fd5e590732a39e
+size 8001024

onw/npukit/patches/npu_compiler-20e7912.patch

@@ -51,6 +51,44 @@ index 3943fd7..c15b6aa 100644
  
  VPU::DistributionInfoAttr updateSliceLikeOpsAlignment(mlir::MLIRContext* ctx, vpux::ShapeRef inShape,
                                                        vpux::ShapeRef sliceShape,
+diff --git a/src/vpux_compiler/include/vpux/compiler/dialect/VPU/utils/mpe_engine_utils.hpp b/src/vpux_compiler/include/vpux/compiler/dialect/VPU/utils/mpe_engine_utils.hpp
+index b058405..e8d6cb5 100644
+--- a/src/vpux_compiler/include/vpux/compiler/dialect/VPU/utils/mpe_engine_utils.hpp
++++ b/src/vpux_compiler/include/vpux/compiler/dialect/VPU/utils/mpe_engine_utils.hpp
+@@ -5,7 +5,10 @@
+ 
+ #pragma once
+ 
++#include <cstdlib>
++
+ #include "vpux/compiler/core/types/quantile_float/types.hpp"
++#include "vpux/compiler/dialect/IE/IR/ops/convolution.hpp"
+ #include "vpux/compiler/dialect/IE/IR/ops/eltwise.hpp"
+ #include "vpux/compiler/dialect/IE/IR/ops/pooling.hpp"
+ #include "vpux/compiler/dialect/IE/utils/matmul.hpp"
+@@ -23,8 +26,20 @@ namespace vpux::VPU {
+  */
+ class MPEEngineConfig {
+ public:
+-    static bool useNewWeightTableFormat(mlir::Operation*, bool) {
+-        return false;
++    // onw: ONW_NEW_WT=1 - the split weight table (data pointers / scale / bias as separate operands) on NPU 50XX: the
++    // per-channel scales then come from the model's constants (weightless: computed by the init schedule) instead of
++    // being baked into the table rows, so graphs that differ only in weights compile to the same blob (test).
++    static bool useNewWeightTableFormat(mlir::Operation* op, bool isCompressConv) {
++        static const bool on = std::getenv("ONW_NEW_WT") != nullptr;
++        if (!on || op == nullptr || isCompressConv) {
++            return false;
++        }
++        // only plain convolutions: depthwise / pooling / eltwise NCE ops do not take the split table in this build
++        // (NCE.DepthConvolution got null scale / bias operands)
++        if (!mlir::isa<IE::ConvolutionOp, VPU::NCEConvolutionOp>(op)) {
++            return false;
++        }
++        return config::getArch(op) == config::ArchKind::NPU50XX;
+     }
+ 
+     static bool isNewWeightTableFormatSupportedWithDwOps([[maybe_unused]] config::ArchKind arch) {
 diff --git a/src/vpux_compiler/include/vpux/compiler/dialect/VPU/utils/multi_cluster_strategy_utils.hpp b/src/vpux_compiler/include/vpux/compiler/dialect/VPU/utils/multi_cluster_strategy_utils.hpp
 index 8895bf9..1e0b4d3 100644
 --- a/src/vpux_compiler/include/vpux/compiler/dialect/VPU/utils/multi_cluster_strategy_utils.hpp
@@ -74,6 +112,67 @@ index 8895bf9..1e0b4d3 100644
  };
  vpux::VPU::StrategyCost correctSwOpCost(VPU::SWOpInterface swOp, ArrayRef<vpux::NDTypeInterface> tiledInputTypes,
                                          vpux::VPU::StrategyCost cost);
+diff --git a/src/vpux_compiler/include/vpux/compiler/dialect/VPU/utils/sibling_ops_analysis.hpp b/src/vpux_compiler/include/vpux/compiler/dialect/VPU/utils/sibling_ops_analysis.hpp
+index 2eaaece..513fb6e 100644
+--- a/src/vpux_compiler/include/vpux/compiler/dialect/VPU/utils/sibling_ops_analysis.hpp
++++ b/src/vpux_compiler/include/vpux/compiler/dialect/VPU/utils/sibling_ops_analysis.hpp
+@@ -10,6 +10,7 @@
+ #include <mlir/Dialect/Func/IR/FuncOps.h>
+ #include <mlir/IR/BuiltinTypes.h>
+ 
++#include <deque>
+ #include <set>
+ #include <vector>
+ 
+@@ -33,13 +34,48 @@ public:
+     // ClusteredOp(producer) -> View-like op -> ClusteredOp(consumer)
+     virtual std::set<ClusteredOpInterface> getConsumers(ClusteredOpInterface);
+ 
++    // onw: the same groups as getSiblings / getConsumers (same order), as views of the stored group - those return a
++    // copy of the std::set, which was 21% of the strategy search on 2-bit (Bonsai) graphs. The views stay valid while
++    // the analysis lives. Eager analyses (no lazy fill) answer through the copying calls.
++    ArrayRef<ClusteredOpInterface> onwSiblingsView(ClusteredOpInterface);
++    ArrayRef<ClusteredOpInterface> onwConsumersView(ClusteredOpInterface);
++
+ protected:
++    bool _onwLazy = true;
++    std::deque<SmallVector<ClusteredOpInterface>> _onwGroupVecs{};
++    llvm::DenseMap<mlir::Operation*, size_t> _onwConsumerGroup{};
++    SmallVector<ClusteredOpInterface> _onwScratch{};
+     std::set<ClusteredOpInterface> lookupOpSiblings(mlir::Operation* op);
+     mlir::Operation* getConsumerOp(ClusteredOpInterface);
++    // onw: adds a group and indexes it by its members and by the op it was computed for
++    void addGroup(mlir::Operation* queried, std::set<ClusteredOpInterface> group);
+     std::vector<std::set<ClusteredOpInterface>> _siblingGroups{};
++    // onw: op -> index of the first group that holds it (or was computed for it); lookups were a linear scan of every
++    // group, and an op found in none recomputed its siblings (and appended another group) on every call - the strategy
++    // search of 2-bit (Bonsai) graphs, whose weights split into 20 per-group convolutions on one input, spent 56% there
++    llvm::DenseMap<mlir::Operation*, size_t> _onwGroupOf{};
+     llvm::DenseMap<ClusteredOpInterface, std::set<ClusteredOpInterface>> _consumers{};
+ };
+ 
++// onw: the analysis of the pass running on this thread (MultiClusterStrategyAssignment publishes its own) - the
++// overload of getOutputOverlappedParams without an analysis (the split-over-height checks) built a fresh one on every
++// call, so the sibling search ran again for every op and strategy (2-bit Bonsai parts: 89% of a 6-minute compile).
++// ONW_NO_SIBLING_CACHE=1: as before.
++extern thread_local SiblingOpsAnalysis* onwCurrentSiblingAnalysis;
++// onw: drops this thread's overlapped-params cache (overlap_distribution_utils.cpp)
++void onwClearOverlapCache();
++struct OnwSiblingAnalysisScope {
++    SiblingOpsAnalysis* prev;
++    explicit OnwSiblingAnalysisScope(SiblingOpsAnalysis* a): prev(onwCurrentSiblingAnalysis) {
++        onwCurrentSiblingAnalysis = a;
++        onwClearOverlapCache();
++    }
++    ~OnwSiblingAnalysisScope() {
++        onwCurrentSiblingAnalysis = prev;
++        onwClearOverlapCache();
++    }
++};
++
+ // Eager version of SiblingOpsAnalysis which doesn't modify internal state
+ // on calls to getConsumers/getSiblings. Suitable for usage with multithreading.
+ class EagerSiblingOpsAnalysis : public SiblingOpsAnalysis {
 diff --git a/src/vpux_compiler/include/vpux/compiler/dialect/VPU/utils/vertical_fusion/v2/merge_vf_region_rewriter.hpp b/src/vpux_compiler/include/vpux/compiler/dialect/VPU/utils/vertical_fusion/v2/merge_vf_region_rewriter.hpp
 index fe48940..c819038 100644
 --- a/src/vpux_compiler/include/vpux/compiler/dialect/VPU/utils/vertical_fusion/v2/merge_vf_region_rewriter.hpp
@@ -87,8 +186,32 @@ index fe48940..c819038 100644
      StrategyCost getOriginalVFCost(VPU::VerticalFusionOp vfOp) const;
  
      VFCacheAnalysis& _cache;
+diff --git a/src/vpux_compiler/include/vpux/compiler/dialect/VPU/utils/weights_separation.hpp b/src/vpux_compiler/include/vpux/compiler/dialect/VPU/utils/weights_separation.hpp
+index 0beeab6..3d54599 100644
+--- a/src/vpux_compiler/include/vpux/compiler/dialect/VPU/utils/weights_separation.hpp
++++ b/src/vpux_compiler/include/vpux/compiler/dialect/VPU/utils/weights_separation.hpp
+@@ -10,6 +10,9 @@
+ #include "vpux/compiler/utils/logging.hpp"
+ 
+ #include "mlir/Dialect/Func/IR/FuncOps.h"
++
++#include <map>
++#include <vector>
+ #include "mlir/Pass/AnalysisManager.h"
+ 
+ namespace vpux::VPU {
+@@ -234,6 +237,9 @@ class ConstOpConverter {
+     // Note: operation cache is pimpl-ed to not expose it as a public class.
+     class OperationCache;
+     std::unique_ptr<OperationCache> _operationCache;
++    // onw: results of folded sub-byte view chains (foldSubByteView) by input value and chain - identical chains (the
++    // tiles of one weight) must give one value, i.e. one Init result, as the per-transformation cache does
++    std::map<std::vector<const void*>, mlir::Value> _onwViewCache;
+ 
+     // IR building function with internal caching semantics.
+     std::tuple<ArrayRef<Const::TransformAttrInterface>, mlir::Value> createMatchingOperation(
 diff --git a/src/vpux_compiler/src/compiler.cpp b/src/vpux_compiler/src/compiler.cpp
-index 263a325..acf446c 100644
+index 263a325..419221a 100644
 --- a/src/vpux_compiler/src/compiler.cpp
 +++ b/src/vpux_compiler/src/compiler.cpp
 @@ -1087,7 +1087,11 @@ void compileIEtoVPU(mlir::OwningOpRef<mlir::ModuleOp>& moduleOp,
@@ -117,6 +240,22 @@ index 263a325..acf446c 100644
          vpuipPM.addPass(VPU::createDestructWsAnalysisPass(log));
      }
  
+@@ -1154,6 +1160,7 @@ std::vector<CompilationResult> compileImplWsOneShot(
+     mlir::OwningOpRef<mlir::ModuleOp> moduleMain =
+             importNetwork(ctx, model, originalParameters, originalResults, config, devConf, rootTiming, log,
+                           /*enableWeightsSeparationPath=*/true);
++    moduleMain.get()->setAttr("VPU.OnwWeightless", mlir::UnitAttr::get(ctx));   // onw: see MergeParallelFullyConnected
+ 
+     OV_ITT_TASK_NEXT(COMPILER_IMPLEMENTATION, "compile");
+     auto factoryMethod = createDialectPipelineStrategyFn(config);
+@@ -1276,6 +1283,7 @@ std::tuple<mlir::OwningOpRef<mlir::ModuleOp>, bool> compileImplWsIterative(
+     OV_ITT_TASK_NEXT(COMPILER_IMPLEMENTATION, "importNetwork");
+     auto moduleOp = importNetwork(ctx, model, originalParameters, originalResults, config, devConf, rootTiming, log,
+                                   /*enableWeightsSeparationPath=*/true);
++    moduleOp.get()->setAttr("VPU.OnwWeightless", mlir::UnitAttr::get(ctx));   // onw: see MergeParallelFullyConnected
+ 
+     auto factoryMethod = createDialectPipelineStrategyFn(config);
+     auto mainPipelineStrategy = factoryMethod(config::CompilationMode::WSMain);
 diff --git a/src/vpux_compiler/src/conversion/passes/IE2VPU/convert_layers_to_VPU.cpp b/src/vpux_compiler/src/conversion/passes/IE2VPU/convert_layers_to_VPU.cpp
 index f80f56e..3cfef97 100644
 --- a/src/vpux_compiler/src/conversion/passes/IE2VPU/convert_layers_to_VPU.cpp
@@ -203,6 +342,76 @@ index f80f56e..3cfef97 100644
  
      mlir::ConversionTarget target(ctx);
      target.addIllegalDialect<IE::IEDialect>();
+diff --git a/src/vpux_compiler/src/dialect/IE/transforms/passes/forbid_four_bit_outputs.cpp b/src/vpux_compiler/src/dialect/IE/transforms/passes/forbid_four_bit_outputs.cpp
+index bc08bb0..d67d826 100644
+--- a/src/vpux_compiler/src/dialect/IE/transforms/passes/forbid_four_bit_outputs.cpp
++++ b/src/vpux_compiler/src/dialect/IE/transforms/passes/forbid_four_bit_outputs.cpp
+@@ -23,6 +23,8 @@ namespace vpux::IE {
+ #include "vpux/compiler/dialect/IE/passes.hpp.inc"
+ }  // namespace vpux::IE
+ 
++#include <cstdlib>
++
+ using namespace vpux;
+ 
+ namespace {
+@@ -41,6 +43,11 @@ private:
+         if (archKind != config::ArchKind::NPU37XX) {
+             return;
+         }
++        // onw: graphs without weights - the init schedule hands its 4-bit weights straight to the main schedule
++        // (slices of the model's packed codes, see weights_separation foldSubByteView); 4-bit inputs already work
++        if (moduleOp->hasAttr("VPU.OnwWeightless") && std::getenv("ONW_FORBID_4BIT_OUT") == nullptr) {
++            return;
++        }
+         auto netInfo = net::getNetworkInfo(moduleOp);
+         auto outputsInfo = netInfo.getOutputsDataInfo();
+         for (auto outputInfo : outputsInfo) {
+diff --git a/src/vpux_compiler/src/dialect/IE/transforms/passes/merge_fully_connected.cpp b/src/vpux_compiler/src/dialect/IE/transforms/passes/merge_fully_connected.cpp
+index baa9fec..029e104 100644
+--- a/src/vpux_compiler/src/dialect/IE/transforms/passes/merge_fully_connected.cpp
++++ b/src/vpux_compiler/src/dialect/IE/transforms/passes/merge_fully_connected.cpp
+@@ -3,6 +3,7 @@
+ // SPDX-License-Identifier: Apache-2.0
+ //
+ 
++#include <cstdlib>
+ #include "vpux/compiler/dialect/IE/IR/dialect.hpp"
+ #include "vpux/compiler/dialect/IE/IR/ops/convolution.hpp"
+ #include "vpux/compiler/dialect/IE/IR/ops/data_movement.hpp"
+@@ -1516,6 +1517,13 @@ mlir::LogicalResult MergeFullyConnectedPass::initialize(mlir::MLIRContext* ctx)
+ }
+ 
+ void MergeFullyConnectedPass::safeRunOnFunc() {
++    // onw: ONW_WS_NO_MERGE_FC=1 - weightless compiles keep the unrolled group matmuls as they are. (Merged, the
++    // weight of k groups is a Concat of k group constants: VPU::WeightsSeparationInfo rewrites it into one chain on the
++    // OV constant again, so it goes weightless - before, it stayed in the blob as a second copy of the codes.)
++    if (auto mod = getOperation()->getParentOfType<mlir::ModuleOp>();
++        mod != nullptr && mod->hasAttr("VPU.OnwWeightless") && std::getenv("ONW_WS_NO_MERGE_FC") != nullptr) {
++        return;
++    }
+     auto& ctx = getContext();
+     auto func = getOperation();
+ 
+diff --git a/src/vpux_compiler/src/dialect/IE/transforms/passes/merge_parallel_fully_connected.cpp b/src/vpux_compiler/src/dialect/IE/transforms/passes/merge_parallel_fully_connected.cpp
+index c2c08f6..a3edc00 100644
+--- a/src/vpux_compiler/src/dialect/IE/transforms/passes/merge_parallel_fully_connected.cpp
++++ b/src/vpux_compiler/src/dialect/IE/transforms/passes/merge_parallel_fully_connected.cpp
+@@ -838,6 +838,14 @@ private:
+ //
+ 
+ void MergeParallelFullyConnectedPass::safeRunOnFunc() {
++    // onw: not when compiling without weights - the merged weights become one Const.Concat of several model constants,
++    // which the weights separation cannot trace back to the model's files (every merged weight stays in the graph:
++    // Ternary Bonsai 2 9B 1-token parts 331.8 MB weightless against 332.6 MB). ONW_WS_MERGE_FC=1 keeps the merge.
++    if (auto mod = getOperation()->getParentOfType<mlir::ModuleOp>();
++        mod != nullptr && mod->hasAttr("VPU.OnwWeightless") && std::getenv("ONW_WS_MERGE_FC") == nullptr) {
++        return;
++    }
++
+     auto& ctx = getContext();
+     auto func = getOperation();
+ 
 diff --git a/src/vpux_compiler/src/dialect/IE/transforms/passes/swap_convert_with_reshape_kind_ops.cpp b/src/vpux_compiler/src/dialect/IE/transforms/passes/swap_convert_with_reshape_kind_ops.cpp
 index d16690c..93f292f 100644
 --- a/src/vpux_compiler/src/dialect/IE/transforms/passes/swap_convert_with_reshape_kind_ops.cpp
@@ -304,7 +513,7 @@ index 7390492..4dd29e4 100644
      case VPU::MultiClusterStrategy::Clustering:
          isStrategyCompatible = true;
 diff --git a/src/vpux_compiler/src/dialect/VPU/transforms/passes/multi_cluster_strategy_assignment.cpp b/src/vpux_compiler/src/dialect/VPU/transforms/passes/multi_cluster_strategy_assignment.cpp
-index adaca74..fab49de 100644
+index adaca74..c5d7f27 100644
 --- a/src/vpux_compiler/src/dialect/VPU/transforms/passes/multi_cluster_strategy_assignment.cpp
 +++ b/src/vpux_compiler/src/dialect/VPU/transforms/passes/multi_cluster_strategy_assignment.cpp
 @@ -5,6 +5,10 @@
@@ -318,7 +527,14 @@ index adaca74..fab49de 100644
  #include "vpux/compiler/dialect/VPU/utils/cost_model/layer_vpunn_cost.hpp"
  #include "vpux/compiler/dialect/VPU/utils/manual_strategy_utils.hpp"
  #include "vpux/compiler/dialect/VPU/utils/multi_cluster_strategy_utils.hpp"
-@@ -121,10 +125,21 @@ void MultiClusterStrategyAssignmentPass::safeRunOnFunc() {
+@@ -115,16 +119,28 @@ void MultiClusterStrategyAssignmentPass::safeRunOnFunc() {
+     cache.enableIfNecessary(clusteredOpCount > _clusteredOpThreshold);
+ 
+     auto& siblingAnalysis = getAnalysis<SiblingOpsAnalysis>();
++    VPU::OnwSiblingAnalysisScope onwSiblingScope(&siblingAnalysis);   // (onw: see sibling_ops_analysis.hpp)
+     StrategyManager strategyManager(func, tileOp.getCount(), _enablePrefetchTiling, _mcOptimizationScope,
+                                     siblingAnalysis, std::move(layerCostModel), _enableTilingFullSearchSpace,
+                                     _log.nest());
      _log.debug("Greedy Strategy Assignment");
      auto enableMultiClusterForSWLayer =
              config::getAvailableExecutor(module, config::ExecutorKind::SHAVE_ACT) != nullptr;
@@ -340,6 +556,331 @@ index adaca74..fab49de 100644
  
      _log.trace("Remove Temporary Strategy");
      strategyManager.removeTemporaryMulticlusterStrategy();
+diff --git a/src/vpux_compiler/src/dialect/VPU/utils/overlap_distribution_utils.cpp b/src/vpux_compiler/src/dialect/VPU/utils/overlap_distribution_utils.cpp
+index 62d2c5b..94ebf19 100644
+--- a/src/vpux_compiler/src/dialect/VPU/utils/overlap_distribution_utils.cpp
++++ b/src/vpux_compiler/src/dialect/VPU/utils/overlap_distribution_utils.cpp
+@@ -4,6 +4,7 @@
+ //
+ 
+ #include "vpux/compiler/dialect/VPU/utils/overlap_distribution_utils.hpp"
++#include <cstdlib>
+ #include "vpux/compiler/core/attributes/shape.hpp"
+ #include "vpux/compiler/core/tiling.hpp"
+ #include "vpux/compiler/dialect/VPU/IR/native_attributes/distribution_info.hpp"
+@@ -18,6 +19,8 @@
+ #include <mlir/Dialect/Func/IR/FuncOps.h>
+ 
+ #include <stack>
++#include <unordered_map>
++#include <vector>
+ 
+ using namespace vpux;
+ using namespace VPU;
+@@ -323,11 +326,106 @@ std::pair<Shape, Shape> getActualDataMemOffsetsAndShapes(VPU::SparseTensorType s
+ }
+ 
+ // Version with extension of halo
++static OverlapDistributionParams onwOverlapCompute(
++        NDTypeInterface tensorType, ArrayRef<VPU::ClusteredOpInterface> consumerSubgraph, const int64_t numClusters,
++        ArrayRef<int64_t> numTiles, const bool uniformDistributedSegments, const vpux::TileInfo& tileInfo,
++        const std::optional<ArrayRef<int64_t>> outputTensorMemoryNumTiles,
++        const std::optional<ArrayRef<int64_t>> alignment);
++
++namespace {
++// onw: the strategy search asks for the same overlapped params once per sibling and strategy check - a 2-bit Bonsai
++// matmul is split into one conv per weight group, so ~20 siblings each recomputed the whole group (94% of the strategy
++// pass). The result depends only on the values in the key, so it is cached while a pass publishes its sibling analysis
++// (OnwSiblingAnalysisScope clears it on entry and exit). ONW_NO_OVERLAP_CACHE=1: as before.
++struct OnwKeyHash {
++    size_t operator()(const std::vector<int64_t>& k) const {
++        return llvm::hash_combine_range(k.begin(), k.end());
++    }
++};
++thread_local std::unordered_map<std::vector<int64_t>, OverlapDistributionParams, OnwKeyHash> onwOverlapCache;
++// kernel / strides / pads per NCE op: getKernelSizeVal builds the filter shape from attributes each time (more than
++// half of the key cost); the strategy pass only sets strategies, so they stay fixed while the scope is active
++thread_local llvm::DenseMap<mlir::Operation*, SmallVector<int64_t, 12>> onwNceGeometry;
++const bool onwOverlapCacheOff = std::getenv("ONW_NO_OVERLAP_CACHE") != nullptr;
++// ONW_NO_SIBLING_VIEW=1: siblings / consumers copied out of the analysis as before
++const bool onwSiblingViewOn = std::getenv("ONW_NO_SIBLING_VIEW") == nullptr;
++}  // namespace
++
++void vpux::VPU::onwClearOverlapCache() {
++    onwOverlapCache.clear();
++    onwNceGeometry.clear();
++}
++
+ OverlapDistributionParams vpux::VPU::getOverlappedDistributionParameters(
+         NDTypeInterface tensorType, ArrayRef<VPU::ClusteredOpInterface> consumerSubgraph, const int64_t numClusters,
+         ArrayRef<int64_t> numTiles, const bool uniformDistributedSegments, const vpux::TileInfo& tileInfo,
+         const std::optional<ArrayRef<int64_t>> outputTensorMemoryNumTiles,
+         const std::optional<ArrayRef<int64_t>> alignment) {
++    if (onwCurrentSiblingAnalysis == nullptr || onwOverlapCacheOff || tensorType == nullptr) {
++        return onwOverlapCompute(tensorType, consumerSubgraph, numClusters, numTiles, uniformDistributedSegments,
++                                 tileInfo, outputTensorMemoryNumTiles, alignment);
++    }
++    std::vector<int64_t> key;
++    key.reserve(16 + 12 * consumerSubgraph.size());
++    key.push_back(reinterpret_cast<int64_t>(tensorType.getAsOpaquePointer()));
++    key.push_back(numClusters);
++    key.push_back(uniformDistributedSegments ? 1 : 0);
++    key.push_back(static_cast<int64_t>(numTiles.size()));
++    key.insert(key.end(), numTiles.begin(), numTiles.end());
++    key.push_back(outputTensorMemoryNumTiles.has_value() ? static_cast<int64_t>(outputTensorMemoryNumTiles->size())
++                                                         : -1);
++    if (outputTensorMemoryNumTiles.has_value()) {
++        key.insert(key.end(), outputTensorMemoryNumTiles->begin(), outputTensorMemoryNumTiles->end());
++    }
++    key.push_back(alignment.has_value() ? static_cast<int64_t>(alignment->size()) : -1);
++    if (alignment.has_value()) {
++        key.insert(key.end(), alignment->begin(), alignment->end());
++    }
++    bool anyNce = false;
++    for (auto clusteredOp : consumerSubgraph) {
++        auto nceOp = mlir::dyn_cast<VPU::NCEOpInterface>(clusteredOp.getOperation());
++        if (nceOp == nullptr) {
++            continue;
++        }
++        anyNce = true;
++        auto [geo, fresh] = onwNceGeometry.try_emplace(nceOp.getOperation());
++        if (fresh) {
++            const auto kernel = nceOp.getKernelSizeVal();
++            const auto strides = nceOp.getStridesVal();
++            const auto pads = VPU::Padding::getClassFromAttr(nceOp.getPad());
++            auto& g = geo->second;
++            g.push_back(-7);
++            g.push_back(static_cast<int64_t>(kernel.size()));
++            g.append(kernel.begin(), kernel.end());
++            g.push_back(static_cast<int64_t>(strides.size()));
++            g.append(strides.begin(), strides.end());
++            g.push_back(pads.getLeftPad());
++            g.push_back(pads.getRightPad());
++            g.push_back(pads.getTopPad());
++            g.push_back(pads.getBottomPad());
++        }
++        key.insert(key.end(), geo->second.begin(), geo->second.end());
++        key.push_back(reinterpret_cast<int64_t>(nceOp->getOperand(0).getType().getAsOpaquePointer()));
++    }
++    if (anyNce) {
++        const auto clusteringAxis = VPU::getDistributedTilingAxis(numTiles);
++        const auto tileOffset = !tileInfo.offsets.empty() ? tileInfo.offsets[Dim(clusteringAxis)] : 0;
++        key.push_back(tileOffset != 0 ? 1 : 0);
++    }
++    if (auto it = onwOverlapCache.find(key); it != onwOverlapCache.end()) {
++        return it->second;
++    }
++    auto result = onwOverlapCompute(tensorType, consumerSubgraph, numClusters, numTiles, uniformDistributedSegments,
++                                    tileInfo, outputTensorMemoryNumTiles, alignment);
++    onwOverlapCache.emplace(std::move(key), result);
++    return result;
++}
++
++static OverlapDistributionParams onwOverlapCompute(
++        NDTypeInterface tensorType, ArrayRef<VPU::ClusteredOpInterface> consumerSubgraph, const int64_t numClusters,
++        ArrayRef<int64_t> numTiles, const bool uniformDistributedSegments, const vpux::TileInfo& tileInfo,
++        const std::optional<ArrayRef<int64_t>> outputTensorMemoryNumTiles,
++        const std::optional<ArrayRef<int64_t>> alignment) {
+     VPUX_THROW_WHEN(tensorType == nullptr, "getOverlappedDistributionParameters: tensorType cannot be nullptr");
+ 
+     auto neutralKernel = SmallVector<int64_t>{1, 1};
+@@ -701,18 +799,23 @@ OverlapDistributionParams vpux::VPU::getActivationOverlappedParams(VPU::Clustere
+     auto sparseType = mlir::dyn_cast<vpux::VPU::SparseTensorType>(clusteredOp->getOperand(0).getType());
+     const bool isSparseTypeInputWithSeTable = (sparseType != nullptr) && (sparseType.getSeAttr() != nullptr);
+     SmallVector<VPU::ClusteredOpInterface> siblingSubgraph{};
++    ArrayRef<VPU::ClusteredOpInterface> siblingView;
+     if (isSparseTypeInputWithSeTable) {
+         siblingSubgraph.push_back(clusteredOp);
++        siblingView = siblingSubgraph;
++    } else if (onwSiblingViewOn) {
++        siblingView = siblingsAnalysis.onwSiblingsView(clusteredOp);
+     } else {
+         auto siblings = siblingsAnalysis.getSiblings(clusteredOp);
+         siblingSubgraph.append(siblings.begin(), siblings.end());
++        siblingView = siblingSubgraph;
+     }
+ 
+     const auto clusteringAxis = VPU::getDistributedTilingAxis(activationTensorNumTiles);
+     return getOverlappedDistributionParameters(
+             (inputType != nullptr) ? inputType
+                                    : mlir::cast<vpux::NDTypeInterface>(clusteredOp->getOperand(0).getType()),
+-            siblingSubgraph, activationTensorNumTiles[clusteringAxis], activationTensorNumTiles,
++            siblingView, activationTensorNumTiles[clusteringAxis], activationTensorNumTiles,
+             uniformDistributedSegments, tileInfo);
+ }
+ 
+@@ -920,12 +1023,20 @@ OverlapDistributionParams vpux::VPU::getOutputOverlappedParams(
+         return getOutputOverlappedParamsNoHalo(clusteredOp, outputTensorNumTiles);
+     }
+ 
+-    auto consumers = siblingsAnalysis.getConsumers(clusteredOp);
++    SmallVector<VPU::ClusteredOpInterface> consumerCopy;
++    ArrayRef<VPU::ClusteredOpInterface> consumerView;
++    if (onwSiblingViewOn) {
++        consumerView = siblingsAnalysis.onwConsumersView(clusteredOp);
++    } else {
++        auto consumers = siblingsAnalysis.getConsumers(clusteredOp);
++        consumerCopy.assign(consumers.begin(), consumers.end());
++        consumerView = consumerCopy;
++    }
+     const auto clusteringAxis = VPU::getDistributedTilingAxis(outputTensorNumTiles);
+     return getOverlappedDistributionParameters(
+             (outputType != nullptr) ? outputType
+                                     : mlir::cast<vpux::NDTypeInterface>(clusteredOp->getResult(0).getType()),
+-            SmallVector<VPU::ClusteredOpInterface>(consumers.begin(), consumers.end()),
++            consumerView,
+             outputTensorNumTiles[clusteringAxis], outputTensorNumTiles, uniformDistributedSegments, tileInfo,
+             outputTensorMemoryNumTiles, alignment);
+ }
+@@ -941,6 +1052,10 @@ OverlapDistributionParams vpux::VPU::getOutputOverlappedParams(
+         return getOutputOverlappedParamsNoHalo(clusteredOp, outputTensorNumTiles);
+     }
+ 
++    if (auto* shared = onwCurrentSiblingAnalysis; shared != nullptr && std::getenv("ONW_NO_SIBLING_CACHE") == nullptr) {
++        return getOutputOverlappedParams(clusteredOp, outputTensorNumTiles, uniformDistributedSegments, outputType,
++                                         tileInfo, *shared, outputTensorMemoryNumTiles, alignment);
++    }
+     auto siblingsAnalysis = SiblingOpsAnalysis(clusteredOp);
+     return getOutputOverlappedParams(clusteredOp, outputTensorNumTiles, uniformDistributedSegments, outputType,
+                                      tileInfo, siblingsAnalysis, outputTensorMemoryNumTiles, alignment);
+diff --git a/src/vpux_compiler/src/dialect/VPU/utils/sibling_ops_analysis.cpp b/src/vpux_compiler/src/dialect/VPU/utils/sibling_ops_analysis.cpp
+index c5f0be3..c28612d 100644
+--- a/src/vpux_compiler/src/dialect/VPU/utils/sibling_ops_analysis.cpp
++++ b/src/vpux_compiler/src/dialect/VPU/utils/sibling_ops_analysis.cpp
+@@ -8,6 +8,8 @@
+ 
+ namespace vpux::VPU {
+ 
++thread_local SiblingOpsAnalysis* onwCurrentSiblingAnalysis = nullptr;
++
+ // Empty constructor to satisfy getAnalysis API requirements.
+ SiblingOpsAnalysis::SiblingOpsAnalysis(mlir::Operation*) {
+ }
+@@ -16,17 +18,74 @@ std::set<ClusteredOpInterface> SiblingOpsAnalysis::lookupOpSiblings(mlir::Operat
+     if (op == nullptr) {
+         return {};
+     }
+-    if (auto clusteredOp = mlir::dyn_cast<VPU::ClusteredOpInterface>(op)) {
+-        for (const auto& siblingGroup : _siblingGroups) {
+-            if (siblingGroup.find(clusteredOp) != siblingGroup.end()) {
+-                return siblingGroup;
+-            }
+-        }
++    if (auto it = _onwGroupOf.find(op); it != _onwGroupOf.end()) {
++        return _siblingGroups[it->second];
+     }
+-
+     return {};
+ }
+ 
++void SiblingOpsAnalysis::addGroup(mlir::Operation* queried, std::set<ClusteredOpInterface> group) {
++    const size_t idx = _siblingGroups.size();
++    for (auto member : group) {                         // (by value: getOperation is not const)
++        _onwGroupOf.try_emplace(member.getOperation(), idx);
++    }
++    if (queried != nullptr) {
++        _onwGroupOf.try_emplace(queried, idx);
++    }
++    _onwGroupVecs.emplace_back(group.begin(), group.end());
++    _siblingGroups.emplace_back(std::move(group));
++}
++
++// (eager analyses are read from many threads at once - TilingStrategyAssignment's workers: their copies go into a
++// buffer of the calling thread; a shared member buffer was overwritten by other threads -> access violations)
++static thread_local SmallVector<ClusteredOpInterface> onwThreadScratch;
++
++ArrayRef<ClusteredOpInterface> SiblingOpsAnalysis::onwSiblingsView(ClusteredOpInterface clusteredOp) {
++    if (!_onwLazy) {
++        auto s = getSiblings(clusteredOp);
++        onwThreadScratch.assign(s.begin(), s.end());
++        return onwThreadScratch;
++    }
++    auto it = _onwGroupOf.find(clusteredOp.getOperation());
++    if (it == _onwGroupOf.end() || _siblingGroups[it->second].empty()) {
++        // as getSiblings: compute (an empty group is computed again, which gives an empty group again)
++        addGroup(clusteredOp.getOperation(), getSiblingOps(clusteredOp));
++        return _onwGroupVecs.back();
++    }
++    return _onwGroupVecs[it->second];
++}
++
++ArrayRef<ClusteredOpInterface> SiblingOpsAnalysis::onwConsumersView(ClusteredOpInterface clusteredOp) {
++    if (!_onwLazy) {
++        auto s = getConsumers(clusteredOp);
++        onwThreadScratch.assign(s.begin(), s.end());
++        return onwThreadScratch;
++    }
++    if (auto it = _onwConsumerGroup.find(clusteredOp.getOperation()); it != _onwConsumerGroup.end()) {
++        return _onwGroupVecs[it->second];
++    }
++    if (_consumers.contains(clusteredOp)) {
++        // filled by getConsumers before: answer with that copy
++        const auto& s = _consumers.at(clusteredOp);
++        onwThreadScratch.assign(s.begin(), s.end());
++        return onwThreadScratch;
++    }
++    auto consumerOp = getConsumerOp(clusteredOp);
++    if (consumerOp == nullptr) {
++        return {};
++    }
++    size_t idx;
++    auto it = _onwGroupOf.find(consumerOp);
++    if (it != _onwGroupOf.end() && !_siblingGroups[it->second].empty()) {
++        idx = it->second;
++    } else {
++        addGroup(consumerOp, getSiblingOps(consumerOp));
++        idx = _siblingGroups.size() - 1;
++    }
++    _onwConsumerGroup.try_emplace(clusteredOp.getOperation(), idx);
++    return _onwGroupVecs[idx];
++}
++
+ mlir::Operation* SiblingOpsAnalysis::getConsumerOp(ClusteredOpInterface clusteredOp) {
+     if (isPassthroughOp(clusteredOp.getOperation())) {
+         // For passthrough ops, ensure input and output tensors use the same pool of ops to
+@@ -47,7 +106,7 @@ std::set<ClusteredOpInterface> SiblingOpsAnalysis::getSiblings(ClusteredOpInterf
+     auto siblings = lookupOpSiblings(clusteredOp);
+     if (siblings.empty()) {
+         siblings = getSiblingOps(clusteredOp);
+-        _siblingGroups.emplace_back(siblings);
++        addGroup(clusteredOp.getOperation(), siblings);
+     }
+     return siblings;
+ }
+@@ -63,13 +122,14 @@ std::set<ClusteredOpInterface> SiblingOpsAnalysis::getConsumers(ClusteredOpInter
+     auto consumers = lookupOpSiblings(consumerOp);
+     if (consumers.empty()) {
+         consumers = getSiblingOps(consumerOp);
+-        _siblingGroups.emplace_back(consumers);
++        addGroup(consumerOp, consumers);
+     }
+     _consumers.insert(std::make_pair(clusteredOp, consumers));
+     return consumers;
+ }
+ 
+ EagerSiblingOpsAnalysis::EagerSiblingOpsAnalysis(mlir::func::FuncOp func): SiblingOpsAnalysis(func) {
++    _onwLazy = false;
+     // Skip siblings for architectures that do not support halo overlap to save some compile time
+     if (!outputOverlappedParamsIsHaloSupported(func)) {
+         return;
+@@ -77,7 +137,7 @@ EagerSiblingOpsAnalysis::EagerSiblingOpsAnalysis(mlir::func::FuncOp func): Sibli
+     func->walk([&](VPU::ClusteredOpInterface clusteredOp) {
+         if (lookupOpSiblings(clusteredOp.getOperation()).empty()) {
+             auto opSiblings = getSiblingOps(clusteredOp.getOperation());
+-            _siblingGroups.emplace_back(opSiblings);
++            addGroup(clusteredOp.getOperation(), opSiblings);
+         }
+         auto consumerOp = getConsumerOp(clusteredOp);
+         if (consumerOp == nullptr) {
+@@ -86,7 +146,7 @@ EagerSiblingOpsAnalysis::EagerSiblingOpsAnalysis(mlir::func::FuncOp func): Sibli
+         auto consumerSiblings = lookupOpSiblings(consumerOp);
+         if (consumerSiblings.empty()) {
+             consumerSiblings = getSiblingOps(consumerOp);
+-            _siblingGroups.emplace_back(consumerSiblings);
++            addGroup(consumerOp, consumerSiblings);
+         }
+         _consumers.insert(std::make_pair(clusteredOp, consumerSiblings));
+     });
 diff --git a/src/vpux_compiler/src/dialect/VPU/utils/strategy_manager/multi_cluster_strategy_utils.cpp b/src/vpux_compiler/src/dialect/VPU/utils/strategy_manager/multi_cluster_strategy_utils.cpp
 index 4bfec49..26e3194 100644
 --- a/src/vpux_compiler/src/dialect/VPU/utils/strategy_manager/multi_cluster_strategy_utils.cpp
@@ -772,7 +1313,7 @@ index 4207b33..7c9f3d1 100644
      return mlir::success();
  }
 diff --git a/src/vpux_compiler/src/dialect/VPU/utils/vertical_fusion/v2/merge_vf_region_rewriter.cpp b/src/vpux_compiler/src/dialect/VPU/utils/vertical_fusion/v2/merge_vf_region_rewriter.cpp
-index f3d8660..2e84289 100644
+index f3d8660..2d4e4fb 100644
 --- a/src/vpux_compiler/src/dialect/VPU/utils/vertical_fusion/v2/merge_vf_region_rewriter.cpp
 +++ b/src/vpux_compiler/src/dialect/VPU/utils/vertical_fusion/v2/merge_vf_region_rewriter.cpp
 @@ -28,6 +28,43 @@
@@ -925,7 +1466,7 @@ index f3d8660..2e84289 100644
          return std::nullopt;
      }
  
-@@ -543,8 +630,61 @@ bool MergeVFRegionRewriter::checkVFCostFunction(VPU::VerticalFusionOp prevOp, VP
+@@ -543,23 +630,108 @@ bool MergeVFRegionRewriter::checkVFCostFunction(VPU::VerticalFusionOp prevOp, VP
      return true;
  }
  
@@ -953,12 +1494,12 @@ index f3d8660..2e84289 100644
 +llvm::hash_code onwHashVF(VPU::VerticalFusionOp op) {
 +    llvm::hash_code h = llvm::hash_value(op->getAttrDictionary().getAsOpaquePointer());
 +    for (auto v : op->getOperands()) {
-+        h = llvm::hash_combine(h, onwHashType(v.getType()));
++        h = llvm::hash_combine(h, v ? onwHashType(v.getType()) : llvm::hash_code(0));
 +    }
 +    op.getBody()->walk([&](mlir::Operation* o) {
 +        h = llvm::hash_combine(h, o->getName().getAsOpaquePointer(), o->getAttrDictionary().getAsOpaquePointer());
-+        for (auto v : o->getOperands()) {
-+            h = llvm::hash_combine(h, onwHashType(v.getType()));
++        for (auto v : o->getOperands()) {             // (a null operand: an op being rewritten)
++            h = llvm::hash_combine(h, v ? onwHashType(v.getType()) : llvm::hash_code(0));
 +        }
 +        for (auto t : o->getResultTypes()) {
 +            h = llvm::hash_combine(h, onwHashType(t));
@@ -987,7 +1528,26 @@ index f3d8660..2e84289 100644
      _log.trace("Starting vertical fusion for region with VerticalFusionOp {0} at location {1}", vfOp, vfOp->getLoc());
      if (vfOp.getIsManualConfigured()) {
          _log.trace("Skipping vertical fusion for manually configured VerticalFusionOp at location {0}", vfOp->getLoc());
-@@ -553,13 +693,27 @@ mlir::LogicalResult MergeVFRegionRewriter::matchAndRewrite(VPU::VerticalFusionOp
+         return mlir::failure();
+     }
++    // onw: regions with split-weight-table convolutions (ONW_NEW_WT) are not merged - this build's merge leaves their
++    // scale / table operands unset (null) in the candidate regions
++    {
++        bool splitTable = false;
++        vfOp.getBody()->walk([&](VPU::NCEConvolutionOp conv) {
++            splitTable |= conv.getWeightTableScale() != nullptr;
++        });
++        for (auto operand : vfOp->getOperands()) {
++            if (auto prev = operand.getDefiningOp<VPU::VerticalFusionOp>()) {
++                prev.getBody()->walk([&](VPU::NCEConvolutionOp conv) {
++                    splitTable |= conv.getWeightTableScale() != nullptr;
++                });
++            }
++        }
++        if (splitTable) {
++            return mlir::failure();
++        }
++    }
  
      VPU::VerticalFusionOp vfBlock = nullptr;
      VPU::VerticalFusionOp parentVFOp = nullptr;
@@ -1015,7 +1575,7 @@ index f3d8660..2e84289 100644
  
          _log.trace("Analyzing vertical fusion region with parent VerticalFusionOp {0} at location {1}", parentVFOp,
                     parentVFOp->getLoc());
-@@ -580,7 +734,10 @@ mlir::LogicalResult MergeVFRegionRewriter::matchAndRewrite(VPU::VerticalFusionOp
+@@ -580,7 +752,10 @@ mlir::LogicalResult MergeVFRegionRewriter::matchAndRewrite(VPU::VerticalFusionOp
              continue;
          }
  
@@ -1027,7 +1587,7 @@ index f3d8660..2e84289 100644
          _log.trace("Merged subgraph candidate {0}", vfBlock.getLoc());
          // `findVFCase` internally compares operations using IR order which is calculated "on the fly" in e.g.
          // `isBeforeInBlock()`
-@@ -590,15 +747,34 @@ mlir::LogicalResult MergeVFRegionRewriter::matchAndRewrite(VPU::VerticalFusionOp
+@@ -590,15 +765,34 @@ mlir::LogicalResult MergeVFRegionRewriter::matchAndRewrite(VPU::VerticalFusionOp
          // As `findVFCase` is expected to not modify the IR, what can be done is an eager recalculation
          // of the operation orders that would ensure later calls to e.g. `isBeforeInBlock()` become concurrent
          // reads instead of concurrent writes (which is no longer a data race).
@@ -1066,7 +1626,7 @@ index f3d8660..2e84289 100644
              if (checkOtherVFInput(vfOp, parentVFOp)) {
                  continue;
              }
-@@ -614,7 +790,10 @@ mlir::LogicalResult MergeVFRegionRewriter::matchAndRewrite(VPU::VerticalFusionOp
+@@ -614,7 +808,10 @@ mlir::LogicalResult MergeVFRegionRewriter::matchAndRewrite(VPU::VerticalFusionOp
      }
  
      _log.trace("Merged subgraph {0}", vfBlock->getLoc());
@@ -1078,8 +1638,47 @@ index f3d8660..2e84289 100644
  
      return mlir::success();
  }
+diff --git a/src/vpux_compiler/src/dialect/VPU/utils/vertical_fusion/v2/vertical_fusion_config.cpp b/src/vpux_compiler/src/dialect/VPU/utils/vertical_fusion/v2/vertical_fusion_config.cpp
+index d25b349..e5fef7c 100644
+--- a/src/vpux_compiler/src/dialect/VPU/utils/vertical_fusion/v2/vertical_fusion_config.cpp
++++ b/src/vpux_compiler/src/dialect/VPU/utils/vertical_fusion/v2/vertical_fusion_config.cpp
+@@ -3,6 +3,7 @@
+ // SPDX-License-Identifier: Apache-2.0
+ //
+ 
++#include <cstdlib>
+ #include "vpux/compiler/dialect/VPU/utils/vertical_fusion/v2/vertical_fusion_config.hpp"
+ #include "vpux/compiler/dialect/VPU/utils/distributed_tensor_utils.hpp"
+ #include "vpux/compiler/dialect/VPU/utils/hash_utils.hpp"
+@@ -167,6 +168,16 @@ void VFConfig::init() {
+ 
+     if (_inputOps.empty()) {
+         auto operations = getVFOperations();
++        if (std::getenv("ONW_DEBUG_NULLOP") != nullptr) {      // onw: report ops with a null operand
++            for (auto* operation : operations) {
++                for (unsigned i = 0; i < operation->getNumOperands(); ++i) {
++                    if (!operation->getOperand(i)) {
++                        llvm::errs() << "ONW null operand " << i << " of " << operation->getName() << " at "
++                                     << operation->getLoc() << "\n";
++                    }
++                }
++            }
++        }
+         const auto allOperandsInputs = [&](auto* current) -> bool {
+             return llvm::all_of(current->getOperands(), [&](mlir::Value operand) {
+                 return mlir::dyn_cast<mlir::BlockArgument>(operand) != nullptr ||
+@@ -197,7 +208,8 @@ void VFConfig::init() {
+                                 break;
+                             }
+ 
+-                            parent = parent->getOperand(0).getDefiningOp();
++                            // onw: an op without operands (a constant feeding the split weight table's scale)
++                            parent = parent->getNumOperands() > 0 ? parent->getOperand(0).getDefiningOp() : nullptr;
+                         }
+                     }
+                 }
 diff --git a/src/vpux_compiler/src/dialect/VPU/utils/weights_separation.cpp b/src/vpux_compiler/src/dialect/VPU/utils/weights_separation.cpp
-index 35a6c00..d48e08c 100644
+index 35a6c00..cb47654 100644
 --- a/src/vpux_compiler/src/dialect/VPU/utils/weights_separation.cpp
 +++ b/src/vpux_compiler/src/dialect/VPU/utils/weights_separation.cpp
 @@ -4,6 +4,7 @@
@@ -1090,7 +1689,50 @@ index 35a6c00..d48e08c 100644
  #include "vpux/compiler/core/force_link_macros.hpp"
  #include "vpux/compiler/dialect/IE/IR/ops/data_movement.hpp"
  #include "vpux/compiler/dialect/IE/IR/ops/data_type.hpp"
-@@ -345,6 +346,9 @@ mlir::Value createAvgPoolForInterQuantizedConvert(mlir::OpBuilder& builder, mlir
+@@ -128,10 +129,24 @@ bool isTrivialForWeightsSeparation(Const::DeclareOp constOp, bool skipViewLikeOn
+ bool isSuitableForWeightlessCompilation(Const::DeclareOp constOp, bool skipViewLikeOnly) {
+     // ignore all non-OV constants
+     if (!Const::isOpenVINOConstant(constOp)) {
++        if (std::getenv("ONW_DEBUG_WS") != nullptr) {
++            const auto bt = mlir::cast<NDTypeInterface>(constOp.getContentAttr().getBaseContent().getType());
++            if (bt.getTotalAllocSize().count() >= (1 << 14)) {
++                llvm::errs() << "ONW ws non-OV constant: " << bt << " " << constOp.getLoc() << char(10);
++            }
++        }
+         return false;
+     }
+ 
+     if (isTrivialForWeightsSeparation(constOp, skipViewLikeOnly)) {
++        if (std::getenv("ONW_DEBUG_WS") != nullptr) {
++            const auto bt = mlir::cast<NDTypeInterface>(constOp.getContentAttr().getBaseContent().getType());
++            if (bt.getTotalAllocSize().count() >= (1 << 14)) {
++                llvm::errs() << "ONW ws trivial: " << bt << " splat=" << constOp.getContentAttr().isSplat()
++                             << " skipViewLikeOnly=" << skipViewLikeOnly << " n="
++                             << constOp.getContentAttr().getTransformations().size() << char(10);
++            }
++        }
+         return false;
+     }
+ 
+@@ -160,7 +175,16 @@ bool isSuitableForWeightlessCompilation(Const::DeclareOp constOp, bool skipViewL
+         return conversions::isSupportedTransformation(baseType, transformations.back());
+     };
+ 
+-    return hasOnlySupportedTransformations(constOp);
++    const bool supported = hasOnlySupportedTransformations(constOp);
++    if (!supported && std::getenv("ONW_DEBUG_WS") != nullptr) {
++        llvm::errs() << "ONW ws rejected: " << constOp.getContentAttr().getBaseContent().getType() << " "
++                     << constOp.getContentAttr().getTransformations().size() << " transformations:";
++        for (auto t : constOp.getContentAttr().getTransformations()) {
++            llvm::errs() << " " << t;
++        }
++        llvm::errs() << char(10);
++    }
++    return supported;
+ }
+ 
+ // Choose only constants that are used in multiple kernel functions
+@@ -345,6 +369,9 @@ mlir::Value createAvgPoolForInterQuantizedConvert(mlir::OpBuilder& builder, mlir
  /// Returns an IE operation for the given constant transformation.
  mlir::Value createMatchingIeOperation(mlir::OpBuilder& builder, mlir::Location loc, mlir::Value input,
                                        Const::TransformAttrInterface t) {
@@ -1100,7 +1742,7 @@ index 35a6c00..d48e08c 100644
      return llvm::TypeSwitch<Const::TransformAttrInterface, mlir::Value>(t)
              .Case<Const::AddAttr>([&](Const::AddAttr add) {
                  const auto biasValue = checked_cast<float>(add.getBias().getValueAsDouble());
-@@ -398,15 +402,94 @@ mlir::Value createMatchingIeOperation(mlir::OpBuilder& builder, mlir::Location l
+@@ -398,15 +425,94 @@ mlir::Value createMatchingIeOperation(mlir::OpBuilder& builder, mlir::Location l
                  const auto inElemType = mlir::cast<NDTypeInterface>(input.getType()).getElementType();
                  const auto outElemType = cast.getElemType();
  
@@ -1197,6 +1839,518 @@ index 35a6c00..d48e08c 100644
                  const auto inElemType = mlir::dyn_cast_or_null<mlir::quant::QuantizedType>(
                          mlir::cast<NDTypeInterface>(input.getType()).getElementType());
                  const auto outElemType = mlir::dyn_cast_or_null<mlir::quant::QuantizedType>(convert.getElemType());
+@@ -480,15 +586,25 @@ mlir::Value createMatchingIeOperation(mlir::OpBuilder& builder, mlir::Location l
+             })
+             .Case<Const::RescaleAttr>([&](Const::RescaleAttr rescale) {
+                 auto foldedAttr = rescale.getScale().fold();
+-                const auto scaleValue = foldedAttr.getSplatValue<float>();
+-
+                 const auto scaleLoc = appendLoc(loc, "scale");
+-                SmallVector<int64_t> shapeRank = {1};
+-                auto scaleType = mlir::RankedTensorType::get({shapeRank}, mlir::Float32Type::get(builder.getContext()));
+                 auto transform = [&](Const::ContentSetup& setup) -> Const::ContentSetup {
+                     return setup.castElemType(mlir::cast<NDTypeInterface>(input.getType()).getElementType());
+                 };
+-                auto scale = Const::createConst<float>(builder, scaleLoc, scaleType, {scaleValue}, transform);
++                mlir::Value scale;
++                if (foldedAttr.isSplat()) {
++                    const auto scaleValue = foldedAttr.getSplatValue<float>();
++                    SmallVector<int64_t> shapeRank = {1};
++                    auto scaleType =
++                            mlir::RankedTensorType::get({shapeRank}, mlir::Float32Type::get(builder.getContext()));
++                    scale = Const::createConst<float>(builder, scaleLoc, scaleType, {scaleValue}, transform);
++                } else {
++                    // onw: per-channel scales (2-bit weights): a constant of the scale's own shape, broadcast by the
++                    // Multiply (numpy rules, as Const::RescaleAttr::transform applies it)
++                    const auto values = to_small_vector(foldedAttr.getValues<float>());
++                    const auto shape = to_small_vector(rescale.getScale().getType().getShape().raw());
++                    auto scaleType = mlir::RankedTensorType::get(shape, mlir::Float32Type::get(builder.getContext()));
++                    scale = Const::createConst<float>(builder, scaleLoc, scaleType, values, transform);
++                }
+ 
+                 return builder.create<IE::MultiplyOp>(loc, input, scale, IE::AutoBroadcastType::NUMPY,
+                                                       /*postOp=*/nullptr, /*clamp=*/nullptr,
+@@ -1071,6 +1187,330 @@ std::tuple<ArrayRef<Const::TransformAttrInterface>, mlir::Value> ConstOpConverte
+     return {transformations.drop_front(), outputValue};
+ }
+ 
++namespace {
++// onw: sub-byte weights (2-bit Bonsai, 4-bit) in init schedules. Their constant chains unpack the codes to f16 only to
++// move them around (transpose, subview, reshape) and cast them back to a sub-byte type of the same storage - which the
++// NPU 40XX Convert kernels cannot write for 2 bits ("f16 -> ui2 not supported"). When the whole chain only moves the
++// values (value-preserving casts and view ops), its result is a box of the original tensor in its own order, reshaped:
++// that is a Slice + Reshape on the packed codes plus a reinterpretation of the storage - no unpacking at all.
++
++mlir::Type subByteStorage(mlir::Type t) {
++    for (int k = 0; k < 2; ++k) {
++        if (auto q = mlir::dyn_cast<mlir::quant::QuantizedType>(t)) {
++            t = q.getStorageType();
++        } else if (auto q = mlir::dyn_cast<vpux::type::QuantileType>(t)) {
++            t = q.getStorageType();
++        }
++    }
++    return t;
++}
++
++// a cast that keeps the code values of `storage` (unsigned / signed integers of width w): integer widening of the same
++// signedness, integer -> float, float -> float of >= 16 bits, float / integer -> a type stored as `storage`, and
++// quantized / quantile reinterpretations of the same storage
++bool keepsCodes(mlir::Type in, mlir::Type out, mlir::IntegerType storage) {
++    const auto inS = subByteStorage(in), outS = subByteStorage(out);
++    auto isInt = [](mlir::Type t) {
++        return mlir::isa<mlir::IntegerType>(t);
++    };
++    auto isFloat16Plus = [](mlir::Type t) {
++        return mlir::isa<mlir::FloatType>(t) && mlir::cast<mlir::FloatType>(t).getWidth() >= 16;
++    };
++    if (outS == storage) {
++        return inS == storage || isFloat16Plus(inS) ||
++               (isInt(inS) && mlir::cast<mlir::IntegerType>(inS).isSigned() == storage.isSigned());
++    }
++    // the lookup table of NPU 40XX+ (convert_to_palletization_lut_strategy): signed codes become unsigned table
++    // indices with the same bits (-7 = 4'b1001 is index 9) - a reinterpretation of storage, as the compiler meant it
++    if (inS == storage && isInt(outS) && mlir::cast<mlir::IntegerType>(outS).getWidth() == storage.getWidth() &&
++        mlir::isa<mlir::quant::QuantizedType>(out) &&
++        mlir::isa<vpux::type::QuantileType>(mlir::cast<mlir::quant::QuantizedType>(out).getStorageType())) {
++        return true;
++    }
++    if (isInt(inS) && isInt(outS)) {
++        const auto a = mlir::cast<mlir::IntegerType>(inS), b = mlir::cast<mlir::IntegerType>(outS);
++        return b.getWidth() >= a.getWidth() && a.isSigned() == b.isSigned() && a.isSigned() == storage.isSigned();
++    }
++    if (isInt(inS) && isFloat16Plus(outS)) {
++        return true;
++    }
++    return isFloat16Plus(inS) && isFloat16Plus(outS);
++}
++
++struct View {
++    SmallVector<int64_t> shape, strides;
++    int64_t offset = 0;
++};
++
++// strides of `newShape` over the same elements as `v` (a reshape of a view), or false when not expressible
++bool reshapeView(View& v, ArrayRef<int64_t> newShape) {
++    // drop size-1 dims, then match products of old and new dims chunk by chunk
++    SmallVector<int64_t> os, ost;
++    for (size_t i = 0; i < v.shape.size(); ++i) {
++        if (v.shape[i] != 1) {
++            os.push_back(v.shape[i]);
++            ost.push_back(v.strides[i]);
++        }
++    }
++    SmallVector<int64_t> nst(newShape.size(), 0);
++    size_t oi = 0;
++    size_t ni = 0;
++    while (ni < newShape.size()) {
++        if (newShape[ni] == 1) {
++            nst[ni] = 1;
++            ++ni;
++            continue;
++        }
++        if (oi >= os.size()) {
++            return false;
++        }
++        // a chunk: old dims oi..oj (contiguous in the view) against new dims ni..nj with the same product
++        int64_t op = os[oi], np = newShape[ni];
++        size_t oj = oi, nj = ni;
++        while (op != np) {
++            if (op < np) {
++                if (++oj >= os.size() || ost[oj - 1] != ost[oj] * os[oj]) {
++                    return false;
++                }
++                op *= os[oj];
++            } else {
++                if (++nj >= newShape.size()) {
++                    return false;
++                }
++                np *= newShape[nj];
++            }
++        }
++        for (size_t k = oi; k < oj; ++k) {
++            if (ost[k] != ost[k + 1] * os[k + 1]) {
++                return false;
++            }
++        }
++        int64_t st = ost[oj];
++        for (size_t k = nj + 1; k-- > ni;) {
++            nst[k] = st;
++            st *= newShape[k];
++        }
++        oi = oj + 1;
++        ni = nj + 1;
++    }
++    if (oi != os.size()) {
++        return false;
++    }
++    v.shape.assign(newShape.begin(), newShape.end());
++    v.strides = nst;
++    return true;
++}
++
++// The chain `ts` applied to `input` (sub-byte codes) as Slice + Reshape + QuantizeCast, or nullptr.
++// [onw] Group rows: a view that takes k consecutive groups of a weight [N, G, K] as [k, N, K] (the merged group
++// matmuls of a one-token graph: groups and rows swapped) is no box of the weight; it is built as a Concat of the k
++// group slices [N, 1, K] (byte moves of the packed codes), then sliced / reshaped like a box of that [k, N, K] stack.
++mlir::Value foldSubByteView(mlir::OpBuilder& builder, mlir::Location loc, mlir::Value input,
++                            ArrayRef<Const::TransformAttrInterface> ts) {
++    if (std::getenv("ONW_WS_VIEWFOLD") != nullptr && std::string(std::getenv("ONW_WS_VIEWFOLD")) == "0") {
++        return nullptr;
++    }
++    auto inType = mlir::cast<NDTypeInterface>(input.getType());
++    auto storage = mlir::dyn_cast<mlir::IntegerType>(subByteStorage(inType.getElementType()));
++    if (storage == nullptr || storage.getWidth() >= 8 || ts.empty() || !inType.getDimsOrder().isIdentity()) {
++        return nullptr;
++    }
++    const auto origShape = to_small_vector(inType.getShape().raw());
++    SmallVector<int64_t> srcShape = origShape;           // what the view addresses: the weight, or the group stack
++    std::optional<std::pair<int64_t, int64_t>> groupRows;  // (first group, groups) once rebased onto the stack
++    View v;
++    v.shape = origShape;
++    v.strides.assign(origShape.size(), 1);
++    for (size_t i = origShape.size() - 1; i-- > 0;) {
++        v.strides[i] = v.strides[i + 1] * origShape[i + 1];
++    }
++    // the view as k groups x N rows x K of a [N, G, K] weight (all rows, all of K) -> the same view on the stack
++    const auto rebase = [&]() -> bool {
++        if (groupRows.has_value() || origShape.size() != 3 || std::getenv("ONW_WS_NO_GROUP_ROWS") != nullptr) {
++            return false;
++        }
++        const int64_t N = origShape[0], G = origShape[1], K = origShape[2];
++        SmallVector<size_t> dims;
++        for (size_t i = 0; i < v.shape.size(); ++i) {
++            if (v.shape[i] != 1) {
++                dims.push_back(i);
++            }
++        }
++        if (dims.size() != 3 || v.strides[dims[0]] != K || v.shape[dims[1]] != N || v.strides[dims[1]] != G * K ||
++            v.shape[dims[2]] != K || v.strides[dims[2]] != 1 || v.offset % K != 0) {
++            return false;
++        }
++        const int64_t g0 = v.offset / K, k = v.shape[dims[0]];
++        if (g0 < 0 || g0 + k > G || k < 2) {
++            return false;
++        }
++        groupRows = std::make_pair(g0, k);
++        srcShape = {k, N, K};
++        v.strides[dims[0]] = N * K;
++        v.strides[dims[1]] = K;
++        v.strides[dims[2]] = 1;
++        v.offset = 0;
++        return true;
++    };
++    NDTypeInterface cur = inType;
++    for (auto t : ts) {
++        const auto next = t.inferOutputType(cur);
++        const auto reshapeTo = [&](ArrayRef<int64_t> shape) {
++            View keep = v;
++            if (reshapeView(v, shape)) {
++                return true;
++            }
++            v = keep;
++            return rebase() && reshapeView(v, shape);
++        };
++        const bool ok = llvm::TypeSwitch<Const::TransformAttrInterface, bool>(t)
++                .Case<Const::CastElemTypeAttr, Const::ConvertElemTypeAttr>([&](auto) {
++                    return keepsCodes(cur.getElementType(), next.getElementType(), storage);
++                })
++                .Case<Const::TransposeAttr>([&](Const::TransposeAttr tr) {
++                    const auto map = tr.getOrder().getValue();
++                    View w;
++                    for (unsigned i = 0; i < map.getNumResults(); ++i) {
++                        const auto d = map.getDimPosition(i);
++                        w.shape.push_back(v.shape[d]);
++                        w.strides.push_back(v.strides[d]);
++                    }
++                    w.offset = v.offset;
++                    v = w;
++                    return true;
++                })
++                .Case<Const::SubViewAttr>([&](Const::SubViewAttr sv) {
++                    const auto off = parseIntArrayAttr<int64_t>(sv.getOffset());
++                    const auto shp = parseIntArrayAttr<int64_t>(sv.getShape());
++                    for (size_t i = 0; i < off.size(); ++i) {
++                        v.offset += off[i] * v.strides[i];
++                    }
++                    v.shape.assign(shp.begin(), shp.end());
++                    return true;
++                })
++                .Case<Const::ReshapeAttr>([&](Const::ReshapeAttr r) {
++                    return reshapeTo(parseIntArrayAttr<int64_t>(r.getShape()));
++                })
++                .Case<Const::AffineReshapeAttr>([&](Const::AffineReshapeAttr r) {
++                    return reshapeTo(parseIntArrayAttr<int64_t>(r.getShapeValue()));
++                })
++                .Default([](Const::TransformAttrInterface) {
++                    return false;
++                });
++        if (!ok || !next.getDimsOrder().isIdentity()) {
++            return nullptr;
++        }
++        cur = next;
++    }
++    {
++        auto outStorage = mlir::dyn_cast<mlir::IntegerType>(subByteStorage(cur.getElementType()));
++        if (outStorage == nullptr || outStorage.getWidth() != storage.getWidth()) {
++            return nullptr;
++        }
++    }
++    // the result as a box of the source (the weight or the group stack) in its own row-major order: walk the non-1
++    // result dims from the innermost out, each covering one source dim (or several whole inner ones)
++    SmallVector<int64_t> boxOff, boxSize;
++    const auto findBox = [&]() -> bool {
++        boxOff.assign(srcShape.size(), 0);
++        boxSize.assign(srcShape.size(), 1);
++        int64_t rem = v.offset;
++        SmallVector<int64_t> ost(srcShape.size(), 1);
++        for (size_t i = srcShape.size() - 1; i-- > 0;) {
++            ost[i] = ost[i + 1] * srcShape[i + 1];
++        }
++        for (size_t i = 0; i < srcShape.size(); ++i) {
++            boxOff[i] = rem / ost[i];
++            rem %= ost[i];
++        }
++        int d = static_cast<int>(srcShape.size()) - 1;
++        for (size_t k = v.shape.size(); k-- > 0;) {
++            int64_t n = v.shape[k];
++            if (n == 1) {
++                continue;
++            }
++            const int64_t s = v.strides[k];
++            while (d >= 0 && (ost[d] != s || srcShape[d] == 1)) {
++                if (ost[d] < s) {
++                    --d;          // a source dim skipped: the box takes one element of it (its offset)
++                    continue;
++                }
++                return false;     // a stride that is no source dim's: the values are reordered
++            }
++            if (d < 0) {
++                return false;
++            }
++            // n elements from dim d up (the dims above d must then be taken whole)
++            while (n > 1) {
++                if (d < 0) {
++                    return false;
++                }
++                if (n <= srcShape[d] - boxOff[d] && (n == 1 || boxSize[d] == 1)) {
++                    boxSize[d] = n;
++                    n = 1;
++                } else if (boxOff[d] == 0 && n % srcShape[d] == 0 && boxSize[d] == 1) {
++                    boxSize[d] = srcShape[d];
++                    n /= srcShape[d];
++                } else {
++                    return false;
++                }
++                --d;
++            }
++        }
++        return true;
++    };
++    if (!findBox()) {
++        if (!rebase() || !findBox()) {
++            return nullptr;
++        }
++    }
++    // the box in row-major order must be exactly the result's element order: the result's dims walked above came in
++    // decreasing source dims, so it is; check the element count and the byte alignment of the innermost box
++    int64_t total = 1, boxTotal = 1;
++    for (auto n : v.shape) {
++        total *= n;
++    }
++    for (auto n : boxSize) {
++        boxTotal *= n;
++    }
++    const auto bits = storage.getWidth();
++    const auto inner = srcShape.size() - 1;
++    if (total != boxTotal || (boxOff[inner] * bits) % 8 != 0 || (boxSize[inner] * bits) % 8 != 0) {
++        return nullptr;
++    }
++    auto ctx = builder.getContext();
++    mlir::Value x = input;
++    if (inType.getElementType() != storage) {             // (quantized input: slice the raw codes, not the axes)
++        x = builder.create<IE::QuantizeCastOp>(appendLoc(loc, "onw_view_raw"), x, storage);
++    }
++    if (groupRows.has_value()) {
++        const auto [g0, k] = groupRows.value();
++        SmallVector<mlir::Value> groups;
++        for (int64_t g = g0; g < g0 + k; ++g) {
++            auto sl = builder.create<IE::SliceOp>(appendLoc(loc, "onw_group_{0}", g), x,
++                                                  getIntArrayAttr(ctx, SmallVector<int64_t>{0, g, 0}),
++                                                  getIntArrayAttr(ctx, SmallVector<int64_t>{origShape[0], 1, origShape[2]}));
++            groups.push_back(builder.create<IE::ReshapeOp>(appendLoc(loc, "onw_group_row_{0}", g), sl,
++                                                           getIntArrayAttr(ctx, SmallVector<int64_t>{1, origShape[0],
++                                                                                                     origShape[2]})));
++        }
++        x = builder.create<IE::ConcatOp>(appendLoc(loc, "onw_group_rows"), groups, Dim(0));
++    }
++    if (boxSize != srcShape) {
++        x = builder.create<IE::SliceOp>(appendLoc(loc, "onw_view_slice"), x, getIntArrayAttr(ctx, boxOff),
++                                        getIntArrayAttr(ctx, boxSize));
++    }
++    const auto outShape = to_small_vector(cur.getShape().raw());
++    if (to_small_vector(mlir::cast<NDTypeInterface>(x.getType()).getShape().raw()) != outShape) {
++        x = builder.create<IE::ReshapeOp>(appendLoc(loc, "onw_view_reshape"), x, getIntArrayAttr(ctx, outShape));
++    }
++    if (mlir::cast<NDTypeInterface>(x.getType()).getElementType() != cur.getElementType()) {
++        x = builder.create<IE::QuantizeCastOp>(appendLoc(loc, "onw_view_cast"), x, cur.getElementType());
++    }
++    return x;
++}
++}  // namespace
++
+ mlir::Value ConstOpConverter::convertToIrForm(mlir::Location loc, const VPU::TransformationsSplit::Projection& split,
+                                               mlir::BlockArgument argValue, const IoBoundaryAdapter& ioAdaptor,
+                                               WeightsSeparationSchedule scheduleKind) {
+@@ -1091,6 +1531,29 @@ mlir::Value ConstOpConverter::convertToIrForm(mlir::Location loc, const VPU::Tra
+     _log.trace("Creating matching operations for '{0}'", contentAttr);
+     _log.trace("  These transformations are converted into IR '{0}'", transformations);
+ 
++    if (scheduleKind == WeightsSeparationSchedule::Init && !transformations.empty()) {
++        std::vector<const void*> viewKey{value.getAsOpaquePointer()};
++        for (auto t : transformations) {
++            viewKey.push_back(t.getAsOpaquePointer());
++        }
++        mlir::Value folded;
++        if (auto it = _onwViewCache.find(viewKey); it != _onwViewCache.end()) {
++            folded = it->second;
++        } else {
++            folded = foldSubByteView(_opBuilder, appendLoc(loc, "onw_view"), value, transformations);
++            if (folded) {
++                _onwViewCache.emplace(std::move(viewKey), folded);
++            }
++        }
++        if (folded) {
++            if (std::getenv("ONW_DEBUG_WS") != nullptr) {
++                llvm::errs() << "ONW ws viewfold: " << value.getType() << " -> " << folded.getType() << char(10);
++            }
++            value = folded;
++            transformations = {};
++        }
++    }
++
+     // convert transformation chain to operations
+     size_t count = 0;
+     while (!transformations.empty()) {
+@@ -1152,8 +1615,119 @@ void removeRepeatedConstantsForHostPipeline(std::vector<Const::DeclareOp>& const
+ }
+ }  // namespace
+ 
++namespace {
++// onw: MergeFullyConnected joins the unrolled group matmuls of a 2-bit weight [N, G, K] (one-token graphs): the weight
++// of k groups becomes a Concat of k constants, each the same chain on the same OV constant ending with a one-group
++// SubView, on a dummy base - not an OV constant, so it stayed in the blob (a second copy of the codes) and the merged
++// matmul could not go weightless. When the parts are consecutive groups of one constant the Concat is that chain with
++// a k-group SubView: rewritten so, the weight is an OV constant again (its init builds it, see foldSubByteView).
++// ONW_WS_NO_CODE_CONCAT=1: as before.
++void onwCanonicalizeCodeConcats(mlir::ModuleOp moduleOp) {
++    if (!moduleOp->hasAttr("VPU.OnwWeightless") || std::getenv("ONW_WS_NO_CODE_CONCAT") != nullptr) {
++        return;
++    }
++    SmallVector<Const::DeclareOp> ops;
++    moduleOp.walk([&](Const::DeclareOp op) {
++        ops.push_back(op);
++    });
++    size_t rewritten = 0;
++    for (auto op : ops) {
++        const auto content = op.getContentAttr();
++        const auto ts = content.getTransformations();
++        if (ts.empty() || !Const::isDummyBaseContent(content.getBaseContent())) {
++            continue;
++        }
++        auto concat = mlir::dyn_cast<Const::ConcatAttr>(ts.front());
++        if (concat == nullptr || concat.getAxis().getInt() != 0) {
++            continue;
++        }
++        const auto constants = concat.getConstants().getValue();
++        const auto offsets = parseIntArrayOfArrayAttr<int64_t>(concat.getStaticOffsets());
++        if (constants.size() < 2 || offsets.size() != constants.size()) {
++            continue;
++        }
++        const auto first = constants[0];
++        const auto base = first.getBaseContent();
++        const auto baseType = mlir::cast<NDTypeInterface>(base.getType());
++        auto storage = mlir::dyn_cast<mlir::IntegerType>(baseType.getElementType());
++        if (!getResourceName(base).starts_with(Const::IMPORTED_WEIGHT_PREFIX) || storage == nullptr || storage.getWidth() >= 8 ||
++            baseType.getRank() != 3) {
++            continue;
++        }
++        const auto firstTs = first.getTransformations();
++        if (firstTs.empty() || !mlir::isa<Const::SubViewAttr>(firstTs.back())) {
++            continue;
++        }
++        const auto prefix = firstTs.drop_back(1);
++        // the SubView's input: [G', ...] with the groups on dim 0 (the parts' chains transpose them there)
++        const auto prefixType = Const::inferFinalType(baseType, prefix);
++        bool ok = true;
++        int64_t g0 = -1;
++        SmallVector<int64_t> partShape;
++        for (size_t i = 0; i < constants.size() && ok; ++i) {
++            const auto c = constants[i];
++            const auto cts = c.getTransformations();
++            ok = c.getBaseContent() == base && cts.size() == firstTs.size();
++            for (size_t j = 0; ok && j + 1 < cts.size(); ++j) {
++                ok = cts[j] == firstTs[j];
++            }
++            if (!ok) {
++                break;
++            }
++            auto sv = mlir::dyn_cast<Const::SubViewAttr>(cts.back());
++            ok = sv != nullptr;
++            if (!ok) {
++                break;
++            }
++            const auto off = parseIntArrayAttr<int64_t>(sv.getOffset());
++            const auto shp = parseIntArrayAttr<int64_t>(sv.getShape());
++            const auto full = to_small_vector(prefixType.getShape().raw());
++            ok = off.size() == full.size() && shp.size() == full.size() && shp[0] == 1;
++            for (size_t d = 1; ok && d < full.size(); ++d) {
++                ok = off[d] == 0 && shp[d] == full[d];
++            }
++            if (!ok) {
++                break;
++            }
++            if (i == 0) {
++                g0 = off[0];
++                partShape = shp;
++            }
++            ok = off[0] == g0 + static_cast<int64_t>(i) && offsets[i].size() == shp.size() &&
++                 offsets[i][0] == static_cast<int64_t>(i);
++            for (size_t d = 1; ok && d < offsets[i].size(); ++d) {
++                ok = offsets[i][d] == 0;
++            }
++        }
++        if (!ok || g0 < 0) {
++            continue;
++        }
++        SmallVector<int64_t> newOff(partShape.size(), 0), newShape(partShape.begin(), partShape.end());
++        newOff[0] = g0;
++        newShape[0] = static_cast<int64_t>(constants.size());
++        SmallVector<Const::TransformAttrInterface> newTs(prefix.begin(), prefix.end());
++        newTs.push_back(Const::SubViewAttr::get(getIntArrayAttr(op.getContext(), newOff),
++                                                getIntArrayAttr(op.getContext(), newShape)));
++        newTs.append(ts.begin() + 1, ts.end());
++        const auto newContent = Const::ContentAttr::get(base, newTs);
++        if (newContent.getType() != op.getType()) {
++            continue;
++        }
++        mlir::OpBuilder builder(op);
++        auto newOp = builder.create<Const::DeclareOp>(op.getLoc(), op.getType(), newContent);
++        op.replaceAllUsesWith(newOp.getOutput());
++        op.erase();
++        ++rewritten;
++    }
++    if (rewritten != 0 && std::getenv("ONW_DEBUG_WS") != nullptr) {
++        llvm::errs() << "ONW ws code concats rewritten: " << rewritten << char(10);
++    }
++}
++}  // namespace
++
+ WeightsSeparationInfo::WeightsSeparationInfo(mlir::ModuleOp moduleOp) {
+     auto log = Logger::global().nest("weights-separation-info", 0);
++    onwCanonicalizeCodeConcats(moduleOp);
+     auto mainFuncOp = net::getMainFunc(moduleOp);
+     auto tree = VPU::getOutliningRepresentation(mainFuncOp);
+ 
 diff --git a/src/vpux_compiler/src/dialect/VPUIP/transforms/passes/convert_async_ops_to_tasks.cpp b/src/vpux_compiler/src/dialect/VPUIP/transforms/passes/convert_async_ops_to_tasks.cpp
 index f637bad..6c3db25 100644
 --- a/src/vpux_compiler/src/dialect/VPUIP/transforms/passes/convert_async_ops_to_tasks.cpp
@@ -1293,7 +2447,7 @@ index 23fe4fc..b93a938 100644
  }
  
 diff --git a/src/vpux_compiler/src/dialect/const/attributes/content.cpp b/src/vpux_compiler/src/dialect/const/attributes/content.cpp
-index 3f4f862..f5d66fe 100644
+index 3f4f862..69d23e8 100644
 --- a/src/vpux_compiler/src/dialect/const/attributes/content.cpp
 +++ b/src/vpux_compiler/src/dialect/const/attributes/content.cpp
 @@ -7,6 +7,14 @@
@@ -1311,7 +2465,7 @@ index 3f4f862..f5d66fe 100644
  #include "vpux/compiler/dialect/const/dialect.hpp"
  #include "vpux/compiler/dialect/const/utils/const_logger.hpp"
  #include "vpux/compiler/utils/stable_hash.hpp"
-@@ -468,11 +476,117 @@ mlir::DenseElementsAttr Const::detail::createConstContentWithConversion(mlir::Sh
+@@ -468,11 +476,126 @@ mlir::DenseElementsAttr Const::detail::createConstContentWithConversion(mlir::Sh
  // ContentAttr::fold
  //
  
@@ -1348,18 +2502,24 @@ index 3f4f862..f5d66fe 100644
 -    for (const auto& attr : getTransformations()) {
 -        res = attr.transform(res);
 +    const auto transformations = getTransformations();
-+    // the last SubView that follows a Concat: the prefix before it is memoized
++    // the last SubView that follows a Concat or a permutation: the prefix before it is memoized. [onw] Weightless
++    // folding keeps SubViews last (ws::getWsOptimizations: moveSubViewAfter, so the init schedule transforms a weight
++    // once), so every group slice of a 2-bit weight [N, G, 128] (and of its scales) transposed the whole tensor again
++    // when folded on the host - SplitFakeQuant and the ELF export of 2-bit Bonsai parts (~10 s of a 16 s compile).
 +    int64_t cut = -1;
 +    bool sawConcat = false;
++    static const bool permuteMemo = std::getenv("ONW_NO_PERMUTE_FOLD_MEMO") == nullptr;
 +    for (size_t i = 0; i < transformations.size(); ++i) {
-+        if (mlir::isa<Const::ConcatAttr>(transformations[i])) {
++        if (mlir::isa<Const::ConcatAttr>(transformations[i]) ||
++            (permuteMemo && mlir::isa<Const::TransposeAttr, Const::MemPermuteAttr>(transformations[i]))) {
 +            sawConcat = true;
 +        } else if (sawConcat && mlir::isa<Const::SubViewAttr>(transformations[i])) {
 +            cut = static_cast<int64_t>(i);
 +        }
 +    }
 +    static std::atomic<int64_t> nHit{0}, nMiss{0}, nPlain{0};
-+    if (cut < 0 || std::getenv("ONW_NO_FOLD_MEMO") != nullptr) {
++    static const bool noFoldMemo = std::getenv("ONW_NO_FOLD_MEMO") != nullptr;
++    if (cut < 0 || noFoldMemo) {
 +        ++nPlain;
 +        auto res = wrapBaseContent(baseContent);
 +        for (const auto& attr : transformations) {
@@ -1403,7 +2563,10 @@ index 3f4f862..f5d66fe 100644
 +        for (size_t i = static_cast<size_t>(cut) + 1; i < transformations.size(); ++i) {
 +            res = transformations[i].transform(res);
 +        }
-+        return res;
++        // a result that still points into the memo's buffer (a SubView / reshape that moved its input instead of
++        // copying) must own its data: another thread can evict the entry and free the buffer once `held` is gone
++        // (multi-threaded compiles of 2-bit Bonsai 9B parts crashed with an access violation, now and then)
++        return Const::Content::copyUnownedBuffer(std::move(res));
 +    }
 +    if (!hit) {
 +        ++nMiss;
@@ -1432,6 +2595,105 @@ index 3f4f862..f5d66fe 100644
      }
      return res;
  }
+@@ -747,8 +870,18 @@ llvm::hash_code StableHashStorage<QuantizeAttrStorage>::calculateStableHash() co
+ template <>
+ llvm::hash_code StableHashStorage<RescaleAttrStorage>::calculateStableHash() const {
+     if (!this->scale.isSplat()) {
+-        // Note: non-splat rescale is currently not supported
+-        return {};
++        // onw: non-splat rescale (per-channel scales, e.g. 2-bit weights) - its shape and values, so that the weights
++        // separation can name the transformation chain (upstream: "not supported", QueryWSInfo threw)
++        const auto content = this->scale.fold();
++        SmallVector<uint32_t> bits;
++        for (const auto v : content.getValues<float>()) {
++            uint32_t b;
++            std::memcpy(&b, &v, sizeof(b));
++            bits.push_back(b);
++        }
++        const auto shape = this->scale.getType().getShape().raw();
++        return llvm::hash_combine(RescaleAttr::getMnemonic(), llvm::hash_combine_range(shape.begin(), shape.end()),
++                                  llvm::hash_combine_range(bits.begin(), bits.end()));
+     }
+ 
+     const auto scale = this->scale.fold().getSplatValue<double>();
+@@ -868,7 +1001,6 @@ llvm::hash_code StableHashStorage<CumSumAttrStorage>::calculateStableHash() cons
+ }  // namespace details
+ 
+ llvm::hash_code RescaleAttr::getStableHashValue() const {
+-    VPUX_THROW_UNLESS(getScale().isSplat(), "RescaleAttr scale must be splat");
+     return static_cast<details::StableHashStorage<details::RescaleAttrStorage>*>(getImpl())->stableHash;
+ }
+ 
+diff --git a/src/vpux_compiler/src/pipelines/developer_config.cpp b/src/vpux_compiler/src/pipelines/developer_config.cpp
+index 175c65d..5bde0f7 100644
+--- a/src/vpux_compiler/src/pipelines/developer_config.cpp
++++ b/src/vpux_compiler/src/pipelines/developer_config.cpp
+@@ -19,6 +19,7 @@
+ #include "vpux/utils/core/developer_path_utils.hpp"
+ #endif
+ 
++#include <cstdlib>
+ #include <mlir/Pass/Pass.h>
+ #include <mlir/Support/Timing.h>
+ 
+@@ -30,6 +31,18 @@
+ namespace vpux {
+ 
+ DeveloperConfig::DeveloperConfig(Logger log, mlir::MLIRContext* ctx, const vpux::OV::Config& config): _log(log) {
++#if !defined(VPUX_DEVELOPER_BUILD) && defined(NDEBUG)
++    // onw: IR dumps in release builds too (ONW_IR_PRINTING_FILTER / _LOCATION / _ORDER)
++    if (const char* v = std::getenv("ONW_IR_PRINTING_FILTER")) {
++        _irPrintingFilter = v;
++    }
++    if (const char* v = std::getenv("ONW_IR_PRINTING_LOCATION")) {
++        _irPrintingLocation = v;
++    }
++    if (const char* v = std::getenv("ONW_IR_PRINTING_ORDER")) {
++        _irPrintingOrderStr = v;
++    }
++#endif
+ #if defined(VPUX_DEVELOPER_BUILD) || !defined(NDEBUG)
+     parseEnv("IE_NPU_CRASH_REPRODUCER_FILE", _crashReproducerFile);
+     parseEnv("IE_NPU_GEN_LOCAL_REPRODUCER", _localReproducer);
+@@ -218,8 +231,38 @@ private:
+     Logger _log;
+ };
+ 
++// onw: ONW_FIND_NULLOP=1 - after every pass, report the first op that has a null operand (debugging passes that
++// leave half-built ops behind)
++class OnwNullOperandCheck final : public mlir::PassInstrumentation {
++public:
++    void runAfterPass(mlir::Pass* pass, mlir::Operation* op) final {
++        if (_reported) {
++            return;
++        }
++        op->walk([&](mlir::Operation* o) {
++            if (_reported) {
++                return;
++            }
++            for (unsigned i = 0; i < o->getNumOperands(); ++i) {
++                if (!o->getOperand(i)) {
++                    llvm::errs() << "ONW null operand " << i << " of " << o->getName() << " after pass "
++                                 << pass->getName() << " at " << o->getLoc() << "\n";
++                    _reported = true;
++                    return;
++                }
++            }
++        });
++    }
++
++private:
++    bool _reported = false;
++};
++
+ void addLogging(mlir::PassManager& pm, Logger log) {
+     pm.addInstrumentation(std::make_unique<PassLogging>(log));
++    if (std::getenv("ONW_FIND_NULLOP") != nullptr) {
++        pm.addInstrumentation(std::make_unique<OnwNullOperandCheck>());
++    }
+ }
+ 
+ void DeveloperConfig::setup(mlir::PassManager& pm, const vpux::OV::Config& config, bool isSubPipeline) const {
 diff --git a/src/vpux_compiler/src/utils/dma_transaction_utils.cpp b/src/vpux_compiler/src/utils/dma_transaction_utils.cpp
 index 10cb1f4..45e02e8 100644
 --- a/src/vpux_compiler/src/utils/dma_transaction_utils.cpp
@@ -1453,3 +2715,9 @@ index 10cb1f4..45e02e8 100644
      VPUX_THROW_WHEN(std::any_of(std::begin(reducedBitStrides), std::prev(std::end(reducedBitStrides)),
                                  [](auto bitStride) {
                                      return bitStride.count() % CHAR_BIT != 0;
+diff --git a/thirdparty/vpucostmodel b/thirdparty/vpucostmodel
+--- a/thirdparty/vpucostmodel
++++ b/thirdparty/vpucostmodel
+@@ -1 +1 @@
+-Subproject commit 4b5da8484d7a5c8cb097486c1248c4d2a753ae4b
++Subproject commit 4b5da8484d7a5c8cb097486c1248c4d2a753ae4b-dirty

onw/npukit/patches/openvino-6db412d-npu_plugin.patch

@@ -13,10 +13,10 @@ index c5f05a69..e976d53b 100644
       * @brief Tensors pointing towards the buffers found in "_mainInputsAllocatedTensors".
       * @details Each map entry corresponds to one input of the main schedule.
 diff --git a/src/plugins/intel_npu/src/compiler_adapter/src/weightless_graph.cpp b/src/plugins/intel_npu/src/compiler_adapter/src/weightless_graph.cpp
-index b56d95b5..24790c3c 100644
+index b56d95b5..2d43d8fe 100644
 --- a/src/plugins/intel_npu/src/compiler_adapter/src/weightless_graph.cpp
 +++ b/src/plugins/intel_npu/src/compiler_adapter/src/weightless_graph.cpp
-@@ -5,6 +5,11 @@
+@@ -5,8 +5,14 @@
  #include "weightless_graph.hpp"
  
  #include <cinttypes>
@@ -27,8 +27,19 @@ index b56d95b5..24790c3c 100644
 +#include <cstdio>
  #include <condition_variable>
  #include <iterator>
++#include <map>
  #include <mutex>
-@@ -30,6 +35,102 @@ namespace intel_npu {
+ #include <queue>
+ 
+@@ -24,12 +30,200 @@
+ #include "openvino/util/common_util.hpp"
+ #include "openvino/util/mmap_object.hpp"
+ 
++#include <algorithm>
++
+ #define USE_SINGLE_THREADED_RUN_INIT 1
+ 
+ namespace intel_npu {
  
  namespace {
  
@@ -45,6 +56,10 @@ index b56d95b5..24790c3c 100644
 +};
 +std::mutex onwWsMutex;
 +std::unordered_multimap<uint64_t, OnwWsEntry> onwWsRegistry;
++// outputs of 64 KB and more, for outputs that are a part of one (onw 3.2): a 64-token graph keeps a 2-bit weight per
++// group, the one-token graph the same groups merged k at a time - group g of a merged weight is its bytes at
++// (g - g0) x the group's size. ONW_WS_SUBRANGE=0: whole outputs only.
++std::vector<OnwWsEntry> onwWsBig;
 +
 +uint64_t onwSampledHash(const unsigned char* p, size_t n) {
 +    uint64_t h = 1469598103934665603ULL ^ n;
@@ -67,46 +82,98 @@ index b56d95b5..24790c3c 100644
 +    return e == nullptr || std::string(e) != "0";
 +}
 +
-+// returns the owners to keep alive when every output was found (and rewrites the map to point at them), else registers
-+// this buffer's outputs and returns nothing
++// Per output (onw 3.2): every output of the schedule is looked up on its own. Found ones are bound to the existing
++// buffer; when all were found the new buffer is released as before; when only some were, the others are copied into a
++// compact buffer of their own (registered for later graphs) and the big one is released too - so graphs that hold the
++// same weights in a different set (the 1-token and the 64-token graphs of a 2-bit model: one output per weight group,
++// split differently over their init schedules) share every tensor they have in common. Returns the buffers to keep
++// alive when the new buffer can go, else registers its outputs and returns nothing (ONW_WS_SHARE_PARTIAL=0: all or
++// nothing, as before).
 +std::optional<std::vector<std::shared_ptr<ZeroTensor>>> onwWsShare(
++    const std::shared_ptr<ZeroInitStructsHolder>& initStruct,
 +    const std::shared_ptr<ZeroTensor>& hostTensor,
 +    std::unordered_map<std::string, std::shared_ptr<ov::ITensor>>& tensorsMap) {
 +    if (!onwWsShareEnabled() || hostTensor == nullptr) {
 +        return std::nullopt;
 +    }
 +    std::lock_guard<std::mutex> lock(onwWsMutex);
-+    std::vector<std::pair<std::string, std::pair<std::shared_ptr<ZeroTensor>, const unsigned char*>>> found;
-+    size_t bytes = 0;
++    const bool log = std::getenv("ONW_WS_SHARE_LOG") != nullptr;
++    const char* pe = std::getenv("ONW_WS_SHARE_PARTIAL");
++    const bool partial = pe == nullptr || std::string(pe) != "0";
++    // found[name] = (owner, data) of an identical live buffer
++    std::unordered_map<std::string, std::pair<std::shared_ptr<ZeroTensor>, const unsigned char*>> found;
++    size_t foundBytes = 0, restBytes = 0;
 +    for (const auto& [name, t] : tensorsMap) {
 +        const auto* p = static_cast<const unsigned char*>(t->data());
 +        const size_t n = t->get_byte_size();
-+        const uint64_t h = onwSampledHash(p, n);
 +        bool hit = false;
-+        auto range = onwWsRegistry.equal_range(h);
++        auto range = onwWsRegistry.equal_range(onwSampledHash(p, n));
 +        for (auto it = range.first; it != range.second; ++it) {
 +            auto owner = it->second.owner.lock();
 +            if (owner == nullptr || owner == hostTensor || it->second.size != n) {
 +                continue;
 +            }
 +            if (std::memcmp(it->second.data, p, n) == 0) {
-+                found.push_back({name, {owner, it->second.data}});
-+                bytes += n;
++                found.emplace(name, std::make_pair(owner, it->second.data));
++                foundBytes += n;
 +                hit = true;
 +                break;
 +            }
 +        }
++        static const bool subrange = std::getenv("ONW_WS_SUBRANGE") == nullptr ||
++                                     std::string(std::getenv("ONW_WS_SUBRANGE")) != "0";
++        if (!hit && subrange && partial && n >= (size_t(1) << 15)) {
++            for (const auto& big : onwWsBig) {
++                if (big.size <= n || big.size % n != 0) {
++                    continue;
++                }
++                auto owner = big.owner.lock();
++                if (owner == nullptr || owner == hostTensor) {
++                    continue;
++                }
++                for (size_t off = 0; off + n <= big.size; off += n) {
++                    if (std::memcmp(big.data + off, p, 64) == 0 && std::memcmp(big.data + off, p, n) == 0) {
++                        found.emplace(name, std::make_pair(owner, big.data + off));
++                        foundBytes += n;
++                        hit = true;
++                        break;
++                    }
++                }
++                if (hit) {
++                    break;
++                }
++            }
++        }
 +        if (!hit) {
-+            break;
++            restBytes += n;
++            if (!partial) {
++                break;
++            }
 +        }
 +    }
-+    const bool log = std::getenv("ONW_WS_SHARE_LOG") != nullptr;
-+    if (found.size() != tensorsMap.size()) {
-+        for (const auto& [name, t] : tensorsMap) {
++    auto registerAll = [&](const std::shared_ptr<ZeroTensor>& owner,
++                           const std::unordered_map<std::string, std::shared_ptr<ov::ITensor>>& m,
++                           const std::unordered_map<std::string, bool>* only) {
++        for (const auto& [name, t] : m) {
++            if (only != nullptr && only->count(name) == 0) {
++                continue;
++            }
 +            const auto* p = static_cast<const unsigned char*>(t->data());
 +            const size_t n = t->get_byte_size();
-+            onwWsRegistry.emplace(onwSampledHash(p, n), OnwWsEntry{hostTensor, p, n});
++            onwWsRegistry.emplace(onwSampledHash(p, n), OnwWsEntry{owner, p, n});
++            if (n >= (size_t(1) << 16)) {
++                onwWsBig.push_back(OnwWsEntry{owner, p, n});
++            }
 +        }
++        // (entries of released buffers go)
++        onwWsBig.erase(std::remove_if(onwWsBig.begin(), onwWsBig.end(),
++                                      [](const OnwWsEntry& e) {
++                                          return e.owner.expired();
++                                      }),
++                       onwWsBig.end());
++    };
++    if (found.empty() || (!partial && found.size() != tensorsMap.size())) {
++        registerAll(hostTensor, tensorsMap, nullptr);
 +        if (log) {
 +            std::fprintf(stderr, "[onw ws-share] new weights: %zu outputs, %.1f MB%c", tensorsMap.size(),
 +                         hostTensor->get_byte_size() / 1e6, 10);
@@ -120,6 +187,40 @@ index b56d95b5..24790c3c 100644
 +                                           const_cast<unsigned char*>(src.second));
 +        owners.push_back(src.first);
 +    }
++    if (found.size() != tensorsMap.size()) {
++        // the outputs not found: copied into a compact buffer (each at a 64-byte aligned offset)
++        auto align = [](size_t v) {
++            return (v + 63) & ~size_t(63);
++        };
++        size_t total = 0;
++        for (const auto& [name, t] : tensorsMap) {
++            if (found.count(name) == 0) {
++                total += align(t->get_byte_size());
++            }
++        }
++        auto compact = std::make_shared<ZeroTensor>(initStruct, ov::element::Type_t::u8, ov::Shape({total}), false);
++        auto* base = static_cast<unsigned char*>(const_cast<void*>(compact->data(ov::element::Type_t::u8)));
++        size_t off = 0;
++        std::unordered_map<std::string, bool> moved;
++        for (auto& [name, t] : tensorsMap) {
++            if (found.count(name) != 0) {
++                continue;
++            }
++            const size_t n = t->get_byte_size();
++            std::memcpy(base + off, t->data(), n);
++            t = ov::make_tensor(t->get_element_type(), t->get_shape(), base + off);
++            moved.emplace(name, true);
++            off += align(n);
++        }
++        registerAll(compact, tensorsMap, &moved);
++        owners.push_back(compact);
++        if (log) {
++            std::fprintf(stderr, "[onw ws-share] partly shared: %zu of %zu outputs, %.1f MB shared, %.1f MB kept "
++                         "(of %.1f MB)%c", found.size(), tensorsMap.size(), foundBytes / 1e6, total / 1e6,
++                         hostTensor->get_byte_size() / 1e6, 10);
++        }
++        return owners;
++    }
 +    if (log) {
 +        std::fprintf(stderr, "[onw ws-share] shared: %zu outputs, %.1f MB released%c", found.size(),
 +                     hostTensor->get_byte_size() / 1e6, 10);
@@ -131,7 +232,7 @@ index b56d95b5..24790c3c 100644
  constexpr uint8_t MAIN_SCHEDULE_INDEX = 0;
  constexpr std::string_view WEIGHTS_IR_EXTENSION = ".bin";
  constexpr std::string_view WEIGHTS_ONNX_EXTENSION = ".data";
-@@ -423,6 +524,7 @@ void WeightlessGraph::initialize_impl(const FilteredConfig& config) {
+@@ -423,6 +617,7 @@ void WeightlessGraph::initialize_impl(const FilteredConfig& config) {
      _initsFences.clear();
      _initsMetadata.clear();
      _initsCommandQueue.reset();
@@ -139,7 +240,7 @@ index b56d95b5..24790c3c 100644
  
      // The main schedule is initialized after the weights initialization ones in order to save some memory
      Graph::initialize_impl(config);
-@@ -502,7 +604,13 @@ WeightlessGraph::InputData WeightlessGraph::allocate_inputs(
+@@ -502,7 +697,13 @@ WeightlessGraph::InputData WeightlessGraph::allocate_inputs(
          // model appears in exactly one schedule. Thus, one can delete
          // the handle to the constant memory early.
          if (auto it = constants.find(id); it != constants.end()) {
@@ -154,11 +255,40 @@ index b56d95b5..24790c3c 100644
              constants.erase(it);
          }
      }
-@@ -557,6 +665,14 @@ void WeightlessGraph::run_init_single_threaded() {
+@@ -525,6 +726,28 @@ WeightlessGraph::OutputData WeightlessGraph::allocate_outputs(const size_t initI
+         initOutputsByteSize += size.value();
+     }
+ 
++    if (std::getenv("ONW_WS_SHARE_LOG") != nullptr && initOutputsByteSize >= (size_t(1) << 30)) {
++        std::map<std::string, std::pair<size_t, size_t>> byType;   // type -> (count, bytes)
++        for (const IODescriptor& descriptor : _initsMetadata.at(initIndex).outputs) {
++            auto& e = byType[descriptor.precision.to_string()];
++            e.first += 1;
++            e.second += ov::util::get_memory_size(descriptor.precision, shape_size(descriptor.shapeFromCompiler.to_shape()));
++        }
++        std::fprintf(stderr, "[onw ws-share] init %zu outputs: %zu, %.1f MB%c", initIndex,
++                     _initsMetadata.at(initIndex).outputs.size(), initOutputsByteSize / 1e6, 10);
++        for (const auto& [t, e] : byType) {
++            std::fprintf(stderr, "[onw ws-share]   type %s: %zu outputs, %.1f MB%c", t.c_str(), e.first, e.second / 1e6,
++                         10);
++        }
++        size_t k = 0;
++        for (const IODescriptor& descriptor : _initsMetadata.at(initIndex).outputs) {
++            if (k++ < 4) {
++                std::fprintf(stderr, "[onw ws-share]   e.g. %s %s %s%c", descriptor.nameFromCompiler.c_str(),
++                             descriptor.precision.to_string().c_str(),
++                             descriptor.shapeFromCompiler.to_string().c_str(), 10);
++            }
++        }
++    }
+     const std::shared_ptr<ZeroTensor> initOutputsAllocatedTensor =
+         std::make_shared<ZeroTensor>(_zeroInitStruct, ov::element::Type_t::u8, ov::Shape({initOutputsByteSize}), false);
+ 
+@@ -557,6 +780,14 @@ void WeightlessGraph::run_init_single_threaded() {
          create_pipeline(initIndex, initInputsViewTensors, initOutputsViewTensors);
          run_pipeline(initIndex);
  
-+        if (auto owners = onwWsShare(initOutputsAllocatedTensor, initOutputsViewTensorsMap)) {
++        if (auto owners = onwWsShare(_zeroInitStruct, initOutputsAllocatedTensor, initOutputsViewTensorsMap)) {
 +            merge_two_maps(_mainInputsViewTensors, initOutputsViewTensorsMap);
 +            for (auto& o : *owners) {
 +                _mainInputsAllocatedTensors.push_back(o);
@@ -169,11 +299,11 @@ index b56d95b5..24790c3c 100644
          merge_two_maps(_mainInputsViewTensors, initOutputsViewTensorsMap);
          _mainInputsAllocatedTensors.push_back(std::move(initOutputsAllocatedTensor));
      }
-@@ -600,6 +716,14 @@ void WeightlessGraph::run_init_multi_threaded() {
+@@ -600,6 +831,14 @@ void WeightlessGraph::run_init_multi_threaded() {
              run_pipeline(data.initIndex);
  
              // TODO: pre-allocate those well in advance? (outside of this loop)
-+            if (auto owners = onwWsShare(data.outputs.hostTensor, data.outputs.tensorsMap)) {
++            if (auto owners = onwWsShare(_zeroInitStruct, data.outputs.hostTensor, data.outputs.tensorsMap)) {
 +                merge_two_maps(_mainInputsViewTensors, data.outputs.tensorsMap);
 +                for (auto& o : *owners) {
 +                    _mainInputsAllocatedTensors.push_back(o);

onw/runtime.py

@@ -311,7 +311,9 @@ def all_sizes(meta):
     """[2.0] Every block size the model folder has graphs for: "sizes" (what onw before 2.0 reads) + "prompt_sizes"
     (larger prompt blocks added to a published model later, see compact.graft_sizes - out of older onw's sight, whose
     loader cannot read their seg<i>x.bin)."""
-    return sorted(set(meta["sizes"]) | set(meta.get("prompt_sizes") or []))
+    # [3.2] "kit_sizes": the 2-bit (Bonsai) 64-token graphs - out of the sight of onw before 3.2, which would compute
+    # them wrong without the kit (npu_compiler #355) and hold the 2-bit weights twice with it
+    return sorted(set(meta["sizes"]) | set(meta.get("prompt_sizes") or []) | set(meta.get("kit_sizes") or []))
 
 
 _CONST_RE = re.compile(rb'<layer id="\d+" name="([^"]+)" type="Const"([^>]*)>(\s*)<data ([^>]*?)offset="(\d+)" size="(\d+)"([^>]*)/>')
@@ -348,6 +350,14 @@ def _xbin_model(core, d, name, m, weights):
     return model
 
 
+def _own_mapping():
+    """[3.2] Weightless blobs on Windows: the weights are read through our own file mapping (as with NPUW), so that
+    release_pages can drop their pages from the working set once the init schedules put them on the NPU - read by
+    OpenVINO's own mapping they stayed (Panther Lake, Bonsai 4B: working set 0.9 GB above the private memory).
+    ONW_MAP_BINS=0: OpenVINO's mapping as before."""
+    return os.name == "nt" and os.environ.get("ONW_MAP_BINS") != "0" and weightless_wanted()
+
+
 def read_segment(core, d, name, m, npuw, bins, T, built_T):
     """The segment's ov.Model, reshaped to context T. With NPUW every segment of a .bin reads against the same
     in-memory copy (bins cache), so the weights bank sees one set of constants."""
@@ -358,23 +368,65 @@ def read_segment(core, d, name, m, npuw, bins, T, built_T):
         model = _xbin_model(core, d, name, m, bins[b])
         if npuw:
             model = _no_bool_constants(model)
-    elif npuw:
+    elif npuw or _own_mapping():
         b = m.get("bin") or name[:-4] + ".bin"
         if b not in bins:
             # a file-backed mapping (OpenVINO shares it, see mapped_bin): NPUW's weights bank keeps reading the
             # constants after compiling, so the buffer lives as long as the model (SegmentedModel._bins), and as
             # mapped pages it costs no private memory
             bins[b] = ov.Tensor(mapped_bin(os.path.join(d, b)), shared_memory=True)
-        model = _no_bool_constants(core.read_model(open(os.path.join(d, name), encoding="utf8").read(), bins[b]))
+        model = core.read_model(open(os.path.join(d, name), encoding="utf8").read(), bins[b])
+        if npuw:
+            model = _no_bool_constants(model)
     else:
         model = core.read_model(os.path.join(d, name), os.path.join(d, m["bin"]) if "bin" in m else None)
     if T != built_T:
         model.reshape(reshaped_inputs(model, m["S"], built_T, T))
     if os.environ.get("ONW_CODES_SCALE") == "const" and codes_const_scale(model):   # (opt-in, see there; marked
         model.set_rt_info("1", "onw_codes_const")                                    # for the compile keys)
+    if os.environ.get("ONW_LEFTMM") != "0" and right_const_matmuls(model):          # (marked for the compile keys)
+        model.set_rt_info("1", "onw_leftmm")
     return model
 
 
+def right_const_matmuls(model):
+    """[3.2] MatMul(C, X) - a constant 2-D C times a batched X [..., a, b] (Bonsai's Hadamard rotation, ir.hadamard:
+    H_a X H_b) - becomes Transpose(MatMul(Transpose(X), C^T)): the NPU compiler unrolls a constant on the left into one
+    tiny matmul per batch (512 per 64-token block of Bonsai 4B, 93% of a part's matmuls), which made its compile 3-4x
+    slower; on the right it is one matmul. The same values (CPU: equal outputs). Returns how many were rewritten."""
+    import openvino.opset13 as ov_ops
+
+    def const2d(v):
+        n, conv = v.get_node(), None
+        if n.get_type_name() == "Convert":
+            conv, n = n, n.input_value(0).get_node()
+        if n.get_type_name() != "Constant" or len(n.get_output_partial_shape(0)) != 2:
+            return None
+        return n, conv
+    k = 0
+    for op in list(model.get_ordered_ops()):
+        if op.get_type_name() != "MatMul":
+            continue
+        at = op.get_attributes()
+        x = op.input_value(1)
+        r = len(x.get_partial_shape())
+        got = const2d(op.input_value(0))
+        if at.get("transpose_a") or at.get("transpose_b") or got is None or r < 3 or const2d(x) is not None:
+            continue
+        cn, conv = got
+        ct = ov_ops.constant(np.ascontiguousarray(cn.get_data().T))
+        if conv is not None:
+            ct = ov_ops.convert(ct, conv.get_destination_type())
+        perm = ov_ops.constant(np.array(list(range(r - 2)) + [r - 1, r - 2], np.int64))
+        y = ov_ops.transpose(ov_ops.matmul(ov_ops.transpose(x, perm), ct, False, False), perm)
+        for t in op.output(0).get_target_inputs():
+            t.replace_source_output(y.output(0))
+        k += 1
+    if k:
+        model.validate_nodes_and_infer_types()
+    return k
+
+
 GROUPED_MIN = int(os.environ.get("ONW_GROUPED_MIN", "2"))   # [2.0] prompts from this long go the grouped-MoE path
 GROUPED_RAM_GB = 24            # [2.0] RAM the grouped-MoE prompt path needs
 
@@ -944,6 +996,13 @@ class _Tried:
             pass
 
 
+class _NoBank(dict):
+    """[3.2] A precompile child's stand-in for the expert bank: truthy like a loaded one (the compile routes test it),
+    holding nothing."""
+    def __bool__(self):
+        return True
+
+
 class Precompiled(Exception):
     """Raised by a precompile child (ONW_PRECOMPILE_ONLY) once its segments are in the compile cache."""
 
@@ -962,9 +1021,34 @@ def compile_jobs(core, device, npuw):
     (Panther Lake, Ornith-1.5-9B NF4: 8.4 instead of 5.2 GB of NPU memory, onw 1.4.2 / 1.4.3 did this by default).
     Each child holds about 3.5 GB at its peak."""
     env = os.environ.get("ONW_COMPILE_JOBS")
-    if not env or device != "NPU" or not npuw or mem_saver():
+    if device != "NPU":
         return 1
-    return max(1, int(env))
+    if npuw:
+        return max(1, int(env)) if env and not mem_saver() else 1
+    if env:
+        return max(1, int(env))
+    # [3.2] onw's own blobs (no NPUW): the children only compile and write the blobs (export_only, no import - no
+    # weights on the NPU, no copies), so as many as the free memory holds at ~2.5 GB a child (a 2-bit Bonsai part
+    # peaks at ~1.1 GB), at most half the cores, 6; 2 in the memory-saving mode. Panther Lake compiled Bonsai 4B's 26
+    # parts one after the other for 19 minutes on one of its 16 cores. ONW_COMPILE_JOBS=1: one at a time.
+    try:
+        import psutil
+        free = psutil.virtual_memory().available / 2**30
+    except Exception:
+        return 1
+    n = max(1, min(int((free - 3) / 2.5), (os.cpu_count() or 2) // 2, 6))
+    return min(n, 2) if mem_saver() else n
+
+
+def export_only():
+    """[3.2] A precompile child of the own-blob route: compile and write the blobs, import nothing (the parent does)."""
+    return bool(os.environ.get("ONW_PRECOMPILE_ONLY")) and os.environ.get("ONW_PRECOMPILE_EXPORT") == "1"
+
+
+class _Exported:
+    """What _compile returns in an export-only child: the graph's inputs / outputs, nothing to run."""
+    def __init__(self, model):
+        self.model_inputs, self.model_outputs = model.inputs, model.outputs
 
 
 def setup_mark(cache, config, tried_key):
@@ -1059,12 +1143,19 @@ class SegmentedModel:
         config = enable_turbo(core, device, config)        # (see turbo(): on unless ONW_TURBO=0)
         config = enable_compiler(core, device, config)     # [2.0.3] (see npu_compiler())
         t0 = time.time()
-        if os.path.exists(os.path.join(d, "experts.bin")):
+        child = bool(os.environ.get("ONW_PRECOMPILE_ONLY"))
+        if os.path.exists(os.path.join(d, "experts.bin")) and child:
+            # [3.2] a precompile child only compiles (it stops before binding anything): no copy of the expert bank in
+            # NPU-visible memory - six children holding Qwen3.6-35B-A3B's bank each hung the NPU driver (Panther Lake)
+            idx = json.load(open(os.path.join(d, "experts.bin.json")))
+            self.bank, self.E, self.K, self.streaming = _NoBank(), idx["E"], idx["K"], False
+        elif os.path.exists(os.path.join(d, "experts.bin")):
             self.bank, self.E, self.K = self._expert_bank(core, d, device, sizes)
         else:                                                   # dense model
             self.bank, self.E, self.K, self.streaming = {}, 0, 0, False
         sp = os.path.join(d, "shared.bin")
-        self.shared = load_shared(core, sp, device, self._shared_inputs(d)) if os.path.exists(sp) else {}
+        # (a precompile child: every shared tensor stays mapped from the file, none copied into NPU-visible memory)
+        self.shared = load_shared(core, sp, device, set() if child else self._shared_inputs(d)) if os.path.exists(sp)             else {}
         self.embed_rows = self._table("embed")
         self.window = self.meta.get("masks", {}).get("sliding_window")
         self.host_inputs = {}
@@ -1089,7 +1180,8 @@ class SegmentedModel:
         # INT4 weights of the plain paged models as well: never separated by the compiler (Granite 4.2 3B INT4 part:
         # 63.5 MB weightless against 63.6 MB, multi-row parts 20-35x slower to compile) - INT8 and NF4 are (llm-jp INT8
         # part: 2.5 against 216 MB). (NPUW and MoE models as they were checked: gemma-4, Qwen3.6)
-        _WEIGHTLESS["skip_model"] = self.meta.get("needs") == "u2" and os.environ.get("ONW_U2_WEIGHTLESS") != "1"
+        # [3.2] 2-bit weights weightless by default (onw's kit: compiler fixes; ONW_U2_WEIGHTLESS=0: full blobs)
+        _WEIGHTLESS["skip_model"] = self.meta.get("needs") == "u2" and os.environ.get("ONW_U2_WEIGHTLESS", "1") == "0"
         _WEIGHTLESS["full"] = False                    # (this model: a part with its weights inside, see fused.wanted)
         if device == "NPU" and not _WEIGHTLESS.get("told"):
             from . import npukit
@@ -1107,7 +1199,7 @@ class SegmentedModel:
             print(f"[npuw] off: ~{parts} compiled parts (each attention layer cuts a segment) - more than the NPU driver "
                   f"holds with NPUW (~30); ONW_NPUW=1 forces it", flush=True)
             npuw = False
-        if not npuw and not self.bank and int4_weights(d, self.meta) and os.environ.get("ONW_U2_WEIGHTLESS") != "1":
+        if not npuw and not self.bank and int4_weights(d, self.meta) and os.environ.get("ONW_INT4_WEIGHTLESS") != "1":
             _WEIGHTLESS["skip_model"] = True           # (decided here: NPUW may have been turned off just above)
         if npuw:
             # experimental: NPUW pulls the weights out of every compiled graph into one weights bank; constants match
@@ -1198,9 +1290,16 @@ class SegmentedModel:
                           and os.path.exists(os.path.join(d, m.get("bin") or n[:-4] + ".bin"))] or [0.0])
             npuw_check_runs(cache, graphs, covered, newest)
         jobs = compile_jobs(core, device, npuw) if cache and own and not (only or precompile) else 1
+        # [3.2] the own-blob route: its own mark (a load with these graphs went through: all blobs are there)
+        own_mark = loaded_mark(cache, config, tried.key, graphs) if cache and own and not npuw and device == "NPU" \
+            else None
         if jobs > 1 and mark and not covered:
             self._precompile(jobs, sorted(n for n in self.meta["segments"] if n[:-4] in graphs), todo)
             tried = self._ctx["tried"] = _Tried(config, core, device, npuw)    # (what the children recorded)
+        elif jobs > 1 and own_mark and not os.path.exists(own_mark):
+            self._precompile(jobs, sorted(n for n in self.meta["segments"] if n[:-4] in graphs), todo,
+                             export=True)
+            tried = self._ctx["tried"] = _Tried(config, core, device, npuw)
         for attempt in range(2):
             k, tc = 0, time.time()
             for name, m in sorted(self.meta["segments"].items()):
@@ -1275,6 +1374,8 @@ class SegmentedModel:
         runs = set(NPUW_SEEN.values())
         if mark and not (NPUW_SEEN and (len(runs) > 1 or None in runs)):   # (every graph is in the cache, one run)
             write_mark(mark, graphs)
+        if own_mark and not dropped:                   # (every blob is in the store: no precompile next time)
+            write_mark(own_mark, graphs)
         # the MTP layer (draft tokens, Qwen3.5 dense): compiled without NPUW, before the paged KV cache is set up
         # (its attention layer gets a cache of its own); mtp: wanted (chat.use_mtp)
         self.mtp, self.last_h = None, None
@@ -1330,7 +1431,12 @@ class SegmentedModel:
         # memory-saving mode (ONW_LAZY16=1): block sizes come and go, so the state lives in tensors of its own (else in
         # the smallest size's requests)
         self.swapping = bool(deferred)
-        mk = host_tensor(core, device) if self.swapping else None
+        # [3.2] the state in tensors of its own as well when the paged graphs may go for the fused ones
+        # (_drop_paged_for_fused): in the dropped requests' buffers the fused graphs wrote into freed memory (Panther
+        # Lake, Bonsai 2 9B: an access violation in the NPU driver at the first prompt block)
+        self._own_state = self.swapping or (self.paged and device == "NPU"
+                                            and os.environ.get("ONW_FUSED_DROP_PAGED", "1") == "1")
+        mk = host_tensor(core, device) if self._own_state else None
         for req, m, names in self.segs[min(self.segs)]:
             for n in names:
                 if n.startswith(("past_k.", "past_v.", "conv.", "ssm.")):
@@ -1400,8 +1506,10 @@ class SegmentedModel:
     def fused_keep_all(self):
         """[3.0] every fused variant stays loaded (onw's NPU kit, NPU 4000+, not the memory-saving mode)."""
         from . import npukit
+        # [3.2] not for 2-bit weights: every variant has its own weight tables (1 bit per weight: Bonsai 2 27B +3.4 GB
+        # a variant) - one variant at a time, swapped as the chat grows
         return os.environ.get("ONW_FUSED_KEEP", "1" if npukit.active() and npu_arch(self._ctx["core"]) >= 4000
-                              and not mem_saver() else "0") == "1"
+                              and not mem_saver() and self.meta.get("needs") != "u2" else "0") == "1"
 
     def fused_prepare(self, need, S=None):
         """Make the fused S-token graphs (default: the answer size) covering `need` positions the ones in use for that
@@ -1444,6 +1552,9 @@ class SegmentedModel:
                 return None
             F = next((keep[(S, B)] for B in bs if B >= want and (S, B) in keep), None)
             if F is None:
+                n = self._fused_jobs_parallel()
+                if n > 1 and not (self._ctx["config"] or {}).get("NPU_USE_NPUW") == "YES":
+                    self._fused_children([(S, want)], n)   # [3.2] (its graphs at the same time; the load imports them)
                 with self._mutex:
                     try:
                         F = keep[(S, want)] = _f.FusedDecode(self, S, want)
@@ -1452,6 +1563,7 @@ class SegmentedModel:
                         self._fused_sizes = [x for x in sizes if x != S]
                         return None
             self._fused[S] = F
+            self._drop_paged_for_fused(S)
             self.S = max([*self.segs, *self._fused])
             self._fused_background(caps)
             return F
@@ -1467,6 +1579,9 @@ class SegmentedModel:
             return None
         if F is not None and F.B >= want:
             return F
+        n = self._fused_jobs_parallel()                # [3.2] (its graphs compiled at the same time first)
+        if n > 1 and not (self._ctx["config"] or {}).get("NPU_USE_NPUW") == "YES":
+            self._fused_children([(S, want)], n)
         F = None                                       # [2.1.0] (this reference kept the old variant alive while the
         with self._mutex:                              # next one loaded: peak working set +3.3 GB, NPU 3720 Granite 3B)
             self._fused.pop(S, None)                   # (out first: never two copies of the weights)
@@ -1490,6 +1605,44 @@ class SegmentedModel:
         self._fused_background(caps)
         return self._fused[S]
 
+    def _drop_paged_for_fused(self, S, before=False):
+        """[3.2] Memory first: the fused S-token graphs run this size for every position they cover, so the paged
+        S-token graphs go (their weight tables are as large as the fused ones': Bonsai 4B +1.3 GB on the NPU with both;
+        the weights themselves are shared). They come back when a block needs them - images (explicit positions),
+        past the fused variants (_paged_back). Not with MTP (its verification blocks use them). ONW_FUSED_DROP_PAGED=0:
+        kept as before."""
+        if (os.environ.get("ONW_FUSED_DROP_PAGED", "1") != "1" or S not in self.segs or self.mtp is not None
+                or not self._own_state):
+            return
+        only = os.environ.get("ONW_FUSED_DROP_SIZES")      # (test: only these sizes, e.g. "64")
+        if only and str(S) not in only.split(","):
+            return
+        if before and not self._fused_ready(S):           # (no fused blobs yet: dropped once they are prepared)
+            return
+        with self._mutex:
+            self.drop_size(S, force=True)
+            self._wired = self._wire_for = None
+            self.__dict__.setdefault("_paged_dropped", set()).add(S)
+        self._trim()
+        print(f"[fused] the paged {S}-token graphs out (the fused ones run this size; back when a block needs them)",
+              flush=True)
+
+    def _fused_ready(self, S):
+        """[3.2] Whether the paged S-token graphs may go before the fused ones are prepared: always - the fused graphs
+        compile in child processes (or here, from the files) without them, and if they fail the paged ones come back
+        when a block needs them (_paged_back)."""
+        return S in (self._fused_sizes or [])
+
+    def _paged_back(self):
+        """[3.2] The paged graphs dropped for the fused ones, loaded again (a block the fused graphs cannot run)."""
+        dropped = self.__dict__.get("_paged_dropped")
+        if not dropped:
+            return
+        for S in sorted(dropped):
+            self.swap_to(S)
+        dropped.clear()
+        self._wired = self._wire_for = None
+
     def _fused_background(self, caps):
         npuw_ = (self._ctx["config"] or {}).get("NPU_USE_NPUW") == "YES"   # (NPUW: compiled when needed, no background)
         if os.environ.get("ONW_FUSED_BG", "1") != "0" and not self._fused_bg and not npuw_:
@@ -1509,14 +1662,17 @@ class SegmentedModel:
         env = os.environ.get("ONW_FUSED_JOBS")
         if env:
             return max(1, int(env))
-        if mem_saver() or self._ctx["device"] != "NPU" or os.environ.get("ONW_FUSED_JOBS_AUTO") != "1":
-            return 1                                   # (3.1.0: opt-in - ONW_FUSED_JOBS=n or ONW_FUSED_JOBS_AUTO=1)
+        # [3.2] on by default (the children only compile and write blobs; ONW_FUSED_JOBS_AUTO=0: one at a time here);
+        # 2 at most in the memory-saving mode
+        if self._ctx["device"] != "NPU" or os.environ.get("ONW_FUSED_JOBS_AUTO") == "0":
+            return 1
         try:
             import psutil
             free = psutil.virtual_memory().available / 2**30
         except Exception:
             return 1
-        return max(1, min(int((free - 4) / 3), (os.cpu_count() or 2) // 2, 6))
+        n = max(1, min(int((free - 4) / 3), (os.cpu_count() or 2) // 2, 6))
+        return min(n, 2) if mem_saver() else n
 
     def _fused_precompile(self, jobs):
         from . import fused as _f
@@ -1546,7 +1702,10 @@ class SegmentedModel:
         cfg = {k: v for k, v in (c["config"] or {}).items()}
         kt = self.kv.kt.get_type_name()
         env = dict(os.environ, ONW_WS_NO_CONCAT=os.environ.get("ONW_WS_NO_CONCAT", "1"))
-        todo = sorted(jobs, key=lambda j: (j[1], j[0]))
+        # [3.2] one child per graph (segment) of a variant, the longest first: a variant's graphs compile at the same
+        # time (Panther Lake, Bonsai 4B: the 64-token 4K variant's 5 graphs took 590 s one after another)
+        segs = {S_: [n_ for n_, sm in sorted(self.meta["segments"].items()) if sm["S"] == S_] for S_ in {j[0] for j in jobs}}
+        todo = [(S, B, n_) for S, B in sorted(jobs, key=lambda j: (j[1], j[0])) for n_ in segs[S]]
         running, ok = [], True
         while todo or running:
             for p in [p for p in running if p.poll() is not None]:
@@ -1555,11 +1714,11 @@ class SegmentedModel:
             if not ok:
                 break
             if todo and len(running) < n and (not running or self._fused_jobs_parallel() > len(running)):
-                S, B = todo.pop(0)
+                S, B, seg = todo.pop(0)
                 arg = json.dumps([c["d"], c["device"], cfg, c["tried"].key, c["built_T"], self.kv.capacity, kt, S, B],
                                  default=str)
                 flags = 0x00004000 if os.name == "nt" else 0      # (BELOW_NORMAL_PRIORITY_CLASS: answers go first)
-                running.append(subprocess.Popen([sys.executable, "-m", "onw.fusedjob", arg], env=env,
+                running.append(subprocess.Popen([sys.executable, "-m", "onw.fusedjob", arg], env={**env, "ONW_FUSED_SEG": seg},
                                                 stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL,
                                                 creationflags=flags))
                 continue
@@ -1568,7 +1727,7 @@ class SegmentedModel:
             p.wait()
         if not ok:
             print("[fused] a background compile process failed: the rest one after another here", flush=True)
-            for S, B in todo:
+            for S, B in sorted({(j[0], j[1]) for j in todo}):
                 try:
                     from . import fused as _f
                     _f.FusedDecode(self, S, B, compile_only=True, log=lambda *a, **k: None)
@@ -1733,12 +1892,17 @@ class SegmentedModel:
             lg[np.isin(self.mtp["vocab"], self.banned)] = -np.inf
         return int(self.mtp["vocab"][int(lg.argmax())])
 
-    def _precompile(self, jobs, names, todo):
+    def _precompile(self, jobs, names, todo, export=False):
         """Compile these segments into the cache in child processes, `jobs` at a time (see compile_jobs); the load
-        then imports them. A child that fails leaves its segment to the load (compiled there, as without this)."""
+        then imports them. A child that fails leaves its segment to the load (compiled there, as without this).
+        export: the children only write onw's blobs (export_only), the biggest segments first."""
         import subprocess, sys
         kw = {"creationflags": subprocess.CREATE_NO_WINDOW} if os.name == "nt" else {}
         env = {**os.environ, "PYTHONUTF8": "1", "ONW_NPUW_SESSION": NPUW_SESSION}
+        if export:
+            env["ONW_PRECOMPILE_EXPORT"] = "1"
+            d = self._ctx["d"]                         # (the longest compiles start first: less waiting at the end)
+            names = sorted(names, key=lambda n: -self.meta["segments"][n]["S"] * os.path.getsize(os.path.join(d, n)))
         args = json.dumps(self._args)
         print(f"[compile] {len(names)} segments, {jobs} processes at a time", flush=True)
         t0, waiting, running, done = time.time(), list(names), [], 0
@@ -1812,7 +1976,7 @@ class SegmentedModel:
         the expert bank; note which carried values each segment's successors still read."""
         for req, m, names in lst:
             for n in names:
-                if n in self._owner and (self.swapping or S != min(self.segs, default=S)):
+                if n in self._owner and (self._own_state or S != min(self.segs, default=S)):
                     req.set_tensor(n, self._owner[n])
                 elif n.startswith("shared.") and n[7:] in self.shared:
                     req.set_tensor(n, self.shared[n[7:]])
@@ -2013,6 +2177,8 @@ class SegmentedModel:
             [xml, size, S, "paged-1", part, ring, _signature(model), tried.key]
         if part is None and model.has_rt_info("onw_codes_const"):   # (a part's signature tells the forms apart)
             src.append("codes-const")
+        if part is None and model.has_rt_info("onw_leftmm"):
+            src.append("leftmm")
         # DeltaNet parts of the 4- and 64-token blocks (Qwen3.5): REP folds the repeated rows / chunks inside a layer
         # into graphs of undecided shape, which no NPU compiler takes (2-6 s lost per part, an empty cache folder
         # left); written so that REP takes them, they ran 6-12% slower than as one graph (Panther Lake) - so NONE first
@@ -2050,6 +2216,8 @@ class SegmentedModel:
                 except OSError:
                     pass
             if bpw and os.path.exists(bpw) and weightless_wanted():
+                if export_only():                      # [3.2] (a precompile child: already there)
+                    return _Exported(model)
                 try:
                     req = import_blob(core, bpw, device, {**nocache, "MODEL_PTR": model}).create_infer_request()
                     USED_BLOBS.add(bpw)
@@ -2065,6 +2233,8 @@ class SegmentedModel:
                     except OSError:
                         pass
             if own and os.path.exists(bp):
+                if export_only():
+                    return _Exported(model)
                 try:
                     req = import_blob(core, bp, device, nocache).create_infer_request()
                     USED_BLOBS.add(bp)
@@ -2101,6 +2271,12 @@ class SegmentedModel:
                             f.write(buf.getbuffer())
                         del buf
                         os.replace(f"{bpw}.{os.getpid()}.tmp", bpw)
+                    if export_only():                  # [3.2] the parent imports it
+                        _WEIGHTLESS["ok"] = True
+                        tried.set(gname, how)
+                        tried.save()
+                        print(f"[compile] {gname}: {how} (weightless, exported)", flush=True)
+                        return _Exported(model)
                     req = import_blob(core, bpw, device, {**nocache, "MODEL_PTR": model}).create_infer_request()
                     USED_BLOBS.add(bpw)
                     _WEIGHTLESS["ok"] = True
@@ -2131,6 +2307,10 @@ class SegmentedModel:
                     # child compiles without CACHE_DIR (no second copy in OpenVINO's cache), otherwise it keeps
                     # CACHE_DIR and the compile below is a cache hit
                     r = _probe(d, name, nocache if (own or npuw_dir) else cfg, npuw, T, export=bp, ring=ring)
+                    if r is True and own and os.path.exists(bp) and export_only():
+                        tried.set(gname, how)
+                        tried.save()
+                        return _Exported(model)
                     if r is True and own and os.path.exists(bp):
                         try:
                             req = import_blob(core, bp, device, nocache).create_infer_request()
@@ -2175,6 +2355,10 @@ class SegmentedModel:
                     # import what was just written and drop the compiled model: the first load then holds no more
                     # memory than later ones (a compiled model keeps its blob next to the driver's copy)
                     del cm
+                    if export_only():                  # [3.2] the parent imports it
+                        tried.set(gname, how)
+                        tried.save()
+                        return _Exported(model)
                     req = import_blob(core, bp, device, nocache).create_infer_request()
                 else:
                     req = cm.create_infer_request()
@@ -2559,6 +2743,7 @@ class SegmentedModel:
                     lg = F.step([int(t) for t in ids], want_logits, embeds)
                     if lg is not False:
                         return lg
+        self._paged_back()                             # [3.2] (dropped for the fused graphs: needed here)
         w = self._wired
         if self._wire_for is not (self.segs.get(min(self.segs)) if self.segs else None):   # (a size dropped / loaded since)
             w = self._wire(quiet=True)
@@ -2587,6 +2772,10 @@ class SegmentedModel:
         return logits
 
     def _prefill_layers(self, ids, embeds=None, pos=None, want_logits=True, stop=None):
+        self._paged_back()                             # [3.2] (dropped for the fused graphs: needed here)
+        return self._prefill_layers2(ids, embeds, pos, want_logits, stop)
+
+    def _prefill_layers2(self, ids, embeds=None, pos=None, want_logits=True, stop=None):
         """A whole prompt, layer by layer: every block goes through segment 0, then every block through segment 1,
         ... Each layer's state only depends on that layer's earlier blocks, so this is exact, and with experts
         streamed from the SSD each layer's experts are read once for the whole prompt instead of once per block.

onw/visionjob.py

@@ -0,0 +1,19 @@
+"""[3.2] A background child that compiles the vision encoder into onw's blob store at a first load (chat.export_vision_blob),
+so the first image only imports it. usage (internal): python -m onw.visionjob '[MODEL_DIR, DEVICE, CONFIG]'"""
+import json
+import sys
+
+from . import npukit
+
+
+def main():
+    d, device, cfg = json.loads(sys.argv[1])
+    from .chat import export_vision_blob
+    from .runtime import enable_compiler
+    core = npukit.core()
+    enable_compiler(core, device, {}, quiet=True)
+    export_vision_blob(core, d, device, cfg)
+
+
+if __name__ == "__main__":
+    main()

onw/wired.py

@@ -7,10 +7,12 @@ SegmentedModel._step (NPU 3720, Qwen3.6-35B-A3B: 146.5 -> 139.5 ms a token). ONW
 [2.0.4] Dense models too (paged attention), on their decode size: the 1-token graphs, or the 2-token ones that answer on
 NPU 4000+ since 2.0.3 (one real row, the second masked by token_valid). Granite 4.2 8B on Panther Lake: the stock path
 spent ~40 of 167 ms a token on the host."""
-import os
+import os, time
 import numpy as np
 from .store import host_tensor
 
+PROFILE = os.environ.get("ONW_PROFILE") == "1"   # [3.2] per-part NPU / attention / host times, printed every 64 tokens
+
 COMMON = ("inputs_embeds", "rope_cos", "rope_sin", "position_ids", "token_valid", "mask")
 
 
@@ -80,6 +82,11 @@ class Wired:
         m._tok += 1
         m._last = {"start": m.cur, "cx": {}, "rows": {}, "prev": {} if n > 1 else None, "pp": None}
         cur, later, logits = m.cur, None, None
+        if PROFILE:
+            t_step = time.perf_counter()
+            prof = getattr(self, "_prof", None)
+            if prof is None:
+                prof = self._prof = {"n": 0, "npu": [0.0] * len(self.plan), "attn": 0.0, "step": 0.0}
         for pi, st in enumerate(self.plan):
             req, meta = st["req"], st["meta"]
             if st["head"] and not want_logits:
@@ -101,11 +108,16 @@ class Wired:
                 bufs, c = m._pp[k]
                 req.set_tensor(k, bufs[c])
                 req.set_tensor("ssm_new." + k[4:], bufs[1 - c])
+            if PROFILE:
+                t0 = time.perf_counter()
             req.start_async()
             if later is not None:
                 later()
                 later = None
             req.wait()
+            if PROFILE:
+                t1 = time.perf_counter()
+                prof["npu"][pi] += t1 - t0
             L = st["kv"]
             if L is not None:                             # paged attention: rows into the cache, attention on the NPU
                 if st["kvw"]:                             # (a layer sharing another's cache only reads it)
@@ -113,6 +125,8 @@ class Wired:
                 q = req.get_tensor(f"q.{L}").data
                 self.parts[pi + 1][0].get_tensor(f"carry.attn.{L}").data[:] = m.kv.attend(
                     L, q[0], cur, S, n, sink=m._sink.get(L), rows_out=True)
+                if PROFILE:
+                    prof["attn"] += time.perf_counter() - t1
 
             def book(st=st, req=req):
                 for o in st["cx"]:
@@ -135,6 +149,16 @@ class Wired:
         if later is not None:
             later()
         m.cur += n
+        if PROFILE:
+            prof["step"] += time.perf_counter() - t_step
+            prof["n"] += 1
+            if prof["n"] % 64 == 0:
+                k = prof["n"]
+                npu = sum(prof["npu"])
+                print(f"[profile] {k} tokens: {prof['step'] / k * 1e3:.2f} ms a token = parts on the NPU "
+                      f"{npu / k * 1e3:.2f} + attention {prof['attn'] / k * 1e3:.2f} + host "
+                      f"{(prof['step'] - npu - prof['attn']) / k * 1e3:.2f}; parts: "
+                      + " ".join(f"{v / k * 1e3:.2f}" for v in prof["npu"]), flush=True)
         if logits is not None:
             if not np.isfinite(logits).all():
                 raise FloatingPointError(f"the {S}-token graph returned NaN/inf logits")

pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "onw"
-version = "3.1.0"
+version = "3.2.0"
 description = "Ore no NPU ga konna ni ugoku wake nai - LLMs entirely on the Intel NPU"
 readme = "README.md"
 license = {text = "Apache-2.0"}