onw 3.5.0: benchmark page (prefill + decode at 128-8K tokens) instead of the check page; prefill / decode speed in the chat header

CHANGELOG.md

@@ -5,6 +5,10 @@
 日付は日本時間です。各版は Hugging Face の `ryugyosoft/onw` にタグ（`v0.4.2` 以降）付きで残っています。
 onw ウィンドウは、新しい版があるとこのファイルから変更点を表示します（`## v…` の見出しで区切っています）。
 
+## v3.5.0 (2026-10-31)
+- 検証ページを **ベンチマーク**（`/bench`）に変えました。128・1K・2K・4K（8K も選べます）トークンのプロンプトで、プロンプトの読み込み（プリフィル）の速さと、そのあとの生成（デコード）の速さを表にします。プロンプトはそのモデル自身のトークナイザーで長さを合わせ、毎回違う一文で始めるので前の会話は再利用されません。「結果をコピー（Markdown）」で、アプリ・モデル・CPU・OS と一緒に表をコピーできます（記事などに）。トレイとウィンドウの「検証ページ」も「ベンチマーク」になり、前のアドレス（`/check`）はベンチマークへ移ります
+- チャット画面の上部に、直前の回答のプリフィル速度と生成速度を出すようにしました
+
 ## v3.4.0 (2026-10-30)
 - **API からモデルを切り替えられるように**：`/v1/models` にこのパソコンにあるモデルが全部出て、リクエストの `model` に別のモデルの ID を指定すると、今のモデルを閉じてそのモデルを読み込んでから答えます（同時に読み込むのは 1 つだけ）。知らない名前のときは今のモデルが答えます。設定「API からモデルを切り替えられるようにする」でオフにできます。
 - `/v1/models` にモデルの上限（`context_length`・`max_model_len`・`max_context_length`・`meta.n_ctx`、`max_output_tokens`）を入れました。

CHANGELOG_en.md

@@ -4,6 +4,10 @@
 
 Dates are JST. Every version is on Hugging Face `ryugyosoft/onw`, tagged from `v0.4.2` on.
 
+## v3.5.0 (2026-10-31)
+- The check page is now a **benchmark** (`/bench`): prompts of 128, 1K, 2K and 4K tokens (8K can be added), with how fast the prompt is read (prefill) and how fast the answer is then generated (decode). The prompts are cut to length by the model's own tokenizer and start with a different sentence every time, so nothing of an earlier conversation is reused. "Copy the results (Markdown)" copies the table with the app, model, CPU and OS (for articles and posts). The tray's and the window's "check page" entries open the benchmark, and the old address (`/check`) leads there
+- The chat page shows the last answer's prefill and decode speed at the top
+
 ## v3.4.0 (2026-10-30)
 - **Model choice from the API**: `/v1/models` lists every model on this PC, and a request whose `model` names another one closes the running model and loads that one first (one model at a time). A name the server does not know is answered by the running model. The setting "Let API requests switch models" turns it off.
 - `/v1/models` carries the model's limits (`context_length`, `max_model_len`, `max_context_length`, `meta.n_ctx`, `max_output_tokens`).

CONVERT_LFM.md

@@ -112,8 +112,8 @@ src=~/lfm-src/LFM2.5-2.6B
 
 ### 5. 動作を確かめる
 
-- 「検証ページ」を開いて「検証を実行」を押し、すべての項目が「OK」になることを確かめます。生成速度の目安は
-  上の表のとおりです（PC によって変わります）。
+- チャット画面で何か聞いて、答えが返ることを確かめます。速さは「ベンチマーク」を開いて「ベンチマークを実行」で測れます。
+  生成速度の目安は上の表のとおりです（PC によって変わります）。
 - LFM2.5 は、既定では考えずにすぐ答えます。考えさせたいときは、チャット画面の「思考モード」をオンにするか、API で
   `"chat_template_kwargs": {"enable_thinking": true}` を指定します。
 
@@ -144,8 +144,8 @@ src=~/lfm-src/LFM2.5-2.6B
 - 手順 1〜5 を上から順に、ユーザーの OS に合ったほう（Windows なら PowerShell、Ubuntu なら bash）だけ実行する。
   変数（`$py`、`$models`、`$src`）は同じシェルの中で使い続ける。
 - 手順 1 で onw の版が 0.9.0 未満（LFM2.5-24B-A2B では 3.3.0 未満）なら、先に進まずユーザーに更新を頼む。
-- 各手順の「成功の目安」（版の表示、`done in …s`、出力フォルダのファイル、検証ページの OK）を確かめてから次に進み、
+- 各手順の「成功の目安」（版の表示、`done in …s`、出力フォルダのファイル、チャットの答え）を確かめてから次に進み、
   失敗したら「うまくいかないとき」の表に沿って対処する。
 - モデルの取得（数 GB のダウンロード）の前と、手順 6 の削除の前には、ユーザーに確認する。
 - ライセンスの判断はユーザーに任せる（上の「ライセンスについて」を伝えるだけにする）。
-- 終わったら、変換先のフォルダ、変換にかかった時間、検証ページの生成速度をユーザーに報告する。
+- 終わったら、変換先のフォルダ、変換にかかった時間、ベンチマークの生成速度をユーザーに報告する。

CONVERT_LFM_en.md

@@ -117,8 +117,8 @@ folder holds `engine.json`, `seg00.bin`..., `seg00_S1.xml`..., `shared.bin`, `to
 
 ### 5. Check it
 
-- Open the "Check page" and press "Run checks": every row should say "OK". Decode speeds should be near the table
-  above (they vary by PC).
+- Ask something on the chat page and see that an answer comes back. "Run the benchmark" on the benchmark page
+  measures the speeds; decode should be near the table above (it varies by PC).
 - LFM2.5 answers without thinking by default. To let it think, switch on the chat page's thinking mode or pass
   `"chat_template_kwargs": {"enable_thinking": true}` in the API.
 
@@ -149,8 +149,8 @@ Rules for running these steps on the user's behalf:
 - Run steps 1-5 in order, only the variant for the user's OS (PowerShell on Windows, bash on Ubuntu), keeping the
   variables (`$py`, `$models`, `$src`) in the same shell.
 - If step 1 shows an onw version below 0.9.0 (3.3.0 for LFM2.5-24B-A2B), stop and ask the user to update first.
-- Check each step's success signs (the version, `done in ...s`, the files in the output folder, OK on the check page)
+- Check each step's success signs (the version, `done in ...s`, the files in the output folder, an answer on the chat page)
   before going on; on a failure follow the troubleshooting table.
 - Ask the user before downloading the model (several GB) and before deleting anything in step 6.
 - Leave the license decision to the user (just pass on "About the license").
-- When done, report the output folder, the conversion time and the decode speed from the check page.
+- When done, report the output folder, the conversion time and the decode speed from the benchmark page.

MANUAL.md

@@ -9,7 +9,7 @@
 - [「モデル」タブ：モデルを選んでダウンロード](#「モデル」タブ：モデルを選んでダウンロード)
 - [「サーバー」タブ：読み込んで使う](#「サーバー」タブ：読み込んで使う)
 - [チャット画面](#チャット画面)
-- [検証ページ](#検証ページ)
+- [ベンチマーク](#ベンチマーク)
 - [「設定」タブ](#「設定」タブ)
 - [他のアプリから使う（OpenAI 互換 API ほか）](#他のアプリから使う（openai-互換-api-ほか）)
 - [更新](#更新)
@@ -26,7 +26,7 @@
 | メニュー | 内容 |
 |---|---|
 | onw を開く（モデル・サーバー・設定） | onw ウィンドウを開きます（アイコンのダブルクリックでも開きます） |
-| チャットを開く／検証ページを開く | ブラウザでチャット画面・検証ページを開きます（モデルが動作中のとき） |
+| チャットを開く／ベンチマークを開く | ブラウザでチャット画面・ベンチマークを開きます（モデルが動作中のとき） |
 | OpenAI API のURLをコピー | 他のアプリに貼る base URL をコピーします |
 | モデルをロード／アンロード（サーバー停止） | 前回のモデルを読み込む／止めてメモリを空けます |
 | 終了（サーバーも停止） | onw ごと終わります |
@@ -75,7 +75,7 @@
 | 項目・ボタン | 内容 |
 |---|---|
 | モデル名・速度・メモリ | 動いているモデルと、その目安 |
-| チャット／検証ページ | ブラウザでチャット画面・検証ページを開きます |
+| チャット／ベンチマーク | ブラウザでチャット画面・ベンチマークを開きます |
 | ログ | サーバーのログ（読み込みやエラーの記録）を開きます |
 | アンロード | サーバーを止めてメモリを空けます |
 | 場所 | モデルのフォルダ |
@@ -124,12 +124,9 @@
 - 会話はこの PC の onw の設定フォルダ（Windows は `%APPDATA%\onw\chats`、Ubuntu は `~/.config/onw/chats`）にだけ保存されます。
   添付した画像は、縮小した写しが保存されます。
 
-## 検証ページ
+## ベンチマーク
 
-<img src="images/onw-check.png" width="760" alt="検証ページで検証を実行したところ">
-
-「検証を実行」で、テキスト・先読み検証・2 ターンの会話・画像（読めるモデルのとき）を順に試し、処理時間と生成速度を
-表にします。動作確認や、設定を変えたときの速さの比較に使えます。「繰り返し抑制」は検証のリクエストに付けて送る値です（Gemma では 1.2 以上を推奨）。
+「ベンチマークを実行」で、128・1K・2K・4K（8K も選べます）トークンのプロンプトを読ませ、プロンプトの読み込み（プリフィル）の速さと、そのあとの生成（デコード）の速さを表にします。プロンプトはこのモデル自身のトークナイザーで長さを合わせ、毎回違う一文で始めるので、前の会話は再利用されません。温度 0・思考オフで、「回数」を増やすと一番速かった回を出します。「結果をコピー（Markdown）」で、アプリ・モデル・CPU・OS と一緒に表をコピーできます（記事などに）。チャット画面の上部にも、直前の回答のプリフィル速度と生成速度が出ます。
 
 ## 「設定」タブ
 
@@ -256,7 +253,7 @@ Microsoft Store 版の Python で入れた場合、設定とモデルはその P
 |---|---|
 | 最初のロードがなかなか終わらない | 初回は NPU 向けのコンパイルに 10〜30 分かかります。「サーバー」タブの「NPU 向けの準備 k/n」が進んでいれば正常です |
 | メモリが足りない | 設定の「メモリ節約モード」（と「厳格」）、Qwen3.6 系なら「エキスパートのメモリ」を試してください。コンテキスト長を小さくしても、メモリはほとんど減りません |
-| 同じ言葉を何度も繰り返す（例：「訪れる訪れる訪れる…」） | 温度 0 では、一度繰り返しに入ると抜け出せないことがあります（特に Gemma。NPU の計算は CPU とわずかに違うため、CPU では出ない質問でも起きます）。チャット画面や検証ページの「繰り返し抑制」を 1.2 くらいにしてください。Gemma を読み込むと、横に赤字で推奨値が出ます。API では `repetition_penalty: 1.2` を付けます |
+| 同じ言葉を何度も繰り返す（例：「訪れる訪れる訪れる…」） | 温度 0 では、一度繰り返しに入ると抜け出せないことがあります（特に Gemma。NPU の計算は CPU とわずかに違うため、CPU では出ない質問でも起きます）。チャット画面の「繰り返し抑制」を 1.2 くらいにしてください。Gemma を読み込むと、横に赤字で推奨値が出ます。API では `repetition_penalty: 1.2` を付けます |
 | 回答が出ず、思考だけで終わる | 最大トークンが足りません。チャット画面の「最大トークン」を増やしてください（「常に考える」モデルは 2048 以上がおすすめ） |
 | 「タスクトレイの onw は … のまま動いています」と出る | トレイのメニューから「終了」して、onw を起動し直してください |
 | モデルの状態が「途中」 | ダウンロードが途中で止まっています。「削除」してからダウンロードし直してください |

MANUAL_en.md

@@ -9,7 +9,7 @@ How to install it and the list of models are in the [README](README_en.md).
 - [Models tab: pick and download a model](#models-tab-pick-and-download-a-model)
 - [Server tab: load and use it](#server-tab-load-and-use-it)
 - [The chat page](#the-chat-page)
-- [The check page](#the-check-page)
+- [The benchmark](#the-benchmark)
 - [Settings tab](#settings-tab)
 - [Use it from other apps (OpenAI-compatible API and more)](#use-it-from-other-apps-openai-compatible-api-and-more)
 - [Updates](#updates)
@@ -26,7 +26,7 @@ Right-click the onw icon in the tray for this menu:
 | Menu item | What it does |
 |---|---|
 | Open onw (models, server, settings) | Opens the onw window (double-clicking the icon does too) |
-| Open chat / Open check page | Opens the chat page or the check page in the browser (while a model is running) |
+| Open chat / Open the benchmark | Opens the chat page or the benchmark page in the browser (while a model is running) |
 | Copy the OpenAI API URL | Copies the base URL to paste into other apps |
 | Load model / Unload (stop the server) | Loads the last model / stops it to free the memory |
 | Quit (stops the server too) | Ends onw |
@@ -81,7 +81,7 @@ Once it is running, the top card shows:
 | Item / button | What it is |
 |---|---|
 | Model name, speed, memory | The running model and its estimates |
-| Chat / Check page | Opens the chat page or the check page in the browser |
+| Chat / Benchmark | Opens the chat page or the benchmark page in the browser |
 | Log | Opens the server log (loading, errors) |
 | Unload | Stops the server and frees the memory |
 | Location | The model's folder |
@@ -137,13 +137,9 @@ The left panel keeps your conversations (☰ shows or hides it).
 - They are kept only on this PC, in onw's settings folder (`%APPDATA%\onw\chats` on Windows, `~/.config/onw/chats` on
   Ubuntu). Attached images are kept as small copies.
 
-## The check page
+## The benchmark
 
-<img src="images/onw-check-en.png" width="760" alt="The check page after a run">
-
-"Run checks" tries text, prompt lookup, a two-turn conversation and an image (for models that read images) one after
-another and tables the processing times and decode speeds. Handy to see that everything works, or to compare speeds
-after changing a setting. "Repetition penalty" is sent with the check's requests (1.2 or more is recommended for Gemma).
+"Run the benchmark" has the model read prompts of 128, 1K, 2K and 4K tokens (8K can be added) and tables how fast it reads the prompt (prefill) and how fast it then generates (decode). The prompts are cut to length by the model's own tokenizer and start with a different sentence every time, so nothing of an earlier conversation is reused. Temperature 0, thinking off; with more runs the fastest one is shown. "Copy the results (Markdown)" copies the table with the app, model, CPU and OS (for articles and posts). The chat page shows the last answer's prefill and decode speed at the top too.
 
 ## Settings tab
 
@@ -269,7 +265,7 @@ When you install again, setup asks to install the python.org Python (the Store o
 |---|---|
 | The first load takes long | The first load compiles for the NPU, 10-30 min. If "Preparing for the NPU k/n" on the Server tab moves, all is well |
 | Not enough memory | Try the memory-saving mode (and Strict) in the settings, and for Qwen3.6 models the expert memory. A shorter context length saves almost nothing |
-| It repeats the same word over and over | At temperature 0 a repetition loop may never end (Gemma especially; the NPU computes slightly differently from the CPU, so it happens for prompts that are fine on the CPU). Set "Repetition penalty" on the chat page or the check page to about 1.2; with Gemma loaded, the recommended value shows next to it in red. In the API, add `repetition_penalty: 1.2` |
+| It repeats the same word over and over | At temperature 0 a repetition loop may never end (Gemma especially; the NPU computes slightly differently from the CPU, so it happens for prompts that are fine on the CPU). Set "Repetition penalty" on the chat page to about 1.2; with Gemma loaded, the recommended value shows next to it in red. In the API, add `repetition_penalty: 1.2` |
 | Only thinking, no answer | The max tokens ran out. Raise "Max tokens" in the chat (2048 or more for "always" thinking models) |
 | "…the onw in the task tray still runs …" | Quit from the tray menu and start onw again |
 | A model's status is "Incomplete" | The download stopped part-way. Delete it and download it again |

README.md

@@ -90,7 +90,7 @@ Python や git の知識は要りません。Python が入っていなければ
 
 <img src="images/onw-server.png" width="760" alt="onw ウィンドウの「サーバー」タブ（Gemma 4 E4B が動作中）">
 
-画面ごとの詳しい使い方（モデルの管理、チャット画面と会話の履歴、検証ページ、設定の各項目、困ったとき）は
+画面ごとの詳しい使い方（モデルの管理、チャット画面と会話の履歴、ベンチマーク、設定の各項目、困ったとき）は
 **[使い方マニュアル](MANUAL.md)** にあります。
 
 ### モデル一覧
@@ -180,7 +180,7 @@ Hugging Face から取得して、手元で変換します（onw だけで変換
 他のアプリからは base URL に `http://localhost:8000/v1`、モデル名にウィンドウに表示される名前を指定します。
 `GET /` でサーバーの案内（JSON）、`/health` でモデルの読み込み完了を確認できます。CORS は開放済みで、LAN に公開する
 場合は API キー（`--api-key` または設定画面）で `/v1` を保護できます。動作確認用に、チャット画面（`/chat`）と、
-テキスト・先読み検証・2 ターン会話・画像の検証を一括実行して速度を表にする検証ページ（`/check`）もあり、どちらも
+プロンプトの読み込み（プリフィル）と生成（デコード）の速さを 128〜8K トークンで測るベンチマーク（`/bench`）もあり、どちらも
 トレイのメニューとウィンドウから開けます。NPU ドライバーがない環境では CPU で動きます。
 
 **ツール呼び出し（function calling）**にも対応しています。OpenAI と同じく `tools` を渡すと、モデルが呼び出しを決めたときは
@@ -254,7 +254,7 @@ KV キャッシュは INT8 で、会話が実際に使った長さの分だけ
 | `onw/store.py`、`onw/xio.py` | エキスパートの保管（全部メモリ／SSD から読む LRU キャッシュ）、OS のキャッシュを通さない直接読み込み |
 | `onw/chat.py` | 生成ループ：画像（Qwen／Gemma）、サンプリング、停止文字列、先読み検証（正確な巻き戻し、再帰状態を持つモデルでは遅延やり直し、効く場合だけ自動で有効）、ターン間の差分処理 |
 | `onw/models/` | アーキテクチャごとの組み立て定義：`lfm2_moe`（dense の `lfm2` も）、`qwen3_5_moe`（dense と MoE）、`qwen_vision`、`gemma4`、`gpt_oss` |
-| `onw/hub.py`、`onw/server.py`、`onw/web/` | モデルのダウンロードと LFS 対策、OpenAI 互換サーバー、チャット画面（`/chat`）、検証ページ（`/check`） |
+| `onw/hub.py`、`onw/server.py`、`onw/web/` | モデルのダウンロードと LFS 対策、OpenAI 互換サーバー、チャット画面（`/chat`）、ベンチマーク（`/bench`） |
 | `onw/app.py`、`onw/tray.py`、`onw/manager.py`、`onw/update.py`、`onw/i18n.py` | onw ウィンドウ（モデル／サーバー／設定）、トレイアイコン、ウィンドウの裏側（ローカル API）、自動更新、表示言語 |
 | `setup.*`、`install.*`、`find_python.bat` | インストーラー（Python の検出と winget） |
 | `check_ref.py`、`check_vision.py`、`test_*.py`、`probe_segments.py`、`prof_*.py`、`mem_*.py`、`bench_*.py` | 元の HF モデルとの比較、生成・画像・複数ターンの検証、区間ごとの NPU と CPU の比較、速度・メモリ計測 |

README_en.md

@@ -77,7 +77,7 @@ Three steps in the **onw window**, which opens when the install is done:
 
 <img src="images/onw-server-en.png" width="760" alt="The Server tab of the onw window (Gemma 4 E4B running)">
 
-The full guide, screen by screen (managing models, the chat page and its history, the check page, every setting,
+The full guide, screen by screen (managing models, the chat page and its history, the benchmark, every setting,
 troubleshooting), is the **[user guide](MANUAL_en.md)**.
 
 ### Models
@@ -170,8 +170,8 @@ like). How to connect them: [user guide](MANUAL_en.md#use-it-from-other-apps-ope
 
 Point other apps at `http://localhost:8000/v1` with the model name shown in the window (`GET /` describes the
 server, `/health` answers once the model is loaded, CORS is open, `--api-key` / the settings' API key protects `/v1`).
-For trying things out it also serves a chat UI at `/chat` and a check page at `/check` that runs a text,
-prompt-lookup, two-turn and image test and shows the speeds; both open from the tray menu and the window. Without
+For trying things out it also serves a chat UI at `/chat` and a benchmark page at `/bench` that measures
+prompt reading (prefill) and generation (decode) speed at 128 to 8K tokens; both open from the tray menu and the window. Without
 an NPU driver everything runs on the CPU.
 
 **Tool calls (function calling)** work as with OpenAI: pass `tools`, and when the model decides to call one the reply
@@ -244,7 +244,7 @@ install it anywhere with `pip install -e . --extra-index-url https://storage.ope
 | `onw/store.py`, `onw/xio.py` | expert storage (all in memory / LRU cache streaming from the SSD), direct I/O that bypasses the OS cache |
 | `onw/chat.py` | chat loop: images (Qwen / Gemma), sampling, stop strings, prompt lookup decoding (exact rewind, or lazy redo for recurrent models, switched on only when it pays), prefix reuse between turns |
 | `onw/models/` | one builder per architecture: `lfm2_moe` (and dense `lfm2`), `qwen3_5_moe` (dense and MoE), `qwen_vision`, `gemma4`, `gpt_oss` |
-| `onw/hub.py`, `onw/server.py`, `onw/web/` | model download + LFS guard, OpenAI-compatible server, chat UI (`/chat`), check page (`/check`) |
+| `onw/hub.py`, `onw/server.py`, `onw/web/` | model download + LFS guard, OpenAI-compatible server, chat UI (`/chat`), benchmark page (`/bench`) |
 | `onw/app.py`, `onw/tray.py`, `onw/manager.py`, `onw/update.py`, `onw/i18n.py` | onw window (models / server / settings), tray icon, the window's backend (local API), self-update, UI languages |
 | `setup.*`, `install.*`, `find_python.bat` | installers (Python detection + winget) |
 | `check_ref.py`, `check_vision.py`, `test_*.py`, `probe_segments.py`, `prof_*.py`, `mem_*.py`, `bench_*.py` | vs the HF model, generation / image / multi-turn checks, NPU-vs-CPU per segment, timing and memory |

onw/__init__.py

@@ -2,4 +2,4 @@
 
 A small engine that runs LLMs entirely on the Intel NPU: static segment graphs + host-managed state.
 """
-__version__ = "3.4.0"
+__version__ = "3.5.0"

onw/i18n.json

@@ -388,7 +388,36 @@
 "ダウンロードを中止しますか？\nダウンロード済みの分は残り、次はその続きから取ります。": "Stop the download?\nWhat has been downloaded stays; the next download continues from there.",
 "エラー": "Error",
 "onw の状態を読めません: {0}": "Could not read onw's state: {0}",
-"Lunar Lake 以降専用の高速・省メモリ版": "Lunar Lake and later only: faster, less memory"
+"Lunar Lake 以降専用の高速・省メモリ版": "Lunar Lake and later only: faster, less memory",
+"ベンチマーク": "Benchmark",
+"このサーバーで、プロンプトの読み込み（プリフィル）と回答の生成（デコード）の速さを測ります。": "Measures how fast this server reads prompts (prefill) and writes answers (decode).",
+"プロンプトの長さ:": "Prompt length:",
+"生成するトークン数": "Tokens to generate",
+"回数": "Runs",
+"ベンチマークを実行": "Run the benchmark",
+"結果をコピー（Markdown）": "Copy the results (Markdown)",
+"プリフィル": "Prefill",
+"プリフィル速度": "Prefill speed",
+"生成": "Generated",
+"プリフィル＝プロンプトを読み終えるまで（最初のトークンが出るまで）の時間と、1秒あたりに読んだトークン数。生成＝そのあと回答を書く速さ（デコード）。温度 0、思考オフ。毎回、文頭に違う一文を入れて、前の会話を再利用しないようにしています。回数を増やすと、いちばん速かった回を表示します（最初に短い慣らし運転を1回します）。": "Prefill = the time until the prompt is read (until the first token) and the tokens read a second. Decode = how fast the answer is written after that. Temperature 0, thinking off. Every prompt starts with a different sentence, so nothing of an earlier one is reused. With more runs the fastest one is shown (one short warm-up run comes first).",
+"上の文章の要点を、できるだけ詳しく説明してください。": "Explain the main points of the text above in as much detail as you can.",
+"ベンチマーク {0}（{1}）": "Benchmark {0} ({1})",
+"{0}トークン": "{0} tokens",
+"慣らし運転…": "Warming up…",
+"コンテキスト長（{0}）を超えるので飛ばしました": "Skipped: longer than the context ({0})",
+"{0}（{1}/{2}回目）…": "{0} (run {1} of {2})…",
+"アプリ": "App",
+"CPU": "CPU",
+"OS": "OS",
+"条件": "Settings",
+"温度 0・思考オフ・生成 {0} トークン・{1} 回の最良値": "temperature 0, thinking off, {0} tokens generated, best of {1}",
+"測定日": "Date",
+"{0} ベンチマーク": "{0} benchmark",
+"先読み（MTP・ドラフト）": "Look-ahead (MTP / draft)",
+"プリフィル {0} tok/s": "Prefill {0} tok/s",
+"生成 {0} tok/s": "Decode {0} tok/s",
+"直前の回答：プロンプト {0} トークン（新しく読んだ分 {1}）を {2} 秒で読み、{3} トークンを生成": "Last answer: read a prompt of {0} tokens ({1} new) in {2} s, generated {3} tokens",
+"ベンチマークを開く": "Open the benchmark"
 },
 "zh": {
 "Lunar Lake 以降専用の高速版（重み NF4、アテンションの入力と down_proj は INT8。INT4 版より高精度）": "仅限 Lunar Lake 及更新机型的高速版（NF4 权重，注意力输入与 down_proj 为 INT8；比 INT4 版更精确）",
@@ -779,6 +808,35 @@
 "ダウンロードを中止しますか？\nダウンロード済みの分は残り、次はその続きから取ります。": "要中止下载吗？\n已下载的部分会保留，下次从中断处继续。",
 "エラー": "错误",
 "onw の状態を読めません: {0}": "无法读取 onw 的状态：{0}",
-"Lunar Lake 以降専用の高速・省メモリ版": "仅限 Lunar Lake 及以后的快速省内存版"
+"Lunar Lake 以降専用の高速・省メモリ版": "仅限 Lunar Lake 及以后的快速省内存版",
+"ベンチマーク": "基准测试",
+"このサーバーで、プロンプトの読み込み（プリフィル）と回答の生成（デコード）の速さを測ります。": "测量此服务器读取提示词（预填充）和生成回答（解码）的速度。",
+"プロンプトの長さ:": "提示词长度：",
+"生成するトークン数": "生成的令牌数",
+"回数": "次数",
+"ベンチマークを実行": "运行基准测试",
+"結果をコピー（Markdown）": "复制结果（Markdown）",
+"プリフィル": "预填充",
+"プリフィル速度": "预填充速度",
+"生成": "生成",
+"プリフィル＝プロンプトを読み終えるまで（最初のトークンが出るまで）の時間と、1秒あたりに読んだトークン数。生成＝そのあと回答を書く速さ（デコード）。温度 0、思考オフ。毎回、文頭に違う一文を入れて、前の会話を再利用しないようにしています。回数を増やすと、いちばん速かった回を表示します（最初に短い慣らし運転を1回します）。": "预填充 = 读完提示词（输出��一个令牌之前）的时间及每秒读取的令牌数。解码 = 之后写出回答的速度。温度 0，关闭思考。每次都以不同的句子开头，不会复用之前的对话。增加次数时显示最快的一次（开始前先进行一次简短的预热）。",
+"上の文章の要点を、できるだけ詳しく説明してください。": "请尽可能详细地说明上文的要点。",
+"ベンチマーク {0}（{1}）": "基准测试 {0}（{1}）",
+"{0}トークン": "{0} 个令牌",
+"慣らし運転…": "预热中…",
+"コンテキスト長（{0}）を超えるので飛ばしました": "已跳过：超过上下文长度（{0}）",
+"{0}（{1}/{2}回目）…": "{0}（第 {1}/{2} 次）…",
+"アプリ": "应用",
+"CPU": "CPU",
+"OS": "OS",
+"条件": "条件",
+"温度 0・思考オフ・生成 {0} トークン・{1} 回の最良値": "温度 0、关闭思考、生成 {0} 个令牌、{1} 次中的最佳值",
+"測定日": "测量日期",
+"{0} ベンチマーク": "{0} 基准测试",
+"先読み（MTP・ドラフト）": "预读（MTP / 草稿模型）",
+"プリフィル {0} tok/s": "预填充 {0} tok/s",
+"生成 {0} tok/s": "生成 {0} tok/s",
+"直前の回答：プロンプト {0} トークン（新しく読んだ分 {1}）を {2} 秒で読み、{3} トークンを生成": "上一个回答：用 {2} 秒读取了 {0} 个令牌的提示词（新读取 {1} 个），生成了 {3} 个令牌",
+"ベンチマークを開く": "打开基准测试"
+}
 }
-}

onw/manager.py

@@ -694,8 +694,8 @@ class Manager:
         elif what == "folder":
             p = app.installed(cfg).get(key) if key else None
             app.open_path(p or app.models_dir(cfg))
-        elif what in ("chat", "check"):
-            webbrowser.open(f"http://127.0.0.1:{app.server_port(cfg)}/{what}")
+        elif what in ("chat", "check", "bench"):        # [bench] ("check": the old name of the benchmark page)
+            webbrowser.open(f"http://127.0.0.1:{app.server_port(cfg)}/{'chat' if what == 'chat' else 'bench'}")
         return {}
 
 

onw/server.py

@@ -1,900 +1,975 @@
-"""OpenAI-compatible API server for an onw model on the Intel NPU (point other apps at http://HOST:PORT/v1).
-
-  GET  /                      what this server is: model, OpenAI base URL, endpoints (JSON)
-  GET  /v1/models             the loaded model with its limits: context_length (= max_model_len, max_context_length,
-                              meta.n_ctx) and max_output_tokens (what a request without max_tokens gets), "loaded":
-                              true; then the app's other models ("loaded": false). A request whose "model" names one
-                              of them loads it first (the running one is closed: one model in memory at a time); a
-                              name this server does not know is answered by the loaded model. ONW_API_SWITCH=0 (the
-                              app's setting off): only the loaded model is listed / used
-  GET  /v1/models/{id}
-  POST /v1/chat/completions   (stream true/false; text and image_url parts: data: URLs or http(s)); responses
-                              also carry "onw_stats" (prefill / decode speed, reused tokens) - extra field;
-                              "usage" (token counts) in a whole answer and on a stream's last chunk
-  GET  /health                200 when the model is loaded
-  GET  /chat                  browser chat UI for trying the model
-  GET  /check                 verification page: speed, image, multi-turn reuse
-  GET  /api/info              model and device details
-
-usage: onw serve MODEL [--port 8000] [--host 0.0.0.0] [--api-key KEY] [--device NPU|CPU] [--open]
-CORS is open, so browser-based clients work too; with --api-key every /v1 request needs "Authorization: Bearer KEY"
-(except from this PC's own programs and pages: the chat and check pages send no key).
-Thinking mode: "chat_template_kwargs": {"enable_thinking": true}, or "reasoning_effort": "low" | "medium" | "high"
-(gpt-oss uses the level; "none" / "minimal" turn thinking off). Whatever the model thought comes back as
-message.reasoning_content (delta.reasoning_content when streaming; also for models that think with thinking off, as vLLM's
-reasoning parsers do), the answer as content, and the thinking's token count as OpenAI's
-usage.completion_tokens_details.reasoning_tokens.
-Tool calls: "tools" (OpenAI format) -> message.tool_calls / finish_reason "tool_calls"; results as role "tool".
-tool_choice "none" hides the tools; a named function narrows them to that one ("required" cannot be enforced).
-response_format json_object / json_schema is best effort (an instruction is added; the output is not constrained).
-Not supported: n > 1 (400), logprobs (null).
-One NPU pipeline serves requests one at a time (a lock serializes them).
-"""
-import argparse, asyncio, base64, io, ipaddress, json, os, re, socket, threading, time, uuid, urllib.parse, urllib.request
-
-import uvicorn
-from fastapi import FastAPI, HTTPException, Request
-from fastapi.exceptions import RequestValidationError
-from fastapi.responses import HTMLResponse, JSONResponse, StreamingResponse
-from PIL import Image
-from onw import __version__, i18n
-from onw import tools as tool_calls
-
-MODEL_ID = "npu-model"      # set from the engine directory in main()
-app = FastAPI(title="LLM on Intel NPU")
-pipe = None                 # onw.chat.ChatEngine, created in main()
-info = {}
-lock = threading.Lock()     # one generation at a time on the NPU
-WEB = os.path.join(os.path.dirname(os.path.abspath(__file__)), "web")
-API_KEY = None              # --api-key: required as a Bearer token on /v1
-API_SAMPLING = None         # sampling for requests that send none (ONW_API_SAMPLING: "recommended" or a JSON of values)
-TOO_LONG = re.compile(r"the prompt is (\d+) tokens; the context holds (\d+)")   # (ChatEngine.stream_chat's ValueError)
-
-
-def oai_error(status, message, type_="invalid_request_error", code=None):
-    """An error in OpenAI's shape: {"error": {"message", "type", "code"}} (a prompt over the context: code
-    "context_length_exceeded", as OpenAI's)."""
-    code = code or ("context_length_exceeded" if TOO_LONG.match(message) else None)
-    return JSONResponse({"error": {"message": message, "type": type_, "param": None, "code": code}}, status_code=status)
-
-
-def anthropic_message(message):
-    """Anthropic's wording for a prompt over the context ("prompt is too long: N tokens > M maximum" - Claude Code
-    compacts the conversation on it); other messages as they are."""
-    m = TOO_LONG.match(message)
-    return f"prompt is too long: {m[1]} tokens > {m[2]} maximum" if m else message
-
-
-def api_error(request, status, message, type_="invalid_request_error", code=None):
-    """oai_error, or on the Anthropic Messages API (/v1/messages...) Anthropic's shape:
-    {"type": "error", "error": {"type", "message"}}."""
-    if request.url.path.startswith("/v1/messages"):
-        typ = {401: "authentication_error", 403: "permission_error", 404: "not_found_error",
-               413: "request_too_large"}.get(status, "api_error" if status >= 500 else "invalid_request_error")
-        return JSONResponse({"type": "error", "error": {"type": typ, "message": anthropic_message(message)}},
-                            status_code=status)
-    return oai_error(status, message, type_, code)
-
-
-@app.exception_handler(HTTPException)
-async def _http_error(request, e):
-    return api_error(request, e.status_code, str(e.detail),
-                     "invalid_request_error" if e.status_code < 500 else "server_error")
-
-
-@app.exception_handler(RequestValidationError)
-async def _validation_error(request, e):
-    return api_error(request, 400, str(e))
-
-
-@app.exception_handler(Exception)
-async def _server_error(request, e):
-    return api_error(request, 500, f"{type(e).__name__}: {e}", "server_error")
-
-
-CORS = [(b"access-control-allow-origin", b"*"), (b"access-control-allow-headers", b"*"),
-        (b"access-control-allow-methods", b"GET, POST, OPTIONS")]
-
-
-class CorsAndKey:
-    """Open CORS (browser-based OpenAI clients) and the optional API key on /v1 (not asked of this PC's own programs and
-    pages - the chat and check pages send none; the key keeps the LAN out). A plain ASGI middleware, not
-    @app.middleware("http"): behind that one (Starlette's BaseHTTPMiddleware) request.is_disconnected() never turns
-    true, and a client that left a whole (non-streamed) answer kept the NPU busy to max_tokens."""
-    def __init__(self, inner):
-        self.inner = inner
-
-    async def __call__(self, scope, receive, send):
-        if scope["type"] != "http":
-            return await self.inner(scope, receive, send)
-        request = Request(scope)                       # (headers and client only: the body stays with the app)
-        path = scope["path"]
-        if path.startswith("/api/chats"):              # saved conversations: this PC's own chat page only
-            why = _chats_refused(request)
-            if why:
-                return await JSONResponse({"error": why}, status_code=403)(scope, receive, send)   # (no CORS)
-            return await self.inner(scope, receive, send)
-
-        async def with_cors(msg):
-            if msg["type"] == "http.response.start":
-                msg = {**msg, "headers": [h for h in msg.get("headers", []) if h[0].lower() not in dict(CORS)] + CORS}
-            await send(msg)
-        if scope["method"] == "OPTIONS":
-            return await JSONResponse({})(scope, receive, with_cors)
-        if API_KEY and path.startswith("/v1") and not _local(request) and \
-                request.headers.get("authorization", "") != f"Bearer {API_KEY}" and \
-                request.headers.get("x-api-key", "") != API_KEY:            # (Anthropic clients send x-api-key)
-            return await api_error(request, 401, "invalid API key", code="invalid_api_key")(scope, receive, with_cors)
-        await self.inner(scope, receive, with_cors)
-
-
-app.add_middleware(CorsAndKey)
-
-
-# ---------------------------------------------------------------- saved conversations (the chat page's history)
-LOCAL_HOSTS = ("localhost", "127.0.0.1", "[::1]", "::1")
-CHAT_ID = __import__("re").compile(r"^[a-z0-9-]{8,48}$")
-CHAT_MAX = 32 << 20                                     # bytes per conversation (images are kept small by the page)
-
-
-def _chats_refused(request):
-    """Why a request for the saved conversations is refused, or None: only from this PC (also when the server listens on
-    the LAN), only under a localhost name (a rebound DNS name of some web site would otherwise pass as same-origin) and
-    only from the chat page itself (no cross-site reads or writes: these responses carry no CORS headers)."""
-    client = request.client.host if request.client else ""
-    try:
-        if not ipaddress.ip_address(client).is_loopback:
-            return "only from this PC"
-    except ValueError:
-        return "only from this PC"
-    host = urllib.parse.urlsplit("//" + request.headers.get("host", "")).hostname or ""
-    if host not in LOCAL_HOSTS:
-        return "only under localhost"
-    origin = request.headers.get("origin")
-    if origin and (urllib.parse.urlsplit(origin).hostname or "") not in LOCAL_HOSTS:
-        return "not from another site"
-    return None
-
-
-def _local(request):
-    """True for a request from this PC itself as _chats_refused judges it (this PC's address, a localhost name, no other
-    site's page) and not one a proxy forwards (with X-Forwarded-For from a proxy on this PC, uvicorn already puts the
-    real client's address in request.client; a bare "Forwarded" it does not read)."""
-    return _chats_refused(request) is None and "forwarded" not in request.headers and \
-        "x-forwarded-for" not in request.headers
-
-
-def chats_dir():
-    from onw import app as _app
-    d = os.path.join(_app.config_dir(), "chats")
-    os.makedirs(d, exist_ok=True)
-    return d
-
-
-def _chat_path(cid):
-    if not CHAT_ID.match(cid or ""):
-        raise HTTPException(400, "bad conversation id")
-    return os.path.join(chats_dir(), cid + ".json")
-
-
-@app.get("/api/chats")
-def chats_list():
-    out = []
-    for f in os.listdir(chats_dir()):
-        if not f.endswith(".json"):
-            continue
-        try:
-            c = json.load(open(os.path.join(chats_dir(), f), encoding="utf8"))
-            out.append({k: c.get(k) for k in ("id", "title", "model", "created", "updated")} | {"n": len(c.get("messages", []))})
-        except (OSError, ValueError):
-            continue
-    return sorted(out, key=lambda c: c.get("updated") or 0, reverse=True)
-
-
-@app.get("/api/chats/{cid}")
-def chat_get(cid: str):
-    try:
-        return json.load(open(_chat_path(cid), encoding="utf8"))
-    except FileNotFoundError:
-        raise HTTPException(404, "no such conversation")
-
-
-@app.put("/api/chats/{cid}")
-async def chat_put(cid: str, request: Request):
-    p = _chat_path(cid)
-    raw = await request.body()
-    if len(raw) > CHAT_MAX:
-        raise HTTPException(413, "conversation too large")
-    try:
-        c = json.loads(raw)
-        assert isinstance(c, dict) and isinstance(c.get("messages"), list)
-    except (ValueError, AssertionError):
-        raise HTTPException(400, "not a conversation")
-    c["id"] = cid
-    with open(p + ".tmp", "w", encoding="utf8") as f:
-        json.dump(c, f, ensure_ascii=False)
-    os.replace(p + ".tmp", p)
-    return {"ok": True}
-
-
-@app.delete("/api/chats/{cid}")
-def chat_delete(cid: str):
-    try:
-        os.remove(_chat_path(cid))
-    except FileNotFoundError:
-        pass
-    return {"ok": True}
-
-
-@app.delete("/api/chats")
-def chats_delete_all():
-    n = 0
-    for f in os.listdir(chats_dir()):
-        if f.endswith(".json"):
-            try:
-                os.remove(os.path.join(chats_dir(), f))
-                n += 1
-            except OSError:
-                pass
-    return {"ok": True, "deleted": n}
-
-
-def start_generation(msgs, kw):
-    """Run stream_chat in a worker thread; returns (asyncio queue of deltas/stats/exceptions, None at the end; stop
-    event). Called on the event loop: the worker hands its items over to it, so no thread waits per queued request."""
-    loop, q, stop = asyncio.get_running_loop(), asyncio.Queue(), threading.Event()
-
-    def put(x):
-        try:
-            loop.call_soon_threadsafe(q.put_nowait, x)
-        except RuntimeError:                        # the event loop is gone (the server is stopping)
-            stop.set()
-
-    def work():
-        try:
-            with lock:
-                if stop.is_set():                   # the client left while this request waited for the NPU
-                    return
-                set_priority(True)
-                try:
-                    for d in pipe.stream_chat(msgs, should_stop=stop.is_set, split_reasoning=True, **kw):
-                        put(d)
-                finally:
-                    set_priority(False)
-        except Exception as e:                      # surfaced to the client
-            put(e)
-        finally:
-            put(None)
-    threading.Thread(target=work, daemon=True).start()
-    return q, stop
-
-
-def reader(q, stop, request=None):
-    """-> async function returning start_generation's next item. With request (a whole response, not a stream): a
-    client that leaves meanwhile stops the generation (checked about once a second; a stream stops when it closes)."""
-    last = [time.monotonic()]
-
-    async def next_item():
-        while request is not None and not stop.is_set():
-            wait = last[0] + 1.0 - time.monotonic()
-            if wait <= 0:
-                last[0] = time.monotonic()
-                if await request.is_disconnected():
-                    stop.set()
-                continue
-            try:
-                return await asyncio.wait_for(q.get(), wait)
-            except asyncio.TimeoutError:
-                pass
-        return await q.get()
-    return next_item
-
-
-MAX_IMAGE = 20 << 20                                    # bytes
-
-
-def _public_host(url):
-    """True when every address the URL's host resolves to is a public one: the server does not fetch from the local
-    network on a client's behalf (ONW_ALLOW_LOCAL_IMAGES=1 allows it)."""
-    if os.environ.get("ONW_ALLOW_LOCAL_IMAGES") == "1":
-        return True
-    host = urllib.parse.urlsplit(url).hostname
-    try:
-        addrs = {a[4][0] for a in socket.getaddrinfo(host, None)}
-    except (socket.gaierror, UnicodeError, TypeError):
-        return False
-    return bool(addrs) and all(ipaddress.ip_address(a.split("%")[0]).is_global for a in addrs)
-
-
-def load_image(url):
-    try:
-        if url.startswith("data:"):
-            data = base64.b64decode(url.split(",", 1)[1])
-        elif url.startswith(("http://", "https://")):
-            if not _public_host(url):
-                raise HTTPException(400, "image_url: only public http(s) addresses are fetched (send a data: URL instead)")
-            req = urllib.request.Request(url, headers={"User-Agent": f"onw/{__version__}"})
-            with urllib.request.urlopen(req, timeout=20) as r:
-                data = r.read(MAX_IMAGE + 1)
-            if len(data) > MAX_IMAGE:
-                raise HTTPException(400, "image_url: the image is larger than 20 MB")
-        else:
-            raise HTTPException(400, "image_url must be a data: URL or http(s) URL")
-        return Image.open(io.BytesIO(data)).convert("RGB")
-    except HTTPException:
-        raise
-    except Exception as e:                                # bad base64, not an image, unreachable URL
-        raise HTTPException(400, f"image_url: {type(e).__name__}: {e}")
-
-
-def to_hf_messages(messages):
-    """OpenAI chat messages -> HF chat format with PIL images."""
-    out = []
-    for m in tool_calls.history(messages):
-        keep = {k: m[k] for k in ("tool_calls", "tool_call_id", "name", "reasoning_content") if m.get(k) is not None}
-        c = m.get("content")
-        if c is None or isinstance(c, str):
-            out.append({"role": m["role"], "content": c or "", **keep})
-            continue
-        parts = []
-        for p in c:
-            if p.get("type") == "text":
-                parts.append({"type": "text", "text": p.get("text") or ""})
-            elif p.get("type") == "image_url":
-                iu = p.get("image_url") or {}
-                url = iu.get("url") if isinstance(iu, dict) else iu
-                if not isinstance(url, str):
-                    raise HTTPException(400, "image_url part without a url")
-                parts.append({"type": "image", "image": load_image(url)})
-            else:
-                raise HTTPException(400, f"unsupported content part type: {p.get('type')}")
-        out.append({"role": m["role"], "content": parts, **keep})
-    return out
-
-
-def _num(body, key, default, cast=float):
-    v = body.get(key)
-    if v is None:
-        return default
-    try:
-        return cast(v)
-    except (TypeError, ValueError):
-        raise HTTPException(400, f"{key} must be a number")
-
-
-LEVELS = ("off", "low", "medium", "high")   # the thinking levels of the window and the chat page (reasoning_effort)
-_PREFS = [None, None]                       # (config mtime, this model's defaults)
-
-
-def model_defaults():
-    """This model's defaults from the onw window (config model_prefs, read again when the file changes - no restart):
-    {"think": level, "sampling": "greedy" | "recommended" | "custom", values...}; {} when none were saved. Settings
-    of earlier versions (ONW_THINK_DEFAULT, ONW_API_SAMPLING) apply where these say nothing."""
-    key = os.environ.get("ONW_MODEL_KEY")
-    if not key:
-        return {}
-    try:
-        from onw import app as _app
-        p = os.path.join(_app.config_dir(), "config.json")
-        mt = os.path.getmtime(p)
-        if _PREFS[0] != mt:
-            _PREFS[:] = [mt, (_app.load_config().get("model_prefs") or {}).get(key) or {}]
-        return _PREFS[1]
-    except (OSError, ValueError):
-        return {}
-
-
-def default_think():
-    """Thinking for a request that says nothing about it: the model's saved level, else the old switch."""
-    lvl = model_defaults().get("think")
-    if lvl == "off":
-        return False
-    if lvl in LEVELS:
-        return lvl
-    return True if os.environ.get("ONW_THINK_DEFAULT") == "1" else None
-
-
-def default_sampling():
-    """Sampling for a request that sends none: the model's saved choice, else the old setting (API_SAMPLING)."""
-    d = model_defaults()
-    mode = d.get("sampling")
-    if mode == "greedy":
-        return None
-    if mode == "recommended":
-        return {k: v for k, v in (info.get("sampling") or {}).items() if k != "source"} or None
-    if mode == "custom":
-        return {k: d[k] for k in ("temperature", "top_p", "top_k", "presence_penalty") if k in d}
-    return API_SAMPLING
-
-
-def gen_args(body):
-    """OpenAI request fields -> ChatEngine.stream_chat arguments."""
-    if _num(body, "n", 1, int) != 1:
-        raise HTTPException(400, "n > 1 is not supported (one NPU pipeline)")
-    stop = body.get("stop") or []
-    mx = body.get("max_completion_tokens", body.get("max_tokens"))
-    # a request without max_tokens: the setting (ONW_MAX_TOKENS; 0 = up to the end of the context - the engine stops
-    # there anyway); agents send none and write whole files in one answer
-    if mx is None:
-        mx = int(os.environ.get("ONW_MAX_TOKENS", "8192") or 0) or (1 << 30)
-    else:
-        mx = _num({"max_tokens": mx}, "max_tokens", 512, int)
-    if mx < 1:
-        raise HTTPException(400, "max_tokens must be at least 1")
-    think = (body.get("chat_template_kwargs") or {}).get("enable_thinking")
-    effort = body.get("reasoning_effort")
-    if isinstance(effort, str):                            # a level: low / medium / high (gpt-oss, llm-jp: trained;
-        think = False if effort in ("none", "minimal", "off") else effort   # others: a thinking budget)
-    if think is None:                                      # nothing said: the window's default for this model
-        think = default_think()
-    tools, choice = body.get("tools") or None, body.get("tool_choice")
-    if choice == "none":
-        tools = None
-    elif isinstance(choice, dict) and tools:              # a named function: offer just that one
-        name = (choice.get("function") or {}).get("name")
-        tools = [t for t in tools if (t.get("function") or {}).get("name") == name] or tools
-    # what the request leaves out: greedy, or the settings' API defaults (API_SAMPLING; thinking has its own set)
-    dflt = dict(default_sampling() or {})
-    if think and isinstance(dflt.get("think"), dict):
-        dflt.update(dflt["think"])
-    return dict(max_new_tokens=mx, temperature=_num(body, "temperature", dflt.get("temperature", 0.0)),
-                top_p=_num(body, "top_p", dflt.get("top_p", 1.0)),
-                stop=[stop] if isinstance(stop, str) else list(stop), seed=body.get("seed"), think=think,
-                top_k=_num(body, "top_k", dflt.get("top_k", 0), int), repetition_penalty=_num(body, "repetition_penalty", 1.0),
-                presence_penalty=_num(body, "presence_penalty", dflt.get("presence_penalty", 0.0)),
-                frequency_penalty=_num(body, "frequency_penalty", 0.0), tools=tools)
-
-
-def with_format(msgs, body):
-    """response_format json_object / json_schema: best effort - an instruction as a system message (the output is not
-    constrained)."""
-    rf = body.get("response_format") or {}
-    kind = rf.get("type") if isinstance(rf, dict) else None
-    if kind not in ("json_object", "json_schema"):
-        return msgs
-    note = "Respond with a single JSON object only, no other text."
-    if kind == "json_schema":
-        note += " It must follow this JSON schema: " + json.dumps((rf.get("json_schema") or {}).get("schema") or {},
-                                                                   ensure_ascii=False)
-    if msgs and msgs[0]["role"] == "system" and isinstance(msgs[0]["content"], str):
-        return [dict(msgs[0], content=msgs[0]["content"] + "\n\n" + note)] + msgs[1:]
-    return [{"role": "system", "content": note}] + msgs
-
-
-@app.get("/")
-def index(request: Request):
-    base = str(request.base_url).rstrip("/")
-    return {"object": "onw.server", "name": "onw - 俺のNPUがこんなに動くわけない", "version": __version__, "model": MODEL_ID,
-            "openai_base_url": base + "/v1", "api_key_required": bool(API_KEY),
-            "endpoints": {"models": "/v1/models", "chat_completions": "/v1/chat/completions", "health": "/health",
-                          "chat_ui": "/chat", "check_page": "/check", "info": "/api/info"},
-            "context": info.get("max_context"), "max_output_tokens": max_output_tokens(), "vision": info.get("vision")}
-
-
-@app.get("/health")
-def health():
-    if SWITCHING.is_set() or pipe is None:            # a model switch under way
-        return JSONResponse({"status": "loading", "model": MODEL_ID}, status_code=503)
-    return {"status": "ok", "model": MODEL_ID}
-
-
-@app.get("/chat")
-def chat_ui():
-    return HTMLResponse(i18n.page(os.path.join(WEB, "index.html")))
-
-
-@app.get("/check")
-def check_page():
-    return HTMLResponse(i18n.page(os.path.join(WEB, "check.html")))
-
-
-@app.get("/api/info")
-def api_info():
-    # the window's defaults for this model, read now (they change without a restart)
-    d, lvl = model_defaults(), default_think()
-    samp = default_sampling()
-    mode = d.get("sampling") or ("custom" if API_SAMPLING and os.environ.get("ONW_API_SAMPLING") != "recommended"
-                                 else "recommended" if API_SAMPLING else "greedy")
-    return {**info, "api_sampling": samp, "api_sampling_mode": mode,
-            "think_default": "off" if lvl is False else ("medium" if lvl is True else lvl)}
-
-
-STARTED = int(time.time())
-
-
-def max_output_tokens():
-    """The answer length a request without max_tokens gets (the setting; 0 = up to the context's end)."""
-    ctx = info.get("max_context") or 0
-    mx = int(os.environ.get("ONW_MAX_TOKENS", "8192") or 0)
-    return min(mx, ctx) if mx and ctx else (mx or ctx or None)
-
-
-def _model_card(mid=None):
-    # the limits under the names clients read them from: context_length (OpenRouter / Open WebUI), max_model_len
-    # (vLLM), max_context_length (LM Studio), meta.n_ctx (llama.cpp's server). Another installed model (mid): the
-    # context setting it would load with (at most its own maximum, which shows once it is loaded).
-    loaded = mid is None or mid == MODEL_ID
-    ctx = info.get("max_context") if loaded else SERVE.get("context")
-    mx = int(os.environ.get("ONW_MAX_TOKENS", "8192") or 0)
-    out = max_output_tokens() if loaded else (min(mx, ctx) if mx and ctx else (mx or ctx or None))
-    return {"id": MODEL_ID if loaded else mid, "object": "model", "created": STARTED, "owned_by": "local-npu",
-            "loaded": loaded, "context_length": ctx, "max_model_len": ctx, "max_context_length": ctx,
-            "max_output_tokens": out, "meta": {"n_ctx": ctx}}
-
-
-@app.get("/v1/models")
-def models():
-    """The loaded model first, then (when switching is allowed) every other model the app has: a request naming one
-    loads it."""
-    others = [m for m in installed_models() if m != MODEL_ID] if switch_allowed() else []
-    return {"object": "list", "data": [_model_card()] + [_model_card(m) for m in others]}
-
-
-@app.get("/v1/models/{model_id:path}")
-def model(model_id: str):
-    if model_id == MODEL_ID:
-        return _model_card()
-    hit = next((m for m, (k, _) in installed_models().items() if model_id in (m, k)), None) if switch_allowed() else None
-    if hit is None:
-        raise HTTPException(404, f"model {model_id!r} not found (this server runs {MODEL_ID!r})")
-    return _model_card(hit)
-
-
-def finalize(text, tools, finish):
-    """The answer's end: tool calls parsed out of the text (OpenAI tool_calls), finish_reason "tool_calls" when there
-    are any. -> (text, calls, finish_reason)."""
-    if not tools:
-        return text, [], finish
-    text, calls = tool_calls.parse(text, tools)
-    return text, calls, ("tool_calls" if calls else finish)
-
-
-def usage_of(st):
-    return {"prompt_tokens": st["prompt_tokens"], "completion_tokens": st["completion_tokens"],
-            "total_tokens": st["prompt_tokens"] + st["completion_tokens"],
-            "prompt_tokens_details": {"cached_tokens": st.get("cached_tokens", 0)},
-            # (OpenAI's field: how many of the completion tokens were thinking)
-            "completion_tokens_details": {"reasoning_tokens": st.get("reasoning_tokens", 0)}}
-
-
-def stats_of(st):
-    return {k: (round(v, 2) if isinstance(v, float) else v) for k, v in st.items() if k != "finish_reason"}
-
-
-def worker_error(e):
-    """A generation error -> (HTTP status, OpenAI error type): the request's fault (a prompt too long, an image for a
-    text-only model) or the server's (the NPU, the driver)."""
-    return (400, "invalid_request_error") if isinstance(e, (ValueError, KeyError, TypeError)) else (500, "server_error")
-
-
-@app.post("/v1/chat/completions")
-async def chat(request: Request):
-    try:
-        body = await request.json()
-    except ValueError:
-        raise HTTPException(400, "the request body is not valid JSON")
-    if not isinstance(body, dict) or not isinstance(body.get("messages"), list):
-        raise HTTPException(400, "messages (a list) is required")
-    await ensure_model(body.get("model"))               # another installed model named: load it
-    kw = gen_args(body)
-    loop = asyncio.get_running_loop()
-    try:                                                  # images may be downloaded: not on the event loop
-        msgs = await loop.run_in_executor(None, to_hf_messages, body["messages"])
-    except (KeyError, TypeError, AttributeError, ValueError) as e:
-        raise HTTPException(400, f"malformed messages: {e}")
-    msgs = with_format(msgs, body)
-    cid, created = f"chatcmpl-{uuid.uuid4().hex[:24]}", int(time.time())
-    tools = kw["tools"]
-    if body.get("stream"):
-        def chunk(delta, finish=None, **extra):
-            d = {"id": cid, "object": "chat.completion.chunk", "created": created, "model": MODEL_ID,
-                 "choices": [{"index": 0, "delta": delta, "logprobs": None, "finish_reason": finish}], **extra}
-            return f"data: {json.dumps(d, ensure_ascii=False)}\n\n"
-
-        async def events():
-            full, sent = "", 0                             # with tools: text is held back once a call may start
-            # (started here, not before the response: one whose stream never began would generate on to max_tokens)
-            q, stop = start_generation(msgs, kw)
-            next_item = reader(q, stop)
-            try:
-                yield chunk({"role": "assistant", "content": ""})
-                while True:
-                    d = await next_item()
-                    if d is None:
-                        break
-                    if isinstance(d, Exception):
-                        status, typ = worker_error(d)
-                        err = {"error": {"message": str(d), "type": typ, "param": None,
-                                         "code": "context_length_exceeded" if TOO_LONG.match(str(d)) else None}}
-                        yield f"data: {json.dumps(err, ensure_ascii=False)}\n\n"
-                    elif isinstance(d, dict) and "reasoning" in d:
-                        yield chunk({"reasoning_content": d["reasoning"]})
-                    elif isinstance(d, dict):
-                        rest, calls, finish = finalize(full, tools, d["finish_reason"])
-                        if tools and rest.startswith(full[:sent]) and rest[sent:].strip():   # text after the calls
-                            yield chunk({"content": rest[sent:]})
-                        for i, tc in enumerate(calls):
-                            yield chunk({"tool_calls": [dict(tc, index=i)]})
-                        # the token counts on the last chunk too (as llama.cpp's server does): clients that do not
-                        # send stream_options.include_usage still see them
-                        yield chunk({}, finish, usage=usage_of(d), onw_stats=stats_of(d))
-                        if (body.get("stream_options") or {}).get("include_usage"):   # OpenAI: its own last chunk
-                            yield f"data: {json.dumps({'id': cid, 'object': 'chat.completion.chunk', 'created': created, 'model': MODEL_ID, 'choices': [], 'usage': usage_of(d)})}\n\n"
-                    elif tools:
-                        full += d
-                        upto = tool_calls.safe_upto(full)
-                        if upto > sent:
-                            yield chunk({"content": full[sent:upto]})
-                            sent = upto
-                    else:
-                        yield chunk({"content": d})
-                yield "data: [DONE]\n\n"
-            finally:  # client disconnected or finished: stop the NPU loop so the lock is released
-                stop.set()
-        return StreamingResponse(events(), media_type="text/event-stream",
-                                 headers={"Cache-Control": "no-cache", "X-Accel-Buffering": "no"})
-
-    parts, thought, st = [], [], None
-    q, stop = start_generation(msgs, kw)
-    next_item = reader(q, stop, request)                  # (a client that leaves stops the generation)
-    try:
-        while True:
-            d = await next_item()
-            if d is None:
-                break
-            if isinstance(d, Exception):
-                status, typ = worker_error(d)
-                return oai_error(status, str(d), typ)
-            if isinstance(d, dict) and "reasoning" in d:
-                thought.append(d["reasoning"])
-            elif isinstance(d, dict):
-                st = d
-            else:
-                parts.append(d)
-    finally:
-        stop.set()
-    if st is None:
-        return oai_error(500, "the generation ended without a result", "server_error")
-    text, calls, finish = finalize("".join(parts), tools, st["finish_reason"])
-    msg = {"role": "assistant", "content": (text.strip() or None) if tools else text}
-    if thought:
-        msg["reasoning_content"] = "".join(thought)
-    if calls:
-        msg["tool_calls"] = calls
-    return JSONResponse({
-        "id": cid, "object": "chat.completion", "created": created, "model": MODEL_ID,
-        "choices": [{"index": 0, "message": msg, "logprobs": None, "finish_reason": finish}],
-        "usage": usage_of(st),
-        "onw_stats": stats_of(st),
-    })
-
-
-def recommended_sampling(arch, engine_dir=None):
-    """The chat page's starting sampling values ({temperature, top_p, top_k, presence_penalty?, think?, source}) or
-    None (greedy): the model's own generation_config.json first (Gemma 4, Granite 4.2), else onw/sampling.json by
-    model_type prefix. The API itself stays greedy unless asked."""
-    keys = ("temperature", "top_p", "top_k")
-    try:
-        with open(os.path.join(engine_dir, "generation_config.json"), encoding="utf8") as f:
-            g = json.load(f)
-        if g.get("do_sample") and any(k in g for k in keys):
-            return {"source": "model", "temperature": g.get("temperature", 1.0), "top_p": g.get("top_p", 1.0),
-                    "top_k": g.get("top_k", 0)}
-    except (OSError, ValueError, TypeError):
-        pass
-    try:
-        with open(os.path.join(WEB, "..", "sampling.json"), encoding="utf8") as f:
-            table = {k: v for k, v in json.load(f).items() if not k.startswith("_")}
-    except (OSError, ValueError):
-        return None
-    hit = max((k for k in table if arch.startswith(k)), key=len, default=None)
-    if hit is None:
-        return None
-    return {k: v for k, v in table[hit].items() if k != "note"}
-
-
-from onw import compat  # noqa: E402,F401  (/v1/messages and /v1/responses on this app)
-
-
-def set_priority(busy):
-    """Windows: the server's priority class - normal while loading and waiting, above normal while it generates
-    (busy). Windows puts a below-normal process on the low-power cores at low clocks (onw 1.4.5 and earlier started
-    the server below normal: Lunar Lake, Qwen3.5-9B NF4 with MTP ~12 tok/s, 16 above normal); generating takes one or two
-    cores, so the rest of the PC hardly notices. ONW_PRIORITY: the class while generating, normal | above (default) |
-    off (leave it as started)."""
-    want = os.environ.get("ONW_PRIORITY", "above")
-    if os.name != "nt" or want not in ("normal", "above"):
-        return
-    import ctypes
-    k32 = ctypes.WinDLL("kernel32")
-    k32.GetCurrentProcess.restype = ctypes.c_void_p
-    k32.SetPriorityClass.argtypes = [ctypes.c_void_p, ctypes.c_uint32]
-    k32.SetPriorityClass(k32.GetCurrentProcess(), 0x8000 if busy and want == "above" else 0x20)
-
-
-def load_model(path, name=None):
-    """Load a model as the one this server runs: pipe, MODEL_ID, info and the sampling defaults. main() and a
-    model switch (switch_model) both come here."""
-    global pipe, MODEL_ID, API_SAMPLING
-    from onw.chat import ChatEngine
-    pipe = ChatEngine(path, SERVE["device"], context=SERVE["context"], pld=SERVE["pld"])
-    MODEL_ID = name or os.path.basename(os.path.normpath(path))
-    v = pipe.vision
-    info.update(model=MODEL_ID, title=MODEL_ID, devices={"lm": SERVE["device"]}, fast=False, vision=bool(v),
-                image_px=v.get("image_px") if v else None,
-                think=pipe.think_style is not None, max_context=pipe.max_context,
-                always_thinks=pipe.think_style == "harmony",   # (gpt-oss, llm-jp: the thinking counts against max_tokens)
-                note=f"文脈は約{pipe.max_context}トークン", context=pipe.max_context, pld=pipe.pld,
-                mtp=getattr(pipe.model, "mtp", None) is not None)
-    try:
-        with open(os.path.join(path, "config.json"), encoding="utf8") as f:
-            arch = json.load(f).get("model_type") or ""
-    except (OSError, ValueError):
-        arch = ""
-    # greedy Gemma on the NPU can fall into repeating one word (fp16 on the NPU tips a near tie; the same happens with
-    # any onw version): the pages suggest a repetition penalty, the API default stays 1.0 (OpenAI behaviour)
-    info.update(arch=arch, rep_hint=1.2 if arch.startswith("gemma") else None, sampling=recommended_sampling(arch, path))
-    want = os.environ.get("ONW_API_SAMPLING")
-    API_SAMPLING = None
-    if want == "recommended":
-        API_SAMPLING = {k: v for k, v in (info["sampling"] or {}).items() if k != "source"} or None
-    elif want:
-        try:
-            API_SAMPLING = json.loads(want)
-        except ValueError:
-            API_SAMPLING = None
-    info.update(api_sampling=API_SAMPLING, api_sampling_mode="recommended" if want == "recommended" else
-                "custom" if API_SAMPLING else "greedy")
-    if API_SAMPLING:
-        print(f"[api] requests without sampling values use {API_SAMPLING}", flush=True)
-
-
-# ---------------------------------------------------------------- model choice from the API
-# A request's "model" naming another model the app has (its id in /v1/models, or the app's key) unloads the running one
-# and loads that one first (one model in memory at a time), as Ollama and LM Studio do. A name this server does not
-# know - clients send their own defaults ("gpt-4o", ...) - is served by the loaded model, as before. The app's setting
-# "api_model_switch" (ONW_API_SWITCH=0) turns switching off.
-MODEL_KEY = None            # the app's key of the loaded model (catalog repo id or folder name; ONW_MODEL_KEY)
-SERVE = {}                  # how main() loads models: device, context, pld, port
-SWITCHING = threading.Event()   # set while a switch unloads / loads (/health answers 503: the window shows loading)
-
-
-def switch_allowed():
-    return os.environ.get("ONW_API_SWITCH", "1") != "0"
-
-
-def installed_models():
-    """{API id: (app key, path)} of every model the app finds in its folders (the ids of /v1/models)."""
-    try:
-        from onw import app as A
-        found = A.installed(A.load_config())
-    except Exception:
-        return {}
-    out = {}
-    for key, path in found.items():
-        if path and os.path.isdir(path):
-            out.setdefault(key.split("/")[-1], (key, path))
-    return out
-
-
-def resolve_model(name):
-    """(app key, path) when a request's model names another installed model, else None (the loaded one serves)."""
-    if not name or not switch_allowed():
-        return None
-    n = str(name).strip().lower()
-    if n in ((MODEL_ID or "").lower(), (MODEL_KEY or "").lower()):
-        return None
-    for mid, (key, path) in installed_models().items():
-        if n in (mid.lower(), key.lower()):
-            return None if key == MODEL_KEY else (key, path)
-    return None
-
-
-def switch_model(key, path):
-    """Unload the running model and load `key` (called with `lock` held, so no generation runs). If the new one does
-    not load, the old one is loaded again and the error goes to the client."""
-    global pipe, MODEL_KEY
-    import gc
-    old_key = MODEL_KEY
-    old_path = installed_models().get((MODEL_KEY or "").split("/")[-1], (None, None))[1]
-    print(f"[api] switching model: {MODEL_ID} -> {key.split('/')[-1]}", flush=True)
-    SWITCHING.set()
-    t = time.time()
-    try:
-        old = pipe
-        pipe = None
-        close = getattr(getattr(old, "model", None), "close", None)
-        if close:
-            close()
-        del old
-        gc.collect()
-        try:
-            os.environ["ONW_MODEL_KEY"] = key
-            load_model(path, key.split("/")[-1])
-            MODEL_KEY = key
-        except Exception:
-            if old_path:                               # back to the model that ran
-                os.environ["ONW_MODEL_KEY"] = old_key or ""
-                load_model(old_path, (old_key or "").split("/")[-1] or None)
-            raise
-        print(f"[api] {MODEL_ID} loaded in {time.time() - t:.1f}s", flush=True)
-        _record_model(key, path)
-    finally:
-        SWITCHING.clear()
-
-
-async def ensure_model(name):
-    """Before a request's arguments are read (the sampling defaults and thinking style are the model's): the model it
-    names loaded, after any generation that runs now (the same lock). A failed load: 500 with why."""
-    if resolve_model(name) is None:
-        return
-
-    def work():
-        with lock:
-            want = resolve_model(name)                 # (another request may have switched to it meanwhile)
-            if want:
-                switch_model(*want)
-    try:
-        await asyncio.get_running_loop().run_in_executor(None, work)
-    except Exception as e:
-        raise HTTPException(500, f"model {name!r} could not be loaded: {type(e).__name__}: {e}")
-
-
-def _record_model(key, path):
-    """The window and the next start follow the switch: the server record (pid file) and the chosen model."""
-    try:
-        from onw import app as A
-        p = A.PIDFILE()
-        with open(p, encoding="utf8") as f:
-            d = json.load(f)
-        if int(d.get("port", -1)) == int(SERVE.get("port", -2)) and not d.get("stopping"):
-            d.update(model=key, path=path)
-            with open(p + ".tmp", "w") as f:
-                json.dump(d, f)
-            os.replace(p + ".tmp", p)
-        A.update_config(model=key)
-    except Exception as e:
-        print(f"[api] could not record the model switch: {e}", flush=True)
-
-
-def main(argv=None):
-    global API_KEY, MODEL_KEY
-    ap = argparse.ArgumentParser(prog="onw serve", description="chat UI + OpenAI-compatible API on the Intel NPU")
-    ap.add_argument("model", help="onw model folder, or a Hugging Face repo id (downloaded into ./models)")
-    ap.add_argument("--host", default="127.0.0.1")
-    ap.add_argument("--port", type=int, default=8000)
-    ap.add_argument("--device", default="NPU", help="run every component on this device (CPU works without an NPU driver)")
-    ap.add_argument("--name", help="model id / title (default: engine directory name)")
-    ap.add_argument("--context", type=int, default=None,
-                    help="context length in tokens (default 32768, at most the model's maximum); no recompile")
-    ap.add_argument("--open", action="store_true", help="open the chat UI in the default browser once loaded")
-    ap.add_argument("--no-pld", action="store_true", help="disable prompt lookup decoding")
-    ap.add_argument("--api-key", help="require this key as a Bearer token on /v1 from other machines "
-                                         "(e.g. with --host 0.0.0.0)")
-    a = ap.parse_args(argv)
-    API_KEY = a.api_key or os.environ.get("ONW_API_KEY") or None
-    from onw.hub import resolve
-    a.engine = resolve(a.model)
-    import openvino as ov
-    from . import npukit
-    from onw.runtime import npu_info
-    print(f"[onw] {__version__} | OpenVINO {ov.__version__} | {npu_info(npukit.core())}", flush=True)
-    if a.device == "NPU" and "NPU" not in npukit.core().available_devices:
-        print("WARNING: no NPU found (is the NPU driver installed?) - running everything on the CPU instead.", flush=True)
-        a.device = "CPU"
-    SERVE.update(device=a.device, context=a.context, pld=not a.no_pld, port=a.port)
-    MODEL_KEY = os.environ.get("ONW_MODEL_KEY") or a.name
-    load_model(a.engine, a.name)
-    set_priority(False)
-    if a.open:
-        import webbrowser
-        threading.Timer(1.5, webbrowser.open, [f"http://{'localhost' if a.host in ('0.0.0.0', '127.0.0.1') else a.host}:{a.port}/chat"]).start()
-    print(f"\n  OpenAI API:  http://localhost:{a.port}/v1   (model id: {MODEL_ID})"
-          f"\n  Chat UI:     http://localhost:{a.port}/chat\n  Check page:  http://localhost:{a.port}/check\n", flush=True)
-    uvicorn.run(app, host=a.host, port=a.port, log_level="warning")
-
-
-if __name__ == "__main__":
-    # (python -m onw.server: run the module that onw.compat put its routes on - this one is a second copy of it)
-    from onw.server import main as _main
-    _main()
+"""OpenAI-compatible API server for an onw model on the Intel NPU (point other apps at http://HOST:PORT/v1).
+
+  GET  /                      what this server is: model, OpenAI base URL, endpoints (JSON)
+  GET  /v1/models             the loaded model with its limits: context_length (= max_model_len, max_context_length,
+                              meta.n_ctx) and max_output_tokens (what a request without max_tokens gets), "loaded":
+                              true; then the app's other models ("loaded": false). A request whose "model" names one
+                              of them loads it first (the running one is closed: one model in memory at a time); a
+                              name this server does not know is answered by the loaded model. ONW_API_SWITCH=0 (the
+                              app's setting off): only the loaded model is listed / used
+  GET  /v1/models/{id}
+  POST /v1/chat/completions   (stream true/false; text and image_url parts: data: URLs or http(s)); responses
+                              also carry "onw_stats" (prefill / decode speed, reused tokens) - extra field;
+                              "usage" (token counts) in a whole answer and on a stream's last chunk
+  GET  /health                200 when the model is loaded
+  GET  /chat                  browser chat UI for trying the model
+  GET  /bench                 benchmark page: prompt reading (prefill) and generation (decode) speed at 128 to 8K
+                              tokens (/check, the old verification page, redirects here)
+  GET  /api/bench_text        ?tokens=N: prose of N tokens of this model's tokenizer (the benchmark's prompts)
+  GET  /api/info              model and device details
+
+usage: onw serve MODEL [--port 8000] [--host 0.0.0.0] [--api-key KEY] [--device NPU|CPU] [--open]
+CORS is open, so browser-based clients work too; with --api-key every /v1 request needs "Authorization: Bearer KEY"
+(except from this PC's own programs and pages: the chat and check pages send no key).
+Thinking mode: "chat_template_kwargs": {"enable_thinking": true}, or "reasoning_effort": "low" | "medium" | "high"
+(gpt-oss uses the level; "none" / "minimal" turn thinking off). Whatever the model thought comes back as
+message.reasoning_content (delta.reasoning_content when streaming; also for models that think with thinking off, as vLLM's
+reasoning parsers do), the answer as content, and the thinking's token count as OpenAI's
+usage.completion_tokens_details.reasoning_tokens.
+Tool calls: "tools" (OpenAI format) -> message.tool_calls / finish_reason "tool_calls"; results as role "tool".
+tool_choice "none" hides the tools; a named function narrows them to that one ("required" cannot be enforced).
+response_format json_object / json_schema is best effort (an instruction is added; the output is not constrained).
+Not supported: n > 1 (400), logprobs (null).
+One NPU pipeline serves requests one at a time (a lock serializes them).
+"""
+import argparse, asyncio, base64, io, ipaddress, json, os, re, socket, threading, time, uuid, urllib.parse, urllib.request
+
+import uvicorn
+from fastapi import FastAPI, HTTPException, Request
+from fastapi.exceptions import RequestValidationError
+from fastapi.responses import HTMLResponse, JSONResponse, RedirectResponse, StreamingResponse
+from PIL import Image
+from onw import __version__, i18n
+from onw import tools as tool_calls
+
+MODEL_ID = "npu-model"      # set from the engine directory in main()
+app = FastAPI(title="LLM on Intel NPU")
+pipe = None                 # onw.chat.ChatEngine, created in main()
+info = {}
+lock = threading.Lock()     # one generation at a time on the NPU
+WEB = os.path.join(os.path.dirname(os.path.abspath(__file__)), "web")
+API_KEY = None              # --api-key: required as a Bearer token on /v1
+API_SAMPLING = None         # sampling for requests that send none (ONW_API_SAMPLING: "recommended" or a JSON of values)
+TOO_LONG = re.compile(r"the prompt is (\d+) tokens; the context holds (\d+)")   # (ChatEngine.stream_chat's ValueError)
+
+
+def oai_error(status, message, type_="invalid_request_error", code=None):
+    """An error in OpenAI's shape: {"error": {"message", "type", "code"}} (a prompt over the context: code
+    "context_length_exceeded", as OpenAI's)."""
+    code = code or ("context_length_exceeded" if TOO_LONG.match(message) else None)
+    return JSONResponse({"error": {"message": message, "type": type_, "param": None, "code": code}}, status_code=status)
+
+
+def anthropic_message(message):
+    """Anthropic's wording for a prompt over the context ("prompt is too long: N tokens > M maximum" - Claude Code
+    compacts the conversation on it); other messages as they are."""
+    m = TOO_LONG.match(message)
+    return f"prompt is too long: {m[1]} tokens > {m[2]} maximum" if m else message
+
+
+def api_error(request, status, message, type_="invalid_request_error", code=None):
+    """oai_error, or on the Anthropic Messages API (/v1/messages...) Anthropic's shape:
+    {"type": "error", "error": {"type", "message"}}."""
+    if request.url.path.startswith("/v1/messages"):
+        typ = {401: "authentication_error", 403: "permission_error", 404: "not_found_error",
+               413: "request_too_large"}.get(status, "api_error" if status >= 500 else "invalid_request_error")
+        return JSONResponse({"type": "error", "error": {"type": typ, "message": anthropic_message(message)}},
+                            status_code=status)
+    return oai_error(status, message, type_, code)
+
+
+@app.exception_handler(HTTPException)
+async def _http_error(request, e):
+    return api_error(request, e.status_code, str(e.detail),
+                     "invalid_request_error" if e.status_code < 500 else "server_error")
+
+
+@app.exception_handler(RequestValidationError)
+async def _validation_error(request, e):
+    return api_error(request, 400, str(e))
+
+
+@app.exception_handler(Exception)
+async def _server_error(request, e):
+    return api_error(request, 500, f"{type(e).__name__}: {e}", "server_error")
+
+
+CORS = [(b"access-control-allow-origin", b"*"), (b"access-control-allow-headers", b"*"),
+        (b"access-control-allow-methods", b"GET, POST, OPTIONS")]
+
+
+class CorsAndKey:
+    """Open CORS (browser-based OpenAI clients) and the optional API key on /v1 (not asked of this PC's own programs and
+    pages - the chat and check pages send none; the key keeps the LAN out). A plain ASGI middleware, not
+    @app.middleware("http"): behind that one (Starlette's BaseHTTPMiddleware) request.is_disconnected() never turns
+    true, and a client that left a whole (non-streamed) answer kept the NPU busy to max_tokens."""
+    def __init__(self, inner):
+        self.inner = inner
+
+    async def __call__(self, scope, receive, send):
+        if scope["type"] != "http":
+            return await self.inner(scope, receive, send)
+        request = Request(scope)                       # (headers and client only: the body stays with the app)
+        path = scope["path"]
+        if path.startswith("/api/chats"):              # saved conversations: this PC's own chat page only
+            why = _chats_refused(request)
+            if why:
+                return await JSONResponse({"error": why}, status_code=403)(scope, receive, send)   # (no CORS)
+            return await self.inner(scope, receive, send)
+
+        async def with_cors(msg):
+            if msg["type"] == "http.response.start":
+                msg = {**msg, "headers": [h for h in msg.get("headers", []) if h[0].lower() not in dict(CORS)] + CORS}
+            await send(msg)
+        if scope["method"] == "OPTIONS":
+            return await JSONResponse({})(scope, receive, with_cors)
+        if API_KEY and path.startswith("/v1") and not _local(request) and \
+                request.headers.get("authorization", "") != f"Bearer {API_KEY}" and \
+                request.headers.get("x-api-key", "") != API_KEY:            # (Anthropic clients send x-api-key)
+            return await api_error(request, 401, "invalid API key", code="invalid_api_key")(scope, receive, with_cors)
+        await self.inner(scope, receive, with_cors)
+
+
+app.add_middleware(CorsAndKey)
+
+
+# ---------------------------------------------------------------- saved conversations (the chat page's history)
+LOCAL_HOSTS = ("localhost", "127.0.0.1", "[::1]", "::1")
+CHAT_ID = __import__("re").compile(r"^[a-z0-9-]{8,48}$")
+CHAT_MAX = 32 << 20                                     # bytes per conversation (images are kept small by the page)
+
+
+def _chats_refused(request):
+    """Why a request for the saved conversations is refused, or None: only from this PC (also when the server listens on
+    the LAN), only under a localhost name (a rebound DNS name of some web site would otherwise pass as same-origin) and
+    only from the chat page itself (no cross-site reads or writes: these responses carry no CORS headers)."""
+    client = request.client.host if request.client else ""
+    try:
+        if not ipaddress.ip_address(client).is_loopback:
+            return "only from this PC"
+    except ValueError:
+        return "only from this PC"
+    host = urllib.parse.urlsplit("//" + request.headers.get("host", "")).hostname or ""
+    if host not in LOCAL_HOSTS:
+        return "only under localhost"
+    origin = request.headers.get("origin")
+    if origin and (urllib.parse.urlsplit(origin).hostname or "") not in LOCAL_HOSTS:
+        return "not from another site"
+    return None
+
+
+def _local(request):
+    """True for a request from this PC itself as _chats_refused judges it (this PC's address, a localhost name, no other
+    site's page) and not one a proxy forwards (with X-Forwarded-For from a proxy on this PC, uvicorn already puts the
+    real client's address in request.client; a bare "Forwarded" it does not read)."""
+    return _chats_refused(request) is None and "forwarded" not in request.headers and \
+        "x-forwarded-for" not in request.headers
+
+
+def chats_dir():
+    from onw import app as _app
+    d = os.path.join(_app.config_dir(), "chats")
+    os.makedirs(d, exist_ok=True)
+    return d
+
+
+def _chat_path(cid):
+    if not CHAT_ID.match(cid or ""):
+        raise HTTPException(400, "bad conversation id")
+    return os.path.join(chats_dir(), cid + ".json")
+
+
+@app.get("/api/chats")
+def chats_list():
+    out = []
+    for f in os.listdir(chats_dir()):
+        if not f.endswith(".json"):
+            continue
+        try:
+            c = json.load(open(os.path.join(chats_dir(), f), encoding="utf8"))
+            out.append({k: c.get(k) for k in ("id", "title", "model", "created", "updated")} | {"n": len(c.get("messages", []))})
+        except (OSError, ValueError):
+            continue
+    return sorted(out, key=lambda c: c.get("updated") or 0, reverse=True)
+
+
+@app.get("/api/chats/{cid}")
+def chat_get(cid: str):
+    try:
+        return json.load(open(_chat_path(cid), encoding="utf8"))
+    except FileNotFoundError:
+        raise HTTPException(404, "no such conversation")
+
+
+@app.put("/api/chats/{cid}")
+async def chat_put(cid: str, request: Request):
+    p = _chat_path(cid)
+    raw = await request.body()
+    if len(raw) > CHAT_MAX:
+        raise HTTPException(413, "conversation too large")
+    try:
+        c = json.loads(raw)
+        assert isinstance(c, dict) and isinstance(c.get("messages"), list)
+    except (ValueError, AssertionError):
+        raise HTTPException(400, "not a conversation")
+    c["id"] = cid
+    with open(p + ".tmp", "w", encoding="utf8") as f:
+        json.dump(c, f, ensure_ascii=False)
+    os.replace(p + ".tmp", p)
+    return {"ok": True}
+
+
+@app.delete("/api/chats/{cid}")
+def chat_delete(cid: str):
+    try:
+        os.remove(_chat_path(cid))
+    except FileNotFoundError:
+        pass
+    return {"ok": True}
+
+
+@app.delete("/api/chats")
+def chats_delete_all():
+    n = 0
+    for f in os.listdir(chats_dir()):
+        if f.endswith(".json"):
+            try:
+                os.remove(os.path.join(chats_dir(), f))
+                n += 1
+            except OSError:
+                pass
+    return {"ok": True, "deleted": n}
+
+
+def start_generation(msgs, kw):
+    """Run stream_chat in a worker thread; returns (asyncio queue of deltas/stats/exceptions, None at the end; stop
+    event). Called on the event loop: the worker hands its items over to it, so no thread waits per queued request."""
+    loop, q, stop = asyncio.get_running_loop(), asyncio.Queue(), threading.Event()
+
+    def put(x):
+        try:
+            loop.call_soon_threadsafe(q.put_nowait, x)
+        except RuntimeError:                        # the event loop is gone (the server is stopping)
+            stop.set()
+
+    def work():
+        try:
+            with lock:
+                if stop.is_set():                   # the client left while this request waited for the NPU
+                    return
+                set_priority(True)
+                try:
+                    for d in pipe.stream_chat(msgs, should_stop=stop.is_set, split_reasoning=True, **kw):
+                        put(d)
+                finally:
+                    set_priority(False)
+        except Exception as e:                      # surfaced to the client
+            put(e)
+        finally:
+            put(None)
+    threading.Thread(target=work, daemon=True).start()
+    return q, stop
+
+
+def reader(q, stop, request=None):
+    """-> async function returning start_generation's next item. With request (a whole response, not a stream): a
+    client that leaves meanwhile stops the generation (checked about once a second; a stream stops when it closes)."""
+    last = [time.monotonic()]
+
+    async def next_item():
+        while request is not None and not stop.is_set():
+            wait = last[0] + 1.0 - time.monotonic()
+            if wait <= 0:
+                last[0] = time.monotonic()
+                if await request.is_disconnected():
+                    stop.set()
+                continue
+            try:
+                return await asyncio.wait_for(q.get(), wait)
+            except asyncio.TimeoutError:
+                pass
+        return await q.get()
+    return next_item
+
+
+MAX_IMAGE = 20 << 20                                    # bytes
+
+
+def _public_host(url):
+    """True when every address the URL's host resolves to is a public one: the server does not fetch from the local
+    network on a client's behalf (ONW_ALLOW_LOCAL_IMAGES=1 allows it)."""
+    if os.environ.get("ONW_ALLOW_LOCAL_IMAGES") == "1":
+        return True
+    host = urllib.parse.urlsplit(url).hostname
+    try:
+        addrs = {a[4][0] for a in socket.getaddrinfo(host, None)}
+    except (socket.gaierror, UnicodeError, TypeError):
+        return False
+    return bool(addrs) and all(ipaddress.ip_address(a.split("%")[0]).is_global for a in addrs)
+
+
+def load_image(url):
+    try:
+        if url.startswith("data:"):
+            data = base64.b64decode(url.split(",", 1)[1])
+        elif url.startswith(("http://", "https://")):
+            if not _public_host(url):
+                raise HTTPException(400, "image_url: only public http(s) addresses are fetched (send a data: URL instead)")
+            req = urllib.request.Request(url, headers={"User-Agent": f"onw/{__version__}"})
+            with urllib.request.urlopen(req, timeout=20) as r:
+                data = r.read(MAX_IMAGE + 1)
+            if len(data) > MAX_IMAGE:
+                raise HTTPException(400, "image_url: the image is larger than 20 MB")
+        else:
+            raise HTTPException(400, "image_url must be a data: URL or http(s) URL")
+        return Image.open(io.BytesIO(data)).convert("RGB")
+    except HTTPException:
+        raise
+    except Exception as e:                                # bad base64, not an image, unreachable URL
+        raise HTTPException(400, f"image_url: {type(e).__name__}: {e}")
+
+
+def to_hf_messages(messages):
+    """OpenAI chat messages -> HF chat format with PIL images."""
+    out = []
+    for m in tool_calls.history(messages):
+        keep = {k: m[k] for k in ("tool_calls", "tool_call_id", "name", "reasoning_content") if m.get(k) is not None}
+        c = m.get("content")
+        if c is None or isinstance(c, str):
+            out.append({"role": m["role"], "content": c or "", **keep})
+            continue
+        parts = []
+        for p in c:
+            if p.get("type") == "text":
+                parts.append({"type": "text", "text": p.get("text") or ""})
+            elif p.get("type") == "image_url":
+                iu = p.get("image_url") or {}
+                url = iu.get("url") if isinstance(iu, dict) else iu
+                if not isinstance(url, str):
+                    raise HTTPException(400, "image_url part without a url")
+                parts.append({"type": "image", "image": load_image(url)})
+            else:
+                raise HTTPException(400, f"unsupported content part type: {p.get('type')}")
+        out.append({"role": m["role"], "content": parts, **keep})
+    return out
+
+
+def _num(body, key, default, cast=float):
+    v = body.get(key)
+    if v is None:
+        return default
+    try:
+        return cast(v)
+    except (TypeError, ValueError):
+        raise HTTPException(400, f"{key} must be a number")
+
+
+LEVELS = ("off", "low", "medium", "high")   # the thinking levels of the window and the chat page (reasoning_effort)
+_PREFS = [None, None]                       # (config mtime, this model's defaults)
+
+
+def model_defaults():
+    """This model's defaults from the onw window (config model_prefs, read again when the file changes - no restart):
+    {"think": level, "sampling": "greedy" | "recommended" | "custom", values...}; {} when none were saved. Settings
+    of earlier versions (ONW_THINK_DEFAULT, ONW_API_SAMPLING) apply where these say nothing."""
+    key = os.environ.get("ONW_MODEL_KEY")
+    if not key:
+        return {}
+    try:
+        from onw import app as _app
+        p = os.path.join(_app.config_dir(), "config.json")
+        mt = os.path.getmtime(p)
+        if _PREFS[0] != mt:
+            _PREFS[:] = [mt, (_app.load_config().get("model_prefs") or {}).get(key) or {}]
+        return _PREFS[1]
+    except (OSError, ValueError):
+        return {}
+
+
+def default_think():
+    """Thinking for a request that says nothing about it: the model's saved level, else the old switch."""
+    lvl = model_defaults().get("think")
+    if lvl == "off":
+        return False
+    if lvl in LEVELS:
+        return lvl
+    return True if os.environ.get("ONW_THINK_DEFAULT") == "1" else None
+
+
+def default_sampling():
+    """Sampling for a request that sends none: the model's saved choice, else the old setting (API_SAMPLING)."""
+    d = model_defaults()
+    mode = d.get("sampling")
+    if mode == "greedy":
+        return None
+    if mode == "recommended":
+        return {k: v for k, v in (info.get("sampling") or {}).items() if k != "source"} or None
+    if mode == "custom":
+        return {k: d[k] for k in ("temperature", "top_p", "top_k", "presence_penalty") if k in d}
+    return API_SAMPLING
+
+
+def gen_args(body):
+    """OpenAI request fields -> ChatEngine.stream_chat arguments."""
+    if _num(body, "n", 1, int) != 1:
+        raise HTTPException(400, "n > 1 is not supported (one NPU pipeline)")
+    stop = body.get("stop") or []
+    mx = body.get("max_completion_tokens", body.get("max_tokens"))
+    # a request without max_tokens: the setting (ONW_MAX_TOKENS; 0 = up to the end of the context - the engine stops
+    # there anyway); agents send none and write whole files in one answer
+    if mx is None:
+        mx = int(os.environ.get("ONW_MAX_TOKENS", "8192") or 0) or (1 << 30)
+    else:
+        mx = _num({"max_tokens": mx}, "max_tokens", 512, int)
+    if mx < 1:
+        raise HTTPException(400, "max_tokens must be at least 1")
+    think = (body.get("chat_template_kwargs") or {}).get("enable_thinking")
+    effort = body.get("reasoning_effort")
+    if isinstance(effort, str):                            # a level: low / medium / high (gpt-oss, llm-jp: trained;
+        think = False if effort in ("none", "minimal", "off") else effort   # others: a thinking budget)
+    if think is None:                                      # nothing said: the window's default for this model
+        think = default_think()
+    tools, choice = body.get("tools") or None, body.get("tool_choice")
+    if choice == "none":
+        tools = None
+    elif isinstance(choice, dict) and tools:              # a named function: offer just that one
+        name = (choice.get("function") or {}).get("name")
+        tools = [t for t in tools if (t.get("function") or {}).get("name") == name] or tools
+    # what the request leaves out: greedy, or the settings' API defaults (API_SAMPLING; thinking has its own set)
+    dflt = dict(default_sampling() or {})
+    if think and isinstance(dflt.get("think"), dict):
+        dflt.update(dflt["think"])
+    return dict(max_new_tokens=mx, temperature=_num(body, "temperature", dflt.get("temperature", 0.0)),
+                top_p=_num(body, "top_p", dflt.get("top_p", 1.0)),
+                stop=[stop] if isinstance(stop, str) else list(stop), seed=body.get("seed"), think=think,
+                top_k=_num(body, "top_k", dflt.get("top_k", 0), int), repetition_penalty=_num(body, "repetition_penalty", 1.0),
+                presence_penalty=_num(body, "presence_penalty", dflt.get("presence_penalty", 0.0)),
+                frequency_penalty=_num(body, "frequency_penalty", 0.0), tools=tools)
+
+
+def with_format(msgs, body):
+    """response_format json_object / json_schema: best effort - an instruction as a system message (the output is not
+    constrained)."""
+    rf = body.get("response_format") or {}
+    kind = rf.get("type") if isinstance(rf, dict) else None
+    if kind not in ("json_object", "json_schema"):
+        return msgs
+    note = "Respond with a single JSON object only, no other text."
+    if kind == "json_schema":
+        note += " It must follow this JSON schema: " + json.dumps((rf.get("json_schema") or {}).get("schema") or {},
+                                                                   ensure_ascii=False)
+    if msgs and msgs[0]["role"] == "system" and isinstance(msgs[0]["content"], str):
+        return [dict(msgs[0], content=msgs[0]["content"] + "\n\n" + note)] + msgs[1:]
+    return [{"role": "system", "content": note}] + msgs
+
+
+@app.get("/")
+def index(request: Request):
+    base = str(request.base_url).rstrip("/")
+    return {"object": "onw.server", "name": "onw - 俺のNPUがこんなに動くわけない", "version": __version__, "model": MODEL_ID,
+            "openai_base_url": base + "/v1", "api_key_required": bool(API_KEY),
+            "endpoints": {"models": "/v1/models", "chat_completions": "/v1/chat/completions", "health": "/health",
+                          "chat_ui": "/chat", "bench_page": "/bench", "info": "/api/info"},
+            "context": info.get("max_context"), "max_output_tokens": max_output_tokens(), "vision": info.get("vision")}
+
+
+@app.get("/health")
+def health():
+    if SWITCHING.is_set() or pipe is None:            # a model switch under way
+        return JSONResponse({"status": "loading", "model": MODEL_ID}, status_code=503)
+    return {"status": "ok", "model": MODEL_ID}
+
+
+@app.get("/chat")
+def chat_ui():
+    return HTMLResponse(i18n.page(os.path.join(WEB, "index.html")))
+
+
+@app.get("/bench")
+def bench_page():
+    return HTMLResponse(i18n.page(os.path.join(WEB, "bench.html")))
+
+
+@app.get("/check")
+def check_page(request: Request):                     # [bench] the old verification page's address
+    return RedirectResponse("/bench" + ("?" + request.url.query if request.url.query else ""))
+
+
+# [bench] the benchmark page's prompts: ordinary prose, cut to an exact token count by the model's own tokenizer (so
+# "2K tokens" is 2K for this model); every section differs a little, which keeps look-ahead (MTP / PLD) honest
+BENCH_SENTENCES = [
+    "The harbor town woke slowly, its fishing boats rocking against the pier while gulls argued over the morning catch.",
+    "Engineers measured the bridge every spring, noting how the steel expanded in summer heat and contracted in winter.",
+    "A small library on the corner kept its lights on late, so students could read after their shifts at the factory.",
+    "Farmers in the valley rotated wheat, beans and clover, which kept the soil rich without much fertilizer.",
+    "The orchestra rehearsed the second movement twice, slowing the tempo until the strings and winds breathed together.",
+    "Old maps of the region show a river that has since moved two kilometers east, leaving a chain of quiet ponds.",
+    "In the laboratory, the team compared three batteries by charging them to full and draining them at a steady load.",
+    "Travelers crossing the pass in autumn carried extra blankets, because the weather could turn within an hour.",
+    "The bakery sold out of rye bread by noon on Saturdays, so regular customers learned to order it a day ahead.",
+    "Historians argue about why the trade route declined, pointing to new ships, higher taxes and a long drought.",
+    "A volunteer group repaired the hiking trail, replacing rotten steps and clearing the fallen trees after the storm.",
+    "The museum's new wing displays tools from daily life, from clay lamps to the first electric kettles.",
+]
+
+
+@app.get("/api/bench_text")
+def bench_text(tokens: int = 2048):
+    if pipe is None or not hasattr(pipe, "tok"):
+        raise HTTPException(503, "the model is not loaded")
+    n = max(8, min(int(tokens), 65536))
+    parts, k = [], 0
+    while True:
+        sec = " ".join(BENCH_SENTENCES[(k + i) % len(BENCH_SENTENCES)] for i in range(5))
+        parts.append(f"Section {k + 1}. {sec}")
+        k += 1
+        if k % 8 == 0 and len(pipe.tok("\n\n".join(parts), add_special_tokens=False)["input_ids"]) >= n:
+            break
+    ids = pipe.tok("\n\n".join(parts), add_special_tokens=False)["input_ids"][:n]
+    return {"text": pipe.tok.decode(ids, skip_special_tokens=True), "tokens": len(ids)}
+
+
+@app.get("/api/info")
+def api_info():
+    # the window's defaults for this model, read now (they change without a restart)
+    d, lvl = model_defaults(), default_think()
+    samp = default_sampling()
+    mode = d.get("sampling") or ("custom" if API_SAMPLING and os.environ.get("ONW_API_SAMPLING") != "recommended"
+                                 else "recommended" if API_SAMPLING else "greedy")
+    return {**info, "api_sampling": samp, "api_sampling_mode": mode,
+            "think_default": "off" if lvl is False else ("medium" if lvl is True else lvl),
+            "app": APP, "cpu": cpu_name(), "os": os_name()}   # [bench] for the benchmark page's report
+
+
+APP = f"{getattr(__import__(__package__ or 'onwx'), 'NAME', __package__ or 'onwx')} {__version__}"
+
+
+def cpu_name():
+    """[bench] the processor's marketing name ("AMD Ryzen AI Max+ 395 w/ Radeon 8060S"), "" if unknown."""
+    try:
+        if os.name == "nt":
+            import winreg
+            with winreg.OpenKey(winreg.HKEY_LOCAL_MACHINE, r"HARDWARE\DESCRIPTION\System\CentralProcessor\0") as k:
+                return str(winreg.QueryValueEx(k, "ProcessorNameString")[0]).strip()
+        for line in open("/proc/cpuinfo", encoding="utf8", errors="replace"):
+            if line.startswith("model name"):
+                return line.split(":", 1)[1].strip()
+    except Exception:
+        pass
+    return ""
+
+
+def os_name():
+    """[bench] "Windows 11 (26200)" / "Ubuntu 24.04.3 LTS" ..."""
+    import platform
+    try:
+        if os.name == "nt":
+            build = int(platform.version().split(".")[-1])
+            return f"Windows {'11' if build >= 22000 else '10'} ({build})"
+        for line in open("/etc/os-release", encoding="utf8"):
+            if line.startswith("PRETTY_NAME="):
+                return line.split("=", 1)[1].strip().strip('"')
+    except Exception:
+        pass
+    return platform.platform(terse=True)
+
+
+STARTED = int(time.time())
+
+
+def max_output_tokens():
+    """The answer length a request without max_tokens gets (the setting; 0 = up to the context's end)."""
+    ctx = info.get("max_context") or 0
+    mx = int(os.environ.get("ONW_MAX_TOKENS", "8192") or 0)
+    return min(mx, ctx) if mx and ctx else (mx or ctx or None)
+
+
+def _model_card(mid=None):
+    # the limits under the names clients read them from: context_length (OpenRouter / Open WebUI), max_model_len
+    # (vLLM), max_context_length (LM Studio), meta.n_ctx (llama.cpp's server). Another installed model (mid): the
+    # context setting it would load with (at most its own maximum, which shows once it is loaded).
+    loaded = mid is None or mid == MODEL_ID
+    ctx = info.get("max_context") if loaded else SERVE.get("context")
+    mx = int(os.environ.get("ONW_MAX_TOKENS", "8192") or 0)
+    out = max_output_tokens() if loaded else (min(mx, ctx) if mx and ctx else (mx or ctx or None))
+    return {"id": MODEL_ID if loaded else mid, "object": "model", "created": STARTED, "owned_by": "local-npu",
+            "loaded": loaded, "context_length": ctx, "max_model_len": ctx, "max_context_length": ctx,
+            "max_output_tokens": out, "meta": {"n_ctx": ctx}}
+
+
+@app.get("/v1/models")
+def models():
+    """The loaded model first, then (when switching is allowed) every other model the app has: a request naming one
+    loads it."""
+    others = [m for m in installed_models() if m != MODEL_ID] if switch_allowed() else []
+    return {"object": "list", "data": [_model_card()] + [_model_card(m) for m in others]}
+
+
+@app.get("/v1/models/{model_id:path}")
+def model(model_id: str):
+    if model_id == MODEL_ID:
+        return _model_card()
+    hit = next((m for m, (k, _) in installed_models().items() if model_id in (m, k)), None) if switch_allowed() else None
+    if hit is None:
+        raise HTTPException(404, f"model {model_id!r} not found (this server runs {MODEL_ID!r})")
+    return _model_card(hit)
+
+
+def finalize(text, tools, finish):
+    """The answer's end: tool calls parsed out of the text (OpenAI tool_calls), finish_reason "tool_calls" when there
+    are any. -> (text, calls, finish_reason)."""
+    if not tools:
+        return text, [], finish
+    text, calls = tool_calls.parse(text, tools)
+    return text, calls, ("tool_calls" if calls else finish)
+
+
+def usage_of(st):
+    return {"prompt_tokens": st["prompt_tokens"], "completion_tokens": st["completion_tokens"],
+            "total_tokens": st["prompt_tokens"] + st["completion_tokens"],
+            "prompt_tokens_details": {"cached_tokens": st.get("cached_tokens", 0)},
+            # (OpenAI's field: how many of the completion tokens were thinking)
+            "completion_tokens_details": {"reasoning_tokens": st.get("reasoning_tokens", 0)}}
+
+
+def stats_of(st):
+    return {k: (round(v, 2) if isinstance(v, float) else v) for k, v in st.items() if k != "finish_reason"}
+
+
+def worker_error(e):
+    """A generation error -> (HTTP status, OpenAI error type): the request's fault (a prompt too long, an image for a
+    text-only model) or the server's (the NPU, the driver)."""
+    return (400, "invalid_request_error") if isinstance(e, (ValueError, KeyError, TypeError)) else (500, "server_error")
+
+
+@app.post("/v1/chat/completions")
+async def chat(request: Request):
+    try:
+        body = await request.json()
+    except ValueError:
+        raise HTTPException(400, "the request body is not valid JSON")
+    if not isinstance(body, dict) or not isinstance(body.get("messages"), list):
+        raise HTTPException(400, "messages (a list) is required")
+    await ensure_model(body.get("model"))               # another installed model named: load it
+    kw = gen_args(body)
+    loop = asyncio.get_running_loop()
+    try:                                                  # images may be downloaded: not on the event loop
+        msgs = await loop.run_in_executor(None, to_hf_messages, body["messages"])
+    except (KeyError, TypeError, AttributeError, ValueError) as e:
+        raise HTTPException(400, f"malformed messages: {e}")
+    msgs = with_format(msgs, body)
+    cid, created = f"chatcmpl-{uuid.uuid4().hex[:24]}", int(time.time())
+    tools = kw["tools"]
+    if body.get("stream"):
+        def chunk(delta, finish=None, **extra):
+            d = {"id": cid, "object": "chat.completion.chunk", "created": created, "model": MODEL_ID,
+                 "choices": [{"index": 0, "delta": delta, "logprobs": None, "finish_reason": finish}], **extra}
+            return f"data: {json.dumps(d, ensure_ascii=False)}\n\n"
+
+        async def events():
+            full, sent = "", 0                             # with tools: text is held back once a call may start
+            # (started here, not before the response: one whose stream never began would generate on to max_tokens)
+            q, stop = start_generation(msgs, kw)
+            next_item = reader(q, stop)
+            try:
+                yield chunk({"role": "assistant", "content": ""})
+                while True:
+                    d = await next_item()
+                    if d is None:
+                        break
+                    if isinstance(d, Exception):
+                        status, typ = worker_error(d)
+                        err = {"error": {"message": str(d), "type": typ, "param": None,
+                                         "code": "context_length_exceeded" if TOO_LONG.match(str(d)) else None}}
+                        yield f"data: {json.dumps(err, ensure_ascii=False)}\n\n"
+                    elif isinstance(d, dict) and "reasoning" in d:
+                        yield chunk({"reasoning_content": d["reasoning"]})
+                    elif isinstance(d, dict):
+                        rest, calls, finish = finalize(full, tools, d["finish_reason"])
+                        if tools and rest.startswith(full[:sent]) and rest[sent:].strip():   # text after the calls
+                            yield chunk({"content": rest[sent:]})
+                        for i, tc in enumerate(calls):
+                            yield chunk({"tool_calls": [dict(tc, index=i)]})
+                        # the token counts on the last chunk too (as llama.cpp's server does): clients that do not
+                        # send stream_options.include_usage still see them
+                        yield chunk({}, finish, usage=usage_of(d), onw_stats=stats_of(d))
+                        if (body.get("stream_options") or {}).get("include_usage"):   # OpenAI: its own last chunk
+                            yield f"data: {json.dumps({'id': cid, 'object': 'chat.completion.chunk', 'created': created, 'model': MODEL_ID, 'choices': [], 'usage': usage_of(d)})}\n\n"
+                    elif tools:
+                        full += d
+                        upto = tool_calls.safe_upto(full)
+                        if upto > sent:
+                            yield chunk({"content": full[sent:upto]})
+                            sent = upto
+                    else:
+                        yield chunk({"content": d})
+                yield "data: [DONE]\n\n"
+            finally:  # client disconnected or finished: stop the NPU loop so the lock is released
+                stop.set()
+        return StreamingResponse(events(), media_type="text/event-stream",
+                                 headers={"Cache-Control": "no-cache", "X-Accel-Buffering": "no"})
+
+    parts, thought, st = [], [], None
+    q, stop = start_generation(msgs, kw)
+    next_item = reader(q, stop, request)                  # (a client that leaves stops the generation)
+    try:
+        while True:
+            d = await next_item()
+            if d is None:
+                break
+            if isinstance(d, Exception):
+                status, typ = worker_error(d)
+                return oai_error(status, str(d), typ)
+            if isinstance(d, dict) and "reasoning" in d:
+                thought.append(d["reasoning"])
+            elif isinstance(d, dict):
+                st = d
+            else:
+                parts.append(d)
+    finally:
+        stop.set()
+    if st is None:
+        return oai_error(500, "the generation ended without a result", "server_error")
+    text, calls, finish = finalize("".join(parts), tools, st["finish_reason"])
+    msg = {"role": "assistant", "content": (text.strip() or None) if tools else text}
+    if thought:
+        msg["reasoning_content"] = "".join(thought)
+    if calls:
+        msg["tool_calls"] = calls
+    return JSONResponse({
+        "id": cid, "object": "chat.completion", "created": created, "model": MODEL_ID,
+        "choices": [{"index": 0, "message": msg, "logprobs": None, "finish_reason": finish}],
+        "usage": usage_of(st),
+        "onw_stats": stats_of(st),
+    })
+
+
+def recommended_sampling(arch, engine_dir=None):
+    """The chat page's starting sampling values ({temperature, top_p, top_k, presence_penalty?, think?, source}) or
+    None (greedy): the model's own generation_config.json first (Gemma 4, Granite 4.2), else onw/sampling.json by
+    model_type prefix. The API itself stays greedy unless asked."""
+    keys = ("temperature", "top_p", "top_k")
+    try:
+        with open(os.path.join(engine_dir, "generation_config.json"), encoding="utf8") as f:
+            g = json.load(f)
+        if g.get("do_sample") and any(k in g for k in keys):
+            return {"source": "model", "temperature": g.get("temperature", 1.0), "top_p": g.get("top_p", 1.0),
+                    "top_k": g.get("top_k", 0)}
+    except (OSError, ValueError, TypeError):
+        pass
+    try:
+        with open(os.path.join(WEB, "..", "sampling.json"), encoding="utf8") as f:
+            table = {k: v for k, v in json.load(f).items() if not k.startswith("_")}
+    except (OSError, ValueError):
+        return None
+    hit = max((k for k in table if arch.startswith(k)), key=len, default=None)
+    if hit is None:
+        return None
+    return {k: v for k, v in table[hit].items() if k != "note"}
+
+
+from onw import compat  # noqa: E402,F401  (/v1/messages and /v1/responses on this app)
+
+
+def set_priority(busy):
+    """Windows: the server's priority class - normal while loading and waiting, above normal while it generates
+    (busy). Windows puts a below-normal process on the low-power cores at low clocks (onw 1.4.5 and earlier started
+    the server below normal: Lunar Lake, Qwen3.5-9B NF4 with MTP ~12 tok/s, 16 above normal); generating takes one or two
+    cores, so the rest of the PC hardly notices. ONW_PRIORITY: the class while generating, normal | above (default) |
+    off (leave it as started)."""
+    want = os.environ.get("ONW_PRIORITY", "above")
+    if os.name != "nt" or want not in ("normal", "above"):
+        return
+    import ctypes
+    k32 = ctypes.WinDLL("kernel32")
+    k32.GetCurrentProcess.restype = ctypes.c_void_p
+    k32.SetPriorityClass.argtypes = [ctypes.c_void_p, ctypes.c_uint32]
+    k32.SetPriorityClass(k32.GetCurrentProcess(), 0x8000 if busy and want == "above" else 0x20)
+
+
+def load_model(path, name=None):
+    """Load a model as the one this server runs: pipe, MODEL_ID, info and the sampling defaults. main() and a
+    model switch (switch_model) both come here."""
+    global pipe, MODEL_ID, API_SAMPLING
+    from onw.chat import ChatEngine
+    pipe = ChatEngine(path, SERVE["device"], context=SERVE["context"], pld=SERVE["pld"])
+    MODEL_ID = name or os.path.basename(os.path.normpath(path))
+    v = pipe.vision
+    info.update(model=MODEL_ID, title=MODEL_ID, devices={"lm": SERVE["device"]}, fast=False, vision=bool(v),
+                image_px=v.get("image_px") if v else None,
+                think=pipe.think_style is not None, max_context=pipe.max_context,
+                always_thinks=pipe.think_style == "harmony",   # (gpt-oss, llm-jp: the thinking counts against max_tokens)
+                note=f"文脈は約{pipe.max_context}トークン", context=pipe.max_context, pld=pipe.pld,
+                mtp=getattr(pipe.model, "mtp", None) is not None)
+    try:
+        with open(os.path.join(path, "config.json"), encoding="utf8") as f:
+            arch = json.load(f).get("model_type") or ""
+    except (OSError, ValueError):
+        arch = ""
+    # greedy Gemma on the NPU can fall into repeating one word (fp16 on the NPU tips a near tie; the same happens with
+    # any onw version): the pages suggest a repetition penalty, the API default stays 1.0 (OpenAI behaviour)
+    info.update(arch=arch, rep_hint=1.2 if arch.startswith("gemma") else None, sampling=recommended_sampling(arch, path))
+    want = os.environ.get("ONW_API_SAMPLING")
+    API_SAMPLING = None
+    if want == "recommended":
+        API_SAMPLING = {k: v for k, v in (info["sampling"] or {}).items() if k != "source"} or None
+    elif want:
+        try:
+            API_SAMPLING = json.loads(want)
+        except ValueError:
+            API_SAMPLING = None
+    info.update(api_sampling=API_SAMPLING, api_sampling_mode="recommended" if want == "recommended" else
+                "custom" if API_SAMPLING else "greedy")
+    if API_SAMPLING:
+        print(f"[api] requests without sampling values use {API_SAMPLING}", flush=True)
+
+
+# ---------------------------------------------------------------- model choice from the API
+# A request's "model" naming another model the app has (its id in /v1/models, or the app's key) unloads the running one
+# and loads that one first (one model in memory at a time), as Ollama and LM Studio do. A name this server does not
+# know - clients send their own defaults ("gpt-4o", ...) - is served by the loaded model, as before. The app's setting
+# "api_model_switch" (ONW_API_SWITCH=0) turns switching off.
+MODEL_KEY = None            # the app's key of the loaded model (catalog repo id or folder name; ONW_MODEL_KEY)
+SERVE = {}                  # how main() loads models: device, context, pld, port
+SWITCHING = threading.Event()   # set while a switch unloads / loads (/health answers 503: the window shows loading)
+
+
+def switch_allowed():
+    return os.environ.get("ONW_API_SWITCH", "1") != "0"
+
+
+def installed_models():
+    """{API id: (app key, path)} of every model the app finds in its folders (the ids of /v1/models)."""
+    try:
+        from onw import app as A
+        found = A.installed(A.load_config())
+    except Exception:
+        return {}
+    out = {}
+    for key, path in found.items():
+        if path and os.path.isdir(path):
+            out.setdefault(key.split("/")[-1], (key, path))
+    return out
+
+
+def resolve_model(name):
+    """(app key, path) when a request's model names another installed model, else None (the loaded one serves)."""
+    if not name or not switch_allowed():
+        return None
+    n = str(name).strip().lower()
+    if n in ((MODEL_ID or "").lower(), (MODEL_KEY or "").lower()):
+        return None
+    for mid, (key, path) in installed_models().items():
+        if n in (mid.lower(), key.lower()):
+            return None if key == MODEL_KEY else (key, path)
+    return None
+
+
+def switch_model(key, path):
+    """Unload the running model and load `key` (called with `lock` held, so no generation runs). If the new one does
+    not load, the old one is loaded again and the error goes to the client."""
+    global pipe, MODEL_KEY
+    import gc
+    old_key = MODEL_KEY
+    old_path = installed_models().get((MODEL_KEY or "").split("/")[-1], (None, None))[1]
+    print(f"[api] switching model: {MODEL_ID} -> {key.split('/')[-1]}", flush=True)
+    SWITCHING.set()
+    t = time.time()
+    try:
+        old = pipe
+        pipe = None
+        close = getattr(getattr(old, "model", None), "close", None)
+        if close:
+            close()
+        del old
+        gc.collect()
+        try:
+            os.environ["ONW_MODEL_KEY"] = key
+            load_model(path, key.split("/")[-1])
+            MODEL_KEY = key
+        except Exception:
+            if old_path:                               # back to the model that ran
+                os.environ["ONW_MODEL_KEY"] = old_key or ""
+                load_model(old_path, (old_key or "").split("/")[-1] or None)
+            raise
+        print(f"[api] {MODEL_ID} loaded in {time.time() - t:.1f}s", flush=True)
+        _record_model(key, path)
+    finally:
+        SWITCHING.clear()
+
+
+async def ensure_model(name):
+    """Before a request's arguments are read (the sampling defaults and thinking style are the model's): the model it
+    names loaded, after any generation that runs now (the same lock). A failed load: 500 with why."""
+    if resolve_model(name) is None:
+        return
+
+    def work():
+        with lock:
+            want = resolve_model(name)                 # (another request may have switched to it meanwhile)
+            if want:
+                switch_model(*want)
+    try:
+        await asyncio.get_running_loop().run_in_executor(None, work)
+    except Exception as e:
+        raise HTTPException(500, f"model {name!r} could not be loaded: {type(e).__name__}: {e}")
+
+
+def _record_model(key, path):
+    """The window and the next start follow the switch: the server record (pid file) and the chosen model."""
+    try:
+        from onw import app as A
+        p = A.PIDFILE()
+        with open(p, encoding="utf8") as f:
+            d = json.load(f)
+        if int(d.get("port", -1)) == int(SERVE.get("port", -2)) and not d.get("stopping"):
+            d.update(model=key, path=path)
+            with open(p + ".tmp", "w") as f:
+                json.dump(d, f)
+            os.replace(p + ".tmp", p)
+        A.update_config(model=key)
+    except Exception as e:
+        print(f"[api] could not record the model switch: {e}", flush=True)
+
+
+def main(argv=None):
+    global API_KEY, MODEL_KEY
+    ap = argparse.ArgumentParser(prog="onw serve", description="chat UI + OpenAI-compatible API on the Intel NPU")
+    ap.add_argument("model", help="onw model folder, or a Hugging Face repo id (downloaded into ./models)")
+    ap.add_argument("--host", default="127.0.0.1")
+    ap.add_argument("--port", type=int, default=8000)
+    ap.add_argument("--device", default="NPU", help="run every component on this device (CPU works without an NPU driver)")
+    ap.add_argument("--name", help="model id / title (default: engine directory name)")
+    ap.add_argument("--context", type=int, default=None,
+                    help="context length in tokens (default 32768, at most the model's maximum); no recompile")
+    ap.add_argument("--open", action="store_true", help="open the chat UI in the default browser once loaded")
+    ap.add_argument("--no-pld", action="store_true", help="disable prompt lookup decoding")
+    ap.add_argument("--api-key", help="require this key as a Bearer token on /v1 from other machines "
+                                         "(e.g. with --host 0.0.0.0)")
+    a = ap.parse_args(argv)
+    API_KEY = a.api_key or os.environ.get("ONW_API_KEY") or None
+    from onw.hub import resolve
+    a.engine = resolve(a.model)
+    import openvino as ov
+    from . import npukit
+    from onw.runtime import npu_info
+    print(f"[onw] {__version__} | OpenVINO {ov.__version__} | {npu_info(npukit.core())}", flush=True)
+    if a.device == "NPU" and "NPU" not in npukit.core().available_devices:
+        print("WARNING: no NPU found (is the NPU driver installed?) - running everything on the CPU instead.", flush=True)
+        a.device = "CPU"
+    SERVE.update(device=a.device, context=a.context, pld=not a.no_pld, port=a.port)
+    MODEL_KEY = os.environ.get("ONW_MODEL_KEY") or a.name
+    load_model(a.engine, a.name)
+    set_priority(False)
+    if a.open:
+        import webbrowser
+        threading.Timer(1.5, webbrowser.open, [f"http://{'localhost' if a.host in ('0.0.0.0', '127.0.0.1') else a.host}:{a.port}/chat"]).start()
+    print(f"\n  OpenAI API:  http://localhost:{a.port}/v1   (model id: {MODEL_ID})"
+          f"\n  Chat UI:     http://localhost:{a.port}/chat\n  Benchmark:   http://localhost:{a.port}/bench\n", flush=True)
+    uvicorn.run(app, host=a.host, port=a.port, log_level="warning")
+
+
+if __name__ == "__main__":
+    # (python -m onw.server: run the module that onw.compat put its routes on - this one is a second copy of it)
+    from onw.server import main as _main
+    _main()

onw/tray.py

@@ -221,7 +221,7 @@ def main(argv=None):
         pystray.MenuItem(lambda _: T("onw を開く（モデル・サーバー・設定）"), lambda *_: open_window(), default=True),
         # (the running server's port: one changed in the settings applies from the next start)
         pystray.MenuItem(lambda _: T("チャットを開く"), lambda *_: webbrowser.open(f"http://127.0.0.1:{app.server_port(cfg())}/chat"), enabled=running),
-        pystray.MenuItem(lambda _: T("検証ページを開く"), lambda *_: webbrowser.open(f"http://127.0.0.1:{app.server_port(cfg())}/check"), enabled=running),
+        pystray.MenuItem(lambda _: T("ベンチマークを開く"), lambda *_: webbrowser.open(f"http://127.0.0.1:{app.server_port(cfg())}/bench"), enabled=running),
         pystray.MenuItem(lambda _: T("OpenAI API のURLをコピー"),
                          lambda *_: copy_text(app.api_url(cfg())) or open_window("server")),   # no clipboard tool: shown there
         pystray.MenuItem(lambda _: T("onw を更新（v{0}）", manager.upd["latest"]), do_update,

onw/web/{check.html → bench.html}

@@ -4,7 +4,7 @@
 <meta charset="utf-8">
 <meta name="viewport" content="width=device-width, initial-scale=1">
 <link rel="icon" href="data:image/svg+xml,%3Csvg xmlns=%22http://www.w3.org/2000/svg%22 viewBox=%2270 90 846 846%22%3E%3Crect x=%22332%22 y=%22256%22 width=%22512%22 height=%22511%22 rx=%2280%22 fill=%22%231868f0%22/%3E%3Cline x1=%22181%22 y1=%22360%22 x2=%22208%22 y2=%22360%22 stroke=%22%231868f0%22 stroke-width=%2238%22 stroke-linecap=%22round%22/%3E%3Cline x1=%22181%22 y1=%22664%22 x2=%22212%22 y2=%22664%22 stroke=%22%231868f0%22 stroke-width=%2238%22 stroke-linecap=%22round%22/%3E%3Cline x1=%22436%22 y1=%22212%22 x2=%22436%22 y2=%22300%22 stroke=%22%231868f0%22 stroke-width=%2238%22 stroke-linecap=%22round%22/%3E%3Cline x1=%22436%22 y1=%22815%22 x2=%22436%22 y2=%22720%22 stroke=%22%231868f0%22 stroke-width=%2238%22 stroke-linecap=%22round%22/%3E%3Cline x1=%22512%22 y1=%22212%22 x2=%22512%22 y2=%22300%22 stroke=%22%231868f0%22 stroke-width=%2238%22 stroke-linecap=%22round%22/%3E%3Cline x1=%22512%22 y1=%22815%22 x2=%22512%22 y2=%22720%22 stroke=%22%231868f0%22 stroke-width=%2238%22 stroke-linecap=%22round%22/%3E%3Cline x1=%22588%22 y1=%22212%22 x2=%22588%22 y2=%22300%22 stroke=%22%231868f0%22 stroke-width=%2238%22 stroke-linecap=%22round%22/%3E%3Cline x1=%22588%22 y1=%22815%22 x2=%22588%22 y2=%22720%22 stroke=%22%231868f0%22 stroke-width=%2238%22 stroke-linecap=%22round%22/%3E%3Cline x1=%22664%22 y1=%22212%22 x2=%22664%22 y2=%22300%22 stroke=%22%231868f0%22 stroke-width=%2238%22 stroke-linecap=%22round%22/%3E%3Cline x1=%22664%22 y1=%22815%22 x2=%22664%22 y2=%22720%22 stroke=%22%231868f0%22 stroke-width=%2238%22 stroke-linecap=%22round%22/%3E%3Cline x1=%22740%22 y1=%22212%22 x2=%22740%22 y2=%22300%22 stroke=%22%231868f0%22 stroke-width=%2238%22 stroke-linecap=%22round%22/%3E%3Cline x1=%22740%22 y1=%22815%22 x2=%22740%22 y2=%22720%22 stroke=%22%231868f0%22 stroke-width=%2238%22 stroke-linecap=%22round%22/%3E%3Cline x1=%22891%22 y1=%22360%22 x2=%22800%22 y2=%22360%22 stroke=%22%231868f0%22 stroke-width=%2238%22 stroke-linecap=%22round%22/%3E%3Cline x1=%22891%22 y1=%22436%22 x2=%22800%22 y2=%22436%22 stroke=%22%231868f0%22 stroke-width=%2238%22 stroke-linecap=%22round%22/%3E%3Cline x1=%22891%22 y1=%22512%22 x2=%22800%22 y2=%22512%22 stroke=%22%231868f0%22 stroke-width=%2238%22 stroke-linecap=%22round%22/%3E%3Cline x1=%22891%22 y1=%22588%22 x2=%22800%22 y2=%22588%22 stroke=%22%231868f0%22 stroke-width=%2238%22 stroke-linecap=%22round%22/%3E%3Cline x1=%22891%22 y1=%22664%22 x2=%22800%22 y2=%22664%22 stroke=%22%231868f0%22 stroke-width=%2238%22 stroke-linecap=%22round%22/%3E%3Cline x1=%22268%22 y1=%22360%22 x2=%22380%22 y2=%22360%22 stroke=%22%231868f0%22 stroke-width=%2238%22 stroke-linecap=%22round%22/%3E%3Cline x1=%22124%22 y1=%22436%22 x2=%22380%22 y2=%22436%22 stroke=%22%231868f0%22 stroke-width=%2238%22 stroke-linecap=%22round%22/%3E%3Cline x1=%22209%22 y1=%22512%22 x2=%22380%22 y2=%22512%22 stroke=%22%231868f0%22 stroke-width=%2238%22 stroke-linecap=%22round%22/%3E%3Cline x1=%2295%22 y1=%22588%22 x2=%22380%22 y2=%22588%22 stroke=%22%231868f0%22 stroke-width=%2238%22 stroke-linecap=%22round%22/%3E%3Cline x1=%22275%22 y1=%22664%22 x2=%22380%22 y2=%22664%22 stroke=%22%231868f0%22 stroke-width=%2238%22 stroke-linecap=%22round%22/%3E%3Cellipse cx=%22588%22 cy=%22512%22 rx=%22120%22 ry=%22117%22 fill=%22%23fff%22/%3E%3Cellipse cx=%22588%22 cy=%22512%22 rx=%2257%22 ry=%2267%22 fill=%22%231868f0%22/%3E%3C/svg%3E">
-<title>onw 検証ページ</title>
+<title>ベンチマーク</title>
 <script>/*i18n*/</script>
 <style>
 :root { --bg: #f6f7f9; --fg: #1d2330; --muted: #5d6675; --card: #fff; --line: #dfe3ea; --ok: #1f8a4c; --bad: #c0392b; --accent: #2f6fde; }
@@ -17,34 +17,50 @@ h1 { font-size: 22px; margin: 0 0 4px; }
 .card { background: var(--card); border: 1px solid var(--line); border-radius: 10px; padding: 14px 16px; margin-bottom: 14px; }
 code { background: rgba(127,127,127,.15); padding: 1px 5px; border-radius: 4px; }
 button { font: inherit; padding: 8px 16px; border-radius: 8px; border: 1px solid var(--accent); background: var(--accent); color: #fff; cursor: pointer; }
+button.ghost { background: transparent; color: var(--accent); }
 button:disabled { opacity: .5; cursor: default; }
 table { width: 100%; border-collapse: collapse; }
 th, td { text-align: left; padding: 7px 8px; border-bottom: 1px solid var(--line); vertical-align: top; }
 th { color: var(--muted); font-weight: 600; font-size: 13px; }
-td.num { font-variant-numeric: tabular-nums; white-space: nowrap; }
-.ok { color: var(--ok); } .bad { color: var(--bad); }
-.ans { color: var(--muted); font-size: 13px; max-width: 420px; }
+td.num, th.num { text-align: right; font-variant-numeric: tabular-nums; white-space: nowrap; }
+td.big { font-weight: 700; }
+.bad { color: var(--bad); }
 #info td:first-child { color: var(--muted); width: 180px; }
+.opts { display: flex; flex-wrap: wrap; gap: 6px 16px; align-items: center; margin-bottom: 10px; }
+.opts label { white-space: nowrap; }
+.note { color: var(--muted); font-size: 13px; margin-top: 10px; }
 </style>
 </head>
 <body>
 <main>
-  <h1>onw 検証ページ</h1>
-  <p class="sub">このサーバー（OpenAI互換API）に実際のリクエストを送って、動作と速度を確かめます。</p>
+  <h1 id="h1">ベンチマーク</h1>
+  <p class="sub">このサーバーで、プロンプトの読み込み（プリフィル）と回答の生成（デコード）の速さを測ります。</p>
 
   <div class="card">
     <table id="info"><tbody><tr><td>読み込み中…</td><td></td></tr></tbody></table>
   </div>
 
   <div class="card">
-    <button id="run">検証を実行</button>
-    <label class="sub" style="margin-left:10px">繰り返し抑制 <input id="rep" type="number" min="1" max="2" step="0.05" value="1" style="width:60px"></label>
-    <span id="rephint" class="sub" style="display:none"></span>
+    <div class="opts">
+      <span>プロンプトの長さ:</span>
+      <label><input type="checkbox" class="len" value="128" checked> 128</label>
+      <label><input type="checkbox" class="len" value="1024" checked> 1K</label>
+      <label><input type="checkbox" class="len" value="2048" checked> 2K</label>
+      <label><input type="checkbox" class="len" value="4096" checked> 4K</label>
+      <label><input type="checkbox" class="len" value="8192"> 8K</label>
+    </div>
+    <div class="opts">
+      <label>生成するトークン数 <select id="gen"><option>128</option><option selected>256</option><option>512</option></select></label>
+      <label>回数 <select id="reps"><option>1</option><option selected>2</option><option>3</option></select></label>
+    </div>
+    <button id="run">ベンチマークを実行</button>
+    <button id="copy" class="ghost" disabled>結果をコピー（Markdown）</button>
     <span id="status" class="sub" style="margin-left:10px"></span>
     <table style="margin-top:12px">
-      <thead><tr><th>項目</th><th>結果</th><th>プロンプト</th><th>処理時間</th><th>生成速度</th><th>回答</th></tr></thead>
+      <thead><tr><th>プロンプト</th><th class="num">プリフィル</th><th class="num">プリフィル速度</th><th class="num">生成</th><th class="num">生成速度</th></tr></thead>
       <tbody id="rows"></tbody>
     </table>
+    <div class="note">プリフィル＝プロンプトを読み終えるまで（最初のトークンが出るまで）の時間と、1秒あたりに読んだトークン数。生成＝そのあと回答を書く速さ（デコード）。温度 0、思考オフ。毎回、文頭に違う一文を入れて、前の会話を再利用しないようにしています。回数を増やすと、いちばん速かった回を表示します（最初に短い慣らし運転を1回します）。</div>
   </div>
 
   <div class="card sub" data-t>他のアプリからは <code id="base"></code> を OpenAI の base URL に、モデル名は <code id="mid"></code> を指定してください（APIキー設定時は <code>Authorization: Bearer キー</code>）。</div>
@@ -52,83 +68,105 @@ td.num { font-variant-numeric: tabular-nums; white-space: nowrap; }
 <script>
 translate(document.body);
 const $ = id => document.getElementById(id);
-let INFO = {}, KEY = new URLSearchParams(location.search).get("key") || "";
-const TEXT = T("東京は日本の首都であり、政治・経済・文化の中心地である。江戸時代には徳川幕府が置かれ、1868年の明治維新で東京と改称された。現在は約1400万人が暮らす世界有数の大都市で、多くの企業や大学が集まっている。");
+let INFO = {}, KEY = new URLSearchParams(location.search).get("key") || "", RESULTS = [];
 
 const esc = s => String(s ?? "").replace(/[&<>"]/g, c => ({ "&": "&amp;", "<": "&lt;", ">": "&gt;", '"': "&quot;" }[c]));
+const fmt = (x, d) => (x == null || !isFinite(x)) ? "-" : Number(x).toLocaleString(undefined, { minimumFractionDigits: d, maximumFractionDigits: d });
 
 async function api(messages, max_tokens) {
-  // thinking at its lowest ("off"; gpt-oss / llm-jp always think - at "low" - and that counts against max_tokens)
+  // thinking at its lowest ("off"; models that always think do it at "low" - that counts against max_tokens)
   if (INFO.always_thinks) max_tokens += 1024;
-  const t0 = performance.now();
   const r = await fetch("/v1/chat/completions", {
     method: "POST",
     headers: Object.assign({ "Content-Type": "application/json" }, KEY ? { Authorization: "Bearer " + KEY } : {}),
-    body: JSON.stringify({ model: INFO.model, messages, max_tokens, temperature: 0, repetition_penalty: +$("rep").value || 1,
-                          reasoning_effort: "off" })
+    body: JSON.stringify({ model: INFO.model, messages, max_tokens, temperature: 0, reasoning_effort: "off" })
   });
   if (!r.ok) throw new Error((await r.text()).slice(0, 200));
-  const j = await r.json(), c = j.choices[0];
-  if (!(c.message.content || "").trim()) throw new Error(T("回答が空です（最大トークンを思考で使い切った可能性があります）"));
-  if (c.finish_reason === "length") throw new Error(T("最大ト���クンで打ち切られました") + ": " + c.message.content);
-  j.wall = (performance.now() - t0) / 1000;
-  return j;
+  return r.json();
 }
 
-function row(name, j, err) {
-  const tr = document.createElement("tr");
-  if (err) {
-    tr.innerHTML = `<td>${esc(name)}</td><td class="bad">${T("失敗")}</td><td colspan="4" class="ans">${esc(err)}</td>`;
-  } else {
-    const s = j.onw_stats || {}, u = j.usage || {};
-    const cached = u.prompt_tokens_details && u.prompt_tokens_details.cached_tokens ? T("（再利用 {0}）", u.prompt_tokens_details.cached_tokens) : "";
-    const vis = s.vision_ms ? T(" + 画像 {0}秒", (s.vision_ms / 1000).toFixed(1)) : "";
-    const ans = esc(j.choices[0].message.content);
-    tr.innerHTML = `<td>${esc(name)}</td><td class="ok">OK</td><td class="num">${esc(u.prompt_tokens)}${cached}</td>` +
-      `<td class="num">${T("{0}秒", ((s.prefill_ms || 0) / 1000).toFixed(2))}${vis}</td>` +
-      `<td class="num">${(s.decode_tok_s || 0).toFixed(1)} tok/s</td><td class="ans">${ans}</td>`;
-  }
-  $("rows").appendChild(tr);
+// a prompt of about n tokens from the server's own tokenizer (so 2K means 2K for this model), started by a sentence
+// that differs every time: nothing of an earlier prompt is reused
+async function prompt(n, run) {
+  const ask = T("上の文章の要点を、できるだけ詳しく説明してください。");
+  const r = await fetch(`/api/bench_text?tokens=${Math.max(16, n - 48)}`);
+  if (!r.ok) throw new Error((await r.text()).slice(0, 200));
+  const t = await r.json();
+  return [{ role: "user", content: T("ベンチマーク {0}（{1}）", run, Date.now()) + "\n\n" + t.text + "\n\n" + ask }];
+}
+
+function measure(j) {
+  const s = j.onwx_stats || j.onw_stats || {}, u = j.usage || {};
+  const reused = u.prompt_tokens_details?.cached_tokens || 0;
+  const pre = (s.prefill_ms || 0) / 1000;
+  return { prompt: u.prompt_tokens || 0, reused, prefill_s: pre,
+           prefill_tps: pre > 0 ? (u.prompt_tokens - reused) / pre : null,
+           gen: (u.completion_tokens || 0), decode_tps: s.decode_tok_s || null };
 }
 
-function testImage() {                      // drawn here, so no image file is needed
-  const c = document.createElement("canvas"); c.width = c.height = 512;
-  const g = c.getContext("2d");
-  g.fillStyle = "#ffffff"; g.fillRect(0, 0, 512, 512);
-  g.fillStyle = "#e53935"; g.beginPath(); g.arc(150, 170, 90, 0, Math.PI * 2); g.fill();
-  g.fillStyle = "#1e88e5"; g.fillRect(290, 90, 160, 160);
-  g.fillStyle = "#111111"; g.font = "bold 96px sans-serif"; g.fillText("NPU", 150, 430);
-  return c.toDataURL("image/png");
+function show() {
+  $("rows").innerHTML = RESULTS.map(r => r.err
+    ? `<tr><td>${esc(r.label)}</td><td colspan="4" class="bad">${esc(r.err)}</td></tr>`
+    : `<tr><td>${esc(r.label)}${r.reused ? " " + esc(T("（再利用 {0}）", r.reused)) : ""}</td>` +
+      `<td class="num">${esc(T("{0}秒", fmt(r.prefill_s, 2)))}</td><td class="num big">${fmt(r.prefill_tps, 0)} tok/s</td>` +
+      `<td class="num">${esc(T("{0}トークン", r.gen))}</td><td class="num big">${fmt(r.decode_tps, 1)} tok/s</td></tr>`).join("");
 }
 
 async function run() {
-  $("run").disabled = true; $("rows").innerHTML = "";
-  const steps = [
-    [T("テキスト生成"), () => api([{ role: "user", content: T("日本の四季について、それぞれ一文ずつ説明してください。") }], 320)],
-    [T("原文を再利用する要約（PLD）"), () => api([{ role: "user", content: T("次の文章を、元の表現をなるべく使って2文に要約してください。") + "\n\n" + TEXT }], 240)],
-  ];
-  const t1 = [{ role: "user", content: T("日本の首都はどこですか？一文で。") }];
-  steps.push([T("2ターン会話: 1ターン目"), async () => { const j = await api(t1, 120); t1.push({ role: "assistant", content: j.choices[0].message.content }); return j; }]);
-  steps.push([T("2ターン会話: 2ターン目"), () => api(t1.concat([{ role: "user", content: T("その都市の人口は？一文で。") }]), 120)]);
-  if (INFO.vision) steps.push([T("画像入力"), () => api([{ role: "user", content: [
-      { type: "image_url", image_url: { url: testImage() } },
-      { type: "text", text: T("この画像に描かれている図形と文字を、色も含めて一文で答えてください。") }] }], 160)]);
-  for (const [name, f] of steps) {
-    $("status").textContent = name + " …";
-    try { row(name, await f()); } catch (e) { row(name, null, e.message); }
+  const lens = [...document.querySelectorAll(".len")].filter(c => c.checked).map(c => +c.value);
+  const gen = +$("gen").value, reps = +$("reps").value;
+  const ctx = INFO.context || INFO.max_context || 0;
+  $("run").disabled = true; $("copy").disabled = true; RESULTS = []; show();
+  try {
+    $("status").textContent = T("慣らし運転…");
+    await api(await prompt(64, 0), 8);
+  } catch (e) { $("status").textContent = T("失敗") + ": " + e.message; $("run").disabled = false; return; }
+  let k = 0;
+  for (const n of lens) {
+    const label = T("{0} トークン", n.toLocaleString());
+    if (ctx && n + gen + 64 > ctx) { RESULTS.push({ label, err: T("コンテキスト長（{0}）を超えるので飛ばしました", ctx) }); show(); continue; }
+    let best = null;
+    for (let i = 0; i < reps; i++) {
+      $("status").textContent = T("{0}（{1}/{2}回目）…", label, i + 1, reps);
+      try {
+        const m = measure(await api(await prompt(n, ++k), gen));
+        best = best ? { ...best, prefill_s: Math.min(best.prefill_s, m.prefill_s),
+                        prefill_tps: Math.max(best.prefill_tps || 0, m.prefill_tps || 0),
+                        decode_tps: Math.max(best.decode_tps || 0, m.decode_tps || 0) } : m;
+      } catch (e) { best = { err: e.message }; break; }
+    }
+    RESULTS.push({ label: best.err ? label : T("{0} トークン", best.prompt.toLocaleString()), ...best }); show();
   }
-  $("status").textContent = T("完了"); $("run").disabled = false;
+  $("status").textContent = T("完了"); $("run").disabled = false; $("copy").disabled = !RESULTS.some(r => !r.err);
 }
 
+function markdown() {
+  const env = [[T("アプリ"), INFO.app], [T("モデル"), INFO.model], [T("デバイス"), Object.values(INFO.devices || {}).join(", ")],
+               [T("CPU"), INFO.cpu], [T("OS"), INFO.os], [T("条件"), T("温度 0・思考オフ・生成 {0} トークン・{1} 回の最良値", $("gen").value, $("reps").value)],
+               [T("測定日"), new Date().toLocaleDateString()]].filter(([, v]) => v);
+  const head = `| ${T("プロンプト")} | ${T("プリフィル")} | ${T("プリフィル速度")} | ${T("生成速度")} |\n|---|---:|---:|---:|`;
+  const body = RESULTS.filter(r => !r.err).map(r =>
+    `| ${r.label} | ${T("{0}秒", fmt(r.prefill_s, 2))} | ${fmt(r.prefill_tps, 0)} tok/s | ${fmt(r.decode_tps, 1)} tok/s |`).join("\n");
+  return env.map(([k, v]) => `- ${k}: ${v}`).join("\n") + "\n\n" + head + "\n" + body + "\n";
+}
+
+$("copy").onclick = async () => {
+  const text = markdown();
+  try { await navigator.clipboard.writeText(text); $("status").textContent = T("コピーしました"); }
+  catch { const a = document.createElement("textarea"); a.value = text; document.body.appendChild(a); a.select();
+          document.execCommand("copy"); a.remove(); $("status").textContent = T("コピーしました"); }
+};
+
 fetch("/api/info").then(r => r.json()).then(i => {
   INFO = i;
-  const rows = [[T("モデル名"), i.model], [T("デバイス"), Object.values(i.devices || {}).join(", ")],
-    [T("コンテキスト長"), T("{0} トークン", i.context || i.max_context)], [T("画像入力"), i.vision ? T("対応") : T("なし")],
-    [T("先読み検証（PLD）"), i.pld && !i.mtp ? T("有効") : T("無効")], [T("MTP 先読み"), i.mtp ? T("有効") : T("なし")],
-    [T("思考モード"), i.think ? T("対応") : T("なし")]];
+  document.title = $("h1").textContent = T("{0} ベンチマーク", (i.app || "").split(" ")[0] || "LLM");
+  const rows = [[T("アプリ"), i.app], [T("モデル名"), i.model], [T("デバイス"), Object.values(i.devices || {}).join(", ")],
+    [T("CPU"), i.cpu], [T("OS"), i.os], [T("コンテキスト長"), T("{0} トークン", i.context || i.max_context)],
+    [T("先読み（MTP・ドラフト）"), i.mtp ? T("有効") : T("なし")]].filter(([, v]) => v);
   $("info").innerHTML = rows.map(([k, v]) => `<tr><td>${esc(k)}</td><td>${esc(v)}</td></tr>`).join("");
   $("base").textContent = location.origin + "/v1"; $("mid").textContent = i.model;
-  if (i.rep_hint) { $("rephint").textContent = T("{0} 以上推奨（1 では同じ言葉を繰り返すことがあります）", i.rep_hint); $("rephint").style.display = ""; }
+  const ctx = i.context || i.max_context || 0;
+  for (const c of document.querySelectorAll(".len")) if (ctx && +c.value + 512 > ctx) { c.checked = false; c.disabled = true; }
 }).catch(() => { $("info").innerHTML = `<tr><td class='bad'>${T("サーバーに接続できません")}</td></tr>`; });
 $("run").onclick = run;
 </script>

onw/web/index.html

@@ -110,6 +110,7 @@
   <button id="sidebtn" title="会話の履歴">☰</button>
   <h1 id="title">NPU</h1>
   <span id="dev" class="badge">接続中…</span>
+  <span id="speed" class="badge ok" style="display:none"></span>
   <span class="spacer"></span>
   <button id="clear" title="会話をクリア">新しい会話</button>
 </header>
@@ -537,10 +538,26 @@ async function send() {
   return answer();
 }
 
+// [bench] the last answer's speeds in the header: prompt reading (the tokens read anew / the server's prefill time)
+// and generation (the server's decode tokens a second)
+function showSpeed(usage, s) {
+  if (!usage || !s) return;
+  const fresh = (usage.prompt_tokens || 0) - (usage.prompt_tokens_details?.cached_tokens || 0);
+  const pre = (s.prefill_ms || 0) / 1000;
+  const parts = [];
+  if (fresh > 0 && pre > 0) parts.push(T("プリフィル {0} tok/s", Math.round(fresh / pre).toLocaleString()));
+  if (s.decode_tok_s) parts.push(T("生成 {0} tok/s", s.decode_tok_s.toFixed(1)));
+  const b = document.getElementById("speed");
+  b.textContent = parts.join(" · ");
+  b.title = T("直前の回答：プロンプト {0} トークン（新しく読んだ分 {1}）を {2} 秒で読み、{3} トークンを生成",
+              usage.prompt_tokens, fresh, pre.toFixed(2), usage.completion_tokens);
+  b.style.display = parts.length ? "" : "none";
+}
+
 async function answer() {                // an answer to the conversation as it stands (send, regenerate)
   let lastThink = "";                    // (this answer's thinking, saved with it)
   const out = addMsg("assistant", "…");
-  let acc = "", usage = null; const t0 = performance.now(); let first = 0;
+  let acc = "", usage = null, speed = null; const t0 = performance.now(); let first = 0;
   const mine = chat, ctl = new AbortController();   // (a new / another conversation may be opened meanwhile)
   controller = ctl; $("send").textContent = T("停止");
   let tm = 0, connected = false;         // the answer is drawn at most every 100 ms (not on every token), and once at the end
@@ -592,6 +609,7 @@ async function answer() {                // an answer to the conversation as it
         }
         if (j.choices?.[0]?.finish_reason) finish = j.choices[0].finish_reason;
         if (j.usage) usage = j.usage;
+        if (j.onwx_stats || j.onw_stats) speed = j.onwx_stats || j.onw_stats;   // [bench] the server's own timing
       }
     }
     if (!finish) throw new TypeError("cut off");   // (the stream ended without an end: the server went away)
@@ -613,6 +631,7 @@ async function answer() {                // an answer to the conversation as it
     }
     if (acc && finish === "length") st += (st ? " · " : "") + T("最大トークンで打ち切り");   // (cut off by the token limit)
     if (st) { const s = document.createElement("div"); s.className = "stats"; s.textContent = st; out.bubble.appendChild(s); }
+    showSpeed(usage, speed);
     mine.msgs.push({ role: "assistant", text: acc, reasoning: lastThink, stats: st });
     saveChat(mine);
     addTools(out, acc);

onw/web/manage.html

@@ -179,7 +179,7 @@
         <span class="muted small" id="sinfo"></span>
         <span class="grow"></span>
         <button id="b-chat" onclick="post('open',{what:'chat'})">チャット</button>
-        <button id="b-check" onclick="post('open',{what:'check'})">検証ページ</button>
+        <button id="b-check" onclick="post('open',{what:'bench'})">ベンチマーク</button>
         <button onclick="post('open',{what:'log'})">ログ</button>
       </div>
       <div class="kv">