onw AMD 0.1.3: model file check, installer restart, download progress
Browse files- .gitattributes +2 -0
- CHANGELOG.md +16 -0
- CHANGELOG_en.md +16 -0
- find_python.bat +45 -45
- flm_compare.py +119 -0
- flm_verify.py +79 -0
- onw/__init__.py +1 -1
- onw/app.py +5 -2
- onw/dl.py +88 -19
- onw/i18n.json +16 -0
- onw/lemonade.py +116 -2
- onw/manager.py +74 -4
- onw/server.py +13 -4
- onw/tray.py +54 -2
- onw/web/manage.html +8 -5
- pyproject.toml +1 -1
- setup.bat +57 -56
- setup.sh +2 -1
- start.bat +52 -52
.gitattributes
CHANGED
|
@@ -34,3 +34,5 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
|
| 34 |
*.zst filter=lfs diff=lfs merge=lfs -text
|
| 35 |
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
| 36 |
images/onw-banner.jpg filter=lfs diff=lfs merge=lfs -text
|
|
|
|
|
|
|
|
|
| 34 |
*.zst filter=lfs diff=lfs merge=lfs -text
|
| 35 |
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
| 36 |
images/onw-banner.jpg filter=lfs diff=lfs merge=lfs -text
|
| 37 |
+
*.bat text eol=crlf
|
| 38 |
+
*.sh text eol=lf
|
CHANGELOG.md
CHANGED
|
@@ -1,5 +1,21 @@
|
|
| 1 |
# 更新履歴
|
| 2 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 3 |
## v0.1.2
|
| 4 |
|
| 5 |
- Linux:インストーラー(`setup.sh`)が memlock の上限を設定するように(FastFlowLM の Linux 手順で必須。Ubuntu の既定値では
|
|
|
|
| 1 |
# 更新履歴
|
| 2 |
|
| 3 |
+
## v0.1.3
|
| 4 |
+
|
| 5 |
+
- 動いている最中に 1 行インストールをもう一度実行(更新)したとき、トレイが新しい版で自動的に再起動するように(使っていた
|
| 6 |
+
モデルも読み込み直す)。これまでは古いトレイが動き続け、手で再起動する必要があった。0.1.2 以前のトレイは止めてから
|
| 7 |
+
新しいトレイが引き継ぐ。
|
| 8 |
+
- モデルのダウンロードの進み具合を、FastFlowLM が表示するバイト数とモデルの実際のファイルサイズから出すように。これまでは
|
| 9 |
+
完了したファイルの数で出していたため、大きな重みファイル(qwen3.6-moe-35b-a3b で約 20 GB)を落とす間は 0% のままだった。
|
| 10 |
+
- モデルのファイルを FastFlowLM のハッシュと照合するように。ダウンロード直後に全ファイルを照合し、壊れていれば壊れたファイル
|
| 11 |
+
だけ自動で取り直す。「モデル」タブの「⋯」→「ファイルを照合」で、入っているモデルをあとから確かめられ、壊れていれば
|
| 12 |
+
「ダウンロードし直す」で直せる。ロードの前にはファイルのサイズを確かめ、欠けていればロードしない(壊れた重みは
|
| 13 |
+
`<unused49>` のような意味のない出力になる)。
|
| 14 |
+
- FastFlowLM の一覧(`flm list`)が、壊れたモデルが 1 つあるだけで空になる場合も、同梱の `model_list.json` から一覧を出すように。
|
| 15 |
+
- FastFlowLM を入れたあと、「使うモデル」の設定に `engine` という名前が入ることがあったのを修正。
|
| 16 |
+
- Gemma 4 の思考(`<|channel>thought`)も、途中で打ち切られたときに思考欄へ分けるように。
|
| 17 |
+
- Windows:二重起動防止のロックを、起動できなかった側が握ったままにしていたのを修正(引き継げなかった原因)。
|
| 18 |
+
|
| 19 |
## v0.1.2
|
| 20 |
|
| 21 |
- Linux:インストーラー(`setup.sh`)が memlock の上限を設定するように(FastFlowLM の Linux 手順で必須。Ubuntu の既定値では
|
CHANGELOG_en.md
CHANGED
|
@@ -1,5 +1,21 @@
|
|
| 1 |
# Changelog
|
| 2 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 3 |
## v0.1.2
|
| 4 |
|
| 5 |
- Linux: the installer (`setup.sh`) raises the memlock limit (required by FastFlowLM's Linux guide; Ubuntu's default
|
|
|
|
| 1 |
# Changelog
|
| 2 |
|
| 3 |
+
## v0.1.3
|
| 4 |
+
|
| 5 |
+
- Running the one-line installer again while onw is running (an update) now restarts the tray on the new version,
|
| 6 |
+
loading the model again, instead of leaving the old tray running until a manual restart. Trays of 0.1.2 and older
|
| 7 |
+
are stopped and the new tray takes over.
|
| 8 |
+
- Model download progress comes from the bytes FastFlowLM reports and the model's real file sizes; it used to count
|
| 9 |
+
finished files, so it sat at 0% while a big weights file came down (about 20 GB for qwen3.6-moe-35b-a3b).
|
| 10 |
+
- Model files are checked against FastFlowLM's hashes: right after a download (damaged files are fetched again
|
| 11 |
+
once), on demand from the Models tab (... -> "Check the files", then "Download again" for damaged ones), and by size
|
| 12 |
+
before every load (broken weights give garbage such as `<unused49>`).
|
| 13 |
+
- The model list no longer goes empty when `flm list` returns nothing because of one damaged model folder: it then
|
| 14 |
+
comes from the `model_list.json` that ships with flm.
|
| 15 |
+
- Fixed: installing FastFlowLM could set the chosen model to "engine".
|
| 16 |
+
- Gemma 4's thinking (`<|channel>thought`) is split off too when the answer was cut off while thinking.
|
| 17 |
+
- Windows: a process that did not get the single-instance lock no longer keeps holding it (it blocked the take-over).
|
| 18 |
+
|
| 19 |
## v0.1.2
|
| 20 |
|
| 21 |
- Linux: the installer (`setup.sh`) raises the memlock limit (required by FastFlowLM's Linux guide; Ubuntu's default
|
find_python.bat
CHANGED
|
@@ -1,45 +1,45 @@
|
|
| 1 |
-
@echo off
|
| 2 |
-
rem Sets PY to a Python 3.10+ command, offering to install Python 3.12 with winget when none is found.
|
| 3 |
-
rem Used by setup.bat and start.bat: call find_python.bat then if not defined PY exit /b 1
|
| 4 |
-
set "PY="
|
| 5 |
-
set "CHECK=import sys; sys.exit(0 if sys.version_info >= (3, 10) else 1)"
|
| 6 |
-
|
| 7 |
-
rem 1) the py launcher, 2) python on PATH (the Microsoft Store alias fails this check, it is not a real Python),
|
| 8 |
-
rem 3) the per-user install locations winget / python.org use
|
| 9 |
-
if not defined ONW_TEST_NOPY (
|
| 10 |
-
py -3 -c "%CHECK%" >nul 2>nul && set "PY=py -3"
|
| 11 |
-
if not defined PY python -c "%CHECK%" >nul 2>nul && set "PY=python"
|
| 12 |
-
if not defined PY for %%V in (313 312 311 310) do (
|
| 13 |
-
if not defined PY if exist "%LOCALAPPDATA%\Programs\Python\Python%%V\python.exe" set "PY="%LOCALAPPDATA%\Programs\Python\Python%%V\python.exe""
|
| 14 |
-
)
|
| 15 |
-
)
|
| 16 |
-
if defined PY exit /b 0
|
| 17 |
-
|
| 18 |
-
echo.
|
| 19 |
-
echo Python 3.10 or newer was not found.
|
| 20 |
-
where winget >nul 2>nul
|
| 21 |
-
if errorlevel 1 (
|
| 22 |
-
echo winget is not available either. Please install Python from https://www.python.org/downloads/
|
| 23 |
-
echo ^(check "Add python.exe to PATH"^), then run this again.
|
| 24 |
-
start "" https://www.python.org/downloads/
|
| 25 |
-
exit /b 1
|
| 26 |
-
)
|
| 27 |
-
choice /c YN /m "Install Python 3.12 for this user with winget now (accepts the Python license)"
|
| 28 |
-
if errorlevel 2 (
|
| 29 |
-
echo Please install Python 3.10+ from https://www.python.org/downloads/ and run this again.
|
| 30 |
-
exit /b 1
|
| 31 |
-
)
|
| 32 |
-
winget install -e --id Python.Python.3.12 --scope user --accept-package-agreements --accept-source-agreements
|
| 33 |
-
if errorlevel 1 (
|
| 34 |
-
echo winget could not install Python. Please install it from https://www.python.org/downloads/
|
| 35 |
-
exit /b 1
|
| 36 |
-
)
|
| 37 |
-
rem the new PATH is not visible in this window yet: use the installed files directly
|
| 38 |
-
if exist "%LOCALAPPDATA%\Programs\Python\Python312\python.exe" set "PY="%LOCALAPPDATA%\Programs\Python\Python312\python.exe""
|
| 39 |
-
if not defined PY if exist "%LOCALAPPDATA%\Programs\Python\Launcher\py.exe" set "PY="%LOCALAPPDATA%\Programs\Python\Launcher\py.exe" -3"
|
| 40 |
-
if not defined PY (
|
| 41 |
-
echo Python was installed but could not be found yet. Please open a new window and run this again.
|
| 42 |
-
exit /b 1
|
| 43 |
-
)
|
| 44 |
-
echo [setup] using %PY%
|
| 45 |
-
exit /b 0
|
|
|
|
| 1 |
+
@echo off
|
| 2 |
+
rem Sets PY to a Python 3.10+ command, offering to install Python 3.12 with winget when none is found.
|
| 3 |
+
rem Used by setup.bat and start.bat: call find_python.bat then if not defined PY exit /b 1
|
| 4 |
+
set "PY="
|
| 5 |
+
set "CHECK=import sys; sys.exit(0 if sys.version_info >= (3, 10) else 1)"
|
| 6 |
+
|
| 7 |
+
rem 1) the py launcher, 2) python on PATH (the Microsoft Store alias fails this check, it is not a real Python),
|
| 8 |
+
rem 3) the per-user install locations winget / python.org use
|
| 9 |
+
if not defined ONW_TEST_NOPY (
|
| 10 |
+
py -3 -c "%CHECK%" >nul 2>nul && set "PY=py -3"
|
| 11 |
+
if not defined PY python -c "%CHECK%" >nul 2>nul && set "PY=python"
|
| 12 |
+
if not defined PY for %%V in (313 312 311 310) do (
|
| 13 |
+
if not defined PY if exist "%LOCALAPPDATA%\Programs\Python\Python%%V\python.exe" set "PY="%LOCALAPPDATA%\Programs\Python\Python%%V\python.exe""
|
| 14 |
+
)
|
| 15 |
+
)
|
| 16 |
+
if defined PY exit /b 0
|
| 17 |
+
|
| 18 |
+
echo.
|
| 19 |
+
echo Python 3.10 or newer was not found.
|
| 20 |
+
where winget >nul 2>nul
|
| 21 |
+
if errorlevel 1 (
|
| 22 |
+
echo winget is not available either. Please install Python from https://www.python.org/downloads/
|
| 23 |
+
echo ^(check "Add python.exe to PATH"^), then run this again.
|
| 24 |
+
start "" https://www.python.org/downloads/
|
| 25 |
+
exit /b 1
|
| 26 |
+
)
|
| 27 |
+
choice /c YN /m "Install Python 3.12 for this user with winget now (accepts the Python license)"
|
| 28 |
+
if errorlevel 2 (
|
| 29 |
+
echo Please install Python 3.10+ from https://www.python.org/downloads/ and run this again.
|
| 30 |
+
exit /b 1
|
| 31 |
+
)
|
| 32 |
+
winget install -e --id Python.Python.3.12 --scope user --accept-package-agreements --accept-source-agreements
|
| 33 |
+
if errorlevel 1 (
|
| 34 |
+
echo winget could not install Python. Please install it from https://www.python.org/downloads/
|
| 35 |
+
exit /b 1
|
| 36 |
+
)
|
| 37 |
+
rem the new PATH is not visible in this window yet: use the installed files directly
|
| 38 |
+
if exist "%LOCALAPPDATA%\Programs\Python\Python312\python.exe" set "PY="%LOCALAPPDATA%\Programs\Python\Python312\python.exe""
|
| 39 |
+
if not defined PY if exist "%LOCALAPPDATA%\Programs\Python\Launcher\py.exe" set "PY="%LOCALAPPDATA%\Programs\Python\Launcher\py.exe" -3"
|
| 40 |
+
if not defined PY (
|
| 41 |
+
echo Python was installed but could not be found yet. Please open a new window and run this again.
|
| 42 |
+
exit /b 1
|
| 43 |
+
)
|
| 44 |
+
echo [setup] using %PY%
|
| 45 |
+
exit /b 0
|
flm_compare.py
ADDED
|
@@ -0,0 +1,119 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""onw AMD test: one model through FastFlowLM directly (no onw, no browser), changing one thing at a time, to find what
|
| 2 |
+
breaks it (e.g. <unused49> from Gemma 4 12B on one PC while Lemonade is fine). Standard library only.
|
| 3 |
+
|
| 4 |
+
python3 flm_compare.py gemma4-it:12b /path/to/photo.jpg
|
| 5 |
+
curl -fsSL https://huggingface.co/ryugyosoft/onwAMD/resolve/main/flm_compare.py | python3 - gemma4-it:12b ~/photo.jpg
|
| 6 |
+
|
| 7 |
+
Unload the model in onw (and stop Lemonade) first: they would compete for the NPU. Compared:
|
| 8 |
+
flm program onw's (portable build, bundled XRT) / the system's (fastflowlm package, system XRT) when installed
|
| 9 |
+
--ctx-len 4096 (what onw passes) / not passed (flm's own default)
|
| 10 |
+
think false (what onw sends) / not sent (what Lemonade does)
|
| 11 |
+
Each answer is shortened; "<unused" in it is flagged. The table is saved as onw-amd-flm-compare.txt in the home folder.
|
| 12 |
+
"""
|
| 13 |
+
import base64, glob, json, os, shutil, subprocess, sys, time, urllib.request
|
| 14 |
+
|
| 15 |
+
WIN = os.name == "nt"
|
| 16 |
+
PORT = 8098
|
| 17 |
+
OUT = []
|
| 18 |
+
|
| 19 |
+
|
| 20 |
+
def say(*a):
|
| 21 |
+
s = " ".join(str(x) for x in a)
|
| 22 |
+
print(s, flush=True)
|
| 23 |
+
OUT.append(s)
|
| 24 |
+
|
| 25 |
+
|
| 26 |
+
def cfg_dir():
|
| 27 |
+
base = os.environ.get("APPDATA") if WIN else os.path.join(os.path.expanduser("~"), ".config")
|
| 28 |
+
return os.path.join(base or os.path.expanduser("~"), "onw-amd")
|
| 29 |
+
|
| 30 |
+
|
| 31 |
+
def flms():
|
| 32 |
+
exe = "flm.exe" if WIN else "flm"
|
| 33 |
+
out = []
|
| 34 |
+
mine = glob.glob(os.path.join(cfg_dir(), "bin", "flm", "npu", "**", exe), recursive=True)
|
| 35 |
+
if mine:
|
| 36 |
+
out.append(("onw", mine[0]))
|
| 37 |
+
for p in (shutil.which("flm"), "/opt/fastflowlm/bin/flm", r"C:\Program Files\flm\flm.exe"):
|
| 38 |
+
if p and os.path.exists(p) and all(os.path.realpath(p) != os.path.realpath(q) for _, q in out):
|
| 39 |
+
out.append(("system", p))
|
| 40 |
+
return out
|
| 41 |
+
|
| 42 |
+
|
| 43 |
+
def models_dir():
|
| 44 |
+
try:
|
| 45 |
+
d = json.load(open(os.path.join(cfg_dir(), "config.json"), encoding="utf8")).get("models_dir")
|
| 46 |
+
except Exception:
|
| 47 |
+
d = None
|
| 48 |
+
return d or os.path.join(cfg_dir(), "flm")
|
| 49 |
+
|
| 50 |
+
|
| 51 |
+
def ask(model, content, think):
|
| 52 |
+
body = {"model": model, "max_tokens": 120, "temperature": 0, "messages": [{"role": "user", "content": content}]}
|
| 53 |
+
if think is not None:
|
| 54 |
+
body["think"] = think
|
| 55 |
+
t0 = time.time()
|
| 56 |
+
try:
|
| 57 |
+
j = json.loads(urllib.request.urlopen(urllib.request.Request(
|
| 58 |
+
f"http://127.0.0.1:{PORT}/v1/chat/completions", json.dumps(body).encode(),
|
| 59 |
+
{"Content-Type": "application/json"}), timeout=900).read())
|
| 60 |
+
m = j["choices"][0]["message"]
|
| 61 |
+
text = (m.get("content") or "").replace("\n", " ")
|
| 62 |
+
bad = "<unused" in text or "<unused" in (m.get("reasoning_content") or "")
|
| 63 |
+
return f"{'BROKEN ' if bad else 'ok '}{time.time() - t0:5.1f}s think={len(m.get('reasoning_content') or ''):4d}ch {text[:90]!r}"
|
| 64 |
+
except Exception as e:
|
| 65 |
+
return f"ERROR {e}"
|
| 66 |
+
|
| 67 |
+
|
| 68 |
+
def main():
|
| 69 |
+
args = [a for a in sys.argv[1:] if a != "-"]
|
| 70 |
+
model = args[0] if args else "gemma4-it:12b"
|
| 71 |
+
image = os.path.expanduser(args[1]) if len(args) > 1 else None
|
| 72 |
+
env = {**os.environ, "FLM_MODEL_PATH": models_dir()}
|
| 73 |
+
say("model", model, "| image", image or "(none)", "| FLM_MODEL_PATH", env["FLM_MODEL_PATH"])
|
| 74 |
+
progs = flms()
|
| 75 |
+
if not progs:
|
| 76 |
+
sys.exit("no flm found (install FastFlowLM from the onw window first)")
|
| 77 |
+
contents = [("text", "日本の首都はどこですか?一文で。")]
|
| 78 |
+
if image:
|
| 79 |
+
ext = os.path.splitext(image)[1].lower().lstrip(".").replace("jpg", "jpeg") or "jpeg"
|
| 80 |
+
url = f"data:image/{ext};base64," + base64.b64encode(open(image, "rb").read()).decode()
|
| 81 |
+
contents.append(("image", [{"type": "image_url", "image_url": {"url": url}},
|
| 82 |
+
{"type": "text", "text": "写真に写っているものはなんですか"}]))
|
| 83 |
+
for name, exe in progs:
|
| 84 |
+
rc = subprocess.run([exe, "version"], capture_output=True, text=True, env=env)
|
| 85 |
+
say(f"\n######## {name}: {exe} ({(rc.stdout or rc.stderr).strip()[:60]})")
|
| 86 |
+
for ctx in (4096, None):
|
| 87 |
+
cmd = [exe, "serve", model, "--port", str(PORT), "--host", "127.0.0.1"] + (["--ctx-len", str(ctx)] if ctx else [])
|
| 88 |
+
log = open(os.path.join(os.path.expanduser("~"), f"onw-amd-flm-compare-{name}-{ctx or 'default'}.log"), "w")
|
| 89 |
+
p = subprocess.Popen(cmd, env=env, stdout=log, stderr=subprocess.STDOUT)
|
| 90 |
+
try:
|
| 91 |
+
up = False
|
| 92 |
+
for _ in range(600):
|
| 93 |
+
if p.poll() is not None:
|
| 94 |
+
break
|
| 95 |
+
try:
|
| 96 |
+
urllib.request.urlopen(f"http://127.0.0.1:{PORT}/api/tags", timeout=2)
|
| 97 |
+
up = True
|
| 98 |
+
break
|
| 99 |
+
except OSError:
|
| 100 |
+
time.sleep(1)
|
| 101 |
+
say(f"-- ctx-len {ctx or 'flm default'}: " + ("up" if up else f"did not start (exit {p.poll()}), see {log.name}"))
|
| 102 |
+
if up:
|
| 103 |
+
for think in (False, None):
|
| 104 |
+
for kind, content in contents:
|
| 105 |
+
say(f" think={'false' if think is False else 'unsent'} {kind:<5} {ask(model, content, think)}")
|
| 106 |
+
finally:
|
| 107 |
+
p.terminate()
|
| 108 |
+
try:
|
| 109 |
+
p.wait(20)
|
| 110 |
+
except subprocess.TimeoutExpired:
|
| 111 |
+
p.kill()
|
| 112 |
+
time.sleep(2)
|
| 113 |
+
path = os.path.join(os.path.expanduser("~"), "onw-amd-flm-compare.txt")
|
| 114 |
+
open(path, "w", encoding="utf8").write("\n".join(OUT) + "\n")
|
| 115 |
+
print(f"\nSaved: {path} (send this file)")
|
| 116 |
+
|
| 117 |
+
|
| 118 |
+
if __name__ == "__main__":
|
| 119 |
+
main()
|
flm_verify.py
ADDED
|
@@ -0,0 +1,79 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""onw AMD test: are a FastFlowLM model's files intact? Checks each file against the hashes in flm's own
|
| 2 |
+
model_info.json (sha256 for the big files, git's blob sha1 for the small ones). Standard library only; reads only.
|
| 3 |
+
|
| 4 |
+
python3 flm_verify.py gemma4-it:12b the copy in onw's model folder
|
| 5 |
+
python3 flm_verify.py gemma4-it:12b /path/to/flm another FLM_MODEL_PATH (e.g. Lemonade's, ~/.config/flm)
|
| 6 |
+
"""
|
| 7 |
+
import glob, hashlib, json, os, sys
|
| 8 |
+
|
| 9 |
+
WIN = os.name == "nt"
|
| 10 |
+
|
| 11 |
+
|
| 12 |
+
def cfg_dir():
|
| 13 |
+
base = os.environ.get("APPDATA") if WIN else os.path.join(os.path.expanduser("~"), ".config")
|
| 14 |
+
return os.path.join(base or os.path.expanduser("~"), "onw-amd")
|
| 15 |
+
|
| 16 |
+
|
| 17 |
+
def flm_dir():
|
| 18 |
+
exe = "flm.exe" if WIN else "flm"
|
| 19 |
+
hits = glob.glob(os.path.join(cfg_dir(), "bin", "flm", "npu", "**", exe), recursive=True)
|
| 20 |
+
for d in [os.path.dirname(h) for h in hits] + ["/opt/fastflowlm/share/flm", r"C:\Program Files\flm"]:
|
| 21 |
+
if os.path.exists(os.path.join(d, "model_info.json")):
|
| 22 |
+
return d
|
| 23 |
+
sys.exit("model_info.json not found (install FastFlowLM from the onw window first)")
|
| 24 |
+
|
| 25 |
+
|
| 26 |
+
def models_root(arg):
|
| 27 |
+
if arg:
|
| 28 |
+
return os.path.expanduser(arg)
|
| 29 |
+
try:
|
| 30 |
+
d = json.load(open(os.path.join(cfg_dir(), "config.json"), encoding="utf8")).get("models_dir")
|
| 31 |
+
except Exception:
|
| 32 |
+
d = None
|
| 33 |
+
return d or os.path.join(cfg_dir(), "flm")
|
| 34 |
+
|
| 35 |
+
|
| 36 |
+
def digest(path, kind, size):
|
| 37 |
+
h = hashlib.sha256() if kind == "sha256" else hashlib.sha1(f"blob {size}\0".encode())
|
| 38 |
+
with open(path, "rb") as f:
|
| 39 |
+
for b in iter(lambda: f.read(1 << 22), b""):
|
| 40 |
+
h.update(b)
|
| 41 |
+
return h.hexdigest()
|
| 42 |
+
|
| 43 |
+
|
| 44 |
+
def main():
|
| 45 |
+
args = [a for a in sys.argv[1:] if a != "-"]
|
| 46 |
+
ck = args[0] if args else "gemma4-it:12b"
|
| 47 |
+
d = flm_dir()
|
| 48 |
+
family, tag = ck.split(":", 1)
|
| 49 |
+
entry = json.load(open(os.path.join(d, "model_list.json"), encoding="utf8"))["models"][family][tag]
|
| 50 |
+
info = {f["path"]: f for f in json.load(open(os.path.join(d, "model_info.json"), encoding="utf8"))[ck]}
|
| 51 |
+
folder = os.path.join(models_root(args[1] if len(args) > 1 else None), "models", entry["name"])
|
| 52 |
+
print("model", ck, "| folder", folder)
|
| 53 |
+
bad = 0
|
| 54 |
+
for name in entry["files"]:
|
| 55 |
+
f, p = info.get(name), os.path.join(folder, name)
|
| 56 |
+
if not os.path.exists(p):
|
| 57 |
+
print(f" MISSING {name}")
|
| 58 |
+
bad += 1
|
| 59 |
+
continue
|
| 60 |
+
size = os.path.getsize(p)
|
| 61 |
+
if f is None:
|
| 62 |
+
print(f" ? {name} ({size} bytes; no hash to compare)")
|
| 63 |
+
continue
|
| 64 |
+
want_size = f.get("size")
|
| 65 |
+
kind, want = ("sha256", f["lfs"]["oid"]) if f.get("lfs") else ("sha1", f.get("oid"))
|
| 66 |
+
if want_size and size != want_size:
|
| 67 |
+
print(f" BAD SIZE {name}: {size} bytes, expected {want_size}")
|
| 68 |
+
bad += 1
|
| 69 |
+
continue
|
| 70 |
+
print(f" ... {name} ({size / 2**20:.0f} MB) hashing", end="\r", flush=True)
|
| 71 |
+
got = digest(p, kind, size)
|
| 72 |
+
ok = got == want
|
| 73 |
+
bad += not ok
|
| 74 |
+
print(f" {'OK ' if ok else 'BAD HASH'} {name} ({size / 2**20:.0f} MB){'' if ok else f' {kind} {got[:16]}…, expected {want[:16]}…'}")
|
| 75 |
+
print("\nall files intact" if not bad else f"\n{bad} file(s) damaged: download the model again (onw: delete it in the Models tab, then Download)")
|
| 76 |
+
|
| 77 |
+
|
| 78 |
+
if __name__ == "__main__":
|
| 79 |
+
main()
|
onw/__init__.py
CHANGED
|
@@ -2,4 +2,4 @@
|
|
| 2 |
|
| 3 |
LLMs entirely on the AMD Ryzen AI NPU (XDNA2), through FastFlowLM, the NPU-only engine of Lemonade Server.
|
| 4 |
"""
|
| 5 |
-
__version__ = "0.1.
|
|
|
|
| 2 |
|
| 3 |
LLMs entirely on the AMD Ryzen AI NPU (XDNA2), through FastFlowLM, the NPU-only engine of Lemonade Server.
|
| 4 |
"""
|
| 5 |
+
__version__ = "0.1.3"
|
onw/app.py
CHANGED
|
@@ -102,11 +102,14 @@ def single_instance(name):
|
|
| 102 |
import ctypes
|
| 103 |
k = ctypes.WinDLL("kernel32", use_last_error=True)
|
| 104 |
k.CreateMutexW.restype = ctypes.c_void_p
|
| 105 |
-
h = k.CreateMutexW(None, False, f"Local\\onw-{name}")
|
| 106 |
if not h:
|
| 107 |
return True
|
|
|
|
|
|
|
|
|
|
| 108 |
_LOCKS.append(h)
|
| 109 |
-
return
|
| 110 |
import fcntl
|
| 111 |
f = open(os.path.join(config_dir(), f"{name}.lock"), "w")
|
| 112 |
try:
|
|
|
|
| 102 |
import ctypes
|
| 103 |
k = ctypes.WinDLL("kernel32", use_last_error=True)
|
| 104 |
k.CreateMutexW.restype = ctypes.c_void_p
|
| 105 |
+
h = k.CreateMutexW(None, False, f"Local\\onw-{name}") # the name 0.1.x trays hold (renaming it lets two run)
|
| 106 |
if not h:
|
| 107 |
return True
|
| 108 |
+
if ctypes.get_last_error() == 183: # ERROR_ALREADY_EXISTS: let go of it again, or the lock
|
| 109 |
+
k.CloseHandle(ctypes.c_void_p(h)) # would outlive the other process (tray take-over)
|
| 110 |
+
return False
|
| 111 |
_LOCKS.append(h)
|
| 112 |
+
return True
|
| 113 |
import fcntl
|
| 114 |
f = open(os.path.join(config_dir(), f"{name}.lock"), "w")
|
| 115 |
try:
|
onw/dl.py
CHANGED
|
@@ -2,7 +2,9 @@
|
|
| 2 |
a crash inside a download cannot take the tray down. Progress goes to a small JSON file the manager reads.
|
| 3 |
|
| 4 |
usage (internal): python -m onw.dl KEY STATUS_FILE a model ("...-FLM"): FastFlowLM first when missing,
|
| 5 |
-
|
|
|
|
|
|
|
| 6 |
python -m onw.dl --engine STATUS_FILE just FastFlowLM
|
| 7 |
"""
|
| 8 |
import json, os, re, subprocess, sys, time
|
|
@@ -14,12 +16,12 @@ class Status:
|
|
| 14 |
def __init__(self, path):
|
| 15 |
self.path, self.last = path, 0.0
|
| 16 |
|
| 17 |
-
def __call__(self, frac, text, force=False):
|
| 18 |
if not force and time.time() - self.last < 0.5:
|
| 19 |
return
|
| 20 |
self.last = time.time()
|
| 21 |
with open(self.path + ".tmp", "w", encoding="utf8") as f:
|
| 22 |
-
json.dump({"frac": round(frac, 4), "text": text}, f, ensure_ascii=False)
|
| 23 |
os.replace(self.path + ".tmp", self.path)
|
| 24 |
|
| 25 |
|
|
@@ -29,26 +31,77 @@ def engine(status):
|
|
| 29 |
status(1.0, "FastFlowLM " + lemonade.VERSION, force=True)
|
| 30 |
|
| 31 |
|
| 32 |
-
def
|
| 33 |
"""`flm pull <checkpoint>`; progress from its "[FLM] Downloading X/Y: file" lines (lemonade: flm_download)."""
|
| 34 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 35 |
ok, msg = lemonade.flm_validate(cfg)
|
| 36 |
if not ok:
|
| 37 |
sys.exit(f"FastFlowLM NPU check failed: {msg}")
|
| 38 |
m = lemonade.flm_models(cfg).get(key)
|
| 39 |
if m is None:
|
| 40 |
sys.exit(f"FastFlowLM has no model {key}")
|
| 41 |
-
|
| 42 |
-
env=lemonade.flm_env(cfg), text=True, encoding="utf8", errors="replace", **lemonade.NO_WINDOW)
|
| 43 |
-
pat = re.compile(r"Downloading\s+(\d+)/(\d+):\s*(.*)")
|
| 44 |
-
for line in p.stdout:
|
| 45 |
-
print(line, end="", flush=True)
|
| 46 |
-
g = pat.search(line)
|
| 47 |
-
if g:
|
| 48 |
-
i, n = int(g.group(1)), int(g.group(2))
|
| 49 |
-
status((i - 1) / max(n, 1), f"{i}/{n}: {g.group(3).strip()[:60]}", force=True)
|
| 50 |
-
if p.wait() != 0:
|
| 51 |
-
sys.exit(f"flm pull failed (exit code {p.returncode})")
|
| 52 |
|
| 53 |
|
| 54 |
def main():
|
|
@@ -56,11 +109,27 @@ def main():
|
|
| 56 |
if args[0] == "--engine":
|
| 57 |
engine(Status(args[1]))
|
| 58 |
return
|
| 59 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
| 60 |
status(0.0, "…", force=True)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 61 |
engine(lambda frac, text, force=False: status(0.0, text, force))
|
| 62 |
-
|
| 63 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 64 |
|
| 65 |
|
| 66 |
if __name__ == "__main__":
|
|
|
|
| 2 |
a crash inside a download cannot take the tray down. Progress goes to a small JSON file the manager reads.
|
| 3 |
|
| 4 |
usage (internal): python -m onw.dl KEY STATUS_FILE a model ("...-FLM"): FastFlowLM first when missing,
|
| 5 |
+
`flm pull`, then every file checked against flm's hashes
|
| 6 |
+
python -m onw.dl --repair KEY STATUS_FILE the model's damaged files fetched again (after a check)
|
| 7 |
+
python -m onw.dl --verify KEY STATUS_FILE just the check; the result (damaged files) in STATUS_FILE
|
| 8 |
python -m onw.dl --engine STATUS_FILE just FastFlowLM
|
| 9 |
"""
|
| 10 |
import json, os, re, subprocess, sys, time
|
|
|
|
| 16 |
def __init__(self, path):
|
| 17 |
self.path, self.last = path, 0.0
|
| 18 |
|
| 19 |
+
def __call__(self, frac, text, force=False, **extra):
|
| 20 |
if not force and time.time() - self.last < 0.5:
|
| 21 |
return
|
| 22 |
self.last = time.time()
|
| 23 |
with open(self.path + ".tmp", "w", encoding="utf8") as f:
|
| 24 |
+
json.dump({"frac": round(frac, 4), "text": text, **extra}, f, ensure_ascii=False)
|
| 25 |
os.replace(self.path + ".tmp", self.path)
|
| 26 |
|
| 27 |
|
|
|
|
| 31 |
status(1.0, "FastFlowLM " + lemonade.VERSION, force=True)
|
| 32 |
|
| 33 |
|
| 34 |
+
def pull(m, status, cfg, force=False):
|
| 35 |
"""`flm pull <checkpoint>`; progress from its "[FLM] Downloading X/Y: file" lines (lemonade: flm_download)."""
|
| 36 |
+
p = subprocess.Popen(lemonade.flm_pull_command(m["checkpoint"], force), stdout=subprocess.PIPE,
|
| 37 |
+
stderr=subprocess.STDOUT, env=lemonade.flm_env(cfg), text=True, encoding="utf8",
|
| 38 |
+
errors="replace", **lemonade.NO_WINDOW)
|
| 39 |
+
# flm prints "Downloading 2/6: model.q4nx" per file and, with \r, "Downloading: 44.6% (85.7MB / 191.9MB)" while a
|
| 40 |
+
# file comes down: the bytes give the progress (one big weights file would otherwise sit at 0% for minutes)
|
| 41 |
+
file_pat = re.compile(r"Downloading\s+(\d+)/(\d+):\s*(.*)")
|
| 42 |
+
byte_pat = re.compile(r"Downloading:\s*([\d.]+)%\s*\(([\d.]+)\s*([KMGT]?B)\s*/\s*([\d.]+)\s*([KMGT]?B)\)")
|
| 43 |
+
unit = {"B": 1, "KB": 2**10, "MB": 2**20, "GB": 2**30, "TB": 2**40}
|
| 44 |
+
total = lemonade.flm_download_size(m["checkpoint"]) # the files' sizes (the footprint is memory, not download)
|
| 45 |
+
done = cur = cur_total = 0.0 # bytes of the finished files, of the current one
|
| 46 |
+
i = n = 0
|
| 47 |
+
name = ""
|
| 48 |
+
|
| 49 |
+
def report():
|
| 50 |
+
whole = max(total, done + cur_total, 1)
|
| 51 |
+
frac = min((done + cur) / whole, 0.99)
|
| 52 |
+
status(frac, f"{(done + cur) / 2**30:.1f} / {whole / 2**30:.1f} GB ({frac * 100:.0f}%)"
|
| 53 |
+
+ (f" · {i}/{n} {name[:40]}" if n else ""))
|
| 54 |
+
|
| 55 |
+
for line in p.stdout:
|
| 56 |
+
print(line, end="", flush=True)
|
| 57 |
+
line = re.sub(r"\x1b\[[0-9;?]*[A-Za-z]", "", line)
|
| 58 |
+
g = file_pat.search(line)
|
| 59 |
+
if g:
|
| 60 |
+
done, cur, cur_total = done + cur_total, 0.0, 0.0 # the previous file is complete
|
| 61 |
+
i, n, name = int(g.group(1)), int(g.group(2)), g.group(3).strip()
|
| 62 |
+
report()
|
| 63 |
+
continue
|
| 64 |
+
g = byte_pat.search(line)
|
| 65 |
+
if g:
|
| 66 |
+
cur = float(g.group(2)) * unit.get(g.group(3).upper(), 1)
|
| 67 |
+
cur_total = float(g.group(4)) * unit.get(g.group(5).upper(), 1)
|
| 68 |
+
report()
|
| 69 |
+
if p.wait() != 0:
|
| 70 |
+
sys.exit(f"flm pull failed (exit code {p.returncode})")
|
| 71 |
+
|
| 72 |
+
|
| 73 |
+
def verify(m, status, cfg):
|
| 74 |
+
"""The model's files against flm's hashes. -> damaged file names ([] intact, None: no hashes to check)."""
|
| 75 |
+
status(0.0, "checking…", force=True)
|
| 76 |
+
bad = lemonade.flm_verify(m["checkpoint"], cfg,
|
| 77 |
+
lambda frac, text: status(frac, f"checking {text} ({frac * 100:.0f}%)"))
|
| 78 |
+
print(f"[onw] verify {m['checkpoint']}: " + ("no hashes" if bad is None else ", ".join(bad) or "intact"), flush=True)
|
| 79 |
+
return bad
|
| 80 |
+
|
| 81 |
+
|
| 82 |
+
def repair(m, bad, status, cfg):
|
| 83 |
+
"""Fetch the damaged files again: remove them so that `flm pull` downloads them; if flm still counts the model
|
| 84 |
+
as complete, pull it whole with --force. Exits when the files are still damaged."""
|
| 85 |
+
for name, path, *_ in lemonade.flm_files(m["checkpoint"], cfg):
|
| 86 |
+
if name in bad and os.path.exists(path):
|
| 87 |
+
os.remove(path)
|
| 88 |
+
pull(m, status, cfg)
|
| 89 |
+
bad = verify(m, status, cfg)
|
| 90 |
+
if bad:
|
| 91 |
+
pull(m, status, cfg, force=True)
|
| 92 |
+
bad = verify(m, status, cfg)
|
| 93 |
+
if bad:
|
| 94 |
+
sys.exit("downloaded files are damaged (checked against FastFlowLM's hashes): " + ", ".join(bad))
|
| 95 |
+
|
| 96 |
+
|
| 97 |
+
def model_entry(key, cfg):
|
| 98 |
ok, msg = lemonade.flm_validate(cfg)
|
| 99 |
if not ok:
|
| 100 |
sys.exit(f"FastFlowLM NPU check failed: {msg}")
|
| 101 |
m = lemonade.flm_models(cfg).get(key)
|
| 102 |
if m is None:
|
| 103 |
sys.exit(f"FastFlowLM has no model {key}")
|
| 104 |
+
return m
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 105 |
|
| 106 |
|
| 107 |
def main():
|
|
|
|
| 109 |
if args[0] == "--engine":
|
| 110 |
engine(Status(args[1]))
|
| 111 |
return
|
| 112 |
+
if args[0] in ("--verify", "--repair"):
|
| 113 |
+
mode, key, status = args[0], args[1], Status(args[2])
|
| 114 |
+
else:
|
| 115 |
+
mode, key, status = "--download", args[0], Status(args[1])
|
| 116 |
+
cfg = app.load_config()
|
| 117 |
status(0.0, "…", force=True)
|
| 118 |
+
if mode == "--verify":
|
| 119 |
+
m = lemonade.flm_models(cfg).get(key)
|
| 120 |
+
if m is None:
|
| 121 |
+
sys.exit(f"FastFlowLM has no model {key}")
|
| 122 |
+
bad = verify(m, status, cfg)
|
| 123 |
+
status(1.0, "done", force=True, bad=bad)
|
| 124 |
+
return
|
| 125 |
engine(lambda frac, text, force=False: status(0.0, text, force))
|
| 126 |
+
m = model_entry(key, cfg)
|
| 127 |
+
if mode == "--download":
|
| 128 |
+
pull(m, status, cfg)
|
| 129 |
+
bad = verify(m, status, cfg) # (--repair: finds the damaged files again, here)
|
| 130 |
+
if bad:
|
| 131 |
+
repair(m, bad, status, cfg)
|
| 132 |
+
status(1.0, "done", force=True, bad=[])
|
| 133 |
|
| 134 |
|
| 135 |
if __name__ == "__main__":
|
onw/i18n.json
CHANGED
|
@@ -31,6 +31,7 @@
|
|
| 31 |
"v{0} に更新しました": "Updated to v{0}",
|
| 32 |
"v{0} に更新しました。onw を起動し直してください": "Updated to v{0}. Please restart onw",
|
| 33 |
"v{0} に更新しました。再起動します…": "Updated to v{0}. Restarting…",
|
|
|
|
| 34 |
"{0} を削除しますか?\n{1}": "Delete {0}?\n{1}",
|
| 35 |
"{0} トークン": "{0} tokens",
|
| 36 |
"{0}秒": "{0} s",
|
|
@@ -68,6 +69,7 @@
|
|
| 68 |
"サーバーに接続できません": "Cannot connect to the server",
|
| 69 |
"ダウンロード": "Download",
|
| 70 |
"ダウンロードが終わってから更新してください": "Update after the download finishes",
|
|
|
|
| 71 |
"ダウンロードに失敗: {0}": "Download failed: {0}",
|
| 72 |
"ダウンロード中 {0}/{1}": "Downloading {0}/{1}",
|
| 73 |
"ダウンロード中 {0}・ウィンドウを閉じても続きます": "Downloading {0} · keeps going if you close the window",
|
|
@@ -79,6 +81,10 @@
|
|
| 79 |
"デバイス": "Device",
|
| 80 |
"ドライバーの入れ方": "How to install the driver",
|
| 81 |
"パッケージを更新中(数分かかることがあります)": "Updating packages (may take a few minutes)",
|
|
|
|
|
|
|
|
|
|
|
|
|
| 82 |
"フォルダが見つかりません: {0}": "Folder not found: {0}",
|
| 83 |
"フォルダのパスを入力してください": "Enter the folder path",
|
| 84 |
"フォルダを開く": "Open folder",
|
|
@@ -136,6 +142,7 @@
|
|
| 136 |
"可": "Yes",
|
| 137 |
"回答": "Answer",
|
| 138 |
"場所": "Location",
|
|
|
|
| 139 |
"変更…": "Change…",
|
| 140 |
"変更点": "What's new",
|
| 141 |
"失敗": "Failed",
|
|
@@ -180,6 +187,7 @@
|
|
| 180 |
"止まった理由: {0}": "Why it stopped: {0}",
|
| 181 |
"温度": "Temperature",
|
| 182 |
"準備中…": "Preparing…",
|
|
|
|
| 183 |
"状態": "Status",
|
| 184 |
"生成速度": "Decode speed",
|
| 185 |
"画像": "Image",
|
|
@@ -240,6 +248,7 @@
|
|
| 240 |
"v{0} に更新しました": "已更新到 v{0}",
|
| 241 |
"v{0} に更新しました。onw を起動し直してください": "已更新到 v{0},请重新启动 onw",
|
| 242 |
"v{0} に更新しました。再起動します…": "已更新到 v{0},正在重启…",
|
|
|
|
| 243 |
"{0} を削除しますか?\n{1}": "删除 {0}?\n{1}",
|
| 244 |
"{0} トークン": "{0} token",
|
| 245 |
"{0}秒": "{0} 秒",
|
|
@@ -277,6 +286,7 @@
|
|
| 277 |
"サーバーに接続できません": "无法连接服务器",
|
| 278 |
"ダウンロード": "下载",
|
| 279 |
"ダウンロードが終わってから更新してください": "请在下载完成后再更新",
|
|
|
|
| 280 |
"ダウンロードに失敗: {0}": "下载失败:{0}",
|
| 281 |
"ダウンロード中 {0}/{1}": "下载中 {0}/{1}",
|
| 282 |
"ダウンロード中 {0}・ウィンドウを閉じても続きます": "下载中 {0}・关闭窗口也会继续",
|
|
@@ -288,6 +298,10 @@
|
|
| 288 |
"デバイス": "设备",
|
| 289 |
"ドライバーの入れ方": "如何安装驱动",
|
| 290 |
"パッケージを更新中(数分かかることがあります)": "正在更新软件包(可能需要几分钟)",
|
|
|
|
|
|
|
|
|
|
|
|
|
| 291 |
"フォルダが見つかりません: {0}": "找不到文件夹:{0}",
|
| 292 |
"フォルダのパスを入力してください": "请输入文件夹路径",
|
| 293 |
"フォルダを開く": "打开文件夹",
|
|
@@ -345,6 +359,7 @@
|
|
| 345 |
"可": "支持",
|
| 346 |
"回答": "回答",
|
| 347 |
"場所": "位置",
|
|
|
|
| 348 |
"変更…": "更改…",
|
| 349 |
"変更点": "更新内容",
|
| 350 |
"失敗": "失败",
|
|
@@ -389,6 +404,7 @@
|
|
| 389 |
"止まった理由: {0}": "停止原因:{0}",
|
| 390 |
"温度": "温度",
|
| 391 |
"準備中…": "准备中…",
|
|
|
|
| 392 |
"状態": "状态",
|
| 393 |
"生成速度": "生成速度",
|
| 394 |
"画像": "图像",
|
|
|
|
| 31 |
"v{0} に更新しました": "Updated to v{0}",
|
| 32 |
"v{0} に更新しました。onw を起動し直してください": "Updated to v{0}. Please restart onw",
|
| 33 |
"v{0} に更新しました。再起動します…": "Updated to v{0}. Restarting…",
|
| 34 |
+
"{0} のファイルが壊れています({1})。「モデル」タブの「ダウンロードし直す」で直せます": "The files of {0} are damaged ({1}). Fix them with \"Download again\" in the Models tab",
|
| 35 |
"{0} を削除しますか?\n{1}": "Delete {0}?\n{1}",
|
| 36 |
"{0} トークン": "{0} tokens",
|
| 37 |
"{0}秒": "{0} s",
|
|
|
|
| 69 |
"サーバーに接続できません": "Cannot connect to the server",
|
| 70 |
"ダウンロード": "Download",
|
| 71 |
"ダウンロードが終わってから更新してください": "Update after the download finishes",
|
| 72 |
+
"ダウンロードし直す": "Download again",
|
| 73 |
"ダウンロードに失敗: {0}": "Download failed: {0}",
|
| 74 |
"ダウンロード中 {0}/{1}": "Downloading {0}/{1}",
|
| 75 |
"ダウンロード中 {0}・ウィンドウを閉じても続きます": "Downloading {0} · keeps going if you close the window",
|
|
|
|
| 81 |
"デバイス": "Device",
|
| 82 |
"ドライバーの入れ方": "How to install the driver",
|
| 83 |
"パッケージを更新中(数分かかることがあります)": "Updating packages (may take a few minutes)",
|
| 84 |
+
"ファイルが壊れています({0})。「ダウンロードし直す」で壊れたファイルだけ取り直します": "Damaged files ({0}). \"Download again\" fetches just those again",
|
| 85 |
+
"ファイルを照合": "Check the files",
|
| 86 |
+
"ファイルを照合しました:問題ありません": "Files checked: all intact",
|
| 87 |
+
"ファイルを照合中 {0}": "Checking the files {0}",
|
| 88 |
"フォルダが見つかりません: {0}": "Folder not found: {0}",
|
| 89 |
"フォルダのパスを入力してください": "Enter the folder path",
|
| 90 |
"フォルダを開く": "Open folder",
|
|
|
|
| 142 |
"可": "Yes",
|
| 143 |
"回答": "Answer",
|
| 144 |
"場所": "Location",
|
| 145 |
+
"壊れています": "Damaged",
|
| 146 |
"変更…": "Change…",
|
| 147 |
"変更点": "What's new",
|
| 148 |
"失敗": "Failed",
|
|
|
|
| 187 |
"止まった理由: {0}": "Why it stopped: {0}",
|
| 188 |
"温度": "Temperature",
|
| 189 |
"準備中…": "Preparing…",
|
| 190 |
+
"照合中": "checking",
|
| 191 |
"状態": "Status",
|
| 192 |
"生成速度": "Decode speed",
|
| 193 |
"画像": "Image",
|
|
|
|
| 248 |
"v{0} に更新しました": "已更新到 v{0}",
|
| 249 |
"v{0} に更新しました。onw を起動し直してください": "已更新到 v{0},请重新启动 onw",
|
| 250 |
"v{0} に更新しました。再起動します…": "已更新到 v{0},正在重启…",
|
| 251 |
+
"{0} のファイルが壊れています({1})。「モデル」タブの「ダウンロードし直す」で直せます": "{0} 的文件已损坏({1})。可在“模型”页点击“重新下载”修复",
|
| 252 |
"{0} を削除しますか?\n{1}": "删除 {0}?\n{1}",
|
| 253 |
"{0} トークン": "{0} token",
|
| 254 |
"{0}秒": "{0} 秒",
|
|
|
|
| 286 |
"サーバーに接続できません": "无法连接服务器",
|
| 287 |
"ダウンロード": "下载",
|
| 288 |
"ダウンロードが終わってから更新してください": "请在下载完成后再更新",
|
| 289 |
+
"ダウンロードし直す": "重新下载",
|
| 290 |
"ダウンロードに失敗: {0}": "下载失败:{0}",
|
| 291 |
"ダウンロード中 {0}/{1}": "下载中 {0}/{1}",
|
| 292 |
"ダウンロード中 {0}・ウィンドウを閉じても続きます": "下载中 {0}・关闭窗口也会继续",
|
|
|
|
| 298 |
"デバイス": "设备",
|
| 299 |
"ドライバーの入れ方": "如何安装驱动",
|
| 300 |
"パッケージを更新中(数分かかることがあります)": "正在更新软件包(可能需要几分钟)",
|
| 301 |
+
"ファイルが壊れています({0})。「ダウンロードし直す」で壊れたファイルだけ取り直します": "文件已损坏({0})。点击“重新下载”只重新获取损坏的文件",
|
| 302 |
+
"ファイルを照合": "校验文件",
|
| 303 |
+
"ファイルを照合しました:問題ありません": "文件已校验:没有问题",
|
| 304 |
+
"ファイルを照合中 {0}": "正在校验文件 {0}",
|
| 305 |
"フォルダが見つかりません: {0}": "找不到文件夹:{0}",
|
| 306 |
"フォルダのパスを入力してください": "请输入文件夹路径",
|
| 307 |
"フォルダを開く": "打开文件夹",
|
|
|
|
| 359 |
"可": "支持",
|
| 360 |
"回答": "回答",
|
| 361 |
"場所": "位置",
|
| 362 |
+
"壊れています": "已损坏",
|
| 363 |
"変更…": "更改…",
|
| 364 |
"変更点": "更新内容",
|
| 365 |
"失敗": "失败",
|
|
|
|
| 404 |
"止まった理由: {0}": "停止原因:{0}",
|
| 405 |
"温度": "温度",
|
| 406 |
"準備中…": "准备中…",
|
| 407 |
+
"照合中": "校验中",
|
| 408 |
"状態": "状态",
|
| 409 |
"生成速度": "生成速度",
|
| 410 |
"画像": "图像",
|
onw/lemonade.py
CHANGED
|
@@ -249,18 +249,132 @@ def flm_name(checkpoint):
|
|
| 249 |
def flm_models(cfg=None):
|
| 250 |
"""{name: {"checkpoint", "size", "labels", "installed"}} of flm's chat models (`flm list --json`). Embedding and
|
| 251 |
speech-to-text models are left out: onw is a chat server."""
|
| 252 |
-
r = _flm(["list", "--json"], cfg)
|
| 253 |
out = {}
|
| 254 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 255 |
ck = m.get("name")
|
| 256 |
if not isinstance(ck, str) or "whisper" in ck.lower() or "embed" in ck.lower():
|
| 257 |
continue
|
| 258 |
out[flm_name(ck)] = {"checkpoint": ck, "size": m.get("footprint") or 0.0,
|
| 259 |
"labels": [l for l in m.get("label") or [] if isinstance(l, str)],
|
| 260 |
"installed": bool(m.get("installed"))}
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 261 |
return out
|
| 262 |
|
| 263 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 264 |
def flm_pull_command(checkpoint, force=False):
|
| 265 |
exe = binary()
|
| 266 |
if not exe:
|
|
|
|
| 249 |
def flm_models(cfg=None):
|
| 250 |
"""{name: {"checkpoint", "size", "labels", "installed"}} of flm's chat models (`flm list --json`). Embedding and
|
| 251 |
speech-to-text models are left out: onw is a chat server."""
|
|
|
|
| 252 |
out = {}
|
| 253 |
+
try:
|
| 254 |
+
r = _flm(["list", "--json"], cfg)
|
| 255 |
+
listed = _json_in(r.stdout).get("models", [])
|
| 256 |
+
except (OSError, ValueError, subprocess.SubprocessError):
|
| 257 |
+
listed = []
|
| 258 |
+
for m in listed:
|
| 259 |
ck = m.get("name")
|
| 260 |
if not isinstance(ck, str) or "whisper" in ck.lower() or "embed" in ck.lower():
|
| 261 |
continue
|
| 262 |
out[flm_name(ck)] = {"checkpoint": ck, "size": m.get("footprint") or 0.0,
|
| 263 |
"labels": [l for l in m.get("label") or [] if isinstance(l, str)],
|
| 264 |
"installed": bool(m.get("installed"))}
|
| 265 |
+
# one damaged model folder (an unreadable config.json) makes `flm list` return nothing at all: then the list
|
| 266 |
+
# comes from model_list.json, which ships with flm, and "installed" from the files being there in full size
|
| 267 |
+
return out or flm_models_offline(cfg)
|
| 268 |
+
|
| 269 |
+
|
| 270 |
+
def flm_models_offline(cfg=None):
|
| 271 |
+
"""flm_models without running flm: model_list.json, and the model's files present with their sizes."""
|
| 272 |
+
exe = binary()
|
| 273 |
+
if not exe:
|
| 274 |
+
return {}
|
| 275 |
+
try:
|
| 276 |
+
families = json.load(open(os.path.join(os.path.dirname(exe), "model_list.json"), encoding="utf8"))["models"]
|
| 277 |
+
except (OSError, ValueError, KeyError):
|
| 278 |
+
return {}
|
| 279 |
+
out = {}
|
| 280 |
+
for family, tags in families.items():
|
| 281 |
+
for tag, e in (tags or {}).items():
|
| 282 |
+
ck = f"{family}:{tag}"
|
| 283 |
+
if not isinstance(e, dict) or "whisper" in ck.lower() or "embed" in ck.lower():
|
| 284 |
+
continue
|
| 285 |
+
files = flm_files(ck, cfg)
|
| 286 |
+
out[flm_name(ck)] = {"checkpoint": ck, "size": e.get("footprint") or 0.0,
|
| 287 |
+
"labels": [l for l in e.get("label") or [] if isinstance(l, str)],
|
| 288 |
+
"installed": bool(files) and all(os.path.isfile(p) for _, p, *_ in files)}
|
| 289 |
+
return out
|
| 290 |
+
|
| 291 |
+
|
| 292 |
+
def flm_download_size(checkpoint):
|
| 293 |
+
"""Bytes `flm pull` downloads for a checkpoint ("qwen3.5:0.8b"): the files model_list.json names, sized by
|
| 294 |
+
model_info.json (both ship with flm), else model_list's rough "size"; 0 when unknown."""
|
| 295 |
+
exe = binary()
|
| 296 |
+
if not exe:
|
| 297 |
+
return 0
|
| 298 |
+
d = os.path.dirname(exe)
|
| 299 |
+
try:
|
| 300 |
+
family, tag = checkpoint.split(":", 1)
|
| 301 |
+
entry = json.load(open(os.path.join(d, "model_list.json"), encoding="utf8"))["models"][family][tag]
|
| 302 |
+
except (OSError, ValueError, KeyError):
|
| 303 |
+
return 0
|
| 304 |
+
try:
|
| 305 |
+
sizes = {f["path"]: f.get("size") or 0 for f in
|
| 306 |
+
json.load(open(os.path.join(d, "model_info.json"), encoding="utf8"))[checkpoint]}
|
| 307 |
+
total = sum(sizes.get(f, 0) for f in entry.get("files") or [])
|
| 308 |
+
if total:
|
| 309 |
+
return total
|
| 310 |
+
except (OSError, ValueError, KeyError, TypeError):
|
| 311 |
+
pass
|
| 312 |
+
return int(entry.get("size") or 0)
|
| 313 |
+
|
| 314 |
+
|
| 315 |
+
def flm_files(checkpoint, cfg=None):
|
| 316 |
+
"""[(name, path, size, hash kind, hash)] of the files `flm pull` puts in the model's folder, with the hashes flm
|
| 317 |
+
ships in model_info.json: sha256 (git LFS) for the big files, git's blob sha1 for the small ones (None when flm
|
| 318 |
+
has no hash for a file). [] when unknown."""
|
| 319 |
+
exe = binary()
|
| 320 |
+
if not exe:
|
| 321 |
+
return []
|
| 322 |
+
d = os.path.dirname(exe)
|
| 323 |
+
try:
|
| 324 |
+
family, tag = checkpoint.split(":", 1)
|
| 325 |
+
entry = json.load(open(os.path.join(d, "model_list.json"), encoding="utf8"))["models"][family][tag]
|
| 326 |
+
info = {f["path"]: f for f in json.load(open(os.path.join(d, "model_info.json"), encoding="utf8")).get(checkpoint, [])}
|
| 327 |
+
except (OSError, ValueError, KeyError):
|
| 328 |
+
return []
|
| 329 |
+
folder = os.path.join(flm_dir(cfg), "models", entry["name"])
|
| 330 |
+
out = []
|
| 331 |
+
for name in entry.get("files") or []:
|
| 332 |
+
f = info.get(name) or {}
|
| 333 |
+
kind, want = ("sha256", f["lfs"]["oid"]) if f.get("lfs") else ("sha1", f.get("oid"))
|
| 334 |
+
out.append((name, os.path.join(folder, name), f.get("size"), kind, want))
|
| 335 |
return out
|
| 336 |
|
| 337 |
|
| 338 |
+
def flm_check_sizes(checkpoint, cfg=None):
|
| 339 |
+
"""The quick check before a load: names of the model's files that are missing or have the wrong size."""
|
| 340 |
+
bad = []
|
| 341 |
+
for name, path, size, _, _ in flm_files(checkpoint, cfg):
|
| 342 |
+
if not os.path.isfile(path) or (size and os.path.getsize(path) != size):
|
| 343 |
+
bad.append(name)
|
| 344 |
+
return bad
|
| 345 |
+
|
| 346 |
+
|
| 347 |
+
def flm_verify(checkpoint, cfg=None, progress=lambda frac, text: None, should_stop=lambda: False):
|
| 348 |
+
"""Hash every file of the model against flm's hashes (minutes for a big model). -> names of the damaged or missing
|
| 349 |
+
files ([] = intact). None when flm has no hashes for this model."""
|
| 350 |
+
import hashlib
|
| 351 |
+
files = flm_files(checkpoint, cfg)
|
| 352 |
+
if not files or not any(want for *_, want in files):
|
| 353 |
+
return None
|
| 354 |
+
total = sum(size or 0 for _, _, size, _, _ in files) or 1
|
| 355 |
+
done, bad = 0, []
|
| 356 |
+
for name, path, size, kind, want in files:
|
| 357 |
+
if not os.path.isfile(path) or (size and os.path.getsize(path) != size):
|
| 358 |
+
bad.append(name)
|
| 359 |
+
done += size or 0
|
| 360 |
+
continue
|
| 361 |
+
if not want:
|
| 362 |
+
done += size or 0
|
| 363 |
+
continue
|
| 364 |
+
real = os.path.getsize(path)
|
| 365 |
+
h = hashlib.sha256() if kind == "sha256" else hashlib.sha1(f"blob {real}\0".encode())
|
| 366 |
+
with open(path, "rb") as f:
|
| 367 |
+
for b in iter(lambda: f.read(8 << 20), b""):
|
| 368 |
+
if should_stop():
|
| 369 |
+
raise RuntimeError("cancelled")
|
| 370 |
+
h.update(b)
|
| 371 |
+
done += len(b)
|
| 372 |
+
progress(done / total, f"{done / 2**30:.1f} / {total / 2**30:.1f} GB")
|
| 373 |
+
if h.hexdigest() != want:
|
| 374 |
+
bad.append(name)
|
| 375 |
+
return bad
|
| 376 |
+
|
| 377 |
+
|
| 378 |
def flm_pull_command(checkpoint, force=False):
|
| 379 |
exe = binary()
|
| 380 |
if not exe:
|
onw/manager.py
CHANGED
|
@@ -50,12 +50,14 @@ class Manager:
|
|
| 50 |
self.dl = None # {"key", "proc", "status", "frac", "text", "error", "done", "log"}
|
| 51 |
self.npu = None
|
| 52 |
self.check = None # lemonade.flm_check() + "text" (the reasons in the UI language)
|
|
|
|
| 53 |
self.ram = app.total_ram_gb()
|
| 54 |
self.busy = "" # a message about a long action (loading / deleting) for the page
|
| 55 |
self.busy_err = False
|
| 56 |
self.upd = {"latest": None, "tag": None, "notes": "", "checked": None, "state": "idle", "msg": "",
|
| 57 |
"checking": False, "error": ""}
|
| 58 |
self.on_updated = None # set by the tray: restart into the new version
|
|
|
|
| 59 |
self.on_update_found = None # set by the tray: notification
|
| 60 |
threading.Thread(target=self._update_loop, daemon=True).start()
|
| 61 |
threading.Thread(target=self._npu_info, daemon=True).start()
|
|
@@ -109,11 +111,20 @@ class Manager:
|
|
| 109 |
"mem_gb": mem, "fits": app.fits(mem, self.ram), "deletable": bool(path),
|
| 110 |
"note": "・".join(T(NOTES[l]) for l in c["labels"] if l in NOTES)}
|
| 111 |
m["state"] = self._state_of(key, path, info, status, dl)
|
|
|
|
| 112 |
if dl and dl["key"] == key and not dl.get("done"):
|
| 113 |
m["progress"] = dl["frac"]
|
| 114 |
-
|
|
|
|
|
|
|
|
|
|
| 115 |
elif dl and dl.get("done") == key and dl.get("error"):
|
| 116 |
m["note"] = T("ダウンロードに失敗: {0}", dl["error"])
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 117 |
models.append(m)
|
| 118 |
eng_dl = dl if dl and dl["key"] == "engine" else None
|
| 119 |
return {"version": __version__, "npu": self.npu or "", "npu_ok": lemonade.npu().get("arch") == "XDNA2",
|
|
@@ -144,7 +155,7 @@ class Manager:
|
|
| 144 |
return "missing"
|
| 145 |
|
| 146 |
# ---------------------------------------------------------------- downloads (a child process per download)
|
| 147 |
-
def _spawn(self, key, args):
|
| 148 |
log = os.path.join(app.config_dir(), "download.log")
|
| 149 |
status = os.path.join(app.config_dir(), "download.json")
|
| 150 |
try:
|
|
@@ -156,7 +167,7 @@ class Manager:
|
|
| 156 |
proc = subprocess.Popen([sys.executable, "-m", "onw.dl", *args, status], stdout=out,
|
| 157 |
stderr=subprocess.STDOUT, env={**os.environ, "PYTHONUTF8": "1"}, **kw)
|
| 158 |
self.dl = {"key": key, "proc": proc, "status": status, "frac": 0.0, "text": T("準備中…"), "error": None,
|
| 159 |
-
"done": None, "log": log}
|
| 160 |
|
| 161 |
def download(self, key):
|
| 162 |
cfg = app.load_config()
|
|
@@ -179,6 +190,40 @@ class Manager:
|
|
| 179 |
self._spawn(key, [key])
|
| 180 |
return {}
|
| 181 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 182 |
def install_engine(self):
|
| 183 |
if self.upd["state"] == "updating":
|
| 184 |
return {"error": T("onw の更新中です。終わってからダウンロードしてください")}
|
|
@@ -213,6 +258,7 @@ class Manager:
|
|
| 213 |
dl = self.dl
|
| 214 |
if not dl or dl.get("done"):
|
| 215 |
return
|
|
|
|
| 216 |
try:
|
| 217 |
st = json.load(open(dl["status"], encoding="utf8"))
|
| 218 |
dl["frac"], dl["text"] = min(float(st.get("frac", 0)), 0.99), st.get("text") or dl["text"]
|
|
@@ -227,12 +273,17 @@ class Manager:
|
|
| 227 |
except OSError:
|
| 228 |
out = ""
|
| 229 |
dl["error"] = " ".join(out.split()[-30:]) or f"exit code {rc}"
|
|
|
|
|
|
|
| 230 |
else:
|
|
|
|
|
|
|
|
|
|
| 231 |
dl["frac"] = 1.0
|
| 232 |
app.flm_catalog(refresh=True) # flm pulled a model / FastFlowLM was installed
|
| 233 |
if dl["key"] == "engine":
|
| 234 |
threading.Thread(target=self._check_npu, daemon=True).start()
|
| 235 |
-
if
|
| 236 |
with app._CFG_LOCK:
|
| 237 |
if not app.load_config().get("model"):
|
| 238 |
app.update_config(model=dl["key"])
|
|
@@ -241,6 +292,11 @@ class Manager:
|
|
| 241 |
def load(self, key):
|
| 242 |
if key not in app.installed(app.load_config()):
|
| 243 |
return {"error": T("そのモデルはありません")}
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 244 |
cfg = app.update_config(model=key)
|
| 245 |
self.busy, self.busy_err = T("ロードしています…"), False
|
| 246 |
|
|
@@ -393,6 +449,14 @@ class Manager:
|
|
| 393 |
threading.Thread(target=work, daemon=True).start()
|
| 394 |
return {}
|
| 395 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 396 |
def open(self, what, key=None):
|
| 397 |
cfg = app.load_config()
|
| 398 |
if what == "log":
|
|
@@ -464,6 +528,10 @@ class _Handler(BaseHTTPRequestHandler):
|
|
| 464 |
r = m.delete(body["key"])
|
| 465 |
elif act == "install_engine":
|
| 466 |
r = m.install_engine()
|
|
|
|
|
|
|
|
|
|
|
|
|
| 467 |
elif act == "save_dir":
|
| 468 |
r = m.save_dir(body["path"])
|
| 469 |
elif act == "settings":
|
|
@@ -472,6 +540,8 @@ class _Handler(BaseHTTPRequestHandler):
|
|
| 472 |
r = m.start_check()
|
| 473 |
elif act == "update":
|
| 474 |
r = m.run_update()
|
|
|
|
|
|
|
| 475 |
elif act == "open":
|
| 476 |
r = m.open(body.get("what"), body.get("key"))
|
| 477 |
else:
|
|
|
|
| 50 |
self.dl = None # {"key", "proc", "status", "frac", "text", "error", "done", "log"}
|
| 51 |
self.npu = None
|
| 52 |
self.check = None # lemonade.flm_check() + "text" (the reasons in the UI language)
|
| 53 |
+
self.damaged = {} # model key -> files a check found damaged (or [] = checked, intact)
|
| 54 |
self.ram = app.total_ram_gb()
|
| 55 |
self.busy = "" # a message about a long action (loading / deleting) for the page
|
| 56 |
self.busy_err = False
|
| 57 |
self.upd = {"latest": None, "tag": None, "notes": "", "checked": None, "state": "idle", "msg": "",
|
| 58 |
"checking": False, "error": ""}
|
| 59 |
self.on_updated = None # set by the tray: restart into the new version
|
| 60 |
+
self.on_restart = None # set by the tray: restart into the files the installer just replaced
|
| 61 |
self.on_update_found = None # set by the tray: notification
|
| 62 |
threading.Thread(target=self._update_loop, daemon=True).start()
|
| 63 |
threading.Thread(target=self._npu_info, daemon=True).start()
|
|
|
|
| 111 |
"mem_gb": mem, "fits": app.fits(mem, self.ram), "deletable": bool(path),
|
| 112 |
"note": "・".join(T(NOTES[l]) for l in c["labels"] if l in NOTES)}
|
| 113 |
m["state"] = self._state_of(key, path, info, status, dl)
|
| 114 |
+
bad = self.damaged.get(key)
|
| 115 |
if dl and dl["key"] == key and not dl.get("done"):
|
| 116 |
m["progress"] = dl["frac"]
|
| 117 |
+
if dl["kind"] == "verify":
|
| 118 |
+
m["state"], m["note"] = "verifying", T("ファイルを照合中 {0}", dl["text"].replace("checking", "").strip())
|
| 119 |
+
else:
|
| 120 |
+
m["note"] = T("ダウンロード中 {0}・ウィンドウを閉じても続きます", dl["text"].replace("checking", T("照合中")))
|
| 121 |
elif dl and dl.get("done") == key and dl.get("error"):
|
| 122 |
m["note"] = T("ダウンロードに失敗: {0}", dl["error"])
|
| 123 |
+
if bad and m["state"] in ("ready", "failed", "missing") and not (dl and dl["key"] == key and not dl.get("done")):
|
| 124 |
+
m["state"] = "damaged"
|
| 125 |
+
m["note"] = T("ファイルが壊れています({0})。「ダウンロードし直す」で壊れたファイルだけ取り直します", ", ".join(bad))
|
| 126 |
+
elif bad == [] and m["state"] == "ready" and not m.get("progress"):
|
| 127 |
+
m["note"] = T("ファイルを照合しました:問題ありません") + ("・" + m["note"] if m["note"] else "")
|
| 128 |
models.append(m)
|
| 129 |
eng_dl = dl if dl and dl["key"] == "engine" else None
|
| 130 |
return {"version": __version__, "npu": self.npu or "", "npu_ok": lemonade.npu().get("arch") == "XDNA2",
|
|
|
|
| 155 |
return "missing"
|
| 156 |
|
| 157 |
# ---------------------------------------------------------------- downloads (a child process per download)
|
| 158 |
+
def _spawn(self, key, args, kind="download"):
|
| 159 |
log = os.path.join(app.config_dir(), "download.log")
|
| 160 |
status = os.path.join(app.config_dir(), "download.json")
|
| 161 |
try:
|
|
|
|
| 167 |
proc = subprocess.Popen([sys.executable, "-m", "onw.dl", *args, status], stdout=out,
|
| 168 |
stderr=subprocess.STDOUT, env={**os.environ, "PYTHONUTF8": "1"}, **kw)
|
| 169 |
self.dl = {"key": key, "proc": proc, "status": status, "frac": 0.0, "text": T("準備中…"), "error": None,
|
| 170 |
+
"done": None, "log": log, "kind": kind}
|
| 171 |
|
| 172 |
def download(self, key):
|
| 173 |
cfg = app.load_config()
|
|
|
|
| 190 |
self._spawn(key, [key])
|
| 191 |
return {}
|
| 192 |
|
| 193 |
+
def _busy_error(self, key):
|
| 194 |
+
if self.upd["state"] == "updating":
|
| 195 |
+
return T("onw の更新中です。終わってからダウンロードしてください")
|
| 196 |
+
info = app.server_info()
|
| 197 |
+
if info and info.get("model") == key:
|
| 198 |
+
return T("このモデルはロード中です。先にアンロードしてください。")
|
| 199 |
+
if self.dl and self.dl["proc"].poll() is None:
|
| 200 |
+
return T("別のダウンロードが進行中です")
|
| 201 |
+
return None
|
| 202 |
+
|
| 203 |
+
def verify(self, key):
|
| 204 |
+
"""Check an installed model's files against FastFlowLM's hashes (a child process, like a download)."""
|
| 205 |
+
if key not in app.installed(app.load_config()):
|
| 206 |
+
return {"error": T("そのモデルはありません")}
|
| 207 |
+
with self.lock:
|
| 208 |
+
err = self._busy_error(key)
|
| 209 |
+
if err:
|
| 210 |
+
return {"error": err}
|
| 211 |
+
self.damaged.pop(key, None) # a stale result must not outlive a new check
|
| 212 |
+
self._spawn(key, ["--verify", key], "verify")
|
| 213 |
+
return {}
|
| 214 |
+
|
| 215 |
+
def repair(self, key):
|
| 216 |
+
"""Fetch a damaged model's broken files again (then check them)."""
|
| 217 |
+
if app.catalog_entry(key) is None:
|
| 218 |
+
return {"error": T("そのモデルはありません")}
|
| 219 |
+
with self.lock:
|
| 220 |
+
err = self._busy_error(key)
|
| 221 |
+
if err:
|
| 222 |
+
return {"error": err}
|
| 223 |
+
self.damaged.pop(key, None)
|
| 224 |
+
self._spawn(key, ["--repair", key], "repair")
|
| 225 |
+
return {}
|
| 226 |
+
|
| 227 |
def install_engine(self):
|
| 228 |
if self.upd["state"] == "updating":
|
| 229 |
return {"error": T("onw の更新中です。終わってからダウンロードしてください")}
|
|
|
|
| 258 |
dl = self.dl
|
| 259 |
if not dl or dl.get("done"):
|
| 260 |
return
|
| 261 |
+
st = {}
|
| 262 |
try:
|
| 263 |
st = json.load(open(dl["status"], encoding="utf8"))
|
| 264 |
dl["frac"], dl["text"] = min(float(st.get("frac", 0)), 0.99), st.get("text") or dl["text"]
|
|
|
|
| 273 |
except OSError:
|
| 274 |
out = ""
|
| 275 |
dl["error"] = " ".join(out.split()[-30:]) or f"exit code {rc}"
|
| 276 |
+
if "damaged" in out: # the download came down broken twice
|
| 277 |
+
self.damaged[dl["key"]] = ["?"]
|
| 278 |
else:
|
| 279 |
+
if dl["kind"] in ("verify", "repair", "download") and dl["key"] != "engine":
|
| 280 |
+
bad = st.get("bad")
|
| 281 |
+
self.damaged[dl["key"]] = bad if isinstance(bad, list) else []
|
| 282 |
dl["frac"] = 1.0
|
| 283 |
app.flm_catalog(refresh=True) # flm pulled a model / FastFlowLM was installed
|
| 284 |
if dl["key"] == "engine":
|
| 285 |
threading.Thread(target=self._check_npu, daemon=True).start()
|
| 286 |
+
if dl["key"] != "engine" and dl["kind"] == "download":
|
| 287 |
with app._CFG_LOCK:
|
| 288 |
if not app.load_config().get("model"):
|
| 289 |
app.update_config(model=dl["key"])
|
|
|
|
| 292 |
def load(self, key):
|
| 293 |
if key not in app.installed(app.load_config()):
|
| 294 |
return {"error": T("そのモデルはありません")}
|
| 295 |
+
m = app.catalog_entry(key)
|
| 296 |
+
bad = lemonade.flm_check_sizes(m["checkpoint"]) if m else []
|
| 297 |
+
if bad: # a file missing or cut short: never load broken weights
|
| 298 |
+
self.damaged[key] = bad
|
| 299 |
+
return {"error": T("{0} のファイルが壊れています({1})。「モデル」タブの「ダウンロードし直す」で直せます", key, ", ".join(bad))}
|
| 300 |
cfg = app.update_config(model=key)
|
| 301 |
self.busy, self.busy_err = T("ロードしています…"), False
|
| 302 |
|
|
|
|
| 449 |
threading.Thread(target=work, daemon=True).start()
|
| 450 |
return {}
|
| 451 |
|
| 452 |
+
def restart_tray(self):
|
| 453 |
+
if not self.on_restart:
|
| 454 |
+
return {"error": "no tray"}
|
| 455 |
+
if self.dl and self.dl["proc"].poll() is None: # the restart would orphan the download
|
| 456 |
+
return {"error": T("ダウンロードが終わってから更新してください")}
|
| 457 |
+
threading.Timer(0.5, self.on_restart).start() # answer first, then restart
|
| 458 |
+
return {}
|
| 459 |
+
|
| 460 |
def open(self, what, key=None):
|
| 461 |
cfg = app.load_config()
|
| 462 |
if what == "log":
|
|
|
|
| 528 |
r = m.delete(body["key"])
|
| 529 |
elif act == "install_engine":
|
| 530 |
r = m.install_engine()
|
| 531 |
+
elif act == "verify":
|
| 532 |
+
r = m.verify(body["key"])
|
| 533 |
+
elif act == "repair":
|
| 534 |
+
r = m.repair(body["key"])
|
| 535 |
elif act == "save_dir":
|
| 536 |
r = m.save_dir(body["path"])
|
| 537 |
elif act == "settings":
|
|
|
|
| 540 |
r = m.start_check()
|
| 541 |
elif act == "update":
|
| 542 |
r = m.run_update()
|
| 543 |
+
elif act == "restart_tray":
|
| 544 |
+
r = m.restart_tray()
|
| 545 |
elif act == "open":
|
| 546 |
r = m.open(body.get("what"), body.get("key"))
|
| 547 |
else:
|
onw/server.py
CHANGED
|
@@ -132,10 +132,13 @@ class ThinkSplit:
|
|
| 132 |
"""FastFlowLM writes the thinking into the content as <think>...</think>; OpenAI clients (and the chat UI) expect it
|
| 133 |
in reasoning_content. Fed the content piece by piece (a tag may be cut between two pieces), returns
|
| 134 |
(reasoning, content) for each piece."""
|
| 135 |
-
|
|
|
|
|
|
|
| 136 |
|
| 137 |
def __init__(self):
|
| 138 |
self.state, self.buf = "start", "" # start (nothing seen yet) -> think -> gap -> text, or start -> text
|
|
|
|
| 139 |
|
| 140 |
def feed(self, s, final=False):
|
| 141 |
self.buf += s
|
|
@@ -143,9 +146,11 @@ class ThinkSplit:
|
|
| 143 |
while True:
|
| 144 |
if self.state == "start": # only a leading <think> opens the thinking
|
| 145 |
head = self.buf.lstrip()
|
| 146 |
-
if head.startswith(
|
| 147 |
-
|
| 148 |
-
|
|
|
|
|
|
|
| 149 |
break # "", "<", "<thi": wait for more
|
| 150 |
else:
|
| 151 |
self.state = "text"
|
|
@@ -300,6 +305,10 @@ def resolve(key):
|
|
| 300 |
print(f"[setup] flm pull {m['checkpoint']} ...", flush=True)
|
| 301 |
if subprocess.run(lemonade.flm_pull_command(m["checkpoint"]), env=lemonade.flm_env(cfg)).returncode != 0:
|
| 302 |
sys.exit("ERROR: flm pull failed")
|
|
|
|
|
|
|
|
|
|
|
|
|
| 303 |
return m["checkpoint"], m["labels"]
|
| 304 |
|
| 305 |
|
|
|
|
| 132 |
"""FastFlowLM writes the thinking into the content as <think>...</think>; OpenAI clients (and the chat UI) expect it
|
| 133 |
in reasoning_content. Fed the content piece by piece (a tag may be cut between two pieces), returns
|
| 134 |
(reasoning, content) for each piece."""
|
| 135 |
+
# (opening, closing) tags: <think> (Qwen, DeepSeek, ...), Gemma 4's thought channel. flm splits the thinking off
|
| 136 |
+
# itself for some models, but not from an answer cut off by max_tokens while still thinking
|
| 137 |
+
TAGS = (("<think>", "</think>"), ("<|channel>thought", "<channel|>"))
|
| 138 |
|
| 139 |
def __init__(self):
|
| 140 |
self.state, self.buf = "start", "" # start (nothing seen yet) -> think -> gap -> text, or start -> text
|
| 141 |
+
self.CLOSE = self.TAGS[0][1]
|
| 142 |
|
| 143 |
def feed(self, s, final=False):
|
| 144 |
self.buf += s
|
|
|
|
| 146 |
while True:
|
| 147 |
if self.state == "start": # only a leading <think> opens the thinking
|
| 148 |
head = self.buf.lstrip()
|
| 149 |
+
hit = next(((o, c) for o, c in self.TAGS if head.startswith(o)), None)
|
| 150 |
+
if hit:
|
| 151 |
+
self.CLOSE = hit[1]
|
| 152 |
+
self.state, self.buf = "think", head[len(hit[0]):]
|
| 153 |
+
elif any(o.startswith(head) for o, _ in self.TAGS) and not final:
|
| 154 |
break # "", "<", "<thi": wait for more
|
| 155 |
else:
|
| 156 |
self.state = "text"
|
|
|
|
| 305 |
print(f"[setup] flm pull {m['checkpoint']} ...", flush=True)
|
| 306 |
if subprocess.run(lemonade.flm_pull_command(m["checkpoint"]), env=lemonade.flm_env(cfg)).returncode != 0:
|
| 307 |
sys.exit("ERROR: flm pull failed")
|
| 308 |
+
bad = lemonade.flm_check_sizes(m["checkpoint"], cfg)
|
| 309 |
+
if bad: # a file missing or cut short: broken weights make garbage
|
| 310 |
+
sys.exit(f"ERROR: the model's files are damaged ({', '.join(bad)}); download it again "
|
| 311 |
+
"(onw window: Models -> Download again)")
|
| 312 |
return m["checkpoint"], m["labels"]
|
| 313 |
|
| 314 |
|
onw/tray.py
CHANGED
|
@@ -40,10 +40,61 @@ def copy_text(text):
|
|
| 40 |
return False
|
| 41 |
|
| 42 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 43 |
def main(argv=None):
|
|
|
|
| 44 |
if not app.single_instance("tray"): # already in the tray (e.g. started again from the menu)
|
| 45 |
-
|
| 46 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 47 |
import pystray
|
| 48 |
from . import manager as mgr
|
| 49 |
app.clear_stale_server() # a pid file from before a crash / reboot
|
|
@@ -109,6 +160,7 @@ def main(argv=None):
|
|
| 109 |
open_window("server")
|
| 110 |
|
| 111 |
manager.on_updated = lambda: restart_after_update()
|
|
|
|
| 112 |
manager.on_update_found = update_found
|
| 113 |
running = lambda _: state["s"] == "running"
|
| 114 |
menu = pystray.Menu(
|
|
|
|
| 40 |
return False
|
| 41 |
|
| 42 |
|
| 43 |
+
def replace_running():
|
| 44 |
+
"""The installer ran again while a tray is running (an update by the one-line installer): ask that tray to restart
|
| 45 |
+
into the new files, as the in-app update does (the model that was running is loaded again). False when no
|
| 46 |
+
tray answered."""
|
| 47 |
+
import urllib.request
|
| 48 |
+
from . import manager as mgr
|
| 49 |
+
d = mgr.find_manager()
|
| 50 |
+
if not d:
|
| 51 |
+
return False
|
| 52 |
+
try:
|
| 53 |
+
req = urllib.request.Request(f"http://127.0.0.1:{d['port']}/api/restart_tray", b"{}",
|
| 54 |
+
{"X-Onw-Token": d["token"], "Content-Type": "application/json"})
|
| 55 |
+
return "error" not in json.loads(urllib.request.urlopen(req, timeout=10).read())
|
| 56 |
+
except Exception:
|
| 57 |
+
return False
|
| 58 |
+
|
| 59 |
+
|
| 60 |
+
def take_over():
|
| 61 |
+
"""A running tray that cannot restart itself (older than 0.1.3): stop it and its server, remember the model for the
|
| 62 |
+
new tray to load again, and wait for its single-instance lock. True once the lock is ours."""
|
| 63 |
+
from . import manager as mgr
|
| 64 |
+
info = app.server_info()
|
| 65 |
+
with open(os.path.join(app.config_dir(), "resume.json"), "w") as f:
|
| 66 |
+
json.dump({"model": info["model"] if info else None}, f)
|
| 67 |
+
app.stop_server()
|
| 68 |
+
d = mgr.find_manager()
|
| 69 |
+
if d:
|
| 70 |
+
try:
|
| 71 |
+
import psutil
|
| 72 |
+
p = psutil.Process(d["pid"])
|
| 73 |
+
procs = p.children(recursive=True) + [p]
|
| 74 |
+
for q in procs:
|
| 75 |
+
q.terminate()
|
| 76 |
+
psutil.wait_procs(procs, timeout=10)
|
| 77 |
+
except Exception:
|
| 78 |
+
pass
|
| 79 |
+
for _ in range(40):
|
| 80 |
+
if app.single_instance("tray"):
|
| 81 |
+
return True
|
| 82 |
+
time.sleep(0.5)
|
| 83 |
+
return False
|
| 84 |
+
|
| 85 |
+
|
| 86 |
def main(argv=None):
|
| 87 |
+
argv = argv or []
|
| 88 |
if not app.single_instance("tray"): # already in the tray (e.g. started again from the menu)
|
| 89 |
+
if "--replace" in argv: # the installer ran again: the files are new
|
| 90 |
+
if replace_running(): # the old tray restarts itself on the new code
|
| 91 |
+
return
|
| 92 |
+
if not take_over(): # an older tray: stop it and start here instead
|
| 93 |
+
open_window()
|
| 94 |
+
return
|
| 95 |
+
else:
|
| 96 |
+
open_window()
|
| 97 |
+
return
|
| 98 |
import pystray
|
| 99 |
from . import manager as mgr
|
| 100 |
app.clear_stale_server() # a pid file from before a crash / reboot
|
|
|
|
| 160 |
open_window("server")
|
| 161 |
|
| 162 |
manager.on_updated = lambda: restart_after_update()
|
| 163 |
+
manager.on_restart = lambda: restart_after_update() # the installer ran again (onw tray --replace)
|
| 164 |
manager.on_update_found = update_found
|
| 165 |
running = lambda _: state["s"] == "running"
|
| 166 |
menu = pystray.Menu(
|
onw/web/manage.html
CHANGED
|
@@ -73,7 +73,8 @@
|
|
| 73 |
tbody tr.loading { background: var(--warn-soft); box-shadow: inset 3px 0 0 var(--warn); }
|
| 74 |
tbody tr.failed { background: var(--danger-soft); box-shadow: inset 3px 0 0 var(--danger); }
|
| 75 |
tbody tr.nofit { background: var(--danger-soft); }
|
| 76 |
-
tbody tr.downloading { background: linear-gradient(90deg, var(--accent-soft) var(--p), transparent var(--p)); }
|
|
|
|
| 77 |
td.name { font-weight: 600; }
|
| 78 |
td.name .sub { display: block; font-weight: 400; color: var(--muted); font-size: 11px; max-width: 240px; overflow: hidden; text-overflow: ellipsis; }
|
| 79 |
td.acts { text-align: right; }
|
|
@@ -281,7 +282,7 @@
|
|
| 281 |
}
|
| 282 |
|
| 283 |
// ---- render from /api/state
|
| 284 |
-
const PILL = {live: ["ok", T("使用中")], loading: ["warn", T("ロード中…")], failed: ["danger", T("ロード失敗")], ready: ["accent", T("取得済み")],
|
| 285 |
};
|
| 286 |
function render() {
|
| 287 |
const s = S, sv = s.server, st = sv.status;
|
|
@@ -340,12 +341,12 @@
|
|
| 340 |
$("raminfo").textContent = s.ram_gb ? T("この PC のメモリ {0} GB ・ メモリ目安は今の設定での値", Math.round(s.ram_gb)) : "";
|
| 341 |
const rowsHtml = s.models.map(m => {
|
| 342 |
const cls = m.state + ((!m.fits && m.state !== "live") ? " nofit" : "");
|
| 343 |
-
const style = m.state === "downloading" ? ` style="--p:${Math.round((m.progress || 0) * 100)}%"` : "";
|
| 344 |
let note = m.note || "";
|
| 345 |
if (!m.fits && m.mem_gb && m.state !== "downloading") note += T("・メモリ 約{0} GB はこの PC では不足の可能性", Math.round(m.mem_gb));
|
| 346 |
const acts = actions(m);
|
| 347 |
let pill = "";
|
| 348 |
-
if (m.state === "downloading") pill = `<span class="pill accent">${Math.round((m.progress || 0) * 100)}%</span>`;
|
| 349 |
else if (m.state === "missing") pill = `<span class="muted small">${T("未取得")}</span>`;
|
| 350 |
else if (PILL[m.state]) pill = `<span class="pill ${PILL[m.state][0]}">${PILL[m.state][1]}</span>`;
|
| 351 |
const cols = `<td class="c${m.image ? "" : " muted"}">${m.image ? T("可") : "—"}</td><td class="c">${m.size_gb ? m.size_gb.toFixed(1) + " GB" : "—"}</td><td class="c${m.fits ? "" : " bad"}">${m.mem_gb ? T("約{0} GB", Math.round(m.mem_gb)) : "—"}</td>`;
|
|
@@ -365,7 +366,8 @@
|
|
| 365 |
switch (m.state) {
|
| 366 |
case "live": return `<button disabled>${T("ロード済み")}</button>${more}`;
|
| 367 |
case "loading": return `<button disabled>${T("ロード中…")}</button>${more}`;
|
| 368 |
-
case "downloading": return `<button onclick="post('cancel_download')">${T("キャンセル")}</button>`;
|
|
|
|
| 369 |
case "missing": return `<button class="primary"${S.npu_ok ? "" : " disabled"} onclick="post('download',{key:'${k}'})">${T("ダウンロード")}</button>`;
|
| 370 |
default: return `<button class="primary" onclick="post('load',{key:'${k}'})">${S.server.model ? T("切り替え") : T("ロード")}</button>${more}`;
|
| 371 |
}
|
|
@@ -375,6 +377,7 @@
|
|
| 375 |
const m = S.models.find(x => x.key === key), mn = $("menu");
|
| 376 |
const k = esc(key);
|
| 377 |
mn.innerHTML = `<button onclick="post('open',{what:'folder',key:'${k}'})">${T("フォルダを開く")}</button>` +
|
|
|
|
| 378 |
(m.deletable ? `<button class="danger" onclick="deleteModel('${k}')">${T("モデルを削除")}</button>`
|
| 379 |
: `<button disabled>${T("削除(onw が入れたモデルではありません)")}</button>`);
|
| 380 |
const r = ev.currentTarget.getBoundingClientRect();
|
|
|
|
| 73 |
tbody tr.loading { background: var(--warn-soft); box-shadow: inset 3px 0 0 var(--warn); }
|
| 74 |
tbody tr.failed { background: var(--danger-soft); box-shadow: inset 3px 0 0 var(--danger); }
|
| 75 |
tbody tr.nofit { background: var(--danger-soft); }
|
| 76 |
+
tbody tr.downloading, tbody tr.verifying { background: linear-gradient(90deg, var(--accent-soft) var(--p), transparent var(--p)); }
|
| 77 |
+
tbody tr.damaged { background: var(--danger-soft); box-shadow: inset 3px 0 0 var(--danger); }
|
| 78 |
td.name { font-weight: 600; }
|
| 79 |
td.name .sub { display: block; font-weight: 400; color: var(--muted); font-size: 11px; max-width: 240px; overflow: hidden; text-overflow: ellipsis; }
|
| 80 |
td.acts { text-align: right; }
|
|
|
|
| 282 |
}
|
| 283 |
|
| 284 |
// ---- render from /api/state
|
| 285 |
+
const PILL = {live: ["ok", T("使用中")], loading: ["warn", T("ロード中…")], failed: ["danger", T("ロード失敗")], ready: ["accent", T("取得済み")], damaged: ["danger", T("壊れています")],
|
| 286 |
};
|
| 287 |
function render() {
|
| 288 |
const s = S, sv = s.server, st = sv.status;
|
|
|
|
| 341 |
$("raminfo").textContent = s.ram_gb ? T("この PC のメモリ {0} GB ・ メモリ目安は今の設定での値", Math.round(s.ram_gb)) : "";
|
| 342 |
const rowsHtml = s.models.map(m => {
|
| 343 |
const cls = m.state + ((!m.fits && m.state !== "live") ? " nofit" : "");
|
| 344 |
+
const style = m.state === "downloading" || m.state === "verifying" ? ` style="--p:${Math.round((m.progress || 0) * 100)}%"` : "";
|
| 345 |
let note = m.note || "";
|
| 346 |
if (!m.fits && m.mem_gb && m.state !== "downloading") note += T("・メモリ 約{0} GB はこの PC では不足の可能性", Math.round(m.mem_gb));
|
| 347 |
const acts = actions(m);
|
| 348 |
let pill = "";
|
| 349 |
+
if (m.state === "downloading" || m.state === "verifying") pill = `<span class="pill accent">${Math.round((m.progress || 0) * 100)}%</span>`;
|
| 350 |
else if (m.state === "missing") pill = `<span class="muted small">${T("未取得")}</span>`;
|
| 351 |
else if (PILL[m.state]) pill = `<span class="pill ${PILL[m.state][0]}">${PILL[m.state][1]}</span>`;
|
| 352 |
const cols = `<td class="c${m.image ? "" : " muted"}">${m.image ? T("可") : "—"}</td><td class="c">${m.size_gb ? m.size_gb.toFixed(1) + " GB" : "—"}</td><td class="c${m.fits ? "" : " bad"}">${m.mem_gb ? T("約{0} GB", Math.round(m.mem_gb)) : "—"}</td>`;
|
|
|
|
| 366 |
switch (m.state) {
|
| 367 |
case "live": return `<button disabled>${T("ロード済み")}</button>${more}`;
|
| 368 |
case "loading": return `<button disabled>${T("ロード中…")}</button>${more}`;
|
| 369 |
+
case "downloading": case "verifying": return `<button onclick="post('cancel_download')">${T("キャンセル")}</button>`;
|
| 370 |
+
case "damaged": return `<button class="primary" onclick="post('repair',{key:'${k}'})">${T("ダウンロードし直す")}</button>${more}`;
|
| 371 |
case "missing": return `<button class="primary"${S.npu_ok ? "" : " disabled"} onclick="post('download',{key:'${k}'})">${T("ダウンロード")}</button>`;
|
| 372 |
default: return `<button class="primary" onclick="post('load',{key:'${k}'})">${S.server.model ? T("切り替え") : T("ロード")}</button>${more}`;
|
| 373 |
}
|
|
|
|
| 377 |
const m = S.models.find(x => x.key === key), mn = $("menu");
|
| 378 |
const k = esc(key);
|
| 379 |
mn.innerHTML = `<button onclick="post('open',{what:'folder',key:'${k}'})">${T("フォルダを開く")}</button>` +
|
| 380 |
+
(m.path ? `<button onclick="post('verify',{key:'${k}'})">${T("ファイルを照合")}</button>` : "") +
|
| 381 |
(m.deletable ? `<button class="danger" onclick="deleteModel('${k}')">${T("モデルを削除")}</button>`
|
| 382 |
: `<button disabled>${T("削除(onw が入れたモデルではありません)")}</button>`);
|
| 383 |
const r = ev.currentTarget.getBoundingClientRect();
|
pyproject.toml
CHANGED
|
@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
|
|
| 4 |
|
| 5 |
[project]
|
| 6 |
name = "onw-amd"
|
| 7 |
-
version = "0.1.
|
| 8 |
description = "Ore no NPU ga konna ni ugoku wake nai - LLMs entirely on the AMD Ryzen AI NPU (FastFlowLM, as in Lemonade Server)"
|
| 9 |
readme = "README.md"
|
| 10 |
license = {text = "Apache-2.0"}
|
|
|
|
| 4 |
|
| 5 |
[project]
|
| 6 |
name = "onw-amd"
|
| 7 |
+
version = "0.1.3"
|
| 8 |
description = "Ore no NPU ga konna ni ugoku wake nai - LLMs entirely on the AMD Ryzen AI NPU (FastFlowLM, as in Lemonade Server)"
|
| 9 |
readme = "README.md"
|
| 10 |
license = {text = "Apache-2.0"}
|
setup.bat
CHANGED
|
@@ -1,56 +1,57 @@
|
|
| 1 |
-
@echo off
|
| 2 |
-
rem onw AMD - Ore no NPU ga konna ni ugoku wake nai: install the resident tray app (Windows, AMD Ryzen AI NPU).
|
| 3 |
-
rem Finds Python 3.10+ (offers to install it with winget), creates .venv, installs onw + the tray, adds "onw" to
|
| 4 |
-
rem the Start menu and starts it in the task tray. FastFlowLM and the models are downloaded later from the
|
| 5 |
-
rem onw window (Models tab).
|
| 6 |
-
setlocal
|
| 7 |
-
cd /d "%~dp0"
|
| 8 |
-
set PYTHONUTF8=1
|
| 9 |
-
|
| 10 |
-
if not exist ".venv\installed-tray.txt" goto create
|
| 11 |
-
rem an older install has the tray but not the window's web view package: install it
|
| 12 |
-
".venv\Scripts\python.exe" -c "import webview" >nul 2>&1 && goto shortcut
|
| 13 |
-
:create
|
| 14 |
-
if exist ".venv\Scripts\python.exe" goto install
|
| 15 |
-
call "%~dp0find_python.bat"
|
| 16 |
-
if not defined PY (
|
| 17 |
-
pause
|
| 18 |
-
exit /b 1
|
| 19 |
-
)
|
| 20 |
-
echo [setup] creating Python environment in .venv ...
|
| 21 |
-
%PY% -m venv .venv
|
| 22 |
-
if errorlevel 1 (
|
| 23 |
-
echo [setup] could not create the Python environment.
|
| 24 |
-
pause
|
| 25 |
-
exit /b 1
|
| 26 |
-
)
|
| 27 |
-
:install
|
| 28 |
-
rem (the bundled pip is fine; a newer pip once failed to install packages that have scripts, so it is not upgraded)
|
| 29 |
-
".venv\Scripts\python.exe" -m pip install -e .[tray]
|
| 30 |
-
if not errorlevel 1 goto installed
|
| 31 |
-
if defined RETRIED (
|
| 32 |
-
echo [setup] package installation failed.
|
| 33 |
-
pause
|
| 34 |
-
exit /b 1
|
| 35 |
-
)
|
| 36 |
-
echo [setup] installation failed; rebuilding the Python environment once ...
|
| 37 |
-
set RETRIED=1
|
| 38 |
-
rmdir /s /q .venv
|
| 39 |
-
if not defined PY call "%~dp0find_python.bat"
|
| 40 |
-
%PY% -m venv .venv
|
| 41 |
-
goto install
|
| 42 |
-
:installed
|
| 43 |
-
echo ok> ".venv\installed.txt"
|
| 44 |
-
echo ok> ".venv\installed-tray.txt"
|
| 45 |
-
|
| 46 |
-
:shortcut
|
| 47 |
-
rem Start menu entry: pythonw -m onw tray (no console window)
|
| 48 |
-
set "PYW=%CD%\.venv\Scripts\pythonw.exe"
|
| 49 |
-
powershell -NoProfile -Command ^
|
| 50 |
-
"$s = (New-Object -ComObject WScript.Shell).CreateShortcut([Environment]::GetFolderPath('Programs') + '\onw AMD.lnk');" ^
|
| 51 |
-
"$s.TargetPath = '%PYW%'; $s.Arguments = '-m onw tray'; $s.WorkingDirectory = '%CD%';" ^
|
| 52 |
-
"$s.Description = 'onw AMD - LLMs on the AMD Ryzen AI NPU'; $s.IconLocation = '%CD%\onw\web\icon.ico,0'; $s.Save()"
|
| 53 |
-
echo [setup] Start menu: onw AMD
|
| 54 |
-
|
| 55 |
-
echo [start] onw AMD is starting in the task tray. Pick a model to download in the window that opens.
|
| 56 |
-
|
|
|
|
|
|
| 1 |
+
@echo off
|
| 2 |
+
rem onw AMD - Ore no NPU ga konna ni ugoku wake nai: install the resident tray app (Windows, AMD Ryzen AI NPU).
|
| 3 |
+
rem Finds Python 3.10+ (offers to install it with winget), creates .venv, installs onw + the tray, adds "onw" to
|
| 4 |
+
rem the Start menu and starts it in the task tray. FastFlowLM and the models are downloaded later from the
|
| 5 |
+
rem onw window (Models tab).
|
| 6 |
+
setlocal
|
| 7 |
+
cd /d "%~dp0"
|
| 8 |
+
set PYTHONUTF8=1
|
| 9 |
+
|
| 10 |
+
if not exist ".venv\installed-tray.txt" goto create
|
| 11 |
+
rem an older install has the tray but not the window's web view package: install it
|
| 12 |
+
".venv\Scripts\python.exe" -c "import webview" >nul 2>&1 && goto shortcut
|
| 13 |
+
:create
|
| 14 |
+
if exist ".venv\Scripts\python.exe" goto install
|
| 15 |
+
call "%~dp0find_python.bat"
|
| 16 |
+
if not defined PY (
|
| 17 |
+
pause
|
| 18 |
+
exit /b 1
|
| 19 |
+
)
|
| 20 |
+
echo [setup] creating Python environment in .venv ...
|
| 21 |
+
%PY% -m venv .venv
|
| 22 |
+
if errorlevel 1 (
|
| 23 |
+
echo [setup] could not create the Python environment.
|
| 24 |
+
pause
|
| 25 |
+
exit /b 1
|
| 26 |
+
)
|
| 27 |
+
:install
|
| 28 |
+
rem (the bundled pip is fine; a newer pip once failed to install packages that have scripts, so it is not upgraded)
|
| 29 |
+
".venv\Scripts\python.exe" -m pip install -e .[tray]
|
| 30 |
+
if not errorlevel 1 goto installed
|
| 31 |
+
if defined RETRIED (
|
| 32 |
+
echo [setup] package installation failed.
|
| 33 |
+
pause
|
| 34 |
+
exit /b 1
|
| 35 |
+
)
|
| 36 |
+
echo [setup] installation failed; rebuilding the Python environment once ...
|
| 37 |
+
set RETRIED=1
|
| 38 |
+
rmdir /s /q .venv
|
| 39 |
+
if not defined PY call "%~dp0find_python.bat"
|
| 40 |
+
%PY% -m venv .venv
|
| 41 |
+
goto install
|
| 42 |
+
:installed
|
| 43 |
+
echo ok> ".venv\installed.txt"
|
| 44 |
+
echo ok> ".venv\installed-tray.txt"
|
| 45 |
+
|
| 46 |
+
:shortcut
|
| 47 |
+
rem Start menu entry: pythonw -m onw tray (no console window)
|
| 48 |
+
set "PYW=%CD%\.venv\Scripts\pythonw.exe"
|
| 49 |
+
powershell -NoProfile -Command ^
|
| 50 |
+
"$s = (New-Object -ComObject WScript.Shell).CreateShortcut([Environment]::GetFolderPath('Programs') + '\onw AMD.lnk');" ^
|
| 51 |
+
"$s.TargetPath = '%PYW%'; $s.Arguments = '-m onw tray'; $s.WorkingDirectory = '%CD%';" ^
|
| 52 |
+
"$s.Description = 'onw AMD - LLMs on the AMD Ryzen AI NPU'; $s.IconLocation = '%CD%\onw\web\icon.ico,0'; $s.Save()"
|
| 53 |
+
echo [setup] Start menu: onw AMD
|
| 54 |
+
|
| 55 |
+
echo [start] onw AMD is starting in the task tray. Pick a model to download in the window that opens.
|
| 56 |
+
rem --replace: a tray already running (an update by the one-line installer) restarts on the new files
|
| 57 |
+
start "" "%PYW%" -m onw tray --replace
|
setup.sh
CHANGED
|
@@ -82,4 +82,5 @@ if [ -n "$relogin" ]; then
|
|
| 82 |
fi
|
| 83 |
|
| 84 |
echo "[start] onw AMD is starting in the top bar ..."
|
| 85 |
-
|
|
|
|
|
|
| 82 |
fi
|
| 83 |
|
| 84 |
echo "[start] onw AMD is starting in the top bar ..."
|
| 85 |
+
# --replace: a tray already running (an update by the one-line installer) restarts on the new files
|
| 86 |
+
nohup .venv/bin/onw tray --replace >/dev/null 2>&1 &
|
start.bat
CHANGED
|
@@ -1,52 +1,52 @@
|
|
| 1 |
-
@echo off
|
| 2 |
-
rem onw AMD - Ore no NPU ga konna ni ugoku wake nai: run the server once from a console (Windows).
|
| 3 |
-
rem start.bat qwen3-it-4b-FLM (default)
|
| 4 |
-
rem start.bat llama3.2-3b-FLM any model name from "onw list"
|
| 5 |
-
rem start.bat MODEL --port 8080 further options go to "onw serve"
|
| 6 |
-
rem First run: finds / installs Python, creates .venv, downloads FastFlowLM and the model.
|
| 7 |
-
rem For the resident tray app with a model manager use setup.bat instead.
|
| 8 |
-
setlocal
|
| 9 |
-
cd /d "%~dp0"
|
| 10 |
-
set PYTHONUTF8=1
|
| 11 |
-
set "MODEL=qwen3-it-4b-FLM"
|
| 12 |
-
set "FIRST=%~1"
|
| 13 |
-
if not "%FIRST%"=="" if not "%FIRST:~0,1%"=="-" (
|
| 14 |
-
set "MODEL=%FIRST%"
|
| 15 |
-
shift
|
| 16 |
-
)
|
| 17 |
-
|
| 18 |
-
if exist ".venv\installed.txt" goto args
|
| 19 |
-
if exist ".venv\Scripts\python.exe" goto install
|
| 20 |
-
call "%~dp0find_python.bat"
|
| 21 |
-
if not defined PY (
|
| 22 |
-
pause
|
| 23 |
-
exit /b 1
|
| 24 |
-
)
|
| 25 |
-
echo [setup] creating Python environment in .venv ...
|
| 26 |
-
%PY% -m venv .venv
|
| 27 |
-
if errorlevel 1 (
|
| 28 |
-
echo [setup] could not create the Python environment.
|
| 29 |
-
pause
|
| 30 |
-
exit /b 1
|
| 31 |
-
)
|
| 32 |
-
:install
|
| 33 |
-
".venv\Scripts\python.exe" -m pip install -U pip
|
| 34 |
-
".venv\Scripts\python.exe" -m pip install -e .
|
| 35 |
-
if errorlevel 1 (
|
| 36 |
-
echo [setup] package installation failed.
|
| 37 |
-
pause
|
| 38 |
-
exit /b 1
|
| 39 |
-
)
|
| 40 |
-
echo ok> ".venv\installed.txt"
|
| 41 |
-
|
| 42 |
-
:args
|
| 43 |
-
set "ARGS="
|
| 44 |
-
:collect
|
| 45 |
-
if "%~1"=="" goto run
|
| 46 |
-
set "ARGS=%ARGS% %1"
|
| 47 |
-
shift
|
| 48 |
-
goto collect
|
| 49 |
-
:run
|
| 50 |
-
echo [start] %MODEL% (first time: NPU engine + model download, please wait) ...
|
| 51 |
-
".venv\Scripts\onw.exe" serve %MODEL% --open %ARGS%
|
| 52 |
-
pause
|
|
|
|
| 1 |
+
@echo off
|
| 2 |
+
rem onw AMD - Ore no NPU ga konna ni ugoku wake nai: run the server once from a console (Windows).
|
| 3 |
+
rem start.bat qwen3-it-4b-FLM (default)
|
| 4 |
+
rem start.bat llama3.2-3b-FLM any model name from "onw list"
|
| 5 |
+
rem start.bat MODEL --port 8080 further options go to "onw serve"
|
| 6 |
+
rem First run: finds / installs Python, creates .venv, downloads FastFlowLM and the model.
|
| 7 |
+
rem For the resident tray app with a model manager use setup.bat instead.
|
| 8 |
+
setlocal
|
| 9 |
+
cd /d "%~dp0"
|
| 10 |
+
set PYTHONUTF8=1
|
| 11 |
+
set "MODEL=qwen3-it-4b-FLM"
|
| 12 |
+
set "FIRST=%~1"
|
| 13 |
+
if not "%FIRST%"=="" if not "%FIRST:~0,1%"=="-" (
|
| 14 |
+
set "MODEL=%FIRST%"
|
| 15 |
+
shift
|
| 16 |
+
)
|
| 17 |
+
|
| 18 |
+
if exist ".venv\installed.txt" goto args
|
| 19 |
+
if exist ".venv\Scripts\python.exe" goto install
|
| 20 |
+
call "%~dp0find_python.bat"
|
| 21 |
+
if not defined PY (
|
| 22 |
+
pause
|
| 23 |
+
exit /b 1
|
| 24 |
+
)
|
| 25 |
+
echo [setup] creating Python environment in .venv ...
|
| 26 |
+
%PY% -m venv .venv
|
| 27 |
+
if errorlevel 1 (
|
| 28 |
+
echo [setup] could not create the Python environment.
|
| 29 |
+
pause
|
| 30 |
+
exit /b 1
|
| 31 |
+
)
|
| 32 |
+
:install
|
| 33 |
+
".venv\Scripts\python.exe" -m pip install -U pip
|
| 34 |
+
".venv\Scripts\python.exe" -m pip install -e .
|
| 35 |
+
if errorlevel 1 (
|
| 36 |
+
echo [setup] package installation failed.
|
| 37 |
+
pause
|
| 38 |
+
exit /b 1
|
| 39 |
+
)
|
| 40 |
+
echo ok> ".venv\installed.txt"
|
| 41 |
+
|
| 42 |
+
:args
|
| 43 |
+
set "ARGS="
|
| 44 |
+
:collect
|
| 45 |
+
if "%~1"=="" goto run
|
| 46 |
+
set "ARGS=%ARGS% %1"
|
| 47 |
+
shift
|
| 48 |
+
goto collect
|
| 49 |
+
:run
|
| 50 |
+
echo [start] %MODEL% (first time: NPU engine + model download, please wait) ...
|
| 51 |
+
".venv\Scripts\onw.exe" serve %MODEL% --open %ARGS%
|
| 52 |
+
pause
|