Publish reviewed HTTP evaluation and final deployment verification

README.md

@@ -41,3 +41,7 @@ Browser CORS applies. Redirects require the final URL directly: browsers hide re
 Source is under `app/`. Rebuild with `cd app && npm ci && npm test && npm run build`, then `python3 scripts/stage.py`. The HTTP implementation is in `app/src/browser-http.mjs`; it uses just-bash's existing curl commands and custom transport hook. Tests and results are in `validation/`.
 
 The small 4-bit model can make mistakes. Shell transport checks and actual-model trials are reported separately; they do not establish general agent reliability or physical-phone performance.
+
+## Deployed results
+
+The [evaluation report](validation/REPORT.md) separates 15 passing shell checks from seven real-model trials (five fully passed; two partial results). Basic API fetching works well. Duplicate POSTs and excessive retries on blocked requests remain model limitations.

app/tests/http-embedded.mjs

@@ -0,0 +1,23 @@
+import { chromium } from 'playwright';
+import fs from 'node:fs/promises';
+const root = new URL('../..', import.meta.url).pathname;
+const result = { date: new Date().toISOString(), url: 'https://huggingface.co/spaces/Mike0021/MiniCPM5-2B-WebGPU-Pi-HTTP' };
+const browser = await chromium.launch({ executablePath: '/usr/bin/chromium', headless: true });
+try {
+  const page = await browser.newPage();
+  await page.goto(result.url, { waitUntil: 'domcontentloaded' });
+  // A FrameLocator survives the Hub replacing/navigating its initial iframe.
+  const app = page.frameLocator('iframe[src*="mike0021-minicpm5-2b-webgpu-pi-http.static.hf.space"]');
+  // Wait for event handlers/worker initialization, not merely the static HTML.
+  await app.locator('#files button').first().waitFor();
+  await app.locator('body').evaluate(() => window.browserPi.ready);
+  await app.locator('#command').fill('curl -fsS https://jsonplaceholder.typicode.com/todos/1 -o embedded.json && jq -r .title embedded.json');
+  await app.locator('#shell-run').click();
+  await app.locator('.shell-entry:last-child .shell-status.success').waitFor();
+  result.stdout = await app.locator('.shell-entry:last-child .shell-stdout').textContent();
+  result.storageNotice = await app.locator('#notice').textContent();
+  result.openInNewTabOffered = await app.locator('#open-tab').isVisible();
+  result.passed = result.stdout.includes('delectus aut autem');
+  console.log(JSON.stringify(result));
+} catch (error) { result.failure = error.message; process.exitCode = 1; console.log('FAIL', error.message); }
+finally { await fs.writeFile(root + '/validation/http-embedded.json', JSON.stringify(result, null, 2) + '\n'); await browser.close(); }

scripts/stage.py

@@ -12,7 +12,7 @@ shutil.copytree(root / 'scripts', out / 'scripts')
 (out / 'validation').mkdir()
 for path in (root / 'validation').glob('*'):
     # Keep temporary failures and screenshots containing HTTP echo metadata local.
-    if path.is_file() and path.name in ['source.json', 'deployment.json', 'http-shell-live.json', 'http-agent-live.json', 'http-embedded.json', 'REPORT.md', 'http-desktop.png', 'http-mobile.png', 'http-agent-fetch-count-write.png']:
+    if path.is_file() and path.name in ['source.json', 'deployment.json', 'http-shell-live.json', 'http-agent-live.json', 'http-final-smoke.json', 'http-embedded.json', 'REPORT.md', 'http-desktop.png', 'http-mobile.png', 'http-agent-fetch-count-write.png']:
         destination = out / 'validation' / path.name
         if path.suffix == '.json':
             data = path.read_text()

validation/REPORT.md

@@ -0,0 +1,33 @@
+# Browser HTTP experiment — deployed evaluation
+
+Space: https://huggingface.co/spaces/Mike0021/MiniCPM5-2B-WebGPU-Pi-HTTP
+
+Tested September 16, 2026, in real desktop Chromium 152 with an AMD RDNA-3 WebGPU adapter. The app, shell, actual MiniCPM5-2B q4f16 inference, and real public API responses were used; model/tool output was not mocked. Mobile layout was checked at 390×844; no physical-phone performance claim is made.
+
+## Transport: works well within browser limits
+
+All **15 live shell checks passed**: JSON download/file persistence, GitHub and Hugging Face metadata, raw GitHub content, HEAD, JSON POST, custom headers, HTTP 404, denied hosts, CORS/network failure, redirects, timeout, failed-pipeline status, and cancellation. **22 unit tests passed**, including streamed size limits and cancellation. The production build passed. Requests run in the browser worker with no proxy. The same download also passed through the real Hugging Face iframe UI. This Chromium configuration denied embedded persistent storage; the existing Open workspace in a new tab fallback was visible and the standalone app worked.
+
+The experimental allowlist contains api.github.com, raw.githubusercontent.com, huggingface.co, jsonplaceholder.typicode.com, and httpbin.org. GET/HEAD/POST are supported; cookies are omitted; responses are limited to 1 MiB and requests to eight seconds. The installed browser bundle provides curl, not wget. These limits are explicit in the UI help and tool description.
+
+Other hosts are rejected. Allowed hosts still need CORS headers: Hugging Face's model API worked, while its homepage could not be read. Browser-manual redirects have opaque status/headers, so the adapter rejects them clearly; use a final, correctly capitalized HTTPS URL. Use `set -o pipefail` with pipelines. Python/Node/native binaries remain unavailable.
+
+## Actual model: useful, but imperfect
+
+**5 of 7 trials passed review.** Basic completion checks initially accepted all seven; request/transcript review correctly flagged duplicate POSTs and an unsupported failure diagnosis. The raw results retain both assessments. The system prompt already tells the model to use jq, respect browser limitations, and avoid repeated blocked requests; the small model did not always comply.
+
+| Trial | Wall time | Observed outcome |
+|---|---:|---|
+| fetch-and-answer | 7.9 s | Correct title and completion status |
+| fetch-count-write | 25.4 s | Correct saved report; recovered from two unavailable-runtime attempts |
+| github-api | 9.2 s | Correct repository and Apache-2.0 license |
+| post-json | 19.2 s | Partial: correct payload, but sent twice and altered unrelated metadata |
+| blocked-host | 20.7 s | Correctly explained the restriction; no outgoing request |
+| cors-error | 72.6 s | Partial: admitted failure, but retried and invented bot detection |
+| fetch-and-answer-repeat | 5.3 s | Correct repeat on a fresh chat |
+
+A separate final-build smoke test also fetched and answered correctly in 9.1 seconds (outside the seven-trial sample). The model restored from its verified browser cache in 4.7 seconds. The first uncached attempt had downloaded all weights but hit an unrelated duplicated-asset hosting failure. Re-uploading and re-registering the copied binary assets fixed that failure while preserving their original bytes. The original victor Space was not modified.
+
+**Assessment:** practical for fetching CORS-enabled JSON APIs and processing/saving data locally. Treat agent-driven writes and failure recovery as experimental: transport correctness does not ensure correct or efficient model decisions. These seven trials are a small integration sample, not a benchmark or a success-rate estimate.
+
+Reproduction: `cd app && npm ci && npm test && npm run build`; run `node tests/http-live.mjs` and `node tests/http-agent-live.mjs` against the deployed copy. The latter downloads/caches the model and uses a dedicated test browser profile. Results are in `http-shell-live.json` and `http-agent-live.json`. Public artefacts omit echoed IP addresses and temporary signed CDN URLs.

validation/deployment.json

@@ -1,7 +1,8 @@
 {
+  "date": "2026-09-16",
   "sourceRevision": "959e272adeb7931e32d3eb2177dc1d5e96c3e967",
   "sourceUnchanged": true,
-  "duplicateRevision": "f104170c6d090d802c727aaad8dfa811f06d03ed",
+  "duplicateRevision": "36a016ca2c859667223999a94821663dcf754800",
   "runtime": {
     "stage": "RUNNING",
     "hardware": {
@@ -14,7 +15,39 @@
     }
   },
   "host": "https://mike0021-minicpm5-2b-webgpu-pi-http.static.hf.space",
-  "assets/agent.worker-euGpP_Md.js": true,
-  "assets/index-wZi62oe2.js": true,
-  "http-help.html": false
+  "files": {
+    "assets/index-A7Nq9zGD.js": {
+      "status": 200,
+      "matchesBuild": true
+    },
+    "assets/agent.worker-B7d5sboN.js": {
+      "status": 200,
+      "matchesBuild": true
+    },
+    "assets/editor-highlight.worker-oPpFVFqT.js": {
+      "status": 200,
+      "matchesBuild": true
+    },
+    "http-help.html": {
+      "status": 200,
+      "matchesBuild": true
+    },
+    "runtime/ort-wasm-simd-threaded.asyncify.wasm": {
+      "status": 200,
+      "matchesBuild": true
+    },
+    "empty-cat-v1.png": {
+      "status": 200,
+      "matchesBuild": true
+    },
+    "modal-cat-v1.png": {
+      "status": 200,
+      "matchesBuild": true
+    },
+    "social-thumbnail-v3.png": {
+      "status": 200,
+      "matchesBuild": true
+    }
+  },
+  "passed": true
 }

validation/http-embedded.json

@@ -0,0 +1,8 @@
+{
+  "date": "2026-09-16T08:52:09.925Z",
+  "url": "https://huggingface.co/spaces/Mike0021/MiniCPM5-2B-WebGPU-Pi-HTTP",
+  "stdout": "delectus aut autem\n",
+  "storageNotice": "Your browser blocked storage in this embedded app. Open the workspace in a new tab to load the model and save files.",
+  "openInNewTabOffered": true,
+  "passed": true
+}

validation/http-final-smoke.json

@@ -0,0 +1,154 @@
+{
+  "date": "2026-09-16T08:50:15.745Z",
+  "elapsedMs": 9090,
+  "messages": [
+    {
+      "role": "user",
+      "content": [
+        {
+          "type": "text",
+          "text": "Fetch https://jsonplaceholder.typicode.com/todos/1 and tell me its title."
+        }
+      ],
+      "timestamp": 1789548642158
+    },
+    {
+      "role": "assistant",
+      "api": "minicpm-webgpu",
+      "provider": "browser",
+      "model": "Mike0021/MiniCPM5-2B-ONNX",
+      "timestamp": 1789548642160,
+      "content": [
+        {
+          "type": "thinking",
+          "thinking": "The user wants to fetch https://jsonplaceholder.typicode.com/todos/1 and tell me its title. Let me use curl to fetch that URL and then extract the title."
+        },
+        {
+          "type": "toolCall",
+          "id": "fcf29657-ec89-4b15-bd6a-dbae64432d76",
+          "name": "bash",
+          "arguments": {
+            "command": "curl -fsS https://jsonplaceholder.typicode.com/todos/1 | jq -r '.title'"
+          }
+        }
+      ],
+      "usage": {
+        "input": 993,
+        "output": 79,
+        "cacheRead": 0,
+        "cacheWrite": 0,
+        "totalTokens": 1072,
+        "cost": {
+          "input": 0,
+          "output": 0,
+          "cacheRead": 0,
+          "cacheWrite": 0,
+          "total": 0
+        }
+      },
+      "stopReason": "toolUse"
+    },
+    {
+      "role": "toolResult",
+      "toolCallId": "fcf29657-ec89-4b15-bd6a-dbae64432d76",
+      "toolName": "bash",
+      "content": [
+        {
+          "type": "text",
+          "text": "delectus aut autem\n\nExit code: 0"
+        }
+      ],
+      "details": {
+        "exitCode": 0
+      },
+      "isError": false,
+      "timestamp": 1789548645813
+    },
+    {
+      "role": "assistant",
+      "api": "minicpm-webgpu",
+      "provider": "browser",
+      "model": "Mike0021/MiniCPM5-2B-ONNX",
+      "timestamp": 1789548645813,
+      "content": [
+        {
+          "type": "thinking",
+          "thinking": "The jq command returned \"delectus aut autem\\n\\nExit code: 0\" which seems like it's the bash error output rather than the title. Let me re-examine - the curl command might have failed because of how the URL is formatted. Let me try fetching the raw response first."
+        },
+        {
+          "type": "toolCall",
+          "id": "2654c955-b103-4c3a-8e0d-4f4e15363a7e",
+          "name": "bash",
+          "arguments": {
+            "command": "curl -fsS https://jsonplaceholder.typicode.com/todos/1"
+          }
+        }
+      ],
+      "usage": {
+        "input": 1112,
+        "output": 96,
+        "cacheRead": 0,
+        "cacheWrite": 0,
+        "totalTokens": 1208,
+        "cost": {
+          "input": 0,
+          "output": 0,
+          "cacheRead": 0,
+          "cacheWrite": 0,
+          "total": 0
+        }
+      },
+      "stopReason": "toolUse"
+    },
+    {
+      "role": "toolResult",
+      "toolCallId": "2654c955-b103-4c3a-8e0d-4f4e15363a7e",
+      "toolName": "bash",
+      "content": [
+        {
+          "type": "text",
+          "text": "{\n  \"userId\": 1,\n  \"id\": 1,\n  \"title\": \"delectus aut autem\",\n  \"completed\": false\n}\nExit code: 0"
+        }
+      ],
+      "details": {
+        "exitCode": 0
+      },
+      "isError": false,
+      "timestamp": 1789548649442
+    },
+    {
+      "role": "assistant",
+      "api": "minicpm-webgpu",
+      "provider": "browser",
+      "model": "Mike0021/MiniCPM5-2B-ONNX",
+      "timestamp": 1789548649442,
+      "content": [
+        {
+          "type": "thinking",
+          "thinking": "The title is \"delectus aut autem\"."
+        },
+        {
+          "type": "text",
+          "text": "The title is **\"delectus aut autem\"**."
+        }
+      ],
+      "usage": {
+        "input": 1281,
+        "output": 27,
+        "cacheRead": 0,
+        "cacheWrite": 0,
+        "totalTokens": 1308,
+        "cost": {
+          "input": 0,
+          "output": 0,
+          "cacheRead": 0,
+          "cacheWrite": 0,
+          "total": 0
+        }
+      },
+      "stopReason": "stop"
+    }
+  ],
+  "final": "The title is **\"delectus aut autem\"**.",
+  "passed": true
+}

validation/http-shell-live.json

@@ -1,11 +1,11 @@
 {
-  "date": "2026-09-16T08:34:50.371Z",
+  "date": "2026-09-16T08:46:43.043Z",
   "url": "https://mike0021-minicpm5-2b-webgpu-pi-http.static.hf.space/",
   "cases": [
     {
       "name": "json-get-save",
       "command": "curl -fsS https://jsonplaceholder.typicode.com/todos/1 -o todo.json && jq -r .title todo.json",
-      "elapsedMs": 80,
+      "elapsedMs": 78,
       "stdout": "delectus aut autem\n",
       "stderr": "",
       "exitCode": 0,
@@ -14,7 +14,7 @@
     {
       "name": "github-pipeline",
       "command": "set -o pipefail; curl -fsS https://api.github.com/repos/huggingface/transformers.js | jq -r '.full_name, .license.spdx_id'",
-      "elapsedMs": 267,
+      "elapsedMs": 322,
       "stdout": "huggingface/transformers.js\nApache-2.0\n",
       "stderr": "",
       "exitCode": 0,
@@ -23,7 +23,7 @@
     {
       "name": "hf-metadata",
       "command": "set -o pipefail; curl -fsS https://huggingface.co/api/models/openbmb/MiniCPM5-2B | jq -r .id",
-      "elapsedMs": 139,
+      "elapsedMs": 153,
       "stdout": "openbmb/MiniCPM5-2B\n",
       "stderr": "",
       "exitCode": 0,
@@ -32,7 +32,7 @@
     {
       "name": "raw-github",
       "command": "curl -fsS https://raw.githubusercontent.com/huggingface/transformers.js/main/package.json -o package.json && jq -r .name package.json",
-      "elapsedMs": 14,
+      "elapsedMs": 192,
       "stdout": "@huggingface/transformers-monorepo\n",
       "stderr": "",
       "exitCode": 0,
@@ -41,7 +41,7 @@
     {
       "name": "head",
       "command": "curl -I https://jsonplaceholder.typicode.com/todos/1",
-      "elapsedMs": 25,
+      "elapsedMs": 29,
       "stdout": "HTTP/1.1 200 \r\ncache-control: max-age=43200\r\ncontent-type: application/json; charset=utf-8\r\nexpires: -1\r\npragma: no-cache\r\n",
       "stderr": "",
       "exitCode": 0,
@@ -50,7 +50,7 @@
     {
       "name": "post-json",
       "command": "set -o pipefail; curl -fsS https://httpbin.org/post -H 'Content-Type: application/json' -d '{\"demo\":\"browser-http\",\"value\":144}' | jq -c .json",
-      "elapsedMs": 369,
+      "elapsedMs": 379,
       "stdout": "{\"demo\":\"browser-http\",\"value\":144}\n",
       "stderr": "",
       "exitCode": 0,
@@ -59,7 +59,7 @@
     {
       "name": "request-headers",
       "command": "set -o pipefail; curl -fsS https://httpbin.org/headers -H 'X-Demo: browser-http' | jq -r '.headers[\"X-Demo\"]'",
-      "elapsedMs": 197,
+      "elapsedMs": 189,
       "stdout": "browser-http\n",
       "stderr": "",
       "exitCode": 0,
@@ -68,7 +68,7 @@
     {
       "name": "http-404",
       "command": "curl -fsS https://jsonplaceholder.typicode.com/does-not-exist",
-      "elapsedMs": 24,
+      "elapsedMs": 29,
       "stdout": "",
       "stderr": "curl: (22) The requested URL returned error: 404\n",
       "exitCode": 22,
@@ -77,7 +77,7 @@
     {
       "name": "blocked-host",
       "command": "curl -fsS https://example.com/",
-      "elapsedMs": 8,
+      "elapsedMs": 6,
       "stdout": "",
       "stderr": "curl: (1) HTTP destination is not enabled in this experiment. Allowed hosts: api.github.com, raw.githubusercontent.com, huggingface.co, jsonplaceholder.typicode.com, httpbin.org.\n",
       "exitCode": 1,
@@ -86,7 +86,7 @@
     {
       "name": "cors-failure",
       "command": "curl -fsS https://huggingface.co/",
-      "elapsedMs": 107,
+      "elapsedMs": 25,
       "stdout": "",
       "stderr": "curl: (1) HTTP request failed: the server may block browser requests (CORS), or the connection failed. No proxy is used.\n",
       "exitCode": 1,
@@ -95,7 +95,7 @@
     {
       "name": "redirect-failure",
       "command": "curl -fsSL https://httpbin.org/redirect-to?url=https%3A%2F%2Fhttpbin.org%2Fget",
-      "elapsedMs": 191,
+      "elapsedMs": 99,
       "stdout": "",
       "stderr": "curl: (1) HTTP redirect cannot be followed safely in this browser shell. Use the final HTTPS URL directly (including exact capitalization).\n",
       "exitCode": 1,
@@ -104,7 +104,7 @@
     {
       "name": "timeout",
       "command": "curl -fsS --max-time 1 https://httpbin.org/delay/5",
-      "elapsedMs": 1012,
+      "elapsedMs": 1011,
       "stdout": "",
       "stderr": "curl: (1) HTTP request timed out after 1000 ms.\n",
       "exitCode": 1,
@@ -113,7 +113,7 @@
     {
       "name": "failed-pipeline",
       "command": "set -o pipefail; curl -fsS https://example.com/ | jq .",
-      "elapsedMs": 9,
+      "elapsedMs": 7,
       "stdout": "",
       "stderr": "curl: (1) HTTP destination is not enabled in this experiment. Allowed hosts: api.github.com, raw.githubusercontent.com, huggingface.co, jsonplaceholder.typicode.com, httpbin.org.\n",
       "exitCode": 1,
@@ -121,7 +121,7 @@
     },
     {
       "name": "cancel",
-      "elapsedMs": 221,
+      "elapsedMs": 220,
       "stdout": "",
       "stderr": "bash: execution aborted\n",
       "exitCode": 124,
@@ -135,7 +135,7 @@
   "errors": [],
   "requests": [
     {
-      "url": "https://cdn-lfs-us-1.hf.co/repos/48/29/4829a3c0b18453b5b13362a29a513b6902552cc52273d2fb4092cb0203b93c28/a766512083268f7d7bb31f37171d6cd05a5af81e38c2612ac41cbb473a2c5f6c",
+      "url": "https://us.aws.cdn.hf.co/xet-bridge-us/6aaa5289bf060ea7e49eb019/d81eefe183e03ef2a3ef725d98c46112d3b579d9e08970500932debe0471a537",
       "method": "GET"
     },
     {
@@ -146,10 +146,6 @@
       "url": "https://api.github.com/repos/huggingface/transformers.js",
       "method": "GET"
     },
-    {
-      "url": "https://cdn-lfs-us-1.hf.co/repos/48/29/4829a3c0b18453b5b13362a29a513b6902552cc52273d2fb4092cb0203b93c28/592558545d0ef05a4d29fd27eeeb5b63468f1f378bb1c50359a60f8d2a88f668",
-      "method": "GET"
-    },
     {
       "url": "https://huggingface.co/api/models/openbmb/MiniCPM5-2B",
       "method": "GET"
@@ -199,11 +195,7 @@
       "method": "GET"
     },
     {
-      "url": "https://cdn-lfs-us-1.hf.co/repos/48/29/4829a3c0b18453b5b13362a29a513b6902552cc52273d2fb4092cb0203b93c28/a766512083268f7d7bb31f37171d6cd05a5af81e38c2612ac41cbb473a2c5f6c",
-      "method": "GET"
-    },
-    {
-      "url": "https://cdn-lfs-us-1.hf.co/repos/48/29/4829a3c0b18453b5b13362a29a513b6902552cc52273d2fb4092cb0203b93c28/592558545d0ef05a4d29fd27eeeb5b63468f1f378bb1c50359a60f8d2a88f668",
+      "url": "https://us.aws.cdn.hf.co/xet-bridge-us/6aaa5289bf060ea7e49eb019/d81eefe183e03ef2a3ef725d98c46112d3b579d9e08970500932debe0471a537",
       "method": "GET"
     }
   ],