Runtime image for Pi Web workspace machines

Dockerfile

@@ -0,0 +1,42 @@
+# Runtime image for Pi Web on HF Sandboxes. Every workspace machine is an HF Sandbox (a Job
+# in its user's own account) started from this image, so the Space is public: Jobs in any
+# namespace can pull it. It holds no secret. The Space itself only serves a short page.
+FROM node:22-bookworm
+
+# Tools for the coding agent, and zstd for the workspace archives.
+RUN apt-get update \
+ && apt-get install -y --no-install-recommends \
+      python3-venv python3-pip python-is-python3 zstd ripgrep fd-find jq less vim-tiny procps htop git-lfs unzip zip rsync \
+ && ln -sf /usr/bin/fdfind /usr/local/bin/fd \
+ && rm -rf /var/lib/apt/lists/*
+
+ARG UV_VERSION=0.12.21
+RUN curl -LsSf "https://github.com/astral-sh/uv/releases/download/${UV_VERSION}/uv-x86_64-unknown-linux-gnu.tar.gz" \
+    | tar -xz --strip-components=1 -C /usr/local/bin uv-x86_64-unknown-linux-gnu/uv uv-x86_64-unknown-linux-gnu/uvx
+
+# The Hugging Face CLI and its agent skills, for every project: pi reads ~/.agents/skills.
+# AWS SageMaker skills are left out (no AWS credentials in a sandbox).
+ARG HF_CLI_VERSION=2.1.1
+ARG HF_SKILLS_EXCLUDE="^hf-cloud-"
+RUN UV_TOOL_DIR=/opt/uv-tools UV_TOOL_BIN_DIR=/usr/local/bin uv tool install --python /usr/bin/python3 "huggingface_hub==${HF_CLI_VERSION}" \
+ && hf skills add --global --force \
+ && for skill in $(hf skills list -q 2>/dev/null | grep -vx 'hf-cli' | grep -Ev "${HF_SKILLS_EXCLUDE}"); do hf skills add "$skill" --global --force; done \
+ && rm -rf /root/.claude /root/.cache/huggingface \
+ && ls /root/.agents/skills
+
+# git authenticates to huggingface.co with the token lent by the sign-in, if any.
+COPY sandbox/git-credential-huggingface /usr/local/bin/git-credential-huggingface
+RUN chmod 755 /usr/local/bin/git-credential-huggingface \
+ && git config --system credential.https://huggingface.co.helper /usr/local/bin/git-credential-huggingface \
+ && git config --system credential.https://hf.co.helper /usr/local/bin/git-credential-huggingface
+
+# Pi Web, unmodified, from its npm release (prebuilt Next.js app + pi SDK).
+ARG PI_WEB_VERSION=0.12.1
+RUN npm install -g --no-audit --no-fund "@agegr/pi-web@${PI_WEB_VERSION}" && npm cache clean --force
+
+COPY sandbox /opt/hfpi
+COPY www /opt/hfpi/www
+RUN chmod -R a+rX /opt/hfpi
+
+EXPOSE 7860
+CMD ["python3", "-m", "http.server", "7860", "--directory", "/opt/hfpi/www"]

README.md

@@ -1,10 +1,29 @@
 ---
-title: Pi Web Runtime
-emoji: 🏃
-colorFrom: purple
-colorTo: pink
+title: Pi Web runtime
+emoji: 🥧
+colorFrom: yellow
+colorTo: gray
 sdk: docker
+app_port: 7860
 pinned: false
+short_description: Sandbox image for Pi Web on Hugging Face Sandboxes
 ---
 
-Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference
+# Pi Web runtime image
+
+The image every workspace machine of **Pi Web on HF Sandboxes** runs. A machine is an HF Sandbox, an
+[HF Job](https://huggingface.co/docs/huggingface_hub/guides/jobs) started in the signed-in user's own account from
+`hf.co/spaces/Mike0021/pi-web-runtime`. That is why this Space is public: Jobs in any namespace must be able to pull
+it. It contains no secret.
+
+What is inside:
+
+- [Pi Web](https://github.com/agegr/pi-web) 0.12.1 (unmodified npm release) and the pi coding agent
+- Node 22, Python 3, uv, git, git-lfs, build tools, ripgrep, fd, jq, zstd
+- the `hf` CLI (`huggingface_hub` 2.1.1) and the Hugging Face agent skills in `~/.agents/skills`
+- `sandbox/hfpi.mjs`: the supervisor that restores the user's projects and sessions from their bucket, runs Pi Web,
+  saves state while it changes, and stops the machine when idle
+- `sandbox/git-credential-huggingface`: git authenticates to huggingface.co with the token the user lent at sign-in
+- `sandbox/AGENTS.md`: what the agent is told about this environment
+
+The Space app itself only serves a short page; the gateway lives in a separate Space.

sandbox/AGENTS.md

@@ -0,0 +1,18 @@
+# Environment: Hugging Face Sandbox
+
+You run inside a disposable Hugging Face Sandbox (a container on HF Jobs, in the signed-in user's own account) driven from Pi Web in a browser.
+
+- You are `root` on Debian 12 with Node 22, npm, Python 3 (`python`), uv, git, git-lfs, gcc/make, ripgrep, fd, jq and zstd. You can `apt-get install` more, but system packages are gone after a restart.
+- Hardware: see `$ACCELERATOR`, `$CPU_CORES` and `$MEMORY`. Internet access is available.
+- Projects live in `/workspace/<name>`. Only `/workspace` is saved: every project is archived to a private bucket in the user's account every couple of minutes while it changes, and when the machine stops. Anything outside `/workspace` (global installs, `~/.cache`, `/tmp`) is lost on restart.
+- Dependency and build directories are not saved: `node_modules`, Python virtualenvs, `__pycache__`, `.next` and folders with a `CACHEDIR.TAG` (such as Rust `target/`). After a restart, reinstall before running anything (`npm ci`, `uv sync`, `pip install -r requirements.txt`, ...).
+- Nobody can open a server you start here in their browser. To check a web app, start it in the background (`nohup ... > /tmp/app.log 2>&1 &`) and query it with `curl`. To show the user a running app, deploy it to a Hugging Face Space.
+- The machine stops after a period without user activity; an agent turn in progress keeps it alive.
+
+## Hugging Face
+
+- The Hugging Face CLI `hf` is installed, and Hugging Face skills are in `~/.agents/skills` (`hf-cli`, `huggingface-spaces`, `huggingface-datasets`, `huggingface-llm-trainer`, `huggingface-gradio`, ...). Use them for anything on the Hub.
+- When the user granted agent access at sign-in, `~/.cache/huggingface/token` holds their scoped, expiring token: `hf`, `huggingface_hub` and `git` (clone and push on huggingface.co) act as that user, within the permissions they granted. Check with `hf auth whoami`. If there is no token, say so instead of asking for one. The token is renewed in place; read the file when you need it rather than copying it.
+- Never print, log, copy or commit the token. For tools that only read `HF_TOKEN`, pass it for that command: `HF_TOKEN=$(cat ~/.cache/huggingface/token) node script.mjs`.
+- What you do with it is real and billed to the user: Jobs, Inference Providers calls, new repos, pushes, pull requests. Ask before starting paid hardware or long Jobs, deleting or overwriting anything on the Hub, or publishing to a public repo; prefer a pull request over pushing to someone's main branch.
+- There is no GitHub token: commit locally; the user can download a patch from the dashboard.

sandbox/git-credential-huggingface

@@ -0,0 +1,10 @@
+#!/bin/sh
+# Git credential helper for huggingface.co (set in /etc/gitconfig): answers with the token
+# the Space lent to the agent through Sign in with Hugging Face, if there is one, so the
+# token never ends up in a remote URL or in .git/config.
+cat >/dev/null
+[ "$1" = "get" ] || exit 0
+dir="${HF_HOME:-$HOME/.cache/huggingface}"
+[ -s "$dir/token" ] || exit 0
+user=$(sed -n 's/.*"user": *"\([^"]*\)".*/\1/p' "$dir/hfpi-identity.json" 2>/dev/null)
+printf 'username=%s\npassword=%s\n' "${user:-hf}" "$(cat "$dir/token")"

sandbox/hfpi.mjs

@@ -0,0 +1,708 @@
+#!/usr/bin/env node
+// hfpi: supervisor and state manager for Pi Web inside a Hugging Face Sandbox.
+//
+// The sandbox is disposable; the bucket mounted at $HFPI_PERSIST is not. Working
+// trees live on fast local disk under $HFPI_WORKSPACE and are saved as one
+// tar.zst per project (a FUSE bucket mount is too slow for git and drops modes and
+// symlinks, a single archive keeps both). The pi agent directory (sessions,
+// settings, models) is saved the same way, more often.
+//
+//   hfpi run                       boot: restore, configure pi, start Pi Web, keep state saved
+//   hfpi snapshot [--name N] [--agent] [--force]
+//   hfpi import --name N (--git URL [--ref R] | --tar PATH | --empty) [--source JSON]
+//   hfpi delete --name N
+//   hfpi changes --name N          files changed since the import, for a Hub PR
+//   hfpi patch --name N            the same changes as a git patch
+//   hfpi status                    JSON for the gateway
+//   hfpi heartbeat                 user activity seen by the gateway
+//   hfpi stop                      final snapshot, then stop Pi Web and the supervisor
+
+import { spawn, spawnSync } from "node:child_process";
+import fs from "node:fs";
+import os from "node:os";
+import path from "node:path";
+
+const WORKSPACE = process.env.HFPI_WORKSPACE || "/workspace";
+const PERSIST = process.env.HFPI_PERSIST || "/persist";
+const HOME = os.homedir();
+const AGENT_DIR = process.env.PI_CODING_AGENT_DIR || path.join(HOME, ".pi", "agent");
+const RUN_DIR = process.env.HFPI_RUN_DIR || "/tmp/hfpi";
+const PORT = Number(process.env.HFPI_PIWEB_PORT || 30141);
+const PROJECTS_DIR = path.join(PERSIST, "projects");
+const AGENT_ARCHIVE = path.join(PERSIST, "agent", "agent.tar.zst");
+const MACHINE_DIR = path.join(PERSIST, "machine");
+const IDLE_MINUTES = Number(process.env.HFPI_IDLE_MINUTES ?? 30);
+const MAX_LIFETIME_MINUTES = Number(process.env.HFPI_MAX_LIFETIME_MINUTES ?? 23 * 60 + 40);
+const AGENT_SAVE_SECONDS = Number(process.env.HFPI_AGENT_SAVE_SECONDS ?? 20);
+const PROJECT_SAVE_SECONDS = Number(process.env.HFPI_PROJECT_SAVE_SECONDS ?? 120);
+const NAME_RE = /^[A-Za-z0-9][A-Za-z0-9._-]{0,63}$/;
+const MANAGED_MARK = "<!-- managed by hfpi: delete this line to keep your own edits -->";
+
+// Regenerable directories are not saved: they make archives 20x larger and slower.
+// Python virtualenvs are found by their pyvenv.cfg, Rust/pytest caches by CACHEDIR.TAG.
+const PROJECT_EXCLUDES = [
+  "node_modules", "__pycache__", ".next", ".nuxt", ".turbo", ".svelte-kit",
+  ".parcel-cache", ".pytest_cache", ".mypy_cache", ".ruff_cache", ".gradle",
+  ...(process.env.HFPI_EXTRA_EXCLUDES || "").split(",").map((s) => s.trim()).filter(Boolean),
+];
+const TAR_EXCLUDE_ARGS = [
+  ...PROJECT_EXCLUDES.map((e) => `--exclude=${e}`),
+  "--exclude-caches-all",
+  "--exclude-tag-all=pyvenv.cfg",
+];
+const AGENT_EXCLUDES = ["./bin", "./tmp"];
+
+// ---------------------------------------------------------------- helpers
+
+function log(...args) {
+  console.error(`[hfpi ${new Date().toISOString()}]`, ...args);
+}
+
+function run(cmd, args, opts = {}) {
+  const res = spawnSync(cmd, args, { encoding: "utf8", maxBuffer: 64 * 1024 * 1024, ...opts });
+  if (res.error) throw res.error;
+  if (res.status !== 0 && !opts.allowFail) {
+    const err = new Error(`${cmd} ${args.join(" ")} exited ${res.status}: ${(res.stderr || res.stdout || "").trim().slice(-2000)}`);
+    err.result = res;
+    throw err;
+  }
+  return res;
+}
+
+function exists(p) {
+  try {
+    fs.lstatSync(p);
+    return true;
+  } catch {
+    return false;
+  }
+}
+
+function readJson(p, fallback = null) {
+  try {
+    return JSON.parse(fs.readFileSync(p, "utf8"));
+  } catch {
+    return fallback;
+  }
+}
+
+function writeJson(p, data) {
+  fs.mkdirSync(path.dirname(p), { recursive: true });
+  const tmp = `${p}.tmp-${process.pid}`;
+  fs.writeFileSync(tmp, JSON.stringify(data, null, 2) + "\n");
+  fs.renameSync(tmp, p);
+}
+
+function touch(p) {
+  fs.mkdirSync(path.dirname(p), { recursive: true });
+  const now = new Date();
+  try {
+    fs.utimesSync(p, now, now);
+  } catch {
+    fs.writeFileSync(p, "");
+  }
+}
+
+function mtimeMs(p) {
+  try {
+    return fs.statSync(p).mtimeMs;
+  } catch {
+    return 0;
+  }
+}
+
+function checkName(name) {
+  if (!name || !NAME_RE.test(name)) throw new Error(`invalid project name ${JSON.stringify(name)}: use letters, digits, '.', '_' or '-'`);
+  return name;
+}
+
+function parseArgs(argv) {
+  const out = { _: [] };
+  for (let i = 0; i < argv.length; i++) {
+    const a = argv[i];
+    if (a.startsWith("--")) {
+      const key = a.slice(2);
+      const next = argv[i + 1];
+      if (next === undefined || next.startsWith("--")) out[key] = true;
+      else {
+        out[key] = next;
+        i++;
+      }
+    } else out._.push(a);
+  }
+  return out;
+}
+
+// A mkdir lock, so the supervisor loop and gateway-triggered commands never write
+// the same archive at once. A lock whose owner died is taken over.
+function withLock(key, fn) {
+  const dir = path.join(RUN_DIR, "locks", key);
+  fs.mkdirSync(path.dirname(dir), { recursive: true });
+  const deadline = Date.now() + 10 * 60 * 1000;
+  for (;;) {
+    try {
+      fs.mkdirSync(dir);
+      fs.writeFileSync(path.join(dir, "pid"), String(process.pid));
+      break;
+    } catch (e) {
+      if (e.code !== "EEXIST") throw e;
+      let owner = 0;
+      try {
+        owner = Number(fs.readFileSync(path.join(dir, "pid"), "utf8")) || 0;
+      } catch {}
+      let alive = false;
+      if (owner) {
+        try {
+          process.kill(owner, 0);
+          alive = true;
+        } catch {}
+      }
+      if (!alive && Date.now() - mtimeMs(dir) > 2000) {
+        fs.rmSync(dir, { recursive: true, force: true });
+        continue;
+      }
+      if (Date.now() > deadline) throw new Error(`timed out waiting for lock ${key}`);
+      spawnSync("sleep", ["0.2"]);
+    }
+  }
+  try {
+    return fn();
+  } finally {
+    fs.rmSync(dir, { recursive: true, force: true });
+  }
+}
+
+// True when anything under dir (minus excluded directories) is newer than marker.
+// Directory mtimes change on create/delete/rename, so removals are caught too.
+function changedSince(dir, marker, excludes) {
+  if (!exists(marker)) return true;
+  const prune = excludes.flatMap((e, i) => (i ? ["-o"] : []).concat(e.startsWith("./") ? ["-path", path.join(dir, e.slice(2))] : ["-name", e]));
+  const args = [dir, "(", ...prune, ")", "-prune", "-o", "-newer", marker, "-print", "-quit"];
+  const res = run("find", args, { allowFail: true });
+  return Boolean(res.stdout.trim());
+}
+
+function listProjectDirs() {
+  if (!exists(WORKSPACE)) return [];
+  return fs.readdirSync(WORKSPACE, { withFileTypes: true })
+    .filter((d) => d.isDirectory() && NAME_RE.test(d.name))
+    .map((d) => d.name)
+    .sort();
+}
+
+function projectMetaPath(name) {
+  return path.join(PROJECTS_DIR, name, "meta.json");
+}
+
+// Folders nobody imported: Pi Web's default directory, or one the agent created.
+function defaultSource(name) {
+  return { kind: name === "pi-cwd" ? "default-dir" : "local" };
+}
+
+function projectArchivePath(name) {
+  return path.join(PROJECTS_DIR, name, "snapshot.tar.zst");
+}
+
+function gitHead(dir) {
+  const res = run("git", ["-C", dir, "rev-parse", "HEAD"], { allowFail: true });
+  return res.status === 0 ? res.stdout.trim() : null;
+}
+
+function gitBranch(dir) {
+  const res = run("git", ["-C", dir, "rev-parse", "--abbrev-ref", "HEAD"], { allowFail: true });
+  return res.status === 0 ? res.stdout.trim() : null;
+}
+
+// ---------------------------------------------------------------- snapshots
+
+// Archive to a temporary file next to the target, then rename: a sandbox killed
+// mid-upload leaves the previous archive intact.
+function writeArchive(archive, tarArgs) {
+  fs.mkdirSync(path.dirname(archive), { recursive: true });
+  const tmp = `${archive}.partial`;
+  run("tar", ["--zstd", "-cf", tmp, ...tarArgs]);
+  fs.renameSync(tmp, archive);
+  return fs.statSync(archive).size;
+}
+
+function snapshotProject(name, { force = false } = {}) {
+  checkName(name);
+  const dir = path.join(WORKSPACE, name);
+  if (!exists(dir)) return { name, skipped: "missing" };
+  const marker = path.join(RUN_DIR, "markers", `project-${name}`);
+  if (!force && !changedSince(dir, marker, PROJECT_EXCLUDES)) return { name, skipped: "unchanged" };
+  return withLock(`project-${name}`, () => {
+    const started = new Date();
+    const pending = `${marker}.pending`;
+    touch(pending); // taken before tar reads anything: edits made during tar count for the next round
+    const bytes = writeArchive(projectArchivePath(name), ["-C", WORKSPACE, ...TAR_EXCLUDE_ARGS, name]);
+    fs.renameSync(pending, marker);
+    const meta = readJson(projectMetaPath(name), { name, source: defaultSource(name), created_at: started.toISOString() });
+    meta.snapshot = { at: new Date().toISOString(), bytes, seconds: (Date.now() - started.getTime()) / 1000 };
+    meta.head = gitHead(dir);
+    meta.branch = gitBranch(dir);
+    writeJson(projectMetaPath(name), meta);
+    log(`saved project ${name}: ${(bytes / 1e6).toFixed(1)} MB in ${meta.snapshot.seconds.toFixed(1)}s`);
+    return { name, bytes, at: meta.snapshot.at };
+  });
+}
+
+function snapshotAgent({ force = false } = {}) {
+  if (!exists(AGENT_DIR)) return { skipped: "missing" };
+  const marker = path.join(RUN_DIR, "markers", "agent");
+  if (!force && !changedSince(AGENT_DIR, marker, AGENT_EXCLUDES)) return { skipped: "unchanged" };
+  return withLock("agent", () => {
+    const pending = `${marker}.pending`;
+    touch(pending);
+    const bytes = writeArchive(AGENT_ARCHIVE, ["-C", AGENT_DIR, ...AGENT_EXCLUDES.map((e) => `--exclude=${e}`), "."]);
+    fs.renameSync(pending, marker);
+    return { bytes, at: new Date().toISOString() };
+  });
+}
+
+function snapshotAll({ force = false } = {}) {
+  const results = { agent: null, projects: [] };
+  try {
+    results.agent = snapshotAgent({ force });
+  } catch (e) {
+    log("agent snapshot failed:", e.message);
+    results.agent = { error: e.message };
+  }
+  for (const name of listProjectDirs()) {
+    try {
+      results.projects.push(snapshotProject(name, { force }));
+    } catch (e) {
+      log(`project ${name} snapshot failed:`, e.message);
+      results.projects.push({ name, error: e.message });
+    }
+  }
+  return results;
+}
+
+// ---------------------------------------------------------------- restore
+
+function restoreAgent() {
+  if (!exists(AGENT_ARCHIVE)) {
+    log("no saved agent state yet");
+    return false;
+  }
+  fs.mkdirSync(AGENT_DIR, { recursive: true });
+  run("tar", ["--zstd", "-xf", AGENT_ARCHIVE, "-C", AGENT_DIR]);
+  touch(path.join(RUN_DIR, "markers", "agent"));
+  log("restored agent state");
+  return true;
+}
+
+function restoreProjects() {
+  if (!exists(PROJECTS_DIR)) return [];
+  const restored = [];
+  for (const name of fs.readdirSync(PROJECTS_DIR)) {
+    if (!NAME_RE.test(name)) continue;
+    const archive = projectArchivePath(name);
+    const dir = path.join(WORKSPACE, name);
+    if (!exists(archive) || exists(dir)) continue;
+    const started = Date.now();
+    run("tar", ["--zstd", "-xf", archive, "-C", WORKSPACE]);
+    touch(path.join(RUN_DIR, "markers", `project-${name}`));
+    log(`restored project ${name} in ${((Date.now() - started) / 1000).toFixed(1)}s`);
+    restored.push(name);
+  }
+  return restored;
+}
+
+// ---------------------------------------------------------------- pi configuration
+
+function configurePi() {
+  fs.mkdirSync(AGENT_DIR, { recursive: true });
+
+  // Route the built-in Hugging Face provider through the gateway relay: the real HF
+  // token stays in the Space, the sandbox only holds a relay key scoped to inference.
+  const relayUrl = process.env.HFPI_RELAY_URL;
+  const modelsPath = path.join(AGENT_DIR, "models.json");
+  const models = readJson(modelsPath, {}) || {};
+  models.providers = models.providers || {};
+  if (relayUrl) {
+    models.providers.huggingface = {
+      ...(models.providers.huggingface || {}),
+      baseUrl: relayUrl,
+      apiKey: "$HFPI_RELAY_KEY",
+    };
+  }
+  writeJson(modelsPath, models);
+
+  const settingsPath = path.join(AGENT_DIR, "settings.json");
+  const settings = readJson(settingsPath, {}) || {};
+  if (!settings.defaultProvider) {
+    settings.defaultProvider = "huggingface";
+    settings.defaultModel = process.env.HFPI_DEFAULT_MODEL || "zai-org/GLM-5.3";
+    settings.defaultThinkingLevel = settings.defaultThinkingLevel || "medium";
+  }
+  if (!settings.enabledModels && process.env.HFPI_ENABLED_MODELS) {
+    settings.enabledModels = process.env.HFPI_ENABLED_MODELS.split(",").map((s) => s.trim()).filter(Boolean);
+  }
+  settings.enableInstallTelemetry = false;
+  writeJson(settingsPath, settings);
+
+  // Global context about this environment, rewritten each boot unless the user
+  // removed the managed marker to keep their own version.
+  const agentsPath = path.join(AGENT_DIR, "AGENTS.md");
+  const template = path.join(path.dirname(new URL(import.meta.url).pathname), "AGENTS.md");
+  const current = exists(agentsPath) ? fs.readFileSync(agentsPath, "utf8") : null;
+  if (exists(template) && (current === null || current.startsWith(MANAGED_MARK))) {
+    fs.writeFileSync(agentsPath, `${MANAGED_MARK}\n${fs.readFileSync(template, "utf8")}`);
+  }
+
+  const gitName = process.env.HFPI_GIT_NAME || "pi";
+  const gitEmail = process.env.HFPI_GIT_EMAIL || "pi@users.noreply.huggingface.co";
+  for (const [k, v] of [
+    ["safe.directory", "*"],
+    ["init.defaultBranch", "main"],
+    ["user.name", gitName],
+    ["user.email", gitEmail],
+  ]) {
+    run("git", ["config", "--global", k, v], { allowFail: true });
+  }
+
+  // Pi Web's "Use default directory" writes under ~/pi-cwd: keep it inside the
+  // saved workspace, and make the workspace easy to reach from the directory picker.
+  fs.mkdirSync(path.join(WORKSPACE, "pi-cwd"), { recursive: true });
+  for (const [link, target] of [
+    [path.join(HOME, "pi-cwd"), path.join(WORKSPACE, "pi-cwd")],
+    [path.join(HOME, "workspace"), WORKSPACE],
+  ]) {
+    if (!exists(link)) fs.symlinkSync(target, link);
+  }
+}
+
+// ---------------------------------------------------------------- pi web process
+
+async function piWebGet(pathname, timeoutMs = 3000) {
+  const ctl = new AbortController();
+  const t = setTimeout(() => ctl.abort(), timeoutMs);
+  try {
+    const res = await fetch(`http://127.0.0.1:${PORT}${pathname}`, { signal: ctl.signal });
+    if (!res.ok) return null;
+    return await res.json();
+  } catch {
+    return null;
+  } finally {
+    clearTimeout(t);
+  }
+}
+
+async function runningSessions() {
+  const data = await piWebGet("/api/agent/running");
+  if (!data) return null;
+  return data.runningSessionIds || [];
+}
+
+// ---------------------------------------------------------------- commands
+
+async function cmdRun() {
+  const bootStarted = Date.now();
+  fs.mkdirSync(RUN_DIR, { recursive: true });
+  // Two gateways can overlap while the Space swaps containers: only one supervisor runs.
+  const previous = readJson(path.join(RUN_DIR, "supervisor.json"), null);
+  if (previous?.pid && previous.pid !== process.pid && previous.phase !== "stopped") {
+    try {
+      process.kill(previous.pid, 0);
+      log(`supervisor ${previous.pid} is already running`);
+      return;
+    } catch {}
+  }
+  fs.mkdirSync(WORKSPACE, { recursive: true });
+  writeJson(path.join(RUN_DIR, "supervisor.json"), { pid: process.pid, started_at: new Date(bootStarted).toISOString(), phase: "restoring" });
+  touch(path.join(RUN_DIR, "heartbeat"));
+
+  restoreAgent();
+  const restored = restoreProjects();
+  configurePi();
+  log(`restore done in ${((Date.now() - bootStarted) / 1000).toFixed(1)}s (${restored.length} projects)`);
+
+  const jobId = process.env.JOB_ID || "";
+  const allowedHosts = [process.env.PI_WEB_ALLOWED_HOSTS, jobId && `${jobId}--${PORT}.hf.jobs`].filter(Boolean).join(",");
+  const env = {
+    ...process.env,
+    PI_WEB_NO_OPEN: "1",
+    PI_WEB_SKIP_VERSION_CHECK: "1",
+    PI_WEB_ALLOWED_HOSTS: allowedHosts,
+    PI_WEB_APP_NAME: process.env.PI_WEB_APP_NAME || "Pi Web",
+  };
+  delete env.PORT;
+
+  let child = null;
+  let stopping = false;
+  let restarts = 0;
+  const logFile = path.join(RUN_DIR, "piweb.log");
+  const startPiWeb = () => {
+    const out = fs.openSync(logFile, "a");
+    child = spawn("pi-web", ["--port", String(PORT), "--hostname", "0.0.0.0", "--no-open"], {
+      cwd: WORKSPACE,
+      env,
+      stdio: ["ignore", out, out],
+    });
+    log(`started pi-web pid ${child.pid}`);
+    child.on("exit", (code, signal) => {
+      log(`pi-web exited code=${code} signal=${signal}`);
+      child = null;
+      if (stopping) return;
+      restarts++;
+      setTimeout(startPiWeb, Math.min(30000, 1000 * 2 ** Math.min(restarts, 5)));
+    });
+  };
+  startPiWeb();
+  writeJson(path.join(RUN_DIR, "supervisor.json"), { pid: process.pid, started_at: new Date(bootStarted).toISOString(), phase: "running", restored });
+
+  const shutdown = async (reason) => {
+    if (stopping) return;
+    stopping = true;
+    log(`shutting down: ${reason}`);
+    writeJson(path.join(RUN_DIR, "supervisor.json"), { pid: process.pid, started_at: new Date(bootStarted).toISOString(), phase: "stopping", reason });
+    // Stop Pi Web first so session files are complete, then save everything that changed.
+    if (child) {
+      child.kill("SIGTERM");
+      for (let i = 0; i < 50 && child; i++) await new Promise((r) => setTimeout(r, 100));
+      if (child) child.kill("SIGKILL");
+    }
+    const results = snapshotAll();
+    writeJson(path.join(MACHINE_DIR, "last-shutdown.json"), { reason, at: new Date().toISOString(), job_id: jobId, results });
+    writeJson(path.join(RUN_DIR, "supervisor.json"), { pid: process.pid, started_at: new Date(bootStarted).toISOString(), phase: "stopped", reason });
+    process.exit(0);
+  };
+  process.on("SIGTERM", () => shutdown("SIGTERM"));
+  process.on("SIGINT", () => shutdown("SIGINT"));
+
+  let lastAgentSave = 0;
+  let lastProjectSave = Date.now();
+  for (;;) {
+    await new Promise((r) => setTimeout(r, 5000));
+    if (stopping) continue;
+    const now = Date.now();
+
+    if (exists(path.join(RUN_DIR, "stop-requested"))) {
+      fs.rmSync(path.join(RUN_DIR, "stop-requested"), { force: true });
+      await shutdown("requested");
+      continue;
+    }
+    if (now - lastAgentSave > AGENT_SAVE_SECONDS * 1000) {
+      lastAgentSave = now;
+      try {
+        snapshotAgent();
+      } catch (e) {
+        log("agent save failed:", e.message);
+      }
+    }
+    if (now - lastProjectSave > PROJECT_SAVE_SECONDS * 1000) {
+      lastProjectSave = now;
+      for (const name of listProjectDirs()) {
+        try {
+          snapshotProject(name);
+        } catch (e) {
+          log(`project ${name} save failed:`, e.message);
+        }
+      }
+    }
+    if (now - bootStarted > MAX_LIFETIME_MINUTES * 60000) {
+      // Jobs end at 24h; leave on our own terms with everything saved.
+      await shutdown("max-lifetime");
+      continue;
+    }
+    if (IDLE_MINUTES > 0 && now - mtimeMs(path.join(RUN_DIR, "heartbeat")) > IDLE_MINUTES * 60000) {
+      const running = await runningSessions();
+      if (running && running.length === 0) await shutdown("idle");
+      else if (running) touch(path.join(RUN_DIR, "heartbeat")); // an agent turn is still going: wait for it
+    }
+  }
+}
+
+function cmdImport(args) {
+  const name = checkName(args.name);
+  const dir = path.join(WORKSPACE, name);
+  if (exists(dir) || exists(projectMetaPath(name))) throw new Error(`project ${name} already exists`);
+  fs.mkdirSync(WORKSPACE, { recursive: true });
+  let source = args.source ? JSON.parse(args.source) : {};
+  const started = Date.now();
+  try {
+    if (args.git) {
+      const cloneArgs = ["clone", "--filter=blob:none"];
+      if (args.ref && args.ref !== true) cloneArgs.push("--branch", String(args.ref));
+      cloneArgs.push("--", String(args.git), dir);
+      run("git", cloneArgs, { env: { ...process.env, GIT_TERMINAL_PROMPT: "0", GIT_LFS_SKIP_SMUDGE: "1" } });
+      source = { kind: "git", url: String(args.git), ref: args.ref && args.ref !== true ? String(args.ref) : null, ...source };
+    } else if (args.tar) {
+      fs.mkdirSync(dir);
+      run("tar", ["-xf", String(args.tar), "-C", dir]);
+      fs.rmSync(String(args.tar), { force: true });
+      if (!exists(path.join(dir, ".git"))) {
+        run("git", ["-C", dir, "init", "-q"]);
+        run("git", ["-C", dir, "add", "-A"]);
+        run("git", ["-C", dir, "commit", "-q", "--allow-empty", "-m", `Import ${source.repo_id || name}`]);
+      }
+      source = { kind: "archive", ...source };
+    } else if (args.empty) {
+      fs.mkdirSync(dir);
+      run("git", ["-C", dir, "init", "-q"]);
+      run("git", ["-C", dir, "commit", "-q", "--allow-empty", "-m", "Initial commit"]);
+      source = { kind: "empty", ...source };
+    } else {
+      throw new Error("import needs --git URL, --tar PATH or --empty");
+    }
+  } catch (e) {
+    fs.rmSync(dir, { recursive: true, force: true });
+    throw e;
+  }
+  const meta = {
+    name,
+    source,
+    created_at: new Date().toISOString(),
+    base_commit: gitHead(dir),
+    import_seconds: (Date.now() - started) / 1000,
+  };
+  writeJson(projectMetaPath(name), meta);
+  const snap = snapshotProject(name, { force: true });
+  return { ...meta, path: dir, snapshot: snap };
+}
+
+function cmdDelete(args) {
+  const name = checkName(args.name);
+  return withLock(`project-${name}`, () => {
+    fs.rmSync(path.join(WORKSPACE, name), { recursive: true, force: true });
+    fs.rmSync(path.join(PROJECTS_DIR, name), { recursive: true, force: true });
+    fs.rmSync(path.join(RUN_DIR, "markers", `project-${name}`), { force: true });
+    return { name, deleted: true };
+  });
+}
+
+// A virtualenv (pyvenv.cfg) or cache (CACHEDIR.TAG) above `file` inside the project.
+function insideTaggedDir(root, file) {
+  let rel = path.dirname(file);
+  while (rel && rel !== ".") {
+    const d = path.join(root, rel);
+    if (exists(path.join(d, "pyvenv.cfg")) || exists(path.join(d, "CACHEDIR.TAG"))) return true;
+    rel = path.dirname(rel);
+  }
+  return false;
+}
+
+// Stage the whole working tree (minus excluded directories) in a throwaway index, so
+// a diff against the import commit sees committed and uncommitted work alike while the
+// user's own staging area stays untouched.
+function withWorkingIndex(name, base, fn) {
+  const dir = path.join(WORKSPACE, checkName(name));
+  const meta = readJson(projectMetaPath(name), {});
+  base = base || meta.base_commit;
+  if (!base) throw new Error(`project ${name} has no base commit to compare with`);
+  const index = path.join(RUN_DIR, `index-${name}-${process.pid}`);
+  const env = { ...process.env, GIT_INDEX_FILE: index };
+  fs.mkdirSync(RUN_DIR, { recursive: true });
+  try {
+    run("git", ["-C", dir, "read-tree", "HEAD"], { env });
+    // The same directories the archives leave out never belong in a PR or patch either.
+    const pathspec = [".", ...PROJECT_EXCLUDES.map((e) => `:(exclude,glob)**/${e}/**`)];
+    run("git", ["-C", dir, "add", "-A", "--", ...pathspec], { env });
+    for (const f of listStaged(dir, env)) {
+      if (insideTaggedDir(dir, f)) run("git", ["-C", dir, "rm", "-q", "--cached", "--", f], { env });
+    }
+    return fn({ dir, base, env });
+  } finally {
+    fs.rmSync(index, { force: true });
+  }
+}
+
+function listStaged(dir, env) {
+  const res = run("git", ["-C", dir, "diff", "--cached", "--name-only", "--diff-filter=AM", "-z", "HEAD"], { env });
+  return res.stdout.split("\0").filter(Boolean);
+}
+
+// Every file that differs from the import commit, as a Hub PR would see it.
+function cmdChanges(args) {
+  const name = checkName(args.name);
+  return withWorkingIndex(name, args.base, ({ dir, base, env }) => {
+    const res = run("git", ["-C", dir, "diff", "--cached", "--name-status", "--no-renames", "-z", base], { env });
+    const parts = res.stdout.split("\0").filter(Boolean);
+    const files = [];
+    for (let i = 0; i + 1 < parts.length; i += 2) {
+      const status = parts[i];
+      const file = parts[i + 1];
+      const full = path.join(dir, file);
+      const st = status !== "D" && exists(full) ? fs.statSync(full) : null;
+      files.push({ status, path: file, size: st ? st.size : 0 });
+    }
+    return { name, base, head: gitHead(dir), files };
+  });
+}
+
+// The same changes as a patch that `git apply` takes on any checkout of the base.
+function cmdPatch(args) {
+  const name = checkName(args.name);
+  return withWorkingIndex(name, args.base, ({ dir, base, env }) => {
+    const res = run("git", ["-C", dir, "diff", "--cached", "--binary", base], { env });
+    process.stdout.write(res.stdout);
+  });
+}
+
+async function cmdStatus() {
+  const sup = readJson(path.join(RUN_DIR, "supervisor.json"), null);
+  let supervisorAlive = false;
+  if (sup?.pid) {
+    try {
+      process.kill(sup.pid, 0);
+      supervisorAlive = true;
+    } catch {}
+  }
+  const projects = [];
+  for (const name of listProjectDirs()) {
+    const meta = readJson(projectMetaPath(name), { name, source: defaultSource(name) });
+    const marker = path.join(RUN_DIR, "markers", `project-${name}`);
+    projects.push({ ...meta, name, path: path.join(WORKSPACE, name), dirty: changedSince(path.join(WORKSPACE, name), marker, PROJECT_EXCLUDES) });
+  }
+  const running = supervisorAlive ? await runningSessions() : null;
+  return {
+    supervisor: sup ? { ...sup, alive: supervisorAlive } : null,
+    piweb: { healthy: running !== null, running_sessions: running || [] },
+    idle_seconds: Math.round((Date.now() - mtimeMs(path.join(RUN_DIR, "heartbeat"))) / 1000),
+    idle_minutes_limit: IDLE_MINUTES,
+    agent_saved_at: exists(AGENT_ARCHIVE) ? new Date(mtimeMs(AGENT_ARCHIVE)).toISOString() : null,
+    projects,
+  };
+}
+
+async function main() {
+  const [command, ...rest] = process.argv.slice(2);
+  const args = parseArgs(rest);
+  const print = (data) => process.stdout.write(JSON.stringify(data) + "\n");
+  switch (command) {
+    case "run":
+      return cmdRun();
+    case "snapshot": {
+      if (args.name) return print(snapshotProject(String(args.name), { force: Boolean(args.force) }));
+      if (args.agent) return print(snapshotAgent({ force: Boolean(args.force) }));
+      return print(snapshotAll({ force: Boolean(args.force) }));
+    }
+    case "import":
+      return print(cmdImport(args));
+    case "delete":
+      return print(cmdDelete(args));
+    case "changes":
+      return print(cmdChanges(args));
+    case "patch":
+      return cmdPatch(args);
+    case "status":
+      return print(await cmdStatus());
+    case "heartbeat":
+      touch(path.join(RUN_DIR, "heartbeat"));
+      return print({ ok: true });
+    case "stop":
+      touch(path.join(RUN_DIR, "stop-requested"));
+      return print({ ok: true });
+    default:
+      console.error("usage: hfpi run|snapshot|import|delete|changes|patch|status|heartbeat|stop");
+      process.exit(2);
+  }
+}
+
+main().catch((e) => {
+  console.error(`hfpi: ${e.message}`);
+  process.stdout.write(JSON.stringify({ error: e.message }) + "\n");
+  process.exit(1);
+});

www/index.html

@@ -0,0 +1,27 @@
+<!doctype html>
+<html lang="en">
+  <head>
+    <meta charset="utf-8" />
+    <meta name="viewport" content="width=device-width, initial-scale=1" />
+    <title>Pi Web runtime</title>
+    <style>
+      :root { --bg: #fff; --text: #1a1a1a; --muted: #5e6673; color-scheme: light; }
+      @media (prefers-color-scheme: dark) { :root { --bg: #1a1a1a; --text: #e8e8e8; --muted: #a4a4a4; color-scheme: dark; } }
+      body { margin: 0; background: var(--bg); color: var(--text); font: 15px/1.6 system-ui, sans-serif; }
+      main { max-width: 640px; margin: 0 auto; padding: 48px 16px; }
+      h1 { font-size: 22px; margin: 0 0 8px; }
+      p { color: var(--muted); }
+      code { font-size: 13px; }
+    </style>
+  </head>
+  <body>
+    <main>
+      <h1>🥧 Pi Web runtime</h1>
+      <p>
+        This Space only holds the Docker image that workspace machines of <b>Pi Web on Hugging Face Sandboxes</b> run.
+        Each machine is an HF Sandbox started in its user's own account from <code>hf.co/spaces/Mike0021/pi-web-runtime</code>.
+      </p>
+      <p>Nothing runs here. See the Files tab for what the image contains.</p>
+    </main>
+  </body>
+</html>