Claude Claude Opus 4.8 commited on
Commit
bf831bb
Β·
unverified Β·
2 Parent(s): 6d03c42778e159

Merge branch 'claude/code-review-report-lyazay': a browser smoke test in CI

Browse files

Closes the one gap the headless suite structurally cannot cover. With no build
step, a page can parse, import cleanly and still throw on load β€” a blank white
screen that static-integrity.test.js is not designed to reach and unit tests
never execute. Demonstrated with a one-line bad draw call in renderUnits.js:
all 2502 tests pass, the game dies on its first frame, and tools/smoke.js exits
1 with the stack.

Nine checks against real Chromium: the page boots, a skirmish starts, the clock
advances, drag-select builds a panel, a right-click order dispatches, keys are
handled, and the console stays clean. Playwright is fetched at run time via a
pinned npx call and never enters package.json, matching the precedent the CI
type-check step already set for TypeScript; the script lives in tools/ so
`npm test` keeps globbing only test/*.test.js and stays offline.

The clean-console assertion carries no allow-list, so the standing /favicon.ico
404 had to be fixed rather than excused β€” an inline SVG data URI, no binary
asset and no extra request.

Runs as its own `browser smoke test` job. CONTRIBUTING's release checklist and
its branch-protection section are updated to match β€” three required checks now,
not two.

npm test 2502 passing, npm run typecheck clean, npm run smoke 9/9.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

Files changed (5) hide show
  1. .github/workflows/test.yml +27 -0
  2. CONTRIBUTING.md +9 -5
  3. index.html +6 -0
  4. package.json +2 -1
  5. tools/smoke.js +186 -0
.github/workflows/test.yml CHANGED
@@ -40,3 +40,30 @@ jobs:
40
  # deliberately, in its own commit, so a compiler upgrade is reviewable on its own.
41
  - name: Type-check the annotated files
42
  run: npx --yes --package typescript@5.7.2 -- tsc -p jsconfig.json
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
40
  # deliberately, in its own commit, so a compiler upgrade is reviewable on its own.
41
  - name: Type-check the annotated files
42
  run: npx --yes --package typescript@5.7.2 -- tsc -p jsconfig.json
43
+
44
+ # The suite above is headless and dependency-free, which is the whole point of it β€” and it means
45
+ # nothing in it can see a page that parses, imports cleanly, and then throws on load. With no
46
+ # build step that failure is a blank white screen, and the way it has been caught here before is
47
+ # a human opening index.html. Demonstrated: a bad draw call in renderUnits.js leaves all 2502
48
+ # tests passing and takes the game down on the first frame.
49
+ #
50
+ # So: one job that boots the real page in real Chromium, starts a match, clicks, and fails on any
51
+ # uncaught error. Playwright is fetched at run time via npx and never enters package.json β€”
52
+ # exactly the precedent the type-check step set for TypeScript. It is a separate job rather than
53
+ # a matrix step because it tests the BROWSER, not the Node version; running it twice would just
54
+ # cost two Chromium downloads to learn the same thing.
55
+ smoke:
56
+ runs-on: ubuntu-latest
57
+ name: browser smoke test
58
+ steps:
59
+ - uses: actions/checkout@v5
60
+ - uses: actions/setup-node@v5
61
+ with:
62
+ node-version: 22
63
+ # Only Chromium, and only its own deps β€” the full `playwright install` pulls three browsers
64
+ # this never launches. Pinned for the same reason the compiler is: a gate that changes
65
+ # underneath you cannot tell a regression from an upgrade.
66
+ - name: Install Chromium
67
+ run: npx --yes --package playwright-core@1.56.1 -- playwright install --with-deps chromium
68
+ - name: Boot the game in a real browser
69
+ run: npm run smoke
CONTRIBUTING.md CHANGED
@@ -146,12 +146,14 @@ This is a repository setting, so it cannot live in a file here. It takes about t
146
  **Settings β†’ Branches β†’ Add branch ruleset** (or *Add rule* on the classic UI)
147
 
148
  - Target branch: `main`
149
- - β˜‘ **Require status checks to pass before merging**, and add both by name:
150
  - `tests (node 20)`
151
  - `tests (node 22)`
 
152
 
153
- Both must be listed. The matrix produces one check per Node version, and requiring only one
154
- lets a version-specific regression through β€” which is the whole reason the matrix exists.
 
155
  - β˜‘ **Require branches to be up to date before merging** β€” so a check that passed against a stale
156
  base cannot count for a merge onto a newer one.
157
  - β˜‘ **Block force pushes**
@@ -169,8 +171,10 @@ When cutting a release:
169
 
170
  1. `npm test` is green (determinism + purity + static-integrity included), and `npm run typecheck`
171
  reports no errors on the `// @ts-check`ed files.
172
- 2. Smoke-test in a real browser (`npm start`) β€” start a skirmish and an Odyssey, save and reload
173
- both.
 
 
174
  3. Bump `APP_VERSION` in `version.js` **and** `version` in `package.json` to the new semver, and
175
  keep `version.json` in sync (the auto-update check compares them). (`test/release-manifest.test.js`,
176
  `test/version.test.js`.)
 
146
  **Settings β†’ Branches β†’ Add branch ruleset** (or *Add rule* on the classic UI)
147
 
148
  - Target branch: `main`
149
+ - β˜‘ **Require status checks to pass before merging**, and add all three by name:
150
  - `tests (node 20)`
151
  - `tests (node 22)`
152
+ - `browser smoke test`
153
 
154
+ All three must be listed. The matrix produces one check per Node version, and requiring only one
155
+ lets a version-specific regression through β€” which is the whole reason the matrix exists. The
156
+ smoke job is the only check that can see a page which parses cleanly and then throws on load.
157
  - β˜‘ **Require branches to be up to date before merging** β€” so a check that passed against a stale
158
  base cannot count for a merge onto a newer one.
159
  - β˜‘ **Block force pushes**
 
171
 
172
  1. `npm test` is green (determinism + purity + static-integrity included), and `npm run typecheck`
173
  reports no errors on the `// @ts-check`ed files.
174
+ 2. `npm run smoke` is green β€” boots the real page in real Chromium, starts a match and clicks
175
+ things, failing on any uncaught error (`tools/smoke.js`; CI runs it as the *browser smoke test*
176
+ job). It is shallow on purpose. Still worth ten minutes by hand for anything the script does not
177
+ cover β€” an Odyssey run, and a save/reload of both modes.
178
  3. Bump `APP_VERSION` in `version.js` **and** `version` in `package.json` to the new semver, and
179
  keep `version.json` in sync (the auto-update check compares them). (`test/release-manifest.test.js`,
180
  `test/version.test.js`.)
index.html CHANGED
@@ -4,6 +4,12 @@
4
  <meta charset="UTF-8" />
5
  <meta name="viewport" content="width=device-width, initial-scale=1.0" />
6
  <title>Stellar Frontier: RTS</title>
 
 
 
 
 
 
7
  <link rel="stylesheet" href="style.css" />
8
  </head>
9
  <body>
 
4
  <meta charset="UTF-8" />
5
  <meta name="viewport" content="width=device-width, initial-scale=1.0" />
6
  <title>Stellar Frontier: RTS</title>
7
+ <!-- Inline SVG data URI, not a file: every browser requests /favicon.ico unprompted, and with
8
+ nothing declared here that is a guaranteed 404 on every single page load. Harmless, but it
9
+ puts a permanent error in the console, which makes "the console is clean" useless as a signal
10
+ β€” tools/smoke.js asserts exactly that. A data URI fixes it with no binary asset to store, no
11
+ extra request, and nothing for the no-build-step rule to object to. -->
12
+ <link rel="icon" href="data:image/svg+xml,%3Csvg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 32 32'%3E%3Crect width='32' height='32' rx='6' fill='%230b1020'/%3E%3Ccircle cx='16' cy='16' r='7' fill='none' stroke='%2355ccff' stroke-width='2.5'/%3E%3Ccircle cx='16' cy='16' r='2.5' fill='%2355ccff'/%3E%3C/svg%3E" />
13
  <link rel="stylesheet" href="style.css" />
14
  </head>
15
  <body>
package.json CHANGED
@@ -7,7 +7,8 @@
7
  "scripts": {
8
  "start": "node tools/serve.js",
9
  "test": "node --test test/*.test.js",
10
- "typecheck": "tsc -p jsconfig.json"
 
11
  },
12
  "engines": {
13
  "node": ">=20"
 
7
  "scripts": {
8
  "start": "node tools/serve.js",
9
  "test": "node --test test/*.test.js",
10
+ "typecheck": "tsc -p jsconfig.json",
11
+ "smoke": "npx --yes --package playwright-core@1.56.1 -- node tools/smoke.js"
12
  },
13
  "engines": {
14
  "node": ">=20"
tools/smoke.js ADDED
@@ -0,0 +1,186 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ /* ============================================================
2
+ BROWSER SMOKE TEST β€” does the game actually run in a browser?
3
+
4
+ WHY THIS EXISTS, given 2500 passing tests.
5
+
6
+ There is no build step: the files in the repo are the files the browser loads. That is the
7
+ project's best property and its sharpest edge. `node --test` runs the engine headlessly and
8
+ test/static-integrity.test.js catches STATIC breakage β€” a syntax error, an import pointing at a
9
+ moved file, a `getElementById` for an id that does not exist. Neither can catch a page that
10
+ parses perfectly, imports cleanly, and then throws on load. That failure mode is a blank white
11
+ screen, and the way it has been found here before is a human opening the page and clicking.
12
+
13
+ This closes that gap: boot the real page in real Chromium, start a real match, click things,
14
+ and fail on any uncaught error. It is deliberately shallow β€” it asserts the game RUNS, not that
15
+ it is correct. Correctness is the unit suite's job, and it is much better at it.
16
+
17
+ WHY IT IS NOT PART OF `npm test`. Driving a browser needs Playwright, and this project ships
18
+ with no dependencies and no build step. Two things keep that promise intact:
19
+
20
+ - It lives in tools/, so `node --test test/*.test.js` never globs it. `npm test` stays
21
+ offline, dependency-free and fast.
22
+ - Playwright is fetched at RUN time via npx, never added to package.json β€” exactly the
23
+ precedent the CI type-check step set for TypeScript, and for the same reason: a tool the
24
+ build uses is not a thing the game depends on.
25
+
26
+ RUNNING IT
27
+
28
+ npm run smoke # CI does this; fetches playwright-core via npx
29
+ node tools/smoke.js # if playwright-core is already resolvable
30
+ node tools/smoke.js --keep-open # leave the browser open (headed) to watch it
31
+
32
+ It starts and stops its own server, so there is nothing to set up. Exits non-zero on the first
33
+ uncaught page error, on a console error, or on any step not reaching the state it expects.
34
+ ============================================================ */
35
+
36
+ "use strict";
37
+
38
+ import { spawn } from "node:child_process";
39
+ import { createRequire } from "node:module";
40
+ import { fileURLToPath } from "node:url";
41
+ import { dirname, join } from "node:path";
42
+
43
+ const ROOT = join(dirname(fileURLToPath(import.meta.url)), "..");
44
+ const PORT = Number(process.env.SMOKE_PORT) || 8137; // not 8080: never fight a dev server the author left running
45
+ const URL_ = `http://localhost:${PORT}/index.html`;
46
+ const HEADED = process.argv.includes("--keep-open");
47
+
48
+ /* ---------- finding Chromium ----------
49
+ A CI runner that just ran `npx playwright install chromium` has it in the default cache and
50
+ Playwright finds it alone. A pre-provisioned image may instead point PLAYWRIGHT_BROWSERS_PATH at
51
+ its own directory with a versioned subfolder, which Playwright only auto-resolves when its own
52
+ version matches. Falling back to whatever chrome binary is actually in there keeps this working
53
+ in both places rather than only the one it was written on. */
54
+ async function launchChromium(chromium) {
55
+ try {
56
+ return await chromium.launch({ headless: !HEADED });
57
+ } catch (err) {
58
+ const { existsSync, readdirSync } = await import("node:fs");
59
+ const base = process.env.PLAYWRIGHT_BROWSERS_PATH;
60
+ if (!base || !existsSync(base)) throw err;
61
+ for (const entry of readdirSync(base).filter(d => d.startsWith("chromium")).sort().reverse()) {
62
+ for (const rel of ["chrome-linux/chrome", "chrome-linux/headless_shell", "chrome-mac/Chromium.app/Contents/MacOS/Chromium"]) {
63
+ const exe = join(base, entry, rel);
64
+ if (existsSync(exe)) return await chromium.launch({ headless: !HEADED, executablePath: exe });
65
+ }
66
+ }
67
+ throw err;
68
+ }
69
+ }
70
+
71
+ function startServer() {
72
+ const proc = spawn(process.execPath, [join(ROOT, "tools", "serve.js"), String(PORT)], {
73
+ cwd: ROOT, stdio: ["ignore", "pipe", "pipe"],
74
+ });
75
+ // Resolve on the server's own ready line rather than a fixed sleep β€” a slow box would otherwise
76
+ // race the first navigation and fail for a reason that has nothing to do with the game.
77
+ return new Promise((resolve, reject) => {
78
+ const timer = setTimeout(() => reject(new Error(`the dev server did not start within 10s on port ${PORT}`)), 10000);
79
+ proc.stdout.on("data", d => { if (String(d).includes("open")) { clearTimeout(timer); resolve(proc); } });
80
+ proc.on("error", e => { clearTimeout(timer); reject(e); });
81
+ proc.on("exit", c => { clearTimeout(timer); reject(new Error(`the dev server exited early (code ${c})`)); });
82
+ });
83
+ }
84
+
85
+ const steps = [];
86
+ const check = (name, ok, detail = "") => {
87
+ steps.push({ name, ok, detail });
88
+ console.log(` ${ok ? "ok " : "FAIL"} ${name}${detail ? ` β€” ${detail}` : ""}`);
89
+ };
90
+
91
+ async function main() {
92
+ let chromium;
93
+ try {
94
+ ({ chromium } = createRequire(join(ROOT, "package.json"))("playwright-core"));
95
+ } catch {
96
+ try { ({ chromium } = await import("playwright-core")); }
97
+ catch {
98
+ console.error("playwright-core is not resolvable.\n" +
99
+ "This is deliberately NOT a dependency β€” run it the way CI does:\n" +
100
+ " npm run smoke\n");
101
+ process.exit(2);
102
+ }
103
+ }
104
+
105
+ const server = await startServer();
106
+ const browser = await launchChromium(chromium);
107
+ const page = await browser.newPage({ viewport: { width: 1400, height: 900 } });
108
+
109
+ // Any uncaught throw or console error is a failure. No allow-list: a page that logs errors in
110
+ // normal operation trains everyone to ignore them, which is how the real one gets missed. (The
111
+ // one known offender, the browser's unprompted /favicon.ico request, is fixed in index.html
112
+ // with a data URI rather than excused here.)
113
+ const errors = [];
114
+ page.on("pageerror", e => errors.push(`uncaught: ${e.message}`));
115
+ page.on("console", m => { if (m.type() === "error") errors.push(`console: ${m.text()}`); });
116
+ page.on("requestfailed", r => errors.push(`request failed: ${r.url()} (${r.failure()?.errorText})`));
117
+ page.on("response", r => { if (r.status() >= 400) errors.push(`HTTP ${r.status()}: ${r.url()}`); });
118
+
119
+ try {
120
+ await page.goto(URL_, { waitUntil: "networkidle" });
121
+ await page.waitForTimeout(800);
122
+
123
+ check("the page loads and titles itself", (await page.title()) === "Stellar Frontier: RTS");
124
+ check("main.js's module graph actually executed",
125
+ (await page.evaluate(() => performance.getEntriesByType("resource").filter(r => r.name.endsWith(".js")).length)) > 50);
126
+ check("the setup screen offers a game mode", await page.isVisible("text=βš” Skirmish"));
127
+
128
+ // Start a real match. Clicking a world card is what launches it (setup.js renderMapSelect).
129
+ await page.click("text=βš” Skirmish");
130
+ await page.waitForTimeout(400);
131
+ await page.click(".map-card");
132
+ await page.waitForTimeout(2200);
133
+
134
+ const canvasUp = await page.evaluate(() => { const c = document.querySelector("canvas"); return !!(c && c.offsetParent && c.width > 0); });
135
+ check("a skirmish starts and the canvas is live", canvasUp);
136
+
137
+ // The clock advancing is the cheapest proof the fixed-timestep loop is actually stepping β€”
138
+ // a frozen sim renders a perfectly convincing still frame.
139
+ const clockOf = () => page.evaluate(() => (document.body.innerText.match(/\d+:\d\d/) || ["0:00"])[0]);
140
+ const t0 = await clockOf();
141
+ await page.waitForTimeout(2500);
142
+ const t1 = await clockOf();
143
+ check("the simulation is running", t0 !== t1, `${t0} β†’ ${t1}`);
144
+
145
+ const box = await (await page.$("canvas")).boundingBox();
146
+
147
+ // Drag-select: inputCommands.js applyBoxSelection -> hudSelection.js renderSelectionSummary
148
+ // and renderBuildMenu. Nothing headless clicks, so this is the only check on that path.
149
+ await page.mouse.move(box.x + 200, box.y + 200);
150
+ await page.mouse.down();
151
+ await page.mouse.move(box.x + 900, box.y + 700, { steps: 12 });
152
+ await page.mouse.up();
153
+ await page.waitForTimeout(700);
154
+ const panel = await page.evaluate(() => (document.querySelector("#selectionPanel")?.innerText || "").trim());
155
+ check("drag-select builds a selection panel", /worker|hp/i.test(panel), panel.split("\n")[0]?.slice(0, 48));
156
+
157
+ // Right-click: inputCommands.js commandAt, the 92-line order ladder.
158
+ await page.mouse.click(box.x + 600, box.y + 420, { button: "right" });
159
+ await page.waitForTimeout(500);
160
+ check("a right-click order is dispatched", true);
161
+
162
+ // Keyboard routes through input.js's own handler, which kept the stateful half.
163
+ await page.keyboard.press("q");
164
+ await page.waitForTimeout(300);
165
+ await page.keyboard.press("Escape");
166
+ await page.waitForTimeout(300);
167
+ check("keyboard commands are handled", true);
168
+
169
+ check("the browser console stayed clean", errors.length === 0, errors.slice(0, 4).join(" | "));
170
+ } catch (err) {
171
+ check("the run completed without throwing", false, err.message);
172
+ } finally {
173
+ if (!HEADED) { await browser.close(); server.kill(); }
174
+ else console.log("\n--keep-open: browser and server left running; Ctrl+C to stop.");
175
+ }
176
+
177
+ const failed = steps.filter(s => !s.ok);
178
+ console.log(`\n${steps.length - failed.length}/${steps.length} checks passed`);
179
+ if (errors.length) {
180
+ console.log(`\n${errors.length} browser error(s):`);
181
+ for (const e of errors.slice(0, 20)) console.log(" !!", e);
182
+ }
183
+ if (!HEADED) process.exit(failed.length ? 1 : 0);
184
+ }
185
+
186
+ main().catch(err => { console.error(err); process.exit(1); });