Spaces:
Paused
Elo: reject a same-name entrant/candidate pairing instead of silently corrupting its rating
Browse filesBlocker from Phase 1 review (docs/competitions-and-elo.md): elo.js's RatingsTable is a
plain object keyed by entrant name, and nothing upstream of it guarded against two
entrants/candidates sharing a name. When aName === bName, `ratings[aName]` and
`ratings[bName]` in applyResult alias the SAME object -- the function computes A's
post-match entry, writes it, then unconditionally overwrites that exact slot with B's
post-match entry (whichever assignment runs second always wins) -- A's half silently
vanishes and the survivor represents neither side. Reproduced live: the in-game Quick
Duel screen with both entrants named "Same Name" ran to completion and showed TWO
Elo cards, both reading the identical, corrupted "1200? (0)" -- no error, no warning.
Three-layer fix, defense in depth:
- elo.js: applyResult now throws a clear Error when aName === bName, before touching
`ratings` at all -- the shared backstop every caller (present or future) gets for
free, per D1's "one implementation" point. applySeries inherits this for free (it
just calls applyResult per row).
- competition.js: buildJob rejects two entrants sharing a name (post-trim) up front,
so the Quick Duel config screen shows a clear inline error instead of running a full
background duel only to throw when the results view folds the rows through
eloFromRows.
- tools/ailab.js: runDuel (the plain --a/--b CLI pairing, which -- unlike the 3+
candidate round-robin/Swiss paths -- had no name-collision guard at all) now rejects
a same-name pair immediately, mirroring assertUniqueCandidateNames' existing message
style.
tools/duelCore.js needed no change: it runs the match itself (owner-parametric, not
name-keyed) and never touches a ratings table -- the corruption is purely in the Elo
bookkeeping step, downstream of it.
Tests (TDD, red before green): test/elo.test.js (applyResult throws on a same-name
pair, before any mutation, even when the name is pre-seeded; applySeries propagates
the guard without undoing earlier valid rows in the same series);
test/competition.test.js (buildJob rejects identical, and post-trim-identical,
entrant names); test/ailab.test.js (runDuel rejects a same-name candidate pair).
Live re-verification (Playwright, headless Chromium, against the running
tools/serve.js on :8080) -- the same check a browser QA pass would run:
- Reverted the fix and reproduced the exact reported failure first: "Same Name" vs
"Same Name" in the Quick Duel screen ran to completion with zero errors and
rendered two Elo cards both reading the identical "1200? (0)" -- the silent
corruption, confirmed live, not just reasoned about.
- Restored the fix: the same steps now show an inline error ("Entrant A and Entrant
B need different names") and never reach the progress/worker view -- zero console
or page errors.
- Regression check: two DIFFERENTLY-named entrants ("Alpha Duelist" / "Beta
Duelist") still run a real duel end-to-end to a results screen with two distinct,
correctly-diverging Elo cards (1198?/1202?) -- the happy path is untouched.
- CLI regression check: `node tools/ailab.js duel --a --b` with same-named candidate
files now fails fast (<0.2s, before any match runs) with a clear message; with
differently-named candidates it still runs and reports two independent Elo values
exactly as before.
Full suite: 2109/2109 passing (2103 baseline + 6 new), 0 failures, 0 regressions.
`npm run typecheck` clean.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013z8dVULQ6Xk8k2YsCF3FS1
- competition.js +6 -0
- elo.js +20 -0
- test/ailab.test.js +7 -0
- test/competition.test.js +13 -0
- test/elo.test.js +38 -0
- tools/ailab.js +8 -0
|
@@ -59,12 +59,18 @@ import { INITIAL_RATING, applySeries } from "./elo.js";
|
|
| 59 |
* pure). Throws a clear, user-facing message for anything that would make competitionWorker.js's
|
| 60 |
* own guards throw anyway β an empty name or an empty world list β so the config screen can catch
|
| 61 |
* it before ever spinning up a Worker.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 62 |
* @param {{ entrantA: {name: string, strategy?: string}, entrantB: {name: string, strategy?: string},
|
| 63 |
* difficulty: string, worlds: string[], seeds: number, seedBase: number, matchTimeLimit?: number }} cfg
|
| 64 |
*/
|
| 65 |
export function buildJob({ entrantA, entrantB, difficulty, worlds, seeds, seedBase, matchTimeLimit } = {}) {
|
| 66 |
if (!entrantA || !entrantA.name || !entrantA.name.trim()) throw new Error("Entrant A needs a name");
|
| 67 |
if (!entrantB || !entrantB.name || !entrantB.name.trim()) throw new Error("Entrant B needs a name");
|
|
|
|
| 68 |
if (!Array.isArray(worlds) || worlds.length === 0) throw new Error("Pick at least one world");
|
| 69 |
const job = {
|
| 70 |
entrantA: { name: entrantA.name.trim(), strategy: entrantA.strategy || "default" },
|
|
|
|
| 59 |
* pure). Throws a clear, user-facing message for anything that would make competitionWorker.js's
|
| 60 |
* own guards throw anyway β an empty name or an empty world list β so the config screen can catch
|
| 61 |
* it before ever spinning up a Worker.
|
| 62 |
+
* Also rejects two entrants sharing a name (post-trim): elo.js's RatingsTable is keyed by name
|
| 63 |
+
* (D1/D2), so a same-name pairing would collide both entrants' Elo into one entry instead of two
|
| 64 |
+
* (elo.js's own applyResult throws on exactly this, as a backstop) β catching it here means the
|
| 65 |
+
* config screen can reject it before ever spinning up a Worker, instead of running a full duel
|
| 66 |
+
* only to throw when the results view folds the rows through eloFromRows.
|
| 67 |
* @param {{ entrantA: {name: string, strategy?: string}, entrantB: {name: string, strategy?: string},
|
| 68 |
* difficulty: string, worlds: string[], seeds: number, seedBase: number, matchTimeLimit?: number }} cfg
|
| 69 |
*/
|
| 70 |
export function buildJob({ entrantA, entrantB, difficulty, worlds, seeds, seedBase, matchTimeLimit } = {}) {
|
| 71 |
if (!entrantA || !entrantA.name || !entrantA.name.trim()) throw new Error("Entrant A needs a name");
|
| 72 |
if (!entrantB || !entrantB.name || !entrantB.name.trim()) throw new Error("Entrant B needs a name");
|
| 73 |
+
if (entrantA.name.trim() === entrantB.name.trim()) throw new Error("Entrant A and Entrant B need different names");
|
| 74 |
if (!Array.isArray(worlds) || worlds.length === 0) throw new Error("Pick at least one world");
|
| 75 |
const job = {
|
| 76 |
entrantA: { name: entrantA.name.trim(), strategy: entrantA.strategy || "default" },
|
|
@@ -80,13 +80,33 @@ export function kFactor(games, rating) {
|
|
| 80 |
* the equal-K and unequal-K cases. This is guaranteed by construction: both deltas are derived from
|
| 81 |
* ONE shared `diff` (A's actual-minus-expected score this match), not from two independently
|
| 82 |
* rounded `expectedScore` calls, so at equal K the negation is exact, not merely close.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 83 |
* @param {RatingsTable} ratings
|
| 84 |
* @param {string} aName
|
| 85 |
* @param {string} bName
|
| 86 |
* @param {number} score A's result: 1 (A won), 0.5 (draw), 0 (A lost).
|
| 87 |
* @returns {RatingsTable} the same `ratings` object, mutated in place.
|
|
|
|
| 88 |
*/
|
| 89 |
export function applyResult(ratings, aName, bName, score) {
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 90 |
const a = ratings[aName] || (ratings[aName] = { rating: INITIAL_RATING, games: 0 });
|
| 91 |
const b = ratings[bName] || (ratings[bName] = { rating: INITIAL_RATING, games: 0 });
|
| 92 |
const kA = kFactor(a.games, a.rating);
|
|
|
|
| 80 |
* the equal-K and unequal-K cases. This is guaranteed by construction: both deltas are derived from
|
| 81 |
* ONE shared `diff` (A's actual-minus-expected score this match), not from two independently
|
| 82 |
* rounded `expectedScore` calls, so at equal K the negation is exact, not merely close.
|
| 83 |
+
*
|
| 84 |
+
* REJECTS aName === bName, before touching `ratings` at all. RatingsTable is keyed by name (see
|
| 85 |
+
* the module header), so a same-name pairing would make `a`/`b` above alias the SAME object:
|
| 86 |
+
* this function would compute A's post-match entry, write it to `ratings[aName]`, then
|
| 87 |
+
* unconditionally overwrite that exact slot with B's post-match entry (`ratings[bName] = ...` runs
|
| 88 |
+
* second and always wins, since aName and bName are one key) β A's half silently vanishes and the
|
| 89 |
+
* survivor represents neither side faithfully. Two entrants that are conceptually different (say,
|
| 90 |
+
* different strategies) but happen to share one display string collide exactly the same way, since
|
| 91 |
+
* this module has no notion of identity beyond the name string. That's not a match anyone can
|
| 92 |
+
* actually referee β an entrant can't play itself β so it's refused up front rather than silently
|
| 93 |
+
* producing a garbage rating; every shipped caller (competition.js's buildJob, tools/ailab.js's
|
| 94 |
+
* runDuel) already rejects this earlier, with a user/CLI-facing message, but this check is the
|
| 95 |
+
* shared backstop any caller gets for free, present or future, per D1's "one implementation" point.
|
| 96 |
* @param {RatingsTable} ratings
|
| 97 |
* @param {string} aName
|
| 98 |
* @param {string} bName
|
| 99 |
* @param {number} score A's result: 1 (A won), 0.5 (draw), 0 (A lost).
|
| 100 |
* @returns {RatingsTable} the same `ratings` object, mutated in place.
|
| 101 |
+
* @throws {Error} if aName === bName.
|
| 102 |
*/
|
| 103 |
export function applyResult(ratings, aName, bName, score) {
|
| 104 |
+
if (aName === bName) {
|
| 105 |
+
throw new Error(
|
| 106 |
+
`applyResult: A and B must be different entrants, both were "${aName}" β RatingsTable is ` +
|
| 107 |
+
`keyed by name, so a same-name pairing would collide both sides into one entry instead of two`
|
| 108 |
+
);
|
| 109 |
+
}
|
| 110 |
const a = ratings[aName] || (ratings[aName] = { rating: INITIAL_RATING, games: 0 });
|
| 111 |
const b = ratings[bName] || (ratings[bName] = { rating: INITIAL_RATING, games: 0 });
|
| 112 |
const kA = kFactor(a.games, a.rating);
|
|
@@ -539,6 +539,13 @@ test("a duel candidate needs a name, on either side", () => {
|
|
| 539 |
assert.throws(() => runDuel(named, { strategy: "default" }, shortDuel()), /candidate B needs a "name"/);
|
| 540 |
});
|
| 541 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 542 |
test("a duel's overrides never leak into what runs next", () => {
|
| 543 |
const before = JSON.stringify(STRATEGIES.aggressive);
|
| 544 |
const a = { name: "A", strategy: "aggressive", overrides: { strategies: { aggressive: { garrisonMult: 0.01 } } } };
|
|
|
|
| 539 |
assert.throws(() => runDuel(named, { strategy: "default" }, shortDuel()), /candidate B needs a "name"/);
|
| 540 |
});
|
| 541 |
|
| 542 |
+
test("a duel's two candidates must have different names -- elo.js's RatingsTable is keyed by name, " +
|
| 543 |
+
"so a same-name pairing would collide both sides into one entry (see test/elo.test.js)", () => {
|
| 544 |
+
const a = { name: "Same", strategy: "default" };
|
| 545 |
+
const b = { name: "Same", strategy: "aggressive" };
|
| 546 |
+
assert.throws(() => runDuel(a, b, shortDuel()), /same|different|distinct/i);
|
| 547 |
+
});
|
| 548 |
+
|
| 549 |
test("a duel's overrides never leak into what runs next", () => {
|
| 550 |
const before = JSON.stringify(STRATEGIES.aggressive);
|
| 551 |
const a = { name: "A", strategy: "aggressive", overrides: { strategies: { aggressive: { garrisonMult: 0.01 } } } };
|
|
@@ -54,6 +54,19 @@ test("buildJob trims entrant names and rejects an empty or blank one", () => {
|
|
| 54 |
assert.throws(() => buildJob({ ...fixedCfg(), entrantA: null }), /name/i);
|
| 55 |
});
|
| 56 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 57 |
test("buildJob throws when no world is selected", () => {
|
| 58 |
assert.throws(() => buildJob({ ...fixedCfg(), worlds: [] }), /world/i);
|
| 59 |
});
|
|
|
|
| 54 |
assert.throws(() => buildJob({ ...fixedCfg(), entrantA: null }), /name/i);
|
| 55 |
});
|
| 56 |
|
| 57 |
+
test("buildJob rejects identical entrant names -- they'd collide into one elo.js RatingsTable entry", () => {
|
| 58 |
+
assert.throws(
|
| 59 |
+
() => buildJob({ ...fixedCfg(), entrantA: { name: "Bob", strategy: "default" }, entrantB: { name: "Bob", strategy: "aggressive" } }),
|
| 60 |
+
/same|different|distinct/i,
|
| 61 |
+
);
|
| 62 |
+
// Compared post-trim, same as the existing blank-name check -- "Bob" and " Bob " are one collision,
|
| 63 |
+
// not two distinct entrants that merely LOOK different in the raw input.
|
| 64 |
+
assert.throws(
|
| 65 |
+
() => buildJob({ ...fixedCfg(), entrantA: { name: " Bob ", strategy: "default" }, entrantB: { name: "Bob", strategy: "aggressive" } }),
|
| 66 |
+
/same|different|distinct/i,
|
| 67 |
+
);
|
| 68 |
+
});
|
| 69 |
+
|
| 70 |
test("buildJob throws when no world is selected", () => {
|
| 71 |
assert.throws(() => buildJob({ ...fixedCfg(), worlds: [] }), /world/i);
|
| 72 |
});
|
|
@@ -91,6 +91,44 @@ test("NOT zero-sum at UNEQUAL K: a provisional entrant meeting a graduated one i
|
|
| 91 |
assert.notEqual(deltaA, -deltaB, "unequal K must NOT produce exact-negative deltas");
|
| 92 |
});
|
| 93 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 94 |
test("both sides' games increment by exactly 1, win or lose", () => {
|
| 95 |
const ratings = {};
|
| 96 |
applyResult(ratings, "A", "B", 0.5);
|
|
|
|
| 91 |
assert.notEqual(deltaA, -deltaB, "unequal K must NOT produce exact-negative deltas");
|
| 92 |
});
|
| 93 |
|
| 94 |
+
/* ---------- applyResult: same-name collision guard ----------
|
| 95 |
+
RatingsTable is a plain object keyed by entrant name (see elo.js's own header). Two entrants
|
| 96 |
+
sharing a name in the SAME match make `ratings[aName]` and `ratings[bName]` the identical slot:
|
| 97 |
+
applyResult computes A's post-match entry, writes it, then unconditionally overwrites it with
|
| 98 |
+
B's post-match entry (`ratings[bName] = ...` runs second and wins) -- A's half silently vanishes
|
| 99 |
+
and the survivor doesn't represent either side faithfully. That must be rejected up front, not
|
| 100 |
+
left to corrupt silently. */
|
| 101 |
+
|
| 102 |
+
test("applyResult throws when aName === bName instead of silently corrupting the shared entry", () => {
|
| 103 |
+
const ratings = {};
|
| 104 |
+
assert.throws(() => applyResult(ratings, "Aggressive", "Aggressive", 1), /same|different|distinct/i);
|
| 105 |
+
});
|
| 106 |
+
|
| 107 |
+
test("applyResult's same-name guard fires before any mutation -- a rejected call leaves ratings untouched", () => {
|
| 108 |
+
const ratings = {};
|
| 109 |
+
assert.throws(() => applyResult(ratings, "X", "X", 1));
|
| 110 |
+
assert.deepEqual(ratings, {}, "the colliding entry must not even be half-created");
|
| 111 |
+
});
|
| 112 |
+
|
| 113 |
+
test("applyResult's same-name guard fires even when that name is already seeded, and leaves it untouched", () => {
|
| 114 |
+
const ratings = { Solo: { rating: 1500, games: 12 } };
|
| 115 |
+
assert.throws(() => applyResult(ratings, "Solo", "Solo", 0.5));
|
| 116 |
+
assert.deepEqual(ratings, { Solo: { rating: 1500, games: 12 } }, "the pre-existing entry must survive intact");
|
| 117 |
+
});
|
| 118 |
+
|
| 119 |
+
test("applySeries propagates the same-name guard for whichever row triggers it", () => {
|
| 120 |
+
const ratings = {};
|
| 121 |
+
const rows = [
|
| 122 |
+
{ aName: "A", bName: "B", score: 1 },
|
| 123 |
+
{ aName: "C", bName: "C", score: 0.5 }, // the offending row
|
| 124 |
+
];
|
| 125 |
+
assert.throws(() => applySeries(ratings, rows), /same|different|distinct/i);
|
| 126 |
+
// The row before the offending one must still have gone through -- applyResult/applySeries never
|
| 127 |
+
// retroactively undoes earlier, valid matches just because a later row in the same series is bad.
|
| 128 |
+
assert.ok(ratings.A, "the earlier, valid row must still have been applied");
|
| 129 |
+
assert.ok(ratings.B, "the earlier, valid row must still have been applied");
|
| 130 |
+
});
|
| 131 |
+
|
| 132 |
test("both sides' games increment by exactly 1, win or lose", () => {
|
| 133 |
const ratings = {};
|
| 134 |
applyResult(ratings, "A", "B", 0.5);
|
|
@@ -651,12 +651,20 @@ export function runLeaderboard(candidates, { worlds, difficulty = "medium", oppo
|
|
| 651 |
// live fight, unlike leaderboard's candidates which never meet) β snapshot/restored around
|
| 652 |
// the whole call exactly like runLeaderboard does around each candidate, via the same
|
| 653 |
// snapshotTables/restoreTables this file already uses.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 654 |
export function runDuel(a, b, {
|
| 655 |
worlds = ["korrath", "ferros", "vesper", "kybernet"], difficulty = "medium",
|
| 656 |
seeds = 2, seedBase = 1, minutes,
|
| 657 |
} = {}) {
|
| 658 |
if (!a || !a.name) throw new Error('candidate A needs a "name"');
|
| 659 |
if (!b || !b.name) throw new Error('candidate B needs a "name"');
|
|
|
|
| 660 |
const dials = pinnedDuelDials(difficulty); // read ONCE β see the FAIRNESS paragraph above
|
| 661 |
assertNoOverrideCollision(a, b);
|
| 662 |
const snap = snapshotTables();
|
|
|
|
| 651 |
// live fight, unlike leaderboard's candidates which never meet) β snapshot/restored around
|
| 652 |
// the whole call exactly like runLeaderboard does around each candidate, via the same
|
| 653 |
// snapshotTables/restoreTables this file already uses.
|
| 654 |
+
//
|
| 655 |
+
// A and B must have DIFFERENT names, same reasoning as assertUniqueCandidateNames below (which
|
| 656 |
+
// covers 3+ candidate round-robins/Swiss but never runs for a plain --a/--b pair): elo.js's
|
| 657 |
+
// RatingsTable is keyed by name, and eloForMatches near the bottom of this file folds every duel's
|
| 658 |
+
// rows through it, so a same-name pair would silently collide both candidates' Elo into one entry
|
| 659 |
+
// (elo.js's own applyResult throws on exactly this, as a backstop β this check just gives the CLI
|
| 660 |
+
// user a clear, immediate reason instead of a duel that runs to completion and THEN throws).
|
| 661 |
export function runDuel(a, b, {
|
| 662 |
worlds = ["korrath", "ferros", "vesper", "kybernet"], difficulty = "medium",
|
| 663 |
seeds = 2, seedBase = 1, minutes,
|
| 664 |
} = {}) {
|
| 665 |
if (!a || !a.name) throw new Error('candidate A needs a "name"');
|
| 666 |
if (!b || !b.name) throw new Error('candidate B needs a "name"');
|
| 667 |
+
if (a.name === b.name) throw new Error(`candidate A and candidate B need different names (both were "${a.name}")`);
|
| 668 |
const dials = pinnedDuelDials(difficulty); // read ONCE β see the FAIRNESS paragraph above
|
| 669 |
assertNoOverrideCollision(a, b);
|
| 670 |
const snap = snapshotTables();
|