Spaces:
Paused
Calibrate the archive's health screen against the shipped AI, not a fixed list
Browse filesThe first real `archive` run refused 39 of its first 41 genomes and filled 2 of
16 cells. The diagnosis was unambiguous and it was not the genomes: arch0-arch3
ARE the four shipped strategies, and the screen refused all four. A filter that
rejects the game's own hand-written AI is wrong by inspection.
The cause was my own parameter choice. CHECKS is calibrated against the
`passive` sparring bot; the descriptor run deliberately uses `turtle`, which
provokes, so the AI spends on army and takes losses and `devFinal` never clears
dev-flatline's threshold. The detector started firing on correct behaviour --
which is precisely the failure this file's own CHECKS header records three
rewrites of. Choosing a different opponent reintroduced it a fourth time, in a
command whose header comment already warned about it.
So the screen is now RELATIVE. The seed strategies are evaluated first and
always seated; the union of what they trip under this run's own opponent and
match length becomes the tolerated set; and a genome is refused only for a
defect the shipped AI does not also show under identical conditions. That is
the question the screen was always trying to ask, and it re-answers it for
whatever settings the caller picks rather than assuming one.
The tolerated set is printed in the results and carried on the result object.
A screen that has degenerated into a no-op has to be visible, not inferred from
a suspiciously clean run.
Two tests: the shipped strategies are never refused and the tolerance is exactly
what they tripped, plus the refusal path itself (forced with no seeds, the one
configuration where an absolute screen IS intended) still refuses before the
expensive duel half rather than after.
Also fixes a syntax error the smoke test caught: backticks around 'evolve' inside
the USAGE template literal terminated it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011KL6GbjkxUNWs3frZUYANK
- test/ailab.test.js +36 -6
- tools/ailab.js +60 -21
|
@@ -1348,16 +1348,21 @@ test("every cell in the archive really is a distinct behaviour", () => {
|
|
| 1348 |
"a cell's key disagrees with its own measured descriptors");
|
| 1349 |
});
|
| 1350 |
|
| 1351 |
-
test("the health screen refuses entry outright,
|
| 1352 |
-
//
|
| 1353 |
-
//
|
| 1354 |
-
//
|
| 1355 |
-
|
|
|
|
|
|
|
|
|
|
| 1356 |
const refused = res.log.filter(e => e.reason === "screen");
|
| 1357 |
assert.ok(refused.length > 0, "a 6-minute descriptor run must trip dev-flatline β CHECKS is calibrated for 40-60m");
|
| 1358 |
for (const e of refused) {
|
| 1359 |
assert.equal(e.seated, false);
|
| 1360 |
-
|
|
|
|
|
|
|
| 1361 |
assert.ok(!res.cells.some(c => c.name === e.name), "a refused genome reached the archive anyway");
|
| 1362 |
}
|
| 1363 |
assert.equal(res.rejected, refused.length);
|
|
@@ -1391,3 +1396,28 @@ test("labWorld forwards an archetype, so a genome's own temperament reaches the
|
|
| 1391 |
opponent: "passive", seed: 1, apm: "real", archetype: "rusher" }).state), "Rusher",
|
| 1392 |
"an explicit archetype must win over the world's");
|
| 1393 |
});
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1348 |
"a cell's key disagrees with its own measured descriptors");
|
| 1349 |
});
|
| 1350 |
|
| 1351 |
+
test("the health screen refuses entry outright, and never duels what it refuses", () => {
|
| 1352 |
+
// Forcing the refusal path takes a deliberate setup now, and the reason is the point: with the
|
| 1353 |
+
// seed strategies present, a 6-minute descriptor run's dev-flatline is TOLERATED, because the
|
| 1354 |
+
// shipped AI trips it here too. Passing no seeds leaves nothing to calibrate against, so the raw
|
| 1355 |
+
// detector list is enforced β which is the only configuration where an absolute screen is the
|
| 1356 |
+
// intended behaviour rather than the bug the test below pins.
|
| 1357 |
+
const res = runArchive({ ...archiveOpts, iterations: 6, seed: 7, screen: true, seedStrategies: [] });
|
| 1358 |
+
assert.deepEqual(res.tolerated, [], "with no seeds there is nothing to calibrate against");
|
| 1359 |
const refused = res.log.filter(e => e.reason === "screen");
|
| 1360 |
assert.ok(refused.length > 0, "a 6-minute descriptor run must trip dev-flatline β CHECKS is calibrated for 40-60m");
|
| 1361 |
for (const e of refused) {
|
| 1362 |
assert.equal(e.seated, false);
|
| 1363 |
+
// The early return is the saving, not a detail: scoring is the expensive half of an evaluation,
|
| 1364 |
+
// so a screen that refused AFTER duelling would cost the same as no screen at all.
|
| 1365 |
+
assert.equal(e.fitness, undefined, "a refused genome must not be duelled");
|
| 1366 |
assert.ok(!res.cells.some(c => c.name === e.name), "a refused genome reached the archive anyway");
|
| 1367 |
}
|
| 1368 |
assert.equal(res.rejected, refused.length);
|
|
|
|
| 1396 |
opponent: "passive", seed: 1, apm: "real", archetype: "rusher" }).state), "Rusher",
|
| 1397 |
"an explicit archetype must win over the world's");
|
| 1398 |
});
|
| 1399 |
+
|
| 1400 |
+
test("the health screen never refuses the SHIPPED strategies β it calibrates against them", () => {
|
| 1401 |
+
// THE REGRESSION THIS EXISTS FOR. The archive's first real run refused 39 of its first 41
|
| 1402 |
+
// genomes, including all four shipped strategies, because CHECKS was calibrated against the
|
| 1403 |
+
// `passive` sparring bot while the descriptor run deliberately uses a provoking one β where the
|
| 1404 |
+
// AI spends on army, takes losses, and never clears dev-flatline's development threshold. A
|
| 1405 |
+
// detector firing on correct behaviour is worse than no detector; the seeds are the definition
|
| 1406 |
+
// of correct behaviour available, so they set the tolerance instead of being judged by it.
|
| 1407 |
+
const res = runArchive({ ...archiveOpts, iterations: 8, seed: 13, screen: true, descriptorMinutes: 6 });
|
| 1408 |
+
const seeds = res.log.filter(e => e.origin.startsWith("seed:"));
|
| 1409 |
+
assert.equal(seeds.length, Object.keys(STRATEGIES).length, "every shipped strategy must be evaluated");
|
| 1410 |
+
for (const s of seeds)
|
| 1411 |
+
assert.notEqual(s.reason, "screen", `the shipped strategy ${s.origin} was refused by the screen`);
|
| 1412 |
+
// A 6-minute descriptor run trips detectors calibrated for 40-60m, so this run MUST have
|
| 1413 |
+
// something to tolerate β otherwise the test would pass vacuously on a screen that never fired.
|
| 1414 |
+
assert.ok(res.tolerated.length > 0, "nothing was tolerated β the calibration path never engaged");
|
| 1415 |
+
// β¦and the tolerance is exactly what the seeds tripped, no more.
|
| 1416 |
+
const seedTrips = new Set(seeds.flatMap(s => s.tripped));
|
| 1417 |
+
assert.deepEqual([...res.tolerated].sort(), [...seedTrips].sort());
|
| 1418 |
+
// Nobody is refused for a defect the shipped AI also shows.
|
| 1419 |
+
for (const e of res.log.filter(e => e.reason === "screen")) {
|
| 1420 |
+
assert.ok(e.novel.length > 0, `${e.name} was refused with no novel defect`);
|
| 1421 |
+
for (const t of e.novel) assert.ok(!res.tolerated.includes(t), `${t} is tolerated but was held against ${e.name}`);
|
| 1422 |
+
}
|
| 1423 |
+
});
|
|
@@ -1202,10 +1202,23 @@ export function runEvolution({
|
|
| 1202 |
four shipped baselines, so its score is absolute and comparable across the whole run β
|
| 1203 |
which dissolves the anchoring problem `evolve` had to correct for (eloForMatches restarts
|
| 1204 |
from a fresh 1200 each generation, so ratings there are only comparable within one).
|
| 1205 |
-
3. THE HEALTH SCREEN IS FREE. Describing a genome means running it solo
|
| 1206 |
-
which is exactly what CHECKS needs
|
| 1207 |
-
|
| 1208 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1209 |
|
| 1210 |
The descriptor run uses the `turtle` sparring bot by default: a real economy behind turrets that
|
| 1211 |
never attacks, but whose turrets kill scouts and so DO provoke. That makes the aggression axis
|
|
@@ -1305,6 +1318,11 @@ export function runArchive({
|
|
| 1305 |
const descOpts = { worlds: descriptorWorlds, minutes: descriptorMinutes, opponent: descriptorOpponent, difficulty, seedBase: seed };
|
| 1306 |
const duelOpts = { worlds: duelWorlds, seeds: duelSeeds, minutes: duelMinutes, difficulty, seedBase: seed };
|
| 1307 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1308 |
// Evaluate one genome and try to seat it. A genome only ever displaces the CURRENT OCCUPANT of
|
| 1309 |
// its own cell, which is the whole mechanism: local competition, global diversity.
|
| 1310 |
const consider = (genome, origin) => {
|
|
@@ -1312,32 +1330,42 @@ export function runArchive({
|
|
| 1312 |
const cand = toCandidate(genome, name, { genes });
|
| 1313 |
const { desc, tripped } = describeCandidate(cand, descOpts);
|
| 1314 |
evaluated++;
|
| 1315 |
-
|
|
|
|
|
|
|
| 1316 |
rejected++;
|
| 1317 |
-
log.push({ name, origin, cell: cellKey(desc), desc, tripped, seated: false, reason: "screen" });
|
| 1318 |
// Report the refusal too, not just the seatings. A screen that is rejecting EVERYTHING looks
|
| 1319 |
// identical to a search that is finding nothing unless each refusal is visible β and there is
|
| 1320 |
// a specific, easy way to land in that state: CHECKS is calibrated for 40-60 sim-minute runs
|
| 1321 |
// (several detectors carry an "β¦and it never resolved in the last third" term keyed to that
|
| 1322 |
// length), so a short --descriptor-minutes makes every healthy genome trip dev-flatline.
|
| 1323 |
if (onCell) onCell(log[log.length - 1], archive);
|
| 1324 |
-
return
|
| 1325 |
}
|
| 1326 |
const key = cellKey(desc);
|
| 1327 |
const fitness = panelStrength(cand, panel, duelOpts);
|
| 1328 |
const incumbent = archive.get(key);
|
| 1329 |
const seated = !incumbent || fitness > incumbent.fitness;
|
| 1330 |
if (seated) archive.set(key, { key, genome, desc, fitness, name, origin, evaluatedAt: evaluated });
|
| 1331 |
-
log.push({ name, origin, cell: key, desc, tripped, fitness: +fitness.toFixed(3), seated,
|
| 1332 |
beat: incumbent ? +incumbent.fitness.toFixed(3) : null });
|
| 1333 |
if (onCell) onCell(log[log.length - 1], archive);
|
| 1334 |
-
return
|
| 1335 |
};
|
| 1336 |
|
| 1337 |
// SEED from the shipped strategies. Four hand-written rows with real search already spent on
|
| 1338 |
// them, and β the point here β four rows that already sit in different parts of the behaviour
|
| 1339 |
// space, so the archive starts spread rather than clustered.
|
| 1340 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1341 |
|
| 1342 |
while (evaluated < iterations) {
|
| 1343 |
const elites = [...archive.values()];
|
|
@@ -1365,6 +1393,7 @@ export function runArchive({
|
|
| 1365 |
return {
|
| 1366 |
cells: [...archive.values()].sort((x, y) => x.key < y.key ? -1 : 1),
|
| 1367 |
dims: ARCHIVE_DIMS, evaluated, rejected, coverage: archive.size / total, genes, log,
|
|
|
|
| 1368 |
};
|
| 1369 |
}
|
| 1370 |
|
|
@@ -1991,7 +2020,7 @@ const CMDS = {
|
|
| 1991 |
console.log(` axes: ${ARCHIVE_DIMS.map(d => `${d.label} [${d.names.join("|")}]`).join(" x ")}`);
|
| 1992 |
console.log(` describe on: ${opts.descriptorWorlds.join(", ")} @ ${opts.descriptorMinutes}m vs ${opts.descriptorOpponent}`);
|
| 1993 |
console.log(` strength vs: ${panel.map(p => p.name).join(", ")} on ${opts.duelWorlds.join(", ")} @ ${opts.duelMinutes}m`);
|
| 1994 |
-
console.log(` screen: ${opts.screen ? "on
|
| 1995 |
console.log();
|
| 1996 |
|
| 1997 |
const res = runArchive({
|
|
@@ -2003,7 +2032,10 @@ const CMDS = {
|
|
| 2003 |
// and still be seated or simply lose its cell contest, and printing that as "refused" would
|
| 2004 |
// report a screen that isn't running.
|
| 2005 |
+ " " + (entry.seated ? (entry.beat != null ? `seated (beat ${(entry.beat * 100).toFixed(0)}%)` : "SEATED β new cell")
|
| 2006 |
-
|
|
|
|
|
|
|
|
|
|
| 2007 |
: `no (holder ${(entry.beat * 100).toFixed(0)}%)`)
|
| 2008 |
+ ` ${archive.size} cells`),
|
| 2009 |
});
|
|
@@ -2013,7 +2045,12 @@ const CMDS = {
|
|
| 2013 |
const [xDim, yDim] = ARCHIVE_DIMS;
|
| 2014 |
const at = (x, y) => res.cells.find(c => c.key === `${xDim.names[x]}/${yDim.names[y]}`);
|
| 2015 |
console.log(`\nTHE CAST β ${res.cells.length}/${xDim.names.length * yDim.names.length} cells filled `
|
| 2016 |
-
+ `(${(res.coverage * 100).toFixed(0)}% coverage), ${res.evaluated} evaluated, ${res.rejected} refused by the health screen
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 2017 |
console.log(pad("", 9) + xDim.names.map(n => pad(n, 16)).join("") + ` <- ${xDim.label}`);
|
| 2018 |
for (let y = 0; y < yDim.names.length; y++) {
|
| 2019 |
console.log(pad(yDim.names[y], 9) + xDim.names.map((_, x) => {
|
|
@@ -2044,11 +2081,10 @@ const CMDS = {
|
|
| 2044 |
// An archive that came back (nearly) empty has one overwhelmingly likely cause, and it is not
|
| 2045 |
// "no good AI exists". Say so, rather than leaving a blank grid to be read as a result.
|
| 2046 |
if (res.rejected > res.evaluated * 0.8)
|
| 2047 |
-
console.log(`\n!! ${res.rejected} of ${res.evaluated} genomes were REFUSED by the health screen
|
| 2048 |
-
+ `\n
|
| 2049 |
-
+ `\n
|
| 2050 |
-
+ `\n --
|
| 2051 |
-
+ `\n to seat genomes unscreened (and lose the free defect filter).`);
|
| 2052 |
if (args.json) { writeFileSync(args.json, JSON.stringify(res.log, null, 1)); console.log(`wrote ${args.json}`); }
|
| 2053 |
},
|
| 2054 |
};
|
|
@@ -2140,11 +2176,14 @@ const USAGE = `AI LAB β a headless bench for the Odyssey opponent.
|
|
| 2140 |
measured on the solo Odyssey bench) and competes ONLY against the
|
| 2141 |
current occupant of its own cell -- so a genome that wins by refusing
|
| 2142 |
to fight can win the "never attacks" cell and nothing else, which is a
|
| 2143 |
-
structural answer to the Goodhart failure
|
| 2144 |
hand-written penalty for it. Strength is measured against a FIXED
|
| 2145 |
panel, so it is absolute and comparable across the whole run. The
|
| 2146 |
-
health screen is ON by default
|
| 2147 |
-
|
|
|
|
|
|
|
|
|
|
| 2148 |
Writes one runnable candidate per filled cell.
|
| 2149 |
|
| 2150 |
Common flags
|
|
|
|
| 1202 |
four shipped baselines, so its score is absolute and comparable across the whole run β
|
| 1203 |
which dissolves the anchoring problem `evolve` had to correct for (eloForMatches restarts
|
| 1204 |
from a fresh 1200 each generation, so ratings there are only comparable within one).
|
| 1205 |
+
3. THE HEALTH SCREEN IS FREE, AND SELF-CALIBRATING. Describing a genome means running it solo
|
| 1206 |
+
on the Odyssey bench, which is exactly what CHECKS needs. So `evolve`'s opt-in `--screen`
|
| 1207 |
+
is ON by default here at no extra cost.
|
| 1208 |
+
But it screens RELATIVE TO THE SHIPPED STRATEGIES, not against the raw detector list, and
|
| 1209 |
+
that is not a nicety β the first real run of this command refused 39 of its first 41
|
| 1210 |
+
genomes INCLUDING all four shipped strategies, which is proof the screen was wrong rather
|
| 1211 |
+
than the genomes. The cause: CHECKS was calibrated against the `passive` sparring bot, and
|
| 1212 |
+
the descriptor run deliberately uses a PROVOKING one (see below), where an AI spends on
|
| 1213 |
+
army and takes losses so `devFinal` never clears dev-flatline's threshold. A detector that
|
| 1214 |
+
fires on correct behaviour is worse than no detector β this file's own CHECKS header
|
| 1215 |
+
records three rewrites of exactly that mistake, and choosing a different opponent
|
| 1216 |
+
reintroduced it a fourth time.
|
| 1217 |
+
So: the seed strategies are evaluated first and always seated, the union of what THEY
|
| 1218 |
+
trip becomes the tolerated set, and a genome is refused only for a defect the shipped AI
|
| 1219 |
+
does not also exhibit under identical conditions. That is the question the screen was
|
| 1220 |
+
always trying to ask, and it now re-answers it for whatever opponent and match length the
|
| 1221 |
+
caller picks instead of assuming one.
|
| 1222 |
|
| 1223 |
The descriptor run uses the `turtle` sparring bot by default: a real economy behind turrets that
|
| 1224 |
never attacks, but whose turrets kill scouts and so DO provoke. That makes the aggression axis
|
|
|
|
| 1318 |
const descOpts = { worlds: descriptorWorlds, minutes: descriptorMinutes, opponent: descriptorOpponent, difficulty, seedBase: seed };
|
| 1319 |
const duelOpts = { worlds: duelWorlds, seeds: duelSeeds, minutes: duelMinutes, difficulty, seedBase: seed };
|
| 1320 |
|
| 1321 |
+
// The defects the SHIPPED strategies themselves exhibit under this run's own opponent and match
|
| 1322 |
+
// length. Null until the seeds have been measured, which is what disables the screen for them β
|
| 1323 |
+
// see point 3 in the header for why screening against the raw detector list was wrong.
|
| 1324 |
+
let tolerated = null;
|
| 1325 |
+
|
| 1326 |
// Evaluate one genome and try to seat it. A genome only ever displaces the CURRENT OCCUPANT of
|
| 1327 |
// its own cell, which is the whole mechanism: local competition, global diversity.
|
| 1328 |
const consider = (genome, origin) => {
|
|
|
|
| 1330 |
const cand = toCandidate(genome, name, { genes });
|
| 1331 |
const { desc, tripped } = describeCandidate(cand, descOpts);
|
| 1332 |
evaluated++;
|
| 1333 |
+
// Only defects the shipped AI does NOT also show here count against a genome.
|
| 1334 |
+
const novel = tolerated ? tripped.filter(t => !tolerated.has(t)) : [];
|
| 1335 |
+
if (screen && novel.length) {
|
| 1336 |
rejected++;
|
| 1337 |
+
log.push({ name, origin, cell: cellKey(desc), desc, tripped, novel, seated: false, reason: "screen" });
|
| 1338 |
// Report the refusal too, not just the seatings. A screen that is rejecting EVERYTHING looks
|
| 1339 |
// identical to a search that is finding nothing unless each refusal is visible β and there is
|
| 1340 |
// a specific, easy way to land in that state: CHECKS is calibrated for 40-60 sim-minute runs
|
| 1341 |
// (several detectors carry an "β¦and it never resolved in the last third" term keyed to that
|
| 1342 |
// length), so a short --descriptor-minutes makes every healthy genome trip dev-flatline.
|
| 1343 |
if (onCell) onCell(log[log.length - 1], archive);
|
| 1344 |
+
return { tripped, seated: false };
|
| 1345 |
}
|
| 1346 |
const key = cellKey(desc);
|
| 1347 |
const fitness = panelStrength(cand, panel, duelOpts);
|
| 1348 |
const incumbent = archive.get(key);
|
| 1349 |
const seated = !incumbent || fitness > incumbent.fitness;
|
| 1350 |
if (seated) archive.set(key, { key, genome, desc, fitness, name, origin, evaluatedAt: evaluated });
|
| 1351 |
+
log.push({ name, origin, cell: key, desc, tripped, novel, fitness: +fitness.toFixed(3), seated,
|
| 1352 |
beat: incumbent ? +incumbent.fitness.toFixed(3) : null });
|
| 1353 |
if (onCell) onCell(log[log.length - 1], archive);
|
| 1354 |
+
return { tripped, seated };
|
| 1355 |
};
|
| 1356 |
|
| 1357 |
// SEED from the shipped strategies. Four hand-written rows with real search already spent on
|
| 1358 |
// them, and β the point here β four rows that already sit in different parts of the behaviour
|
| 1359 |
// space, so the archive starts spread rather than clustered.
|
| 1360 |
+
//
|
| 1361 |
+
// They also CALIBRATE the screen: whatever they trip under this run's own opponent and match
|
| 1362 |
+
// length is what the detectors are saying about the configuration rather than about a genome, so
|
| 1363 |
+
// it becomes the tolerated set and the seeds themselves are never refused. `tolerated` stays null
|
| 1364 |
+
// until this loop finishes, which is what exempts them.
|
| 1365 |
+
const seedTrips = new Set();
|
| 1366 |
+
for (const key of seedStrategies)
|
| 1367 |
+
for (const t of consider(genomeFrom({ strategy: key, genes }), `seed:${key}`).tripped) seedTrips.add(t);
|
| 1368 |
+
tolerated = seedTrips;
|
| 1369 |
|
| 1370 |
while (evaluated < iterations) {
|
| 1371 |
const elites = [...archive.values()];
|
|
|
|
| 1393 |
return {
|
| 1394 |
cells: [...archive.values()].sort((x, y) => x.key < y.key ? -1 : 1),
|
| 1395 |
dims: ARCHIVE_DIMS, evaluated, rejected, coverage: archive.size / total, genes, log,
|
| 1396 |
+
tolerated: [...tolerated], // what the SHIPPED strategies trip here β report it, never hide it
|
| 1397 |
};
|
| 1398 |
}
|
| 1399 |
|
|
|
|
| 2020 |
console.log(` axes: ${ARCHIVE_DIMS.map(d => `${d.label} [${d.names.join("|")}]`).join(" x ")}`);
|
| 2021 |
console.log(` describe on: ${opts.descriptorWorlds.join(", ")} @ ${opts.descriptorMinutes}m vs ${opts.descriptorOpponent}`);
|
| 2022 |
console.log(` strength vs: ${panel.map(p => p.name).join(", ")} on ${opts.duelWorlds.join(", ")} @ ${opts.duelMinutes}m`);
|
| 2023 |
+
console.log(` screen: ${opts.screen ? "on β relative to what the SHIPPED strategies trip here" : "OFF"}`);
|
| 2024 |
console.log();
|
| 2025 |
|
| 2026 |
const res = runArchive({
|
|
|
|
| 2032 |
// and still be seated or simply lose its cell contest, and printing that as "refused" would
|
| 2033 |
// report a screen that isn't running.
|
| 2034 |
+ " " + (entry.seated ? (entry.beat != null ? `seated (beat ${(entry.beat * 100).toFixed(0)}%)` : "SEATED β new cell")
|
| 2035 |
+
// Name the NOVEL defect, not everything it tripped: the tolerated ones are what the
|
| 2036 |
+
// shipped strategies do here too, so listing them would blame a genome for the
|
| 2037 |
+
// configuration.
|
| 2038 |
+
: entry.reason === "screen" ? `refused [${entry.novel.join(",")}]`
|
| 2039 |
: `no (holder ${(entry.beat * 100).toFixed(0)}%)`)
|
| 2040 |
+ ` ${archive.size} cells`),
|
| 2041 |
});
|
|
|
|
| 2045 |
const [xDim, yDim] = ARCHIVE_DIMS;
|
| 2046 |
const at = (x, y) => res.cells.find(c => c.key === `${xDim.names[x]}/${yDim.names[y]}`);
|
| 2047 |
console.log(`\nTHE CAST β ${res.cells.length}/${xDim.names.length * yDim.names.length} cells filled `
|
| 2048 |
+
+ `(${(res.coverage * 100).toFixed(0)}% coverage), ${res.evaluated} evaluated, ${res.rejected} refused by the health screen`);
|
| 2049 |
+
// Never hide the calibration. If the shipped strategies trip everything here, the screen is a
|
| 2050 |
+
// no-op and the reader has to be able to see that rather than infer it from a suspiciously
|
| 2051 |
+
// clean run.
|
| 2052 |
+
console.log(` screen tolerated (what the SHIPPED strategies trip under these settings): `
|
| 2053 |
+
+ `${res.tolerated.length ? res.tolerated.join(", ") : "nothing β the full detector list was enforced"}\n`);
|
| 2054 |
console.log(pad("", 9) + xDim.names.map(n => pad(n, 16)).join("") + ` <- ${xDim.label}`);
|
| 2055 |
for (let y = 0; y < yDim.names.length; y++) {
|
| 2056 |
console.log(pad(yDim.names[y], 9) + xDim.names.map((_, x) => {
|
|
|
|
| 2081 |
// An archive that came back (nearly) empty has one overwhelmingly likely cause, and it is not
|
| 2082 |
// "no good AI exists". Say so, rather than leaving a blank grid to be read as a result.
|
| 2083 |
if (res.rejected > res.evaluated * 0.8)
|
| 2084 |
+
console.log(`\n!! ${res.rejected} of ${res.evaluated} genomes were REFUSED by the health screen,`
|
| 2085 |
+
+ `\n for defects the shipped strategies do NOT show here. That is a lot, and worth reading`
|
| 2086 |
+
+ `\n before trusting: --descriptor-minutes is ${opts.descriptorMinutes} against '${opts.descriptorOpponent}'.`
|
| 2087 |
+
+ `\n Pass --screen false to seat genomes unscreened (and lose the defect filter).`);
|
|
|
|
| 2088 |
if (args.json) { writeFileSync(args.json, JSON.stringify(res.log, null, 1)); console.log(`wrote ${args.json}`); }
|
| 2089 |
},
|
| 2090 |
};
|
|
|
|
| 2176 |
measured on the solo Odyssey bench) and competes ONLY against the
|
| 2177 |
current occupant of its own cell -- so a genome that wins by refusing
|
| 2178 |
to fight can win the "never attacks" cell and nothing else, which is a
|
| 2179 |
+
structural answer to the Goodhart failure 'evolve' hit rather than a
|
| 2180 |
hand-written penalty for it. Strength is measured against a FIXED
|
| 2181 |
panel, so it is absolute and comparable across the whole run. The
|
| 2182 |
+
health screen is ON by default, free (describing a genome IS running
|
| 2183 |
+
it solo), and calibrated RELATIVE to the shipped strategies: a genome
|
| 2184 |
+
is refused only for a defect they do not also show under the same
|
| 2185 |
+
opponent and match length, because CHECKS is calibrated against the
|
| 2186 |
+
'passive' bot and this runs against a provoking one.
|
| 2187 |
Writes one runnable candidate per filled cell.
|
| 2188 |
|
| 2189 |
Common flags
|