Spaces:
Paused
Paused
Commit History
-
The cast's labels do not survive held-out maps -- say so
Claude +1: Claude Opus 5Unverified signatureVerified four cells on replicates the archive never trained on. Three moved bin. The harness was checked before the result was believed: re-run on the archive's own replicates it reproduces the filed descriptor exactly, so it is faithful and the descriptor is not stable. Root cause is that the residual variance is ACROSS WORLDS rather than within one. One genome held fixed measures army 3/3 on korrath, 58/143 on ferros and 47/208 on vesper; a pooled median over that is a description of the roster, not of the genome. Both fixes that landed earlier were necessary and neither was sufficient -- the median really did fix the heavy-tail pathology it was aimed at, and the pinned runSeed key really did make re-measurement possible at all, but neither touches cross-world spread. Coverage fell 15/16 to 10/16, and that part IS the fix working: the first run's 94% was inflated by a descriptor scattering genomes into bins at random. The genomes and their strength numbers stand -- 16 real matches…cc7170f -
The corrected cast: 10 cells from the median-descriptor archive
Claude +1: Claude Opus 5Unverified signatureReplaces the 15 cells the first archive produced, six of which existed only because the old mean-based descriptor was scattering genomes into bins at random. Coverage drops 94% -> 63%, and that drop IS the fix working rather than a regression: the earlier number was inflated by measurement noise, and this one is what a tightly-clustered, honestly-measured population looks like. The behaviours now separate cleanly -- waves 0.0 to 13.5, army 1 to 58 -- and the fitness spread (38-88% against the fixed four-baseline panel) tracks something real, with the small-army column scoring highest and the sprawling waves/large lowest. Held-out-replicate verification is running: the archive measured on replicates 0-1, so 2-3 are maps it never saw. Re-measuring on the SAME maps would be tautological now that runSeed is pinned, which is why the check has to use different ones. The README is updated once that lands -- it is the check the first cast failed, when two cells three bins apart swapped ends …dd5b8f9 -
Descriptors are a MEDIAN, and runSeed stops hashing the candidate's name
Claude +1: Claude Opus 5Unverified signatureThe archive run filled 15 of 16 cells at 94% coverage, and then verification undercut it: two cells filed three bins apart on the army axis (8 vs 113) differ in exactly one active gene, and re-measuring them returned 93 and 2 -- they had swapped ends of the axis. The cells were not distinct; the measurement was dealing cards. Probing it properly, one genome held fixed on pinned maps over 4 replicates per world: korrath 26/23/22/27 (stable), vesper 1/0/8/0, ferros 36/3/1/296. The distribution is HEAVY-TAILED, not merely noisy -- an Odyssey economy that gets going compounds, so about one run in twelve returns an order of magnitude more army than the rest. That rules out the fix already written. Averaging inherits a heavy tail rather than smoothing it, and the same twelve runs say so: the MEAN reads 21.0 / 14.8 / 36.9 at 1 / 2 / 4 replicates -- diverging, because whichever sample holds the 296 decides the answer -- while the MEDIAN over the identical samples reads 26 / 13 / 15 and conve…ee4f10e
-
The first evolved candidate: 12 genomes x 6 generations, 3h21m
Claude +1: Claude Opus 5Unverified signatureChampion is generation 3's g3i0 -- later generations never beat it, which is itself the run's most useful result (see below). Recorded as a candidate file so it can be duelled and swept like any hand-written one; NOT promoted into engine/, and on the evidence so far it should not be. What it actually is, once the inert genes are stripped: max economy, uncapped army, never attack. Of the eleven dials it emits, three do anything -- neverInitiates, workerTargetMult, turretCountMult. attackTimeoutMult/ armyAttackSizeMult/garrisonMult are dead under neverInitiates (aiMilitary.js's readyToAttack gate), warFootingMult/Time are dead without a standingArmyCap, and matchBuffer/matchFloor are dead without matchEnemyForce. Junk DNA, exactly as the design doc's Sec 3.3 predicted, and carried rather than erased because the `|| 1` convention means an unread gene costs nothing. It found this because engine/victory.js's score-at-clock tiebreak weights combat units at 1.35x and banked ore at 0.25x, so…eddfd01
-
Add ailab leaderboard: Tier 0 of the AI championship roadmap
ClaudeUnverified signatureRanks N candidate strategies against the same fixed sparring bot, with difficulty/opponent/worlds/seeds held identical across every candidate so none gets an APM or micro edge — a proxy signal usable today, with no engine changes, ahead of true head-to-head self-play (docs §2.8, which needs engine/ai.js's controller to become owner-parametric). - tools/ailab.js: runLeaderboard + snapshotTables/restoreTables (clean per-candidate override isolation) + the `leaderboard` CLI command. - tools/candidates/: five runnable example candidates. - test/ailab.test.js: determinism, override isolation (including on throw), and ranking-correctness coverage. - docs/odyssey-ai-review.md + .claude/skills/ai-lab/SKILL.md: document the command and its "proxy, not self-play" caveat.d71f0e0