Claude Claude Opus 5 commited on
Commit
b63da6d
·
unverified ·
1 Parent(s): 42d791d

Correct three bench detectors that the fixes turned into false positives

Browse files

Scaling the AI up changed what its healthy behaviour looks like, and three of
the five health checks kept measuring the old shape. A detector that fires on
behaviour the design intends is worse than no detector, because the tuning loop
optimises against it — so each is rewritten and pinned with a test:

- production-stall now needs EVERY Barracks idle, and excludes strategies
that deliberately cap their standing army. An "any Barracks idle while
holding 400" test fired on 42 of 44 HEALTHY runs once surplus opened six of
them, and idle Barracks are the literal point of Economic / Force Parity.
- hoarding now needs a large FINAL bank with a frozen army, not a transient
peak. A working economy peaks high and spends straight back down; the
peak-based test scored it the same as the AI that banked 30,000 and stopped.
- supply-deadlock now needs the AI to be blocked with no Habitat on the way.
A busy AI lives close to its cap and is momentarily blocked all the time —
that is supply pressure, not the state that never resolves. Measured: helix
grew its army 75 -> 327 and drained a 10,000 bank to 854 while the old test
still called it deadlocked.

Also: thrift scores what the AI ENDED holding rather than what it ever touched,
for the same reason.

Review doc gains §2.9, a finding that only appeared once the bench had an
opponent that fights: against a basic rush the neighbour is dead by minute 10
on most of the roster, and none of the five long-game fixes touch it, because
the long game never arrives. Reported rather than rebalanced — razing a
neighbour is galaxy.pacified, an intended conquest milestone, so how hard that
should be is a call about the Odyssey difficulty curve.

The skill picks up the new contract (neverInitiates is Odyssey-conditional now),
the skirmisher opponent, and the rule that a changed metric means re-baselining
the old engine under the new definition rather than comparing across them.

Suite 1211 green, typecheck clean.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V6aK9g6A9dYtyinJSVXHqQ

.claude/skills/ai-lab/SKILL.md CHANGED
@@ -67,7 +67,11 @@ than optimising the default and describing the result as an improvement.
67
  pressure ever arrive at all".
68
  - `--opponent turtle` — a real economy behind turrets that never attacks. Use for
69
  "can the AI crack a defended base", which is the question that decides whether an
70
- Odyssey neighbour is threatening.
 
 
 
 
71
 
72
  A result from one opponent says nothing about the others. State which one you used.
73
 
@@ -86,10 +90,19 @@ noisy mean and will happily overfit three worlds.
86
  - **Layers compose multiplicatively.** Archetype × strategy × difficulty all multiply
87
  (`graceMult`, `workerTargetMult`, …). A dial that looks mild in isolation can be a 3×
88
  swing on Hard + Aggressive + Rusher. Always sweep across difficulties when touching one.
89
- - **`neverInitiates` is absolute**, including the skirmish desperation timeout. It was made
90
- absolute on purpose (see the comment in `engine/aiStrategy.js`) — a passive player was
91
- eating unexplained all-in waves. Do not "fix" the Odyssey side of it by removing the flag;
92
- add a separate Odyssey-only provocation path if that's the goal.
 
 
 
 
 
 
 
 
 
93
  - **Neighbour worlds pick their strategy uniformly at random** (`neighbourAiProfile` in
94
  `engine/galaxy.js`), so any change to the STRATEGIES table changes roughly a quarter of the
95
  galaxy. Weight the sweep accordingly.
 
67
  pressure ever arrive at all".
68
  - `--opponent turtle` — a real economy behind turrets that never attacks. Use for
69
  "can the AI crack a defended base", which is the question that decides whether an
70
+ Odyssey neighbour is threatening. (Its turrets still kill the AI's scouts, so it *does*
71
+ provoke — `passive` is the only bot that never draws blood.)
72
+ - `--opponent skirmisher` — a turtle that also throws its army at the AI. The only bot that
73
+ reliably provokes, so it is the one that answers "does a never-initiating neighbour push
74
+ back once it's entitled to?" and "does the AI hold up under sustained attack?".
75
 
76
  A result from one opponent says nothing about the others. State which one you used.
77
 
 
90
  - **Layers compose multiplicatively.** Archetype × strategy × difficulty all multiply
91
  (`graceMult`, `workerTargetMult`, …). A dial that looks mild in isolation can be a 3×
92
  swing on Hard + Aggressive + Rusher. Always sweep across difficulties when touching one.
93
+ - **`neverInitiates` means "doesn't start fights", and only in Odyssey.** In a skirmish it is
94
+ absolute, including the desperation timeout — made so on purpose, because a passive player
95
+ was eating unexplained all-in waves. In Odyssey a neighbour may still answer a player who
96
+ **provoked** it (`engine/diplomacy.js` `provoked()`: destroyed its ships, or is charging a
97
+ Gate). Never widen provocation to elapsed time or stance alone — that reintroduces the exact
98
+ bug. And note what it means for measurement: against `passive` and `none` those strategies
99
+ correctly commit nothing, so use `--opponent skirmisher` when the question is whether the AI
100
+ pushes back.
101
+ - **A metric that fires on correct behaviour is worse than no metric.** Two detectors had to be
102
+ rewritten after they turned into false positives on a scaled-up AI. When a fix lands, re-read
103
+ the detectors before trusting the scoreboard — and if you change one, re-baseline the OLD
104
+ engine under the NEW metric (a `git worktree` at the pre-fix commit with the current
105
+ `tools/ailab.js` copied in) rather than comparing across metric definitions.
106
  - **Neighbour worlds pick their strategy uniformly at random** (`neighbourAiProfile` in
107
  `engine/galaxy.js`), so any change to the STRATEGIES table changes roughly a quarter of the
108
  galaxy. Weight the sweep accordingly.
CHANGELOG.md CHANGED
@@ -6,6 +6,33 @@ All notable changes to this project are documented here. The format follows
6
 
7
  ## [Unreleased]
8
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
9
  ### Added
10
 
11
  - `tools/ailab.js` — a headless bench for the Odyssey AI. Runs the real sim for 30–60 sim-minutes
@@ -13,7 +40,8 @@ All notable changes to this project are documented here. The format follows
13
  scores the opponent on development / growth / pressure / thrift / liveness / survival. Candidate
14
  AIs are injected as JSON into the archetype, strategy and difficulty tables (`--overrides`), so
15
  the whole space can be searched without touching `engine/`. Subcommands: `probe`, `sweep`,
16
- `compare`, `search`, `check`. No engine behaviour changed.
 
17
  - `docs/odyssey-ai-review.md` — a measured review of how the AI plays in Odyssey, with the
18
  findings the bench turned up (a supply deadlock that permanently freezes a developed AI, a
19
  terminating build order, a Rusher that never develops, and a non-initiating half of the galaxy),
 
6
 
7
  ## [Unreleased]
8
 
9
+ ### Fixed
10
+
11
+ - **The Odyssey AI no longer freezes solid in a long session.** Its Habitat trigger was sized for
12
+ a 2-supply unit while the Odyssey roster has 4- and 8-supply ones, and the unit-mix cycle retries
13
+ the same entry until it succeeds — so a developed neighbour could wedge permanently with an empty
14
+ production queue and tens of thousands of unspent ore. The supply margin is now sized from what
15
+ the AI is actually trying to build, supply already under construction is credited, and Habitats
16
+ are placed at any of its Command Centers rather than only the capital.
17
+ - **The AI develops on worlds where it never used to.** Unit production ran before industry with no
18
+ ore held back, so a cheap-unit archetype spent itself below a Reactor's cost every think cycle and
19
+ never built one; Hard's rusher graduation landed roughly 30 minutes past its own trigger. A
20
+ bounded bootstrap reserve now banks for the power grid and the first two factories.
21
+ - **A rich neighbour spends its income instead of hoarding it.** Sustained surplus opens extra
22
+ Barracks, and — for strategies that cap their standing army — funds a colony ship regardless of
23
+ home depletion, so the build order no longer terminates in a mode that doesn't.
24
+ - **A neighbour dragged to Hostile can now act on it.** The Economic and Force Parity strategies
25
+ could never attack under any circumstances, and Odyssey hands one of them to roughly half the
26
+ galaxy's worlds — so half a galaxy read "Hostile" and never sent anything. In Odyssey those
27
+ strategies now mean "doesn't start fights": they still answer a player who has destroyed their
28
+ ships or begun charging an Antimatter Gate. Skirmish behaviour is unchanged.
29
+ - **The opening grace window can't collapse any more.** The archetype, strategy and difficulty
30
+ grace/grievance multipliers all compounded, cutting the documented 7-minute window to under 40
31
+ seconds on an Aggressive + Hard combination the setup screen offers directly. The layers still
32
+ compose; they're now bounded.
33
+ - A neighbour deploying its own colony ship (or triggering its own Helium Bomb) counted as you
34
+ destroying one of its ships, so it soured its own stance for founding a colony.
35
+
36
  ### Added
37
 
38
  - `tools/ailab.js` — a headless bench for the Odyssey AI. Runs the real sim for 30–60 sim-minutes
 
40
  scores the opponent on development / growth / pressure / thrift / liveness / survival. Candidate
41
  AIs are injected as JSON into the archetype, strategy and difficulty tables (`--overrides`), so
42
  the whole space can be searched without touching `engine/`. Subcommands: `probe`, `sweep`,
43
+ `compare`, `search`, `check`. Four scripted sparring opponents, including a `skirmisher` that
44
+ actually attacks.
45
  - `docs/odyssey-ai-review.md` — a measured review of how the AI plays in Odyssey, with the
46
  findings the bench turned up (a supply deadlock that permanently freezes a developed AI, a
47
  terminating build order, a Rusher that never develops, and a non-initiating half of the galaxy),
docs/odyssey-ai-review.md CHANGED
@@ -272,6 +272,41 @@ controller owner-parametric — `state.ais[owner]` — is a contained refactor,
272
  unlock true self-play: the strongest possible evaluation signal, replacing the scripted
273
  sparring bots below.
274
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
275
  ---
276
 
277
  ## 3. The proposal — searching for better AI with Claude Code
@@ -320,8 +355,9 @@ drives the player side with scripted bots through the engine's public command AP
320
  | bot | what it is | the question it answers |
321
  |---|---|---|
322
  | `none` | no player at all | how does the development curve look on a background world? |
323
- | `passive` | seats a base, never acts | does pressure ever arrive at all? |
324
  | `turtle` | economy behind turrets, never attacks | can the AI crack a defended base? |
 
325
 
326
  **Metrics and score.** Six components, weighted in `WEIGHTS` and printed individually:
327
  `develop` (climbs the chain), `keepGrowing` (still growing in the last third — the
 
272
  unlock true self-play: the strongest possible evaluation signal, replacing the scripted
273
  sparring bots below.
274
 
275
+ ### 2.9 The AI cannot survive an early rush — *new, found while fixing the rest*
276
+
277
+ Building a sparring bot that actually fights (`--opponent skirmisher`: a turtle economy that
278
+ throws its army at the AI whenever it musters six units) opened a measurement none of the
279
+ original findings covered, and the answer is stark. On ferros the neighbour is **dead by minute
280
+ 10** of a 40-minute run — no Command Center, no workers, and the world inert for the remaining
281
+ thirty minutes:
282
+
283
+ ```
284
+ time wrk army dev bldg banked waves
285
+ 5m 7 1 3 7 494 0
286
+ 10m 0 0 0 0 662 0
287
+ 40m 0 0 0 0 662 0
288
+ ```
289
+
290
+ It is not close, and it is not one world: the AI fails to survive on most of the roster. The
291
+ bot is not cheating — it deploys the same opening colony ship, mines with its own workers, and
292
+ pays for every unit.
293
+
294
+ **Status: reported, not fixed — this one is a design decision, not a defect.** Razing a
295
+ neighbour's Command Center is `galaxy.pacified`, an intended conquest milestone with its own
296
+ firework and a domination counter, so "a determined player can kill a neighbour" is the *feature*.
297
+ What the bench can say is that today the bar is very low and the same rush works everywhere.
298
+ Whether that bar should move — and how far — is a call about the difficulty curve of the whole
299
+ Odyssey, and it belongs to whoever owns that curve, not to a defect-fix pass.
300
+
301
+ Worth knowing before deciding: **none of the five fixes above touch this.** They all improve the
302
+ long game, and against a rushing opponent the long game never arrives. That is visible in the
303
+ comparison table at the top of this section — the passive column moves a lot, the skirmisher
304
+ column barely does.
305
+
306
+ ```
307
+ node tools/ailab.js probe --world ferros --opponent skirmisher --minutes 40 --sample 5
308
+ ```
309
+
310
  ---
311
 
312
  ## 3. The proposal — searching for better AI with Claude Code
 
355
  | bot | what it is | the question it answers |
356
  |---|---|---|
357
  | `none` | no player at all | how does the development curve look on a background world? |
358
+ | `passive` | seats a base, never acts | does pressure ever arrive at all? the only bot that never draws blood |
359
  | `turtle` | economy behind turrets, never attacks | can the AI crack a defended base? |
360
+ | `skirmisher` | turtle that also throws its army at the AI | does a provoked neighbour push back — and does the AI survive being rushed? |
361
 
362
  **Metrics and score.** Six components, weighted in `WEIGHTS` and printed individually:
363
  `develop` (climbs the chain), `keepGrowing` (still growing in the last third — the
test/ailab.test.js CHANGED
@@ -166,3 +166,28 @@ test("hoarding means a bank it never spent, not a bank it passed through", () =>
166
  assert.ok(!hoard.hit({ bankedFinal: 2200, armyGrowthTail: 51 }), "a working balance with a growing army is not");
167
  assert.ok(!hoard.hit({ bankedFinal: 30000, armyGrowthTail: 40 }), "…nor is a big balance it's actively converting");
168
  });
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
166
  assert.ok(!hoard.hit({ bankedFinal: 2200, armyGrowthTail: 51 }), "a working balance with a growing army is not");
167
  assert.ok(!hoard.hit({ bankedFinal: 30000, armyGrowthTail: 40 }), "…nor is a big balance it's actively converting");
168
  });
169
+
170
+ test("supply PRESSURE with a Habitat on the way is not a deadlock", () => {
171
+ // A healthy AI at full tilt lives close to its cap and is momentarily unable to fit the next
172
+ // unit all the time. A third detector had to learn that difference: measured, helix grew its army
173
+ // 75 -> 327 and drained a 10,000 bank to 854 while the old test still called it deadlocked.
174
+ const base = { dev: 0, army: 0, waves: 0, hostility: 0, playerBuildings: 1, entitled: true,
175
+ armyValue: 0, workers: 0, buildings: 0, rax: 1, idleRax: 0, aiAlive: true, t: 0 };
176
+ const frac = c => summarise([{ ...base, ...c }]).supplyDeadlockFrac;
177
+ assert.equal(frac({ supplyBlocked: true, habitatPending: true, banked: 5000 }), 0,
178
+ "blocked, but a Habitat is already going up — it resolves itself");
179
+ assert.equal(frac({ supplyBlocked: true, habitatPending: false, banked: 5000 }), 1,
180
+ "blocked with nothing on the way is the state that never resolves");
181
+ });
182
+
183
+ test("a strategy that deliberately caps its army isn't reported as a production stall", () => {
184
+ // Economic keeps 3 units and Force Parity mirrors what it has seen — idle Barracks are the
185
+ // POINT of those strategies, and counting them made the detector fire on the design working.
186
+ const base = { dev: 0, army: 0, waves: 0, hostility: 0, playerBuildings: 1, entitled: true,
187
+ armyValue: 0, workers: 0, buildings: 0, rax: 2, idleRax: 2, banked: 5000,
188
+ supplyBlocked: false, habitatPending: false, aiAlive: true, t: 0 };
189
+ assert.equal(summarise([{ ...base, armyCapped: true }]).idleRichFrac, 0,
190
+ "an army-capped strategy sitting on idle Barracks is doing what it was asked to");
191
+ assert.equal(summarise([{ ...base, armyCapped: false }]).idleRichFrac, 1,
192
+ "…an uncapped one doing the same has run out of things to buy");
193
+ });
tools/ailab.js CHANGED
@@ -20,11 +20,16 @@
20
  node tools/ailab.js probe [--world ferros] [--strategy default] [--difficulty medium]
21
  [--opponent passive] [--minutes 40] [--sample 5]
22
  node tools/ailab.js sweep [--worlds a,b] [--strategies a,b] [--difficulties a,b]
23
- [--seeds 3] [--minutes 40] [--json out.json] [--csv out.csv]
24
  node tools/ailab.js compare --baseline base.json --candidate cand.json
25
  node tools/ailab.js search --strategy aggressive --dials 'attackTimeoutMult=0.3:1,garrisonMult=0:1'
26
- [--trials 24] [--worlds a,b]
27
- node tools/ailab.js check [--worlds a,b] [--minutes 60] # health checks (stall/hoard/flatline)
 
 
 
 
 
28
 
29
  Common flags
30
  --overrides f.json inject rows into the AI tables before running, e.g.
@@ -251,6 +256,10 @@ function sample(s) {
251
  // and the Habitat trigger only fires within 2 supply of the cap, which doesn't cover the
252
  // 4- and 8-supply Odyssey units. That combination is a hard deadlock, so it's measured.
253
  supplyBlocked: !!next && supplyUsed(s, "ai") + (UNITS[next].supplyCost || 0) > supplyCap(s, "ai"),
 
 
 
 
254
  supplyFree: +(supplyCap(s, "ai") - supplyUsed(s, "ai")).toFixed(1),
255
  playerBuildings: playerBuildingsOf(s).length,
256
  aiAlive: done.some(b => b.type === "command"),
@@ -260,6 +269,11 @@ function sample(s) {
260
  // that line — a neighbour leaving an unprovoked player alone is the contract working, and
261
  // scoring it as a failure would push the tuning loop toward breaking it.
262
  entitled: !strategyFor(s).neverInitiates || provoked(s),
 
 
 
 
 
263
  provokedAi: provoked(s), // has the player drawn blood / started a Gate? (diagnostic + the `entitled` input)
264
  };
265
  }
@@ -319,8 +333,10 @@ function summarise(curve) {
319
  // extra Barracks, one of six sitting between jobs is ordinary churn, and an "any" test fired on
320
  // 42 of 44 healthy runs. This is the version that distinguishes a working production line from
321
  // a stopped one.
322
- idleRichFrac: +(curve.filter(c => c.rax > 0 && c.idleRax === c.rax && c.banked > 1000).length / curve.length).toFixed(2),
323
- supplyDeadlockFrac: +(curve.filter(c => c.supplyBlocked && c.banked > 400).length / curve.length).toFixed(2),
 
 
324
  stanceFinal: last.stance,
325
  playerBuildingsFinal: last.playerBuildings,
326
  aiAlive: last.aiAlive,
@@ -403,8 +419,9 @@ export function applyOverrides(ov = {}) {
403
 
404
  export const CHECKS = [
405
  { id: "supply-deadlock",
406
- why: "the mix cycle retries one unit forever because it doesn't fit under the supply cap, "
407
- + "while the Habitat trigger (used >= cap - 2) never fires for a 4- or 8-supply unit",
 
408
  hit: r => r.supplyDeadlockFrac > 0.1 },
409
  { id: "hoarding",
410
  why: "it finished sitting on a large bank AND its army hadn't grown — money it has no way to "
@@ -418,7 +435,8 @@ export const CHECKS = [
418
  + "(samples where a never-initiating strategy was left unprovoked don't count — that's the contract)",
419
  hit: r => r.entitledSamples > 0 && r.hostileIdleFrac > 0.5 },
420
  { id: "production-stall",
421
- why: "a completed Barracks stood idle while the AI held real money",
 
422
  hit: r => r.idleRichFrac > 0.25 },
423
  ];
424
 
 
20
  node tools/ailab.js probe [--world ferros] [--strategy default] [--difficulty medium]
21
  [--opponent passive] [--minutes 40] [--sample 5]
22
  node tools/ailab.js sweep [--worlds a,b] [--strategies a,b] [--difficulties a,b]
23
+ [--seeds 3] [--minutes 40] [--json out.json] [--csv out.csv] [--full]
24
  node tools/ailab.js compare --baseline base.json --candidate cand.json
25
  node tools/ailab.js search --strategy aggressive --dials 'attackTimeoutMult=0.3:1,garrisonMult=0:1'
26
+ [--steps 4] [--worlds a,b]
27
+ node tools/ailab.js check [--worlds a,b] [--minutes 60] [--exit-code] # the named-defect list
28
+
29
+ Opponents (--opponent) decide what the AI is measured AGAINST, and the answer differs
30
+ completely between them: none (no player at all) · passive (seats a base, never acts — the
31
+ only bot that never draws blood) · turtle (economy behind turrets) · skirmisher (a turtle that
32
+ also attacks, so it is the one that provokes).
33
 
34
  Common flags
35
  --overrides f.json inject rows into the AI tables before running, e.g.
 
256
  // and the Habitat trigger only fires within 2 supply of the cap, which doesn't cover the
257
  // 4- and 8-supply Odyssey units. That combination is a hard deadlock, so it's measured.
258
  supplyBlocked: !!next && supplyUsed(s, "ai") + (UNITS[next].supplyCost || 0) > supplyCap(s, "ai"),
259
+ // …and is it doing anything about it? A healthy AI at full tilt runs close to its cap and is
260
+ // momentarily blocked all the time — that's supply PRESSURE, not deadlock. The deadlock is
261
+ // being blocked with no Habitat on the way, which is the state that never resolves itself.
262
+ habitatPending: [...s.buildings.values()].some(b => b.owner === "ai" && b.type === "habitat" && b.constructing),
263
  supplyFree: +(supplyCap(s, "ai") - supplyUsed(s, "ai")).toFixed(1),
264
  playerBuildings: playerBuildingsOf(s).length,
265
  aiAlive: done.some(b => b.type === "command"),
 
269
  // that line — a neighbour leaving an unprovoked player alone is the contract working, and
270
  // scoring it as a failure would push the tuning loop toward breaking it.
271
  entitled: !strategyFor(s).neverInitiates || provoked(s),
272
+ // Is this strategy DELIBERATELY holding its army down (Economic's fixed cap, Force Parity's
273
+ // mirror)? Idle Barracks are the intended output of a standing-army cap, so the stall detector
274
+ // has to exclude them — the same principle as `entitled` above: a detector that fires on
275
+ // behaviour the design asks for teaches the tuning loop to break the design.
276
+ armyCapped: strategyFor(s).standingArmyCap != null || !!strategyFor(s).matchEnemyForce,
277
  provokedAi: provoked(s), // has the player drawn blood / started a Gate? (diagnostic + the `entitled` input)
278
  };
279
  }
 
333
  // extra Barracks, one of six sitting between jobs is ordinary churn, and an "any" test fired on
334
  // 42 of 44 healthy runs. This is the version that distinguishes a working production line from
335
  // a stopped one.
336
+ idleRichFrac: +(curve.filter(c => !c.armyCapped && c.rax > 0 && c.idleRax === c.rax && c.banked > 1000)
337
+ .length / curve.length).toFixed(2),
338
+ supplyDeadlockFrac: +(curve.filter(c => c.supplyBlocked && !c.habitatPending && c.banked > 400)
339
+ .length / curve.length).toFixed(2),
340
  stanceFinal: last.stance,
341
  playerBuildingsFinal: last.playerBuildings,
342
  aiAlive: last.aiAlive,
 
419
 
420
  export const CHECKS = [
421
  { id: "supply-deadlock",
422
+ why: "the mix cycle is stuck on a unit that won't fit under the supply cap, with no Habitat on "
423
+ + "the way to raise it — the state that never resolves itself (being momentarily blocked "
424
+ + "WITH one under construction is ordinary supply pressure on a busy AI, not a deadlock)",
425
  hit: r => r.supplyDeadlockFrac > 0.1 },
426
  { id: "hoarding",
427
  why: "it finished sitting on a large bank AND its army hadn't grown — money it has no way to "
 
435
  + "(samples where a never-initiating strategy was left unprovoked don't count — that's the contract)",
436
  hit: r => r.entitledSamples > 0 && r.hostileIdleFrac > 0.5 },
437
  { id: "production-stall",
438
+ why: "every Barracks stood idle while the AI held real money, on a strategy that isn't "
439
+ + "deliberately capping its army — it had nothing left it knew how to buy",
440
  hit: r => r.idleRichFrac > 0.25 },
441
  ];
442