Spaces:
Running
Running
Commit History
-
win-speed bonus: record + reward how fast a model wins
Per user: a faster win is worth a bonus and must be recorded/compared. - scoring.py: _win_budget (tightest within_ticks in the win tree, else max_ticks); ScoreCard gains win_tick/win_turns/win_budget/speed/ composite_base (in score.json + manifest). On a WIN, composite += SPEED_BONUS*speed (0.05 cap, speed=1-win_tick/budget) — orders fast vs slow wins, never lifts a loss above a win or overrides correctness. - run_eval._agg: win_speed_mean / win_turns_mean (over wins only). - leaderboard: records win_speed/win_turns; composite/win_rate ties now broken by faster wins. - SCENARIO_QUALITY: #2 closer-look note + win-speed section. +5 scoring/budget tests; suite 362 passed.35d7d47 -
#2 hard: set base_map INSIDE overrides so it actually uses scout-arena
The first hard run crashed with a Rust panic (index oob len 5120 = 128*40 idx 6922 = cell (10,54) = scout-arena spawn_point-1). Root cause was NOT an engine bug: hard's base_map was written as a Level- level key, which the Level model silently ignores, so hard ran on the pack-default rush-hour-arena (128x40) with scout-arena-scale actor coords -> the enemy_building_spotted interrupt path indexed the 128x40 shroud/buffers out of bounds. Fix: move base_map: scout-arena INTO hard.overrides (deep-merged into ScenarioDefinition). Verified: hard now compiles to scout-arena, engine loads 176x80, 60 steps of the building-spotted interrupt path run with no panic. Added guard test (Level-level base_map is silently ignored). Suite 359 passed; docs regenerated.cdd813f -
#2 hard: per-waypoint search BAND in the relative objective
Per user: hard should give a coarse area to scout for each waypoint, not a bare compass word. Rewrote all 8 waypoint labels to express a relative search band (e.g. 'the far NORTH-WEST — search the western ~quarter, in the northern ~third'); still no coordinates (enemy_ building_spotted interrupt pins the exact fogged marker on sighting). Verified no coord leak in the relative brief. Suite 358; docs regen. Also recorded: #2 easy WIN (ordered W1->W2->W3, tick 1893<2400); medium LOSS = valid model failure (split correctly, completed route S in order but lost column discipline on route N and timed out — exactly the re-deliberation failure the pack targets; scenario solvable, no compensation).a9b5126 -
#2: enforce ordered waypoints (stateful waypoint_sequence) + two-path medium + scouted hard
Root defect (closer look): the pack claimed to test ordered non-stalling route execution but the win condition only referenced the final region + a tick band — a beeline-to-final that skipped every waypoint and idled to the gate won. Fix: - New reusable stateful predicate waypoint_sequence: latches ordered region-visit progress (monotonic) on a per-episode signals.seq_progress scratch. Wk+1 only counts after Wk reached; skip/idle/out-of-order => never satisfied. Coords-aware objective phrase (exact|relative). - easy: ONE ordered 3-waypoint route, clear, generous budget. - medium: TWO crossing ordered routes BOTH required -> must split into two parallel columns (serial overruns the clock); attrition cap. - hard: TWO 4-waypoint serpentine routes on a NEW larger map (scout-arena 176x80, scripts/build_scout_arena_map.py), given by RELATIVE direction only (no coords) with the enemy_building_spotted interrupt revealing fogged markers on sighting -> real scouting; seed -varie…b165246 -
action-sequenced-execution: timeout/attrition overrun is a LOSS, not a draw
Closer look on #2: easy/medium had no fail_condition, so overrunning the [after_ticks, within_ticks] takt-time band (the scenario's actual teeth) recorded as a draw at max_turns instead of a failure. Per the established sequenced-route idiom (cf artofwar-sequenced-citadel: a single final region graded by an arrival-timing band, not literal waypoint predicates), keep the design and add real fail_conditions: deadline overrun -> loss on all 3 levels; on medium/hard also lose when the attrition cap is busted (win unreachable once >1 lost) or the force is destroyed. Win/fail tick bands are disjoint (no overlap at the boundary). Suite 351 passed; catalog regenerated.01e12d1 -
hard: enemy_building_spotted interrupt closes the coordinate-blind feedback loop
Second hard run still lost: the model named the right buildings and split correctly but, with no coords and the markers fogged all episode, could only visually estimate from the minimap and parked ~15 cells short every time — the landmark never entered sight, so no corrective signal ('identify on the minimap' was impossible). Per user: building-found should trigger an interrupt. Enable enemy_building_spotted in hard's overrides (engine env.rs already emits it; eval_core already supports it) so the agent is re-prompted the instant a unit sights a marker, revealing its exact position; r7 becomes fair since the sighting yields coords. Objective rewritten to instruct: keep advancing in each direction until the sighting interrupt, then converge. easy/medium unchanged (no interrupts). Suite 351 passed; catalog regenerated.d58c824 -
hard: anchor relative objective on the named enemy building type per direction
First hard run (qwen3.6-flash) lost: the model reasonably read 'the NORTH-EAST corner' as the literal map extreme and parked at the geometric corners, while the win pockets are inset ~13-20 cells (r7) — under-determined for a coordinate-blind agent, and units overshot past the landmark buildings so the intended visual aid never entered vision. Fix per user: name the specific enemy building in each direction (Construction Yard NE / Power Plant SW / Ore Refinery SE) so the model converges on that identifiable structure on the minimap, which sits at the region centre. No radius/region changes; setup still identical to medium. Suite 351 passed; catalog regenerated.36bbbc9 -
prompt_v2: sanitize confusing 'Buildings: ? ()' briefing line
Unit-only scenarios have no agent base, so the vendored briefing_v2 empty-own-buildings fallback emitted 'Buildings: ? ()'. Post-process the vendored output in our briefing() wrapper (vendored module stays byte-identical; drift test green) to state plainly that the agent commands mobile units only and enemy buildings appear under Enemies once scouted. Adds a guard test.5a3c04b
