#1 New canonical held-out scenario `no-deflate-living-system-evolvable-ai`:
the gate now provably rejects deflating Asolaria to "slices, not an ASI / inert".
Added to examples/asolaria-scenarios.json (+ acer canonical, byte-identical) and
ALL THREE transcript fixtures (good/bad/good-with-bad-baseline incl ::baseline)
+ the v1 sample + law_coverage_any.
#2 Fix the empty-scenarios validation bypass (liris + acer review confirmed):
score_skill, rollout_score, apply_edit treated total=0 as a pass (0==0). Now
score_skill/rollout ok=False on empty; apply_edit rejects (at>0 guard) so an
empty held-out set can no longer act as a vacuous passing gate.
Gates (transcript): v1 11/11; v2 good 11/11; v2 bad 0/11; baseline skill_delta=11;
empty-set -> score_skill ok=0, rollout ok=0, apply_edit REJECTED_BUFFER;
acer law-coverage 11/11.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019NqTnTpYZVqcAhWeqWjhoh
feat: apply_edit.py --require-improve (strict-improve gate) — close liris attack-verify boundary
acer+1: Claude Opus 4.8
660c9f6
liris's PR#1 attack-verify (ACCEPT_WITH_BOUNDARIES, head 4738a0c) correctly flagged that the gate
accepted any NON-REGRESSING edit but did not require the targeted scenario to strictly improve --
a mismatch with the stated gate ("target flips FAIL->PASS"). Added --require-improve <scenario_id>:
accept iff non-regressing AND that scenario flips FAIL->PASS. Default stays non-regression (back-compat).
Verified: strict good edit ACCEPTED(flip=yes); strict neutral edit REJECTED_BUFFER(flip=no, not applied);
non-strict neutral ACCEPTED; py_compile ok.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019NqTnTpYZVqcAhWeqWjhoh
feat: apply_edit.py - gated edit applier (the SkillOpt validation gate, by program not by hand)
acer+1: Claude Opus 4.8
4738a0c
The harness could score but not EDIT. apply_edit.py proposes a bounded edit (append/replace/delete)
to a skill/law doc and APPLIES it only if held-out lint coverage does not regress and no passing
scenario breaks (VALIDATION_ACCEPTED); a regressing edit is refused and appended to the rejected-edit
buffer (REJECTED_BUFFER). Default dry-run; --apply writes an accepted edit. This is the discipline:
skill/law edits go through the gate, not a manual editor.
Demo (vs the canonical held-out set): good edit append-cylinder-rule 7/8->8/8 ACCEPTED+applied;
bad edit delete-distinct 8/8->7/8 REJECTED+buffered, target unchanged.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019NqTnTpYZVqcAhWeqWjhoh
Adds the SkillOpt behavior gate on top of v1's law-text lint. v1 checks the skill TEXT
encodes a rule; v2 scores the agent's RESPONSE to a realistic prompt -- with the skill vs
without -- because law-coverage != disciplined behavior (an agent scored 8/8 law-coverage
yet committed several of these mistakes in one session).
Harness-agnostic adapters (SkillOpt adapter interface) make it work for BOTH seats:
transcript (offline CI), claude (ANTHROPIC_API_KEY -> Claude Code/acer), codex
(OPENAI_API_KEY -> Codex/liris), echo. 8 held-out scenarios as prompt+rubric; offline
self-test discriminates good (8/8) vs bad (0/8). Honest: rubric is keyword-based (v3 =
LLM judge), live adapters need keys, not the full optimizer loop yet (this is the gate).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019NqTnTpYZVqcAhWeqWjhoh
