> ⚠ **P4B frame:** the **SkillOpt / claims-gate / watcher-correction loop** (validation-gated skill/law edits) — NOT a hot-path JSON fabric. See [`HARNESS-EDIT-ROOT-FRAME.md`](./HARNESS-EDIT-ROOT-FRAME.md) + the system `MAP.md`.

# Harness-edit

SkillOpt-style edit harness for validating changes to agent skill documents.

This repo is the **harness**, not the skill. It implements the missing gate from the SkillOpt paper: a candidate skill/law edit is not accepted just because it sounds right. It must improve a held-out scenario set and avoid regressions.

Source paper: <https://arxiv.org/abs/2605.23904>

## What This Provides

- A small deterministic scorer for skill text.
- A JSON scenario format for held-out mistakes and expected rules.
- One canonical Asolaria scenario set seeded from real correction cases, used by both v1 text lint and v2 rollout scoring.
- A reusable sub-agent brief (`docs/AGENT-BRIEF.md`) so delegated waves inherit the same real-tools law before they touch Asolaria substrate/map/USB work.
- Public-safe scaffolding only. No private feeds, rendered maps, vaults, keys, receipts, or local memory files are included.

## What This Is Not

- Not the full SkillOpt optimizer loop.
- Not a full model-rollout optimizer yet.
- Not a replacement for fabric/canon/GitHub owning gates.

This repo has two gates over the same scenario set: v1 rule-coverage scoring over skill text, and v2 rollout behavior scoring over agent responses.

For delegated work, inject `docs/AGENT-BRIEF.md` into every fresh helper prompt. Fresh helpers do
not reliably inherit the parent law or memory; the brief tells them to use the owning Linux/Ubuntu/WSL
/ USB-raw lane instead of treating Windows report files as authority.

## Quick Start

```bash
python scripts/score_skill.py --skill path/to/SKILL.md --scenarios examples/asolaria-scenarios.json --report out/report.json
```

Exit codes:

- `0`: all scenarios pass.
- `1`: at least one scenario fails.
- `2`: invalid inputs or harness error.

## Scenario Format

Each canonical scenario can serve both gates. For v1 text lint it has:

- `id`: stable identifier.
- `failure`: the mistake the skill should prevent.
- `must_include_any`: at least one phrase from each inner group must appear in the skill text.
- `must_not_include`: phrases that must not appear.

For v2 rollout scoring it also has:

- `prompt`: the realistic situation used for a rollout.
- `rubric.apply_any`: response markers that show the correct behavior.
- `rubric.fail_any`: response markers that show the old mistake.

This is intentionally simple. The scorer is a first gate, not the whole evaluator.

## SkillOpt Mapping

- Trainable artifact: a skill/law Markdown file.
- Held-out set: `examples/asolaria-scenarios.json`.
- Textual learning-rate analogue: bounded candidate diff reviewed separately.
- Validation gate: `score_skill.py` must strictly improve or maintain all held-out cases.
- Rejected buffer: failed reports become future scenarios.