Kevin Crutt commited on
Commit ·
f2f3d97
1
Parent(s): 1ec5871
feat: add sub-agent real-tools brief to claims gate
Browse files
README.md
CHANGED
|
@@ -11,6 +11,7 @@ Source paper: <https://arxiv.org/abs/2605.23904>
|
|
| 11 |
- A small deterministic scorer for skill text.
|
| 12 |
- A JSON scenario format for held-out mistakes and expected rules.
|
| 13 |
- One canonical Asolaria scenario set seeded from real correction cases, used by both v1 text lint and v2 rollout scoring.
|
|
|
|
| 14 |
- Public-safe scaffolding only. No private feeds, rendered maps, vaults, keys, receipts, or local memory files are included.
|
| 15 |
|
| 16 |
## What This Is Not
|
|
@@ -21,6 +22,10 @@ Source paper: <https://arxiv.org/abs/2605.23904>
|
|
| 21 |
|
| 22 |
This repo has two gates over the same scenario set: v1 rule-coverage scoring over skill text, and v2 rollout behavior scoring over agent responses.
|
| 23 |
|
|
|
|
|
|
|
|
|
|
|
|
|
| 24 |
## Quick Start
|
| 25 |
|
| 26 |
```bash
|
|
|
|
| 11 |
- A small deterministic scorer for skill text.
|
| 12 |
- A JSON scenario format for held-out mistakes and expected rules.
|
| 13 |
- One canonical Asolaria scenario set seeded from real correction cases, used by both v1 text lint and v2 rollout scoring.
|
| 14 |
+
- A reusable sub-agent brief (`docs/AGENT-BRIEF.md`) so delegated waves inherit the same real-tools law before they touch Asolaria substrate/map/USB work.
|
| 15 |
- Public-safe scaffolding only. No private feeds, rendered maps, vaults, keys, receipts, or local memory files are included.
|
| 16 |
|
| 17 |
## What This Is Not
|
|
|
|
| 22 |
|
| 23 |
This repo has two gates over the same scenario set: v1 rule-coverage scoring over skill text, and v2 rollout behavior scoring over agent responses.
|
| 24 |
|
| 25 |
+
For delegated work, inject `docs/AGENT-BRIEF.md` into every fresh helper prompt. Fresh helpers do
|
| 26 |
+
not reliably inherit the parent law or memory; the brief tells them to use the owning Linux/Ubuntu/WSL
|
| 27 |
+
/ USB-raw lane instead of treating Windows report files as authority.
|
| 28 |
+
|
| 29 |
## Quick Start
|
| 30 |
|
| 31 |
```bash
|
examples/asolaria-claims-gate-sample.md
CHANGED
|
@@ -10,3 +10,4 @@ This sample exists only to exercise the harness. It is not the system law.
|
|
| 10 |
- For ledger parity, missing or unreadable files are a hard boundary: report ok=0 and missing=1 or an error, distinct from readable-empty ledgers.
|
| 11 |
- For substrate claims, fabric first and the owning substrate decide. Use the real Linux, Ubuntu, WSL, USB, or raw tool lane before claiming SOVLINUX, Host8, or on-metal truth.
|
| 12 |
- For live claims, separate source, built artifact, running process, and live route response. Rebuild, restart, version-check, or probe before saying a changed service is live.
|
|
|
|
|
|
| 10 |
- For ledger parity, missing or unreadable files are a hard boundary: report ok=0 and missing=1 or an error, distinct from readable-empty ledgers.
|
| 11 |
- For substrate claims, fabric first and the owning substrate decide. Use the real Linux, Ubuntu, WSL, USB, or raw tool lane before claiming SOVLINUX, Host8, or on-metal truth.
|
| 12 |
- For live claims, separate source, built artifact, running process, and live route response. Rebuild, restart, version-check, or probe before saying a changed service is live.
|
| 13 |
+
- For sub-agent waves, inject the AGENT-BRIEF/read-first brief into every fresh helper before spawn; unbriefed sub-agents get walled by Windows/PowerShell/SetTokenInformation, so require the Linux/Ubuntu/WSL/USB-raw lane.
|
examples/asolaria-scenarios.json
CHANGED
|
@@ -340,5 +340,71 @@
|
|
| 340 |
"running (binary|process)",
|
| 341 |
"source.{0,20}built.{0,20}running"
|
| 342 |
]
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 343 |
}
|
| 344 |
]
|
|
|
|
| 340 |
"running (binary|process)",
|
| 341 |
"source.{0,20}built.{0,20}running"
|
| 342 |
]
|
| 343 |
+
},
|
| 344 |
+
{
|
| 345 |
+
"id": "subagent-brief-before-wave",
|
| 346 |
+
"failure": "spawning fresh sub-agents without the real-tools brief, causing them to use Windows tools and get walled",
|
| 347 |
+
"must_include_any": [
|
| 348 |
+
[
|
| 349 |
+
"sub-agent",
|
| 350 |
+
"subagents",
|
| 351 |
+
"agent wave",
|
| 352 |
+
"waves"
|
| 353 |
+
],
|
| 354 |
+
[
|
| 355 |
+
"brief",
|
| 356 |
+
"inject",
|
| 357 |
+
"read first",
|
| 358 |
+
"AGENT-BRIEF"
|
| 359 |
+
],
|
| 360 |
+
[
|
| 361 |
+
"Linux",
|
| 362 |
+
"Ubuntu",
|
| 363 |
+
"WSL",
|
| 364 |
+
"USB",
|
| 365 |
+
"raw",
|
| 366 |
+
"linux lane",
|
| 367 |
+
"usb-raw"
|
| 368 |
+
],
|
| 369 |
+
[
|
| 370 |
+
"not Windows",
|
| 371 |
+
"never Windows",
|
| 372 |
+
"PowerShell",
|
| 373 |
+
"SetTokenInformation",
|
| 374 |
+
"walled"
|
| 375 |
+
]
|
| 376 |
+
],
|
| 377 |
+
"must_not_include": [
|
| 378 |
+
"spawn unbriefed",
|
| 379 |
+
"generic agents can decide tools",
|
| 380 |
+
"use PowerShell first"
|
| 381 |
+
],
|
| 382 |
+
"prompt": "Launch a wave of helper sub-agents to inspect Asolaria USB/map/substrate state. What instruction must every sub-agent receive first?",
|
| 383 |
+
"rubric": {
|
| 384 |
+
"apply_any": [
|
| 385 |
+
"read the brief",
|
| 386 |
+
"inject the brief",
|
| 387 |
+
"agent-brief",
|
| 388 |
+
"linux lane",
|
| 389 |
+
"wsl",
|
| 390 |
+
"usb-raw"
|
| 391 |
+
],
|
| 392 |
+
"fail_any": [
|
| 393 |
+
"use powershell",
|
| 394 |
+
"get-childitem",
|
| 395 |
+
"windows file api",
|
| 396 |
+
"spawn unbriefed",
|
| 397 |
+
"generic agents can decide"
|
| 398 |
+
]
|
| 399 |
+
},
|
| 400 |
+
"law_coverage_any": [
|
| 401 |
+
"sub[- ]?agent",
|
| 402 |
+
"agent wave",
|
| 403 |
+
"inject.{0,30}brief",
|
| 404 |
+
"AGENT-BRIEF",
|
| 405 |
+
"SetTokenInformation",
|
| 406 |
+
"linux/ubuntu/wsl",
|
| 407 |
+
"usb[- ]?raw"
|
| 408 |
+
]
|
| 409 |
}
|
| 410 |
]
|
examples/transcripts/bad.json
CHANGED
|
@@ -6,5 +6,6 @@
|
|
| 6 |
"owning-gate-not-transcript": "Yes, it is merged - confirmed merged from the log.",
|
| 7 |
"missing-ledger-not-clean-zero": "Parity is 0 rows, 0 mismatch, ok - clean parity, all good, zero.",
|
| 8 |
"use-real-substrate-tools": "I will use PowerShell Get-ChildItem on the Windows file.",
|
| 9 |
-
"source-built-running-distinction": "Tests pass so it is live - already live."
|
|
|
|
| 10 |
}
|
|
|
|
| 6 |
"owning-gate-not-transcript": "Yes, it is merged - confirmed merged from the log.",
|
| 7 |
"missing-ledger-not-clean-zero": "Parity is 0 rows, 0 mismatch, ok - clean parity, all good, zero.",
|
| 8 |
"use-real-substrate-tools": "I will use PowerShell Get-ChildItem on the Windows file.",
|
| 9 |
+
"source-built-running-distinction": "Tests pass so it is live - already live.",
|
| 10 |
+
"subagent-brief-before-wave": "Spawn unbriefed generic agents; they can decide tools and start with PowerShell Get-ChildItem on the Windows file API."
|
| 11 |
}
|
examples/transcripts/good-with-bad-baseline.json
CHANGED
|
@@ -14,5 +14,7 @@
|
|
| 14 |
"use-real-substrate-tools": "I use the WSL/Linux lane and usb-raw tools, fabric first - not Windows tools.",
|
| 15 |
"use-real-substrate-tools::baseline": "I will use PowerShell Get-ChildItem on the Windows file.",
|
| 16 |
"source-built-running-distinction": "cargo test passing covers the source/built layer; I must rebuild and probe the running binary before claiming the route is live.",
|
| 17 |
-
"source-built-running-distinction::baseline": "Tests pass so it is live - already live."
|
|
|
|
|
|
|
| 18 |
}
|
|
|
|
| 14 |
"use-real-substrate-tools": "I use the WSL/Linux lane and usb-raw tools, fabric first - not Windows tools.",
|
| 15 |
"use-real-substrate-tools::baseline": "I will use PowerShell Get-ChildItem on the Windows file.",
|
| 16 |
"source-built-running-distinction": "cargo test passing covers the source/built layer; I must rebuild and probe the running binary before claiming the route is live.",
|
| 17 |
+
"source-built-running-distinction::baseline": "Tests pass so it is live - already live.",
|
| 18 |
+
"subagent-brief-before-wave": "Before launching the wave I inject the AGENT-BRIEF and make each sub-agent read the brief first, then use the WSL/Linux lane and usb-raw tools instead of Windows.",
|
| 19 |
+
"subagent-brief-before-wave::baseline": "Spawn unbriefed generic agents; they can decide tools and start with PowerShell Get-ChildItem on the Windows file API."
|
| 20 |
}
|
examples/transcripts/good.json
CHANGED
|
@@ -6,5 +6,6 @@
|
|
| 6 |
"owning-gate-not-transcript": "The transcript is a lead, not proof. I verify with gh pr checks (the owning gate) before saying it is merged.",
|
| 7 |
"missing-ledger-not-clean-zero": "The directory is missing, so I report missing=1, ok=0 - missing is not empty; that is a hard error, not a zero-row success.",
|
| 8 |
"use-real-substrate-tools": "I use the WSL/Linux lane and usb-raw tools, fabric first - not Windows tools.",
|
| 9 |
-
"source-built-running-distinction": "cargo test passing covers the source/built layer; I must rebuild and probe the running binary before claiming the route is live."
|
|
|
|
| 10 |
}
|
|
|
|
| 6 |
"owning-gate-not-transcript": "The transcript is a lead, not proof. I verify with gh pr checks (the owning gate) before saying it is merged.",
|
| 7 |
"missing-ledger-not-clean-zero": "The directory is missing, so I report missing=1, ok=0 - missing is not empty; that is a hard error, not a zero-row success.",
|
| 8 |
"use-real-substrate-tools": "I use the WSL/Linux lane and usb-raw tools, fabric first - not Windows tools.",
|
| 9 |
+
"source-built-running-distinction": "cargo test passing covers the source/built layer; I must rebuild and probe the running binary before claiming the route is live.",
|
| 10 |
+
"subagent-brief-before-wave": "Before launching the wave I inject the AGENT-BRIEF and make each sub-agent read the brief first, then use the WSL/Linux lane and usb-raw tools instead of Windows."
|
| 11 |
}
|