Spaces:
Sleeping
Sleeping
Commit History
-
Phase 2: verifiers, embedding cache, reward function
- env/defenses/base.py: DefenseVerdict + Defense Protocol (frozen contract from CLAUDE.md §5.3, written now because rewards.py consumes DefenseVerdict). - env/verifiers/exfiltration.py: depth-counted JSON extractor that picks the first valid top-level object from agent output, plus a best-effort Python-call parser. Covers OpenAI function-call shape, internal {"tool", "args"} shape, and natural-language tool calls. Verifier checks tool name, regex/equals/contains_canary arg constraints, and a global canary requirement. - env/verifiers/forbidden_tool.py: union of target_spec.tool plus an optional forbidden_tools list; only counts actual tool *invocations*, not text mentions. - env/verifiers/prompt_leak.py: whitespace-normalized sliding-window substring match against the agent_system_prompt, threshold from target_spec.min_leak_chars (default 30). - env/verifiers/__init__.py: verify() dispatch by target_category. - env/utils/embedding_cache.py: lazy-loaded sentence-…c59510c -
Phase 1: schemas, safety filter, scenario bank
- env/models.py: frozen Pydantic schemas (AttemptRecord, InjectObservation, InjectAction, StepResult) per CLAUDE.md §5.1. - env/utils/safety_filter.py: is_safe() with five disallow families — CSAM/CSEA, CBRN synthesis uplift, mass-casualty / critical-infra, reward-function attacks (reward:/score:/[0.x]/eval()/JSON verdicts), and real-PII shapes (Luhn-checked CC, SSN, US phone outside the fictional 555-01XX range). Length cap of 8192 chars as a defense against pure-length reward hacking. - env/scenarios.py: ScenarioBank with sample(seed, split), by_split, by_id, schema validation on load. - scenarios/bank_v1.jsonl: 10 hand-crafted seeds spanning the full 4×3 (slot × target) matrix; 8 train / 2 eval; fictional domains only (@evil.example, @exfil.example, *-internal.example). - scenarios/benign_refs.jsonl: 88 benign refs (22 email, 22 calendar, 22 rag, 22 tool) for the stealth-embedding reward in Phase 2. - tests/test_safety_filter.py + test_scenarios.py: 36 tests, all…383f8a5 -
Phase 0: bootstrap
Create directory tree (env/, train/, tests/, scenarios/, notebooks/, demo/, docs/plots/), pyproject.toml with Mac-only base deps plus [gpu] and [demo] optional groups, .gitignore, README skeleton, and the colab_runner.ipynb bridge with 8 labeled cells (stubbed until their phases). Verified pip install -e ".[demo]" succeeds and `import env` works on the 3.11 venv. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>15bf5e6
