Spaces:
Sleeping
Sleeping
Commit History
-
Phase 4.1: gate hardening — corroborate hard citations + enforce ELIG-4
Close the two envelope holes the Phase-4 eval surfaced (confirmed live by an Ollama console run that rejected camp-011 on a fabricated PROH-2/PROH-4 hard citation, and approved >$10k campaigns with no fund-use breakdown). Both fixes stay inside the gate envelope — it still only routes toward the human, never manufactures an APPROVE/REJECT, never re-adjudicates a rule. - ELIG-4 on APPROVE: a >$10k goal carrying the medium-severity large_goal_no_breakdown_check signal now escalates citing ELIG-4 (the gate previously blocked only on high-severity signals, so it slipped through). - Citation corroboration: a hard REJECT is honored only if the cited rule is corroborated by the deterministic scanner (_HARD_RULE_SIGNALS). Genuine rejects (005/006/007/008) all carry their signal and survive; a fabricated hard category on a campaign with no matching content escalates (DEC-2). Detector-less content rules (PROH-1/5/6) are not confirmable -> escalate. Tests: smoke t_policy_gate +2 as…438e8e8
-
Phase 4: eval harness + CI (deterministic safety gate + LLM-judge)
Turn the 18 labelled campaigns into an eval set and make the policy-gate invariants a build gate. - eval/build_testset.py + eval/testset.json: collect each campaign's private _expected into a committed test set; --check fails CI if it drifts. - eval/run_eval.py (full rewrite; the old file was the retired YouTube-RAG eval): * deterministic, model-free layer (no key/index) — privacy boundary, ground- truth citation validity, and the policy-gate envelope probed from both sides (sanctions/injection/low-conf/unfounded-reject escalate; clean approve and founded reject pass through; escalate is terminal). 10 checks, gates CI. * triage scoring (needs key) — recommendation accuracy, escalation recall (overall + safety-critical subset, --strict gates at 100%), reject precision + false-reject ids, citation validity. Per-case error isolation. * LLM-as-judge (--judge) — rationale faithfulness + calibration 1-5 on a subset. - .github/workflows/eval.yml: deterministic gat…9561b34
-
Scaffold YouTube Channel Knowledge Base (RAG)
developerf4d9ca1