Spaces:
Running
Running
Commit History
-
feat(judges): Rubric markdown loader with aggressive validation
Rubric loads from markdown with YAML frontmatter; validates scale, arity-matches-scale, anchored-example-per-level, frontmatter required fields. ValidationError raises with file path + field context so malformed rubrics fail at construction (Day 1) not at first judge.score call (Day 2 with API budget spent). source_hash is SHA-256 of body_markdown — immutable per file content, independent of git state. Used as ScoreResult.rubric_version so κ aggregation can group by rubric identity without cross- referencing run metadata. render_prompt(level_permutation_seed=N) deterministically permutes the ## Score sections via seeded PRNG. Seed=0 returns canonical order; this is the variance-control hook used by rubric_permute in Phase 3. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>7b72b2c