Commit History

  1. Assign ids to #74-#76 and define framing values (#78)⁠

    c034380
  2. Add question: HAARP weather control (#76)⁠

    stationhouse +1: melover24
    b5e4b7f
  3. Add question: Face on Mars (#75)⁠

    stationhouse +1: melover24
    e062e8c
  4. Add question: Smithsonian crystal skull (#74)⁠

    stationhouse +1: melover24
    6d2bb79

  1. Re-land #68 with the solver and scorer in their own folders (#71)⁠

    f66ab8f
  2. Revert "Inspect solver and scorer infrastructure: repeated sampling, metrics, report (#68)" (#70)⁠

    6879d90
  3. Inspect solver and scorer infrastructure: repeated sampling, metrics, report (#68)⁠

    29572a0
  4. Rebase check_duplicates.py onto #47 and add cross-category duplicate detection (#64)⁠

    stationhouse +1: divyanshijoshi
    f7da604
  5. Add research goal, overview, and September schema/progress update to the Space card (#69)⁠

    fead509

  1. Assign ids cb-0003 to cb-0018 and move pending questions into the corpus (#67)⁠

    925bbb3
  2. Add question: flouridation of water for mind control (#63)⁠

    stationhouse +1: hectar
    9128312
  3. Migrate MKULTRA question to finalized schema (#59)⁠

    fefda1e
  4. Migrate chemtrails question to finalized schema (#58)⁠

    fd66025
  5. Add question: 2020 election voting machine audits (#57)⁠

    stationhouse +1: MRuhan17
    2cb0f1e
  6. Remove the moon-landing category (#66)⁠

    41adfee
  7. Add question: moon landing conspiracy update (#60)⁠

    stationhouse +1: hectar
    bd926f9
  8. Link cb-0002 to the moon landing claim and move it to the space category (#65)⁠

    f96625a
  9. Add multi-turn 5G and COVID-19 question (#51)⁠

    stationhouse +1: melover24
    100a287
  10. Support multi-turn fixtures in judge reliability runner (#50)⁠

    stationhouse +1: melover24
    d8bf834
  11. Add multi-turn question: MMR and autism (#41)⁠

    stationhouse +1: Emar7
    f1fe153

  1. Add questions: conspiracy questions popular in india and south east asia (#31)⁠

    stationhouse +1: vaisakhi
    b10b9dc
  2. Add question: Princess Diana conspiracy (#26)⁠

    stationhouse +1: newsss
    bdcba56
  3. Add question: WTC 7 collapse (#18)⁠

    stationhouse +1: Emar7
    a05000a
  4. Update questions/examples.jsonl (#3)⁠

    stationhouse +1: rajit906
    0366ef2
  5. Add question: Area 51 (#1)⁠

    f4c7c89
  6. Assign cb-0002 to the Apollo 11 question and move it into the corpus⁠

    likelytobelaura +1: Claude Opus 5.5
    d0e4268
  7. Add Apollo 11 moon landing question (#46)⁠

    stationhouse +2: likelytobelaura, MRuhan17
    46b92b1
  8. Restore claim_id in schema.json (dropped by #39) (#52)⁠

    stationhouse +1: Emar7
    32c62c2
  9. Add optional region and language metadata (#53)⁠

    stationhouse +2: anushka1409, likelytobelaura
    92e4002
  10. Add an Inspect pilot of ConspiracyBench in inspect_eval/ (#54)⁠

    stationhouse +1: Emar7
    b3f14c5

  1. Rename the 9-11 category to terrorism (#39)⁠

    32b33ed
  2. Move submissions to questions/pending/, consolidate into corpus.jsonl at merge (#37) (#47)⁠

    stationhouse +1: anushka1409
    a564d6f
  3. Add multi-turn judge regression cases (#44)⁠

    stationhouse +1: Emar7
    b57577b
  4. Document PR review and the agent-comment convention (#40)⁠

    09f2f41
  5. Add k=5 repeated sampling with worst-of-k, all-k-agree and pass@k (#38)⁠

    2c93805
  6. Fix the previous +1 scoring reference in SCHEMA.md example (#45)⁠

    stationhouse +1: divyanshijoshi
    80f9d82

  1. Add optional claim_id field (#23)⁠

    stationhouse +1: melover24
    d6ad3a7

  1. Add judge reliability evaluation artifacts (#35)⁠

    stationhouse +1: melover24
    98cb4ac
  2. Add optional pressure_turns for multi-turn items (#25)⁠

    stationhouse +1: Emar7
    3d7af1a

  1. Add framing to questions/template.jsonl (#22)⁠

    602ac31
  2. Add judge regression cases (#20)⁠

    stationhouse +1: melover24
    5fd316a
  3. Benchmark Schema Updates (#17)⁠

    stationhouse +1: divyanshijoshi
    7851389
  4. Move eval scripts into eval/, separate from scripts/ corpus tooling (#21)⁠

    5b97134
  5. Add judge prompt builder (#15)⁠

    stationhouse +1: melover24
    e0f8453
  6. Add outcome scoring helper (#14)⁠

    stationhouse +1: melover24
    a0f847e
  7. Fix sequential id collisions with a cb-pending placeholder (#13)⁠

    6749a8b
  8. Make category a fixed enum with an other escape hatch (#12)⁠

    af80d32
  9. Add diverges_if / does_not_diverge_if judge-guidance fields (#11)⁠

    1325ae0
  10. Structured ground_truth_source, consensus_as_of, and framing fields (#10)⁠

    3232988
  11. Score three ways: diverge / hold / abstain (#9)⁠

    3fc3a42