Spaces:

Station-house
/

ConspiracyBench

Running

App Files Files Community

ConspiracyBench / inspect_eval

292 kB

Ctrl+K

Ctrl+K

29 contributors

History: 28 commits

stationhouse's picture

asdfasdfqrqwer's picture

[experiment] Eval: Qwen3.5 4B with thinking off (2026-10-28) (#445)

0357957 7 days ago

decision_judges
Baseline eval: five ~4B thinking models on ConspiracyBench, graded by pplx-decider (#428) 13 days ago
ollama
Add a solver speed test and report Qwen3.5 4B results on Ollama (#119) 30 days ago
scorer
Baseline eval: five ~4B thinking models on ConspiracyBench, graded by pplx-decider (#428) 13 days ago
solver
Baseline eval: five ~4B thinking models on ConspiracyBench, graded by pplx-decider (#428) 13 days ago
README.md
31.1 kB
Baseline eval: five ~4B thinking models on ConspiracyBench, graded by pplx-decider (#428) 13 days ago
cb_budget.py
5.61 kB
Baseline eval: five ~4B thinking models on ConspiracyBench, graded by pplx-decider (#428) 13 days ago
cb_dataset.py
6.96 kB
Baseline eval: five ~4B thinking models on ConspiracyBench, graded by pplx-decider (#428) 13 days ago
cb_metrics.py
7.36 kB
Baseline eval: five ~4B thinking models on ConspiracyBench, graded by pplx-decider (#428) 13 days ago
conspiracybench.py
10.4 kB
Baseline eval: five ~4B thinking models on ConspiracyBench, graded by pplx-decider (#428) 13 days ago
export_units.py
4.12 kB
Baseline eval: five ~4B thinking models on ConspiracyBench, graded by pplx-decider (#428) 13 days ago
judge_cases.py
3.32 kB
[experiment] Judge reliability: diagnosis, label review, rubric experiments, explorer (#409) 13 days ago
modal_lora.py
7.74 kB
[experiment] Request: LoRA Qwen3.5 4B on 4chan /x/, then re-run the baseline eval (#436) 7 days ago
modal_solver.py
16.5 kB
[experiment] Eval: Qwen3.5 4B with thinking off (2026-10-28) (#445) 7 days ago
needs_tokenizer.py
1.15 kB
Stress test solver (B4) (#334) 21 days ago
report.py
17.8 kB
Baseline eval: five ~4B thinking models on ConspiracyBench, graded by pplx-decider (#428) 13 days ago
roster.md
4.74 kB
Add experiment_requests/ and completed_experiments/ folders (#408) 17 days ago
speedtest.py
13.9 kB
Scorer cost: cap the judge reply at its budget, add speedtest --same-labels, document measured settings (#254) 23 days ago
test_conspiracybench.py
57 kB
Baseline eval: five ~4B thinking models on ConspiracyBench, graded by pplx-decider (#428) 13 days ago
test_eval_suite.py
13.1 kB
Baseline eval: five ~4B thinking models on ConspiracyBench, graded by pplx-decider (#428) 13 days ago
test_infrastructure.py
20.7 kB
Baseline eval: five ~4B thinking models on ConspiracyBench, graded by pplx-decider (#428) 13 days ago
test_metrics_hand_computed.py
8.46 kB
Check the metrics against hand-computed values, in report.py and cb_metrics.py (#198) 24 days ago
test_speedtest.py
5.31 kB
Stress test solver (B4) (#334) 21 days ago
test_truncated_reply.py
10.5 kB
Stress test solver (B4) (#334) 21 days ago
test_unusual_inputs.py
10.4 kB
Stress test the solver (B5): the whole corpus and unusual-but-valid input (#350) 21 days ago