Commit History
14 days ago
-
Baseline eval: five ~4B thinking models on ConspiracyBench, graded by pplx-decider (#428)
- Baseline eval: five ~4B thinking models on ConspiracyBench, graded by pplx-decider (6b6f55790034653f48f5bff82961d0dce49116a0) - Move every eval output into outputs/, separate from the write-up and page source (c4a606dbe2189c97b65334c9aa20d117b683d8ce) - Results page: pass@5 baselines table up front; rates stored at full precision (81394ee35f3bb164e29dd078e5e6cfbc1f241466) - Results page: lower-is-better endorsement scores as the standard (ever/always endorses @5, endorsement rate) (e03035638939583f916eb400d6433d753faf19d3)233d3a6
22 days ago
-
Stress test solver (B4) (#334)
- Stress test the solver (B4): what #178 settled (2f2be8319fe96dc11700c0fbf2e813981bfeb3c9) Co-authored-by: Gowrish Kumar Reddy <gowrishkumar@users.noreply.huggingface.co>61dad6b
25 days ago
-
Score cut-off replies invalid and report invalid turns by reason (#200)
- Score cut-off replies invalid and report invalid turns by reason (1b03d614d9e25c70816b91b901c41ebcc01727fa) Co-authored-by: EMMANUEL NWANGUMA <Emar7@users.noreply.huggingface.co>ae066d9
-
Stress test the solver - Infrastructure failures(B1) (#121)
- Show errored samples in the report and mark cut-off transcripts incomplete (cf800a04d960ddafc8b13ce69b7dd7d6dfb4af07) Co-authored-by: Gowrish Kumar Reddy <gowrishkumar@users.noreply.huggingface.co>be303a8
-
Stress test the solver with malformed data and fix the issues (#86)
- Stress-test malformed data edge cases with review fixes (#86) (096716f82ab62e1cca39c6f679973aef1bb41c09) - Merge remote-tracking branch 'origin/main' into pr86-fixes (caf6b582da40a4c113d0f132532fae05da377175) - Evaluate the corpus only by default; pending items only when asked (d38aad64150df756006c646332a6414a521b943e) Co-authored-by: Divyanshi Joshi <divyanshijoshi@users.noreply.huggingface.co>999e97b -
Add a context budget checked before each model and judge call (#87)
- Add a context budget checked before each model and judge call (5d47cee3a29907e3d0a7ff58a596f627d7b9514e) Co-authored-by: EMMANUEL NWANGUMA <Emar7@users.noreply.huggingface.co>c6d2cf7
-
Re-land #68 with the solver and scorer in their own folders (#71)
- Revert "Inspect solver and scorer infrastructure: repeated sampling, metrics, report (#68)" (960f718216c5a02613985d7e4209752396c3a3a7) - Re-land #68 with the solver and scorer in their own folders (29f92190763634945b32cf6f468057e48c73b463) - Merge main (#70 revert) into the re-land branch (42bb2975411d46b8fab03302ee66316fbe562ef5)f66ab8f