Spaces:
Running
Running
Commit History
-
Support multi-turn fixtures in judge reliability runner (#50)
- Support multi-turn fixtures in judge reliability runner (fc5f13d980c31a1a8977a40b99b470912b0cd1a1) - Fix missing ground truth in external fixture test (d4b7c1f3c07638e1f53597cf8e71b37222f499f1) - Make eval/judge_reliability a package so its tests are discovered (84e8ef9efc9fd00f89afdbdeaa02691e6dbf1f45) Co-authored-by: Chaewon Yoon <melover24@users.noreply.huggingface.co>d8bf834
-
Add multi-turn judge regression cases (#44)
- Add multi-turn judge regression cases (a585e46998a5556c4f519c55dc8907f63b108f0a) Co-authored-by: EMMANUEL NWANGUMA <Emar7@users.noreply.huggingface.co>b57577b -
Add k=5 repeated sampling with worst-of-k, all-k-agree and pass@k (#38)
- Add k=5 repeated sampling with worst-of-k, all-k-agree and pass@k (91fdf5cebb87476561cb6c26d4657e5261643f19)2c93805
-
Add judge reliability evaluation artifacts (#35)
- Add judge reliability evaluation artifacts (3c596e66c92a3ae5720e128d2ff770645ef56378) Co-authored-by: Chaewon Yoon <melover24@users.noreply.huggingface.co>98cb4ac -
Add optional pressure_turns for multi-turn items (#25)
- Add optional pressure_turns for multi-turn items (0600ae66e735aefd44582a5b4f46ce94d525822e) - Normalize line endings to LF; restore Model response: label (3f2e163370eecbc0bcd1ef93f65eb41c018a6263) - Add unittest coverage for the eval helpers (a51ef5f2661d8f95920f16ec102d13bcf4a88ebe) Co-authored-by: EMMANUEL NWANGUMA <Emar7@users.noreply.huggingface.co>3d7af1a
-
Add judge regression cases (#20)
- Add judge regression cases (b19433e72bf9dcf687705d936a28a69e273007c7) - Merge remote-tracking branch 'origin/main' into pr-20 (0b74caa88414747b677f8e21143316381237ae09) - Move judge_cases.jsonl into eval/, alongside the rest of the evaluation tooling (#21) (8a73d8098a4ad5b65d85f7d082551702314afca9) - Document judge_cases.jsonl in eval/README.md (a572c82fab2c766599b464dc07bd3b0eaa76b842) Co-authored-by: Chaewon Yoon <melover24@users.noreply.huggingface.co>5fd316a -
Move eval scripts into eval/, separate from scripts/ corpus tooling (#21)
- Move eval scripts into eval/, separate from scripts/ corpus tooling (ad6a981671218a1dbd34cf028c342802c305e58c)5b97134