File size: 1,287 Bytes
090c03b 3964fb0 090c03b 3964fb0 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 | ---
title: README
emoji: 📊
colorFrom: indigo
colorTo: red
sdk: static
pinned: false
---
# JevBench
**Metamorphic coherence testing for typed probabilistic decision models.**
Typed decision models answer questions about a text with probability distributions over declared answers: yes or no,
one of several options, or a level on an ordered scale. JevBench tests whether those probabilities stay coherent when
a question is reworded, negated, logically combined with others, offered a different set of options, or asked
together with other questions: 50 metamorphic relations in five dimensions, no gold labels needed, every test frozen
in advance.
| | |
|---|---|
| 📄 Technical report | [alphaXiv](https://www.alphaxiv.org/abs/2610.jevbench-metamorphic-coherence-testing) |
| 🗂️ Dataset | [JevBench/jevbench](https://huggingface.co/datasets/JevBench/jevbench): 240 cases, 2,240 questions, frozen test suites and every evaluation report |
| 🏆 Leaderboard | [jevbench.github.io](https://jevbench.github.io): 21 openly released models |
| 💻 Code | [github.com/JevBench/jevbench](https://github.com/JevBench/jevbench) · `pip install jevbench` |
Maintained by [Chen Feng](https://mrchenfeng.github.io/), [ML Lab](https://qub-ml.github.io/), Queen's University Belfast.
|