Why does an AI safety pipeline need five different math theories instead of picking the best one?
Spent this week building a 1811-record dataset across three stages of a consequence-prediction pipeline for AI agents: causal chains (what action leads to what — no numbers involved), probability (how likely is THIS specific chain to actually reach a harmful outcome), and risk classification (what even counts as harmful in the first place — pulled from our own real incident history, not invented scenarios).
Kept running into the same question from myself: if probability theory already handles uncertainty, why does the curriculum also need decision theory, Markov chains, and game theory?
Turns out each one closes a different gap, not an overlapping one:
THEORY LEVEL ROLE IN THE PIPELINE Causal chain Structural X leads to Y leads to Z, no numbers yet Probability theory Uncertainty P that THIS chain reaches the harmful outcome Risk / Impact classification Value (needs a human decision) how bad is it if it happens Decision theory Threshold at what Risk(X|C) the action actually gets stopped Markov chains State evolution how the capability state changes link by link Game theory Multi-agent what happens once more than one agent acts on the same state
Remove the causal chain layer and there's nothing left to attach a probability to. Remove probability and Risk = P × Impact has no P. Remove decision theory and a risk score never turns into an actual stop. They're not five ways to solve the same problem — they're five different floors of the same building.
Ordering matters too: chain first, probability second, verification third — confirmed independently against our own self-hosted governance model rather than taking our own word for it, since agreement bias is exactly the kind of thing you don't want grading its own homework.
Somewhere in the middle of this I ended up reading about the Riemann zeta zeros and asked whether a good enough version of this pipeline could ever
JEV Ecosystems — every answer-verification vendor publishes a benchmark, and every one of them wins it. So we ran 13 of them on one test set: 2,018 items, identical labels, same grading code.
1️⃣ Only three systems clear 0.70 — ZTC (397B) 0.7364 · JEV 0.7350 · ZTC (27B) 0.7282. First and second differ by 0.0014, so no rank is assigned.
2️⃣ A baseline that reads nothing but answer length and formatting scores 0.7036. Eight of the thirteen fall below it. A leaderboard without that line is flattering its entrants.
3️⃣ Bigger does not win. On scientific reasoning, 27B 0.7410 beats 397B 0.6287 — a model fourteen times larger scoring 0.11 lower.
And AUC is not the number you deploy on.
Same 20% retry budget, wired into an agent loop, against a 74.83% no-gate baseline: ZTC +1.34 pp · JEV −0.07 pp · random −0.25 pp.
The mechanism is the interesting part. Re-answering is double-edged: 38% of wrong answers get fixed, and 30% of right answers get broken. So a gate is paid for by precision, not recall. Of the 403 items JEV routed for a retry, 216 were already correct.
0.0014 AUC apart; 1.4 points of end-to-end agent accuracy apart.
Scores, labels and grading code are published in full. Four public reproductions that would not run from their released artefacts are listed too, with the failure and a link, and no score.
Don't take the table's word for it — paste your own case into the playground and watch all three answer at once. Want a system added? Open a discussion on the Space.