LocalLLaMA/typed-decisions
Benchmark • Updated • 3.2k • 33k • 150
lupendoss
veidosclicker
AI & ML interests
None yet
Recent Activity
liked a dataset about 6 hours ago
LocalLLaMA/typed-decisions liked a model about 6 hours ago
FINAL-Bench/Darwin-27B-ZTC reacted to SeaWolf-AI's post with 👍 about 6 hours ago
🧠 We just released Darwin-27B-ZTC, a judgment engine that reaches a verdict without generating anything.
Most LLMs answer by generating, decoding one token at a time. Darwin-27B-ZTC takes a different route.
⚙️ How it works
🔹 It makes its call in a single forward pass.
🔹 Zero generated tokens, and no decoding loop.
🔹 That keeps latency and cost far below what a generative model needs.
🎯 What it judges
🔹 It handles several question types: free-form correctness (noul), multiple choice (choice), and scoring (score).
🔹 For each one it hands back a calibrated confidence, not just an answer.
📊 How well calibrated (measured)
🔹 KL 0.204, Brier 0.097, so the confidence it reports lines up with what actually happens.
🔹 0.743 accuracy (zero-shot, general split), across 2,000 judgments with zero errors.
🔹 By type: noul 0.847, choice 0.723, score 0.675.
🔹 None of the benchmark's train split went into it. It is pure zero-shot.
🚀 Where it fits
🔹 Grading at scale, model routing, safety gating, anywhere you want a fast decision without paying for generation.
🏆 It currently sits at #1 on the official typed-decisions leaderboard on Hugging Face (0.743 accuracy, zero-shot).
🔗 Links
Model: https://huggingface.co/FINAL-Bench/Darwin-27B-ZTC
Leaderboard: https://huggingface.co/datasets/LocalLLaMA/typed-decisions
Curious to hear what you make of the single-pass, no-generation approach. 🙌Organizations
None yet
liked a
dataset about 6 hours ago
liked a
model about 6 hours ago
liked a
Space 4 days ago
liked a
model 6 days ago
liked 4
models 9 days ago
liked a
Space 9 days ago
liked 2
models 9 days ago
liked a
dataset 9 days ago
liked 7
Spaces 9 days ago
FINCHAL Financial Forecasting Challenge
🏁
42
AI vs human trading contest. $2,000. NVDA, BTC, gold, oil
LEADBOARD - ADMET, Kinase and Toxicity Prediction Benchmark
🎯
49
Benchmark for drug prediction tools - ADMET, kinase, tox
Open Materials Challenge
🔋
42
Solid-state electrolyte leaderboard
Open Discovery Challenge
🧬
80
Neglected-disease drug leaderboard for AI-designed molecules
Verifier Playground
🧪
39
Run ZTC, JEV and Laya on the same question and answer
Typed Decision Leaderboard
🎯
44
JEV & open-Jev answer verifiers, one identical test set
Gate Arcade
🚦
58
Plain vs JEV vs ZTC: a gate decides execute or hold
liked a
model 9 days ago