diff --git "a/index.html" "b/index.html" --- "a/index.html" +++ "b/index.html" @@ -3,240 +3,380 @@ Multicultural Riddles Benchmark + + +
-
-
Human evaluation Β· provisional
-

🧩 Multicultural Riddles Benchmark

-
-
-
-
-
-
Heuristic floor β†’ human ceiling
-
- - -
-
-
Grey = automatic heuristic judge (lower bound). Blue = human judgement (upper bound). The gap is the lift from strict string-matching to human scoring.
-
- Download leaderboard (CSV) -
+

Multicultural Riddles Benchmark

+
Native speakers write original riddles that only make sense if you truly know the culture β€” +then humans score how well models answer them. This is the human-evaluation view.
-
-
How models fail (human error types)
-
-
-
+
+
-
-
How hard is each culture?
-
-
+
+ Collection round +
+
-
-
Does the auto-judge agree with humans?
-
-
-
-
+
-
-
Human-eval coverage by community
-
-
-
+
+
+
+
+
+
+
+
-
-
Model performance by region
-
-
+
-
-
Calibration: accuracy vs hallucination
-
-
-
+ \ No newline at end of file + +document.getElementById("foot").innerHTML = + "Methodology: each model response is scored once β€” by a human rater (0–3) or by the reference auto-judge "+ + "for trivially-resolvable cases; pending responses are excluded from accuracy. Mean score is 0–3. "+ + "The two per-community wildcard topics are pooled. Cuba riddle spanish_cuba_history_016 excluded "+ + "(corrected in source). All numbers reconcile to Henry Gagnier's human-eval stats file. Provisional until coverage completes."; + +setVerNote(); renderAll(); + +