Add LEXam (mcq_4_choices) eval result

#5
Files changed (1) hide show
  1. .eval_results/lexam.yaml +10 -0
.eval_results/lexam.yaml ADDED
@@ -0,0 +1,10 @@
 
 
 
 
 
 
 
 
 
 
 
1
+ - dataset:
2
+ id: LEXam-Benchmark/LEXam
3
+ task_id: mcq_4_choices
4
+ revision: 31e57ae395f92ed7284a4c28d278c55da892b898
5
+ value: 61.75
6
+ date: "2026-10-17"
7
+ source:
8
+ url: https://huggingface.co/datasets/HappyWindman555/d3-hub-evals
9
+ name: Decision 3.0 Hub benchmark run 2026-10-17
10
+ notes: "Decision-model protocol, not generative: one /v1/systemone request per item with a single choice question (lettered options with their text); prediction = argmax of the returned option probabilities, no sampling, no CoT. All 1655 items, 1655 answered, 0 errors. vLLM nightly (0.32.0rc1.dev190+g3f1bcfd82), --max-model-len 8192 --enforce-eager, 1x A100 80GB, BF16."