sunyiyou commited on
Commit
4e53bee
·
verified ·
1 Parent(s): cf457fa

Add official Agents Last Exam evaluation results

Browse files

Adds ALE results from the [official leaderboard](https://agents-last-exam.org/leaderboard), captured on 2026-10-01.

- Covers Overall, ALE-CLI, Near-term, Full-spectrum, and Last-exam, with pass rate and score on a 0-100 scale.
- Configuration: GLM-5.2, Claude Code; effort=max. Task coverage is recorded in `notes`.
- Overall: 20.39% pass rate / 41.09% score.

These are existing official leaderboard aggregates, not newly rerun or HF-verified evaluations. The full benchmark contains 152 tasks, including unattempted tasks in the denominator.

.eval_results/ale-claude-code-max.yaml ADDED
@@ -0,0 +1,120 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ - dataset:
2
+ id: agents-last-exam/agents-last-exam
3
+ task_id: overall-pass-rate
4
+ value: 20.394736842105264
5
+ source:
6
+ url: https://agents-last-exam.org/leaderboard
7
+ name: ALE official leaderboard
8
+ user: agents-last-exam
9
+ notes: harness=claude_code; harness_variant=max; attempted=146/152; official leaderboard
10
+ snapshot=2026-10-01T20:39:17.826877+00:00; values are percentages (0-100); equal-task
11
+ aggregation; unattempted tasks contribute zero; pass threshold score>=0.999. This
12
+ is an official leaderboard export, not an HF-verified evaluation.
13
+ - dataset:
14
+ id: agents-last-exam/agents-last-exam
15
+ task_id: overall-score
16
+ value: 41.08540680857251
17
+ source:
18
+ url: https://agents-last-exam.org/leaderboard
19
+ name: ALE official leaderboard
20
+ user: agents-last-exam
21
+ notes: harness=claude_code; harness_variant=max; attempted=146/152; official leaderboard
22
+ snapshot=2026-10-01T20:39:17.826877+00:00; values are percentages (0-100); equal-task
23
+ aggregation; unattempted tasks contribute zero; pass threshold score>=0.999. This
24
+ is an official leaderboard export, not an HF-verified evaluation.
25
+ - dataset:
26
+ id: agents-last-exam/agents-last-exam
27
+ task_id: ale-cli-pass-rate
28
+ value: 23.809523809523807
29
+ source:
30
+ url: https://agents-last-exam.org/leaderboard
31
+ name: ALE official leaderboard
32
+ user: agents-last-exam
33
+ notes: harness=claude_code; harness_variant=max; attempted=104/105; official leaderboard
34
+ snapshot=2026-10-01T20:39:17.826877+00:00; values are percentages (0-100); equal-task
35
+ aggregation; unattempted tasks contribute zero; pass threshold score>=0.999. This
36
+ is an official leaderboard export, not an HF-verified evaluation.
37
+ - dataset:
38
+ id: agents-last-exam/agents-last-exam
39
+ task_id: ale-cli-score
40
+ value: 44.15980756094993
41
+ source:
42
+ url: https://agents-last-exam.org/leaderboard
43
+ name: ALE official leaderboard
44
+ user: agents-last-exam
45
+ notes: harness=claude_code; harness_variant=max; attempted=104/105; official leaderboard
46
+ snapshot=2026-10-01T20:39:17.826877+00:00; values are percentages (0-100); equal-task
47
+ aggregation; unattempted tasks contribute zero; pass threshold score>=0.999. This
48
+ is an official leaderboard export, not an HF-verified evaluation.
49
+ - dataset:
50
+ id: agents-last-exam/agents-last-exam
51
+ task_id: near-term-pass-rate
52
+ value: 32.83582089552239
53
+ source:
54
+ url: https://agents-last-exam.org/leaderboard
55
+ name: ALE official leaderboard
56
+ user: agents-last-exam
57
+ notes: harness=claude_code; harness_variant=max; attempted=66/67; official leaderboard
58
+ snapshot=2026-10-01T20:39:17.826877+00:00; values are percentages (0-100); equal-task
59
+ aggregation; unattempted tasks contribute zero; pass threshold score>=0.999. This
60
+ is an official leaderboard export, not an HF-verified evaluation.
61
+ - dataset:
62
+ id: agents-last-exam/agents-last-exam
63
+ task_id: near-term-score
64
+ value: 60.33126287817978
65
+ source:
66
+ url: https://agents-last-exam.org/leaderboard
67
+ name: ALE official leaderboard
68
+ user: agents-last-exam
69
+ notes: harness=claude_code; harness_variant=max; attempted=66/67; official leaderboard
70
+ snapshot=2026-10-01T20:39:17.826877+00:00; values are percentages (0-100); equal-task
71
+ aggregation; unattempted tasks contribute zero; pass threshold score>=0.999. This
72
+ is an official leaderboard export, not an HF-verified evaluation.
73
+ - dataset:
74
+ id: agents-last-exam/agents-last-exam
75
+ task_id: full-spectrum-pass-rate
76
+ value: 18.181818181818183
77
+ source:
78
+ url: https://agents-last-exam.org/leaderboard
79
+ name: ALE official leaderboard
80
+ user: agents-last-exam
81
+ notes: harness=claude_code; harness_variant=max; attempted=54/55; official leaderboard
82
+ snapshot=2026-10-01T20:39:17.826877+00:00; values are percentages (0-100); equal-task
83
+ aggregation; unattempted tasks contribute zero; pass threshold score>=0.999. This
84
+ is an official leaderboard export, not an HF-verified evaluation.
85
+ - dataset:
86
+ id: agents-last-exam/agents-last-exam
87
+ task_id: full-spectrum-score
88
+ value: 36.974012794692094
89
+ source:
90
+ url: https://agents-last-exam.org/leaderboard
91
+ name: ALE official leaderboard
92
+ user: agents-last-exam
93
+ notes: harness=claude_code; harness_variant=max; attempted=54/55; official leaderboard
94
+ snapshot=2026-10-01T20:39:17.826877+00:00; values are percentages (0-100); equal-task
95
+ aggregation; unattempted tasks contribute zero; pass threshold score>=0.999. This
96
+ is an official leaderboard export, not an HF-verified evaluation.
97
+ - dataset:
98
+ id: agents-last-exam/agents-last-exam
99
+ task_id: last-exam-pass-rate
100
+ value: 2.631578947368421
101
+ source:
102
+ url: https://agents-last-exam.org/leaderboard
103
+ name: ALE official leaderboard
104
+ user: agents-last-exam
105
+ notes: harness=claude_code; harness_variant=max; attempted=34/38; official leaderboard
106
+ snapshot=2026-10-01T20:39:17.826877+00:00; values are percentages (0-100); equal-task
107
+ aggregation; unattempted tasks contribute zero; pass threshold score>=0.999. This
108
+ is an official leaderboard export, not an HF-verified evaluation.
109
+ - dataset:
110
+ id: agents-last-exam/agents-last-exam
111
+ task_id: last-exam-score
112
+ value: 12.862606139319707
113
+ source:
114
+ url: https://agents-last-exam.org/leaderboard
115
+ name: ALE official leaderboard
116
+ user: agents-last-exam
117
+ notes: harness=claude_code; harness_variant=max; attempted=34/38; official leaderboard
118
+ snapshot=2026-10-01T20:39:17.826877+00:00; values are percentages (0-100); equal-task
119
+ aggregation; unattempted tasks contribute zero; pass threshold score>=0.999. This
120
+ is an official leaderboard export, not an HF-verified evaluation.