Loading…
How to review
- Task: is the instruction clear, solvable on the site and gradeable as written? Needs fix for a wording, tag or answer problem you can name; Broken when it cannot be done or graded as intended. Say what in the notes.
- Step through the demonstration; each primitive's span is colored in the steps and the action log, and its "steps a–b" link jumps there. Note a recording that does not do what its primitive or the task says.
- Review changes: compare the original definition, verifier and recording with the current ones. Accept when the changes fix or keep the task without changing what it asks; Reject when they introduce a mistake.
- Test tasks (held-out sites) come first: they are the benchmark's evaluation set.