Spaces:
Running on CPU Upgrade
Running on CPU Upgrade
Request: repeated base-model evaluations and a zero-update control run
#19
by stoiclion - opened
Hi, thanks for running the Arena. The leaderboard compares every run against a single evaluation of the base model, and a run's score moves by about one task between runs with identical inputs. Could you publish (a) at least 10 repeated evaluations of the unchanged base weights on the private set, and (b) one run through the full pipeline with zero weight updates (or learning rate 0)? With those, participants can tell a real gain from evaluation noise, and the board's zero point stops being a single draw. Happy to help if a script or a config would make it easier.