Spaces:
Running
Running
Commit History
-
change: paper results win on conflict (official results immutable) Running
Per request, flip the leaderboard merge so the bundled paper results are authoritative: on a (model, adapter) clash the paper row wins, and community submissions from braindecode/contents can only ADD new model/strategy combos, never override an official result.5d5fb3c -
feat: merge real (paper) results with new community results
Previously the board showed HF contents OR the bundled sample_results.json (fallback) — so the first worker write would have hidden the real paper results. Now the leaderboard treats the bundled paper results as an immutable base and OVERLAYS new results from braindecode/contents, deduped by (model, adapter); a freshly evaluated result wins on a same-key clash, every other paper result is preserved. Falls back to base-only when contents is absent. Also alias oeb 'frozen' -> arena 'probe' in results mapping so new linear-probe results align with the paper rows and the frontend filters. Tests: leaderboard merge + frozen->probe alias (22 passing).19eb9db -
fix: results mapping returns [] when a run produced no results
The live in-Space worker test surfaced this: when every experiment fails, oeb's DataFrame has no 'test_balanced_accuracy' column, and the mapper indexed it directly -> KeyError('test_balanced_accuracy'), which masked the real failure. Guard the missing/empty metric column and return [] so the worker reports the clean 'benchmark produced no completed results' error instead. Adds regressions.e42d567 -
feat: OpenEEGBench submission flow (arena runs the eval)
Replace the leftover LLM-leaderboard submission with a first-class OpenEEGBench flow. Users submit a model reference (model_cls + checkpoint + finetuning strategy/head/datasets); the backend validates it, enforces a 5-per-month per-user cap, and queues a request to braindecode/requests. A worker (stub) runs oeb.benchmark() and maps results to braindecode/contents. Backend: - submission_config: OEB enums/defaults (single source of truth) - submissions: pure validation, request building, monthly cap - results_mapping: OEB DataFrame -> contents schema (balanced_accuracy) - model_validation: EEG/OEB weights-source + license checks - OEB-shaped submit endpoint/service; eval worker stub (run_eval_worker.py) Frontend: - OEB submission form (model_cls, weights source, strategies, heads, PEFT modules, datasets, seeds, contact/visibility) with conditional fields - rewritten submission guide (5/month, help -> OpenEEGBench issues, citation) - queue/limit copy aligned; fixed duplicate style key…4db3be0
