Spaces:
Running
Running
Commit History
-
Upload from GitHub Actions: results: update committed snapshots to current 71-model state
Verified signatureThe repo snapshots were stale (40-model b7b017f restore) while HF/the live dashboard are at 71 fully-covered models. Refresh results.json/models.json/ languages.json/model_health.json so GitHub matches what's published. (Actions runs push to HF but don't commit back, so these are updated by hand.)19fbc15
-
Upload from GitHub Actions: restore results snapshots to b7b017f (canonical 40-model state)
Verified signatureThe "restore" I pushed in 5f30ceb was wrong: it re-aggregated the entire results-detailed log over 132 models (not just current cohort), which let sparse-coverage auto-discovered models bubble to the top of the leaderboard with cherry-picked 100% scores. Dashboard rank #1 was google/gemma-4-31b-it (a model with only a handful of evaluated combos), not actual frontier models. Reverting results/{results,models,languages}.json to b7b017f (40 models, 221 langs, 59115 rows — the 2026-03-10 canonical snapshot). Pushed the same files to the HF datasets so the Space backend picks them up. The running eval workflow (26099052615) will replace this with a fresh 99-model snapshot when it completes via the canonical main.py aggregation filter. Until then, the dashboard shows correct historical data instead of broken current data.4e0fc02 -
Upload from GitHub Actions: guard main.py against partial-scale HF pushes; restore aggregated results
Verified signatureA smoke run earlier today (N_LANGUAGES=5 N_MODELS=150) silently overwrote fair-forward/evals-for-every-language-results with a filtered subset: the aggregation step at main.py:97 filters results_agg to current_models x current_languages, then save() pushes regardless of scale, so a 5-language smoke truncated the dashboard from ~1000 langs to 5. results-detailed was untouched (it merges full history), which made recovery possible. Restoration: re-aggregated the entire results-detailed table (701k rows, 135 models, 221 langs -> 70.9k aggregate rows) and pushed only the `results` dataset back. Local results/*.json snapshots now reflect the restored state. Footgun patch: - evals/main.py: * Add CANONICAL_N_LANGUAGES_FOR_PUSH (1000) and CANONICAL_N_MODELS_FOR_PUSH (100) constants. * Compute ALLOW_HF_PUSH_RESULTS at module load. When either env scale is below canonical, route results/models/languages through save_local_only() instead of save(), keeping the …691e6c2
-
Upload from GitHub Actions: refresh pyproject metadata + README HF frontmatter
Verified signaturepyproject.toml - description: real one (was "Add your description here") - drop bert-score and tiktoken from the dev group; neither is imported anywhere. Kept elevenlabs + jiwer because they pair with the dormant FLEURS/CommonVoice ASR pipeline. README.md (HF Spaces frontmatter) - models: refreshed to 8 representative current frontier models (was 6 stale entries including openai/whisper-large-v3 which is ASR and not active). Added a comment noting evals/models.py is canonical (the full curated list is ~42). - datasets: dropped mozilla-foundation/common_voice_1_0 and google/fleurs (ASR dormant); added the Masakhane benchmarks that actually drive multilingual coverage. - language: tags expanded from English+German only to a small cross-tier sample (CJK, Indic, Romance, Semitic, Bantu, Niger- Congo) for HF discoverability. uv.lock: regenerated.1eccc3f