Spaces:
Runtime error
Runtime error
Commit History
-
Fix leaderboard loading error when no cache exists
Instead of trying to run evaluation on page load (which can fail with network errors), show a user-friendly message asking them to click 'Re-evaluate All' to run the evaluation manually. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>b43b847 -
Add leaderboard caching and fix dataset configurations
- Add caching system to avoid re-evaluating on every page load - Fix ArabicMMLU: add required subset "All" config - Replace deprecated ArSenTD-LEV with arbml/ASTD (Egyptian dialect) - Fix Arabic Sentiment: correct text column to "tweet" - Fix SANAD: use default config with "Article" column - Add "Re-evaluate All" button for manual refresh 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>532fc72
-
Add persistent storage for submitted tokenizers
- Submitted tokenizers are now saved to submissions.json file - Submissions appear on the leaderboard after evaluation - Results persist across app restarts - All submissions are evaluated on all 8 datasets 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>1ea35a4 -
Update leaderboard UI to dark theme for better visibility
Changed all leaderboard tables and submission results from light theme to dark theme matching the app's overall design. Updated colors for headers, rows, text, and metric indicators. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>c6c6ac2 -
Evaluate all tokenizers on all datasets with full data
- Remove dataset and tokenizer selection options - Evaluate ALL tokenizers on ALL 8 datasets automatically - Increase sample sizes significantly (5000 per dataset, ~36K total) - Simplified leaderboard UI - just shows results - Submit tab also evaluates on all datasets 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>d57f9ad -
Add tokenizer submission feature
- New "Submit" tab for users to evaluate any HuggingFace tokenizer - Input fields: Model ID, display name, organization, model type - Dataset selection for evaluation - Professional results display with: - Overall score with quality indicator - Key metrics (fertility, compression, UNK rate) - Per-dataset breakdown table - Submission guidelines and tips 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>0c3f890 -
Auto-run leaderboard evaluation and improve UI colors
- Leaderboard now runs automatically on page load - Options moved to collapsible accordion for cleaner UX - Redesigned tables with professional color scheme: - Neutral dark headers (#2c3e50, #495057) - Subtle row backgrounds for top 3 - Medals for rank display (🥇🥈🥉) - Clean Bootstrap-style metric colors - Improved typography and spacing 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>7b9de45 -
Add HuggingFace Spaces YAML configuration
🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>751def7 -
Refactor: modularize codebase into separate modules
- Split monolithic app.py into logical modules: - config.py: tokenizer registry, datasets, sample texts - tokenizer_manager.py: tokenizer loading and caching - analysis.py: tokenization analysis functions - leaderboard.py: HF dataset evaluation - utils.py: Arabic text utilities - styles.py: CSS styles - ui_components.py: HTML generation - Add .gitignore for Python/Gradio - Add __init__.py for package structure 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>f32d4c7 -
Update README.md
Verified signaturef2a2081 -
Update app.py
Verified signature59baef6 -
Update app.py
Verified signature06d6710 -
Update app.py
Verified signaturef32f063
-
Update app.py
Verified signature2b73d7a
-
Update app.py
Verified signature3f8c44e -
Update app.py
Verified signature27c0e8b -
Update app.py
Verified signatureb613c61 -
Update app.py
Verified signature00fb4c8 -
Update app.py
Verified signature91eabe4 -
Update app.py
Verified signature778be61 -
Update app.py
Verified signature71c7fc4 -
Update app.py
Verified signature8d000e9 -
Create Dockerfile
Verified signaturee0681b7 -
Update app.py
Verified signature0a9a631 -
Update app.py
Verified signature176f915 -
Update app.py
Verified signaturec296fab -
Update app.py
Verified signaturebd02afc -
Update app.py
Verified signature31687bc -
Update app.py
Verified signatureb9f9278
-
Update app.py
Verified signature4c91389 -
Update app.py
Verified signature3b0ce68 -
Update app.py
Verified signature5ad6fed
-
Update app.py
Verified signature7760bbc -
Update app.py
Verified signaturee47d405 -
Update app.py
Verified signature9f4f9aa -
Update app.py
Verified signature4aae5f1 -
Update app.py
Verified signaturec8d4230 -
Update app.py
Verified signature81d456a -
Update app.py
Verified signature6be3180 -
Update app.py
Verified signatureece3f89 -
Update app.py
Verified signature5357bd8 -
Update app.py
Verified signature9079275 -
Update app.py
Verified signature6f9d03f -
Update app.py
Verified signature3567a04 -
Update app.py
Verified signature8b17681 -
Update app.py
Verified signature11bd107 -
Update app.py
Verified signature7ed66d7 -
Update app.py
Verified signature58c0de5 -
Update app.py
Verified signature890f5a9 -
Update app.py
Verified signatured5b1f62