Spaces:
Running
Running
Commit History
-
Remove agenteval.json, use hardcoded mappings as single source of truth
openhandsSimplify the data loader by removing the agenteval.json config file and its loading logic. The benchmark-to-category mappings are now defined directly in the code, eliminating redundancy and potential inconsistencies.24ccc42 -
Fix category names and benchmark assignments
openhands- Use preferred category names: Issue Resolution, Frontend, Greenfield, Testing, Information Gathering - Move swe-bench-multimodal to Frontend category (was incorrectly in Issue Resolution) - Remove multi-swe-bench (not used) - Update about page to reflect correct benchmark-category mappingab22529 -
Remove asta/astabench references
openhands- Update utm_source from asta_leaderboard to openhands_index - Remove 'Asta' from toolset description - Update citation text to use openhands-index - Update legal disclaimer with OpenHands URL - Simplify README Hugging Face integration section - Delete unused agenteval_backup.json file Keep acknowledgement to AstaBench in about.py as it credits the source.35aa299
-
Fix Categories Attempted calculation to handle missing category columns correctly
openhands- Changed logic to check if category score is not None and not 0.0 - This fixes the issue where missing categories were being counted as attempted - Now correctly shows X/5 based on actual categories with data0718569
-
Update data loader to support agent-centric directory structure
openhands +1: openhands- Add support for new results/YYYYMMDD_model/ directory structure - Each agent directory contains metadata.json and scores.json - Maintain backward compatibility with old JSONL format - Update mock data to use new structure Co-authored-by: openhands <openhands@all-hands.dev>e003f7b -
Update UI: All-Hands-AI color scheme, agent version column names, and OpenHands logo
openhands +1: openhands- Apply All-Hands-AI color scheme (yellow accent, indigo links, beige background) - Change 'Agent' column to 'Agent Version' - Remove 'Submitter' column - Change 'Models Used' to 'Model' - Update mock data with version numbers (1.0.2, 1.0.1, etc.) - Replace logo with OpenHands branding Co-authored-by: openhands <openhands@all-hands.dev>0ee2099 -
Fix category cost calculation - add category-level aggregation
openhands +1: openhands- Updated simple_data_loader.py to build proper category-to-benchmarks mapping from agenteval.json - Added logic to calculate category-level aggregate scores and costs from individual benchmark results - Updated agenteval.json to include category tags (Bug Fixing, App Creation, etc.) - Fixes issue where Bug Fixing cost was showing as 0.0 instead of the average of swe-bench and swe-bench-multimodal costs Co-authored-by: openhands <openhands@all-hands.dev>5e9c3b9 -
Add verbose logging and include JSONL data files
openhands +1: openhands- Add comprehensive logging throughout app.py to track initialization progress - Include JSONL data files that were missing from data directory - These files contain the actual leaderboard data needed by SimpleLeaderboardViewer Co-authored-by: openhands <openhands@all-hands.dev>b5df580 -
Initial OpenHands Index leaderboard based on ASTA Bench
openhands +1: openhands- Updated branding (title, README, logo) - Changed dataset references to OpenHands Index datasets - Modified agenteval.json to include 6 OpenHands datasets: * swe-bench * multi-swe-bench * swe-bench-multimodal * swt-bench * commit0 * gaia - Added mock data for initial testing Co-authored-by: openhands <openhands@all-hands.dev>085a012