Spaces:
Running
Running
Commit History
-
Remove agenteval.json, use hardcoded mappings as single source of truth
openhandsSimplify the data loader by removing the agenteval.json config file and its loading logic. The benchmark-to-category mappings are now defined directly in the code, eliminating redundancy and potential inconsistencies.24ccc42 -
Fix category names and benchmark assignments
openhands- Use preferred category names: Issue Resolution, Frontend, Greenfield, Testing, Information Gathering - Move swe-bench-multimodal to Frontend category (was incorrectly in Issue Resolution) - Remove multi-swe-bench (not used) - Update about page to reflect correct benchmark-category mappingab22529 -
Remove asta/astabench references
openhands- Update utm_source from asta_leaderboard to openhands_index - Remove 'Asta' from toolset description - Update citation text to use openhands-index - Update legal disclaimer with OpenHands URL - Simplify README Hugging Face integration section - Delete unused agenteval_backup.json file Keep acknowledgement to AstaBench in about.py as it credits the source.35aa299
-
Fix category cost calculation - add category-level aggregation
openhands +1: openhands- Updated simple_data_loader.py to build proper category-to-benchmarks mapping from agenteval.json - Added logic to calculate category-level aggregate scores and costs from individual benchmark results - Updated agenteval.json to include category tags (Bug Fixing, App Creation, etc.) - Fixes issue where Bug Fixing cost was showing as 0.0 instead of the average of swe-bench and swe-bench-multimodal costs Co-authored-by: openhands <openhands@all-hands.dev>5e9c3b9 -
Add verbose logging and include JSONL data files
openhands +1: openhands- Add comprehensive logging throughout app.py to track initialization progress - Include JSONL data files that were missing from data directory - These files contain the actual leaderboard data needed by SimpleLeaderboardViewer Co-authored-by: openhands <openhands@all-hands.dev>b5df580 -
Initial OpenHands Index leaderboard based on ASTA Bench
openhands +1: openhands- Updated branding (title, README, logo) - Changed dataset references to OpenHands Index datasets - Modified agenteval.json to include 6 OpenHands datasets: * swe-bench * multi-swe-bench * swe-bench-multimodal * swt-bench * commit0 * gaia - Added mock data for initial testing Co-authored-by: openhands <openhands@all-hands.dev>085a012