Spaces:
Running
Running
Commit History
-
Add dedicated API Access section to About page
openhands- New section with links to API documentation - Links to key endpoints (/api/leaderboard, /api/categories) - Includes curl example for quick reference168fd64 -
Add REST API for programmatic access to leaderboard data
openhands- Add api.py with FastAPI endpoints for leaderboard data - Mount API on root path using gr.mount_gradio_app() - Add API documentation link to About page Resources section - API uses same data loading functions as UI for consistency Endpoints: - /health - Health check with cache status - /leaderboard - Full leaderboard with filtering/sorting - /leaderboard/models - List all models - /leaderboard/model/{name} - Get specific model - /categories - List benchmark categories - /benchmarks - List benchmarks - /docs - Swagger UI documentationd05a70e
-
Fix nav links opening in new tab and simplify methodology text
openhands +1: openhands- Remove target="_blank" from navigation links so they open in same tab - Remove unnecessary note about agents without cost/runtime data Co-authored-by: openhands <openhands@all-hands.dev>1ca9fc3 -
Add methodology section and benchmark instance counts to About page
openhands +1: openhands- Add instance/task counts for all five benchmarks - Add new Methodology section explaining: - Per-benchmark scores are percentages (comparable across datasets) - Average score is macro-average with equal weighting - Cost and runtime are per-instance averages - All evals use identical OpenHands Agent SDK configuration Co-authored-by: openhands <openhands@all-hands.dev>d8b77db
-
Update About page structure
openhands- Combine Benchmarks and Scoring into 'Benchmark Details' - Change 'Raw Data' to 'Resources' with links to: - OpenHands (main repo) - Software Agent SDK (agent code) - Benchmarks (benchmarking code) - Results (raw evaluation data) - Update citation URL to https://index.openhands.devb6f00ad -
Fix category names and benchmark assignments
openhands- Use preferred category names: Issue Resolution, Frontend, Greenfield, Testing, Information Gathering - Move swe-bench-multimodal to Frontend category (was incorrectly in Issue Resolution) - Remove multi-swe-bench (not used) - Update about page to reflect correct benchmark-category mappingab22529 -
Simplify and update About page
openhands- Made content more concise - Updated benchmark list to match actual categories - Added links to all benchmark sources - Removed 'Submitting Results' section (not accepting external submissions) - Added Slack link for contact - Updated citation author to 'OpenHands Team'cfd4f2a
-
Remove multi-swe-bench from OpenHands Index
openhands- Remove multi-swe-bench from benchmark mappings in simple_data_loader.py - Remove multi-swe-bench from generate_mock_jsonl.py - Update FRONTEND_DEVELOPMENT_DESCRIPTION to reference SWE-bench Multimodal (not Multi-SWE-bench) - Remove multi-swe-bench from about.py benchmark lists and acknowledgements - Update DATA_STRUCTURE.md to remove multi-swe-bench references - Keep 5 categories (Frontend uses swe-bench-multimodal, not multi-swe-bench)b978a6b
-
UI cleanup and About page updates
openhands +1: openhands1. Removed legend box from graph (OpenHands/Pareto/Company Logos) 2. Combined Pareto, lock, and company logo columns into Language Model column 3. Removed dataset structure SVG image from main page 4. Updated Acknowledgements section - shortened AstaBench credit, added links to component benchmarks 5. Updated Citation to be for OpenHands Index with authors: Juan Michelini, Simon Rosenberg, Xingyao Wang, Graham Neubig Co-authored-by: openhands <openhands@all-hands.dev>6737ff3
-
docs: Update descriptive text to use Average Score and Total Cost
openhands +1: openhands- Changed 'Overall score' to 'Average score' in content.py and about.py - Changed 'Overall cost' to 'Total cost' in content.py and about.py - Updated Total cost description to reflect that it's a sum, not an average Co-authored-by: openhands <openhands@all-hands.dev>bb0f7af
-
Add Acknowledgements section crediting AstaBench
openhands- Added new Acknowledgements section to About page - Credits AstaBench leaderboard (Allen Institute for AI) for UI design and methodology - Lists key aspects adapted: macro-averaging, visualization, UI structure - Positioned before Citation section737a3f2 -
Remove submission page and add submission instructions to About page
openhands +1: openhands- Removed submission page import and route from app.py - Removed redirect_submission_on_close_script (no longer needed) - Added new section in About page: 'Submitting Results & Accessing Raw Data' - Instructions now point users to submit via GitHub PR to openhands-index-results - Added information about accessing raw results from GitHub repository - Clarified data format requirements (metadata.json and scores.json) Submissions are now handled via GitHub PR workflow instead of web form Co-authored-by: openhands <openhands@all-hands.dev>701c496
-
Replace all AstaBench references with OpenHands Index
openhands- Updated branding across all files (about.py, content.py, etc.) - Rewrote about.py to focus on software engineering benchmarks - Updated descriptions to reflect OpenHands' focus on coding agents - Maintained original design and layout structure7904c2d -
Initial OpenHands Index leaderboard based on ASTA Bench
openhands +1: openhands- Updated branding (title, README, logo) - Changed dataset references to OpenHands Index datasets - Modified agenteval.json to include 6 OpenHands datasets: * swe-bench * multi-swe-bench * swe-bench-multimodal * swt-bench * commit0 * gaia - Added mock data for initial testing Co-authored-by: openhands <openhands@all-hands.dev>085a012