Spaces:
Running
Running
Commit History
-
Add harness logos for Claude Code, Codex CLI, Gemini CLI, OpenHands
Debug AgentPopulate the assets/harnesses/ slots that the previous commit carved out. Every row on the Alternative Agents scatter now renders as a composite marker: the model provider logo on top and the harness logo on the bottom. One small adjustment applied in-place to the four source SVGs: the width/height attributes that hard-coded the rendered size to 1em were stripped. Plotly's layout-image renderer doesn't establish a font-size context, so 1em collapses to zero and the artwork renders as a speck in the corner of the marker box. Removing those two attributes lets Plotly size each SVG through its existing viewBox and the scatter's configured sizex/sizey instead. No artwork was modified. Smoke test: Alternative Agents view (7 rows) now produces 14 scatter markers, every point is a full composite pair.e6877f3 -
Move harness logos to assets/harnesses/ with graceful missing-file fallback
Debug AgentThe previous two passes shipped placeholder SVG badges in assets/ directly (first plain letters, then generic CLI prompt glyphs). Neither was a real logo, and the review feedback was to ship the slot without the art so the actual logos can be dropped in by hand later. Restructure so the code no longer assumes any specific file exists: - New dedicated folder assets/harnesses/ with only a README documenting the expected filenames and the mapping from agent_name -> file stem. No placeholder artwork ships with the repo. - leaderboard_transformer.py: replace the hardcoded HARNESS_LOGO_PATHS map with HARNESS_LOGO_STEMS + a small resolver in get_harness_icon() that tries {stem}.svg then {stem}.png under assets/harnesses/ and returns None when no file exists. The three cases that yield None (empty agent_name, unregistered agent_name, or no file on disk) all make the scatter-plot caller skip the harness layer, so every point falls back to a single model-provider marker — the …282557d -
Replace harness badge letters with generic pictograms
Debug AgentThe first pass used two-letter abbreviations (CC/CX/GC/OH) in brand-coloured rounded squares to identify each harness on the Alternative Agents scatter. Letters worked but looked cheap. Replace the three CLI harnesses (Claude Code, Codex CLI, Gemini CLI) with a generic terminal-prompt glyph — a chevron plus cursor, the near-universal visual shorthand for "this is a CLI tool" — and keep the per-harness brand-adjacent background colour as the actual discriminator between them. This mirrors how the evo-claw.com reference image distinguishes Gemini CLI and Codex CLI markers. For the OpenHands harness, draw a simple five-finger hand silhouette (five vertical finger strokes plus a rounded palm) in the existing OpenHands yellow. It's geometric primitives — lines and one quadratic path — not a derivative of any trademarked logo, and is unambiguously a hand at marker size. All four files keep the same 80x80 viewBox and filename, so no code change is needed — leaderboard_transformer.py's comp…8a1c0ed -
Stack model + harness logos on Alternative Agents scatter
Debug AgentOn the Alternative Agents page the same LLM can show up under multiple harnesses (e.g. claude-sonnet-4-5 under Claude Code vs OpenHands Sub-agents). Before this change both points drew the exact same Anthropic company logo as their marker, so they were visually indistinguishable — the only way to tell them apart was the hover tooltip's "Harness:" line from the earlier fix. Now when a row carries an "Agent" column value (which is only the case on the Alternative Agents page — DataTransformer.view() drops the Agent column on the canonical OpenHands pages via the has_mixed_agents check), the scatter plot draws a composite marker: the model provider logo on top and the harness logo on the bottom, stacked symmetrically around the point's true coordinate. Canonical pages keep the single-marker layout with zero visual change. - New HARNESS_LOGO_PATHS map + get_harness_icon() helper next to the existing get_marker_icon(). Kept in sync with AGENT_NAME_BY_TYPE from OpenHands/evaluation pus…fb16d57 -
Add harness logo assets for Alternative Agents composite markers
Debug AgentFour SVG badges used as the bottom half of the composite scatter markers on the Alternative Agents page: Claude Code, Codex CLI, Gemini CLI, and OpenHands (shared by the OpenHands and "OpenHands Sub-agents" harnesses). These are intentionally simple rounded-square badges in brand-adjacent colors rather than hunted-down official CLI marks. The goal is just to make two points for the same LLM run under different harnesses visually distinct at a glance — e.g. claude-sonnet-4-5 under Claude Code vs claude-sonnet-4-5 under OpenHands Sub-agents, which otherwise share the exact same Anthropic company logo. Easy to swap for real logos later without touching the scatter-plot code (just replace the SVG files).6860223 -
Add Alternative Agents page screenshot
Verified signaturePR preview asset. Captures the /alternative-agents route after the ACP benchmark data restoration (openhands-index-results #832-#843): all seven harness x model rows render complete, the scatter plot shows the cost-vs-score frontier populated, and the 'Show incomplete entries' toggle reports zero by default.55dc87f -
Show harness in scatter-plot hover on Alternative Agents page
Debug AgentThe Alternative Agents scatter plot labels each point with the Language Model's company logo but nothing else, so two points for the same LLM run under different harnesses (e.g. claude-sonnet-4-5 under Claude Code vs OpenHands Sub-agents) are visually indistinguishable — the hover tooltip only showed "{lm_name} (SDK {version})". Add a "Harness: {agent}" line to the tooltip when the row has an Agent value. The canonical OpenHands pages drop the Agent column in DataTransformer.view() (has_mixed_agents check), so the new line only shows up on the Alternative Agents page where it carries signal — no visual change on Home or the per-category subpages.f00a617 -
Move alternative agents to a dedicated page
Verified signatureAddress review feedback from the original commit: - Mixing default OpenHands runs and alternative-agent runs (Claude Code, Codex, Gemini CLI, OpenHands Sub-agents) in the same leaderboard table and the same scatter plot is misleading. Their cost/runtime numbers aren't apples-to-apples across harnesses, and conflating them with the OpenHands ranking on Home obscures what the index is actually measuring. - Move alternative-agent rows to a dedicated /alternative-agents page. Home and the per-category subpages stay exactly as before (default-OpenHands only). Changes: - simple_data_loader.SimpleLeaderboardViewer now takes an `agent_filter` argument, defaulting to "openhands". When "openhands" it walks results/{model}/. When "alternative" it walks alternative_agents/{type}/{model}/. - ui_components.get_leaderboard_viewer_instance / get_full_leaderboard_data thread `agent_filter` through and key the in-memory cache on (split, agent_filter) so the two pages don't fig…d7034b8 -
Show ACP agent results in the leaderboard
Verified signature## Summary The HF Space currently only loads `results/{model}/` (default OpenHands runs). The ACP runs (`acp-claude`, `acp-codex`, `acp-gemini`, `openhands_subagents`) live in `alternative_agents/{type}/{model}/` in the openhands-index-results repo and never made it into the dataframe, so the website silently dropped them. After OpenHands/openhands-index-results#820–#829 + #830, all the ACP Claude Code data from the master table in OpenHands/benchmarks#576 is in the canonical location, but the leaderboard still doesn't show it. This PR teaches the loader to ingest `alternative_agents/` and adds an **Agent** column to the leaderboard so OpenHands vs Claude Code vs Codex vs Gemini CLI are visible at a glance. ## Changes - **`setup_data.py`** — copy `alternative_agents/` alongside `results/` when fetching the index repo, so all submissions land in the data dir. - **`simple_data_loader.py`**: - Factor per-directory loading into `_records_from_agent_dir` and have `_load_from_agent_dir…bef7ade
-
Date in summary table is missing (#10)
- Date in summary table is missing (45fc3a4337e324d17f54b9564db1dbc11951ff36)7949205
-
Hide incomplete entries from home page charts by default
openhands0b126a8
-
Fix root path redirect issue on HuggingFace Spaces
openhands +1: openhandsThe issue: When Gradio is mounted at path '/', FastAPI's default behavior causes a 307 redirect from '/' to '//', which breaks routing on HuggingFace Spaces (the page appears blank). The fix: 1. Add RootRedirectMiddleware that intercepts requests to '/' and redirects to '/home' with a 302 status code before Gradio handles it 2. Create FastAPI app with redirect_slashes=False to prevent automatic trailing slash redirects This is a workaround for Gradio bug #11071: https://github.com/gradio-app/gradio/issues/11071 Co-authored-by: openhands <openhands@all-hands.dev>41e5220
-
Add dedicated API Access section to About page
openhands- New section with links to API documentation - Links to key endpoints (/api/leaderboard, /api/categories) - Includes curl example for quick reference168fd64 -
Remove redundant root redirect - now handled by proxy
openhandsc560986 -
Add explicit root redirect with 302 status code
openhandsUse 302 redirect instead of letting Gradio's default 307 redirect handle it. The 307 redirect was causing issues with the Vercel proxy due to malformed redirect URLs (double slash, http instead of https).20c58ab -
Remove conflicting server-side redirect
openhandsThe JavaScript redirect_script already handles / -> /home redirection. The server-side redirect was causing issues with the proxy.9ba6edb -
Fix root route redirect to /home
openhandsThe root FastAPI app needs an explicit handler for / to redirect to /home since Gradio multi-page apps don't handle the bare root path.14690be -
Mount API at /api path instead of root
openhands- API endpoints now available at /api/* (e.g., /api/leaderboard, /api/health) - Swagger docs at /api/docs - Gradio UI remains at root path (/)c744cbf -
Add REST API for programmatic access to leaderboard data
openhands- Add api.py with FastAPI endpoints for leaderboard data - Mount API on root path using gr.mount_gradio_app() - Add API documentation link to About page Resources section - API uses same data loading functions as UI for consistency Endpoints: - /health - Health check with cache status - /leaderboard - Full leaderboard with filtering/sorting - /leaderboard/models - List all models - /leaderboard/model/{name} - Get specific model - /categories - List benchmark categories - /benchmarks - List benchmarks - /docs - Swagger UI documentationd05a70e
-
Fix nav links opening in new tab and simplify methodology text
openhands +1: openhands- Remove target="_blank" from navigation links so they open in same tab - Remove unnecessary note about agents without cost/runtime data Co-authored-by: openhands <openhands@all-hands.dev>1ca9fc3 -
Add methodology section and benchmark instance counts to About page
openhands +1: openhands- Add instance/task counts for all five benchmarks - Add new Methodology section explaining: - Per-benchmark scores are percentages (comparable across datasets) - Average score is macro-average with equal weighting - Cost and runtime are per-instance averages - All evals use identical OpenHands Agent SDK configuration Co-authored-by: openhands <openhands@all-hands.dev>d8b77db -
Add z.ai logo for GLM language model
openhands +1: openhandsCo-authored-by: openhands <openhands@all-hands.dev>3b71657
-
fix: Add Visualization column to main table (not just benchmark tables)
openhands +1: openhandsThe previous commit only added visualization links to individual benchmark detail tables. This commit adds the Visualization column to the main front page table as well. Changes: - Add 'visualization' column to record in simple_data_loader.py - Add 'Visualization' to ending_cols in leaderboard_transformer.py - Add make_visualization_icons() to format the column in ui_components.py - Update column widths and datatypes to include Visualization - Add tooltip description for Visualization column Co-authored-by: openhands <openhands@all-hands.dev>5f628a6 -
feat: Add Visualization column for Laminar eval links
openhands +1: openhandsDisplay eval_visualization_page URLs as clickable 📊 icons in a new Visualization column next to Logs. Changes: - Load eval_visualization_page from score entries in simple_data_loader.py - Store as '{benchmark} visualization' column similar to download URLs - Add 📊 column to benchmark detail tables with clickable links - Only show icon if visualization URL exists, otherwise blank - Add tooltip documentation for the new column Co-authored-by: openhands <openhands@all-hands.dev>dfa8bfc -
Reduce left/right margins on mobile view
openhandsAdd responsive CSS media query for screens 768px and narrower that: - Reduces padding on #page-content-wrapper from 25px to 8px - Reduces padding on #intro-paragraph from 35px to 8px - Reduces padding on .nav-holder from 20px to 8px - Reduces padding on .gradio-container - Makes content full width (max-width: 100%) - Reduces table cell padding for better mobile display0f8031a
-
Fix dark mode modebar styling - transparent background with light gray icons
openhands +1: openhands- Make modebar, modebar-container, modebar-group, and modebar-btn backgrounds transparent - Style modebar icons to match the light gray text color (neutral-400) - Add hover state with slightly lighter color for better UX This fixes the ugly gray background that appeared when hovering over Plotly figures in dark mode. Co-authored-by: openhands <openhands@all-hands.dev>4f4a27d
-
Widen Logs column to prevent vertical stacking of download icons
openhands +1: openhandsCo-authored-by: openhands <openhands@all-hands.dev>9cab912 -
Add download icons to Logs column for benchmark results
openhands +1: openhandsCo-authored-by: openhands <openhands@all-hands.dev>67867ec
-
Skip entries with hide_from_leaderboard=True
openhands +1: openhandsFilter out models from the leaderboard when their metadata has hide_from_leaderboard set to true. This allows hiding models that shouldn't be displayed (e.g., due to lack of comparable models in similar size classes). Co-authored-by: openhands <openhands@all-hands.dev>2123552
-
Format date column to show only date, not time
openhands- Add format_date_column function to leaderboard_transformer.py - Apply date formatting in ui_components.py for all table displays - Strips time portion from ISO datetime strings (e.g. 2025-11-24T19:56:00 -> 2025-11-24)2b7cd27
-
Trigger rebuild
openhands +1: openhandsCo-authored-by: openhands <openhands@all-hands.dev>73df5bb -
Update mock data and docs to match new openhands-index-results format
openhands +1: openhands- Update scores.json: rename total_cost -> cost_per_instance, total_runtime -> average_runtime - Add new required fields to mock data: full_archive, agent_version, submission_time - Add agent_name and country fields to metadata.json - Update DATA_STRUCTURE.md documentation to reflect new schema This ensures mock data works as a proper fallback when GitHub is unavailable and matches the production format from openhands-index-results. Co-authored-by: openhands <openhands@all-hands.dev>f7e6b58
-
Add NVIDIA logo and update company mappings
openhands +1: openhands- Added logo-nvidia.svg asset for NVIDIA branding - Added NVIDIA and Nemotron to COMPANY_LOGO_MAP - Added NVIDIA and Nemotron to MODEL_COUNTRY_MAP (US company) Co-authored-by: openhands <openhands@all-hands.dev>68b25a7
-
Fix navigation links to use relative paths and match category names
openhands +1: openhands1. Add JavaScript to convert absolute navigation hrefs to relative paths - Fixes issue where links pointed to hf.space domain when accessed via index.openhands.dev - Navigation now stays on the same domain regardless of proxy configuration 2. Update route paths to match category names: - Issue Resolution: /bug-fixing → /issue-resolution - Greenfield: /app-creation → /greenfield - Frontend: /frontend-development → /frontend - Testing: /test-generation → /testing Co-authored-by: openhands <openhands@all-hands.dev>0813604 -
Trigger deploy
openhandsbdd9196
-
Empty commit to update results
openhands +1: openhandsCo-authored-by: openhands <openhands@all-hands.dev>2e46852 -
Fix: Preserve mark_by selection during periodic data refresh
openhands +1: openhandsThe 60-second timer that checks for data updates was not passing the current mark_by dropdown value, causing charts to revert to 'Company' when data refreshed. Changes: - Add mark_by parameter to check_and_refresh_data() - Add mark_by parameter to check_and_refresh_all() - Add mark_by parameter to check_and_refresh_simple() - Include mark_by_dropdown in timer inputs for all refresh paths - Pass mark_by to create_cost_scatter_plot() and create_runtime_scatter_plot() Co-authored-by: openhands <openhands@all-hands.dev>df058f7 -
Fix icon positioning for date-based charts
openhands +1: openhandsUse data coordinates (xref='x', yref='y') for date axes instead of domain coordinates. This matches the previous behavior where icons were positioned using actual x/y values. - Date charts: use data coordinates with sizex in milliseconds - Log charts: continue using domain coordinates (0-1 range) Co-authored-by: openhands <openhands@all-hands.dev>88a8840 -
Refactor: Create generic create_scatter_chart() as single source of truth
openhands +1: openhandsMajor refactoring to eliminate code duplication across scatter plots: - Add create_scatter_chart() in leaderboard_transformer.py (~270 lines) - Handles all scatter plot types: cost, runtime, date, params - Configurable x-axis type (log or date) - Configurable Pareto frontier direction - Consistent marker icons, hover text, and styling - Auto-detects column names - Add STANDARD_LAYOUT and STANDARD_FONT constants for shared styling - Simplify visualizations.py from 536 lines to 159 lines - create_evolution_over_time_chart() now uses generic function - create_accuracy_by_size_chart() now uses generic function - Only contains data filtering and column detection logic Benefits: - Single source of truth for all scatter plot styling - Consistent fonts (Arial) across all charts - Easier to maintain and extend - ~375 lines of code removed Co-authored-by: openhands <openhands@all-hands.dev>1f75b60 -
Fix font consistency and remove code duplication in visualizations.py
openhands +1: openhands- Remove duplicated branding code (OPENHANDS_LOGO_PATH, URL_ANNOTATION, get_openhands_logo_images, add_branding_to_figure) - now imported from leaderboard_transformer.py - Fix STANDARD_FONT to use FONT_FAMILY_SHORT constant instead of hardcoded 'Outfit' - All charts now consistently use Arial font from constants.py Co-authored-by: openhands <openhands@all-hands.dev>f1798d2 -
Connect 'Mark systems by' selector to Evolution and Size charts
openhands +1: openhands- Update visualizations.py to import get_marker_icon and MARK_BY_DEFAULT - Add mark_by parameter to create_evolution_over_time_chart() - Add mark_by parameter to create_accuracy_by_size_chart() - Update main_page.py to connect mark_by_dropdown to all charts - Evolution and Size charts now update when mark_by selection changes Co-authored-by: openhands <openhands@all-hands.dev>d731ad4 -
Update intro text: remove 'standardized' from 'we need ways'
openhands +1: openhandsCo-authored-by: openhands <openhands@all-hands.dev>315feb4 -
Add 'Mark systems by' selector for scatter plot icons (Company/Openness/Country)
openhands +1: openhands- Add MARK_BY_CHOICES and MARK_BY_DEFAULT constants - Add country flag SVGs (US, China, France) - Add COUNTRY_FLAG_MAP and MODEL_COUNTRY_MAP for country detection - Add get_openness_icon() and get_country_from_model() functions - Add get_marker_icon() to select appropriate icon based on mark_by setting - Modify _plot_scatter_plotly() to accept mark_by parameter - Add mark_by_dropdown to UI in ui_components.py - Update update_display() to regenerate plots dynamically based on mark_by selection Co-authored-by: openhands <openhands@all-hands.dev>ed6e90d -
Keep CSS font separate from Python constants
openhandsa87403b -
Consolidate font definitions into single source of truth
openhandsCreate constants.py with FONT_FAMILY and FONT_FAMILY_SHORT. All Python files now import and use these constants instead of hardcoded font values.acd5bf8 -
Change plot fonts from Outfit to Arial
openhands0750050 -
Update About page structure
openhands- Combine Benchmarks and Scoring into 'Benchmark Details' - Change 'Raw Data' to 'Resources' with links to: - OpenHands (main repo) - Software Agent SDK (agent code) - Benchmarks (benchmarking code) - Results (raw evaluation data) - Update citation URL to https://index.openhands.devb6f00ad -
Fix font change - update Gradio theme font to Arial
openhands721acc6 -
Change font to Arial for better readability
openhandsfaddaaf -
Remove agenteval.json, use hardcoded mappings as single source of truth
openhandsSimplify the data loader by removing the agenteval.json config file and its loading logic. The benchmark-to-category mappings are now defined directly in the code, eliminating redundancy and potential inconsistencies.24ccc42