# Architecture Blueprint: MedIRT Interactive Leaderboard ## 1. Project Structure We'll use a standard structure for a Gradio application on Hugging Face. All the necessary logic and data will be contained within the repository. * app.py: The core of the application. This single Python script will contain all the backend logic for data processing and the Gradio UI definition. * data/leaderboard_data.csv: A CSV file containing all the necessary pre-computed results from your analysis. Storing processed data is much more efficient than re-calculating it on the fly in a public Space. * requirements.txt: A text file listing the Python dependencies (gradio, pandas, plotly) that Hugging Face needs to install. * README.md: The main page of your Hugging Face Space. You'll use this to explain the project, the MEDIRT framework, and how to use the leaderboard. ## 2. Data Management ### Source Data (leaderboard_data.csv) This is the most critical piece. You will need to structure a single CSV file that contains the data needed to generate all the tables and plots. A suggested schema based on the paper would be: * model_name: The name of the LLM (e.g., openai/gpt-5). * topic: The medical topic (e.g., Cardio, Comm, Bio/Epi). * irt_ability_score: The topic-specific ability score (θ). * accuracy: The topic-specific accuracy. * mean_irt_ability_overall: The model's average θ across all topics. * mean_accuracy_overall: The model's average accuracy. * mean_time_s: Average response time in seconds. * total_cost_usd: Total evaluation cost in USD. ### In-App Data Handling The app.py script will load leaderboard_data.csv into a pandas DataFrame once at startup. All interactive filtering and processing will be performed on this DataFrame in memory. ## 3. Backend Logic (app.py) The backend will consist of Python functions that are triggered by user interactions in the Gradio UI. * Data Loading: A function to load the CSV into a global pandas DataFrame. * Plotting Functions: Separate functions to generate the visualizations. These functions will take filtered data as input and return plot objects. * create_radar_chart(df, models): Takes a list of model names, filters the main DataFrame, and generates an interactive radar chart using Plotly. * create_heatmap(df): Generates the main topic-wise ability heatmap using Plotly. * Data Filtering: Logic within the Gradio event handlers to select and reshape data for the tables and plots based on user input (e.g., selecting models from a dropdown). ## 4. Frontend / UI (app.py with Gradio) The user interface will be built entirely using Gradio components, organized into tabs for clarity. Layout: A gr.Blocks() interface with four main tabs (gr.Tab). * Tab 1: Overall Performance: * A gr.DataFrame to display the main leaderboard (replicating Table 2 from the paper), showing overall ability, accuracy, cost, and time. * Tab 2: Topic-Specific Ability Profiles: * A gr.Markdown for instructions. * A gr.CheckboxGroup or gr.Dropdown(multiselect=True) to allow users to select one or more models to compare. * A gr.Plot element that displays the interactive radar chart (Figure 3) generated by Plotly. * A second gr.Plot element to show the overall heatmap of all top models (Figure 2). * Tab 3: Cost & Efficiency Analysis: * A gr.DataFrame to show the cost-performance trade-offs (replicating Table 3). * (Optional) A gr.Plot with a Plotly scatter plot visualizing Ability vs. Cost. * Tab 4: About the MEDIRT Framework: * A gr.Markdown component to explain the project, methodology, and link to the research paper.