Spaces:
Sleeping
Sleeping
|
Download docs/ProjectStructure.md from Pitt-iRiS-Lab/MedIRT: direct link, hf CLI and curl.
- Browser
- Download file 3.61 kB
-
https://huggingface.co/spaces/Pitt-iRiS-Lab/MedIRT/resolve/main/docs/ProjectStructure.md
- Command line
-
hf download hf://spaces/Pitt-iRiS-Lab/MedIRT/docs/ProjectStructure.md
-
curl -L -o ProjectStructure.md https://huggingface.co/spaces/Pitt-iRiS-Lab/MedIRT/resolve/main/docs/ProjectStructure.md
3.61 kB
| # Architecture Blueprint: MedIRT Interactive Leaderboard | |
| ## 1. Project Structure | |
| We'll use a standard structure for a Gradio application on Hugging Face. All the necessary logic and data will be contained within the repository. | |
| * app.py: The core of the application. This single Python script will contain all the backend logic for data processing and the Gradio UI definition. | |
| * data/leaderboard_data.csv: A CSV file containing all the necessary pre-computed results from your analysis. Storing processed data is much more efficient than re-calculating it on the fly in a public Space. | |
| * requirements.txt: A text file listing the Python dependencies (gradio, pandas, plotly) that Hugging Face needs to install. | |
| * README.md: The main page of your Hugging Face Space. You'll use this to explain the project, the MEDIRT framework, and how to use the leaderboard. | |
| ## 2. Data Management | |
| ### Source Data (leaderboard_data.csv) | |
| This is the most critical piece. You will need to structure a single CSV file that contains the data needed to generate all the tables and plots. A suggested schema based on the paper would be: | |
| * model_name: The name of the LLM (e.g., openai/gpt-5). | |
| * topic: The medical topic (e.g., Cardio, Comm, Bio/Epi). | |
| * irt_ability_score: The topic-specific ability score (θ). | |
| * accuracy: The topic-specific accuracy. | |
| * mean_irt_ability_overall: The model's average θ across all topics. | |
| * mean_accuracy_overall: The model's average accuracy. | |
| * mean_time_s: Average response time in seconds. | |
| * total_cost_usd: Total evaluation cost in USD. | |
| ### In-App Data Handling | |
| The app.py script will load leaderboard_data.csv into a pandas DataFrame once at startup. All interactive filtering and processing will be performed on this DataFrame in memory. | |
| ## 3. Backend Logic (app.py) | |
| The backend will consist of Python functions that are triggered by user interactions in the Gradio UI. | |
| * Data Loading: A function to load the CSV into a global pandas DataFrame. | |
| * Plotting Functions: Separate functions to generate the visualizations. These functions will take filtered data as input and return plot objects. | |
| * create_radar_chart(df, models): Takes a list of model names, filters the main DataFrame, and generates an interactive radar chart using Plotly. | |
| * create_heatmap(df): Generates the main topic-wise ability heatmap using Plotly. | |
| * Data Filtering: Logic within the Gradio event handlers to select and reshape data for the tables and plots based on user input (e.g., selecting models from a dropdown). | |
| ## 4. Frontend / UI (app.py with Gradio) | |
| The user interface will be built entirely using Gradio components, organized into tabs for clarity. Layout: A gr.Blocks() interface with four main tabs (gr.Tab). | |
| * Tab 1: Overall Performance: | |
| * A gr.DataFrame to display the main leaderboard (replicating Table 2 from the paper), showing overall ability, accuracy, cost, and time. | |
| * Tab 2: Topic-Specific Ability Profiles: | |
| * A gr.Markdown for instructions. | |
| * A gr.CheckboxGroup or gr.Dropdown(multiselect=True) to allow users to select one or more models to compare. | |
| * A gr.Plot element that displays the interactive radar chart (Figure 3) generated by Plotly. | |
| * A second gr.Plot element to show the overall heatmap of all top models (Figure 2). | |
| * Tab 3: Cost & Efficiency Analysis: | |
| * A gr.DataFrame to show the cost-performance trade-offs (replicating Table 3). | |
| * (Optional) A gr.Plot with a Plotly scatter plot visualizing Ability vs. Cost. | |
| * Tab 4: About the MEDIRT Framework: | |
| * A gr.Markdown component to explain the project, methodology, and link to the research paper. | |