Spaces:
Running on Zero
Running on Zero
Commit History
-
Keep core models loaded across requests on T4 Running
Unverified signatureOn T4 with concurrency_limit=1, models are now loaded once globally and persist across all sequential requests. Only Paladin models are cleaned up after each request, while core models (CTransPath, Optimus, Aeon, marker_classifier) remain in GPU memory. This eliminates the overhead of reloading 8-12GB of models for every single-slide request on T4 instances. High-memory GPUs continue to use the previous per-batch loading strategy. Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>a2d70a9 -
Centralize hardware detection and optimize T4 GPU memory management
Unverified signature- Create new mosaic.hardware module for centralized GPU detection - Set T4 concurrency limit to 1 to prevent concurrent OOM errors - Update analysis.py, model_manager.py, and ui/app.py to use centralized detection - Simplify cleanup logic: only Paladin models cleared on T4, core models persist - Remove redundant GPU cleanup code (already handled by model_cache.cleanup()) Fixes T4 memory issues by enforcing sequential processing while maintaining efficient batch processing on high-memory GPUs. Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>42a4892 -
Add aggressive GPU memory cleanup after UI analysis
Unverified signature- Clear intermediate data structures (all_slide_masks, all_aeon_results, etc) - Force garbage collection and CUDA cache clearing - Log both allocated and reserved GPU memory after cleanup - Should reduce memory footprint from 4GB to near zero after analysis - Helps prevent OOM on consecutive T4 runs4fda083 -
Add aggressive GPU memory cleanup for T4 instances
Unverified signature- Force CUDA synchronization before cache clearing - Add explicit garbage collection after model cleanup - Log GPU memory usage after cleanup for debugging - Prevents OOM crashes when running multiple slides consecutively - Especially important for T4 GPUs with limited VRAM (~16GB)5549e2b -
Improve model download performance on HF Spaces
Unverified signature- Enable HF_HUB_ENABLE_HF_TRANSFER for faster downloads - Add informative logging about download size and expected time - Helps prevent timeout perception on T4 instances with slower network - Model download is ~1GB and may take several minutes on first runa15a72c -
Handle legacy 'Unknown' sex values gracefully
Unverified signature- Silently convert 'Unknown' to empty string in validation - Handle 'Unknown' in update_settings_column function - Prevents 'Unknown is not in choices' error from Gradio - Update uv.lock for Python version constraint - Users will need to manually select Male/Female for analysis6bb43ff -
Add progressive results display and sex validation to UI
Unverified signature- Add sex requirement check before analysis starts - Display AEON and PALADIN results progressively as slides are processed - Show settings table during multi-slide processing for progress tracking - Add initial yield to make UI responsive immediately - Fix sex dropdown to use None instead of 'Unknown' as default - Convert None to empty string when building settings DataFrame - Ensure Score column is numeric before rounding in PALADIN results - Hide settings table for single-slide mode, show for multi-slide mode7c7a1ba -
Fix premature sex validation warnings
Unverified signature- Only warn about invalid sex values (not Male/Female), not empty/None - Empty or None sex values will be validated at analysis time - Prevents annoying warnings when settings are first created - Improve warning message to show the invalid valuee66cef9 -
Enforce required sex parameter in Gradio CLI app
Unverified signature- Remove 'Unknown' as default sex value - Add validation to ensure sex is provided for single slide processing - Update help text to indicate sex is required parameter - Remove fallback to 'Unknown' when sex is missingac41c6b -
Update AEON inference to require sex parameter
Unverified signature- Remove 'Unknown' from sex choices in CLI arguments - Update docstrings to indicate sex is required - Clarify that sex must be 'Male' or 'Female' for inference0d1b788 -
Remove 'Unknown' sex option from UI utilities
Unverified signature- Update SEX_OPTIONS to only include 'Male' and 'Female' - Remove automatic defaulting to 'Unknown' for missing sex values - Update validation to handle empty/null sex values appropriatelyddfa8f1 -
Make sex parameter required in encode_sex function
Unverified signature- Remove 'Unknown' as valid sex option, only accept 'Male' or 'Female' - Raise ValueError if invalid sex value is provided - Update documentation to clarify sex is required parameter466e964 -
Fix AEON results column naming for multiple slides
Unverified signatureWhen running multiple slides, rename the 'Confidence' column to the slide name before concatenation. This ensures each slide has a uniquely named column in the combined results table instead of duplicate 'Confidence' column names.7774e34
-
Fix missing metadata files in model download
Include metadata/* pattern in snapshot_download to ensure metadata/target_dict.tsv is downloaded. This fixes FileNotFoundError when running Aeon inference. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>df63a56
-
format
Unverified signature2f29552 -
Refactor inference modules to eliminate code duplication
Unverified signatureSimplified run() and run_model() functions to delegate to their _with_model/_with_preloaded counterparts after loading the model, removing ~125 lines of duplicated inference logic across both modules. Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>0ab0da6 -
Add CLI options to control model downloads
Unverified signature- Add --download-models-only flag to download models and exit - Add --skip-model-download flag to skip model download (use cached) - Update Makefile download-models target to use new CLI option - Document new options in README Advanced Usage section - Use getattr() for robust handling in tests Usage: mosaic --download-models-only # Pre-download models mosaic --skip-model-download <args> # Skip download check make download-models # Download via Makefile14c1ba6 -
Update src/mosaic/inference/paladin.py
Unverified signatureCo-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>414e6c1 -
Add GitHub Actions workflows and comprehensive test suite
Unverified signature- Add CI/CD workflows for tests, code quality, and Docker builds - Add comprehensive Makefile with test, lint, format, and Docker targets - Add new test files for CLI, fixtures, UI components, and UI events - Refactor batch analysis into main analysis module - Update model manager and inference modules - Add .dockerignore for cleaner Docker builds - Update dependencies in uv.lock4780d8d
-
Fix remaining hardcoded data paths after rebase
Unverified signatureFound and fixed 3 hardcoded paths that weren't using get_data_directory(): - analysis.py:468 - Paladin model map path in batch processing - aeon.py:77 - Metadata path in run_with_model function - model_manager.py:157 - Data directory in load_all_models function All paths now use get_data_directory() to properly resolve HF cache or local data directory. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>c2c8715 -
Add comprehensive logging for batch processing verification
Unverified signatureImproves logging throughout the batch processing pipeline to clearly demonstrate that the optimization is working correctly. Model Manager (src/mosaic/model_manager.py): - Add detailed GPU detection and memory reporting - Show total GPU memory and memory before/after loading each model - Log memory management strategy (T4 aggressive vs A100 caching) - Clear banner indicating models loaded ONCE per batch - Show Paladin model cache hits vs new loads with ✓ indicators - Distinguish between aggressive mode (load/free) and caching mode Batch Analysis (src/mosaic/batch_analysis.py): - Add comprehensive timing for entire batch and per-slide processing - Log batch start with clear separator - Show per-slide progress with [n/total] indicators - Track and report slide processing times (avg, min, max, total) - Calculate and display batch overhead vs processing time - Add detailed summary at end with: * Success/failure counts * Model loading time (done once) * Total batch time * Per-slid…6e06a36 -
Fix segment_tissue unpacking in batch analysis
Unverified signatureThe analyze_slide_with_models function was incorrectly calling segment_tissue and unpacking its return value. segment_tissue returns 4 values (polygon, _, coords, attrs) but we were only unpacking 2. This fix: - Converts seg_config string to config object (BiopsySegConfig, etc.) - Calls segment_tissue with proper individual parameters - Correctly unpacks all 4 return values - Uses polygon for slide mask visualization Fixes: ValueError: too many values to unpack (expected 2) 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>42bcf72 -
Add batch processing optimization for slide analysis
Unverified signatureImplements batch processing to reduce model loading overhead by ~90% when processing multiple slides. Models are now loaded once per batch instead of once per slide, providing 25-45% overall speedup for multi-slide batches. New features: - ModelCache class with adaptive memory management (T4 vs A100 GPUs) - Batch coordinator that loads models once and reuses across all slides - Automatic batch mode for >1 slide in both Gradio UI and CLI - GPU type detection for memory-optimized strategies - Comprehensive test suite with unit, integration, and regression tests Implementation: - src/mosaic/model_manager.py: Model loading and caching infrastructure - src/mosaic/batch_analysis.py: Batch processing coordinator - src/mosaic/analysis.py: Batch-optimized pipeline functions - src/mosaic/inference/aeon.py: Add run_with_model() for pre-loaded models - src/mosaic/inference/paladin.py: Add run_with_models() for batch mode - src/mosaic/ui/app.py: Integrate batch mode in Gradio UI - src/mosaic/grad…0234c58
-
Fix user directory creation and hide HF Spaces logs locally
Unverified signature- Fix create_user_directory return type inconsistency that caused TypeError - Only show GPU allocation and user auth logs when running on HF Spaces - Prevents confusing log messages when running the app locally 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>07d6e0e
-
Fix circular import by moving get_data_directory to separate module
Unverified signatureMoved get_data_directory() function from gradio_app.py to a new data_directory.py module to avoid circular imports. The circular import occurred because gradio_app.py imports from ui/utils.py, which was trying to import back from gradio_app.py. Changes: - Create src/mosaic/data_directory.py with get_data_directory() and set_data_directory() functions - Update gradio_app.py to import and use set_data_directory() - Update all other modules (aeon.py, data.py, utils.py, analysis.py) to import from data_directory module instead of gradio_app 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>52f8cb9 -
Fix model file location to use HuggingFace cache directory
Unverified signatureChanged all model file references from hardcoded local data/ directory to use HuggingFace cache directory. This fixes the issue where files couldn't be found after downloading from HuggingFace Hub. Changes: - Add get_data_directory() function to gradio_app.py that checks MOSAIC_DATA_DIR environment variable first, then falls back to local data/ for development - Update gradio_app.py to download to HF cache (removed local_dir parameter) and set MOSAIC_DATA_DIR environment variable - Update all model file references in aeon.py, data.py, utils.py, and analysis.py to use get_data_directory() instead of hardcoded paths 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>751062d
-
Define DEFAULT_TISSUE_SITE_IDX constant for "Not Applicable" tissue site
copilot-swe-agent[bot] +1: raylimCo-authored-by: raylim <3074310+raylim@users.noreply.github.com>e1efb8d -
Merge pull request #12 from pathology-data-mining/copilot/sub-pr-7-one-more-time
Unverified signature[WIP] Update sex and tissue site parameter support based on feedback50b5538 -
Add error handling for missing tissue site CSV file
copilot-swe-agent[bot] +1: raylimCo-authored-by: raylim <3074310+raylim@users.noreply.github.com>45cb0d2 -
Merge pull request #10 from pathology-data-mining/copilot/sub-pr-7-another-one
Unverified signatureAdd error handling for missing CSV mapping files2f88344 -
Merge pull request #9 from pathology-data-mining/copilot/sub-pr-7-again
Unverified signatureFix hardcoded relative path in aeon.py for target_dict.tsv8047026 -
Add error handling for missing CSV files in get_tissue_site_map and get_sex_map
copilot-swe-agent[bot] +1: raylimCo-authored-by: raylim <3074310+raylim@users.noreply.github.com>85adf5f -
Fix hardcoded path for target_dict.tsv using robust path construction
copilot-swe-agent[bot] +1: raylimCo-authored-by: raylim <3074310+raylim@users.noreply.github.com>0e9fc10 -
Update src/mosaic/inference/data.py
Unverified signatureCo-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>7b9f0ea -
Update src/mosaic/analysis.py
Unverified signatureCo-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>9cc95d5 -
Remove unused global cancer type mappings from aeon.py
Unverified signature- Remove unused imports of INT_TO_CANCER_TYPE_MAP and CANCER_TYPE_TO_INT_MAP - Remove unused col_indices_to_drop global variable - Simplify CANCER_TYPES_TO_DROP to only the 5 types that are excluded - Update tests to import mappings from data.py instead of aeon.py - All 8 tests pass successfully The Aeon inference now exclusively uses metadata-loaded mappings (160 classes) instead of the old global constants (183 classes), eliminating confusion. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>bac4d5d -
Add Aeon model test suite and reproducibility scripts
Unverified signature- Add comprehensive test summary (AEON_TEST_SUMMARY.md) - Add test results for 3 validation slides (100% accuracy, avg 99.10% confidence) - Add reproducibility scripts: - export_aeon_checkpoint.py: Convert PyTorch Lightning checkpoints to pickle - run_aeon_tests.sh: Automated test runner with validation - verify_aeon_results.py: Result verification against ground truth - Update analysis.py to use standard aeon_model.pkl path - Update .gitignore to exclude slide images and masks Test Results: - Slide 881837 (BLCA): 98.19% confidence ✓ - Slide 744547 (HCC): 99.49% confidence ✓ - Slide 755246 (HCC): 99.61% confidence ✓ 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>0506a57 -
Add comprehensive sex and tissue site parameter support
Unverified signatureThis commit enhances the Aeon inference pipeline with complete support for sex and tissue site parameters across the entire application: - Add sex and tissue site encoding functions with CSV-based mapping - Update Aeon model to load correct target mappings from metadata - Add CLI arguments for sex and tissue site in gradio_app.py - Add validation for sex and tissue site in settings - Update dependencies: add lightning, seaborn, and statsmodels - Fix tissue site map file path references 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>a2b6947
-
Complete implementation of sex and tissue site parameters
Unverified signature- Add UI dropdowns for Sex and Tissue Site selection - Update settings dataframe to include Sex and Tissue Site columns - Pass sex and tissue_site parameters through the analysis pipeline - Convert sex string to index (0=Male, 1=Female, None=Unknown) - Convert tissue site string to index using tissue site mapping - Add get_tissue_sites() helper function in ui/utils.py - Update all inference pipeline functions to handle these parameters - Pass sex and tissue_site_idx to Aeon model for improved inference49fbf68
-
Add sex and tissue site parameters to Aeon inference
Unverified signature- Add encoding functions to data.py (encode_sex, encode_tissue_site, tissue_site_to_one_hot) - Update TileFeatureTensorDataset to optionally include SEX and TISSUE_SITE fields - Add sex and tissue_site_idx parameters to Aeon run() function - Thread sex and tissue_site through analysis pipeline (Aeon only, Paladin unchanged) - Add Sex and Tissue Site dropdowns to UI - Add validation for Sex and Tissue Site in settings CSV - Update SETTINGS_COLUMNS to include new fieldsde40714
-
Fix user detection by decoding JWT token from referer
Unverified signature24b5de2 -
Add debug logging for request object
Unverified signature6d1bfd0
-
Improve user detection: check username and auth headers
Unverified signaturecd62763 -
Add request parameter to analyze_slide function signature
Unverified signature0524123 -
Add request parameter to analyze_slides function signature
Unverified signature3cb7ed4 -
Pass request parameter through UI to analyze_slide
Unverified signature- Add request: gr.Request parameter to analyze_slides - Pass request to analyze_slide call - Fixes 'request is not defined' errord98d38e -
Add gr.Request parameter to analyze_slide signature
Unverified signature7f1cc8e -
Add dynamic GPU duration: 60s for anonymous, 300s for logged-in users
Unverified signature- Split pipeline into _free (60s) and _pro (300s) versions - Both call same _impl function for code reuse - analyze_slide now accepts gr.Request parameter - Logged-in users get 300s (can process 10k+ tiles) - Anonymous users get 60s (limited to ~2000 tiles) - Log user status for debugging This allows both free and PRO users to use the Space0e5928b
-
Fix T4 detection by checking actual GPU name
Unverified signature- No SPACE_HARDWARE env var on HF Spaces T4 - Detect T4 by checking torch.cuda.get_device_name() - Look for 'T4' in GPU name (Tesla T4) - Add hardware logging at module load - Should now correctly use 4 workers on T4cbb7db9 -
Add T4 GPU detection and optimized settings
Unverified signature- Detect T4 hardware via SPACE_HARDWARE env var - T4 settings: batch_size=64, num_workers=4 - ZeroGPU settings: batch_size=128, num_workers=0 - Standard GPU: batch_size=64, num_workers=8 - Auto-adapts to hardware for optimal performance - Ready to test on T4 small instance18b7b6e