Spaces:
Sleeping
Sleeping
Commit History
-
feat: Major scraper enhancements - consistent format, better extraction, per-month costs
Riley +1: ClaudeThis commit significantly improves the grant scraper to extract more complete and consistent data: ## Scraper Improvements (snapshot.py) ### 1. Consistent JSON Structure - Added `id` and `competition_id` fields to all snapshots - Ensures compatibility between manual (snapshot.py) and auto (discover_grants.py) scraping - All existing code handles both old and new formats gracefully ### 2. Enhanced Funding Rates Extraction - Pattern 1: Standard format "up to 60% if you are micro/small" - Pattern 2: Combined format "up to 60% if you are a micro, small or medium" - Pattern 3: KTP format "large companies...50%...SMEs...67%" - Now extracts rates for diverse grant types ### 3. Enhanced Duration Extraction - Range: "last/must be between 12 and 36 months" - Single max: "last up to nine months" (supports word numbers like "nine") - Maximum: "maximum 24 months" - Added word-to-number conversion for written numbers ### 4. Per-Month Cost Calculation (NEW) - Detects patterns like "£8,500 per mo…2ae7490