Spaces:
Sleeping
Sleeping
Commit History
-
feat: Major scraper enhancements - consistent format, better extraction, per-month costs
Riley +1: ClaudeThis commit significantly improves the grant scraper to extract more complete and consistent data: ## Scraper Improvements (snapshot.py) ### 1. Consistent JSON Structure - Added `id` and `competition_id` fields to all snapshots - Ensures compatibility between manual (snapshot.py) and auto (discover_grants.py) scraping - All existing code handles both old and new formats gracefully ### 2. Enhanced Funding Rates Extraction - Pattern 1: Standard format "up to 60% if you are micro/small" - Pattern 2: Combined format "up to 60% if you are a micro, small or medium" - Pattern 3: KTP format "large companies...50%...SMEs...67%" - Now extracts rates for diverse grant types ### 3. Enhanced Duration Extraction - Range: "last/must be between 12 and 36 months" - Single max: "last up to nine months" (supports word numbers like "nine") - Maximum: "maximum 24 months" - Added word-to-number conversion for written numbers ### 4. Per-Month Cost Calculation (NEW) - Detects patterns like "£8,500 per mo…2ae7490 -
feat: Major system enhancements - GPT-5 support, monitoring, translation, and optimizations
Riley +1: ClaudeThis commit includes 15 major improvements to the grant-analyst system: ## New Features ### GPT-5 Model Support (Step 1) - Implemented three-tier GPT-5 model system: - gpt-5-nano for routing and simple decisions - gpt-5-mini for translations and summaries - gpt-5 for complex analysis - Updated token encoding to use o200k_base (GPT-5) with cl100k_base fallback - Updated all configuration files and documentation ### Grant Translation Endpoint (Step 14) - New `/translate` API endpoint for converting grant jargon to simple language - Structured output with 4 sections: eligibility, process, timeline, requirements - Uses gpt-5-mini model for cost-effective translations - Permanent caching in MongoDB (translations don't change) - Batch translation support via `/translate/batch` ### Monitoring & Metrics System (Step 15) - New metrics collection system tracking: - P50/P95/P99 latency per endpoint - Token usage per request (prompt + completion) - Cache hit/miss rates (Redis + Mon…057c21e
-
feat: add PDF & supporting materials integration to context
Riley Coleman- Add build_context_with_supporting() function to embed extracted PDF text - Add get_supporting_docs_for_grant() to retrieve supporting materials - Add _load_supporting_docs_jsonl() helper to index 1,124 supporting documents - Auto-extract grant ID from URL if not present - Support filtering by document type (PDF only or mixed) - Include top k supporting docs with full text in LLM context Benefits: ✓ Model now has direct access to PDF content (not just URLs) ✓ Can answer questions about briefing materials and policies ✓ Can cite specific sections from supporting documents ✓ ~1,120 HTML sections + 4 PDFs available for context ✓ No external API calls needed - content is embedded Usage: from src.analyzer.context_builder import build_context_with_supporting context = build_context_with_supporting(grant, k=5) # Pass to LLM with full supporting materials embedded See PDF_CONTEXT_INTEGRATION.md for complete integration guide. 🤖 Generated with Claude Codebfcc872
-
Initial commit of grant-analyst project
Riley Coleman19de729