Spaces:
Sleeping
Sleeping
Commit History
-
Migrate to Ruff for linting/formatting and add comprehensive import tests
ming- Replace black, isort, and flake8 with ruff (Rust-based, 10-100x faster) - Add ruff.toml configuration with appropriate rules and ignores - Update pre-commit hook to run ruff check --fix and ruff format before tests - Update CLAUDE.md documentation with new ruff commands - Fix code style issues (nested if statements, formatting) - Add comprehensive import tests (test_imports.py) to catch import errors early - Fix config.py to ignore extra environment variables (prevents build failures) - Format all code with ruff (18 files reformatted) - Auto-fix 32 linting issues - Fix remaining test linting issues (B017, SIM117, B023) Benefits: - Single tool instead of three (black/isort/flake8) - Faster pre-commit hooks - Better CI/CD performance - Automatic import validation before deployment29ed661
-
feat: Add V4 NDJSON patch-based structured summarization
ming- Refactor StructuredSummarizer with new NDJSON streaming protocol - Add summarize_structured_stream_ndjson() method for patch-based streaming - Implement state management with _empty_state() and _apply_patch() - Update system prompt to instruct model to output NDJSON patches - Add new /api/v4/scrape-and-summarize/stream-ndjson endpoint - Support incremental state updates via delta/state events - Use deterministic decoding (greedy) for consistent results - Maintain backwards compatibility with existing stream endpoint Event structure: - delta: JSON patch object (set/append/done operations) - state: Current accumulated state - done: Completion flag - tokens_used: Token count - latency_ms: Final latency metric Test suite: - test_v4_ndjson_mock.py: Protocol logic validation (PASSED ✅) - test_v4_ndjson_http.py: HTTP endpoint test - test_v4_ndjson_url.py: URL scraping test (PASSED ✅) - test_v4_ndjson.py: Direct service test Documentation: - NDJSON_REFACTOR_SUMMARY.md:…93c9664
-
feat: Implement V3 Web Scraping + Summarization API
ming✨ New Features: - Add V3 API endpoint: POST /api/v3/scrape-and-summarize/stream - Backend web scraping with trafilatura (95%+ success rate) - In-memory TTL-based caching (1 hour default, configurable) - User-agent rotation to avoid anti-scraping measures - Metadata extraction (title, author, date, site_name) - SSRF protection (blocks localhost and private IPs) - Streaming SSE response with metadata + content chunks 📦 New Components: - ArticleScraperService: High-quality article extraction - SimpleCache: In-memory cache with TTL and max size - V3 Router: Complete API implementation with validation - V3 Schemas: Request/response models with security validators 🧪 Testing: - 30 new tests (100% passing) - Cache tests: TTL, expiration, thread safety - Scraper tests: Success, timeouts, validation - API tests: Security (SSRF), error handling, streaming format 📝 Documentation: - Updated CLAUDE.md with V3 details - Updated README.md with V3 usage examples - Added V3_SCRAPING_IMPLEMENTATION…2ed2bd7
-
chore: initialize FastAPI backend project structure and testing setup
ming9024ad9