Spaces:
Sleeping
Sleeping
Commit History
-
Migrate to Ruff for linting/formatting and add comprehensive import tests
ming- Replace black, isort, and flake8 with ruff (Rust-based, 10-100x faster) - Add ruff.toml configuration with appropriate rules and ignores - Update pre-commit hook to run ruff check --fix and ruff format before tests - Update CLAUDE.md documentation with new ruff commands - Fix code style issues (nested if statements, formatting) - Add comprehensive import tests (test_imports.py) to catch import errors early - Fix config.py to ignore extra environment variables (prevents build failures) - Format all code with ruff (18 files reformatted) - Auto-fix 32 linting issues - Fix remaining test linting issues (B017, SIM117, B023) Benefits: - Single tool instead of three (black/isort/flake8) - Faster pre-commit hooks - Better CI/CD performance - Automatic import validation before deployment29ed661
-
Revert adaptive token logic, restore client-controlled max_tokens
ming +1: Claude- Restore client max_tokens control in V3 API (was being ignored) - Remove min_length parameter from HF streaming service - Re-enable temperature and top_p sampling parameters - Simplify token generation: min_new_tokens = max(20, min(50, max_tokens//4)) - Set neutral length_penalty = 1.0 to avoid length bias - Reduce recursive chunk tokens from 200 to 80 for tighter summaries - Remove adaptive token calculation tests and parameter validation tests - Update default max_tokens back to 256 in schema This simplifies the summarization logic by removing server-side token overrides and allowing the client (Android app) to control summary length directly. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>6a1e8a3 -
fix: Backend ignores client max_tokens to verify Android app hypothesis
ming +1: ClaudeThis is a diagnostic commit to verify that the Android app's max_tokens=256 is causing early stopping, NOT a backend issue. Changes Made: 1. **Remove Client max_tokens Constraint** (app/api/v3/scrape_summarize.py:121-123) - BEFORE: min(text_length // 3, payload.max_tokens, 1024) - AFTER: min(text_length // 3, 1024) - Backend now IGNORES client's max_tokens value - Always uses adaptive calculation for quality 2. **Remove Incompatible Generation Parameters** (app/services/hf_streaming_summarizer.py) - Removed temperature, top_p from gen_kwargs (lines ~375, ~670) - Removed length_penalty from gen_kwargs (lines ~391) - These parameters are invalid with do_sample=False (greedy decoding) - Eliminates transformers warning: "generation flags are not valid and may be ignored: ['temperature', 'top_p', 'length_penalty']" 3. **Update Tests** (tests/test_v3_api.py) - test_adaptive_tokens_medium_article: Now expects 600-700 tokens (not 450-512) - test_user_max_…80ea70f -
fix: Improve V3 summary completeness with enhanced token allocation
ming +1: ClaudeThis commit addresses remaining early stopping issues by fixing three cascading problems that caused summaries to truncate prematurely, especially for long articles. Critical Changes: 1. Increased recursive chunk token limit: 80 → 200 tokens per chunk - Location: app/services/hf_streaming_summarizer.py:480 - Formula: min(max_new_tokens // 2, 200) - Impact: 2.5x more tokens for long article chunks (>1500 chars) 2. Raised min_new_tokens floor: 50 → 200 tokens - Location: app/services/hf_streaming_summarizer.py:389-391, 651-652 - Formula: max(50, min(max_new_tokens // 2, 200)) - Impact: Prevents early stopping at 50 tokens when 512 allocated 3. More aggressive adaptive formula: text_length // 4 → // 3 - Location: app/api/v3/scrape_summarize.py:121 - Impact: 6000-char articles get ~2000 tokens (before 1024 cap) Expected Results: - Short articles (<1500 chars): Better min quality guarantee (50 → 200 tokens) - Medium articles (2000 chars): More complete summaries …6b2de93 -
fix: V3 API mid-sentence cutoff with adaptive token calculation
ming +1: ClaudeThis commit fixes the issue where V3 API summaries were cutting off mid-sentence by implementing adaptive token allocation and improving generation parameters. Changes: - Increase default max_tokens from 256 to 512 (app/api/v3/schemas.py) - Add adaptive token calculation based on input length (app/api/v3/scrape_summarize.py) - Formula: min(max(text_length // 4, 300), user_max, 1024) - Calculate min_length as 60% of max to encourage complete thoughts - Update HF service to accept min_length parameter (app/services/hf_streaming_summarizer.py) - Increase length_penalty from 1.0 to 1.2 to favor complete sentences - Add 10 new tests for adaptive tokens and summary completeness Results: - Short articles (~500 chars): 300-400 tokens - Medium articles (~1500 chars): 500-700 tokens - Long articles (~3000+ chars): 800-1024 tokens - All V3 tests passing (16/16) - 89% coverage for V3-specific code 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <norep…5e83010
-
Fix V3 API to support both URL and text input
ming +1: ClaudeThe V3 endpoint was causing 422 errors when receiving plain text from the Android app because it only accepted URLs. This update adds intelligent input detection to handle both modes: - URL mode: Scrapes article from URL then summarizes - Text mode: Summarizes provided text directly (no scraping) Changes: - Updated schema to make url and text optional (exactly one required) - Added model validator to ensure mutual exclusivity - Enhanced endpoint to detect and route based on input type - Updated metadata to include input_type field - Added comprehensive documentation and examples This fixes the 422 Unprocessable Entity errors and provides a single unified endpoint for the Android app. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>f724bab
-
feat: Implement V3 Web Scraping + Summarization API
ming✨ New Features: - Add V3 API endpoint: POST /api/v3/scrape-and-summarize/stream - Backend web scraping with trafilatura (95%+ success rate) - In-memory TTL-based caching (1 hour default, configurable) - User-agent rotation to avoid anti-scraping measures - Metadata extraction (title, author, date, site_name) - SSRF protection (blocks localhost and private IPs) - Streaming SSE response with metadata + content chunks 📦 New Components: - ArticleScraperService: High-quality article extraction - SimpleCache: In-memory cache with TTL and max size - V3 Router: Complete API implementation with validation - V3 Schemas: Request/response models with security validators 🧪 Testing: - 30 new tests (100% passing) - Cache tests: TTL, expiration, thread safety - Scraper tests: Success, timeouts, validation - API tests: Security (SSRF), error handling, streaming format 📝 Documentation: - Updated CLAUDE.md with V3 details - Updated README.md with V3 usage examples - Added V3_SCRAPING_IMPLEMENTATION…2ed2bd7