Spaces:
Sleeping
Sleeping
Commit History
-
Apply ruff formatting to Loguru migration files
ming +1: ClaudeAutomatic code formatting applied by ruff during pre-commit hook. These are cosmetic changes only (code style, no functional changes). Files formatted: - app/core/logging.py - tests/test_logging.py - tests/test_loguru_context.py - tests/test_loguru_integration.py - tests/test_loguru_setup.py 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>75fe59b -
Migrate logging from stdlib to Loguru for structured logging
ming +1: ClaudeThis migration replaces Python's standard logging module with Loguru, providing structured logging with automatic request ID propagation, environment-aware formatting, and improved developer experience. ## Changes ### Core Implementation - app/core/logging.py: Complete rewrite (224 lines) - Environment-aware formatting (JSON for prod, colored text for dev) - Context variables for automatic request ID propagation - Custom JSON serializer with structured fields - Backward-compatible get_logger() and RequestLogger APIs - app/core/middleware.py: - Integrated context var (request_id_var) for request ID injection - Automatic propagation eliminates manual parameter passing - app/core/config.py: - Added LOG_FORMAT setting (json/text/auto) with validator - Auto-detection based on HF_SPACE_ROOT_PATH environment - requirements.txt: - Added loguru>=0.7.0 dependency ### Testing (44 tests, 100% passing) - tests/test_loguru_context.py: Context variable propagation (7 tests) - …7ab470d -
Migrate to Ruff for linting/formatting and add comprehensive import tests
ming- Replace black, isort, and flake8 with ruff (Rust-based, 10-100x faster) - Add ruff.toml configuration with appropriate rules and ignores - Update pre-commit hook to run ruff check --fix and ruff format before tests - Update CLAUDE.md documentation with new ruff commands - Fix code style issues (nested if statements, formatting) - Add comprehensive import tests (test_imports.py) to catch import errors early - Fix config.py to ignore extra environment variables (prevents build failures) - Format all code with ruff (18 files reformatted) - Auto-fix 32 linting issues - Fix remaining test linting issues (B017, SIM117, B023) Benefits: - Single tool instead of three (black/isort/flake8) - Faster pre-commit hooks - Better CI/CD performance - Automatic import validation before deployment29ed661
-
Implement Option 3: Use FP16 for 2-3x faster inference
mingSpeed optimization: - Added v4_use_fp16_for_speed config option - When enabled, uses FP16 instead of 4-bit quantization - FP16 is 2-3x faster than 4-bit NF4 quantization - Enabled by default in Dockerfile for maximum speed Memory trade-off: - FP16 uses ~2-3GB GPU memory (vs ~1GB for 4-bit) - Still fits comfortably on T4 GPU (16GB total) Expected results: - Generation time: 24.9s → 8-12s (2-3x speedup) - Same output quality - Faster token generation (~20-30 tokens/sec vs ~6 tokens/sec) This completes the speed optimization plan (Option 1 + 2 + 3)7fff563 -
Optimize V4 generation speed: greedy decoding + reduced max_tokens
mingSpeed optimizations: - Switched from sampling to greedy decoding (2-3x faster) - Changed do_sample=True to do_sample=False - Removed temperature and top_p parameters (not needed for greedy) - Reduced max_tokens from 512 to 256 (model only needs ~150 tokens) Expected results: - Generation time: 24.5s → 8-12s (2-3x speedup) - Deterministic output (better for structured JSON) - Lower memory usage (smaller token limit) Trade-offs: - Slightly less diverse output (but fine for structured JSON) - Still using 4-bit quantization (memory efficient)fd2a8c1 -
Optimize V4 output verbosity and generation speed
mingBrevity improvements: - Added explicit length constraints in system prompt - Title: 8-12 words max (was unlimited) - Main summary: 2-3 sentences max - Key points: 3-5 items, each 10-15 words - Category: 1-2 words only Speed improvements: - Switched from greedy decoding to sampling (5-10x faster) - Changed do_sample=False to do_sample=True - Set temperature=0.3 (low but fast) - Added top_p=0.9 for quality Token optimization: - Reduced default max_tokens from 1024 to 512 - More concise outputs complete within token limit Expected results: - Generation time: 49s → 5-10s - Titles: 20+ words → 8-12 words - Complete output with all required fields17499f7
-
feat: Switch V4 to Qwen2.5-1.5B for HF memory compatibility
ming- Replace Phi-3-mini (3.8B) with Qwen2.5-1.5B-Instruct - Phi-3-mini still exceeded 16GB limit even with V2 disabled - Qwen2.5-1.5B: ~3GB RAM (comfortable within 16GB) - 3x larger than 0.5B version (much better quality) - Good structured output and NDJSON generation capability - Memory: V4: 3GB + system: 2GB = ~5GB (plenty of headroom)fe47248 -
perf: Disable V2 warmup to save memory for V4
ming- Disable V2 HuggingFace warmup by default (enable_v2_warmup=False) - Frees up ~1.2GB RAM for V4 Phi-3-mini model - V2 endpoints still work (model loads on first request) - Required to fit Phi-3-mini within HF 16GB limit - V2: 0GB + V4 peak: 15GB + system: 2GB = ~17GB (within limit)1b76b21 -
feat: Switch V4 model to Phi-3-mini for better structured output
ming- Replace Qwen/Qwen2.5-0.5B-Instruct with microsoft/Phi-3-mini-4k-instruct - Phi-3-mini (3.8B params) has much better instruction following capability - Expected improvements: - Valid NDJSON patch generation - Better structured JSON output - More reliable field completion - Trade-off: Slower generation (~20-30s vs ~10s) but higher quality - RAM usage: ~7-8GB (quantized) vs ~1-2GB for Qwen2.5-0.5B7019b66 -
feat: Add V4 NDJSON patch-based structured summarization
ming- Refactor StructuredSummarizer with new NDJSON streaming protocol - Add summarize_structured_stream_ndjson() method for patch-based streaming - Implement state management with _empty_state() and _apply_patch() - Update system prompt to instruct model to output NDJSON patches - Add new /api/v4/scrape-and-summarize/stream-ndjson endpoint - Support incremental state updates via delta/state events - Use deterministic decoding (greedy) for consistent results - Maintain backwards compatibility with existing stream endpoint Event structure: - delta: JSON patch object (set/append/done operations) - state: Current accumulated state - done: Completion flag - tokens_used: Token count - latency_ms: Final latency metric Test suite: - test_v4_ndjson_mock.py: Protocol logic validation (PASSED ✅) - test_v4_ndjson_http.py: HTTP endpoint test - test_v4_ndjson_url.py: URL scraping test (PASSED ✅) - test_v4_ndjson.py: Direct service test Documentation: - NDJSON_REFACTOR_SUMMARY.md:…93c9664
-
feat: Implement V3 Web Scraping + Summarization API
ming✨ New Features: - Add V3 API endpoint: POST /api/v3/scrape-and-summarize/stream - Backend web scraping with trafilatura (95%+ success rate) - In-memory TTL-based caching (1 hour default, configurable) - User-agent rotation to avoid anti-scraping measures - Metadata extraction (title, author, date, site_name) - SSRF protection (blocks localhost and private IPs) - Streaming SSE response with metadata + content chunks 📦 New Components: - ArticleScraperService: High-quality article extraction - SimpleCache: In-memory cache with TTL and max size - V3 Router: Complete API implementation with validation - V3 Schemas: Request/response models with security validators 🧪 Testing: - 30 new tests (100% passing) - Cache tests: TTL, expiration, thread safety - Scraper tests: Success, timeouts, validation - API tests: Security (SSRF), error handling, streaming format 📝 Documentation: - Updated CLAUDE.md with V3 details - Updated README.md with V3 usage examples - Added V3_SCRAPING_IMPLEMENTATION…2ed2bd7
-
Add support for sshleifer/distilbart-cnn-6-6 model for V2 API
ming- Added BART model detection and specific input handling - Updated default model from t5-small to sshleifer/distilbart-cnn-6-6 - BART models now receive direct text input without prefixes - Updated warmup and health check methods for BART compatibility - Updated Dockerfile and README documentation6e48ad3 -
Optimize V2 API for HuggingFace Spaces with t5-small
ming- Change default model from Phi-3-mini to t5-small (~250MB vs 7GB) - Update Dockerfile for V2-only deployment (remove Ollama) - Add t5-small specific handling in HF streaming service - Add HuggingFace cache directory configuration - Update README with V2-primary deployment info - Optimize for HuggingFace Spaces free tier limits - Fast startup: ~30-60 seconds vs 2-3 minutes - Memory efficient: ~500MB RAM vs 8GB+ for Phi-3 - V2 model downloads during warmup for instant first requestd174be4 -
Add V2 API with HuggingFace streaming support
ming- Add V2 API endpoints with HuggingFace TextIteratorStreamer - Implement real-time token-by-token streaming via SSE - Add configurable HuggingFace model support (default: Phi-3-mini-4k-instruct) - Maintain V1 API compatibility - Android app only needs to change /api/v1/ to /api/v2/ - Add conditional warmup (V1 disabled, V2 enabled by default) - Add comprehensive tests for V2 API and HF streaming service - Update README with V2 documentation and Android client examples - Add accelerate package for better device mapping - All 116 tests pass (97 original + 19 new V2 tests)0b6e76d
-
Fix 504 Gateway Timeout: Increase timeout values
ming- Increase Ollama timeout from 30s to 60s in config - Update Nginx timeouts: connect/send 60s, read 90s - Update dynamic timeout cap from 60s to 90s in summarizer - Update Docker Compose and Hugging Face deployment configs - Update timeout optimization tests to reflect new values This should resolve the 504 Gateway Timeout errors on Hugging Face Spaces by providing more time for CPU-bound inference in the shared environment.d3f36f7
-
Optimize latency and performance for Hugging Face Spaces
mingPerformance Optimizations: - Reduce base timeout from 60s to 30s for faster failure detection - Optimize timeout calculation: 3s per 1000 chars (was 5s) with 60s cap (was 120s) - Reduce default max_tokens from 256 to 100 for faster responses - Add text preprocessing: truncate texts >4000 chars for faster processing - Optimize model parameters: * Lower temperature (0.1) for faster, focused output * Add top_p, top_k, repeat_penalty for efficiency * Limit context window to 2048 tokens for speed - Shorter, more direct prompt for faster processing Expected Results: - 2-3x faster response times - Better handling of large texts - Reduced timeout errors - More reliable performance on free tierf2cff39 -
Switch to llama3.2:1b base model for better availability
ming- Use llama3.2:1b instead of llama3.2:1b-instruct - Base model should be more readily available in Ollama - Still small enough to fit within storage limits - Should resolve the model not found error64e4f7a -
Fix storage limit issue by switching to llama3.2:1b-instruct
ming- Switch to 1B parameter model to reduce storage usage - Add model cleanup step to remove existing models and free space - 1B model should fit within Hugging Face Spaces 50GB limit - Still instruction-fine-tuned for good summarization quality - Much faster inference than larger models43c3d8e -
Switch to llama3.2:3b-instruct model for faster performance
ming- Update default model to llama3.2:3b-instruct (3B parameters) - Smaller model should provide faster response times - Still instruction-fine-tuned for better summarization quality - Reduced memory and resource usage4cc543a -
Switch to mistral:7b-instruct model for better instruction following
ming- Update default model in config.py to mistral:7b-instruct - Update Dockerfile to pull the new model during startup - The instruct variant is better optimized for summarization tasksaf820b0 -
Fix Ollama host default to use 0.0.0.0 instead of localhost for container environments
mingae3b1b5 -
Fix Ollama permissions and model configuration for Hugging Face deployment
mingbd3417a -
Optimize timeout configuration for better performance
ming🚀 Timeout Optimizations: - Reduced base timeout from 120s to 60s - Reduced scaling factor from +10s to +5s per 1000 chars - Reduced maximum cap from 300s to 120s - Updated all tests to reflect new timeout values ✅ Benefits: - Faster failure detection for stuck requests - More reasonable timeout values for typical use cases - Still provides dynamic scaling for large text - Prevents extremely long waits (100+ seconds) 📊 New Timeout Formula: - Base: 60 seconds - Scaling: +5 seconds per 1000 characters over 1000 - Maximum: 120 seconds (2 minutes) - Example: 10,000 chars = 60 + (9 * 5) = 105 seconds This addresses the 100+ second timeout issue while maintaining the dynamic timeout system for large text processing.2c658eb
-
Fix Ollama model configuration
ming- Updated default model from llama3.1:8b to llama3.2:latest - Resolves 404 error when calling Ollama API - Summarization endpoint now works correctlyd6249b3 -
Fix Pydantic import: use pydantic-settings for BaseSettings
ming- Updated import to use pydantic_settings.BaseSettings instead of pydantic.BaseSettings - Resolves PydanticImportError for Pydantic v2+ compatibility - Server now starts successfully without import errors10a33a6
-
test: complete comprehensive test suite with 95% coverage
ming- Enhanced configuration tests with validation scenarios - Added comprehensive error handling tests - Added middleware functionality tests - Added logging setup tests - Fixed all test failures and improved coverage - Achieved 95% test coverage across all modules - All 48 tests passing successfully Coverage breakdown: - Core modules: 100% coverage - API endpoints: 100% coverage - Services: 92% coverage - Middleware: 92% coverage - Main app: 83% coverage2aa2b79 -
feat(middleware): add request ID logging and error handling
mingfa85955 -
chore: initialize FastAPI backend project structure and testing setup
ming9024ad9