Optimize connection pool and concurrency to prevent target server crashes

Increase max_workers to 200 for extreme concurrency

Increase max_workers to 50 for faster parallel crawling

Increase HTTP timeouts to prevent fallback to slow Playwright

Improve URL normalization logic and add API docs

Fix silent failure on blocked crawls

Make playwright-stealth optional and fix import errors

Migrate API to FastAPI with async job queue and progress tracking

Update Dockerfile and remove logs/cache from tracking

Fix Import Error: Explicitly install playwright-stealth in Dockerfile and log actual import errors

Enable Verbose Logging: Track all WAF interactions, stealth steps, and navigation events for debugging

Update Stealth: Use playwright-stealth library and Frame-based checkbox solver for improved WAF evasion

Update Stealth Strategy: Hardcode Chrome UA, mock plugins/permissions/languages, and randomize mouse movements

Update Crawler: Add stealth args and challenge detection for Cloudflare

Enhance API stability: Add KeepAlive, increase timeout, and optimize crawler threads

Updated /api/status to accept 'domain' parameter for specific latency checks; Default checks dev.autoalt.ai

Added /api/status endpoint with dynamic region and latency monitoring

Merged short alt detection back into main 'poor quality' report and removed separate 'short_alt' counts as requested

Refined report structure: Ensure mutually exclusive separation of 'short_alt' vs 'poor_quality' (Generic/Filename) to correct double counting

Refined report structure: Separated legacy 'short_alt' (length only) from 'poor_quality' (all issues) metrics

Refined generic text detection: added partial matches for short phrases (e.g. 'company logo') while allowing branded logos

Refined alt text analysis: added slug detection (e.g. product-12345) and reordered prioritization

Enhanced alt text analysis: added detection for short, sparse, generic, and filename-like alt text

Add short alt text detection (<15 chars) to reports

Fix URL normalization: strip trailing slash from start_url

Fix Amazon crawling: Add Start URL Retry logic and more block keywords

Fix crawl evasion: Add WAF block detection to Tier 1 fetch

Fix syntax errors: Restore init, fix indentation, remove invalid arg

Ultra-fast crawler update: Caching, SSL disable, and Thread tuning

Optimize crawler speed: BFS + Reduced delays

Fix syntax error in crawler

Optimize memory usage to prevent crashes

Fix crawling for eminentcoders.com

Initial commit for deployment