Commit History

  1. fix: add DEFAULT_KEY fallback and resilient standard requests fetcher to prevent timeout hangs

    AI Bot
    4e6b2ca

  1. Optimize connection pool and concurrency to prevent target server crashes

    AI Bot
    9c8e3ad
  2. Increase max_workers to 200 for extreme concurrency

    AI Bot
    acde457
  3. Increase max_workers to 50 for faster parallel crawling

    AI Bot
    0b14ba9
  4. Increase HTTP timeouts to prevent fallback to slow Playwright

    AI Bot
    a7bac51

  1. Improve URL normalization logic and add API docs

    prince1604
    3176f9b
  2. Fix silent failure on blocked crawls

    prince1604
    3c07190
  3. Make playwright-stealth optional and fix import errors

    prince1604
    1f8e73e
  4. Migrate API to FastAPI with async job queue and progress tracking

    prince1604
    1b7c76e
  5. Update Dockerfile and remove logs/cache from tracking

    prince1604
    c872ed8

  1. Fix Import Error: Explicitly install playwright-stealth in Dockerfile and log actual import errors

    prince1604
    966b662
  2. Enable Verbose Logging: Track all WAF interactions, stealth steps, and navigation events for debugging

    prince1604
    059751d
  3. Update Stealth: Use playwright-stealth library and Frame-based checkbox solver for improved WAF evasion

    prince1604
    dd23225
  4. Update Stealth Strategy: Hardcode Chrome UA, mock plugins/permissions/languages, and randomize mouse movements

    prince1604
    d0c6b9b
  5. Update Crawler: Add stealth args and challenge detection for Cloudflare

    prince1604
    503011f
  6. Enhance API stability: Add KeepAlive, increase timeout, and optimize crawler threads

    prince1604
    f2e524e

  1. Updated /api/status to accept 'domain' parameter for specific latency checks; Default checks dev.autoalt.ai

    prince1604
    3da7fc2
  2. Added /api/status endpoint with dynamic region and latency monitoring

    prince1604
    67a97b3
  3. Merged short alt detection back into main 'poor quality' report and removed separate 'short_alt' counts as requested

    prince1604
    14aaae8
  4. Refined report structure: Ensure mutually exclusive separation of 'short_alt' vs 'poor_quality' (Generic/Filename) to correct double counting

    prince1604
    d7353b3
  5. Refined report structure: Separated legacy 'short_alt' (length only) from 'poor_quality' (all issues) metrics

    prince1604
    19f9d48
  6. Refined generic text detection: added partial matches for short phrases (e.g. 'company logo') while allowing branded logos

    prince1604
    b3f7149
  7. Refined alt text analysis: added slug detection (e.g. product-12345) and reordered prioritization

    prince1604
    a88e274
  8. Enhanced alt text analysis: added detection for short, sparse, generic, and filename-like alt text

    prince1604
    4ec2569

  1. Add short alt text detection (<15 chars) to reports

    prince1604
    c1d121b
  2. Fix URL normalization: strip trailing slash from start_url

    prince1604
    7e72c45
  3. Fix Amazon crawling: Add Start URL Retry logic and more block keywords

    prince1604
    94af2f2
  4. Fix crawl evasion: Add WAF block detection to Tier 1 fetch

    prince1604
    d92f150
  5. Fix syntax errors: Restore init, fix indentation, remove invalid arg

    prince1604
    747f2d2
  6. Ultra-fast crawler update: Caching, SSL disable, and Thread tuning

    prince1604
    6d07192

  1. Optimize crawler speed: BFS + Reduced delays

    prince1604
    d3e6b5e
  2. Fix syntax error in crawler

    prince1604
    e824b70
  3. Optimize memory usage to prevent crashes

    prince1604
    dc99297
  4. Fix crawling for eminentcoders.com

    prince1604
    53a61cf

  1. Initial commit for deployment

    prince1604
    b458f3d