bharatgraph / scrapers

Commit History

  1. fix: centralize + desktop-mode-adapt the logs directory (P-1)⁠

    6d12c24

  1. fix(ascii): replace all non-ASCII chars in Python source files⁠

    fa87992

  1. fix(BUG-C5): GemScraper never returns fake sample data; returns None/[] with explicit warning on failure

    0c26e07

  1. fix(M-09): guard os.makedirs() in save_json() against empty dirname -- filenames without path component no longer crash

    0e43bbf

  1. fix(NEW-A3 part1): replace em-dashes/smart-quotes in 56+ Python files -- CI requires pure ASCII source

    8a0fba4
  2. fix(BUG-29): add haryana_2024, jk_2024, odisha_2024, ap_2024, sikkim_2024, arunachal_2024, bihar_2025 election URLs

    67cb2b8

  1. fix(encoding): strip non-ASCII from 4 remaining files⁠

    23cf286
  2. fix(scrapers+pipeline): H-03+L-02+H-12 -- MyNeta elections and git bloat⁠

    1ffe057

  1. fix(scrapers): BUG-29 -- logs dir may not exist in Docker environments⁠

    b62bf48

  1. feat(scrapers): complete — 8 new Indian sources, total 21 scrapers⁠

    6bcb5d3

  1. fix( ): add 5 scrapers, fix README links⁠

    8af87f7
  2. fix(scrapers): ICIJ HTML scraping instead of JSON API, restore .env.example⁠

    8958e86

  1. fix(pib+gitignore): PIB HTML scraper + clean gitignore⁠

    8e77b70

  1. fix(scrapers): add missing import os to mca_scraper.py⁠

    421aac9
  2. fix(security): remove real API key from repo, fix gem_scraper bugs⁠

    7590455

  1. fix(scrapers): update PIB RSS feed URLs, add empty feed handling⁠

    0c2b3a5

  1. fix(scrapers): correct GeM stats endpoint, use gem.gov.in/statistics⁠

    a4f3e18
  2. fix(scrapers): load DataGov API key from .env, remove hardcoded key⁠

    4baa251
  3. docs(scrapers): update README with all 6 scrapers + known issues⁠

    2a9acf6
  4. feat(scrapers): add GeM government contracts scraper⁠

    0e9b6d0
  5. feat(scrapers): add CAG audit report scraper⁠

    563cd5e
  6. fix(scrapers): update PIB RSS URLs, add .env API key setup⁠

    f0cecec
  7. docs(scrapers): add README documenting all scrapers and usage⁠

    f452d21
  8. feat(scrapers): add MCA company registry scraper⁠

    50f9e90
  9. feat(scrapers): add MyNeta/ADR candidate affidavit scraper⁠

    67940af
  10. fix: add __init__.py to all packages, create venv setup⁠

    d76057f
  11. feat(scrapers): add PIB press release scraper⁠

    d0cb0ea
  12. feat(scrapers): add DataGov scraper for govt open datasets⁠

    a2a0f68

  1. feat: add requirements.txt and base_scraper.py - Day 2

    1c02240