Spaces:
Build error
Build error
Commit History
-
Add python -m config.credentials CLI (D6, ahead of D5's UI)
What: a thin argparse CLI appended to config/credentials.py -- 'list', 'get [--reveal]', 'set [VALUE]' (prompts hidden if omitted), 'delete' -- over the 6 managed keys. Values are masked to their last 4 chars by default. Calls the exact same get_credential()/set_credential() functions a future D5 setup-wizard UI would call, so nothing here needs to change when that UI exists. Why now rather than waiting for D5: this is the only way to test the 5 keys with no other write path (everything but ADMIN_SECRET) against a REAL OS keychain -- Windows Credential Manager specifically -- since no sandbox in this engagement has ever had one available to test against directly. Small, self-contained, and doesn't conflict with or duplicate D5's eventual work. Verified: python -m py_compile passes; ran 'python -m config.credentials list/get/set/delete' as a real subprocess (not just calling _cli() in-process) to confirm argparse's prog name and --help both render correctly. Behaviorally exercised eve…10d643d -
Wire load_credentials_into_environ() into config/settings.py (D6)
What: after the existing local-config (config.json) injection loop, and before NEO4J_URI/NEO4J_USER/etc are read as module-level constants, call config.credentials.load_credentials_into_environ() -- but ONLY when BHARATGRAPH_DESKTOP_MODE=true. Why gated to desktop mode (a correction from the original design sketch, caught during this delivery's own verification, not assumed): a web deployment gets its credentials from GitHub Secrets/real env vars, never a local OS keychain, so there's no reason to import keyring there at all. Gating the whole block -- not just making load_credentials_into_environ() a no-op -- means keyring is never even imported in web mode, confirmed via a direct sys.modules check. Verified: python -m py_compile passes. Imported config.settings twice in the same isolated venv used for the previous commit: with BHARATGRAPH_DESKTOP_MODE unset, confirmed 'keyring' never appears in sys.modules and NEO4J_URI resolves exactly as before this change; with BHARATGRAPH_DESKTO…e205d9d -
Add config/credentials.py -- OS-keychain credential storage (D6)
What: new module with get_credential(key), set_credential(key, value), and load_credentials_into_environ(). Extends config.settings's existing 2-level precedence (env var > local config.json) with a third, lowest- precedence source: the OS keychain (Windows Credential Manager / macOS Keychain / Linux Secret Service), via the keyring package. A closed, explicit list of 6 managed keys (ADMIN_SECRET, NEO4J_URI, NEO4J_USER, NEO4J_PASSWORD, DATAGOV_API_KEY, OPENSANCTIONS_API_KEY) -- not an arbitrary-key store, since the OS keychain is a shared per-user resource. Not yet wired into anything (next two commits do that) -- this commit is the module on its own, reviewable in isolation. Verified: python -m py_compile passes. In an isolated venv with keyring installed: (1) with no real OS keychain backend available, confirmed get_credential/set_credential/load_credentials_into_environ all degrade to no-ops with no exceptions -- import and keyring.get_keyring() both succeed, and the failure only …89d90ae -
Fix duplicate get_dossier_output_dir() definition in config/settings.py
Root cause: an earlier delivery script was accidentally re-run after a 'git checkout -b' failed with branch-already-exists (bash does not stop on a non-&&-chained command failure). That script's edit anchor matched only the function's own unchanged body, not anything after it, so the second run found the same anchor again and inserted a second, verbatim copy of the function immediately after the first. Trigger condition: any second invocation of that specific delivery step against a repo where it had already succeeded once. Blast radius: definition-order only. Python's later def silently shadowed the earlier one; both bodies were byte-identical, so every call site kept calling the same function with the same behavior. No functional bug reached any user -- this is a code-hygiene / future- drift-risk fix. Verified (in sandbox before delivery): python -m py_compile succeeds; AST-parsed the module and confirmed exactly one top-level FunctionDef named get_dossier_output_dir (was 2, now 1…c43cb01
-
fix: chdir into app-data dir at desktop startup, fixing 30+ writes at once (P-1)
Root cause: grepped every bare (non-__file__-anchored) CWD-relative path in the codebase. Found DATA_RAW_PATH/DATA_PROCESSED_PATH/ DATA_SAMPLES_PATH in config/settings.py were dead code exactly like LOGS_PATH was before its own fix -- zero real consumers reference them. Instead, 30+ real call sites hardcode their own copy of the same path directly: all 19 scrapers' data/samples/*.json sample- fallback output, processing/pipeline.py (data/processed + data/samples), ai/multi_investigator.py, ai/risk_scorer.py, ai/self_learning/schema_learner.py, processing/alias_graph.py, and graph/loader.py's own glob() read of pipeline output. Trigger condition: on a real Windows install, a double-clicked app's CWD is unpredictable, and the install directory itself is typically read-only for a non-admin user. Every one of these 30+ writes would fail outright under that condition -- not just lose regeneratable cache data, but error out entirely (scrapers couldn't run at all). Why this fix looks differ…45cee30
-
fix: desktop-mode-adapt dossier output; unify 3 hardcoded copies of its path (P-1)
Root cause: the dossier/export flow had this same base path hardcoded independently in THREE places, all of which need to agree for the export endpoint to work at all: 1. ai/dossier_generator.py's generate() computed its own <project_root>/data as a path-traversal allowlist (a pre-existing H-07/CodeQL security fix) and joined output_dir onto <project_root> directly. 2. api/routes/export.py passed the literal string "data/processed" as that function's output_dir argument. 3. api/routes/export.py ALSO independently recomputed <project_root>/data/processed as its OWN separate, independent path-traversal allowlist (a second H-07/CodeQL fix) to validate the path generate() handed back before serving the file. New finding while implementing this: (1) and (3) were ALREADY subtly inconsistent with each other before any desktop-mode change -- (1) allowed anything under <project_root>/data/ (e.g. data/samples), while (3) allowed only <project_root>/data/…7650d33 -
fix: centralize + desktop-mode-adapt the logs directory (P-1)
New finding while implementing this: config/settings.py's LOGS_PATH constant was dead code -- grepped every consumer, zero hits anywhere outside its own definition. The 3 real log-writing call sites each independently hardcoded or recomputed their own path instead: - scrapers/base_scraper.py: a bare CWD-relative "logs" string - api/middleware/audit_logger.py: __file__-relative, 3 dirname() calls up to project root - blockchain/audit_chain.py: __file__-relative, 2 dirname() calls up to project root (different file depth, same target) All 3 resolve to the identical directory TODAY only because Docker's fixed WORKDIR makes CWD == project root -- a coincidence, not a guarantee. A desktop build has no such guarantee: a double-clicked app's CWD is unpredictable, so the CWD-relative scraper logs and the __file__-relative audit logs could silently end up in two different, unrelated directories the moment this ships as a desktop app. Fix: LOGS_PATH is now desktop-mode-aware (same…6d12c24
-
feat(D1): local config-file loading + desktop-mode admin-auth hardening
Phase D1 scope (PHASE_ROADMAP.md): local-config-file loading added alongside .env, and the DEBUG_MODE admin-auth bypass replaced with a desktop-appropriate default per SECURITY.md's Desktop Application Threat Model. config/settings.py: added get_app_data_dir() (OS-appropriate config dir), _load_local_config()/set_local_setting() (JSON file at <app-data>/config.json). Mirrors exactly how load_dotenv() already works -- injects values into os.environ so every existing os.getenv(...) call site in the codebase picks them up transparently, without needing to touch api/dependencies.py or any scraper. Precedence: real env var (including .env) always wins; local config only fills gaps. A missing/malformed local config file degrades to a warning, never a crash. api/routes/admin.py: _require_admin() now branches on BHARATGRAPH_DESKTOP_MODE. When unset (every current deployment), web behavior is byte-for-byte unchanged. When set, the DEBUG_MODE bypass path is skipped entirely -- a missing ADMIN_…e26a730
-
fix(F-10): give scraper-pipeline concurrency its own I/O-bound resource dimension
Decision (owner-confirmed): scraper concurrency should NOT follow the same low/medium/high CPU/RAM tiers as the AI investigators -- the investigator pool is CPU-bound, the scraper pipeline is I/O-bound. Added RuntimeProfile.scraper_workers: max(2, min(10, cpu_count * 2)), overridable via BHARATGRAPH_SCRAPER_WORKERS=N. Surfaced in GET /runtime. processing/pipeline.py now reads PROFILE.scraper_workers instead of the hardcoded max_workers=6. Verified: py_compile passes on both files. Formula sanity-checked across a realistic CPU range. Spied on the real ThreadPoolExecutor constructor and confirmed the env-var override reaches it end-to-end.a2f923e
-
chore: bump version to 0.36.0 -- Phase 35 complete
Phase 35 -- Entity Intelligence Dashboard Frontend entity view: 3 tabs -> 6 tabs Before: Findings | Relationship Graph | Evidence Locker After: Findings | Relationship Graph | Timeline | Forensics | Network Analysis | Evidence Locker Frontend entity view: 3 API calls -> 6 API calls Added: Api.timeline(), Api.graphAnalytics(), Api.investigate() New page: #/forensics Network-wide ghost company + shadow director + circular ownership dashboard Nav link added to desktop and mobile navbar Home stats: 4 counters -> 6 counters Added: Audit Reports count + total Relationship count api.js: Api.investigate() method added Tests: 113 total (15 new, was 98)8549189
-
chore: bump version to 0.35.0 -- Phase 34 complete
Phase 34 -- Forensic Intelligence: Wire 8 Detection Engines 8 fully-built AI engines that were importing nothing now have API routes: ai/benfords_analyzer.py -> GET /forensics/benfords/{entity_id} + new RiskFactor in GET /risk/{entity_id} ai/circular_ownership.py -> GET /forensics/circular-ownership ai/ghost_company.py -> GET /forensics/ghost-companies + new RiskFactor in GET /risk/{entity_id} ai/shadow_director.py -> GET /forensics/shadow-directors + new RiskFactor in GET /risk/{entity_id} ai/shadow_draft_detector.py -> GET /forensics/shadow-draft ai/self_learning/*.py -> GET /self-learning/patterns|weights|audit|schema ai/case_memory/case_store.py-> GET /case-memory/stats|similar POST /case-memory/false-positive ai/explainer.py -> wired into /risk/{entity_id} explanation New routes: 3 (forensics, self_learning, case_memo…820c26f
-
chore: bump version to 0.34.0 -- Phase 33 complete
Phase 33 -- Graph Intelligence API New API endpoints: GET /graph/analytics/{entity_id} -- betweenness + pagerank + communities GET /graph/centrality/betweenness -- top 20 network brokers (full graph) GET /graph/centrality/pagerank -- top 20 influential entities GET /graph/communities -- institutional cluster detection GET /timeline/{entity_id} -- chronological event feed GET /timeline/{entity_id}/by-year -- events bucketed by year Frontend fix: EvidencePanel timeline tab was calling /profile/ (no events). Now calls /timeline/ which returns real chronological events. New client methods: Api.timeline(), Api.timelineByYear() Api.graphAnalytics(), Api.graphBetweenness() Api.graphPagerank(), Api.graphCommunities() Tests: 78 total (was 59). 19 new tests for GraphAnalytics + timeline helpers.1b77cde
-
chore: bump version to 0.33.0 -- Phase 32 complete
Phase 32 -- Entity Resolution v2: Canonical Identity Engine New endpoints: GET /resolve -- normalise + canonicalise a name GET /resolve/alias -- O(1) alias graph lookup POST /resolve/batch -- merge duplicates in a list (max 500 records) GET /resolve/stats -- alias graph statistics Infrastructure: AliasGraph singleton loaded at startup from data/processed/alias_graph.json Pipeline rebuilds alias graph after every resolve_dataset() run 15 new tests added (59 total, was 44)681bc37
-
fix(M-09): confirm dirname guard in save_json; bump version to 0.32.0
M-09 was already correctly fixed in a prior sprint with: dir_path = os.path.dirname(filepath) if dir_path: os.makedirs(dir_path, exist_ok=True) The sprint 8 verification reported FAIL because the check searched for the variable name '_save_dir' but the actual fix used 'dir_path'. Both are valid fixes -- no code change needed, verification logic corrected. Version: 0.31.x -> 0.32.0e492711
-
chore: bump version to 0.31.5 -- Sprint 7 complete (31 commits, 30+ bugs fixed)
fc19538
-
chore: bump version to 0.31.4 -- Sprint 5 complete (36 bugs fixed across 36 commits)
1e8b330
-
fix(NEW-A3 part2): encode Indian script string literals as Unicode escapes in 4 files -- CI ASCII requirement met without data loss
611725a -
fix(BUG-16,BUG-27,NEW-A7): remove dead OPENAI_API_KEY; add DATAGOV_API_KEY startup warning; bump VERSION to 0.31.3
3b55bb8
-
fix(encoding): strip non-ASCII from 4 remaining files
investigation.py, myneta_scraper.py, components.js, and settings.py contained UTF-8 em dashes and curly quotes in comments which cause UnicodeDecodeError on GitHub Actions runners.23cf286 -
fix(api+ai): M-01+M-02+M-03 -- version consistency and duplicate assignments
M-01: Three different version strings for the same release: FastAPI app = 0.31.0, root() endpoint = 0.30.0, settings.py = 0.30.0 Fix: settings.py VERSION -> 0.31.0, main.py root() -> 0.31.0. M-02: entity_label = 'Unknown' was written on two consecutive lines in deep_investigator.py (copy-paste residue from the BUG-1+7 fix). Fix: kept one assignment with a clean comment. M-03: _edge_seen = set() was declared twice on consecutive lines in graph.py (same copy-paste residue). Fix: kept one declaration. Also fixed graph.py GraphResponse(depth=depth) -> depth=depth_safe so the response reflects the actual queried depth, not the raw user input.01cf12a
-
feat(config): add ModelSelector for profile-aware model picking
Routes NER, embedding, and translation model choices through the runtime profile so low-profile deployments use small fast models and high-profile deployments use larger accurate models. Helper functions: get_max_workers(), get_batch_size(), get_graph_depth(), get_investigation_layers(), get_cache_ttl()eedbcec -
feat(config): add RuntimeProfile hardware detector
Detects CPU cores, RAM GB, GPU, free disk, Docker environment, and Neo4j URI location at startup. Scoring: cpu*2 + ram*2 + gpu*2 + disk + docker + db_local (max 9) Score 0-3 -> LOW (2 workers, batch 25, depth 2) Score 4-7 -> MEDIUM (4 workers, batch 100, depth 3) Score 8+ -> HIGH (8 workers, batch 500, depth 5) BHARATGRAPH_PROFILE env var overrides auto-detection. Module-level PROFILE singleton imported by all modules. Part of: #5622a359f
-
fix(config): BUG-14 — remove literal default password
NEO4J_PASSWORD was defaulting to the string 'password'. A missing secret now fails loudly (empty string triggers the 'NEO4J_PASSWORD not set' error in loader.py) instead of silently connecting with wrong credentials. Also bumped VERSION from 0.1.0 to 0.30.0 to match actual state.b2d4410
-
feat(BUG-13): expand UI_LABELS to 10 languages, 15 keys (nav/filter/button/heading)
789ce52 -
feat(i18n): expand UI labels to 10 languages, add investigation_title/no_results/tagline keys
88a8ae9
-
fix(config+admin+main): 4 bugs resolved
- config/settings.py: NEO4J_URI default was bolt://localhost:7687 causing HF Space to always try connecting to localhost (fails) Changed to empty string — env var must be set explicitly - graph/queries.py: same bolt://localhost default — same fix - api/routes/admin.py: load_companies() was called with SAMPLE_CONTRACTS instead of SAMPLE_COMPANIES — seed was loading wrong data as companies - api/main.py: load_dotenv() was never called — environment variables from .env and HF Space secrets were not loaded at API startupb70b0c6
-
feat(ai):complete — multilingual support for 22 Indian languages
Confirmed working: - config/languages.py: 22 scheduled languages, risk level translations in 9 languages, UI labels in 6 languages all verified - ai/translator.py: language detection working — Devanagari detected as hi, Tamil script detected as ta, Latin as en. Risk explanation templates confirmed in hi/ta/te/kn/ml. Switched from gated indictrans2-indic-en-1B to public Helsinki-NLP/opus-mt-en-hi (no access approval required) - ai/transliteration.py: Modi correctly transliterated to 5 scripts. Search normalization for Modi generates 8 variants including all scripts. indic-transliteration library optional — lookup table fallback working - api/routes/multilingual.py: /languages, /search/multilingual, /risk/multilingual/{id}, /translate endpoints registeredddeb034
-
fix(security): remove real API key from repo, fix gem_scraper bugs
Fixes #4 - .env.example: replaced real DATAGOV_API_KEY with placeholder text (real key was accidentally committed - security risk) - config/settings.py: removed hardcoded key fallback (now empty string) - gem_scraper.py: fixed 3 bugs: 1. Added missing 'import os' at top 2. Fixed empty __init__ - added self.api_key = os.getenv() 3. Fixed fetch_gem_stats() indentation (body was at class level) - All 7 scrapers now pass syntax check7590455
-
fix: add __init__.py to all packages, create venv setup
- Added __init__.py to scrapers, processing, graph, ai, api, blockchain, config, tests folders - Fixes ModuleNotFoundError when running scrapers - Added main.py as project entry point - Updated .gitignore to exclude venv/ and __pycache__/ - All scripts must now run from project root with: python -m scrapers.Xd76057f
-
feat: initial project structure, config, gitignore, requirements and README
971fa6c