Spaces:
Build error
Build error
Commit History
-
Wire get_graph_depth() into 5 internal investigator queries (R-5, partial)
What: 5 hardcoded [*1..2] variable-length Cypher traversals -- 3 in ai/deep_investigator.py (_layer_3_patterns x2, _layer_4_timeline x1), 1 in ai/investigators/historical_investigator.py, 1 in api/routes/risk.py's ghost-company detection -- now interpolate int(get_graph_depth()) instead of a hardcoded 2, using the same f-string-into-Cypher-variable-length-path pattern ai/graph_analytics.py already established elsewhere in this codebase (Cypher does not accept *min..max as a query parameter -- this project's own BUG-1 fix already established that literal string interpolation is required here, not ). Why these 5 specifically, and why this is genuinely safe (unlike investigation_layers, deliberately NOT touched in this commit): every one of these 5 sites was hardcoded at exactly *1..2, which is also PROFILE['low']'s own graph_depth value. That means a low-tier/desktop install sees ZERO behavioral change from this commit -- it already was getting depth-2 traversal today, for everyone, reg…f064888
-
Extend provenance badges to search results (D8)
What: search results now carry and display source_type, same as the profile page from the previous commit. Three pieces: - api/models.py: SearchResult gets a source_type field (Optional, same convention as risk_level). - api/routes/search.py: all 22 LABEL_QUERIES RETURN clauses, the fulltext-index query, and both result-construction code paths (fulltext success + label-scan fallback) now select/pass source_type through. Applied as one global, count-asserted substitution across the 22 near-identical query strings rather than 22 separate manual edits -- same reasoning as the desktop-mode chdir fix: many genuinely uniform call sites are better fixed once, centrally, when the change to each is identical. - frontend/js/components.js: ResultCard renders the badge by calling Components.ProvenanceBadge(...).outerHTML rather than duplicating its label map inline -- one source of truth for the 4 SOURCE_TYPES labels, and it inherits that component's textContent-based (not in…c91bada -
Add provenance badges to the entity profile view (D8)
What: a colored badge next to the entity-type badge on every profile page, showing graph/schema.py's SOURCE_TYPES value (OFFICIAL today; UNOFFICIAL/COMMUNITY/USER_CREATED are provenance for D7's future user-import paths). Mirrors the existing badge/risk-badge pattern exactly -- same component style (Components.ProvenanceBadge, container appended post-render), same CSS structure, reuses existing color tokens rather than inventing a new palette (OFFICIAL reuses badge--company's green, COMMUNITY reuses badge--contract's blue, USER_CREATED reuses risk-badge--MODERATE's amber, UNOFFICIAL reuses risk-badge--NONE's neutral gray). One commit, not several, because the two halves are inert alone: the backend field with no UI renders nothing, and the UI with no field has nothing to render. Real gap found and fixed, not assumed: source_type was already being read off every node (p = dict(pol['p'])) but silently dropped by the Politician and Company profile handlers' hand-curated overview dicts -…66bf6f4 -
Migrate admin-secret generation to try the OS keychain first (D6)
What: _generate_and_persist_admin_secret() in api/routes/admin.py now calls config.credentials.set_credential("ADMIN_SECRET", ...) first, falling back to config.settings.set_local_setting() (config.json) only if the keychain write fails or isn't available. _require_admin() needed zero changes -- it already reads via os.getenv("ADMIN_SECRET", ""), which transparently picks up whatever config.settings injected regardless of source. Subtlety specifically verified rather than assumed: set_local_setting() already injects its value into os.environ itself, but set_credential() (the keychain path) does not -- that's load_credentials_into_environ()'s job, which only runs once at process startup. Without an explicit os.environ["ADMIN_SECRET"] = new_secret on the keychain-success branch, a secret generated via the keychain would be correct on disk/in-keychain but invisible to the rest of that same running process until next restart. Added explicitly; confirmed by test. An existing install with …188acc4
-
fix: desktop-mode-adapt dossier output; unify 3 hardcoded copies of its path (P-1)
Root cause: the dossier/export flow had this same base path hardcoded independently in THREE places, all of which need to agree for the export endpoint to work at all: 1. ai/dossier_generator.py's generate() computed its own <project_root>/data as a path-traversal allowlist (a pre-existing H-07/CodeQL security fix) and joined output_dir onto <project_root> directly. 2. api/routes/export.py passed the literal string "data/processed" as that function's output_dir argument. 3. api/routes/export.py ALSO independently recomputed <project_root>/data/processed as its OWN separate, independent path-traversal allowlist (a second H-07/CodeQL fix) to validate the path generate() handed back before serving the file. New finding while implementing this: (1) and (3) were ALREADY subtly inconsistent with each other before any desktop-mode change -- (1) allowed anything under <project_root>/data/ (e.g. data/samples), while (3) allowed only <project_root>/data/…7650d33
-
feat(D1): local config-file loading + desktop-mode admin-auth hardening
Phase D1 scope (PHASE_ROADMAP.md): local-config-file loading added alongside .env, and the DEBUG_MODE admin-auth bypass replaced with a desktop-appropriate default per SECURITY.md's Desktop Application Threat Model. config/settings.py: added get_app_data_dir() (OS-appropriate config dir), _load_local_config()/set_local_setting() (JSON file at <app-data>/config.json). Mirrors exactly how load_dotenv() already works -- injects values into os.environ so every existing os.getenv(...) call site in the codebase picks them up transparently, without needing to touch api/dependencies.py or any scraper. Precedence: real env var (including .env) always wins; local config only fills gaps. A missing/malformed local config file degrades to a warning, never a crash. api/routes/admin.py: _require_admin() now branches on BHARATGRAPH_DESKTOP_MODE. When unset (every current deployment), web behavior is byte-for-byte unchanged. When set, the DEBUG_MODE bypass path is skipped entirely -- a missing ADMIN_…e26a730
-
fix(timeline route): invalid Cypher syntax '{id: }' and bare LIMIT
Root cause: GET /timeline/{entity_id} -- the entry point for the EvidencePanel timeline tab -- used '{id: }' (map literal with no value) and a bare 'LIMIT' clause with no bound value. Both are invalid Cypher. This is the most severe instance of this pattern in the codebase besides graph/loader.py (next commit): unlike every other occurrence, this route has NO exception handling at all around the session.run() call, so it returns an unhandled 500 on every single request. It has not yet been hit by real end users only because the frontend has no timeline tab to call it from yet (a separate, already-documented frontend gap) -- the moment that tab is built, every load of it will 500. Evidence: grep -rn '{[a-zA-Z_]*: *}' plus a direct read of the bare 'LIMIT' clause in context; the params dict already passes limit=limit * 3, confirming the intent was always to bind it via $limit. Verified: py_compile passes.4ccfced -
fix(forensics route): invalid Cypher syntax '{id:}' in 3 query blocks
Root cause: same malformed map-literal pattern as the risk.py commit, three more instances in api/routes/forensics.py: - GET /forensics/benfords/{entity_id} (line 136): affidavit fetch - GET /forensics/shadow-draft (line 187): submission-text fetch - GET /forensics/shadow-draft (line 191): bill-text fetch All three are wrapped in try/except Exception blocks that return {"status": "error", ...} on failure -- syntactically valid-looking JSON, no operator-visible signal that the underlying query never parsed. Evidence: grep -rn '{[a-zA-Z_]*: *}', each instance opened in context before fixing (not just pattern-matched). Verified: py_compile passes; confirmed zero remaining '{id:}' instances in this file after the fix.3f1a145 -
fix(risk route): invalid Cypher syntax + wrong field names in 3 Phase-34 blocks
Root cause: three session.run() calls in GET /risk/{entity_id} used the malformed map literal '{id:}' (no value after the colon), which is not valid Cypher and fails to parse on every call. All three sit inside a try/except Exception block, so the failure is silent -- the endpoint returns normal-looking JSON with the corresponding risk factor simply absent, with no operator-visible signal anything is wrong. Evidence: grep -rn '{[a-zA-Z_]*: *}' across the tree, each instance read in context (not just pattern-matched) before fixing: - Benford's Law block (line 176): '(p {id:})' -> '(p {id: $id})' - Ghost-company block (line 208): '(n {id:})' -> '(n {id: $id})' - Shadow-director block (line 243): '(n {id:})' -> '(n {id: $id})' Second, independent bug fixed in the same commit (same block, same root symptom -- a risk factor that can never fire): the ghost-company query fetched co.employee_count AS emp and co.registered_capital_crore AS cap, but GhostCompanyDetector.score_company() r…146a572
-
fix(search): remove dead 'pressrelease' LABEL_QUERIES branch
Every LABEL_QUERIES entry returns columns id/name/state/party except 'pressrelease', which returned entity_id/entity_type/name/source instead. The shared result loop does: if not r.get("id") or not r.get("name"): continue so this branch's rows never had a truthy 'id' key and were silently dropped, 100% of the time. 'pib' already does this job correctly with matching column names. The frontend never sent type=pressrelease (it's unreachable through the UI), so this has no observable effect on real traffic. Note for anyone hitting the API directly: filter_type lookup tries an exact key match first, so an explicit ?type=pressrelease request will now fall through the prefix-match logic, find nothing, and fall back to searching across ALL types -- rather than silently returning zero rows as it did before. That's a behavior change worth knowing about, even though it's strictly better than the old silent-drop.7fd981e
-
feat(phase-34): add api/routes/case_memory.py with 3 endpoints
GET /case-memory/stats -- case count, pattern breakdown, false positive rates GET /case-memory/similar -- find past cases with matching finding types POST /case-memory/false-positive -- record false positives to improve accuracy CaseStore was fully implemented in ai/case_memory/case_store.py. It was imported by zero routes and had no feedback mechanism. The false-positive endpoint closes the learning loop: users flag bad findings, WeightOptimizer picks them up and reduces the offending indicator weight on next optimize() run.538036c -
feat(phase-34): add api/routes/self_learning.py with 4 endpoints
GET /self-learning/patterns -- PatternLearner discovers new investigation motifs GET /self-learning/weights -- WeightOptimizer current investigator weights GET /self-learning/audit -- SelfAudit scraper health check (30s timeout) GET /self-learning/schema -- SchemaLearner pending fields not in schema.py All 4 modules in ai/self_learning/ were fully implemented. None were reachable via any API endpoint. Now wired.5fdaa50 -
feat(phase-34): add api/routes/forensics.py with 5 endpoints
GET /forensics/circular-ownership -- detect shell company ownership rings GET /forensics/ghost-companies -- companies with contracts but no activity GET /forensics/shadow-directors -- directors of 10+ companies / shared addresses GET /forensics/benfords/{entity_id} -- Benford Law analysis on asset declarations GET /forensics/shadow-draft -- policy text similarity to lobbying documents All 5 endpoints use pre-built detection engines in ai/ that were fully implemented but imported by zero routes. Each returns gracefully if the underlying data (affidavits, company filings) is not yet ingested.48337b7 -
feat(phase-34): wire BenfordsAnalyzer + GhostCompanyDetector + ShadowDirectorDetector into risk scorer
Three fully-built detection engines were importing nothing and calling no API routes. Now they contribute new RiskFactor entries: benfords_law_anomaly: Runs Benford Law chi-squared test on affidavit asset values. chi2 > 15.5 (p<0.05) flags fabricated or heavily rounded figures. Weight 0.20, max contribution 20 points. ghost_company_association: Scores linked companies for ghost indicators (no employees, minimal capital, high contract volume). Each ghost company adds 12 points. Weight 0.24, max contribution 24 points. high_directorship_count: Flags entities directing 10+ companies (shadow director pattern). Weight 0.15, max contribution 15 points. Also wired the explainer module for structured natural-language output.4ac80cb
-
feat(phase-33): add api/routes/timeline.py with 2 endpoints
GET /timeline/{entity_id} -- all time-stamped events for an entity sorted newest-first (feeds EvidencePanel) GET /timeline/{entity_id}/by-year -- same events bucketed by year The EvidencePanel timeline tab calls /profile/{entity_id} and receives a ProfileResponse with no events field -- the timeline always shows empty. This dedicated /timeline endpoint queries all connected nodes that have any date field (order_date, date, filing_date, scraped_at, year) and returns structured events with category, title, amount, source. Categories: contract, audit, enforcement, financial, legal, political, electoral, regulatory, vigilance, corporate, other.4435778 -
feat(phase-33): add 4 graph analytics endpoints to api/routes/graph.py
GET /graph/analytics/{entity_id} -- betweenness + pagerank + communities for an entity local sub-graph GET /graph/centrality/betweenness -- top 20 brokers in full graph GET /graph/centrality/pagerank -- top 20 influential entities in full graph GET /graph/communities -- detect institutional clusters All 4 endpoints use the pre-built GraphAnalytics class (ai/graph_analytics.py) which uses NetworkX under the hood. Graceful ImportError if networkx not installed -- returns {status: error} instead of 500.5ac9ba3
-
feat(phase-32): add api/routes/resolve.py with 4 endpoints
GET /resolve?name=...&kind=person -- normalise + canonicalise a single name GET /resolve/alias?name=... -- O(1) lookup in pre-built alias graph POST /resolve/batch -- resolve list of records (max 500), merge duplicates GET /resolve/stats -- alias graph statistics The /resolve/alias and /resolve/stats endpoints share the _alias_graph singleton loaded at startup from data/processed/alias_graph.json. The /resolve/batch endpoint creates a temporary resolver for the request at the caller's threshold.d5dc03e
-
fix(LOGIC-1): remove 2 duplicate pressrelease keys in LABEL_QUERIES; fix LIMIT $lim -> $limit to match query runner kwarg
b348d13 -
fix(SEC-6): reject empty-string ADMIN_SECRET -- empty string was falsy and bypassed authentication check
4040e0f -
fix(CodeQL#16-17): use validated real_out path for FileResponse and media_type check in export.py
673a953 -
fix(CodeQL#23): log exception type only in investigation.py -- full exception message may expose Neo4j URI
7a6c1ee
-
fix(M-08): add pressrelease alias in LABEL_QUERIES so frontend filter button works
9b088fe
-
fix(M-08): add 'pressrelease' alias key in LABEL_QUERIES -- frontend filter button now correctly filters to PressRelease nodes
2d87a38 -
fix(M-03,M-06,M-08): add mobile CSS overrides for inline grid layouts; add NONE risk level for score=0; clamp risk score to 0-100
b0e0450 -
fix(CodeQL#23): log exception type only, not full message in investigation.py -- prevents Neo4j URI/credentials leaking into logs
370bb36 -
fix(H-08): replace substring audit name match with relationship-based + word-boundary match -- eliminates false positives for common words like 'India', 'National'
48f3ef2 -
fix(H-07,CodeQL#12-17): use file-anchored absolute paths for path traversal guards in dossier_generator and export -- relative realpath() was CWD-dependent
6de77e6 -
fix(H-04): enforce ADMIN_SECRET even when DEBUG_MODE=true -- prevents production DB wipe via accidental debug flag
d39cad8
-
fix(BUG-17): wire blockchain.audit_chain.store_daily_root() to dossier export -- README SHA-256 claim now accurate
06f5a34 -
fix(NEW-A3 part1): replace em-dashes/smart-quotes in 56+ Python files -- CI requires pure ASCII source
8a0fba4 -
fix(BUG-15): remove duplicate entity_type param from /search; keep only entity_type_param with alias='type'
e13cf2a -
fix(BUG-6,NEW-A6): add X-Admin-Secret auth on /admin/seed and /admin/pipeline; add threading.Lock for pipeline race condition
e97313c -
fix(BUG-1): use str(depth_safe) literal in Cypher path pattern -- Neo4j rejects parameters in *min..max
391b602
-
fix(encoding): strip non-ASCII from 4 remaining files
investigation.py, myneta_scraper.py, components.js, and settings.py contained UTF-8 em dashes and curly quotes in comments which cause UnicodeDecodeError on GitHub Actions runners.23cf286 -
fix(api+graph+pipeline): M-04+M-05+M-08+M-09 -- misc medium fixes
M-04: /debug/env endpoint was publicly accessible with no auth gate. It revealed NEO4J_URI prefix and credential configuration status. Fix: added same DEBUG_MODE env-var gate as /debug/neo4j. M-05: Wikidata enrichment used exact name match (=) -- any difference in spacing, middle name, honorific, or transliteration caused a silent miss. With Indian names this is extremely common. Fix: changed to CONTAINS match in both directions for fuzzy matching. M-08: --parallel flag had action='store_true', default=True making it always True whether passed or not. The flag had zero effect. Confusing for operators who expected --parallel to mean something. Fix: removed --parallel; --sequential now inverts the default. M-09: startNode(rel).id and endNode(rel).id in Cypher can return null if a node has no 'id' property. This created edges with null source/target silently corrupting graph output. Fix: added isinstance str check to filter null src/tgt before appending.5d9b8a9 -
fix(api+ai): M-01+M-02+M-03 -- version consistency and duplicate assignments
M-01: Three different version strings for the same release: FastAPI app = 0.31.0, root() endpoint = 0.30.0, settings.py = 0.30.0 Fix: settings.py VERSION -> 0.31.0, main.py root() -> 0.31.0. M-02: entity_label = 'Unknown' was written on two consecutive lines in deep_investigator.py (copy-paste residue from the BUG-1+7 fix). Fix: kept one assignment with a clean comment. M-03: _edge_seen = set() was declared twice on consecutive lines in graph.py (same copy-paste residue). Fix: kept one declaration. Also fixed graph.py GraphResponse(depth=depth) -> depth=depth_safe so the response reflects the actual queried depth, not the raw user input.01cf12a -
fix(api+pipeline): C-05+C-06 -- unsearchable nodes and hardcoded ICIJ names
C-05: DataGovDocument and CrimeReport nodes were loaded into Neo4j by the graph loader but had no LABEL_QUERIES entry, SOURCE_MAP entry, or fulltext index definition. They were permanently invisible to search. Fix: added both types to LABEL_QUERIES and SOURCE_MAP. C-06: ICIJ scraper only searched 5 hardcoded names (Adani, Reliance, Tata, Birla, Ambani). OpenSanctions searched 5 names. Every other politician, company, or sanctioned person was completely missed. Fix: expanded both lists to include known high-risk Indian entities.2f11b48 -
fix(api): C-04 -- /investigate was using DeepInvestigator not MultiInvestigator
The /investigate/{id} endpoint used DeepInvestigator (6-layer graph traversal) but the homepage claimed '15 specialist AI investigators run in parallel'. MultiInvestigator -- the one that actually runs all 15 specialists (financial, political, corporate, judicial, procurement, network, asset, international, media, historical, public_interest, doubt, affidavit, benami, math) -- was only instantiated inside the PDF export route and never triggered by normal usage. Fix: /investigate now uses MultiInvestigator by default. Added mode=deep param for backward compat with existing integrations.2a82690 -
fix(api): C-02+H-04 -- risk None crash and missing risk signals
C-02: Neo4j SUM() returns None (Python None) when no matching contract rows exist. 'contract_rows' was always truthy (Cypher RETURN always produces a row), so the 'if contract_rows else 0' guard never triggered. f-string {total_crore:.1f} then crashed with TypeError. Fix: added 'or 0' guard on the SUM value. H-04: risk scoring only checked 4 signals and never queried ElectoralBond, SanctionedEntity, or EnforcementAction nodes even though all three types exist in the graph. High-risk patterns were silently missed. Fix: added 3 new risk factors: electoral_bond_donor (weight 0.20), sanctions_exposure (weight 0.25), enforcement_action_subject (0.25).3c1144f
-
fix(encoding): strip non-ASCII chars from all modified files
All files contained UTF-8 punctuation (em dashes U+2014, arrows U+2192, curly quotes) in comments which cause UnicodeDecodeError on GitHub Actions runners that read files without explicit encoding= argument. Replaced all non-ASCII punctuation with ASCII equivalents: U+2014 em dash -> -- U+2192 arrow -> -> curly quotes -> straight quotes747e613 -
fix(api): BUG-17 -- fragile rstrip(s) filter normalisation in search
filter_type.rstrip('s') stripped ALL trailing s characters, so 'press' -> 'pre' and 'congresss' -> 'congres' (wrong matches). The fallback silently returned all types when no match was found. Fix: priority order: 1. Exact match in LABEL_QUERIES keys 2. Prefix match (covers 'audit' -> 'audit' key) 3. Strip single trailing s as last resort 4. Fallback to all types if still no match77b5a18 -
fix(api): BUG-10 -- deduplicate graph edges before sending to frontend
With depth=2, Neo4j returns many paths sharing intermediate edges. Each shared edge appeared in multiple row[rels] arrays and was appended to a plain list unconditionally. D3 force simulation received duplicate links creating visual double/triple edges and distorting the force calculation (artificially stronger pull). Fix: track seen (src, tgt, type) tuples in _edge_seen set and skip duplicates.719142b -
fix(api): BUG-10 -- deduplicate graph edges before sending to frontend
With depth=2, Neo4j returns many paths sharing intermediate edges. Each shared edge appeared in multiple row[rels] arrays and was appended to a plain list unconditionally. D3 force simulation received duplicate links creating visual double/triple edges and distorting the force calculation (artificially stronger pull). Fix: track seen (src, tgt, type) tuples in _edge_seen set and skip duplicates.7beb5e9
-
feat(api): add GET /runtime endpoint
Returns hardware detection results and active profile settings. Useful for diagnosing performance on different deployment targets.9cf5b3f
-
fix
21e8167 -
fix(admin): BUG-21/22/30 — driver reuse, bond key, 7 missing scrapers
BUG-21: /admin/seed was calling GraphLoader() with no driver, opening a second Neo4j connection. Now passes the existing driver via GraphLoader(driver=driver). BUG-22: SCRAPER_TO_LOADER key was 'electoral_bonds' (wrong) — the scraper outputs key 'electoral_bond'. Corrected. BUG-30: 7 scrapers were running but their loader methods were never registered in SCRAPER_TO_LOADER so all records were silently discarded after the pipeline ran. Added: icij->load_icij_entities, opensanctions->load_sanctioned_entities, njdg->load_court_cases, lgd->load_local_bodies, ncrb->load_crime_reports, wikidata->load_wikidata_enrichments, datagov->load_datagov_documents.d8672be -
fix(search): BUG-1/4/10/18 — type() crash, missing types, source map
BUG-1 (CRITICAL): query param named 'type' shadowed Python's built-in type(). type(ft_err).__name__ crashed with TypeError on every fulltext index miss. Fixed by renaming to entity_type_param with alias='type'. Also changed type(ft_err).__name__ -> ft_err.__class__.__name__. BUG-4: added 6 missing node types to LABEL_QUERIES: ParliamentQuestion, VigilanceCircular, ICIJEntity, SanctionedEntity, CourtCase, LocalBody. BUG-10: expanded SOURCE_MAP from 8 to 20 node types so every search result gets a real source URL instead of '#'. BUG-18: LABEL_QUERIES key renamed 'pressrelease' -> 'pib' to match the scraper output key and fix source attribution.fabf548
-
fix(BUG-03/22): removed duplicate fallback block; added sources field to ProfileResponse
99375ed -
fix(BUG-08): DossierGenerator no longer caches stale Neo4j driver
b7bdaca -
fix(BUG-04): all investigation routes wrapped in try/except — no more raw 500s
6c74dba