Viney commited on
Commit
d1e793b
·
1 Parent(s): 7880373

feat: add company primer research tab

Browse files
ARCHITECTURE.md CHANGED
@@ -18,7 +18,7 @@ flowchart TB
18
  classDef io fill:#fce7f3,stroke:#db2777,color:#831843
19
  classDef gate fill:#fee2e2,stroke:#dc2626,color:#7f1d1d
20
 
21
- USER([PM / senior equity analyst]):::io --> UI["Streamlit decision stack<br/>PM Flash · Evidence & Deltas · Financials<br/>Ask Evidence — secondary"]:::io
22
  UI --> AG
23
 
24
  subgraph LG ["LangGraph · short-term state"]
@@ -99,12 +99,13 @@ flowchart TB
99
 
100
  | Surface | Primary payload |
101
  | --- | --- |
 
102
  | **PM Flash** | As-of and coverage; experimental PM read-through; thesis-confirming/challenging sourced points; experimental Swing Factor; watch items. |
103
  | **Evidence & Deltas** | Source-backed facts, verbatim evidence, deterministic detector outputs labelled heuristic, risks, commentary, and experimental AI hypotheses. |
104
  | **Financials** | Structured historical metrics, trends, guidance history, earnings history, and exports. |
105
  | **Ask Evidence** | Secondary grounded Q&A over the available evidence. |
106
 
107
- The navigation order is intentional: decision memo → audit → model depth. Open-ended Q&A is useful only after the decision question has been framed.
108
 
109
  ## Output taxonomy
110
 
 
18
  classDef io fill:#fce7f3,stroke:#db2777,color:#831843
19
  classDef gate fill:#fee2e2,stroke:#dc2626,color:#7f1d1d
20
 
21
+ USER([PM / senior equity analyst]):::io --> UI["Streamlit research stack<br/>Company Primer · PM Flash · Evidence & Deltas · Financials<br/>Ask Evidence — secondary"]:::io
22
  UI --> AG
23
 
24
  subgraph LG ["LangGraph · short-term state"]
 
99
 
100
  | Surface | Primary payload |
101
  | --- | --- |
102
+ | **Company Primer** | Stable sourced company profile plus independently refreshed price context and public news. Business model, geographic exposure, three-year trends, attention themes, associated price events, and monitoring variables remain concise and inspectable. |
103
  | **PM Flash** | As-of and coverage; experimental PM read-through; thesis-confirming/challenging sourced points; experimental Swing Factor; watch items. |
104
  | **Evidence & Deltas** | Source-backed facts, verbatim evidence, deterministic detector outputs labelled heuristic, risks, commentary, and experimental AI hypotheses. |
105
  | **Financials** | Structured historical metrics, trends, guidance history, earnings history, and exports. |
106
  | **Ask Evidence** | Secondary grounded Q&A over the available evidence. |
107
 
108
+ The navigation order is intentional: company onboarding → decision memo → grounded Q&A → audit → model depth.
109
 
110
  ## Output taxonomy
111
 
README.md CHANGED
@@ -19,6 +19,7 @@ The product is designed for the first review after an earnings release: what cha
19
 
20
  | Surface | Role in the workflow |
21
  | --- | --- |
 
22
  | **PM Flash** | One-screen decision memo: filing date and source coverage, experimental AI read-through, thesis-confirming and thesis-challenging evidence, the swing factor, and the next items to watch. |
23
  | **Evidence & Deltas** | Audit layer: source-backed claims, before/after evidence, heuristic period deltas, risks, management commentary, and experimental AI hypotheses. |
24
  | **Financials** | Historical KPI trends, guidance history, earnings history, and exportable structured data. |
@@ -52,11 +53,12 @@ Legacy briefs without a deterministic `evidence_coverage` result are shown as **
52
 
53
  ## Architecture
54
 
55
- - **Offline ingestion** — SEC EDGAR XBRL into SQLite; 10-K/10-Q sections and earnings-call transcripts into Chroma.
 
56
  - **Runtime agent** — a LangGraph tool loop retrieves structured financials, filings, transcripts, public news, and analyst data before producing a validated Pydantic brief.
57
  - **Retrieval** — vector search over-fetches candidates and reranks them with a cross-encoder.
58
  - **Post-synthesis controls** — deterministic source-reliability rules, corroboration checks, and heuristic period-delta detectors run after model synthesis.
59
- - **Decision UI** — PM Flash leads; Evidence & Deltas provides auditability; Financials provides depth; Ask Evidence stays secondary.
60
 
61
  See [WRITEUP.md](WRITEUP.md) for product and methodology detail and [ARCHITECTURE.md](ARCHITECTURE.md) for data lineage and presentation controls.
62
 
@@ -74,6 +76,10 @@ databases created before the evidence-lineage upgrade: legacy vector chunks and
74
  metrics without verified period context are intentionally hidden rather than
75
  silently upgraded.
76
 
 
 
 
 
77
  ## Intended use
78
 
79
  Amplegest is a research accelerator based on public information. It helps an experienced investor review evidence faster, challenge a thesis, and identify the next falsifiable datapoint. The analyst or portfolio manager retains responsibility for source verification, modelling, valuation, and the investment decision.
 
19
 
20
  | Surface | Role in the workflow |
21
  | --- | --- |
22
+ | **Company Primer** | Concise company onboarding: business model, revenue engines, geographic exposure, three-year evolution, investor-attention themes, relative price events, recent news, and the variables to watch. |
23
  | **PM Flash** | One-screen decision memo: filing date and source coverage, experimental AI read-through, thesis-confirming and thesis-challenging evidence, the swing factor, and the next items to watch. |
24
  | **Evidence & Deltas** | Audit layer: source-backed claims, before/after evidence, heuristic period deltas, risks, management commentary, and experimental AI hypotheses. |
25
  | **Financials** | Historical KPI trends, guidance history, earnings history, and exportable structured data. |
 
53
 
54
  ## Architecture
55
 
56
+ - **Offline ingestion** — SEC EDGAR XBRL into SQLite; 10-K/10-Q Business, Segments/Geography, MD&A, Risk Factors, and earnings-call transcripts into the evidence stores.
57
+ - **Company profile** — a cached, source-fingerprinted English profile separates stable filing/call analysis from independently refreshed prices and public news.
58
  - **Runtime agent** — a LangGraph tool loop retrieves structured financials, filings, transcripts, public news, and analyst data before producing a validated Pydantic brief.
59
  - **Retrieval** — vector search over-fetches candidates and reranks them with a cross-encoder.
60
  - **Post-synthesis controls** — deterministic source-reliability rules, corroboration checks, and heuristic period-delta detectors run after model synthesis.
61
+ - **Decision UI** — Company Primer onboards; PM Flash frames the decision; Evidence & Deltas provides auditability; Financials provides depth; Ask Evidence stays secondary.
62
 
63
  See [WRITEUP.md](WRITEUP.md) for product and methodology detail and [ARCHITECTURE.md](ARCHITECTURE.md) for data lineage and presentation controls.
64
 
 
76
  metrics without verified period context are intentionally hidden rather than
77
  silently upgraded.
78
 
79
+ For an already lineage-compatible database, `python ingest.py AAPL` is enough
80
+ to backfill the Company Primer’s 10-K Business and Segments/Geography sections;
81
+ the delta path reuses any transcript already stored.
82
+
83
  ## Intended use
84
 
85
  Amplegest is a research accelerator based on public information. It helps an experienced investor review evidence faster, challenge a thesis, and identify the next falsifiable datapoint. The analyst or portfolio manager retains responsibility for source verification, modelling, valuation, and the investment decision.
agent/company_profile.py ADDED
@@ -0,0 +1,482 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """Evidence-grounded Company Primer generation and translation."""
2
+ from __future__ import annotations
3
+
4
+ import copy
5
+ import hashlib
6
+ import json
7
+ from datetime import datetime, timezone
8
+ from typing import Callable, Optional
9
+
10
+ from langchain_core.messages import HumanMessage
11
+
12
+ from agent.company_profile_schemas import CompanyProfile
13
+ from agent.evidence import evidence_records_from, verify_brief_evidence, verify_fact
14
+ from agent.llm import RunConfig, build_system_message, make_chat_model
15
+ from agent.tools import get_financial_metrics, search_filing, search_transcript
16
+ from storage import metrics_db
17
+ from storage.sections_db import get_recent_transcripts, get_section, get_periods_for_ticker
18
+
19
+ PROFILE_SCHEMA_VERSION = "company_profile.v1"
20
+
21
+ _PROFILE_PROMPT = """
22
+ You are a senior equity-research analyst preparing a compact Company Primer for
23
+ someone who knows little about the company.
24
+
25
+ Use ONLY the evidence records supplied by the user. Never use prior knowledge.
26
+ Every factual object must include one SourcedFact with:
27
+ - source, reliability, evidence_snippet and evidence_ref copied exactly from one
28
+ supplied evidence record;
29
+ - a short evidence_snippet that appears verbatim in that record;
30
+ - no number that is absent from the supporting record.
31
+
32
+ Content rules:
33
+ - identity: one-line description, how the company makes money, customer types,
34
+ and competitive position; prefer the 10-K Business section.
35
+ - business_lines: 3-5 economic engines. revenue_share_pct must be null unless
36
+ the exact percentage is disclosed in the cited evidence.
37
+ - geographic_exposures: separate disclosed revenue geography from qualitative
38
+ sales, operational, supply-chain, or regulatory exposure. Do not convert a
39
+ country mention into materiality or a revenue percentage.
40
+ - strategic_changes: at most 3 material changes across the available years.
41
+ - attention_themes: exactly the 3 strongest investor questions supported by
42
+ earnings-call or filing evidence. Keep why_it_matters to one sentence.
43
+ - watch_variables: 3-4 variables, each with the next datapoint and a falsifiable
44
+ alert signal. These are monitoring prompts, not recommendations.
45
+
46
+ Keep every prose field concise. Analytical prose such as economics, implication,
47
+ why_it_matters and alert_signal is an AI hypothesis anchored to the cited fact.
48
+ Never provide a buy/sell recommendation, price target, valuation conclusion, or
49
+ claim that an event caused a stock-price move.
50
+ """.strip()
51
+
52
+ _TRANSLATABLE_FIELDS = {
53
+ "text",
54
+ "economics",
55
+ "implication",
56
+ "theme",
57
+ "why_it_matters",
58
+ "variable",
59
+ "next_datapoint",
60
+ "alert_signal",
61
+ }
62
+ _NEVER_TRANSLATE = {
63
+ "evidence_snippet",
64
+ "source",
65
+ "reliability",
66
+ "impact",
67
+ "verification_status",
68
+ "verification_reason",
69
+ "ticker",
70
+ "company_name",
71
+ "period",
72
+ "period_from",
73
+ "period_to",
74
+ "share_period",
75
+ "source_fingerprint",
76
+ "schema_version",
77
+ }
78
+
79
+
80
+ def source_fingerprint(ticker: str) -> str:
81
+ """Fingerprint stable company-profile sources, not live price/news data."""
82
+ ticker = ticker.upper()
83
+ rows = metrics_db.get_all_metrics(ticker)
84
+ latest_k = next((row for row in rows if row.get("form_type") == "10-K"), {})
85
+ latest_q = next((row for row in rows if row.get("form_type") == "10-Q"), {})
86
+ transcripts = get_recent_transcripts(ticker, n=1)
87
+ latest_transcript = transcripts[-1][0] if transcripts else ""
88
+ annual_periods = get_periods_for_ticker(ticker, form_type="10-K")
89
+ latest_annual = annual_periods[0] if annual_periods else ""
90
+ business_digest = hashlib.sha256(
91
+ (get_section(ticker, latest_annual, "business") or "").encode("utf-8")
92
+ ).hexdigest()[:12]
93
+ geography_digest = hashlib.sha256(
94
+ (get_section(ticker, latest_annual, "segments_geography") or "").encode("utf-8")
95
+ ).hexdigest()[:12]
96
+ identity = "\x1f".join(
97
+ (
98
+ ticker,
99
+ str(latest_k.get("accession") or latest_k.get("filing_date") or ""),
100
+ str(latest_q.get("accession") or latest_q.get("filing_date") or ""),
101
+ latest_transcript,
102
+ business_digest,
103
+ geography_digest,
104
+ PROFILE_SCHEMA_VERSION,
105
+ )
106
+ )
107
+ return hashlib.sha256(identity.encode("utf-8")).hexdigest()[:24]
108
+
109
+
110
+ def profile_source_coverage(ticker: str) -> dict:
111
+ ticker = ticker.upper()
112
+ annual_periods = get_periods_for_ticker(ticker, form_type="10-K")
113
+ business_values = [
114
+ get_section(ticker, period, "business") for period in annual_periods
115
+ ]
116
+ geography_values = [
117
+ get_section(ticker, period, "segments_geography") for period in annual_periods
118
+ ]
119
+ business = sum(1 for value in business_values if (value or "").strip())
120
+ geography = sum(1 for value in geography_values if (value or "").strip())
121
+ transcript_count = len(get_recent_transcripts(ticker, n=4))
122
+ status = (
123
+ "COMPLETE"
124
+ if business >= 1 and geography >= 1 and transcript_count >= 2
125
+ else "PARTIAL"
126
+ )
127
+ return {
128
+ "status": status,
129
+ "annual_periods": len(annual_periods),
130
+ "business_sections": business,
131
+ "business_sections_attempted": sum(
132
+ 1 for value in business_values if value is not None
133
+ ),
134
+ "segments_geography_sections": geography,
135
+ "segments_geography_sections_attempted": sum(
136
+ 1 for value in geography_values if value is not None
137
+ ),
138
+ "recent_transcripts": transcript_count,
139
+ }
140
+
141
+
142
+ def _invoke_tool(tool, arguments: dict) -> str:
143
+ return tool.invoke(arguments)
144
+
145
+
146
+ def collect_profile_evidence(
147
+ ticker: str,
148
+ progress: Optional[Callable[[str], None]] = None,
149
+ ) -> list[str]:
150
+ ticker = ticker.upper()
151
+ payloads: list[str] = []
152
+ calls = [
153
+ (get_financial_metrics, {"ticker": ticker}, "Loading verified financial history"),
154
+ (
155
+ search_filing,
156
+ {
157
+ "ticker": ticker,
158
+ "query": "business products services customers distribution competition",
159
+ "period": "",
160
+ "since": "",
161
+ },
162
+ "Reading the business model",
163
+ ),
164
+ (
165
+ search_filing,
166
+ {
167
+ "ticker": ticker,
168
+ "query": "reportable segments revenue geographic information countries international sales",
169
+ "period": "",
170
+ "since": "",
171
+ },
172
+ "Mapping segments and geography",
173
+ ),
174
+ (
175
+ search_filing,
176
+ {
177
+ "ticker": ticker,
178
+ "query": "strategy change product mix capital allocation growth drivers",
179
+ "period": "",
180
+ "since": "",
181
+ },
182
+ "Comparing strategic evolution",
183
+ ),
184
+ (
185
+ search_transcript,
186
+ {
187
+ "ticker": ticker,
188
+ "query": "analyst questions demand competition margins products strategy risks",
189
+ "period": "",
190
+ "since": "",
191
+ },
192
+ "Finding investor attention themes",
193
+ ),
194
+ ]
195
+ for tool, arguments, label in calls:
196
+ if progress:
197
+ progress(label)
198
+ try:
199
+ payloads.append(_invoke_tool(tool, arguments))
200
+ except Exception as exc:
201
+ from agent.evidence import evidence_envelope
202
+
203
+ payloads.append(
204
+ evidence_envelope(
205
+ tool=tool.name,
206
+ query=arguments,
207
+ status="ERROR",
208
+ message=str(exc),
209
+ error_code="PROFILE_RETRIEVAL_ERROR",
210
+ )
211
+ )
212
+ return payloads
213
+
214
+
215
+ def _evidence_prompt(payloads: list[str]) -> str:
216
+ records = evidence_records_from(payloads)
217
+ compact = []
218
+ for record in records:
219
+ compact.append(
220
+ {
221
+ "ref": record.ref.model_dump(mode="json"),
222
+ "content": record.content[:5000],
223
+ "metadata": record.metadata,
224
+ }
225
+ )
226
+ return json.dumps(compact, ensure_ascii=False)
227
+
228
+
229
+ def _annual_trends(ticker: str) -> list[dict]:
230
+ rows = [
231
+ row
232
+ for row in metrics_db.get_all_metrics(ticker)
233
+ if row.get("form_type") == "10-K"
234
+ and row.get("data_quality_status") == "VERIFIED"
235
+ ]
236
+ rows.sort(key=lambda row: str(row.get("report_date") or row.get("filing_date") or ""))
237
+ return [
238
+ {
239
+ "period": row.get("period") or "",
240
+ "revenue_bn": (
241
+ round(float(row["revenue"]) / 1e9, 3)
242
+ if row.get("revenue") is not None
243
+ else None
244
+ ),
245
+ "revenue_yoy_pct": row.get("revenue_yoy_pct"),
246
+ "operating_margin": row.get("operating_margin"),
247
+ "free_cash_flow_bn": (
248
+ round(float(row["free_cash_flow"]) / 1e9, 3)
249
+ if row.get("free_cash_flow") is not None
250
+ else None
251
+ ),
252
+ "eps": row.get("eps"),
253
+ }
254
+ for row in rows[-3:]
255
+ ]
256
+
257
+
258
+ def _is_verified_fact(value) -> bool:
259
+ return isinstance(value, dict) and value.get("verification_status") == "VERIFIED"
260
+
261
+
262
+ def _prune_unverified(profile: dict) -> int:
263
+ removed = 0
264
+ identity = profile.get("identity")
265
+ if isinstance(identity, dict):
266
+ for key in ("one_liner", "business_model", "competitive_position"):
267
+ if identity.get(key) is not None and not _is_verified_fact(identity.get(key)):
268
+ identity[key] = None
269
+ removed += 1
270
+ customers = identity.get("customer_types")
271
+ if isinstance(customers, list):
272
+ kept = [item for item in customers if _is_verified_fact(item)]
273
+ removed += len(customers) - len(kept)
274
+ identity["customer_types"] = kept
275
+
276
+ for key, fact_key in (
277
+ ("business_lines", "description"),
278
+ ("geographic_exposures", "description"),
279
+ ("strategic_changes", "change"),
280
+ ("attention_themes", "evidence"),
281
+ ("watch_variables", "evidence"),
282
+ ):
283
+ values = profile.get(key)
284
+ if not isinstance(values, list):
285
+ continue
286
+ kept = [
287
+ item
288
+ for item in values
289
+ if isinstance(item, dict) and _is_verified_fact(item.get(fact_key))
290
+ ]
291
+ removed += len(values) - len(kept)
292
+ profile[key] = kept
293
+ return removed
294
+
295
+
296
+ def _quarantine_unsupported_shares(profile: dict, records) -> int:
297
+ """Hide parent-level revenue shares unless the cited record supports them."""
298
+ removed = 0
299
+ for key, fact_key in (
300
+ ("business_lines", "description"),
301
+ ("geographic_exposures", "description"),
302
+ ):
303
+ for item in profile.get(key) or []:
304
+ if not isinstance(item, dict) or item.get("revenue_share_pct") is None:
305
+ continue
306
+ fact = item.get(fact_key)
307
+ if not isinstance(fact, dict):
308
+ item["revenue_share_pct"] = None
309
+ item["share_period" if key == "business_lines" else "period"] = None
310
+ removed += 1
311
+ continue
312
+ synthetic = {
313
+ **fact,
314
+ "text": f"{item['revenue_share_pct']}% revenue share",
315
+ }
316
+ verify_fact(synthetic, records)
317
+ if synthetic.get("verification_status") != "VERIFIED":
318
+ item["revenue_share_pct"] = None
319
+ item["share_period" if key == "business_lines" else "period"] = None
320
+ removed += 1
321
+ return removed
322
+
323
+
324
+ def _partial_profile(ticker: str, fingerprint: str, coverage: dict) -> dict:
325
+ rows = metrics_db.get_all_metrics(ticker)
326
+ company_name = rows[0].get("company_name") if rows else ticker
327
+ return CompanyProfile(
328
+ ticker=ticker,
329
+ company_name=company_name or ticker,
330
+ source_fingerprint=fingerprint,
331
+ generated_at=datetime.now(timezone.utc).isoformat(),
332
+ data_as_of=(rows[0].get("filing_date") if rows else None),
333
+ status="PARTIAL",
334
+ annual_trends=_annual_trends(ticker),
335
+ source_coverage=coverage,
336
+ ).model_dump(mode="json")
337
+
338
+
339
+ def generate_company_profile(
340
+ ticker: str,
341
+ config: RunConfig,
342
+ progress: Optional[Callable[[str], None]] = None,
343
+ ) -> dict:
344
+ ticker = ticker.upper()
345
+ fingerprint = source_fingerprint(ticker)
346
+ coverage = profile_source_coverage(ticker)
347
+ payloads = collect_profile_evidence(ticker, progress=progress)
348
+ records = evidence_records_from(payloads)
349
+ if not records:
350
+ return _partial_profile(ticker, fingerprint, coverage)
351
+
352
+ rows = metrics_db.get_all_metrics(ticker)
353
+ company_name = rows[0].get("company_name") if rows else ticker
354
+ if progress:
355
+ progress("Synthesizing the compact company primer")
356
+ llm = make_chat_model(config, max_retries=3, max_tokens=8192)
357
+ structured = llm.with_structured_output(CompanyProfile)
358
+ system = build_system_message(config, _PROFILE_PROMPT)
359
+ user = HumanMessage(
360
+ content=(
361
+ f"Create the Company Primer for {ticker} ({company_name}). "
362
+ "Return the structured CompanyProfile.\n\nEVIDENCE RECORDS:\n"
363
+ + _evidence_prompt(payloads)
364
+ )
365
+ )
366
+ result = structured.invoke([system, user])
367
+ if isinstance(result, CompanyProfile):
368
+ profile = result.model_dump(mode="json")
369
+ elif isinstance(result, dict):
370
+ profile = CompanyProfile.model_validate(result).model_dump(mode="json")
371
+ else:
372
+ raise ValueError("The model did not return a CompanyProfile")
373
+
374
+ profile.update(
375
+ {
376
+ "ticker": ticker,
377
+ "company_name": company_name or ticker,
378
+ "schema_version": PROFILE_SCHEMA_VERSION,
379
+ "source_fingerprint": fingerprint,
380
+ "language": "English",
381
+ "generated_at": datetime.now(timezone.utc).isoformat(),
382
+ "data_as_of": max(
383
+ (record.ref.as_of for record in records if record.ref.as_of),
384
+ default=(rows[0].get("filing_date") if rows else None),
385
+ ),
386
+ "model": config.model,
387
+ "annual_trends": _annual_trends(ticker),
388
+ "source_coverage": coverage,
389
+ "status": coverage["status"],
390
+ }
391
+ )
392
+ verify_brief_evidence(profile, payloads)
393
+ removed = _quarantine_unsupported_shares(profile, records)
394
+ removed += _prune_unverified(profile)
395
+ verification_report = profile.setdefault("verification_report", {})
396
+ verification_report["removed"] = (
397
+ int(verification_report.get("removed") or 0) + removed
398
+ )
399
+ profile["status"] = (
400
+ "COMPLETE"
401
+ if coverage.get("status") == "COMPLETE"
402
+ and (profile.get("evidence_coverage") or {}).get("status") == "VERIFIED"
403
+ and not verification_report["removed"]
404
+ else "PARTIAL"
405
+ )
406
+
407
+ from analysis.company_attention import attach_attention_stats
408
+
409
+ attach_attention_stats(profile, ticker, [])
410
+ return CompanyProfile.model_validate(profile).model_dump(mode="json")
411
+
412
+
413
+ def _collect_translation_segments(node, path=()):
414
+ result = []
415
+ if isinstance(node, dict):
416
+ for key, value in node.items():
417
+ if key in _NEVER_TRANSLATE:
418
+ continue
419
+ if key in _TRANSLATABLE_FIELDS and isinstance(value, str) and value:
420
+ result.append((path + (key,), value))
421
+ else:
422
+ result.extend(_collect_translation_segments(value, path + (key,)))
423
+ elif isinstance(node, list):
424
+ for index, value in enumerate(node):
425
+ result.extend(_collect_translation_segments(value, path + (index,)))
426
+ return result
427
+
428
+
429
+ def _set_path(value, path, replacement):
430
+ current = value
431
+ for step in path[:-1]:
432
+ current = current[step]
433
+ current[path[-1]] = replacement
434
+
435
+
436
+ def translate_company_profile(
437
+ profile: dict,
438
+ target_language: str,
439
+ config: RunConfig,
440
+ ) -> tuple[dict, Optional[dict]]:
441
+ """Translate display prose while preserving evidence snippets and identifiers."""
442
+ if not target_language or target_language == "English":
443
+ return profile, None
444
+ segments = _collect_translation_segments(profile)
445
+ if not segments:
446
+ return profile, None
447
+ payload = {str(index): text for index, (_, text) in enumerate(segments)}
448
+ prompt = (
449
+ f"Translate every JSON value into {target_language}. Return only a JSON "
450
+ "object with the same keys. Use concise professional equity-research "
451
+ "language. Preserve all numbers, tickers, company names, periods and "
452
+ "finance abbreviations."
453
+ )
454
+ try:
455
+ llm = make_chat_model(config, max_retries=2, max_tokens=8192)
456
+ response = llm.invoke(
457
+ [
458
+ build_system_message(config, prompt),
459
+ HumanMessage(content=json.dumps(payload, ensure_ascii=False)),
460
+ ]
461
+ )
462
+ raw = response.content
463
+ if isinstance(raw, list):
464
+ raw = "".join(
465
+ str(block.get("text") or "")
466
+ for block in raw
467
+ if isinstance(block, dict)
468
+ )
469
+ from agent.graph import _extract_json
470
+
471
+ translated_values = json.loads(_extract_json(str(raw)))
472
+ translated = copy.deepcopy(profile)
473
+ for index, (path, _) in enumerate(segments):
474
+ replacement = translated_values.get(str(index))
475
+ if isinstance(replacement, str) and replacement:
476
+ _set_path(translated, path, replacement)
477
+ translated["language"] = target_language
478
+ from agent.cost_log import compute_run_cost
479
+
480
+ return translated, compute_run_cost([response], config.model)
481
+ except Exception:
482
+ return profile, None
agent/company_profile_schemas.py ADDED
@@ -0,0 +1,120 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """Structured payload for the Company Primer research surface."""
2
+ from __future__ import annotations
3
+
4
+ from typing import Any, Literal, Optional
5
+
6
+ from pydantic import BaseModel, ConfigDict, Field
7
+
8
+ from agent.schemas import SourcedFact
9
+
10
+
11
+ class CompanyIdentity(BaseModel):
12
+ model_config = ConfigDict(extra="ignore")
13
+
14
+ one_liner: Optional[SourcedFact] = None
15
+ business_model: Optional[SourcedFact] = None
16
+ customer_types: list[SourcedFact] = Field(default_factory=list)
17
+ competitive_position: Optional[SourcedFact] = None
18
+
19
+
20
+ class BusinessLine(BaseModel):
21
+ model_config = ConfigDict(extra="ignore")
22
+
23
+ name: str
24
+ description: SourcedFact
25
+ economics: str = Field(
26
+ default="",
27
+ description="Concise analytical framing; displayed as AI experimental.",
28
+ )
29
+ trend: Literal["growing", "stable", "declining", "mixed", "not_disclosed"] = "not_disclosed"
30
+ revenue_share_pct: Optional[float] = Field(default=None, ge=0, le=100)
31
+ share_period: Optional[str] = None
32
+
33
+
34
+ class GeographicExposure(BaseModel):
35
+ model_config = ConfigDict(extra="ignore")
36
+
37
+ name: str
38
+ exposure_types: list[
39
+ Literal["revenue", "operations", "supply_chain", "regulation"]
40
+ ] = Field(default_factory=list)
41
+ description: SourcedFact
42
+ revenue_share_pct: Optional[float] = Field(default=None, ge=0, le=100)
43
+ period: Optional[str] = None
44
+
45
+
46
+ class StrategicChange(BaseModel):
47
+ model_config = ConfigDict(extra="ignore")
48
+
49
+ period_from: str = ""
50
+ period_to: str = ""
51
+ change: SourcedFact
52
+ implication: str = Field(
53
+ default="",
54
+ description="One concise AI interpretation, never an investment recommendation.",
55
+ )
56
+
57
+
58
+ class AttentionPoint(BaseModel):
59
+ model_config = ConfigDict(extra="ignore")
60
+
61
+ period: str
62
+ mentions: int = Field(default=0, ge=0)
63
+
64
+
65
+ class AttentionTheme(BaseModel):
66
+ model_config = ConfigDict(extra="ignore")
67
+
68
+ theme: str
69
+ why_it_matters: str
70
+ evidence: SourcedFact
71
+ call_mentions: list[AttentionPoint] = Field(default_factory=list)
72
+ retrieved_news_count: int = Field(default=0, ge=0)
73
+ direction: Literal["rising", "stable", "falling", "new", "insufficient"] = "insufficient"
74
+
75
+
76
+ class WatchVariable(BaseModel):
77
+ model_config = ConfigDict(extra="ignore")
78
+
79
+ variable: str
80
+ why_it_matters: str
81
+ next_datapoint: str
82
+ alert_signal: str
83
+ evidence: SourcedFact
84
+
85
+
86
+ class AnnualTrend(BaseModel):
87
+ model_config = ConfigDict(extra="ignore")
88
+
89
+ period: str
90
+ revenue_bn: Optional[float] = None
91
+ revenue_yoy_pct: Optional[float] = None
92
+ operating_margin: Optional[float] = None
93
+ free_cash_flow_bn: Optional[float] = None
94
+ eps: Optional[float] = None
95
+
96
+
97
+ class CompanyProfile(BaseModel):
98
+ model_config = ConfigDict(extra="ignore")
99
+
100
+ ticker: str
101
+ company_name: str
102
+ schema_version: str = "company_profile.v1"
103
+ source_fingerprint: str = ""
104
+ language: str = "English"
105
+ generated_at: Optional[str] = None
106
+ data_as_of: Optional[str] = None
107
+ model: Optional[str] = None
108
+ status: Literal["COMPLETE", "PARTIAL"] = "PARTIAL"
109
+
110
+ identity: CompanyIdentity = Field(default_factory=CompanyIdentity)
111
+ business_lines: list[BusinessLine] = Field(default_factory=list)
112
+ geographic_exposures: list[GeographicExposure] = Field(default_factory=list)
113
+ strategic_changes: list[StrategicChange] = Field(default_factory=list)
114
+ attention_themes: list[AttentionTheme] = Field(default_factory=list)
115
+ watch_variables: list[WatchVariable] = Field(default_factory=list)
116
+ annual_trends: list[AnnualTrend] = Field(default_factory=list)
117
+
118
+ source_coverage: dict[str, Any] = Field(default_factory=dict)
119
+ evidence_coverage: dict[str, Any] = Field(default_factory=dict)
120
+ verification_report: dict[str, Any] = Field(default_factory=dict)
analysis/company_attention.py ADDED
@@ -0,0 +1,150 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """Deterministic investor-attention signals from earnings-call Q&A."""
2
+ from __future__ import annotations
3
+
4
+ import re
5
+ from collections import Counter
6
+ from typing import Iterable
7
+
8
+ from analysis.transcript_parse import parse_call
9
+ from storage.sections_db import get_recent_transcripts
10
+
11
+ _STOPWORDS = frozenset(
12
+ "about after again also analyst and are been before being between can could "
13
+ "company quarter quarters did does doing for from give given going have how "
14
+ "into just like more much next now our please really say should some talk "
15
+ "than that the their them then there these they think this those through "
16
+ "very want was were what when where which why will with would year years "
17
+ "you your".split()
18
+ )
19
+
20
+
21
+ def _tokens(text: str) -> set[str]:
22
+ return {
23
+ token
24
+ for token in re.findall(r"[a-z][a-z0-9-]{2,}", (text or "").lower())
25
+ if token not in _STOPWORDS
26
+ }
27
+
28
+
29
+ def _similar(left: set[str], right: set[str]) -> bool:
30
+ if not left or not right:
31
+ return False
32
+ overlap = len(left & right)
33
+ return overlap >= 2 and overlap / len(left | right) >= 0.12
34
+
35
+
36
+ def cluster_questions(calls: Iterable[tuple[str, str]], max_themes: int = 5) -> list[dict]:
37
+ """Greedily cluster analyst questions and return deterministic counts."""
38
+ clusters: list[dict] = []
39
+ periods: list[str] = []
40
+ for period, raw in calls:
41
+ periods.append(period)
42
+ parsed = parse_call(period, raw)
43
+ for exchange in parsed.qa:
44
+ question = (exchange.question or "").strip()
45
+ tokens = _tokens(question)
46
+ if len(tokens) < 3:
47
+ continue
48
+ best = None
49
+ best_overlap = 0
50
+ for cluster in clusters:
51
+ overlap = len(tokens & cluster["tokens"])
52
+ if overlap > best_overlap and _similar(tokens, cluster["tokens"]):
53
+ best = cluster
54
+ best_overlap = overlap
55
+ if best is None:
56
+ best = {
57
+ "tokens": set(tokens),
58
+ "questions": [],
59
+ "mentions": Counter(),
60
+ }
61
+ clusters.append(best)
62
+ best["tokens"].update(tokens)
63
+ best["questions"].append(question)
64
+ best["mentions"][period] += 1
65
+
66
+ usable = [
67
+ cluster
68
+ for cluster in clusters
69
+ if len(cluster["questions"]) >= 2
70
+ ]
71
+ usable.sort(
72
+ key=lambda cluster: (
73
+ -len(cluster["mentions"]),
74
+ -len(cluster["questions"]),
75
+ )
76
+ )
77
+
78
+ output: list[dict] = []
79
+ for cluster in usable[:max_themes]:
80
+ counts = Counter()
81
+ for question in cluster["questions"]:
82
+ counts.update(_tokens(question))
83
+ label_tokens = [token for token, _ in counts.most_common(3)]
84
+ output.append(
85
+ {
86
+ "label": " / ".join(label_tokens),
87
+ "tokens": label_tokens,
88
+ "mentions": [
89
+ {"period": period, "mentions": int(cluster["mentions"].get(period, 0))}
90
+ for period in periods
91
+ ],
92
+ "question_count": len(cluster["questions"]),
93
+ "period_count": len(cluster["mentions"]),
94
+ "representative_question": cluster["questions"][0],
95
+ }
96
+ )
97
+ return output
98
+
99
+
100
+ def get_attention_clusters(ticker: str, n: int = 4, max_themes: int = 5) -> list[dict]:
101
+ return cluster_questions(get_recent_transcripts(ticker.upper(), n=n), max_themes=max_themes)
102
+
103
+
104
+ def attach_attention_stats(profile: dict, ticker: str, news_items: list[dict]) -> dict:
105
+ """Attach Q&A and retrieved-news counts to synthesized attention themes."""
106
+ themes = profile.get("attention_themes")
107
+ if not isinstance(themes, list) or not themes:
108
+ return profile
109
+ clusters = get_attention_clusters(ticker)
110
+ unused = set(range(len(clusters)))
111
+
112
+ for index, theme in enumerate(themes):
113
+ if not isinstance(theme, dict):
114
+ continue
115
+ theme_tokens = _tokens(str(theme.get("theme") or ""))
116
+ scored = [
117
+ (len(theme_tokens & set(cluster.get("tokens") or [])), cluster_index)
118
+ for cluster_index, cluster in enumerate(clusters)
119
+ if cluster_index in unused
120
+ ]
121
+ scored.sort(reverse=True)
122
+ cluster_index = scored[0][1] if scored and scored[0][0] > 0 else None
123
+ if cluster_index is None and index < len(clusters) and index in unused:
124
+ cluster_index = index
125
+ if cluster_index is not None:
126
+ unused.discard(cluster_index)
127
+ mentions = clusters[cluster_index]["mentions"]
128
+ theme["call_mentions"] = mentions
129
+ values = [point["mentions"] for point in mentions]
130
+ if values:
131
+ if values[0] == 0 and values[-1] > 0:
132
+ theme["direction"] = "new"
133
+ elif values[-1] > values[0]:
134
+ theme["direction"] = "rising"
135
+ elif values[-1] < values[0]:
136
+ theme["direction"] = "falling"
137
+ else:
138
+ theme["direction"] = "stable"
139
+
140
+ expanded_tokens = theme_tokens | _tokens(
141
+ str((theme.get("evidence") or {}).get("text") or "")
142
+ )
143
+ theme["retrieved_news_count"] = sum(
144
+ 1
145
+ for item in news_items
146
+ if expanded_tokens
147
+ and expanded_tokens
148
+ & _tokens(f"{item.get('title', '')} {item.get('snippet', '')}")
149
+ )
150
+ return profile
analytics/company_market.py ADDED
@@ -0,0 +1,246 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """Relative price history and event candidates for Company Primer."""
2
+ from __future__ import annotations
3
+
4
+ import os
5
+ from datetime import datetime
6
+ from email.utils import parsedate_to_datetime
7
+ from typing import Iterable, Optional
8
+
9
+ from storage.earnings_cache import get as cache_get, set as cache_set
10
+
11
+ SECTOR_ETFS = {
12
+ "technology": "XLK",
13
+ "information technology": "XLK",
14
+ "consumer cyclical": "XLY",
15
+ "consumer discretionary": "XLY",
16
+ "communication services": "XLC",
17
+ "financial services": "XLF",
18
+ "financials": "XLF",
19
+ "healthcare": "XLV",
20
+ "health care": "XLV",
21
+ "industrials": "XLI",
22
+ "energy": "XLE",
23
+ "consumer defensive": "XLP",
24
+ "consumer staples": "XLP",
25
+ "utilities": "XLU",
26
+ "real estate": "XLRE",
27
+ "basic materials": "XLB",
28
+ "materials": "XLB",
29
+ }
30
+
31
+
32
+ def sector_etf(sector: str) -> str:
33
+ return SECTOR_ETFS.get((sector or "").strip().lower(), "SPY")
34
+
35
+
36
+ def select_relative_events(
37
+ dates: list[str],
38
+ company_values: list[float],
39
+ sector_values: list[float],
40
+ *,
41
+ threshold_pct: float = 2.0,
42
+ per_direction: int = 3,
43
+ min_spacing: int = 5,
44
+ ) -> list[dict]:
45
+ """Select balanced, non-overlapping relative one-day moves."""
46
+ candidates: list[dict] = []
47
+ for index in range(1, min(len(dates), len(company_values), len(sector_values))):
48
+ prior_company = company_values[index - 1]
49
+ prior_sector = sector_values[index - 1]
50
+ if not prior_company or not prior_sector:
51
+ continue
52
+ company_return = (company_values[index] / prior_company - 1) * 100
53
+ sector_return = (sector_values[index] / prior_sector - 1) * 100
54
+ excess = company_return - sector_return
55
+ if abs(excess) >= threshold_pct:
56
+ candidates.append(
57
+ {
58
+ "index": index,
59
+ "date": dates[index],
60
+ "company_return_pct": round(company_return, 2),
61
+ "sector_return_pct": round(sector_return, 2),
62
+ "sector_excess_pct": round(excess, 2),
63
+ }
64
+ )
65
+
66
+ selected: list[dict] = []
67
+ for sign in (1, -1):
68
+ side = [item for item in candidates if item["sector_excess_pct"] * sign > 0]
69
+ side.sort(key=lambda item: abs(item["sector_excess_pct"]), reverse=True)
70
+ for item in side:
71
+ if any(abs(item["index"] - chosen["index"]) < min_spacing for chosen in selected):
72
+ continue
73
+ selected.append(item)
74
+ if sum(1 for chosen in selected if chosen["sector_excess_pct"] * sign > 0) >= per_direction:
75
+ break
76
+ selected.sort(key=lambda item: item["date"])
77
+ return selected
78
+
79
+
80
+ def _to_date(value) -> str:
81
+ if hasattr(value, "date"):
82
+ value = value.date()
83
+ return str(value)[:10]
84
+
85
+
86
+ def _parse_published_date(value: object):
87
+ raw = str(value or "").strip()
88
+ if not raw:
89
+ return None
90
+ try:
91
+ return datetime.fromisoformat(raw[:10]).date()
92
+ except ValueError:
93
+ try:
94
+ return parsedate_to_datetime(raw).date()
95
+ except (TypeError, ValueError, OverflowError):
96
+ return None
97
+
98
+
99
+ def _price_rows(frame, symbols: Iterable[str]) -> list[dict]:
100
+ close = frame["Close"] if "Close" in frame else frame
101
+ symbols = list(symbols)
102
+ rows: list[dict] = []
103
+ for index, raw_date in enumerate(close.index):
104
+ row = {"date": _to_date(raw_date)}
105
+ valid = True
106
+ for symbol in symbols:
107
+ try:
108
+ value = close[symbol].iloc[index]
109
+ value = float(value)
110
+ if value != value:
111
+ valid = False
112
+ break
113
+ row[symbol] = value
114
+ except Exception:
115
+ valid = False
116
+ break
117
+ if valid:
118
+ rows.append(row)
119
+ return rows
120
+
121
+
122
+ def _event_context(
123
+ ticker: str,
124
+ company_name: str,
125
+ event_date: str,
126
+ ) -> Optional[dict]:
127
+ cache_key = f"COMPANY_EVENT:v1:{ticker.upper()}:{event_date}"
128
+ cached = cache_get(cache_key, ttl_hours=24 * 7)
129
+ if cached is not None:
130
+ return cached or None
131
+ key = os.getenv("TAVILY_API_KEY", "").strip()
132
+ if not key:
133
+ return None
134
+ try:
135
+ from tavily import TavilyClient
136
+
137
+ results = TavilyClient(api_key=key).search(
138
+ query=f"{ticker.upper()} {company_name} {event_date} stock news",
139
+ max_results=3,
140
+ search_depth="basic",
141
+ topic="news",
142
+ ).get("results", [])
143
+ target = datetime.fromisoformat(event_date).date()
144
+ best = None
145
+ for item in results:
146
+ published = _parse_published_date(item.get("published_date"))
147
+ if published is None:
148
+ continue
149
+ if abs((published - target).days) <= 2:
150
+ best = {
151
+ "headline": str(item.get("title") or ""),
152
+ "url": str(item.get("url") or ""),
153
+ "published_date": published.isoformat(),
154
+ "attribution_status": "CORROBORATED_EVENT",
155
+ }
156
+ break
157
+ cache_set(cache_key, best or {})
158
+ return best
159
+ except Exception:
160
+ return None
161
+
162
+
163
+ def fetch_company_market(
164
+ ticker: str,
165
+ company_name: str,
166
+ ) -> tuple[Optional[dict], Optional[str]]:
167
+ ticker = ticker.upper()
168
+ cache_key = f"COMPANY_MARKET:v1:{ticker}:3y"
169
+ cached = cache_get(cache_key, ttl_hours=6)
170
+ if cached is not None:
171
+ return cached, None
172
+
173
+ try:
174
+ from ingestion.alphavantage import fetch_overview
175
+
176
+ overview, _overview_error = fetch_overview(ticker)
177
+ except Exception:
178
+ overview = None
179
+ sector = str((overview or {}).get("Sector") or "")
180
+ industry = str((overview or {}).get("Industry") or "")
181
+ benchmark = sector_etf(sector)
182
+ symbols = [ticker, "SPY"] + ([] if benchmark == "SPY" else [benchmark])
183
+
184
+ try:
185
+ import yfinance as yf
186
+
187
+ frame = yf.download(
188
+ symbols,
189
+ period="3y",
190
+ interval="1d",
191
+ auto_adjust=True,
192
+ progress=False,
193
+ threads=False,
194
+ )
195
+ rows = _price_rows(frame, symbols)
196
+ except Exception as exc:
197
+ return None, f"price history unavailable: {exc}"
198
+ if len(rows) < 2:
199
+ return None, "price history unavailable"
200
+
201
+ dates = [row["date"] for row in rows]
202
+ company_values = [row[ticker] for row in rows]
203
+ sector_values = [row[benchmark] for row in rows]
204
+ events = select_relative_events(dates, company_values, sector_values)
205
+ for event in events:
206
+ context = _event_context(ticker, company_name, event["date"])
207
+ if context:
208
+ event.update(context)
209
+ else:
210
+ event.update(
211
+ {
212
+ "headline": "No verified catalyst found",
213
+ "url": "",
214
+ "published_date": "",
215
+ "attribution_status": "NO_VERIFIED_CATALYST",
216
+ }
217
+ )
218
+
219
+ bases = {symbol: rows[0][symbol] for symbol in symbols}
220
+ weekly_rows = rows[::5]
221
+ if weekly_rows[-1]["date"] != rows[-1]["date"]:
222
+ weekly_rows.append(rows[-1])
223
+ normalized = [
224
+ {
225
+ "date": row["date"],
226
+ **{
227
+ symbol: round(row[symbol] / bases[symbol] * 100, 2)
228
+ for symbol in symbols
229
+ },
230
+ }
231
+ for row in weekly_rows
232
+ ]
233
+ result = {
234
+ "ticker": ticker,
235
+ "company_name": company_name,
236
+ "sector": sector,
237
+ "industry": industry,
238
+ "benchmark": benchmark,
239
+ "market_cap": (overview or {}).get("MarketCapitalization"),
240
+ "price_source": "Yahoo Finance",
241
+ "history": normalized,
242
+ "events": events,
243
+ "as_of": rows[-1]["date"],
244
+ }
245
+ cache_set(cache_key, result)
246
+ return result, None
analytics/company_news.py ADDED
@@ -0,0 +1,51 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """Public-news feed for Company Primer, cached independently from the profile."""
2
+ from __future__ import annotations
3
+
4
+ import os
5
+ from typing import Optional
6
+
7
+ from storage.earnings_cache import get as cache_get, set as cache_set
8
+
9
+
10
+ def fetch_company_news(
11
+ ticker: str,
12
+ company_name: str,
13
+ *,
14
+ days: int = 90,
15
+ max_results: int = 10,
16
+ ) -> tuple[list[dict], Optional[str]]:
17
+ cache_key = f"COMPANY_NEWS:v1:{ticker.upper()}:{days}:{max_results}"
18
+ cached = cache_get(cache_key, ttl_hours=1)
19
+ if cached is not None:
20
+ return list(cached.get("items") or []), None
21
+
22
+ key = os.getenv("TAVILY_API_KEY", "").strip()
23
+ if not key:
24
+ return [], "TAVILY_API_KEY not set"
25
+ try:
26
+ from tavily import TavilyClient
27
+
28
+ results = TavilyClient(api_key=key).search(
29
+ query=(
30
+ f"{ticker.upper()} {company_name} company strategy products "
31
+ "earnings regulation acquisition"
32
+ ),
33
+ max_results=max_results,
34
+ search_depth="basic",
35
+ days=days,
36
+ topic="news",
37
+ ).get("results", [])
38
+ items = [
39
+ {
40
+ "title": str(item.get("title") or "Untitled"),
41
+ "url": str(item.get("url") or ""),
42
+ "published_date": str(item.get("published_date") or ""),
43
+ "snippet": str(item.get("content") or "")[:500],
44
+ }
45
+ for item in results
46
+ if item.get("url")
47
+ ]
48
+ cache_set(cache_key, {"items": items})
49
+ return items, None
50
+ except Exception as exc:
51
+ return [], f"news fetch failed: {exc}"
app.py CHANGED
@@ -27,12 +27,12 @@ if not st.session_state.get("_reranker_warmed"):
27
  # ── UI language — default to English (finance lingua franca) ───────────────────
28
  st.session_state.setdefault("ui_lang", "en")
29
 
30
- # ── Navigation state — default to Verdict (the decision view) ──────────────────
31
- st.session_state.setdefault("nav_key", "verdict")
32
  # Remap stale route keys from the pre-Decision-Stack navigation (open sessions).
33
  _nk = st.session_state["nav_key"]
34
  if _nk not in VALID_KEYS:
35
- st.session_state["nav_key"] = LEGACY_NAV.get(_nk, "verdict")
36
 
37
  # ── Generation state ────────────────────────────────────────────────────────────
38
  st.session_state.setdefault("gen", {
@@ -44,6 +44,7 @@ st.session_state.setdefault("gen", {
44
  "all_messages": [],
45
  "config": None,
46
  })
 
47
 
48
 
49
 
@@ -566,13 +567,202 @@ def _live_trace_fragment() -> None:
566
 
567
 
568
  # ── Main content routing ──────────────────────────────────────────────────────────
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
569
  gen_error = st.session_state.pop("_gen_error", None)
570
  if gen_error:
571
  err_ticker, err_msg = gen_error
572
  st.error(f"{t('brief_error_prefix')} **{err_ticker}**: {err_msg}")
573
 
574
  gen = st.session_state["gen"]
575
- active_key = st.session_state.get("nav_key", "snapshot")
576
 
577
  if gen["running"] or gen.get("trace"):
578
  _live_trace_fragment()
@@ -596,7 +786,9 @@ else:
596
  if brief:
597
  _render_ticker_bar(ticker_input, brief)
598
 
599
- if active_key == "verdict":
 
 
600
  if brief:
601
  from dashboard.verdict import render as render_verdict
602
  render_verdict(brief)
 
27
  # ── UI language — default to English (finance lingua franca) ───────────────────
28
  st.session_state.setdefault("ui_lang", "en")
29
 
30
+ # ── Navigation state — default to Company Primer (the onboarding view) ─────────
31
+ st.session_state.setdefault("nav_key", "company")
32
  # Remap stale route keys from the pre-Decision-Stack navigation (open sessions).
33
  _nk = st.session_state["nav_key"]
34
  if _nk not in VALID_KEYS:
35
+ st.session_state["nav_key"] = LEGACY_NAV.get(_nk, "company")
36
 
37
  # ── Generation state ────────────────────────────────────────────────────────────
38
  st.session_state.setdefault("gen", {
 
44
  "all_messages": [],
45
  "config": None,
46
  })
47
+ st.session_state.setdefault("profile_jobs", {})
48
 
49
 
50
 
 
567
 
568
 
569
  # ── Main content routing ──────────────────────────────────────────────────────────
570
+ def _run_company_profile_thread(
571
+ ticker: str,
572
+ job: dict,
573
+ config: RunConfig,
574
+ ) -> None:
575
+ """Generate and persist one canonical English Company Primer."""
576
+ try:
577
+ from agent.company_profile import generate_company_profile
578
+ from storage import company_profiles
579
+
580
+ def _progress(label: str) -> None:
581
+ job["label"] = label
582
+ job["completed_steps"] = min(int(job.get("completed_steps") or 0) + 1, 6)
583
+
584
+ profile = generate_company_profile(ticker, config, progress=_progress)
585
+ profile_id = company_profiles.save_profile(ticker, profile)
586
+ job["profile"] = profile
587
+ job["profile_id"] = profile_id
588
+ except Exception as exc:
589
+ import sys
590
+
591
+ print(f"[company profile thread error] {exc}", file=sys.stderr)
592
+ job["error"] = classify_llm_error(exc, config.provider) or str(exc)
593
+ finally:
594
+ job["running"] = False
595
+
596
+
597
+ def _start_company_profile_job(ticker: str, fingerprint: str, config: RunConfig) -> dict:
598
+ jobs = st.session_state.setdefault("profile_jobs", {})
599
+ job = {
600
+ "running": True,
601
+ "ticker": ticker,
602
+ "fingerprint": fingerprint,
603
+ "label": t("primer_generating"),
604
+ "completed_steps": 0,
605
+ "profile": None,
606
+ "profile_id": None,
607
+ "error": None,
608
+ "config": config,
609
+ }
610
+ jobs[ticker] = job
611
+ thread = threading.Thread(
612
+ target=_run_company_profile_thread,
613
+ args=(ticker, job, config),
614
+ daemon=True,
615
+ )
616
+ thread.start()
617
+ return job
618
+
619
+
620
+ def _render_company_profile_progress(job: dict) -> None:
621
+ completed = int(job.get("completed_steps") or 0)
622
+ progress = min(max(completed / 6, 0.04), 0.96 if job.get("running") else 1.0)
623
+ st.markdown(
624
+ f'<div style="background:#ffffff;border:1px solid #e5e7eb;border-radius:12px;'
625
+ f'padding:16px 20px;margin-bottom:16px;">'
626
+ f'<div style="display:flex;justify-content:space-between;gap:12px;">'
627
+ f'<strong>{t("primer_generating")} · {job.get("ticker", "")}</strong>'
628
+ f'<span style="color:#10b981;font-weight:700;">{int(progress * 100)}%</span></div>'
629
+ f'<div style="font-size:0.75rem;color:#6b7280;margin:5px 0 10px;">'
630
+ f'{job.get("label") or ""}</div>'
631
+ f'<div style="height:7px;background:#e5e7eb;border-radius:999px;overflow:hidden;">'
632
+ f'<div style="height:100%;width:{int(progress * 100)}%;background:#10b981;'
633
+ f'border-radius:999px;"></div></div></div>',
634
+ unsafe_allow_html=True,
635
+ )
636
+
637
+
638
+ def _display_company_profile(profile: dict, config: RunConfig | None) -> dict:
639
+ target_language = i18n.report_language()
640
+ if target_language == "English":
641
+ return profile
642
+ profile_id = profile.get("_profile_id")
643
+ if not profile_id:
644
+ return profile
645
+ try:
646
+ from storage import company_profiles
647
+
648
+ cached = company_profiles.get_translation(profile_id, target_language)
649
+ if cached:
650
+ return cached
651
+ if config is None:
652
+ return profile
653
+ from agent.company_profile import translate_company_profile
654
+
655
+ with st.spinner(t("translating_brief")):
656
+ translated, usage = translate_company_profile(profile, target_language, config)
657
+ if translated is not profile and translated.get("language") == target_language:
658
+ company_profiles.save_translation(profile_id, target_language, translated)
659
+ _absorb_translation_cost(str(profile.get("ticker") or ""), usage, config.model)
660
+ return translated
661
+ except Exception:
662
+ pass
663
+ st.caption(t("translation_failed"))
664
+ return profile
665
+
666
+
667
+ def _render_company_route(ticker: str, config: RunConfig | None) -> None:
668
+ from agent.company_profile import profile_source_coverage, source_fingerprint
669
+ from storage import company_profiles
670
+
671
+ fingerprint = source_fingerprint(ticker)
672
+ profile = company_profiles.get_latest_profile(
673
+ ticker,
674
+ source_fingerprint=fingerprint,
675
+ language="English",
676
+ )
677
+ jobs = st.session_state.setdefault("profile_jobs", {})
678
+ job = jobs.get(ticker)
679
+ if job and job.get("fingerprint") != fingerprint and not job.get("running"):
680
+ jobs.pop(ticker, None)
681
+ job = None
682
+
683
+ if profile is None:
684
+ if config is None:
685
+ st.warning(t("model_blocked_caption"))
686
+ return
687
+ if job is None:
688
+ job = _start_company_profile_job(ticker, fingerprint, config)
689
+ if job.get("running"):
690
+ _render_company_profile_progress(job)
691
+ time.sleep(0.4)
692
+ st.rerun()
693
+ if job.get("error"):
694
+ st.error(job["error"])
695
+ if st.button(t("primer_retry"), key=f"primer_retry_{ticker}"):
696
+ _start_company_profile_job(ticker, fingerprint, config)
697
+ st.rerun()
698
+ return
699
+ profile = job.get("profile")
700
+ if profile is None:
701
+ st.info(t("primer_identity_unavailable"))
702
+ return
703
+ profile["_profile_id"] = job.get("profile_id")
704
+
705
+ coverage = profile_source_coverage(ticker)
706
+ if (
707
+ coverage.get("business_sections_attempted", 0) < 1
708
+ or coverage.get("segments_geography_sections_attempted", 0) < 1
709
+ ):
710
+ st.warning(t("primer_backfill_warning").format(ticker=ticker))
711
+
712
+ refresh_col, _ = st.columns([1, 5])
713
+ with refresh_col:
714
+ if st.button(
715
+ t("primer_refresh"),
716
+ key=f"primer_refresh_{ticker}",
717
+ disabled=bool(job and job.get("running")),
718
+ ):
719
+ if config is None:
720
+ st.warning(t("model_blocked_caption"))
721
+ else:
722
+ _start_company_profile_job(ticker, fingerprint, config)
723
+ st.rerun()
724
+ if job and job.get("running"):
725
+ _render_company_profile_progress(job)
726
+ time.sleep(0.4)
727
+ st.rerun()
728
+ if job and job.get("error"):
729
+ st.warning(
730
+ f"{t('primer_refresh_failed')} {job['error']}"
731
+ )
732
+
733
+ display_profile = _display_company_profile(profile, config)
734
+ company_name = str(display_profile.get("company_name") or ticker)
735
+ from analytics.company_news import fetch_company_news
736
+ from analytics.company_market import fetch_company_market
737
+
738
+ with st.spinner(t("primer_loading_market")):
739
+ news_items, news_error = fetch_company_news(ticker, company_name)
740
+ market, market_error = fetch_company_market(ticker, company_name)
741
+
742
+ import copy
743
+ from analysis.company_attention import attach_attention_stats
744
+
745
+ display_profile = attach_attention_stats(copy.deepcopy(display_profile), ticker, news_items)
746
+ from dashboard.company_primer import render as render_company_primer
747
+
748
+ render_company_primer(
749
+ display_profile,
750
+ ticker,
751
+ market=market,
752
+ news_items=news_items,
753
+ news_error=news_error,
754
+ )
755
+ if market_error:
756
+ st.caption(market_error)
757
+
758
+
759
  gen_error = st.session_state.pop("_gen_error", None)
760
  if gen_error:
761
  err_ticker, err_msg = gen_error
762
  st.error(f"{t('brief_error_prefix')} **{err_ticker}**: {err_msg}")
763
 
764
  gen = st.session_state["gen"]
765
+ active_key = st.session_state.get("nav_key", "company")
766
 
767
  if gen["running"] or gen.get("trace"):
768
  _live_trace_fragment()
 
786
  if brief:
787
  _render_ticker_bar(ticker_input, brief)
788
 
789
+ if active_key == "company":
790
+ _render_company_route(ticker_input, run_cfg)
791
+ elif active_key == "verdict":
792
  if brief:
793
  from dashboard.verdict import render as render_verdict
794
  render_verdict(brief)
dashboard/company_primer.py ADDED
@@ -0,0 +1,542 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """Company Primer — compact company-onboarding surface."""
2
+ from __future__ import annotations
3
+
4
+ from datetime import datetime
5
+ from html import escape
6
+
7
+ import streamlit as st
8
+
9
+ from dashboard.components import section_header
10
+ from dashboard.i18n import t
11
+ from dashboard.theme import (
12
+ AI_BG,
13
+ AI_BORDER,
14
+ AI_COLOR,
15
+ BLUE,
16
+ BORDER,
17
+ BRAND_GREEN,
18
+ FS_BODY,
19
+ FS_META,
20
+ FS_PAGE,
21
+ TEXT,
22
+ TEXT_MUTED,
23
+ )
24
+
25
+
26
+ def _fact_text(value) -> str:
27
+ if not isinstance(value, dict):
28
+ return ""
29
+ if value.get("verification_status") != "VERIFIED":
30
+ return ""
31
+ return str(value.get("text") or "")
32
+
33
+
34
+ def _source_meta(fact: dict | None) -> str:
35
+ if not isinstance(fact, dict):
36
+ return ""
37
+ source = escape(str(fact.get("source") or ""))
38
+ ref = fact.get("evidence_ref") or {}
39
+ url = escape(str(ref.get("source_url") or ""))
40
+ period = escape(str(ref.get("as_of") or ""))
41
+ label = " · ".join(value for value in (source, period) if value)
42
+ if url:
43
+ return (
44
+ f'<a href="{url}" target="_blank" style="color:{TEXT_MUTED};'
45
+ f'text-decoration:none;border-bottom:1px solid {BORDER};">{label}</a>'
46
+ )
47
+ return f'<span style="color:{TEXT_MUTED};">{label}</span>'
48
+
49
+
50
+ def _page_header(profile: dict, market: dict | None) -> None:
51
+ company = escape(str(profile.get("company_name") or profile.get("ticker") or ""))
52
+ ticker = escape(str(profile.get("ticker") or ""))
53
+ sector = escape(str((market or {}).get("sector") or ""))
54
+ industry = escape(str((market or {}).get("industry") or ""))
55
+ as_of = escape(str(profile.get("data_as_of") or ""))
56
+ meta = " · ".join(value for value in (ticker, sector, industry, as_of) if value)
57
+ st.markdown(
58
+ f'<div style="margin-bottom:18px;">'
59
+ f'<div style="font-size:{FS_PAGE};font-weight:700;color:{TEXT};'
60
+ f'letter-spacing:-0.02em;">{company}</div>'
61
+ f'<div style="font-size:{FS_META};color:{TEXT_MUTED};margin-top:3px;">{meta}</div>'
62
+ f'</div>',
63
+ unsafe_allow_html=True,
64
+ )
65
+
66
+
67
+ def _identity(profile: dict) -> None:
68
+ identity = profile.get("identity") or {}
69
+ one_liner = _fact_text(identity.get("one_liner"))
70
+ business_model = _fact_text(identity.get("business_model"))
71
+ if not one_liner and not business_model:
72
+ st.info(t("primer_identity_unavailable"))
73
+ return
74
+ source = identity.get("one_liner") or identity.get("business_model")
75
+ body = "".join(
76
+ (
77
+ f'<div style="font-size:1.02rem;font-weight:600;line-height:1.55;'
78
+ f'color:{TEXT};margin-bottom:8px;">{escape(one_liner)}</div>'
79
+ if one_liner
80
+ else "",
81
+ f'<div style="font-size:{FS_BODY};line-height:1.6;color:{TEXT_MUTED};">'
82
+ f'{escape(business_model)}</div>'
83
+ if business_model
84
+ else "",
85
+ )
86
+ )
87
+ st.markdown(
88
+ f'<div style="border-left:4px solid {BRAND_GREEN};padding:2px 0 2px 16px;">'
89
+ f'{body}<div style="font-size:{FS_META};margin-top:8px;">{_source_meta(source)}</div>'
90
+ f'</div>',
91
+ unsafe_allow_html=True,
92
+ )
93
+
94
+
95
+ def _business(profile: dict) -> None:
96
+ st.markdown(section_header(t("primer_business"), accent=BRAND_GREEN), unsafe_allow_html=True)
97
+ lines = profile.get("business_lines") or []
98
+ if not lines:
99
+ st.info(t("primer_business_unavailable"))
100
+ return
101
+
102
+ shares = [
103
+ (str(item.get("name") or ""), item.get("revenue_share_pct"))
104
+ for item in lines
105
+ if item.get("revenue_share_pct") is not None
106
+ ]
107
+ total = sum(float(value) for _, value in shares)
108
+ if len(shares) >= 2 and total <= 100.5:
109
+ import plotly.graph_objects as go
110
+
111
+ fig = go.Figure()
112
+ palette = ["#10b981", "#3b82f6", "#8b5cf6", "#f59e0b", "#64748b"]
113
+ for index, (name, value) in enumerate(shares):
114
+ fig.add_bar(
115
+ y=["Revenue mix"],
116
+ x=[float(value)],
117
+ name=name,
118
+ orientation="h",
119
+ text=f"{name} {float(value):.0f}%",
120
+ textposition="inside",
121
+ marker_color=palette[index % len(palette)],
122
+ hovertemplate=f"{escape(name)}: {float(value):.1f}%<extra></extra>",
123
+ )
124
+ if total < 99.5:
125
+ fig.add_bar(
126
+ y=["Revenue mix"],
127
+ x=[100 - total],
128
+ name=t("primer_other"),
129
+ orientation="h",
130
+ marker_color="#d1d5db",
131
+ hovertemplate=f"{t('primer_other')}: {100-total:.1f}%<extra></extra>",
132
+ )
133
+ fig.update_layout(
134
+ barmode="stack",
135
+ height=105,
136
+ margin=dict(l=0, r=0, t=5, b=5),
137
+ paper_bgcolor="rgba(0,0,0,0)",
138
+ plot_bgcolor="rgba(0,0,0,0)",
139
+ showlegend=False,
140
+ xaxis=dict(range=[0, 100], visible=False),
141
+ yaxis=dict(visible=False),
142
+ font=dict(family="Inter"),
143
+ )
144
+ st.plotly_chart(fig, use_container_width=True, config={"displayModeBar": False})
145
+
146
+ rows = []
147
+ for item in lines[:5]:
148
+ fact = item.get("description") or {}
149
+ framing = str(item.get("economics") or "")
150
+ if framing:
151
+ framing = f"AI · {framing}"
152
+ rows.append(
153
+ {
154
+ t("primer_engine"): item.get("name") or "",
155
+ t("primer_role"): _fact_text(fact),
156
+ t("primer_share"): (
157
+ f"{float(item['revenue_share_pct']):.1f}% · {item.get('share_period') or ''}"
158
+ if item.get("revenue_share_pct") is not None
159
+ else t("primer_not_disclosed")
160
+ ),
161
+ t("primer_trend"): t(
162
+ f"primer_trend_{item.get('trend') or 'not_disclosed'}"
163
+ ),
164
+ t("primer_framing"): framing,
165
+ }
166
+ )
167
+ st.dataframe(rows, use_container_width=True, hide_index=True)
168
+
169
+
170
+ def _geography(profile: dict) -> None:
171
+ st.markdown(section_header(t("primer_geography"), accent=BLUE), unsafe_allow_html=True)
172
+ exposures = profile.get("geographic_exposures") or []
173
+ if not exposures:
174
+ st.info(t("primer_geography_unavailable"))
175
+ return
176
+
177
+ quantitative = [
178
+ item for item in exposures if item.get("revenue_share_pct") is not None
179
+ ]
180
+ total = sum(float(item["revenue_share_pct"]) for item in quantitative)
181
+ if len(quantitative) >= 2 and total <= 100.5:
182
+ import plotly.graph_objects as go
183
+
184
+ fig = go.Figure()
185
+ palette = ["#3b82f6", "#10b981", "#8b5cf6", "#f59e0b", "#64748b"]
186
+ for index, item in enumerate(quantitative):
187
+ value = float(item["revenue_share_pct"])
188
+ name = str(item.get("name") or "")
189
+ fig.add_bar(
190
+ y=[t("primer_revenue_geography")],
191
+ x=[value],
192
+ name=name,
193
+ orientation="h",
194
+ text=f"{name} {value:.0f}%",
195
+ textposition="inside",
196
+ marker_color=palette[index % len(palette)],
197
+ hovertemplate=f"{escape(name)}: {value:.1f}%<extra></extra>",
198
+ )
199
+ fig.update_layout(
200
+ barmode="stack",
201
+ height=105,
202
+ margin=dict(l=0, r=0, t=5, b=5),
203
+ paper_bgcolor="rgba(0,0,0,0)",
204
+ plot_bgcolor="rgba(0,0,0,0)",
205
+ showlegend=False,
206
+ xaxis=dict(range=[0, max(100, total)], visible=False),
207
+ yaxis=dict(visible=False),
208
+ font=dict(family="Inter"),
209
+ )
210
+ st.plotly_chart(fig, use_container_width=True, config={"displayModeBar": False})
211
+
212
+ rows = [
213
+ {
214
+ t("primer_country_region"): item.get("name") or "",
215
+ t("primer_exposure"): ", ".join(
216
+ t(f"primer_exposure_{kind}")
217
+ for kind in (item.get("exposure_types") or [])
218
+ ),
219
+ t("primer_description"): _fact_text(item.get("description")),
220
+ t("primer_share"): (
221
+ f"{float(item['revenue_share_pct']):.1f}% · {item.get('period') or ''}"
222
+ if item.get("revenue_share_pct") is not None
223
+ else t("primer_qualitative")
224
+ ),
225
+ }
226
+ for item in exposures[:8]
227
+ ]
228
+ st.dataframe(rows, use_container_width=True, hide_index=True)
229
+
230
+
231
+ def _evolution(profile: dict) -> None:
232
+ st.markdown(section_header(t("primer_evolution"), accent=BRAND_GREEN), unsafe_allow_html=True)
233
+ trends = profile.get("annual_trends") or []
234
+ changes = profile.get("strategic_changes") or []
235
+ if not trends and not changes:
236
+ st.info(t("primer_evolution_unavailable"))
237
+ return
238
+ if trends:
239
+ from plotly.subplots import make_subplots
240
+ import plotly.graph_objects as go
241
+
242
+ periods = [str(item.get("period") or "") for item in trends]
243
+ specs = [
244
+ ("revenue_bn", t("primer_revenue"), "$B", 1),
245
+ ("operating_margin", t("primer_op_margin"), "%", 2),
246
+ ("free_cash_flow_bn", t("primer_fcf"), "$B", 3),
247
+ ("eps", "EPS", "$", 4),
248
+ ]
249
+ fig = make_subplots(rows=2, cols=2, subplot_titles=[item[1] for item in specs])
250
+ for key, _label, unit, col in specs:
251
+ values = [item.get(key) for item in trends]
252
+ display_values = [
253
+ value * 100 if key == "operating_margin" and value is not None else value
254
+ for value in values
255
+ ]
256
+ fig.add_trace(
257
+ go.Scatter(
258
+ x=periods,
259
+ y=display_values,
260
+ mode="lines+markers+text",
261
+ text=[
262
+ f"{value:.1f}{unit}" if value is not None else ""
263
+ for value in display_values
264
+ ],
265
+ textposition="top center",
266
+ line=dict(color=BRAND_GREEN, width=2),
267
+ marker=dict(size=6),
268
+ hovertemplate="%{x}: %{y:.2f}<extra></extra>",
269
+ showlegend=False,
270
+ ),
271
+ row=1 if col <= 2 else 2,
272
+ col=col if col <= 2 else col - 2,
273
+ )
274
+ fig.update_layout(
275
+ height=430,
276
+ margin=dict(l=20, r=20, t=45, b=20),
277
+ paper_bgcolor="rgba(0,0,0,0)",
278
+ plot_bgcolor="rgba(0,0,0,0)",
279
+ font=dict(family="Inter", size=11),
280
+ )
281
+ fig.update_xaxes(showgrid=False)
282
+ fig.update_yaxes(showgrid=True, gridcolor="#e5e7eb", zeroline=False)
283
+ st.plotly_chart(fig, use_container_width=True, config={"displayModeBar": False})
284
+
285
+ if changes:
286
+ rows = [
287
+ {
288
+ t("primer_period"): " → ".join(
289
+ value for value in (item.get("period_from"), item.get("period_to")) if value
290
+ ),
291
+ t("primer_change"): _fact_text(item.get("change")),
292
+ t("primer_implication"): f"AI · {item.get('implication') or ''}",
293
+ }
294
+ for item in changes[:3]
295
+ ]
296
+ st.dataframe(rows, use_container_width=True, hide_index=True)
297
+
298
+
299
+ def _attention(profile: dict) -> None:
300
+ st.markdown(section_header(t("primer_attention"), accent=AI_COLOR), unsafe_allow_html=True)
301
+ themes = profile.get("attention_themes") or []
302
+ if not themes:
303
+ st.info(t("primer_attention_unavailable"))
304
+ return
305
+ for item in themes[:3]:
306
+ mentions = item.get("call_mentions") or []
307
+ trajectory = " → ".join(
308
+ f"{point.get('period')}: {point.get('mentions', 0)}" for point in mentions
309
+ ) or t("primer_insufficient")
310
+ evidence = item.get("evidence") or {}
311
+ direction_label = t(
312
+ "primer_direction_" + str(item.get("direction") or "insufficient")
313
+ )
314
+ st.markdown(
315
+ f'<div style="border-bottom:1px solid {BORDER};padding:12px 0;">'
316
+ f'<div style="display:flex;justify-content:space-between;gap:10px;flex-wrap:wrap;">'
317
+ f'<div style="font-size:{FS_BODY};font-weight:600;color:{TEXT};">'
318
+ f'{escape(str(item.get("theme") or ""))}</div>'
319
+ f'<span style="background:{AI_BG};border:1px solid {AI_BORDER};color:{AI_COLOR};'
320
+ f'border-radius:4px;padding:1px 7px;font-size:0.65rem;font-weight:600;">'
321
+ f'AI · experimental</span></div>'
322
+ f'<div style="font-size:{FS_BODY};color:{TEXT_MUTED};margin-top:4px;">'
323
+ f'{escape(str(item.get("why_it_matters") or ""))}</div>'
324
+ f'<div style="font-size:{FS_META};color:{TEXT_MUTED};margin-top:7px;">'
325
+ f'Q&amp;A: {escape(trajectory)} · {t("primer_news_count").format(count=int(item.get("retrieved_news_count") or 0))}'
326
+ f' · {escape(direction_label)}</div>'
327
+ f'<div style="font-size:{FS_META};margin-top:4px;">{_source_meta(evidence)}</div>'
328
+ f'</div>',
329
+ unsafe_allow_html=True,
330
+ )
331
+
332
+
333
+ def _market(profile: dict, market: dict | None) -> None:
334
+ st.markdown(section_header(t("primer_price"), accent=BLUE), unsafe_allow_html=True)
335
+ if not market or not market.get("history"):
336
+ st.info(t("primer_price_unavailable"))
337
+ return
338
+ import plotly.graph_objects as go
339
+
340
+ history = market["history"]
341
+ ticker = str(market.get("ticker") or profile.get("ticker") or "")
342
+ benchmark = str(market.get("benchmark") or "SPY")
343
+ series = [ticker, "SPY"] + ([] if benchmark == "SPY" else [benchmark])
344
+ colors = {ticker: BRAND_GREEN, "SPY": "#64748b", benchmark: "#3b82f6"}
345
+ fig = go.Figure()
346
+ for symbol in series:
347
+ fig.add_trace(
348
+ go.Scatter(
349
+ x=[item["date"] for item in history],
350
+ y=[item.get(symbol) for item in history],
351
+ mode="lines",
352
+ name=symbol,
353
+ line=dict(color=colors.get(symbol, "#8b5cf6"), width=2 if symbol == ticker else 1.5),
354
+ hovertemplate=f"{symbol} %{{y:.1f}}<extra></extra>",
355
+ )
356
+ )
357
+ event_dates = []
358
+ event_values = []
359
+ event_symbols = []
360
+ event_colors = []
361
+ event_hover = []
362
+ ticker_points = [
363
+ item for item in history if item.get("date") and item.get(ticker) is not None
364
+ ]
365
+ for event in market.get("events") or []:
366
+ try:
367
+ event_day = datetime.fromisoformat(str(event.get("date") or "")[:10])
368
+ nearest = min(
369
+ ticker_points,
370
+ key=lambda point: abs(
371
+ (
372
+ datetime.fromisoformat(str(point["date"])[:10])
373
+ - event_day
374
+ ).days
375
+ ),
376
+ )
377
+ except (TypeError, ValueError):
378
+ continue
379
+ excess = float(event.get("sector_excess_pct") or 0)
380
+ event_dates.append(nearest["date"])
381
+ event_values.append(nearest[ticker])
382
+ event_symbols.append("triangle-up" if excess >= 0 else "triangle-down")
383
+ event_colors.append(BRAND_GREEN if excess >= 0 else "#dc2626")
384
+ headline_text = (
385
+ t("primer_no_verified_catalyst")
386
+ if event.get("attribution_status") == "NO_VERIFIED_CATALYST"
387
+ else str(event.get("headline") or t("primer_no_verified_catalyst"))
388
+ )
389
+ headline = escape(headline_text)
390
+ event_hover.append(
391
+ f"{escape(str(event.get('date') or ''))}<br>"
392
+ f"{escape(t('primer_relative_move'))}: {excess:+.1f}%<br>"
393
+ f"{headline}"
394
+ )
395
+ if event_dates:
396
+ fig.add_trace(
397
+ go.Scatter(
398
+ x=event_dates,
399
+ y=event_values,
400
+ mode="markers",
401
+ name=t("primer_price_events"),
402
+ marker=dict(
403
+ size=10,
404
+ symbol=event_symbols,
405
+ color=event_colors,
406
+ line=dict(color=BORDER, width=1),
407
+ ),
408
+ text=event_hover,
409
+ hovertemplate="%{text}<extra></extra>",
410
+ )
411
+ )
412
+ fig.update_layout(
413
+ height=330,
414
+ margin=dict(l=20, r=20, t=10, b=20),
415
+ paper_bgcolor="rgba(0,0,0,0)",
416
+ plot_bgcolor="rgba(0,0,0,0)",
417
+ font=dict(family="Inter"),
418
+ legend=dict(orientation="h", y=1.05, x=0),
419
+ yaxis_title=t("primer_indexed_100"),
420
+ hovermode="x unified",
421
+ )
422
+ fig.update_xaxes(showgrid=False)
423
+ fig.update_yaxes(gridcolor="#e5e7eb", zeroline=False)
424
+ st.plotly_chart(fig, use_container_width=True, config={"displayModeBar": False})
425
+ st.caption(t("primer_price_caption").format(benchmark=benchmark))
426
+
427
+ events = market.get("events") or []
428
+ if events:
429
+ for event in events:
430
+ direction = "+" if float(event.get("sector_excess_pct") or 0) >= 0 else ""
431
+ headline_text = (
432
+ t("primer_no_verified_catalyst")
433
+ if event.get("attribution_status") == "NO_VERIFIED_CATALYST"
434
+ else str(event.get("headline") or "")
435
+ )
436
+ headline = escape(headline_text)
437
+ url = escape(str(event.get("url") or ""))
438
+ headline_html = (
439
+ f'<a href="{url}" target="_blank" style="color:{TEXT};text-decoration:none;'
440
+ f'border-bottom:1px solid {BORDER};">{headline}</a>'
441
+ if url else f'<span style="color:{TEXT_MUTED};">{headline}</span>'
442
+ )
443
+ st.markdown(
444
+ f'<div style="display:grid;grid-template-columns:86px 90px 1fr;gap:12px;'
445
+ f'padding:8px 0;border-bottom:1px solid {BORDER};font-size:{FS_META};">'
446
+ f'<span>{escape(str(event.get("date") or ""))}</span>'
447
+ f'<strong>{direction}{float(event.get("sector_excess_pct") or 0):.1f}%</strong>'
448
+ f'<span>{headline_html}</span></div>',
449
+ unsafe_allow_html=True,
450
+ )
451
+
452
+
453
+ def _news(news_items: list[dict], news_error: str | None) -> None:
454
+ st.markdown(section_header(t("primer_news"), accent=BLUE), unsafe_allow_html=True)
455
+ if not news_items:
456
+ st.info(news_error or t("primer_news_unavailable"))
457
+ return
458
+ for item in news_items[:5]:
459
+ title = escape(str(item.get("title") or ""))
460
+ url = escape(str(item.get("url") or ""))
461
+ date = escape(str(item.get("published_date") or "")[:10])
462
+ st.markdown(
463
+ f'<div style="padding:9px 0;border-bottom:1px solid {BORDER};">'
464
+ f'<div style="font-size:{FS_META};color:{TEXT_MUTED};">{date}</div>'
465
+ f'<a href="{url}" target="_blank" style="font-size:{FS_BODY};font-weight:600;'
466
+ f'color:{TEXT};text-decoration:none;">{title}</a></div>',
467
+ unsafe_allow_html=True,
468
+ )
469
+
470
+
471
+ def _watch(profile: dict) -> None:
472
+ st.markdown(section_header(t("primer_watch"), accent="#f59e0b"), unsafe_allow_html=True)
473
+ values = profile.get("watch_variables") or []
474
+ if not values:
475
+ st.info(t("primer_watch_unavailable"))
476
+ return
477
+ rows = [
478
+ {
479
+ t("primer_variable"): item.get("variable") or "",
480
+ t("primer_why"): f"AI · {item.get('why_it_matters') or ''}",
481
+ t("primer_next_datapoint"): item.get("next_datapoint") or "",
482
+ t("primer_alert"): item.get("alert_signal") or "",
483
+ }
484
+ for item in values[:4]
485
+ ]
486
+ st.dataframe(rows, use_container_width=True, hide_index=True)
487
+
488
+
489
+ def render(
490
+ profile: dict,
491
+ ticker: str,
492
+ *,
493
+ market: dict | None = None,
494
+ news_items: list[dict] | None = None,
495
+ news_error: str | None = None,
496
+ ) -> None:
497
+ _page_header(profile, market)
498
+ _identity(profile)
499
+ _business(profile)
500
+ _geography(profile)
501
+ _evolution(profile)
502
+ _attention(profile)
503
+ _market(profile, market)
504
+ _news(news_items or [], news_error)
505
+ _watch(profile)
506
+
507
+ coverage = profile.get("source_coverage") or {}
508
+ with st.expander(t("primer_sources"), expanded=False):
509
+ st.json(
510
+ {
511
+ "profile_status": profile.get("status"),
512
+ "data_as_of": profile.get("data_as_of"),
513
+ "source_coverage": coverage,
514
+ "evidence_coverage": profile.get("evidence_coverage") or {},
515
+ "price_source": (market or {}).get("price_source"),
516
+ "price_as_of": (market or {}).get("as_of"),
517
+ }
518
+ )
519
+ evidence_rows = []
520
+
521
+ def _walk(value):
522
+ if isinstance(value, dict):
523
+ if value.get("verification_status") == "VERIFIED" and value.get("evidence_snippet"):
524
+ ref = value.get("evidence_ref") or {}
525
+ evidence_rows.append(
526
+ {
527
+ "Claim": value.get("text") or "",
528
+ "Source": value.get("source") or "",
529
+ "As of": ref.get("as_of") or "",
530
+ "Evidence": value.get("evidence_snippet") or "",
531
+ "URL": ref.get("source_url") or "",
532
+ }
533
+ )
534
+ for nested in value.values():
535
+ _walk(nested)
536
+ elif isinstance(value, list):
537
+ for nested in value:
538
+ _walk(nested)
539
+
540
+ _walk(profile)
541
+ if evidence_rows:
542
+ st.dataframe(evidence_rows, use_container_width=True, hide_index=True)
dashboard/i18n.py CHANGED
@@ -279,6 +279,7 @@ STRINGS: dict[str, dict[str, str]] = {
279
  },
280
 
281
  # ── Flat navigation (Decision Stack) ───────────────────────────────────
 
282
  "nav_verdict": {"en": "Verdict", "fr": "Verdict", "es": "Veredicto", "de": "Fazit"},
283
  "nav_chat": {"en": "Ask AI", "fr": "Demander à l'IA", "es": "Preguntar a la IA", "de": "KI fragen"},
284
  "nav_signals": {"en": "Signals", "fr": "Signaux", "es": "Señales", "de": "Signale"},
@@ -543,6 +544,73 @@ STRINGS: dict[str, dict[str, str]] = {
543
  "band_strong_bear": {"en": "Strongly Bearish", "fr": "Très baissier", "es": "Muy bajista", "de": "Stark bärisch"},
544
 
545
  # ── Financials screen ──────────────────────────────────────────────────
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
546
  "fin_kpi": {"en": "Latest quarter", "fr": "Dernier trimestre", "es": "Último trimestre", "de": "Letztes Quartal"},
547
  "fin_next_guidance": {"en": "Next period guidance", "fr": "Guidance du prochain trimestre", "es": "Guidance del próximo período", "de": "Guidance nächste Periode"},
548
  "fin_tab_revenue": {"en": "Revenue & Growth", "fr": "CA & croissance", "es": "Ingresos y crecimiento", "de": "Umsatz & Wachstum"},
 
279
  },
280
 
281
  # ── Flat navigation (Decision Stack) ───────────────────────────────────
282
+ "nav_company": {"en": "Company Primer", "fr": "Profil société", "es": "Perfil de empresa", "de": "Unternehmensprofil"},
283
  "nav_verdict": {"en": "Verdict", "fr": "Verdict", "es": "Veredicto", "de": "Fazit"},
284
  "nav_chat": {"en": "Ask AI", "fr": "Demander à l'IA", "es": "Preguntar a la IA", "de": "KI fragen"},
285
  "nav_signals": {"en": "Signals", "fr": "Signaux", "es": "Señales", "de": "Signale"},
 
544
  "band_strong_bear": {"en": "Strongly Bearish", "fr": "Très baissier", "es": "Muy bajista", "de": "Stark bärisch"},
545
 
546
  # ── Financials screen ──────────────────────────────────────────────────
547
+ # Company Primer
548
+ "primer_generating": {"en": "Building the Company Primer", "fr": "Création du profil société", "es": "Creando el perfil de empresa", "de": "Unternehmensprofil wird erstellt"},
549
+ "primer_retry": {"en": "Retry Company Primer", "fr": "Relancer le profil société", "es": "Reintentar el perfil de empresa", "de": "Unternehmensprofil erneut erstellen"},
550
+ "primer_refresh": {"en": "Refresh profile", "fr": "Actualiser le profil", "es": "Actualizar perfil", "de": "Profil aktualisieren"},
551
+ "primer_refresh_failed": {"en": "Profile refresh failed; the cached version remains available.", "fr": "L’actualisation du profil a échoué ; la version en cache reste disponible.", "es": "La actualización del perfil falló; la versión en caché sigue disponible.", "de": "Die Profilaktualisierung ist fehlgeschlagen; die zwischengespeicherte Version bleibt verfügbar."},
552
+ "primer_loading_market": {"en": "Loading current market context…", "fr": "Chargement du contexte de marché actuel…", "es": "Cargando el contexto actual de mercado…", "de": "Aktueller Marktkontext wird geladen…"},
553
+ "primer_backfill_warning": {"en": "Company-profile source coverage is incomplete. Run `python ingest.py {ticker}` to backfill the 10-K Business and segment/geography sections.", "fr": "La couverture des sources du profil est incomplète. Lancez `python ingest.py {ticker}` pour récupérer les sections Business et segments/géographies du 10-K.", "es": "La cobertura de fuentes del perfil está incompleta. Ejecute `python ingest.py {ticker}` para recuperar las secciones Business y segmentos/geografía del 10-K.", "de": "Die Quellenabdeckung des Unternehmensprofils ist unvollständig. Führen Sie `python ingest.py {ticker}` aus, um die 10-K-Abschnitte Business und Segmente/Geografie nachzuladen."},
554
+ "primer_identity_unavailable": {"en": "The business description is not yet available from verified sources.", "fr": "La description de l’activité n’est pas encore disponible dans les sources vérifiées.", "es": "La descripción del negocio aún no está disponible en fuentes verificadas.", "de": "Die Geschäftsbeschreibung ist aus verifizierten Quellen noch nicht verfügbar."},
555
+ "primer_business": {"en": "How it makes money", "fr": "Comment l’entreprise gagne de l’argent", "es": "Cómo gana dinero", "de": "Wie das Unternehmen Geld verdient"},
556
+ "primer_business_unavailable": {"en": "No verified business-line disclosure is available.", "fr": "Aucune ventilation vérifiée des activités n’est disponible.", "es": "No hay desglose verificado de líneas de negocio.", "de": "Keine verifizierte Aufschlüsselung der Geschäftsbereiche verfügbar."},
557
+ "primer_geography": {"en": "Where it is exposed", "fr": "Où l’entreprise est exposée", "es": "Dónde está expuesta", "de": "Geografische Exponierung"},
558
+ "primer_geography_unavailable": {"en": "No verified geographic disclosure is available.", "fr": "Aucune information géographique vérifiée n’est disponible.", "es": "No hay información geográfica verificada.", "de": "Keine verifizierten geografischen Angaben verfügbar."},
559
+ "primer_evolution": {"en": "Three-year evolution", "fr": "Évolution sur trois ans", "es": "Evolución a tres años", "de": "Dreijahresentwicklung"},
560
+ "primer_evolution_unavailable": {"en": "Verified multi-year history is unavailable.", "fr": "L’historique pluriannuel vérifié est indisponible.", "es": "El historial plurianual verificado no está disponible.", "de": "Eine verifizierte Mehrjahreshistorie ist nicht verfügbar."},
561
+ "primer_attention": {"en": "What commands attention", "fr": "Ce qui concentre l’attention", "es": "Lo que concentra la atención", "de": "Was Aufmerksamkeit bindet"},
562
+ "primer_attention_unavailable": {"en": "Not enough verified call evidence to rank attention themes.", "fr": "Les calls vérifiés sont insuffisants pour classer les thèmes d’attention.", "es": "No hay suficiente evidencia verificada de calls para clasificar temas.", "de": "Nicht genügend verifizierte Call-Evidenz für eine Themenrangfolge."},
563
+ "primer_price": {"en": "Price and associated events", "fr": "Cours et événements associés", "es": "Precio y eventos asociados", "de": "Kurs und zugeordnete Ereignisse"},
564
+ "primer_price_unavailable": {"en": "Relative price history is unavailable.", "fr": "L’historique de cours relatif est indisponible.", "es": "El historial de precio relativo no está disponible.", "de": "Relative Kurshistorie ist nicht verfügbar."},
565
+ "primer_price_caption": {"en": "Three-year adjusted prices indexed to 100. Events are date associations, not proven causes. Sector benchmark: {benchmark}.", "fr": "Cours ajustés sur trois ans, base 100. Les événements sont des associations de dates, pas des causalités prouvées. Benchmark sectoriel : {benchmark}.", "es": "Precios ajustados a tres años, base 100. Los eventos son asociaciones de fechas, no causas probadas. Benchmark sectorial: {benchmark}.", "de": "Dreijährige adjustierte Kurse, indexiert auf 100. Ereignisse sind zeitliche Zuordnungen, keine bewiesenen Ursachen. Sektor-Benchmark: {benchmark}."},
566
+ "primer_price_events": {"en": "Relative-move events", "fr": "Événements de performance relative", "es": "Eventos de movimiento relativo", "de": "Ereignisse relativer Kursbewegungen"},
567
+ "primer_relative_move": {"en": "Relative move", "fr": "Mouvement relatif", "es": "Movimiento relativo", "de": "Relative Bewegung"},
568
+ "primer_indexed_100": {"en": "Indexed to 100", "fr": "Base 100", "es": "Base 100", "de": "Indexiert auf 100"},
569
+ "primer_no_verified_catalyst": {"en": "No verified catalyst found", "fr": "Aucun catalyseur vérifié trouvé", "es": "No se encontró un catalizador verificado", "de": "Kein verifizierter Auslöser gefunden"},
570
+ "primer_news": {"en": "Recent news", "fr": "Actualités récentes", "es": "Noticias recientes", "de": "Aktuelle Nachrichten"},
571
+ "primer_news_unavailable": {"en": "Recent news is unavailable.", "fr": "Les actualités récentes sont indisponibles.", "es": "Las noticias recientes no están disponibles.", "de": "Aktuelle Nachrichten sind nicht verfügbar."},
572
+ "primer_watch": {"en": "What to watch", "fr": "À surveiller", "es": "Qué vigilar", "de": "Worauf zu achten ist"},
573
+ "primer_watch_unavailable": {"en": "No verified monitoring variables are available.", "fr": "Aucune variable de suivi vérifiée n’est disponible.", "es": "No hay variables de seguimiento verificadas.", "de": "Keine verifizierten Beobachtungsvariablen verfügbar."},
574
+ "primer_sources": {"en": "Coverage and sources", "fr": "Couverture et sources", "es": "Cobertura y fuentes", "de": "Abdeckung und Quellen"},
575
+ "primer_engine": {"en": "Engine", "fr": "Moteur", "es": "Motor", "de": "Treiber"},
576
+ "primer_role": {"en": "Economic role", "fr": "Rôle économique", "es": "Función económica", "de": "Wirtschaftliche Rolle"},
577
+ "primer_share": {"en": "Revenue share", "fr": "Part du CA", "es": "Parte de ingresos", "de": "Umsatzanteil"},
578
+ "primer_trend": {"en": "Trend", "fr": "Tendance", "es": "Tendencia", "de": "Trend"},
579
+ "primer_trend_growing": {"en": "Growing", "fr": "En croissance", "es": "En crecimiento", "de": "Wachsend"},
580
+ "primer_trend_stable": {"en": "Stable", "fr": "Stable", "es": "Estable", "de": "Stabil"},
581
+ "primer_trend_declining": {"en": "Declining", "fr": "En baisse", "es": "En descenso", "de": "Rückläufig"},
582
+ "primer_trend_mixed": {"en": "Mixed", "fr": "Contrastée", "es": "Mixta", "de": "Gemischt"},
583
+ "primer_trend_not_disclosed": {"en": "Not disclosed", "fr": "Non communiqué", "es": "No publicado", "de": "Nicht offengelegt"},
584
+ "primer_framing": {"en": "Analyst framing", "fr": "Lecture analyste", "es": "Lectura del analista", "de": "Analysteneinordnung"},
585
+ "primer_other": {"en": "Other / undisclosed", "fr": "Autres / non communiqué", "es": "Otros / no publicado", "de": "Sonstige / nicht offengelegt"},
586
+ "primer_not_disclosed": {"en": "Not disclosed", "fr": "Non communiqué", "es": "No publicado", "de": "Nicht offengelegt"},
587
+ "primer_revenue_geography": {"en": "Revenue geography", "fr": "Répartition géographique du CA", "es": "Geografía de ingresos", "de": "Umsatzgeografie"},
588
+ "primer_country_region": {"en": "Country / region", "fr": "Pays / région", "es": "País / región", "de": "Land / Region"},
589
+ "primer_exposure": {"en": "Exposure", "fr": "Exposition", "es": "Exposición", "de": "Exponierung"},
590
+ "primer_exposure_revenue": {"en": "Revenue", "fr": "Chiffre d’affaires", "es": "Ingresos", "de": "Umsatz"},
591
+ "primer_exposure_operations": {"en": "Operations", "fr": "Opérations", "es": "Operaciones", "de": "Betrieb"},
592
+ "primer_exposure_supply_chain": {"en": "Supply chain", "fr": "Chaîne d’approvisionnement", "es": "Cadena de suministro", "de": "Lieferkette"},
593
+ "primer_exposure_regulation": {"en": "Regulation", "fr": "Réglementation", "es": "Regulación", "de": "Regulierung"},
594
+ "primer_description": {"en": "Description", "fr": "Description", "es": "Descripción", "de": "Beschreibung"},
595
+ "primer_qualitative": {"en": "Qualitative", "fr": "Qualitatif", "es": "Cualitativo", "de": "Qualitativ"},
596
+ "primer_revenue": {"en": "Revenue", "fr": "Chiffre d’affaires", "es": "Ingresos", "de": "Umsatz"},
597
+ "primer_op_margin": {"en": "Operating margin", "fr": "Marge opérationnelle", "es": "Margen operativo", "de": "Operative Marge"},
598
+ "primer_fcf": {"en": "Free cash flow", "fr": "Free cash flow", "es": "Flujo de caja libre", "de": "Free Cashflow"},
599
+ "primer_period": {"en": "Period", "fr": "Période", "es": "Período", "de": "Zeitraum"},
600
+ "primer_change": {"en": "What changed", "fr": "Ce qui a changé", "es": "Qué cambió", "de": "Was sich geändert hat"},
601
+ "primer_implication": {"en": "Implication", "fr": "Implication", "es": "Implicación", "de": "Implikation"},
602
+ "primer_insufficient": {"en": "Insufficient history", "fr": "Historique insuffisant", "es": "Historial insuficiente", "de": "Unzureichende Historie"},
603
+ "primer_news_count": {"en": "{count} retrieved news item(s)", "fr": "{count} actualité(s) récupérée(s)", "es": "{count} noticia(s) recuperada(s)", "de": "{count} abgerufene Nachricht(en)"},
604
+ "primer_direction_rising": {"en": "Rising", "fr": "En hausse", "es": "En aumento", "de": "Zunehmend"},
605
+ "primer_direction_stable": {"en": "Stable", "fr": "Stable", "es": "Estable", "de": "Stabil"},
606
+ "primer_direction_falling": {"en": "Falling", "fr": "En baisse", "es": "En descenso", "de": "Abnehmend"},
607
+ "primer_direction_new": {"en": "New", "fr": "Nouveau", "es": "Nuevo", "de": "Neu"},
608
+ "primer_direction_insufficient": {"en": "Insufficient history", "fr": "Historique insuffisant", "es": "Historial insuficiente", "de": "Unzureichende Historie"},
609
+ "primer_variable": {"en": "Variable", "fr": "Variable", "es": "Variable", "de": "Variable"},
610
+ "primer_why": {"en": "Why it matters", "fr": "Pourquoi c’est important", "es": "Por qué importa", "de": "Warum relevant"},
611
+ "primer_next_datapoint": {"en": "Next datapoint", "fr": "Prochain datapoint", "es": "Próximo dato", "de": "Nächster Datenpunkt"},
612
+ "primer_alert": {"en": "Alert signal", "fr": "Signal d’alerte", "es": "Señal de alerta", "de": "Warnsignal"},
613
+
614
  "fin_kpi": {"en": "Latest quarter", "fr": "Dernier trimestre", "es": "Último trimestre", "de": "Letztes Quartal"},
615
  "fin_next_guidance": {"en": "Next period guidance", "fr": "Guidance du prochain trimestre", "es": "Guidance del próximo período", "de": "Guidance nächste Periode"},
616
  "fin_tab_revenue": {"en": "Revenue & Growth", "fr": "CA & croissance", "es": "Ingresos y crecimiento", "de": "Umsatz & Wachstum"},
dashboard/nav.py CHANGED
@@ -1,7 +1,7 @@
1
  """Data-driven sidebar navigation for Amplegest — institutional style.
2
 
3
- Four primary destinations, Chat promoted right after the decision view:
4
- PM Flash → Ask AI → Evidence & Deltas → Financials
5
 
6
  Each item has a stable ``key`` decoupled from its display label,
7
  so renaming a section never breaks routing.
@@ -12,10 +12,11 @@ from dashboard.i18n import t
12
 
13
  # ── Per-section color palette (accent, pastel bg, pastel border) ──────────────
14
  # Deliberately narrow: emerald is the single brand/active accent shared by the
15
- # three data screens (Verdict, Signals, Financials); violet is reserved
16
  # exclusively for the AI tool (Chat), so a glance at the sidebar tells you
17
  # "data screen" from "AI screen" without six competing hues to parse.
18
  SECTION_COLORS: dict[str, tuple[str, str, str]] = {
 
19
  "verdict": ("#10b981", "#ecfdf5", "#a7f3d0"), # emerald (brand)
20
  "signals": ("#10b981", "#ecfdf5", "#a7f3d0"), # emerald (brand)
21
  "financials": ("#10b981", "#ecfdf5", "#a7f3d0"), # emerald (brand)
@@ -23,9 +24,10 @@ SECTION_COLORS: dict[str, tuple[str, str, str]] = {
23
  }
24
 
25
  # ── Navigation items ──────────────────────────────────────────────────────────
26
- # Chat sits right after the decision view (PM Flash) — it's the fastest path
27
- # from "here's the verdict" to "let me interrogate the evidence".
28
  NAV_ITEMS: list[dict] = [
 
29
  {"key": "verdict", "i18n_key": "nav_verdict", "display_label": "PM Flash"},
30
  {"key": "chat", "i18n_key": "nav_chat"},
31
  {"key": "signals", "i18n_key": "nav_signals", "display_label": "Evidence & Deltas"},
@@ -45,6 +47,7 @@ LEGACY_NAV: dict[str, str] = {
45
  "earnings_call": "signals",
46
  "quality_tone": "signals",
47
  "portfolio": "verdict",
 
48
  }
49
 
50
 
@@ -56,6 +59,7 @@ def section_state(brief: dict | None) -> dict[str, str]:
56
  """
57
  has_brief = bool(brief)
58
  return {
 
59
  "verdict": "ready" if has_brief else "—",
60
  "signals": "ready" if has_brief else "—",
61
  "financials": "ready", # reads SQLite directly, brief optional
 
1
  """Data-driven sidebar navigation for Amplegest — institutional style.
2
 
3
+ Five primary destinations, from company onboarding to decision and audit:
4
+ Company Primer → PM Flash → Ask AI → Evidence & Deltas → Financials
5
 
6
  Each item has a stable ``key`` decoupled from its display label,
7
  so renaming a section never breaks routing.
 
12
 
13
  # ── Per-section color palette (accent, pastel bg, pastel border) ──────────────
14
  # Deliberately narrow: emerald is the single brand/active accent shared by the
15
+ # four data screens (Company, Verdict, Signals, Financials); violet is reserved
16
  # exclusively for the AI tool (Chat), so a glance at the sidebar tells you
17
  # "data screen" from "AI screen" without six competing hues to parse.
18
  SECTION_COLORS: dict[str, tuple[str, str, str]] = {
19
+ "company": ("#10b981", "#ecfdf5", "#a7f3d0"), # emerald (brand)
20
  "verdict": ("#10b981", "#ecfdf5", "#a7f3d0"), # emerald (brand)
21
  "signals": ("#10b981", "#ecfdf5", "#a7f3d0"), # emerald (brand)
22
  "financials": ("#10b981", "#ecfdf5", "#a7f3d0"), # emerald (brand)
 
24
  }
25
 
26
  # ── Navigation items ──────────────────────────────────────────────────────────
27
+ # Company Primer onboards the analyst; Chat stays immediately after PM Flash
28
+ # for fast interrogation of the decision evidence.
29
  NAV_ITEMS: list[dict] = [
30
+ {"key": "company", "i18n_key": "nav_company"},
31
  {"key": "verdict", "i18n_key": "nav_verdict", "display_label": "PM Flash"},
32
  {"key": "chat", "i18n_key": "nav_chat"},
33
  {"key": "signals", "i18n_key": "nav_signals", "display_label": "Evidence & Deltas"},
 
47
  "earnings_call": "signals",
48
  "quality_tone": "signals",
49
  "portfolio": "verdict",
50
+ "overview": "company",
51
  }
52
 
53
 
 
59
  """
60
  has_brief = bool(brief)
61
  return {
62
+ "company": "ready",
63
  "verdict": "ready" if has_brief else "—",
64
  "signals": "ready" if has_brief else "—",
65
  "financials": "ready", # reads SQLite directly, brief optional
ingest.py CHANGED
@@ -72,7 +72,18 @@ def ingest(ticker: str, full: bool = False) -> None:
72
  # --- Delta check ---
73
  if not full:
74
  transcript_resolved = get_section(edgar.ticker, edgar.period, "transcript")
75
- if edgar.period in existing_metric_periods and transcript_resolved is not None:
 
 
 
 
 
 
 
 
 
 
 
76
  print(f"[ingest] SKIP {label} (already ingested)")
77
  skipped += 1
78
  continue
@@ -148,6 +159,17 @@ def ingest(ticker: str, full: bool = False) -> None:
148
  upsert_section(edgar.ticker, edgar.period, edgar.form_type, "mda", edgar.mda_text)
149
  if edgar.risk_factors_text:
150
  upsert_section(edgar.ticker, edgar.period, edgar.form_type, "risk_factors", edgar.risk_factors_text)
 
 
 
 
 
 
 
 
 
 
 
151
 
152
  embed_and_store_filing(
153
  ticker=edgar.ticker,
@@ -159,6 +181,8 @@ def ingest(ticker: str, full: bool = False) -> None:
159
  form_type=edgar.form_type,
160
  accession=edgar.accession,
161
  source_url=edgar.source_url,
 
 
162
  )
163
 
164
  # Always mark transcript resolution — even empty — so future delta runs skip the AV call.
 
72
  # --- Delta check ---
73
  if not full:
74
  transcript_resolved = get_section(edgar.ticker, edgar.period, "transcript")
75
+ profile_sections_resolved = (
76
+ edgar.form_type != "10-K"
77
+ or (
78
+ get_section(edgar.ticker, edgar.period, "business") is not None
79
+ and get_section(edgar.ticker, edgar.period, "segments_geography") is not None
80
+ )
81
+ )
82
+ if (
83
+ edgar.period in existing_metric_periods
84
+ and transcript_resolved is not None
85
+ and profile_sections_resolved
86
+ ):
87
  print(f"[ingest] SKIP {label} (already ingested)")
88
  skipped += 1
89
  continue
 
159
  upsert_section(edgar.ticker, edgar.period, edgar.form_type, "mda", edgar.mda_text)
160
  if edgar.risk_factors_text:
161
  upsert_section(edgar.ticker, edgar.period, edgar.form_type, "risk_factors", edgar.risk_factors_text)
162
+ if edgar.form_type == "10-K":
163
+ # Store empty results too: this records that the profile sections
164
+ # were attempted and prevents endless delta backfills.
165
+ upsert_section(edgar.ticker, edgar.period, edgar.form_type, "business", edgar.business_text)
166
+ upsert_section(
167
+ edgar.ticker,
168
+ edgar.period,
169
+ edgar.form_type,
170
+ "segments_geography",
171
+ edgar.segments_geography_text,
172
+ )
173
 
174
  embed_and_store_filing(
175
  ticker=edgar.ticker,
 
181
  form_type=edgar.form_type,
182
  accession=edgar.accession,
183
  source_url=edgar.source_url,
184
+ business_text=edgar.business_text,
185
+ segments_geography_text=edgar.segments_geography_text,
186
  )
187
 
188
  # Always mark transcript resolution — even empty — so future delta runs skip the AV call.
ingestion/edgar.py CHANGED
@@ -24,6 +24,8 @@ class EdgarData:
24
  free_cash_flow: Optional[float]
25
  mda_text: str
26
  risk_factors_text: str
 
 
27
  shares_diluted: Optional[float] = None
28
  effective_tax_rate: Optional[float] = None
29
  interest_expense: Optional[float] = None
@@ -703,15 +705,29 @@ def compute_metrics_for_accn(
703
 
704
  _ITEM_7 = re.compile(r'^\s*item\s+7\b(?!\s*a)', re.IGNORECASE | re.MULTILINE)
705
  _ITEM_7A = re.compile(r'^\s*item\s+7\s*a\b', re.IGNORECASE | re.MULTILINE)
 
706
  _ITEM_1A = re.compile(r'^\s*item\s+1\s*a\b', re.IGNORECASE | re.MULTILINE)
707
  _ITEM_1B = re.compile(r'^\s*item\s+1\s*b\b', re.IGNORECASE | re.MULTILINE)
708
  _ITEM_2_Q = re.compile(r'^\s*item\s+2\b', re.IGNORECASE | re.MULTILINE)
709
  _ITEM_3_Q = re.compile(r'^\s*item\s+3\b', re.IGNORECASE | re.MULTILINE)
 
 
710
 
711
  _MDA_CAP = 80_000
712
  _RISK_CAP = 40_000
 
 
 
713
  _FALLBACK_WIN = 60_000
714
 
 
 
 
 
 
 
 
 
715
 
716
  def _extract_item_section(text: str, start_re: re.Pattern, end_re: re.Pattern, cap: int) -> str:
717
  """Extract text from the last occurrence of start_re up to the first end_re after it."""
@@ -737,8 +753,65 @@ def _extract_risk_factors(text: str, form_type: str = "10-K") -> str:
737
  return _extract_item_section(text, _ITEM_1A, end_re, _RISK_CAP)
738
 
739
 
740
- def get_filing_text_for_accn(cik: str, accn: str, primary_doc: str, form_type: str = "10-Q") -> tuple[str, str]:
741
- """Returns (mda_text, risk_factors_text) for a specific accession."""
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
742
  accn_clean = accn.replace("-", "")
743
  cik_int = int(cik)
744
  doc_url = f"https://www.sec.gov/Archives/edgar/data/{cik_int}/{accn_clean}/{primary_doc}"
@@ -747,18 +820,46 @@ def get_filing_text_for_accn(cik: str, accn: str, primary_doc: str, form_type: s
747
  resp.raise_for_status()
748
  except (httpx.HTTPError, httpx.TimeoutException) as e:
749
  print(f"WARNING: filing fetch failed cik={cik} accn={accn}: {e}")
750
- return "", ""
751
-
752
- from bs4 import BeautifulSoup
753
- text = BeautifulSoup(resp.text, "html.parser").get_text(separator="\n", strip=True)
 
 
 
 
754
  mda = _extract_mda(text, form_type)
755
  risk = _extract_risk_factors(text, form_type)
 
 
756
  if not mda:
757
  print(f"WARNING: MD&A section not found in {accn} ({form_type})")
758
  if not risk:
759
  print(f"WARNING: Risk Factors section not found in {accn} ({form_type})")
760
- print(f"INFO: filing {accn} ({form_type}) extracted MD&A={len(mda)} chars, Risk={len(risk)} chars")
761
- return mda, risk
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
762
 
763
 
764
  def _extract_guidance(mda_text: str) -> Optional[str]:
@@ -816,7 +917,9 @@ def fetch_all_edgar_data(
816
  report_date=acc.get("report_date", ""),
817
  filing_date=acc.get("filing_date", ""),
818
  )
819
- mda_text, risk_text = get_filing_text_for_accn(cik, acc["accn"], acc["primary_doc"], form_type)
 
 
820
  results.append(EdgarData(
821
  ticker=ticker.upper(),
822
  company_name=company_name,
@@ -829,8 +932,10 @@ def fetch_all_edgar_data(
829
  gross_margin=metrics["gross_margin"],
830
  operating_margin=metrics["operating_margin"],
831
  free_cash_flow=metrics["free_cash_flow"],
832
- mda_text=mda_text,
833
- risk_factors_text=risk_text,
 
 
834
  shares_diluted=metrics["shares_diluted"],
835
  effective_tax_rate=metrics["effective_tax_rate"],
836
  interest_expense=metrics["interest_expense"],
 
24
  free_cash_flow: Optional[float]
25
  mda_text: str
26
  risk_factors_text: str
27
+ business_text: str = ""
28
+ segments_geography_text: str = ""
29
  shares_diluted: Optional[float] = None
30
  effective_tax_rate: Optional[float] = None
31
  interest_expense: Optional[float] = None
 
705
 
706
  _ITEM_7 = re.compile(r'^\s*item\s+7\b(?!\s*a)', re.IGNORECASE | re.MULTILINE)
707
  _ITEM_7A = re.compile(r'^\s*item\s+7\s*a\b', re.IGNORECASE | re.MULTILINE)
708
+ _ITEM_1 = re.compile(r'^\s*item\s+1\b(?!\s*[a-z])', re.IGNORECASE | re.MULTILINE)
709
  _ITEM_1A = re.compile(r'^\s*item\s+1\s*a\b', re.IGNORECASE | re.MULTILINE)
710
  _ITEM_1B = re.compile(r'^\s*item\s+1\s*b\b', re.IGNORECASE | re.MULTILINE)
711
  _ITEM_2_Q = re.compile(r'^\s*item\s+2\b', re.IGNORECASE | re.MULTILINE)
712
  _ITEM_3_Q = re.compile(r'^\s*item\s+3\b', re.IGNORECASE | re.MULTILINE)
713
+ _ITEM_8 = re.compile(r'^\s*item\s+8\b', re.IGNORECASE | re.MULTILINE)
714
+ _ITEM_9 = re.compile(r'^\s*item\s+9\b(?!\s*[a-z])', re.IGNORECASE | re.MULTILINE)
715
 
716
  _MDA_CAP = 80_000
717
  _RISK_CAP = 40_000
718
+ _BUSINESS_CAP = 80_000
719
+ _SEGMENT_CAP = 60_000
720
+ _ITEM_8_CAP = 240_000
721
  _FALLBACK_WIN = 60_000
722
 
723
+ _SEGMENT_GEO_HEADING = re.compile(
724
+ r"(?im)^\s*(?:note\s+\d+\s*[-–—.:]?\s*)?"
725
+ r"(?:segment(?:s| information| reporting)?|reportable segments|"
726
+ r"geographic(?:al)? information|revenue by geographic(?: area)?|"
727
+ r"net sales by (?:reportable segment|geographic(?: area)?)|"
728
+ r"long-lived assets by geographic(?: area)?)\s*$"
729
+ )
730
+
731
 
732
  def _extract_item_section(text: str, start_re: re.Pattern, end_re: re.Pattern, cap: int) -> str:
733
  """Extract text from the last occurrence of start_re up to the first end_re after it."""
 
753
  return _extract_item_section(text, _ITEM_1A, end_re, _RISK_CAP)
754
 
755
 
756
+ def _extract_business(text: str, form_type: str = "10-K") -> str:
757
+ """Extract Item 1 Business from an annual filing."""
758
+ if form_type != "10-K":
759
+ return ""
760
+ return _extract_item_section(text, _ITEM_1, _ITEM_1A, _BUSINESS_CAP)
761
+
762
+
763
+ def _extract_segments_geography(text: str, form_type: str = "10-K") -> str:
764
+ """Extract targeted segment/geography disclosures from Item 8."""
765
+ if form_type != "10-K":
766
+ return ""
767
+ item8 = _extract_item_section(text, _ITEM_8, _ITEM_9, _ITEM_8_CAP)
768
+ search_text = item8 or text
769
+ matches = list(_SEGMENT_GEO_HEADING.finditer(search_text))
770
+ if not matches:
771
+ return ""
772
+
773
+ windows: list[str] = []
774
+ last_end = -1
775
+ for match in matches:
776
+ start = max(0, match.start() - 300)
777
+ end = min(len(search_text), match.start() + 12_000)
778
+ if start <= last_end and windows:
779
+ windows[-1] = (windows[-1] + search_text[last_end:end])[:_SEGMENT_CAP]
780
+ last_end = max(last_end, end)
781
+ continue
782
+ windows.append(search_text[start:end].strip())
783
+ last_end = end
784
+ if sum(len(window) for window in windows) >= _SEGMENT_CAP:
785
+ break
786
+ return "\n\n---\n\n".join(windows)[:_SEGMENT_CAP]
787
+
788
+
789
+ def _html_to_filing_text(html: str) -> str:
790
+ """Flatten filing HTML while preserving table rows as pipe-delimited text."""
791
+ from bs4 import BeautifulSoup
792
+
793
+ soup = BeautifulSoup(html, "html.parser")
794
+ for table in soup.find_all("table"):
795
+ rows: list[str] = []
796
+ for row in table.find_all("tr"):
797
+ cells = [
798
+ " ".join(cell.get_text(" ", strip=True).split())
799
+ for cell in row.find_all(["th", "td"])
800
+ ]
801
+ if any(cells):
802
+ rows.append(" | ".join(cells))
803
+ if rows:
804
+ table.replace_with("\n" + "\n".join(rows) + "\n")
805
+ return soup.get_text(separator="\n", strip=True)
806
+
807
+
808
+ def get_filing_sections_for_accn(
809
+ cik: str,
810
+ accn: str,
811
+ primary_doc: str,
812
+ form_type: str = "10-Q",
813
+ ) -> dict[str, str]:
814
+ """Return all filing sections needed by the research surfaces."""
815
  accn_clean = accn.replace("-", "")
816
  cik_int = int(cik)
817
  doc_url = f"https://www.sec.gov/Archives/edgar/data/{cik_int}/{accn_clean}/{primary_doc}"
 
820
  resp.raise_for_status()
821
  except (httpx.HTTPError, httpx.TimeoutException) as e:
822
  print(f"WARNING: filing fetch failed cik={cik} accn={accn}: {e}")
823
+ return {
824
+ "mda": "",
825
+ "risk_factors": "",
826
+ "business": "",
827
+ "segments_geography": "",
828
+ }
829
+
830
+ text = _html_to_filing_text(resp.text)
831
  mda = _extract_mda(text, form_type)
832
  risk = _extract_risk_factors(text, form_type)
833
+ business = _extract_business(text, form_type)
834
+ segments_geography = _extract_segments_geography(text, form_type)
835
  if not mda:
836
  print(f"WARNING: MD&A section not found in {accn} ({form_type})")
837
  if not risk:
838
  print(f"WARNING: Risk Factors section not found in {accn} ({form_type})")
839
+ if form_type == "10-K" and not business:
840
+ print(f"WARNING: Business section not found in {accn} ({form_type})")
841
+ print(
842
+ f"INFO: filing {accn} ({form_type}) extracted "
843
+ f"MD&A={len(mda)} chars, Risk={len(risk)} chars, "
844
+ f"Business={len(business)} chars, Segment/Geo={len(segments_geography)} chars"
845
+ )
846
+ return {
847
+ "mda": mda,
848
+ "risk_factors": risk,
849
+ "business": business,
850
+ "segments_geography": segments_geography,
851
+ }
852
+
853
+
854
+ def get_filing_text_for_accn(
855
+ cik: str,
856
+ accn: str,
857
+ primary_doc: str,
858
+ form_type: str = "10-Q",
859
+ ) -> tuple[str, str]:
860
+ """Backward-compatible wrapper returning only (MD&A, risk factors)."""
861
+ sections = get_filing_sections_for_accn(cik, accn, primary_doc, form_type)
862
+ return sections["mda"], sections["risk_factors"]
863
 
864
 
865
  def _extract_guidance(mda_text: str) -> Optional[str]:
 
917
  report_date=acc.get("report_date", ""),
918
  filing_date=acc.get("filing_date", ""),
919
  )
920
+ filing_sections = get_filing_sections_for_accn(
921
+ cik, acc["accn"], acc["primary_doc"], form_type
922
+ )
923
  results.append(EdgarData(
924
  ticker=ticker.upper(),
925
  company_name=company_name,
 
932
  gross_margin=metrics["gross_margin"],
933
  operating_margin=metrics["operating_margin"],
934
  free_cash_flow=metrics["free_cash_flow"],
935
+ mda_text=filing_sections["mda"],
936
+ risk_factors_text=filing_sections["risk_factors"],
937
+ business_text=filing_sections["business"],
938
+ segments_geography_text=filing_sections["segments_geography"],
939
  shares_diluted=metrics["shares_diluted"],
940
  effective_tax_rate=metrics["effective_tax_rate"],
941
  interest_expense=metrics["interest_expense"],
ingestion/embedder.py CHANGED
@@ -30,16 +30,23 @@ def embed_and_store_filing(
30
  accession: str = "",
31
  source_url: str = "",
32
  document_id: str = "",
 
 
33
  ) -> None:
34
  ticker = ticker.upper()
35
  filing_document_id = document_id or f"sec:{ticker}:{accession or f'{form_type}:{period}:{filing_date}'}"
36
- for section, text in [("MD&A", mda_text), ("Risk Factors", risk_text)]:
 
 
 
 
 
 
37
  if not text.strip():
38
  print(f"WARNING: empty section '{section}' for {ticker} {period} ({form_type}) — Chroma chunk not written")
39
  continue
40
  chunks = chunk_text(text)
41
  total = len(chunks)
42
- section_slug = "mda" if section == "MD&A" else "risk_factors"
43
  chunk_ids = [f"{section_slug}:{i}" for i in range(total)]
44
  metadatas = [
45
  {
 
30
  accession: str = "",
31
  source_url: str = "",
32
  document_id: str = "",
33
+ business_text: str = "",
34
+ segments_geography_text: str = "",
35
  ) -> None:
36
  ticker = ticker.upper()
37
  filing_document_id = document_id or f"sec:{ticker}:{accession or f'{form_type}:{period}:{filing_date}'}"
38
+ sections = [
39
+ ("MD&A", "mda", mda_text),
40
+ ("Risk Factors", "risk_factors", risk_text),
41
+ ("Business", "business", business_text),
42
+ ("Segments & Geography", "segments_geography", segments_geography_text),
43
+ ]
44
+ for section, section_slug, text in sections:
45
  if not text.strip():
46
  print(f"WARNING: empty section '{section}' for {ticker} {period} ({form_type}) — Chroma chunk not written")
47
  continue
48
  chunks = chunk_text(text)
49
  total = len(chunks)
 
50
  chunk_ids = [f"{section_slug}:{i}" for i in range(total)]
51
  metadatas = [
52
  {
storage/company_profiles.py ADDED
@@ -0,0 +1,172 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """Append-only persistence for Company Primer profiles and translations."""
2
+ from __future__ import annotations
3
+
4
+ import hashlib
5
+ import json
6
+ import sqlite3
7
+ from datetime import datetime, timezone
8
+ from pathlib import Path
9
+ from typing import Optional
10
+
11
+ DB_PATH = Path("data/company_profiles.db")
12
+ SCHEMA_VERSION = "company_profile.v1"
13
+
14
+
15
+ def init_db() -> None:
16
+ DB_PATH.parent.mkdir(parents=True, exist_ok=True)
17
+ with sqlite3.connect(DB_PATH) as conn:
18
+ conn.execute(
19
+ """
20
+ CREATE TABLE IF NOT EXISTS company_profiles (
21
+ profile_id TEXT PRIMARY KEY,
22
+ ticker TEXT NOT NULL,
23
+ source_fingerprint TEXT NOT NULL,
24
+ schema_version TEXT NOT NULL,
25
+ generated_at TEXT NOT NULL,
26
+ data_as_of TEXT,
27
+ language TEXT NOT NULL,
28
+ status TEXT NOT NULL,
29
+ profile_json TEXT NOT NULL
30
+ )
31
+ """
32
+ )
33
+ conn.execute(
34
+ """
35
+ CREATE INDEX IF NOT EXISTS idx_company_profiles_latest
36
+ ON company_profiles (ticker, generated_at DESC)
37
+ """
38
+ )
39
+ conn.execute(
40
+ """
41
+ CREATE TABLE IF NOT EXISTS company_profile_translations (
42
+ profile_id TEXT NOT NULL,
43
+ language TEXT NOT NULL,
44
+ profile_json TEXT NOT NULL,
45
+ saved_at TEXT NOT NULL,
46
+ PRIMARY KEY (profile_id, language)
47
+ )
48
+ """
49
+ )
50
+
51
+
52
+ def _profile_id(profile: dict) -> str:
53
+ identity = "\x1f".join(
54
+ (
55
+ str(profile.get("ticker") or "").upper(),
56
+ str(profile.get("source_fingerprint") or ""),
57
+ str(profile.get("schema_version") or SCHEMA_VERSION),
58
+ str(profile.get("generated_at") or ""),
59
+ str(profile.get("language") or "English"),
60
+ )
61
+ )
62
+ return hashlib.sha256(identity.encode("utf-8")).hexdigest()[:32]
63
+
64
+
65
+ def save_profile(ticker: str, profile: dict) -> str:
66
+ init_db()
67
+ payload = dict(profile)
68
+ payload["ticker"] = ticker.upper()
69
+ payload.setdefault("schema_version", SCHEMA_VERSION)
70
+ payload.setdefault("language", "English")
71
+ payload.setdefault("generated_at", datetime.now(timezone.utc).isoformat())
72
+ payload.setdefault("status", "PARTIAL")
73
+ profile_id = _profile_id(payload)
74
+ with sqlite3.connect(DB_PATH) as conn:
75
+ conn.execute(
76
+ """
77
+ INSERT OR IGNORE INTO company_profiles (
78
+ profile_id, ticker, source_fingerprint, schema_version,
79
+ generated_at, data_as_of, language, status, profile_json
80
+ ) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?)
81
+ """,
82
+ (
83
+ profile_id,
84
+ payload["ticker"],
85
+ payload.get("source_fingerprint") or "",
86
+ payload["schema_version"],
87
+ payload["generated_at"],
88
+ payload.get("data_as_of"),
89
+ payload["language"],
90
+ payload["status"],
91
+ json.dumps(payload, ensure_ascii=False),
92
+ ),
93
+ )
94
+ return profile_id
95
+
96
+
97
+ def get_latest_profile(
98
+ ticker: str,
99
+ *,
100
+ source_fingerprint: Optional[str] = None,
101
+ language: str = "English",
102
+ ) -> Optional[dict]:
103
+ if not DB_PATH.exists():
104
+ return None
105
+ init_db()
106
+ clauses = ["ticker = ?", "language = ?"]
107
+ params: list[str] = [ticker.upper(), language]
108
+ if source_fingerprint is not None:
109
+ clauses.append("source_fingerprint = ?")
110
+ params.append(source_fingerprint)
111
+ with sqlite3.connect(DB_PATH) as conn:
112
+ row = conn.execute(
113
+ """
114
+ SELECT profile_id, profile_json
115
+ FROM company_profiles
116
+ WHERE """
117
+ + " AND ".join(clauses)
118
+ + " ORDER BY generated_at DESC LIMIT 1",
119
+ params,
120
+ ).fetchone()
121
+ if row is None:
122
+ return None
123
+ try:
124
+ result = json.loads(row[1])
125
+ result["_profile_id"] = row[0]
126
+ return result
127
+ except json.JSONDecodeError:
128
+ return None
129
+
130
+
131
+ def get_translation(profile_id: str, language: str) -> Optional[dict]:
132
+ if not DB_PATH.exists():
133
+ return None
134
+ init_db()
135
+ with sqlite3.connect(DB_PATH) as conn:
136
+ row = conn.execute(
137
+ """
138
+ SELECT profile_json
139
+ FROM company_profile_translations
140
+ WHERE profile_id = ? AND language = ?
141
+ """,
142
+ (profile_id, language),
143
+ ).fetchone()
144
+ if row is None:
145
+ return None
146
+ try:
147
+ return json.loads(row[0])
148
+ except json.JSONDecodeError:
149
+ return None
150
+
151
+
152
+ def save_translation(profile_id: str, language: str, profile: dict) -> None:
153
+ init_db()
154
+ payload = dict(profile)
155
+ payload["language"] = language
156
+ with sqlite3.connect(DB_PATH) as conn:
157
+ conn.execute(
158
+ """
159
+ INSERT INTO company_profile_translations (
160
+ profile_id, language, profile_json, saved_at
161
+ ) VALUES (?, ?, ?, ?)
162
+ ON CONFLICT(profile_id, language) DO UPDATE SET
163
+ profile_json = excluded.profile_json,
164
+ saved_at = excluded.saved_at
165
+ """,
166
+ (
167
+ profile_id,
168
+ language,
169
+ json.dumps(payload, ensure_ascii=False),
170
+ datetime.now(timezone.utc).isoformat(),
171
+ ),
172
+ )
storage/sections_db.py CHANGED
@@ -1,8 +1,8 @@
1
  """storage/sections_db.py — persist raw section text across filing periods.
2
 
3
- Stores verbatim MD&A, Risk Factors, and transcript text keyed by
4
- (ticker, period, section). Used by analysis/textdiff.py to compare text
5
- across periods without re-fetching from EDGAR or Alpha Vantage.
6
 
7
  This is append-only at ingest time; textdiff reads it at runtime.
8
  """
@@ -40,7 +40,7 @@ def upsert_section(
40
  section: str,
41
  text: str,
42
  ) -> None:
43
- """Write (or overwrite) a section's text. section is one of: mda, risk_factors, transcript."""
44
  from datetime import datetime, timezone
45
  SECTIONS_DB_PATH.parent.mkdir(parents=True, exist_ok=True)
46
  with sqlite3.connect(SECTIONS_DB_PATH) as conn:
 
1
  """storage/sections_db.py — persist raw section text across filing periods.
2
 
3
+ Stores verbatim Business, Segments/Geography, MD&A, Risk Factors, and
4
+ transcript text keyed by (ticker, period, section). Used by company-profile
5
+ generation and analysis/textdiff.py without re-fetching source documents.
6
 
7
  This is append-only at ingest time; textdiff reads it at runtime.
8
  """
 
40
  section: str,
41
  text: str,
42
  ) -> None:
43
+ """Write or overwrite a supported filing or transcript section."""
44
  from datetime import datetime, timezone
45
  SECTIONS_DB_PATH.parent.mkdir(parents=True, exist_ok=True)
46
  with sqlite3.connect(SECTIONS_DB_PATH) as conn:
tests/test_company_profile.py ADDED
@@ -0,0 +1,298 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ from types import SimpleNamespace
2
+ import ast
3
+ from pathlib import Path
4
+
5
+ import pytest
6
+
7
+ from agent.company_profile import (
8
+ _collect_translation_segments,
9
+ _prune_unverified,
10
+ _quarantine_unsupported_shares,
11
+ profile_source_coverage,
12
+ source_fingerprint,
13
+ )
14
+ from agent.evidence import make_evidence_record
15
+ from agent.company_profile_schemas import CompanyProfile
16
+ from analysis.company_attention import attach_attention_stats, cluster_questions
17
+ from analytics.company_market import (
18
+ _parse_published_date,
19
+ sector_etf,
20
+ select_relative_events,
21
+ )
22
+ from storage import company_profiles
23
+
24
+
25
+ @pytest.mark.parametrize(
26
+ "relative_path",
27
+ [
28
+ "app.py",
29
+ "dashboard/company_primer.py",
30
+ "dashboard/i18n.py",
31
+ "dashboard/nav.py",
32
+ "agent/company_profile.py",
33
+ "analytics/company_market.py",
34
+ ],
35
+ )
36
+ def test_company_primer_files_parse(relative_path):
37
+ source = Path(relative_path).read_text(encoding="utf-8")
38
+ ast.parse(source, filename=relative_path)
39
+
40
+
41
+ def _fact(status="VERIFIED"):
42
+ return {
43
+ "text": "Verified claim.",
44
+ "source": "10-K",
45
+ "reliability": "HIGH",
46
+ "evidence_snippet": "Verified claim.",
47
+ "verification_status": status,
48
+ }
49
+
50
+
51
+ def test_company_profile_schema_accepts_partial_payload():
52
+ profile = CompanyProfile(ticker="AAPL", company_name="Apple Inc.")
53
+ assert profile.status == "PARTIAL"
54
+ assert profile.schema_version == "company_profile.v1"
55
+ assert profile.business_lines == []
56
+
57
+
58
+ def test_prune_unverified_removes_nested_profile_claims():
59
+ profile = {
60
+ "identity": {
61
+ "one_liner": _fact("FAILED"),
62
+ "business_model": _fact(),
63
+ "customer_types": [_fact(), _fact("UNVERIFIED")],
64
+ },
65
+ "business_lines": [
66
+ {"name": "Good", "description": _fact()},
67
+ {"name": "Bad", "description": _fact("FAILED")},
68
+ ],
69
+ "geographic_exposures": [],
70
+ "strategic_changes": [],
71
+ "attention_themes": [],
72
+ "watch_variables": [],
73
+ }
74
+ removed = _prune_unverified(profile)
75
+ assert removed == 3
76
+ assert profile["identity"]["one_liner"] is None
77
+ assert len(profile["identity"]["customer_types"]) == 1
78
+ assert [item["name"] for item in profile["business_lines"]] == ["Good"]
79
+
80
+
81
+ def test_translation_segments_skip_verbatim_evidence():
82
+ profile = {
83
+ "identity": {
84
+ "one_liner": {
85
+ "text": "Designs chips.",
86
+ "evidence_snippet": "designs semiconductor chips",
87
+ "source": "10-K",
88
+ }
89
+ },
90
+ "watch_variables": [
91
+ {"variable": "Gross margin", "next_datapoint": "Next earnings"}
92
+ ],
93
+ }
94
+ segments = _collect_translation_segments(profile)
95
+ paths = [path for path, _ in segments]
96
+ assert ("identity", "one_liner", "text") in paths
97
+ assert ("identity", "one_liner", "evidence_snippet") not in paths
98
+ assert ("watch_variables", 0, "next_datapoint") in paths
99
+
100
+
101
+ def test_unsupported_parent_revenue_share_is_hidden():
102
+ record = make_evidence_record(
103
+ source="10-K",
104
+ content="The Compute segment serves data-center customers.",
105
+ document_id="sec:test",
106
+ chunk_id="business:1",
107
+ )
108
+ fact = {
109
+ "text": "Compute serves data-center customers.",
110
+ "source": "10-K",
111
+ "reliability": "HIGH",
112
+ "evidence_snippet": "Compute segment serves data-center customers",
113
+ "evidence_ref": record.ref.model_dump(mode="json"),
114
+ "verification_status": "VERIFIED",
115
+ }
116
+ profile = {
117
+ "business_lines": [
118
+ {
119
+ "name": "Compute",
120
+ "description": fact,
121
+ "revenue_share_pct": 80,
122
+ "share_period": "FY2025",
123
+ }
124
+ ],
125
+ "geographic_exposures": [],
126
+ }
127
+ removed = _quarantine_unsupported_shares(profile, [record])
128
+ assert removed == 1
129
+ assert profile["business_lines"][0]["revenue_share_pct"] is None
130
+ assert profile["business_lines"][0]["share_period"] is None
131
+
132
+
133
+ def test_source_fingerprint_changes_when_business_backfill_changes(monkeypatch):
134
+ rows = [
135
+ {"form_type": "10-K", "accession": "K1", "filing_date": "2025-01-01"},
136
+ {"form_type": "10-Q", "accession": "Q1", "filing_date": "2025-04-01"},
137
+ ]
138
+ monkeypatch.setattr("agent.company_profile.metrics_db.get_all_metrics", lambda _ticker: rows)
139
+ monkeypatch.setattr(
140
+ "agent.company_profile.get_recent_transcripts",
141
+ lambda _ticker, n=1: [("Q12025", "transcript")],
142
+ )
143
+ monkeypatch.setattr(
144
+ "agent.company_profile.get_periods_for_ticker",
145
+ lambda _ticker, form_type=None: ["FY2025"],
146
+ )
147
+ sections = {
148
+ ("FY2025", "business"): "",
149
+ ("FY2025", "segments_geography"): "",
150
+ }
151
+ monkeypatch.setattr(
152
+ "agent.company_profile.get_section",
153
+ lambda _ticker, period, section: sections[(period, section)],
154
+ )
155
+ before = source_fingerprint("AAPL")
156
+ sections[("FY2025", "business")] = "Item 1. Business content"
157
+ after = source_fingerprint("AAPL")
158
+ assert before != after
159
+
160
+
161
+ def test_profile_coverage_requires_business_geography_and_call_history(monkeypatch):
162
+ monkeypatch.setattr(
163
+ "agent.company_profile.get_periods_for_ticker",
164
+ lambda _ticker, form_type=None: ["FY2025"],
165
+ )
166
+ sections = {
167
+ "business": "Business disclosure",
168
+ "segments_geography": "Geographic disclosure",
169
+ }
170
+ monkeypatch.setattr(
171
+ "agent.company_profile.get_section",
172
+ lambda _ticker, _period, section: sections[section],
173
+ )
174
+ monkeypatch.setattr(
175
+ "agent.company_profile.get_recent_transcripts",
176
+ lambda _ticker, n=4: [("Q42024", "a"), ("Q12025", "b")],
177
+ )
178
+
179
+ complete = profile_source_coverage("AAPL")
180
+ assert complete["status"] == "COMPLETE"
181
+ assert complete["segments_geography_sections_attempted"] == 1
182
+ sections["segments_geography"] = ""
183
+ partial = profile_source_coverage("AAPL")
184
+ assert partial["status"] == "PARTIAL"
185
+ assert partial["segments_geography_sections_attempted"] == 1
186
+
187
+
188
+ def test_company_profile_storage_round_trip(tmp_path, monkeypatch):
189
+ monkeypatch.setattr(company_profiles, "DB_PATH", tmp_path / "profiles.db")
190
+ profile = {
191
+ "ticker": "AAPL",
192
+ "company_name": "Apple Inc.",
193
+ "source_fingerprint": "abc",
194
+ "schema_version": "company_profile.v1",
195
+ "generated_at": "2026-01-01T00:00:00+00:00",
196
+ "data_as_of": "2025-12-31",
197
+ "language": "English",
198
+ "status": "PARTIAL",
199
+ }
200
+ profile_id = company_profiles.save_profile("AAPL", profile)
201
+ loaded = company_profiles.get_latest_profile(
202
+ "AAPL", source_fingerprint="abc", language="English"
203
+ )
204
+ assert loaded["_profile_id"] == profile_id
205
+ company_profiles.save_translation(profile_id, "French", {**profile, "language": "French"})
206
+ assert company_profiles.get_translation(profile_id, "French")["language"] == "French"
207
+
208
+
209
+ def test_attention_cluster_counts_questions(monkeypatch):
210
+ parsed = {
211
+ "Q1": SimpleNamespace(
212
+ qa=[
213
+ SimpleNamespace(question="Can you discuss China demand and China inventory?"),
214
+ SimpleNamespace(question="How should we think about pricing and margins?"),
215
+ ]
216
+ ),
217
+ "Q2": SimpleNamespace(
218
+ qa=[
219
+ SimpleNamespace(question="What changed in China demand and channel inventory?"),
220
+ SimpleNamespace(question="Can you discuss China inventory and demand trends?"),
221
+ ]
222
+ ),
223
+ }
224
+ monkeypatch.setattr(
225
+ "analysis.company_attention.parse_call",
226
+ lambda period, _raw: parsed[period],
227
+ )
228
+ clusters = cluster_questions([("Q1", "a"), ("Q2", "b")])
229
+ assert clusters
230
+ assert clusters[0]["question_count"] == 3
231
+ assert clusters[0]["mentions"] == [
232
+ {"period": "Q1", "mentions": 1},
233
+ {"period": "Q2", "mentions": 2},
234
+ ]
235
+
236
+
237
+ def test_attach_attention_stats_adds_news_count(monkeypatch):
238
+ monkeypatch.setattr(
239
+ "analysis.company_attention.get_attention_clusters",
240
+ lambda _ticker: [
241
+ {
242
+ "tokens": ["china", "demand"],
243
+ "mentions": [
244
+ {"period": "Q1", "mentions": 0},
245
+ {"period": "Q2", "mentions": 2},
246
+ ],
247
+ }
248
+ ],
249
+ )
250
+ profile = {
251
+ "attention_themes": [
252
+ {
253
+ "theme": "China demand",
254
+ "evidence": {"text": "China demand remains volatile."},
255
+ }
256
+ ]
257
+ }
258
+ attach_attention_stats(
259
+ profile,
260
+ "AAPL",
261
+ [{"title": "China demand improves", "snippet": ""}, {"title": "Dividend", "snippet": ""}],
262
+ )
263
+ theme = profile["attention_themes"][0]
264
+ assert theme["direction"] == "new"
265
+ assert theme["retrieved_news_count"] == 1
266
+
267
+
268
+ def test_sector_etf_mapping_and_fallback():
269
+ assert sector_etf("Technology") == "XLK"
270
+ assert sector_etf("Consumer Cyclical") == "XLY"
271
+ assert sector_etf("Unknown") == "SPY"
272
+
273
+
274
+ def test_market_news_dates_accept_iso_and_rfc_formats():
275
+ assert _parse_published_date("2026-07-29T08:30:00Z").isoformat() == "2026-07-29"
276
+ assert (
277
+ _parse_published_date("Wed, 29 Jul 2026 08:30:00 GMT").isoformat()
278
+ == "2026-07-29"
279
+ )
280
+ assert _parse_published_date("not a date") is None
281
+
282
+
283
+ def test_relative_events_are_balanced_thresholded_and_spaced():
284
+ dates = [f"2026-01-{day:02d}" for day in range(1, 13)]
285
+ company = [100, 105, 106, 107, 108, 109, 102, 101, 100, 99, 106, 107]
286
+ sector = [100] * len(company)
287
+ events = select_relative_events(
288
+ dates,
289
+ company,
290
+ sector,
291
+ threshold_pct=2,
292
+ per_direction=2,
293
+ min_spacing=3,
294
+ )
295
+ assert any(item["sector_excess_pct"] > 0 for item in events)
296
+ assert any(item["sector_excess_pct"] < 0 for item in events)
297
+ indices = [item["index"] for item in events]
298
+ assert all(abs(a - b) >= 3 for i, a in enumerate(indices) for b in indices[i + 1 :])
tests/test_edgar.py CHANGED
@@ -1,6 +1,16 @@
1
  import pytest
2
  from unittest.mock import patch, MagicMock
3
- from ingestion.edgar import get_cik, compute_metrics_for_accn, get_all_xbrl_facts, _extract_guidance, _extract_mda, _extract_risk_factors
 
 
 
 
 
 
 
 
 
 
4
 
5
  FAKE_TICKERS = {
6
  "0": {"cik_str": 320193, "ticker": "AAPL", "title": "Apple Inc."}
@@ -240,6 +250,57 @@ def test_extract_risk_factors_returns_empty_when_not_found():
240
  assert result == ""
241
 
242
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
243
  # ── XBRL concept expansion ───────────────────────────────────────────────────
244
 
245
  def test_compute_metrics_uses_sales_revenue_net_concept():
 
1
  import pytest
2
  from unittest.mock import patch, MagicMock
3
+ from ingestion.edgar import (
4
+ get_cik,
5
+ compute_metrics_for_accn,
6
+ get_all_xbrl_facts,
7
+ _extract_business,
8
+ _extract_guidance,
9
+ _extract_mda,
10
+ _extract_risk_factors,
11
+ _extract_segments_geography,
12
+ _html_to_filing_text,
13
+ )
14
 
15
  FAKE_TICKERS = {
16
  "0": {"cik_str": 320193, "ticker": "AAPL", "title": "Apple Inc."}
 
250
  assert result == ""
251
 
252
 
253
+ def test_extract_business_uses_item_1_and_stops_before_risks():
254
+ text = """Table of Contents
255
+ Item 1. Business
256
+ Item 1A. Risk Factors
257
+
258
+ Item 1. Business
259
+ We design accelerated computing platforms for data centers.
260
+ Our customers include cloud service providers.
261
+
262
+ Item 1A. Risk Factors
263
+ Export controls may affect sales.
264
+ """
265
+ result = _extract_business(text, "10-K")
266
+ assert "accelerated computing platforms" in result
267
+ assert "Export controls" not in result
268
+
269
+
270
+ def test_extract_business_returns_empty_for_10q():
271
+ assert _extract_business("Item 1. Business\nText\nItem 1A.", "10-Q") == ""
272
+
273
+
274
+ def test_extract_segments_geography_preserves_disclosed_table_text():
275
+ text = """Item 8. Financial Statements and Supplementary Data
276
+
277
+ Note 12 - Segment Information
278
+ Compute | 80 | 70
279
+ Networking | 20 | 30
280
+
281
+ Geographic Information
282
+ United States | 60%
283
+ International | 40%
284
+
285
+ Item 9. Changes in and Disagreements with Accountants
286
+ """
287
+ result = _extract_segments_geography(text, "10-K")
288
+ assert "Compute | 80 | 70" in result
289
+ assert "United States | 60%" in result
290
+ assert "Changes in and Disagreements" not in result
291
+
292
+
293
+ def test_html_to_filing_text_keeps_table_rows():
294
+ html = """
295
+ <html><body><h2>Geographic Information</h2>
296
+ <table><tr><th>Region</th><th>Revenue</th></tr>
297
+ <tr><td>Europe</td><td>25%</td></tr></table></body></html>
298
+ """
299
+ result = _html_to_filing_text(html)
300
+ assert "Region | Revenue" in result
301
+ assert "Europe | 25%" in result
302
+
303
+
304
  # ── XBRL concept expansion ───────────────────────────────────────────────────
305
 
306
  def test_compute_metrics_uses_sales_revenue_net_concept():
tests/test_embedder.py CHANGED
@@ -47,6 +47,36 @@ def test_embed_filing_skips_empty_sections(mock_vs):
47
  assert mock_vs.add_chunks.call_count == 1
48
 
49
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
50
  @patch("ingestion.embedder.vector_store")
51
  def test_embed_filing_chunk_context_in_metadata(mock_vs):
52
  embed_and_store_filing(
 
47
  assert mock_vs.add_chunks.call_count == 1
48
 
49
 
50
+ @patch("ingestion.embedder.vector_store")
51
+ def test_embed_annual_filing_indexes_company_profile_sections(mock_vs):
52
+ embed_and_store_filing(
53
+ ticker="AAPL",
54
+ company_name="Apple Inc.",
55
+ mda_text="Revenue grew 5%.",
56
+ risk_text="Competition remains a risk.",
57
+ business_text="The company designs and sells devices and services.",
58
+ segments_geography_text="Net sales are reported by geography.",
59
+ filing_date="2024-11-01",
60
+ period="FY2024",
61
+ form_type="10-K",
62
+ )
63
+
64
+ assert mock_vs.add_chunks.call_count == 4
65
+ metadata = [call.args[2][0] for call in mock_vs.add_chunks.call_args_list]
66
+ assert [item["section"] for item in metadata] == [
67
+ "MD&A",
68
+ "Risk Factors",
69
+ "Business",
70
+ "Segments & Geography",
71
+ ]
72
+ assert [item["chunk_id"] for item in metadata] == [
73
+ "mda:0",
74
+ "risk_factors:0",
75
+ "business:0",
76
+ "segments_geography:0",
77
+ ]
78
+
79
+
80
  @patch("ingestion.embedder.vector_store")
81
  def test_embed_filing_chunk_context_in_metadata(mock_vs):
82
  embed_and_store_filing(
tests/test_ingest.py CHANGED
@@ -55,6 +55,8 @@ def test_10k_fetches_q4_transcript():
55
  patch("ingest.prune_old_metrics"),
56
  patch("ingest.init_db"),
57
  patch("ingest.init_sections_db"),
 
 
58
  patch("ingest.fetch_transcript", return_value="Q4 transcript text") as mock_transcript,
59
  ):
60
  ingest("NVDA")
@@ -81,8 +83,62 @@ def test_10q_fetches_quarterly_transcript():
81
  patch("ingest.prune_old_metrics"),
82
  patch("ingest.init_db"),
83
  patch("ingest.init_sections_db"),
 
 
84
  patch("ingest.fetch_transcript", return_value="Q1 transcript text") as mock_transcript,
85
  ):
86
  ingest("NVDA")
87
 
88
  mock_transcript.assert_called_once_with("NVDA", "2025Q1")
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
55
  patch("ingest.prune_old_metrics"),
56
  patch("ingest.init_db"),
57
  patch("ingest.init_sections_db"),
58
+ patch("ingest.get_all_metrics", return_value=[]),
59
+ patch("ingest.get_section", return_value=None),
60
  patch("ingest.fetch_transcript", return_value="Q4 transcript text") as mock_transcript,
61
  ):
62
  ingest("NVDA")
 
83
  patch("ingest.prune_old_metrics"),
84
  patch("ingest.init_db"),
85
  patch("ingest.init_sections_db"),
86
+ patch("ingest.get_all_metrics", return_value=[]),
87
+ patch("ingest.get_section", return_value=None),
88
  patch("ingest.fetch_transcript", return_value="Q1 transcript text") as mock_transcript,
89
  ):
90
  ingest("NVDA")
91
 
92
  mock_transcript.assert_called_once_with("NVDA", "2025Q1")
93
+
94
+
95
+ def test_delta_backfills_missing_company_profile_sections_without_refetching_transcript():
96
+ """A previously ingested 10-K must be revisited when its profile sections are absent."""
97
+ from ingest import ingest
98
+
99
+ edgar_10k = _make_edgar("FY2025", "10-K")
100
+ edgar_10k.business_text = "NVIDIA designs accelerated computing platforms."
101
+ edgar_10k.segments_geography_text = "Revenue is disclosed by reportable segment and geography."
102
+
103
+ def stored_section(_ticker, _period, section):
104
+ return "cached transcript" if section == "transcript" else None
105
+
106
+ with (
107
+ patch("ingest.fetch_all_edgar_data", return_value=[edgar_10k]),
108
+ patch("ingest.fill_missing_metrics", side_effect=lambda x: x),
109
+ patch("ingest.get_all_metrics", return_value=[{"period": "FY2025"}]),
110
+ patch("ingest.get_section", side_effect=stored_section),
111
+ patch("ingest._extract_guidance", return_value=None),
112
+ patch("ingest.parse_guidance", return_value={}),
113
+ patch("ingest.upsert_metrics"),
114
+ patch("ingest.upsert_section") as mock_upsert_section,
115
+ patch("ingest.embed_and_store_filing") as mock_embed_filing,
116
+ patch("ingest.embed_and_store_transcript"),
117
+ patch("ingest.prune_old_metrics"),
118
+ patch("ingest.init_db"),
119
+ patch("ingest.init_sections_db"),
120
+ patch("ingest.fetch_transcript") as mock_fetch_transcript,
121
+ ):
122
+ ingest("NVDA")
123
+
124
+ mock_fetch_transcript.assert_not_called()
125
+ mock_embed_filing.assert_called_once()
126
+ assert mock_embed_filing.call_args.kwargs["business_text"] == edgar_10k.business_text
127
+ assert (
128
+ mock_embed_filing.call_args.kwargs["segments_geography_text"]
129
+ == edgar_10k.segments_geography_text
130
+ )
131
+ assert call(
132
+ "NVDA",
133
+ "FY2025",
134
+ "10-K",
135
+ "business",
136
+ edgar_10k.business_text,
137
+ ) in mock_upsert_section.call_args_list
138
+ assert call(
139
+ "NVDA",
140
+ "FY2025",
141
+ "10-K",
142
+ "segments_geography",
143
+ edgar_10k.segments_geography_text,
144
+ ) in mock_upsert_section.call_args_list