Spaces:
Sleeping
Sleeping
feat: add company primer research tab
Browse files- ARCHITECTURE.md +3 -2
- README.md +8 -2
- agent/company_profile.py +482 -0
- agent/company_profile_schemas.py +120 -0
- analysis/company_attention.py +150 -0
- analytics/company_market.py +246 -0
- analytics/company_news.py +51 -0
- app.py +197 -5
- dashboard/company_primer.py +542 -0
- dashboard/i18n.py +68 -0
- dashboard/nav.py +9 -5
- ingest.py +25 -1
- ingestion/edgar.py +116 -11
- ingestion/embedder.py +9 -2
- storage/company_profiles.py +172 -0
- storage/sections_db.py +4 -4
- tests/test_company_profile.py +298 -0
- tests/test_edgar.py +62 -1
- tests/test_embedder.py +30 -0
- tests/test_ingest.py +56 -0
ARCHITECTURE.md
CHANGED
|
@@ -18,7 +18,7 @@ flowchart TB
|
|
| 18 |
classDef io fill:#fce7f3,stroke:#db2777,color:#831843
|
| 19 |
classDef gate fill:#fee2e2,stroke:#dc2626,color:#7f1d1d
|
| 20 |
|
| 21 |
-
USER([PM / senior equity analyst]):::io --> UI["Streamlit
|
| 22 |
UI --> AG
|
| 23 |
|
| 24 |
subgraph LG ["LangGraph · short-term state"]
|
|
@@ -99,12 +99,13 @@ flowchart TB
|
|
| 99 |
|
| 100 |
| Surface | Primary payload |
|
| 101 |
| --- | --- |
|
|
|
|
| 102 |
| **PM Flash** | As-of and coverage; experimental PM read-through; thesis-confirming/challenging sourced points; experimental Swing Factor; watch items. |
|
| 103 |
| **Evidence & Deltas** | Source-backed facts, verbatim evidence, deterministic detector outputs labelled heuristic, risks, commentary, and experimental AI hypotheses. |
|
| 104 |
| **Financials** | Structured historical metrics, trends, guidance history, earnings history, and exports. |
|
| 105 |
| **Ask Evidence** | Secondary grounded Q&A over the available evidence. |
|
| 106 |
|
| 107 |
-
The navigation order is intentional:
|
| 108 |
|
| 109 |
## Output taxonomy
|
| 110 |
|
|
|
|
| 18 |
classDef io fill:#fce7f3,stroke:#db2777,color:#831843
|
| 19 |
classDef gate fill:#fee2e2,stroke:#dc2626,color:#7f1d1d
|
| 20 |
|
| 21 |
+
USER([PM / senior equity analyst]):::io --> UI["Streamlit research stack<br/>Company Primer · PM Flash · Evidence & Deltas · Financials<br/>Ask Evidence — secondary"]:::io
|
| 22 |
UI --> AG
|
| 23 |
|
| 24 |
subgraph LG ["LangGraph · short-term state"]
|
|
|
|
| 99 |
|
| 100 |
| Surface | Primary payload |
|
| 101 |
| --- | --- |
|
| 102 |
+
| **Company Primer** | Stable sourced company profile plus independently refreshed price context and public news. Business model, geographic exposure, three-year trends, attention themes, associated price events, and monitoring variables remain concise and inspectable. |
|
| 103 |
| **PM Flash** | As-of and coverage; experimental PM read-through; thesis-confirming/challenging sourced points; experimental Swing Factor; watch items. |
|
| 104 |
| **Evidence & Deltas** | Source-backed facts, verbatim evidence, deterministic detector outputs labelled heuristic, risks, commentary, and experimental AI hypotheses. |
|
| 105 |
| **Financials** | Structured historical metrics, trends, guidance history, earnings history, and exports. |
|
| 106 |
| **Ask Evidence** | Secondary grounded Q&A over the available evidence. |
|
| 107 |
|
| 108 |
+
The navigation order is intentional: company onboarding → decision memo → grounded Q&A → audit → model depth.
|
| 109 |
|
| 110 |
## Output taxonomy
|
| 111 |
|
README.md
CHANGED
|
@@ -19,6 +19,7 @@ The product is designed for the first review after an earnings release: what cha
|
|
| 19 |
|
| 20 |
| Surface | Role in the workflow |
|
| 21 |
| --- | --- |
|
|
|
|
| 22 |
| **PM Flash** | One-screen decision memo: filing date and source coverage, experimental AI read-through, thesis-confirming and thesis-challenging evidence, the swing factor, and the next items to watch. |
|
| 23 |
| **Evidence & Deltas** | Audit layer: source-backed claims, before/after evidence, heuristic period deltas, risks, management commentary, and experimental AI hypotheses. |
|
| 24 |
| **Financials** | Historical KPI trends, guidance history, earnings history, and exportable structured data. |
|
|
@@ -52,11 +53,12 @@ Legacy briefs without a deterministic `evidence_coverage` result are shown as **
|
|
| 52 |
|
| 53 |
## Architecture
|
| 54 |
|
| 55 |
-
- **Offline ingestion** — SEC EDGAR XBRL into SQLite; 10-K/10-Q
|
|
|
|
| 56 |
- **Runtime agent** — a LangGraph tool loop retrieves structured financials, filings, transcripts, public news, and analyst data before producing a validated Pydantic brief.
|
| 57 |
- **Retrieval** — vector search over-fetches candidates and reranks them with a cross-encoder.
|
| 58 |
- **Post-synthesis controls** — deterministic source-reliability rules, corroboration checks, and heuristic period-delta detectors run after model synthesis.
|
| 59 |
-
- **Decision UI** — PM Flash
|
| 60 |
|
| 61 |
See [WRITEUP.md](WRITEUP.md) for product and methodology detail and [ARCHITECTURE.md](ARCHITECTURE.md) for data lineage and presentation controls.
|
| 62 |
|
|
@@ -74,6 +76,10 @@ databases created before the evidence-lineage upgrade: legacy vector chunks and
|
|
| 74 |
metrics without verified period context are intentionally hidden rather than
|
| 75 |
silently upgraded.
|
| 76 |
|
|
|
|
|
|
|
|
|
|
|
|
|
| 77 |
## Intended use
|
| 78 |
|
| 79 |
Amplegest is a research accelerator based on public information. It helps an experienced investor review evidence faster, challenge a thesis, and identify the next falsifiable datapoint. The analyst or portfolio manager retains responsibility for source verification, modelling, valuation, and the investment decision.
|
|
|
|
| 19 |
|
| 20 |
| Surface | Role in the workflow |
|
| 21 |
| --- | --- |
|
| 22 |
+
| **Company Primer** | Concise company onboarding: business model, revenue engines, geographic exposure, three-year evolution, investor-attention themes, relative price events, recent news, and the variables to watch. |
|
| 23 |
| **PM Flash** | One-screen decision memo: filing date and source coverage, experimental AI read-through, thesis-confirming and thesis-challenging evidence, the swing factor, and the next items to watch. |
|
| 24 |
| **Evidence & Deltas** | Audit layer: source-backed claims, before/after evidence, heuristic period deltas, risks, management commentary, and experimental AI hypotheses. |
|
| 25 |
| **Financials** | Historical KPI trends, guidance history, earnings history, and exportable structured data. |
|
|
|
|
| 53 |
|
| 54 |
## Architecture
|
| 55 |
|
| 56 |
+
- **Offline ingestion** — SEC EDGAR XBRL into SQLite; 10-K/10-Q Business, Segments/Geography, MD&A, Risk Factors, and earnings-call transcripts into the evidence stores.
|
| 57 |
+
- **Company profile** — a cached, source-fingerprinted English profile separates stable filing/call analysis from independently refreshed prices and public news.
|
| 58 |
- **Runtime agent** — a LangGraph tool loop retrieves structured financials, filings, transcripts, public news, and analyst data before producing a validated Pydantic brief.
|
| 59 |
- **Retrieval** — vector search over-fetches candidates and reranks them with a cross-encoder.
|
| 60 |
- **Post-synthesis controls** — deterministic source-reliability rules, corroboration checks, and heuristic period-delta detectors run after model synthesis.
|
| 61 |
+
- **Decision UI** — Company Primer onboards; PM Flash frames the decision; Evidence & Deltas provides auditability; Financials provides depth; Ask Evidence stays secondary.
|
| 62 |
|
| 63 |
See [WRITEUP.md](WRITEUP.md) for product and methodology detail and [ARCHITECTURE.md](ARCHITECTURE.md) for data lineage and presentation controls.
|
| 64 |
|
|
|
|
| 76 |
metrics without verified period context are intentionally hidden rather than
|
| 77 |
silently upgraded.
|
| 78 |
|
| 79 |
+
For an already lineage-compatible database, `python ingest.py AAPL` is enough
|
| 80 |
+
to backfill the Company Primer’s 10-K Business and Segments/Geography sections;
|
| 81 |
+
the delta path reuses any transcript already stored.
|
| 82 |
+
|
| 83 |
## Intended use
|
| 84 |
|
| 85 |
Amplegest is a research accelerator based on public information. It helps an experienced investor review evidence faster, challenge a thesis, and identify the next falsifiable datapoint. The analyst or portfolio manager retains responsibility for source verification, modelling, valuation, and the investment decision.
|
agent/company_profile.py
ADDED
|
@@ -0,0 +1,482 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""Evidence-grounded Company Primer generation and translation."""
|
| 2 |
+
from __future__ import annotations
|
| 3 |
+
|
| 4 |
+
import copy
|
| 5 |
+
import hashlib
|
| 6 |
+
import json
|
| 7 |
+
from datetime import datetime, timezone
|
| 8 |
+
from typing import Callable, Optional
|
| 9 |
+
|
| 10 |
+
from langchain_core.messages import HumanMessage
|
| 11 |
+
|
| 12 |
+
from agent.company_profile_schemas import CompanyProfile
|
| 13 |
+
from agent.evidence import evidence_records_from, verify_brief_evidence, verify_fact
|
| 14 |
+
from agent.llm import RunConfig, build_system_message, make_chat_model
|
| 15 |
+
from agent.tools import get_financial_metrics, search_filing, search_transcript
|
| 16 |
+
from storage import metrics_db
|
| 17 |
+
from storage.sections_db import get_recent_transcripts, get_section, get_periods_for_ticker
|
| 18 |
+
|
| 19 |
+
PROFILE_SCHEMA_VERSION = "company_profile.v1"
|
| 20 |
+
|
| 21 |
+
_PROFILE_PROMPT = """
|
| 22 |
+
You are a senior equity-research analyst preparing a compact Company Primer for
|
| 23 |
+
someone who knows little about the company.
|
| 24 |
+
|
| 25 |
+
Use ONLY the evidence records supplied by the user. Never use prior knowledge.
|
| 26 |
+
Every factual object must include one SourcedFact with:
|
| 27 |
+
- source, reliability, evidence_snippet and evidence_ref copied exactly from one
|
| 28 |
+
supplied evidence record;
|
| 29 |
+
- a short evidence_snippet that appears verbatim in that record;
|
| 30 |
+
- no number that is absent from the supporting record.
|
| 31 |
+
|
| 32 |
+
Content rules:
|
| 33 |
+
- identity: one-line description, how the company makes money, customer types,
|
| 34 |
+
and competitive position; prefer the 10-K Business section.
|
| 35 |
+
- business_lines: 3-5 economic engines. revenue_share_pct must be null unless
|
| 36 |
+
the exact percentage is disclosed in the cited evidence.
|
| 37 |
+
- geographic_exposures: separate disclosed revenue geography from qualitative
|
| 38 |
+
sales, operational, supply-chain, or regulatory exposure. Do not convert a
|
| 39 |
+
country mention into materiality or a revenue percentage.
|
| 40 |
+
- strategic_changes: at most 3 material changes across the available years.
|
| 41 |
+
- attention_themes: exactly the 3 strongest investor questions supported by
|
| 42 |
+
earnings-call or filing evidence. Keep why_it_matters to one sentence.
|
| 43 |
+
- watch_variables: 3-4 variables, each with the next datapoint and a falsifiable
|
| 44 |
+
alert signal. These are monitoring prompts, not recommendations.
|
| 45 |
+
|
| 46 |
+
Keep every prose field concise. Analytical prose such as economics, implication,
|
| 47 |
+
why_it_matters and alert_signal is an AI hypothesis anchored to the cited fact.
|
| 48 |
+
Never provide a buy/sell recommendation, price target, valuation conclusion, or
|
| 49 |
+
claim that an event caused a stock-price move.
|
| 50 |
+
""".strip()
|
| 51 |
+
|
| 52 |
+
_TRANSLATABLE_FIELDS = {
|
| 53 |
+
"text",
|
| 54 |
+
"economics",
|
| 55 |
+
"implication",
|
| 56 |
+
"theme",
|
| 57 |
+
"why_it_matters",
|
| 58 |
+
"variable",
|
| 59 |
+
"next_datapoint",
|
| 60 |
+
"alert_signal",
|
| 61 |
+
}
|
| 62 |
+
_NEVER_TRANSLATE = {
|
| 63 |
+
"evidence_snippet",
|
| 64 |
+
"source",
|
| 65 |
+
"reliability",
|
| 66 |
+
"impact",
|
| 67 |
+
"verification_status",
|
| 68 |
+
"verification_reason",
|
| 69 |
+
"ticker",
|
| 70 |
+
"company_name",
|
| 71 |
+
"period",
|
| 72 |
+
"period_from",
|
| 73 |
+
"period_to",
|
| 74 |
+
"share_period",
|
| 75 |
+
"source_fingerprint",
|
| 76 |
+
"schema_version",
|
| 77 |
+
}
|
| 78 |
+
|
| 79 |
+
|
| 80 |
+
def source_fingerprint(ticker: str) -> str:
|
| 81 |
+
"""Fingerprint stable company-profile sources, not live price/news data."""
|
| 82 |
+
ticker = ticker.upper()
|
| 83 |
+
rows = metrics_db.get_all_metrics(ticker)
|
| 84 |
+
latest_k = next((row for row in rows if row.get("form_type") == "10-K"), {})
|
| 85 |
+
latest_q = next((row for row in rows if row.get("form_type") == "10-Q"), {})
|
| 86 |
+
transcripts = get_recent_transcripts(ticker, n=1)
|
| 87 |
+
latest_transcript = transcripts[-1][0] if transcripts else ""
|
| 88 |
+
annual_periods = get_periods_for_ticker(ticker, form_type="10-K")
|
| 89 |
+
latest_annual = annual_periods[0] if annual_periods else ""
|
| 90 |
+
business_digest = hashlib.sha256(
|
| 91 |
+
(get_section(ticker, latest_annual, "business") or "").encode("utf-8")
|
| 92 |
+
).hexdigest()[:12]
|
| 93 |
+
geography_digest = hashlib.sha256(
|
| 94 |
+
(get_section(ticker, latest_annual, "segments_geography") or "").encode("utf-8")
|
| 95 |
+
).hexdigest()[:12]
|
| 96 |
+
identity = "\x1f".join(
|
| 97 |
+
(
|
| 98 |
+
ticker,
|
| 99 |
+
str(latest_k.get("accession") or latest_k.get("filing_date") or ""),
|
| 100 |
+
str(latest_q.get("accession") or latest_q.get("filing_date") or ""),
|
| 101 |
+
latest_transcript,
|
| 102 |
+
business_digest,
|
| 103 |
+
geography_digest,
|
| 104 |
+
PROFILE_SCHEMA_VERSION,
|
| 105 |
+
)
|
| 106 |
+
)
|
| 107 |
+
return hashlib.sha256(identity.encode("utf-8")).hexdigest()[:24]
|
| 108 |
+
|
| 109 |
+
|
| 110 |
+
def profile_source_coverage(ticker: str) -> dict:
|
| 111 |
+
ticker = ticker.upper()
|
| 112 |
+
annual_periods = get_periods_for_ticker(ticker, form_type="10-K")
|
| 113 |
+
business_values = [
|
| 114 |
+
get_section(ticker, period, "business") for period in annual_periods
|
| 115 |
+
]
|
| 116 |
+
geography_values = [
|
| 117 |
+
get_section(ticker, period, "segments_geography") for period in annual_periods
|
| 118 |
+
]
|
| 119 |
+
business = sum(1 for value in business_values if (value or "").strip())
|
| 120 |
+
geography = sum(1 for value in geography_values if (value or "").strip())
|
| 121 |
+
transcript_count = len(get_recent_transcripts(ticker, n=4))
|
| 122 |
+
status = (
|
| 123 |
+
"COMPLETE"
|
| 124 |
+
if business >= 1 and geography >= 1 and transcript_count >= 2
|
| 125 |
+
else "PARTIAL"
|
| 126 |
+
)
|
| 127 |
+
return {
|
| 128 |
+
"status": status,
|
| 129 |
+
"annual_periods": len(annual_periods),
|
| 130 |
+
"business_sections": business,
|
| 131 |
+
"business_sections_attempted": sum(
|
| 132 |
+
1 for value in business_values if value is not None
|
| 133 |
+
),
|
| 134 |
+
"segments_geography_sections": geography,
|
| 135 |
+
"segments_geography_sections_attempted": sum(
|
| 136 |
+
1 for value in geography_values if value is not None
|
| 137 |
+
),
|
| 138 |
+
"recent_transcripts": transcript_count,
|
| 139 |
+
}
|
| 140 |
+
|
| 141 |
+
|
| 142 |
+
def _invoke_tool(tool, arguments: dict) -> str:
|
| 143 |
+
return tool.invoke(arguments)
|
| 144 |
+
|
| 145 |
+
|
| 146 |
+
def collect_profile_evidence(
|
| 147 |
+
ticker: str,
|
| 148 |
+
progress: Optional[Callable[[str], None]] = None,
|
| 149 |
+
) -> list[str]:
|
| 150 |
+
ticker = ticker.upper()
|
| 151 |
+
payloads: list[str] = []
|
| 152 |
+
calls = [
|
| 153 |
+
(get_financial_metrics, {"ticker": ticker}, "Loading verified financial history"),
|
| 154 |
+
(
|
| 155 |
+
search_filing,
|
| 156 |
+
{
|
| 157 |
+
"ticker": ticker,
|
| 158 |
+
"query": "business products services customers distribution competition",
|
| 159 |
+
"period": "",
|
| 160 |
+
"since": "",
|
| 161 |
+
},
|
| 162 |
+
"Reading the business model",
|
| 163 |
+
),
|
| 164 |
+
(
|
| 165 |
+
search_filing,
|
| 166 |
+
{
|
| 167 |
+
"ticker": ticker,
|
| 168 |
+
"query": "reportable segments revenue geographic information countries international sales",
|
| 169 |
+
"period": "",
|
| 170 |
+
"since": "",
|
| 171 |
+
},
|
| 172 |
+
"Mapping segments and geography",
|
| 173 |
+
),
|
| 174 |
+
(
|
| 175 |
+
search_filing,
|
| 176 |
+
{
|
| 177 |
+
"ticker": ticker,
|
| 178 |
+
"query": "strategy change product mix capital allocation growth drivers",
|
| 179 |
+
"period": "",
|
| 180 |
+
"since": "",
|
| 181 |
+
},
|
| 182 |
+
"Comparing strategic evolution",
|
| 183 |
+
),
|
| 184 |
+
(
|
| 185 |
+
search_transcript,
|
| 186 |
+
{
|
| 187 |
+
"ticker": ticker,
|
| 188 |
+
"query": "analyst questions demand competition margins products strategy risks",
|
| 189 |
+
"period": "",
|
| 190 |
+
"since": "",
|
| 191 |
+
},
|
| 192 |
+
"Finding investor attention themes",
|
| 193 |
+
),
|
| 194 |
+
]
|
| 195 |
+
for tool, arguments, label in calls:
|
| 196 |
+
if progress:
|
| 197 |
+
progress(label)
|
| 198 |
+
try:
|
| 199 |
+
payloads.append(_invoke_tool(tool, arguments))
|
| 200 |
+
except Exception as exc:
|
| 201 |
+
from agent.evidence import evidence_envelope
|
| 202 |
+
|
| 203 |
+
payloads.append(
|
| 204 |
+
evidence_envelope(
|
| 205 |
+
tool=tool.name,
|
| 206 |
+
query=arguments,
|
| 207 |
+
status="ERROR",
|
| 208 |
+
message=str(exc),
|
| 209 |
+
error_code="PROFILE_RETRIEVAL_ERROR",
|
| 210 |
+
)
|
| 211 |
+
)
|
| 212 |
+
return payloads
|
| 213 |
+
|
| 214 |
+
|
| 215 |
+
def _evidence_prompt(payloads: list[str]) -> str:
|
| 216 |
+
records = evidence_records_from(payloads)
|
| 217 |
+
compact = []
|
| 218 |
+
for record in records:
|
| 219 |
+
compact.append(
|
| 220 |
+
{
|
| 221 |
+
"ref": record.ref.model_dump(mode="json"),
|
| 222 |
+
"content": record.content[:5000],
|
| 223 |
+
"metadata": record.metadata,
|
| 224 |
+
}
|
| 225 |
+
)
|
| 226 |
+
return json.dumps(compact, ensure_ascii=False)
|
| 227 |
+
|
| 228 |
+
|
| 229 |
+
def _annual_trends(ticker: str) -> list[dict]:
|
| 230 |
+
rows = [
|
| 231 |
+
row
|
| 232 |
+
for row in metrics_db.get_all_metrics(ticker)
|
| 233 |
+
if row.get("form_type") == "10-K"
|
| 234 |
+
and row.get("data_quality_status") == "VERIFIED"
|
| 235 |
+
]
|
| 236 |
+
rows.sort(key=lambda row: str(row.get("report_date") or row.get("filing_date") or ""))
|
| 237 |
+
return [
|
| 238 |
+
{
|
| 239 |
+
"period": row.get("period") or "",
|
| 240 |
+
"revenue_bn": (
|
| 241 |
+
round(float(row["revenue"]) / 1e9, 3)
|
| 242 |
+
if row.get("revenue") is not None
|
| 243 |
+
else None
|
| 244 |
+
),
|
| 245 |
+
"revenue_yoy_pct": row.get("revenue_yoy_pct"),
|
| 246 |
+
"operating_margin": row.get("operating_margin"),
|
| 247 |
+
"free_cash_flow_bn": (
|
| 248 |
+
round(float(row["free_cash_flow"]) / 1e9, 3)
|
| 249 |
+
if row.get("free_cash_flow") is not None
|
| 250 |
+
else None
|
| 251 |
+
),
|
| 252 |
+
"eps": row.get("eps"),
|
| 253 |
+
}
|
| 254 |
+
for row in rows[-3:]
|
| 255 |
+
]
|
| 256 |
+
|
| 257 |
+
|
| 258 |
+
def _is_verified_fact(value) -> bool:
|
| 259 |
+
return isinstance(value, dict) and value.get("verification_status") == "VERIFIED"
|
| 260 |
+
|
| 261 |
+
|
| 262 |
+
def _prune_unverified(profile: dict) -> int:
|
| 263 |
+
removed = 0
|
| 264 |
+
identity = profile.get("identity")
|
| 265 |
+
if isinstance(identity, dict):
|
| 266 |
+
for key in ("one_liner", "business_model", "competitive_position"):
|
| 267 |
+
if identity.get(key) is not None and not _is_verified_fact(identity.get(key)):
|
| 268 |
+
identity[key] = None
|
| 269 |
+
removed += 1
|
| 270 |
+
customers = identity.get("customer_types")
|
| 271 |
+
if isinstance(customers, list):
|
| 272 |
+
kept = [item for item in customers if _is_verified_fact(item)]
|
| 273 |
+
removed += len(customers) - len(kept)
|
| 274 |
+
identity["customer_types"] = kept
|
| 275 |
+
|
| 276 |
+
for key, fact_key in (
|
| 277 |
+
("business_lines", "description"),
|
| 278 |
+
("geographic_exposures", "description"),
|
| 279 |
+
("strategic_changes", "change"),
|
| 280 |
+
("attention_themes", "evidence"),
|
| 281 |
+
("watch_variables", "evidence"),
|
| 282 |
+
):
|
| 283 |
+
values = profile.get(key)
|
| 284 |
+
if not isinstance(values, list):
|
| 285 |
+
continue
|
| 286 |
+
kept = [
|
| 287 |
+
item
|
| 288 |
+
for item in values
|
| 289 |
+
if isinstance(item, dict) and _is_verified_fact(item.get(fact_key))
|
| 290 |
+
]
|
| 291 |
+
removed += len(values) - len(kept)
|
| 292 |
+
profile[key] = kept
|
| 293 |
+
return removed
|
| 294 |
+
|
| 295 |
+
|
| 296 |
+
def _quarantine_unsupported_shares(profile: dict, records) -> int:
|
| 297 |
+
"""Hide parent-level revenue shares unless the cited record supports them."""
|
| 298 |
+
removed = 0
|
| 299 |
+
for key, fact_key in (
|
| 300 |
+
("business_lines", "description"),
|
| 301 |
+
("geographic_exposures", "description"),
|
| 302 |
+
):
|
| 303 |
+
for item in profile.get(key) or []:
|
| 304 |
+
if not isinstance(item, dict) or item.get("revenue_share_pct") is None:
|
| 305 |
+
continue
|
| 306 |
+
fact = item.get(fact_key)
|
| 307 |
+
if not isinstance(fact, dict):
|
| 308 |
+
item["revenue_share_pct"] = None
|
| 309 |
+
item["share_period" if key == "business_lines" else "period"] = None
|
| 310 |
+
removed += 1
|
| 311 |
+
continue
|
| 312 |
+
synthetic = {
|
| 313 |
+
**fact,
|
| 314 |
+
"text": f"{item['revenue_share_pct']}% revenue share",
|
| 315 |
+
}
|
| 316 |
+
verify_fact(synthetic, records)
|
| 317 |
+
if synthetic.get("verification_status") != "VERIFIED":
|
| 318 |
+
item["revenue_share_pct"] = None
|
| 319 |
+
item["share_period" if key == "business_lines" else "period"] = None
|
| 320 |
+
removed += 1
|
| 321 |
+
return removed
|
| 322 |
+
|
| 323 |
+
|
| 324 |
+
def _partial_profile(ticker: str, fingerprint: str, coverage: dict) -> dict:
|
| 325 |
+
rows = metrics_db.get_all_metrics(ticker)
|
| 326 |
+
company_name = rows[0].get("company_name") if rows else ticker
|
| 327 |
+
return CompanyProfile(
|
| 328 |
+
ticker=ticker,
|
| 329 |
+
company_name=company_name or ticker,
|
| 330 |
+
source_fingerprint=fingerprint,
|
| 331 |
+
generated_at=datetime.now(timezone.utc).isoformat(),
|
| 332 |
+
data_as_of=(rows[0].get("filing_date") if rows else None),
|
| 333 |
+
status="PARTIAL",
|
| 334 |
+
annual_trends=_annual_trends(ticker),
|
| 335 |
+
source_coverage=coverage,
|
| 336 |
+
).model_dump(mode="json")
|
| 337 |
+
|
| 338 |
+
|
| 339 |
+
def generate_company_profile(
|
| 340 |
+
ticker: str,
|
| 341 |
+
config: RunConfig,
|
| 342 |
+
progress: Optional[Callable[[str], None]] = None,
|
| 343 |
+
) -> dict:
|
| 344 |
+
ticker = ticker.upper()
|
| 345 |
+
fingerprint = source_fingerprint(ticker)
|
| 346 |
+
coverage = profile_source_coverage(ticker)
|
| 347 |
+
payloads = collect_profile_evidence(ticker, progress=progress)
|
| 348 |
+
records = evidence_records_from(payloads)
|
| 349 |
+
if not records:
|
| 350 |
+
return _partial_profile(ticker, fingerprint, coverage)
|
| 351 |
+
|
| 352 |
+
rows = metrics_db.get_all_metrics(ticker)
|
| 353 |
+
company_name = rows[0].get("company_name") if rows else ticker
|
| 354 |
+
if progress:
|
| 355 |
+
progress("Synthesizing the compact company primer")
|
| 356 |
+
llm = make_chat_model(config, max_retries=3, max_tokens=8192)
|
| 357 |
+
structured = llm.with_structured_output(CompanyProfile)
|
| 358 |
+
system = build_system_message(config, _PROFILE_PROMPT)
|
| 359 |
+
user = HumanMessage(
|
| 360 |
+
content=(
|
| 361 |
+
f"Create the Company Primer for {ticker} ({company_name}). "
|
| 362 |
+
"Return the structured CompanyProfile.\n\nEVIDENCE RECORDS:\n"
|
| 363 |
+
+ _evidence_prompt(payloads)
|
| 364 |
+
)
|
| 365 |
+
)
|
| 366 |
+
result = structured.invoke([system, user])
|
| 367 |
+
if isinstance(result, CompanyProfile):
|
| 368 |
+
profile = result.model_dump(mode="json")
|
| 369 |
+
elif isinstance(result, dict):
|
| 370 |
+
profile = CompanyProfile.model_validate(result).model_dump(mode="json")
|
| 371 |
+
else:
|
| 372 |
+
raise ValueError("The model did not return a CompanyProfile")
|
| 373 |
+
|
| 374 |
+
profile.update(
|
| 375 |
+
{
|
| 376 |
+
"ticker": ticker,
|
| 377 |
+
"company_name": company_name or ticker,
|
| 378 |
+
"schema_version": PROFILE_SCHEMA_VERSION,
|
| 379 |
+
"source_fingerprint": fingerprint,
|
| 380 |
+
"language": "English",
|
| 381 |
+
"generated_at": datetime.now(timezone.utc).isoformat(),
|
| 382 |
+
"data_as_of": max(
|
| 383 |
+
(record.ref.as_of for record in records if record.ref.as_of),
|
| 384 |
+
default=(rows[0].get("filing_date") if rows else None),
|
| 385 |
+
),
|
| 386 |
+
"model": config.model,
|
| 387 |
+
"annual_trends": _annual_trends(ticker),
|
| 388 |
+
"source_coverage": coverage,
|
| 389 |
+
"status": coverage["status"],
|
| 390 |
+
}
|
| 391 |
+
)
|
| 392 |
+
verify_brief_evidence(profile, payloads)
|
| 393 |
+
removed = _quarantine_unsupported_shares(profile, records)
|
| 394 |
+
removed += _prune_unverified(profile)
|
| 395 |
+
verification_report = profile.setdefault("verification_report", {})
|
| 396 |
+
verification_report["removed"] = (
|
| 397 |
+
int(verification_report.get("removed") or 0) + removed
|
| 398 |
+
)
|
| 399 |
+
profile["status"] = (
|
| 400 |
+
"COMPLETE"
|
| 401 |
+
if coverage.get("status") == "COMPLETE"
|
| 402 |
+
and (profile.get("evidence_coverage") or {}).get("status") == "VERIFIED"
|
| 403 |
+
and not verification_report["removed"]
|
| 404 |
+
else "PARTIAL"
|
| 405 |
+
)
|
| 406 |
+
|
| 407 |
+
from analysis.company_attention import attach_attention_stats
|
| 408 |
+
|
| 409 |
+
attach_attention_stats(profile, ticker, [])
|
| 410 |
+
return CompanyProfile.model_validate(profile).model_dump(mode="json")
|
| 411 |
+
|
| 412 |
+
|
| 413 |
+
def _collect_translation_segments(node, path=()):
|
| 414 |
+
result = []
|
| 415 |
+
if isinstance(node, dict):
|
| 416 |
+
for key, value in node.items():
|
| 417 |
+
if key in _NEVER_TRANSLATE:
|
| 418 |
+
continue
|
| 419 |
+
if key in _TRANSLATABLE_FIELDS and isinstance(value, str) and value:
|
| 420 |
+
result.append((path + (key,), value))
|
| 421 |
+
else:
|
| 422 |
+
result.extend(_collect_translation_segments(value, path + (key,)))
|
| 423 |
+
elif isinstance(node, list):
|
| 424 |
+
for index, value in enumerate(node):
|
| 425 |
+
result.extend(_collect_translation_segments(value, path + (index,)))
|
| 426 |
+
return result
|
| 427 |
+
|
| 428 |
+
|
| 429 |
+
def _set_path(value, path, replacement):
|
| 430 |
+
current = value
|
| 431 |
+
for step in path[:-1]:
|
| 432 |
+
current = current[step]
|
| 433 |
+
current[path[-1]] = replacement
|
| 434 |
+
|
| 435 |
+
|
| 436 |
+
def translate_company_profile(
|
| 437 |
+
profile: dict,
|
| 438 |
+
target_language: str,
|
| 439 |
+
config: RunConfig,
|
| 440 |
+
) -> tuple[dict, Optional[dict]]:
|
| 441 |
+
"""Translate display prose while preserving evidence snippets and identifiers."""
|
| 442 |
+
if not target_language or target_language == "English":
|
| 443 |
+
return profile, None
|
| 444 |
+
segments = _collect_translation_segments(profile)
|
| 445 |
+
if not segments:
|
| 446 |
+
return profile, None
|
| 447 |
+
payload = {str(index): text for index, (_, text) in enumerate(segments)}
|
| 448 |
+
prompt = (
|
| 449 |
+
f"Translate every JSON value into {target_language}. Return only a JSON "
|
| 450 |
+
"object with the same keys. Use concise professional equity-research "
|
| 451 |
+
"language. Preserve all numbers, tickers, company names, periods and "
|
| 452 |
+
"finance abbreviations."
|
| 453 |
+
)
|
| 454 |
+
try:
|
| 455 |
+
llm = make_chat_model(config, max_retries=2, max_tokens=8192)
|
| 456 |
+
response = llm.invoke(
|
| 457 |
+
[
|
| 458 |
+
build_system_message(config, prompt),
|
| 459 |
+
HumanMessage(content=json.dumps(payload, ensure_ascii=False)),
|
| 460 |
+
]
|
| 461 |
+
)
|
| 462 |
+
raw = response.content
|
| 463 |
+
if isinstance(raw, list):
|
| 464 |
+
raw = "".join(
|
| 465 |
+
str(block.get("text") or "")
|
| 466 |
+
for block in raw
|
| 467 |
+
if isinstance(block, dict)
|
| 468 |
+
)
|
| 469 |
+
from agent.graph import _extract_json
|
| 470 |
+
|
| 471 |
+
translated_values = json.loads(_extract_json(str(raw)))
|
| 472 |
+
translated = copy.deepcopy(profile)
|
| 473 |
+
for index, (path, _) in enumerate(segments):
|
| 474 |
+
replacement = translated_values.get(str(index))
|
| 475 |
+
if isinstance(replacement, str) and replacement:
|
| 476 |
+
_set_path(translated, path, replacement)
|
| 477 |
+
translated["language"] = target_language
|
| 478 |
+
from agent.cost_log import compute_run_cost
|
| 479 |
+
|
| 480 |
+
return translated, compute_run_cost([response], config.model)
|
| 481 |
+
except Exception:
|
| 482 |
+
return profile, None
|
agent/company_profile_schemas.py
ADDED
|
@@ -0,0 +1,120 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""Structured payload for the Company Primer research surface."""
|
| 2 |
+
from __future__ import annotations
|
| 3 |
+
|
| 4 |
+
from typing import Any, Literal, Optional
|
| 5 |
+
|
| 6 |
+
from pydantic import BaseModel, ConfigDict, Field
|
| 7 |
+
|
| 8 |
+
from agent.schemas import SourcedFact
|
| 9 |
+
|
| 10 |
+
|
| 11 |
+
class CompanyIdentity(BaseModel):
|
| 12 |
+
model_config = ConfigDict(extra="ignore")
|
| 13 |
+
|
| 14 |
+
one_liner: Optional[SourcedFact] = None
|
| 15 |
+
business_model: Optional[SourcedFact] = None
|
| 16 |
+
customer_types: list[SourcedFact] = Field(default_factory=list)
|
| 17 |
+
competitive_position: Optional[SourcedFact] = None
|
| 18 |
+
|
| 19 |
+
|
| 20 |
+
class BusinessLine(BaseModel):
|
| 21 |
+
model_config = ConfigDict(extra="ignore")
|
| 22 |
+
|
| 23 |
+
name: str
|
| 24 |
+
description: SourcedFact
|
| 25 |
+
economics: str = Field(
|
| 26 |
+
default="",
|
| 27 |
+
description="Concise analytical framing; displayed as AI experimental.",
|
| 28 |
+
)
|
| 29 |
+
trend: Literal["growing", "stable", "declining", "mixed", "not_disclosed"] = "not_disclosed"
|
| 30 |
+
revenue_share_pct: Optional[float] = Field(default=None, ge=0, le=100)
|
| 31 |
+
share_period: Optional[str] = None
|
| 32 |
+
|
| 33 |
+
|
| 34 |
+
class GeographicExposure(BaseModel):
|
| 35 |
+
model_config = ConfigDict(extra="ignore")
|
| 36 |
+
|
| 37 |
+
name: str
|
| 38 |
+
exposure_types: list[
|
| 39 |
+
Literal["revenue", "operations", "supply_chain", "regulation"]
|
| 40 |
+
] = Field(default_factory=list)
|
| 41 |
+
description: SourcedFact
|
| 42 |
+
revenue_share_pct: Optional[float] = Field(default=None, ge=0, le=100)
|
| 43 |
+
period: Optional[str] = None
|
| 44 |
+
|
| 45 |
+
|
| 46 |
+
class StrategicChange(BaseModel):
|
| 47 |
+
model_config = ConfigDict(extra="ignore")
|
| 48 |
+
|
| 49 |
+
period_from: str = ""
|
| 50 |
+
period_to: str = ""
|
| 51 |
+
change: SourcedFact
|
| 52 |
+
implication: str = Field(
|
| 53 |
+
default="",
|
| 54 |
+
description="One concise AI interpretation, never an investment recommendation.",
|
| 55 |
+
)
|
| 56 |
+
|
| 57 |
+
|
| 58 |
+
class AttentionPoint(BaseModel):
|
| 59 |
+
model_config = ConfigDict(extra="ignore")
|
| 60 |
+
|
| 61 |
+
period: str
|
| 62 |
+
mentions: int = Field(default=0, ge=0)
|
| 63 |
+
|
| 64 |
+
|
| 65 |
+
class AttentionTheme(BaseModel):
|
| 66 |
+
model_config = ConfigDict(extra="ignore")
|
| 67 |
+
|
| 68 |
+
theme: str
|
| 69 |
+
why_it_matters: str
|
| 70 |
+
evidence: SourcedFact
|
| 71 |
+
call_mentions: list[AttentionPoint] = Field(default_factory=list)
|
| 72 |
+
retrieved_news_count: int = Field(default=0, ge=0)
|
| 73 |
+
direction: Literal["rising", "stable", "falling", "new", "insufficient"] = "insufficient"
|
| 74 |
+
|
| 75 |
+
|
| 76 |
+
class WatchVariable(BaseModel):
|
| 77 |
+
model_config = ConfigDict(extra="ignore")
|
| 78 |
+
|
| 79 |
+
variable: str
|
| 80 |
+
why_it_matters: str
|
| 81 |
+
next_datapoint: str
|
| 82 |
+
alert_signal: str
|
| 83 |
+
evidence: SourcedFact
|
| 84 |
+
|
| 85 |
+
|
| 86 |
+
class AnnualTrend(BaseModel):
|
| 87 |
+
model_config = ConfigDict(extra="ignore")
|
| 88 |
+
|
| 89 |
+
period: str
|
| 90 |
+
revenue_bn: Optional[float] = None
|
| 91 |
+
revenue_yoy_pct: Optional[float] = None
|
| 92 |
+
operating_margin: Optional[float] = None
|
| 93 |
+
free_cash_flow_bn: Optional[float] = None
|
| 94 |
+
eps: Optional[float] = None
|
| 95 |
+
|
| 96 |
+
|
| 97 |
+
class CompanyProfile(BaseModel):
|
| 98 |
+
model_config = ConfigDict(extra="ignore")
|
| 99 |
+
|
| 100 |
+
ticker: str
|
| 101 |
+
company_name: str
|
| 102 |
+
schema_version: str = "company_profile.v1"
|
| 103 |
+
source_fingerprint: str = ""
|
| 104 |
+
language: str = "English"
|
| 105 |
+
generated_at: Optional[str] = None
|
| 106 |
+
data_as_of: Optional[str] = None
|
| 107 |
+
model: Optional[str] = None
|
| 108 |
+
status: Literal["COMPLETE", "PARTIAL"] = "PARTIAL"
|
| 109 |
+
|
| 110 |
+
identity: CompanyIdentity = Field(default_factory=CompanyIdentity)
|
| 111 |
+
business_lines: list[BusinessLine] = Field(default_factory=list)
|
| 112 |
+
geographic_exposures: list[GeographicExposure] = Field(default_factory=list)
|
| 113 |
+
strategic_changes: list[StrategicChange] = Field(default_factory=list)
|
| 114 |
+
attention_themes: list[AttentionTheme] = Field(default_factory=list)
|
| 115 |
+
watch_variables: list[WatchVariable] = Field(default_factory=list)
|
| 116 |
+
annual_trends: list[AnnualTrend] = Field(default_factory=list)
|
| 117 |
+
|
| 118 |
+
source_coverage: dict[str, Any] = Field(default_factory=dict)
|
| 119 |
+
evidence_coverage: dict[str, Any] = Field(default_factory=dict)
|
| 120 |
+
verification_report: dict[str, Any] = Field(default_factory=dict)
|
analysis/company_attention.py
ADDED
|
@@ -0,0 +1,150 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""Deterministic investor-attention signals from earnings-call Q&A."""
|
| 2 |
+
from __future__ import annotations
|
| 3 |
+
|
| 4 |
+
import re
|
| 5 |
+
from collections import Counter
|
| 6 |
+
from typing import Iterable
|
| 7 |
+
|
| 8 |
+
from analysis.transcript_parse import parse_call
|
| 9 |
+
from storage.sections_db import get_recent_transcripts
|
| 10 |
+
|
| 11 |
+
_STOPWORDS = frozenset(
|
| 12 |
+
"about after again also analyst and are been before being between can could "
|
| 13 |
+
"company quarter quarters did does doing for from give given going have how "
|
| 14 |
+
"into just like more much next now our please really say should some talk "
|
| 15 |
+
"than that the their them then there these they think this those through "
|
| 16 |
+
"very want was were what when where which why will with would year years "
|
| 17 |
+
"you your".split()
|
| 18 |
+
)
|
| 19 |
+
|
| 20 |
+
|
| 21 |
+
def _tokens(text: str) -> set[str]:
|
| 22 |
+
return {
|
| 23 |
+
token
|
| 24 |
+
for token in re.findall(r"[a-z][a-z0-9-]{2,}", (text or "").lower())
|
| 25 |
+
if token not in _STOPWORDS
|
| 26 |
+
}
|
| 27 |
+
|
| 28 |
+
|
| 29 |
+
def _similar(left: set[str], right: set[str]) -> bool:
|
| 30 |
+
if not left or not right:
|
| 31 |
+
return False
|
| 32 |
+
overlap = len(left & right)
|
| 33 |
+
return overlap >= 2 and overlap / len(left | right) >= 0.12
|
| 34 |
+
|
| 35 |
+
|
| 36 |
+
def cluster_questions(calls: Iterable[tuple[str, str]], max_themes: int = 5) -> list[dict]:
|
| 37 |
+
"""Greedily cluster analyst questions and return deterministic counts."""
|
| 38 |
+
clusters: list[dict] = []
|
| 39 |
+
periods: list[str] = []
|
| 40 |
+
for period, raw in calls:
|
| 41 |
+
periods.append(period)
|
| 42 |
+
parsed = parse_call(period, raw)
|
| 43 |
+
for exchange in parsed.qa:
|
| 44 |
+
question = (exchange.question or "").strip()
|
| 45 |
+
tokens = _tokens(question)
|
| 46 |
+
if len(tokens) < 3:
|
| 47 |
+
continue
|
| 48 |
+
best = None
|
| 49 |
+
best_overlap = 0
|
| 50 |
+
for cluster in clusters:
|
| 51 |
+
overlap = len(tokens & cluster["tokens"])
|
| 52 |
+
if overlap > best_overlap and _similar(tokens, cluster["tokens"]):
|
| 53 |
+
best = cluster
|
| 54 |
+
best_overlap = overlap
|
| 55 |
+
if best is None:
|
| 56 |
+
best = {
|
| 57 |
+
"tokens": set(tokens),
|
| 58 |
+
"questions": [],
|
| 59 |
+
"mentions": Counter(),
|
| 60 |
+
}
|
| 61 |
+
clusters.append(best)
|
| 62 |
+
best["tokens"].update(tokens)
|
| 63 |
+
best["questions"].append(question)
|
| 64 |
+
best["mentions"][period] += 1
|
| 65 |
+
|
| 66 |
+
usable = [
|
| 67 |
+
cluster
|
| 68 |
+
for cluster in clusters
|
| 69 |
+
if len(cluster["questions"]) >= 2
|
| 70 |
+
]
|
| 71 |
+
usable.sort(
|
| 72 |
+
key=lambda cluster: (
|
| 73 |
+
-len(cluster["mentions"]),
|
| 74 |
+
-len(cluster["questions"]),
|
| 75 |
+
)
|
| 76 |
+
)
|
| 77 |
+
|
| 78 |
+
output: list[dict] = []
|
| 79 |
+
for cluster in usable[:max_themes]:
|
| 80 |
+
counts = Counter()
|
| 81 |
+
for question in cluster["questions"]:
|
| 82 |
+
counts.update(_tokens(question))
|
| 83 |
+
label_tokens = [token for token, _ in counts.most_common(3)]
|
| 84 |
+
output.append(
|
| 85 |
+
{
|
| 86 |
+
"label": " / ".join(label_tokens),
|
| 87 |
+
"tokens": label_tokens,
|
| 88 |
+
"mentions": [
|
| 89 |
+
{"period": period, "mentions": int(cluster["mentions"].get(period, 0))}
|
| 90 |
+
for period in periods
|
| 91 |
+
],
|
| 92 |
+
"question_count": len(cluster["questions"]),
|
| 93 |
+
"period_count": len(cluster["mentions"]),
|
| 94 |
+
"representative_question": cluster["questions"][0],
|
| 95 |
+
}
|
| 96 |
+
)
|
| 97 |
+
return output
|
| 98 |
+
|
| 99 |
+
|
| 100 |
+
def get_attention_clusters(ticker: str, n: int = 4, max_themes: int = 5) -> list[dict]:
|
| 101 |
+
return cluster_questions(get_recent_transcripts(ticker.upper(), n=n), max_themes=max_themes)
|
| 102 |
+
|
| 103 |
+
|
| 104 |
+
def attach_attention_stats(profile: dict, ticker: str, news_items: list[dict]) -> dict:
|
| 105 |
+
"""Attach Q&A and retrieved-news counts to synthesized attention themes."""
|
| 106 |
+
themes = profile.get("attention_themes")
|
| 107 |
+
if not isinstance(themes, list) or not themes:
|
| 108 |
+
return profile
|
| 109 |
+
clusters = get_attention_clusters(ticker)
|
| 110 |
+
unused = set(range(len(clusters)))
|
| 111 |
+
|
| 112 |
+
for index, theme in enumerate(themes):
|
| 113 |
+
if not isinstance(theme, dict):
|
| 114 |
+
continue
|
| 115 |
+
theme_tokens = _tokens(str(theme.get("theme") or ""))
|
| 116 |
+
scored = [
|
| 117 |
+
(len(theme_tokens & set(cluster.get("tokens") or [])), cluster_index)
|
| 118 |
+
for cluster_index, cluster in enumerate(clusters)
|
| 119 |
+
if cluster_index in unused
|
| 120 |
+
]
|
| 121 |
+
scored.sort(reverse=True)
|
| 122 |
+
cluster_index = scored[0][1] if scored and scored[0][0] > 0 else None
|
| 123 |
+
if cluster_index is None and index < len(clusters) and index in unused:
|
| 124 |
+
cluster_index = index
|
| 125 |
+
if cluster_index is not None:
|
| 126 |
+
unused.discard(cluster_index)
|
| 127 |
+
mentions = clusters[cluster_index]["mentions"]
|
| 128 |
+
theme["call_mentions"] = mentions
|
| 129 |
+
values = [point["mentions"] for point in mentions]
|
| 130 |
+
if values:
|
| 131 |
+
if values[0] == 0 and values[-1] > 0:
|
| 132 |
+
theme["direction"] = "new"
|
| 133 |
+
elif values[-1] > values[0]:
|
| 134 |
+
theme["direction"] = "rising"
|
| 135 |
+
elif values[-1] < values[0]:
|
| 136 |
+
theme["direction"] = "falling"
|
| 137 |
+
else:
|
| 138 |
+
theme["direction"] = "stable"
|
| 139 |
+
|
| 140 |
+
expanded_tokens = theme_tokens | _tokens(
|
| 141 |
+
str((theme.get("evidence") or {}).get("text") or "")
|
| 142 |
+
)
|
| 143 |
+
theme["retrieved_news_count"] = sum(
|
| 144 |
+
1
|
| 145 |
+
for item in news_items
|
| 146 |
+
if expanded_tokens
|
| 147 |
+
and expanded_tokens
|
| 148 |
+
& _tokens(f"{item.get('title', '')} {item.get('snippet', '')}")
|
| 149 |
+
)
|
| 150 |
+
return profile
|
analytics/company_market.py
ADDED
|
@@ -0,0 +1,246 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""Relative price history and event candidates for Company Primer."""
|
| 2 |
+
from __future__ import annotations
|
| 3 |
+
|
| 4 |
+
import os
|
| 5 |
+
from datetime import datetime
|
| 6 |
+
from email.utils import parsedate_to_datetime
|
| 7 |
+
from typing import Iterable, Optional
|
| 8 |
+
|
| 9 |
+
from storage.earnings_cache import get as cache_get, set as cache_set
|
| 10 |
+
|
| 11 |
+
SECTOR_ETFS = {
|
| 12 |
+
"technology": "XLK",
|
| 13 |
+
"information technology": "XLK",
|
| 14 |
+
"consumer cyclical": "XLY",
|
| 15 |
+
"consumer discretionary": "XLY",
|
| 16 |
+
"communication services": "XLC",
|
| 17 |
+
"financial services": "XLF",
|
| 18 |
+
"financials": "XLF",
|
| 19 |
+
"healthcare": "XLV",
|
| 20 |
+
"health care": "XLV",
|
| 21 |
+
"industrials": "XLI",
|
| 22 |
+
"energy": "XLE",
|
| 23 |
+
"consumer defensive": "XLP",
|
| 24 |
+
"consumer staples": "XLP",
|
| 25 |
+
"utilities": "XLU",
|
| 26 |
+
"real estate": "XLRE",
|
| 27 |
+
"basic materials": "XLB",
|
| 28 |
+
"materials": "XLB",
|
| 29 |
+
}
|
| 30 |
+
|
| 31 |
+
|
| 32 |
+
def sector_etf(sector: str) -> str:
|
| 33 |
+
return SECTOR_ETFS.get((sector or "").strip().lower(), "SPY")
|
| 34 |
+
|
| 35 |
+
|
| 36 |
+
def select_relative_events(
|
| 37 |
+
dates: list[str],
|
| 38 |
+
company_values: list[float],
|
| 39 |
+
sector_values: list[float],
|
| 40 |
+
*,
|
| 41 |
+
threshold_pct: float = 2.0,
|
| 42 |
+
per_direction: int = 3,
|
| 43 |
+
min_spacing: int = 5,
|
| 44 |
+
) -> list[dict]:
|
| 45 |
+
"""Select balanced, non-overlapping relative one-day moves."""
|
| 46 |
+
candidates: list[dict] = []
|
| 47 |
+
for index in range(1, min(len(dates), len(company_values), len(sector_values))):
|
| 48 |
+
prior_company = company_values[index - 1]
|
| 49 |
+
prior_sector = sector_values[index - 1]
|
| 50 |
+
if not prior_company or not prior_sector:
|
| 51 |
+
continue
|
| 52 |
+
company_return = (company_values[index] / prior_company - 1) * 100
|
| 53 |
+
sector_return = (sector_values[index] / prior_sector - 1) * 100
|
| 54 |
+
excess = company_return - sector_return
|
| 55 |
+
if abs(excess) >= threshold_pct:
|
| 56 |
+
candidates.append(
|
| 57 |
+
{
|
| 58 |
+
"index": index,
|
| 59 |
+
"date": dates[index],
|
| 60 |
+
"company_return_pct": round(company_return, 2),
|
| 61 |
+
"sector_return_pct": round(sector_return, 2),
|
| 62 |
+
"sector_excess_pct": round(excess, 2),
|
| 63 |
+
}
|
| 64 |
+
)
|
| 65 |
+
|
| 66 |
+
selected: list[dict] = []
|
| 67 |
+
for sign in (1, -1):
|
| 68 |
+
side = [item for item in candidates if item["sector_excess_pct"] * sign > 0]
|
| 69 |
+
side.sort(key=lambda item: abs(item["sector_excess_pct"]), reverse=True)
|
| 70 |
+
for item in side:
|
| 71 |
+
if any(abs(item["index"] - chosen["index"]) < min_spacing for chosen in selected):
|
| 72 |
+
continue
|
| 73 |
+
selected.append(item)
|
| 74 |
+
if sum(1 for chosen in selected if chosen["sector_excess_pct"] * sign > 0) >= per_direction:
|
| 75 |
+
break
|
| 76 |
+
selected.sort(key=lambda item: item["date"])
|
| 77 |
+
return selected
|
| 78 |
+
|
| 79 |
+
|
| 80 |
+
def _to_date(value) -> str:
|
| 81 |
+
if hasattr(value, "date"):
|
| 82 |
+
value = value.date()
|
| 83 |
+
return str(value)[:10]
|
| 84 |
+
|
| 85 |
+
|
| 86 |
+
def _parse_published_date(value: object):
|
| 87 |
+
raw = str(value or "").strip()
|
| 88 |
+
if not raw:
|
| 89 |
+
return None
|
| 90 |
+
try:
|
| 91 |
+
return datetime.fromisoformat(raw[:10]).date()
|
| 92 |
+
except ValueError:
|
| 93 |
+
try:
|
| 94 |
+
return parsedate_to_datetime(raw).date()
|
| 95 |
+
except (TypeError, ValueError, OverflowError):
|
| 96 |
+
return None
|
| 97 |
+
|
| 98 |
+
|
| 99 |
+
def _price_rows(frame, symbols: Iterable[str]) -> list[dict]:
|
| 100 |
+
close = frame["Close"] if "Close" in frame else frame
|
| 101 |
+
symbols = list(symbols)
|
| 102 |
+
rows: list[dict] = []
|
| 103 |
+
for index, raw_date in enumerate(close.index):
|
| 104 |
+
row = {"date": _to_date(raw_date)}
|
| 105 |
+
valid = True
|
| 106 |
+
for symbol in symbols:
|
| 107 |
+
try:
|
| 108 |
+
value = close[symbol].iloc[index]
|
| 109 |
+
value = float(value)
|
| 110 |
+
if value != value:
|
| 111 |
+
valid = False
|
| 112 |
+
break
|
| 113 |
+
row[symbol] = value
|
| 114 |
+
except Exception:
|
| 115 |
+
valid = False
|
| 116 |
+
break
|
| 117 |
+
if valid:
|
| 118 |
+
rows.append(row)
|
| 119 |
+
return rows
|
| 120 |
+
|
| 121 |
+
|
| 122 |
+
def _event_context(
|
| 123 |
+
ticker: str,
|
| 124 |
+
company_name: str,
|
| 125 |
+
event_date: str,
|
| 126 |
+
) -> Optional[dict]:
|
| 127 |
+
cache_key = f"COMPANY_EVENT:v1:{ticker.upper()}:{event_date}"
|
| 128 |
+
cached = cache_get(cache_key, ttl_hours=24 * 7)
|
| 129 |
+
if cached is not None:
|
| 130 |
+
return cached or None
|
| 131 |
+
key = os.getenv("TAVILY_API_KEY", "").strip()
|
| 132 |
+
if not key:
|
| 133 |
+
return None
|
| 134 |
+
try:
|
| 135 |
+
from tavily import TavilyClient
|
| 136 |
+
|
| 137 |
+
results = TavilyClient(api_key=key).search(
|
| 138 |
+
query=f"{ticker.upper()} {company_name} {event_date} stock news",
|
| 139 |
+
max_results=3,
|
| 140 |
+
search_depth="basic",
|
| 141 |
+
topic="news",
|
| 142 |
+
).get("results", [])
|
| 143 |
+
target = datetime.fromisoformat(event_date).date()
|
| 144 |
+
best = None
|
| 145 |
+
for item in results:
|
| 146 |
+
published = _parse_published_date(item.get("published_date"))
|
| 147 |
+
if published is None:
|
| 148 |
+
continue
|
| 149 |
+
if abs((published - target).days) <= 2:
|
| 150 |
+
best = {
|
| 151 |
+
"headline": str(item.get("title") or ""),
|
| 152 |
+
"url": str(item.get("url") or ""),
|
| 153 |
+
"published_date": published.isoformat(),
|
| 154 |
+
"attribution_status": "CORROBORATED_EVENT",
|
| 155 |
+
}
|
| 156 |
+
break
|
| 157 |
+
cache_set(cache_key, best or {})
|
| 158 |
+
return best
|
| 159 |
+
except Exception:
|
| 160 |
+
return None
|
| 161 |
+
|
| 162 |
+
|
| 163 |
+
def fetch_company_market(
|
| 164 |
+
ticker: str,
|
| 165 |
+
company_name: str,
|
| 166 |
+
) -> tuple[Optional[dict], Optional[str]]:
|
| 167 |
+
ticker = ticker.upper()
|
| 168 |
+
cache_key = f"COMPANY_MARKET:v1:{ticker}:3y"
|
| 169 |
+
cached = cache_get(cache_key, ttl_hours=6)
|
| 170 |
+
if cached is not None:
|
| 171 |
+
return cached, None
|
| 172 |
+
|
| 173 |
+
try:
|
| 174 |
+
from ingestion.alphavantage import fetch_overview
|
| 175 |
+
|
| 176 |
+
overview, _overview_error = fetch_overview(ticker)
|
| 177 |
+
except Exception:
|
| 178 |
+
overview = None
|
| 179 |
+
sector = str((overview or {}).get("Sector") or "")
|
| 180 |
+
industry = str((overview or {}).get("Industry") or "")
|
| 181 |
+
benchmark = sector_etf(sector)
|
| 182 |
+
symbols = [ticker, "SPY"] + ([] if benchmark == "SPY" else [benchmark])
|
| 183 |
+
|
| 184 |
+
try:
|
| 185 |
+
import yfinance as yf
|
| 186 |
+
|
| 187 |
+
frame = yf.download(
|
| 188 |
+
symbols,
|
| 189 |
+
period="3y",
|
| 190 |
+
interval="1d",
|
| 191 |
+
auto_adjust=True,
|
| 192 |
+
progress=False,
|
| 193 |
+
threads=False,
|
| 194 |
+
)
|
| 195 |
+
rows = _price_rows(frame, symbols)
|
| 196 |
+
except Exception as exc:
|
| 197 |
+
return None, f"price history unavailable: {exc}"
|
| 198 |
+
if len(rows) < 2:
|
| 199 |
+
return None, "price history unavailable"
|
| 200 |
+
|
| 201 |
+
dates = [row["date"] for row in rows]
|
| 202 |
+
company_values = [row[ticker] for row in rows]
|
| 203 |
+
sector_values = [row[benchmark] for row in rows]
|
| 204 |
+
events = select_relative_events(dates, company_values, sector_values)
|
| 205 |
+
for event in events:
|
| 206 |
+
context = _event_context(ticker, company_name, event["date"])
|
| 207 |
+
if context:
|
| 208 |
+
event.update(context)
|
| 209 |
+
else:
|
| 210 |
+
event.update(
|
| 211 |
+
{
|
| 212 |
+
"headline": "No verified catalyst found",
|
| 213 |
+
"url": "",
|
| 214 |
+
"published_date": "",
|
| 215 |
+
"attribution_status": "NO_VERIFIED_CATALYST",
|
| 216 |
+
}
|
| 217 |
+
)
|
| 218 |
+
|
| 219 |
+
bases = {symbol: rows[0][symbol] for symbol in symbols}
|
| 220 |
+
weekly_rows = rows[::5]
|
| 221 |
+
if weekly_rows[-1]["date"] != rows[-1]["date"]:
|
| 222 |
+
weekly_rows.append(rows[-1])
|
| 223 |
+
normalized = [
|
| 224 |
+
{
|
| 225 |
+
"date": row["date"],
|
| 226 |
+
**{
|
| 227 |
+
symbol: round(row[symbol] / bases[symbol] * 100, 2)
|
| 228 |
+
for symbol in symbols
|
| 229 |
+
},
|
| 230 |
+
}
|
| 231 |
+
for row in weekly_rows
|
| 232 |
+
]
|
| 233 |
+
result = {
|
| 234 |
+
"ticker": ticker,
|
| 235 |
+
"company_name": company_name,
|
| 236 |
+
"sector": sector,
|
| 237 |
+
"industry": industry,
|
| 238 |
+
"benchmark": benchmark,
|
| 239 |
+
"market_cap": (overview or {}).get("MarketCapitalization"),
|
| 240 |
+
"price_source": "Yahoo Finance",
|
| 241 |
+
"history": normalized,
|
| 242 |
+
"events": events,
|
| 243 |
+
"as_of": rows[-1]["date"],
|
| 244 |
+
}
|
| 245 |
+
cache_set(cache_key, result)
|
| 246 |
+
return result, None
|
analytics/company_news.py
ADDED
|
@@ -0,0 +1,51 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""Public-news feed for Company Primer, cached independently from the profile."""
|
| 2 |
+
from __future__ import annotations
|
| 3 |
+
|
| 4 |
+
import os
|
| 5 |
+
from typing import Optional
|
| 6 |
+
|
| 7 |
+
from storage.earnings_cache import get as cache_get, set as cache_set
|
| 8 |
+
|
| 9 |
+
|
| 10 |
+
def fetch_company_news(
|
| 11 |
+
ticker: str,
|
| 12 |
+
company_name: str,
|
| 13 |
+
*,
|
| 14 |
+
days: int = 90,
|
| 15 |
+
max_results: int = 10,
|
| 16 |
+
) -> tuple[list[dict], Optional[str]]:
|
| 17 |
+
cache_key = f"COMPANY_NEWS:v1:{ticker.upper()}:{days}:{max_results}"
|
| 18 |
+
cached = cache_get(cache_key, ttl_hours=1)
|
| 19 |
+
if cached is not None:
|
| 20 |
+
return list(cached.get("items") or []), None
|
| 21 |
+
|
| 22 |
+
key = os.getenv("TAVILY_API_KEY", "").strip()
|
| 23 |
+
if not key:
|
| 24 |
+
return [], "TAVILY_API_KEY not set"
|
| 25 |
+
try:
|
| 26 |
+
from tavily import TavilyClient
|
| 27 |
+
|
| 28 |
+
results = TavilyClient(api_key=key).search(
|
| 29 |
+
query=(
|
| 30 |
+
f"{ticker.upper()} {company_name} company strategy products "
|
| 31 |
+
"earnings regulation acquisition"
|
| 32 |
+
),
|
| 33 |
+
max_results=max_results,
|
| 34 |
+
search_depth="basic",
|
| 35 |
+
days=days,
|
| 36 |
+
topic="news",
|
| 37 |
+
).get("results", [])
|
| 38 |
+
items = [
|
| 39 |
+
{
|
| 40 |
+
"title": str(item.get("title") or "Untitled"),
|
| 41 |
+
"url": str(item.get("url") or ""),
|
| 42 |
+
"published_date": str(item.get("published_date") or ""),
|
| 43 |
+
"snippet": str(item.get("content") or "")[:500],
|
| 44 |
+
}
|
| 45 |
+
for item in results
|
| 46 |
+
if item.get("url")
|
| 47 |
+
]
|
| 48 |
+
cache_set(cache_key, {"items": items})
|
| 49 |
+
return items, None
|
| 50 |
+
except Exception as exc:
|
| 51 |
+
return [], f"news fetch failed: {exc}"
|
app.py
CHANGED
|
@@ -27,12 +27,12 @@ if not st.session_state.get("_reranker_warmed"):
|
|
| 27 |
# ── UI language — default to English (finance lingua franca) ───────────────────
|
| 28 |
st.session_state.setdefault("ui_lang", "en")
|
| 29 |
|
| 30 |
-
# ── Navigation state — default to
|
| 31 |
-
st.session_state.setdefault("nav_key", "
|
| 32 |
# Remap stale route keys from the pre-Decision-Stack navigation (open sessions).
|
| 33 |
_nk = st.session_state["nav_key"]
|
| 34 |
if _nk not in VALID_KEYS:
|
| 35 |
-
st.session_state["nav_key"] = LEGACY_NAV.get(_nk, "
|
| 36 |
|
| 37 |
# ── Generation state ────────────────────────────────────────────────────────────
|
| 38 |
st.session_state.setdefault("gen", {
|
|
@@ -44,6 +44,7 @@ st.session_state.setdefault("gen", {
|
|
| 44 |
"all_messages": [],
|
| 45 |
"config": None,
|
| 46 |
})
|
|
|
|
| 47 |
|
| 48 |
|
| 49 |
|
|
@@ -566,13 +567,202 @@ def _live_trace_fragment() -> None:
|
|
| 566 |
|
| 567 |
|
| 568 |
# ── Main content routing ──────────────────────────────────────────────────────────
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 569 |
gen_error = st.session_state.pop("_gen_error", None)
|
| 570 |
if gen_error:
|
| 571 |
err_ticker, err_msg = gen_error
|
| 572 |
st.error(f"{t('brief_error_prefix')} **{err_ticker}**: {err_msg}")
|
| 573 |
|
| 574 |
gen = st.session_state["gen"]
|
| 575 |
-
active_key = st.session_state.get("nav_key", "
|
| 576 |
|
| 577 |
if gen["running"] or gen.get("trace"):
|
| 578 |
_live_trace_fragment()
|
|
@@ -596,7 +786,9 @@ else:
|
|
| 596 |
if brief:
|
| 597 |
_render_ticker_bar(ticker_input, brief)
|
| 598 |
|
| 599 |
-
if active_key == "
|
|
|
|
|
|
|
| 600 |
if brief:
|
| 601 |
from dashboard.verdict import render as render_verdict
|
| 602 |
render_verdict(brief)
|
|
|
|
| 27 |
# ── UI language — default to English (finance lingua franca) ───────────────────
|
| 28 |
st.session_state.setdefault("ui_lang", "en")
|
| 29 |
|
| 30 |
+
# ── Navigation state — default to Company Primer (the onboarding view) ─────────
|
| 31 |
+
st.session_state.setdefault("nav_key", "company")
|
| 32 |
# Remap stale route keys from the pre-Decision-Stack navigation (open sessions).
|
| 33 |
_nk = st.session_state["nav_key"]
|
| 34 |
if _nk not in VALID_KEYS:
|
| 35 |
+
st.session_state["nav_key"] = LEGACY_NAV.get(_nk, "company")
|
| 36 |
|
| 37 |
# ── Generation state ────────────────────────────────────────────────────────────
|
| 38 |
st.session_state.setdefault("gen", {
|
|
|
|
| 44 |
"all_messages": [],
|
| 45 |
"config": None,
|
| 46 |
})
|
| 47 |
+
st.session_state.setdefault("profile_jobs", {})
|
| 48 |
|
| 49 |
|
| 50 |
|
|
|
|
| 567 |
|
| 568 |
|
| 569 |
# ── Main content routing ──────────────────────────────────────────────────────────
|
| 570 |
+
def _run_company_profile_thread(
|
| 571 |
+
ticker: str,
|
| 572 |
+
job: dict,
|
| 573 |
+
config: RunConfig,
|
| 574 |
+
) -> None:
|
| 575 |
+
"""Generate and persist one canonical English Company Primer."""
|
| 576 |
+
try:
|
| 577 |
+
from agent.company_profile import generate_company_profile
|
| 578 |
+
from storage import company_profiles
|
| 579 |
+
|
| 580 |
+
def _progress(label: str) -> None:
|
| 581 |
+
job["label"] = label
|
| 582 |
+
job["completed_steps"] = min(int(job.get("completed_steps") or 0) + 1, 6)
|
| 583 |
+
|
| 584 |
+
profile = generate_company_profile(ticker, config, progress=_progress)
|
| 585 |
+
profile_id = company_profiles.save_profile(ticker, profile)
|
| 586 |
+
job["profile"] = profile
|
| 587 |
+
job["profile_id"] = profile_id
|
| 588 |
+
except Exception as exc:
|
| 589 |
+
import sys
|
| 590 |
+
|
| 591 |
+
print(f"[company profile thread error] {exc}", file=sys.stderr)
|
| 592 |
+
job["error"] = classify_llm_error(exc, config.provider) or str(exc)
|
| 593 |
+
finally:
|
| 594 |
+
job["running"] = False
|
| 595 |
+
|
| 596 |
+
|
| 597 |
+
def _start_company_profile_job(ticker: str, fingerprint: str, config: RunConfig) -> dict:
|
| 598 |
+
jobs = st.session_state.setdefault("profile_jobs", {})
|
| 599 |
+
job = {
|
| 600 |
+
"running": True,
|
| 601 |
+
"ticker": ticker,
|
| 602 |
+
"fingerprint": fingerprint,
|
| 603 |
+
"label": t("primer_generating"),
|
| 604 |
+
"completed_steps": 0,
|
| 605 |
+
"profile": None,
|
| 606 |
+
"profile_id": None,
|
| 607 |
+
"error": None,
|
| 608 |
+
"config": config,
|
| 609 |
+
}
|
| 610 |
+
jobs[ticker] = job
|
| 611 |
+
thread = threading.Thread(
|
| 612 |
+
target=_run_company_profile_thread,
|
| 613 |
+
args=(ticker, job, config),
|
| 614 |
+
daemon=True,
|
| 615 |
+
)
|
| 616 |
+
thread.start()
|
| 617 |
+
return job
|
| 618 |
+
|
| 619 |
+
|
| 620 |
+
def _render_company_profile_progress(job: dict) -> None:
|
| 621 |
+
completed = int(job.get("completed_steps") or 0)
|
| 622 |
+
progress = min(max(completed / 6, 0.04), 0.96 if job.get("running") else 1.0)
|
| 623 |
+
st.markdown(
|
| 624 |
+
f'<div style="background:#ffffff;border:1px solid #e5e7eb;border-radius:12px;'
|
| 625 |
+
f'padding:16px 20px;margin-bottom:16px;">'
|
| 626 |
+
f'<div style="display:flex;justify-content:space-between;gap:12px;">'
|
| 627 |
+
f'<strong>{t("primer_generating")} · {job.get("ticker", "")}</strong>'
|
| 628 |
+
f'<span style="color:#10b981;font-weight:700;">{int(progress * 100)}%</span></div>'
|
| 629 |
+
f'<div style="font-size:0.75rem;color:#6b7280;margin:5px 0 10px;">'
|
| 630 |
+
f'{job.get("label") or ""}</div>'
|
| 631 |
+
f'<div style="height:7px;background:#e5e7eb;border-radius:999px;overflow:hidden;">'
|
| 632 |
+
f'<div style="height:100%;width:{int(progress * 100)}%;background:#10b981;'
|
| 633 |
+
f'border-radius:999px;"></div></div></div>',
|
| 634 |
+
unsafe_allow_html=True,
|
| 635 |
+
)
|
| 636 |
+
|
| 637 |
+
|
| 638 |
+
def _display_company_profile(profile: dict, config: RunConfig | None) -> dict:
|
| 639 |
+
target_language = i18n.report_language()
|
| 640 |
+
if target_language == "English":
|
| 641 |
+
return profile
|
| 642 |
+
profile_id = profile.get("_profile_id")
|
| 643 |
+
if not profile_id:
|
| 644 |
+
return profile
|
| 645 |
+
try:
|
| 646 |
+
from storage import company_profiles
|
| 647 |
+
|
| 648 |
+
cached = company_profiles.get_translation(profile_id, target_language)
|
| 649 |
+
if cached:
|
| 650 |
+
return cached
|
| 651 |
+
if config is None:
|
| 652 |
+
return profile
|
| 653 |
+
from agent.company_profile import translate_company_profile
|
| 654 |
+
|
| 655 |
+
with st.spinner(t("translating_brief")):
|
| 656 |
+
translated, usage = translate_company_profile(profile, target_language, config)
|
| 657 |
+
if translated is not profile and translated.get("language") == target_language:
|
| 658 |
+
company_profiles.save_translation(profile_id, target_language, translated)
|
| 659 |
+
_absorb_translation_cost(str(profile.get("ticker") or ""), usage, config.model)
|
| 660 |
+
return translated
|
| 661 |
+
except Exception:
|
| 662 |
+
pass
|
| 663 |
+
st.caption(t("translation_failed"))
|
| 664 |
+
return profile
|
| 665 |
+
|
| 666 |
+
|
| 667 |
+
def _render_company_route(ticker: str, config: RunConfig | None) -> None:
|
| 668 |
+
from agent.company_profile import profile_source_coverage, source_fingerprint
|
| 669 |
+
from storage import company_profiles
|
| 670 |
+
|
| 671 |
+
fingerprint = source_fingerprint(ticker)
|
| 672 |
+
profile = company_profiles.get_latest_profile(
|
| 673 |
+
ticker,
|
| 674 |
+
source_fingerprint=fingerprint,
|
| 675 |
+
language="English",
|
| 676 |
+
)
|
| 677 |
+
jobs = st.session_state.setdefault("profile_jobs", {})
|
| 678 |
+
job = jobs.get(ticker)
|
| 679 |
+
if job and job.get("fingerprint") != fingerprint and not job.get("running"):
|
| 680 |
+
jobs.pop(ticker, None)
|
| 681 |
+
job = None
|
| 682 |
+
|
| 683 |
+
if profile is None:
|
| 684 |
+
if config is None:
|
| 685 |
+
st.warning(t("model_blocked_caption"))
|
| 686 |
+
return
|
| 687 |
+
if job is None:
|
| 688 |
+
job = _start_company_profile_job(ticker, fingerprint, config)
|
| 689 |
+
if job.get("running"):
|
| 690 |
+
_render_company_profile_progress(job)
|
| 691 |
+
time.sleep(0.4)
|
| 692 |
+
st.rerun()
|
| 693 |
+
if job.get("error"):
|
| 694 |
+
st.error(job["error"])
|
| 695 |
+
if st.button(t("primer_retry"), key=f"primer_retry_{ticker}"):
|
| 696 |
+
_start_company_profile_job(ticker, fingerprint, config)
|
| 697 |
+
st.rerun()
|
| 698 |
+
return
|
| 699 |
+
profile = job.get("profile")
|
| 700 |
+
if profile is None:
|
| 701 |
+
st.info(t("primer_identity_unavailable"))
|
| 702 |
+
return
|
| 703 |
+
profile["_profile_id"] = job.get("profile_id")
|
| 704 |
+
|
| 705 |
+
coverage = profile_source_coverage(ticker)
|
| 706 |
+
if (
|
| 707 |
+
coverage.get("business_sections_attempted", 0) < 1
|
| 708 |
+
or coverage.get("segments_geography_sections_attempted", 0) < 1
|
| 709 |
+
):
|
| 710 |
+
st.warning(t("primer_backfill_warning").format(ticker=ticker))
|
| 711 |
+
|
| 712 |
+
refresh_col, _ = st.columns([1, 5])
|
| 713 |
+
with refresh_col:
|
| 714 |
+
if st.button(
|
| 715 |
+
t("primer_refresh"),
|
| 716 |
+
key=f"primer_refresh_{ticker}",
|
| 717 |
+
disabled=bool(job and job.get("running")),
|
| 718 |
+
):
|
| 719 |
+
if config is None:
|
| 720 |
+
st.warning(t("model_blocked_caption"))
|
| 721 |
+
else:
|
| 722 |
+
_start_company_profile_job(ticker, fingerprint, config)
|
| 723 |
+
st.rerun()
|
| 724 |
+
if job and job.get("running"):
|
| 725 |
+
_render_company_profile_progress(job)
|
| 726 |
+
time.sleep(0.4)
|
| 727 |
+
st.rerun()
|
| 728 |
+
if job and job.get("error"):
|
| 729 |
+
st.warning(
|
| 730 |
+
f"{t('primer_refresh_failed')} {job['error']}"
|
| 731 |
+
)
|
| 732 |
+
|
| 733 |
+
display_profile = _display_company_profile(profile, config)
|
| 734 |
+
company_name = str(display_profile.get("company_name") or ticker)
|
| 735 |
+
from analytics.company_news import fetch_company_news
|
| 736 |
+
from analytics.company_market import fetch_company_market
|
| 737 |
+
|
| 738 |
+
with st.spinner(t("primer_loading_market")):
|
| 739 |
+
news_items, news_error = fetch_company_news(ticker, company_name)
|
| 740 |
+
market, market_error = fetch_company_market(ticker, company_name)
|
| 741 |
+
|
| 742 |
+
import copy
|
| 743 |
+
from analysis.company_attention import attach_attention_stats
|
| 744 |
+
|
| 745 |
+
display_profile = attach_attention_stats(copy.deepcopy(display_profile), ticker, news_items)
|
| 746 |
+
from dashboard.company_primer import render as render_company_primer
|
| 747 |
+
|
| 748 |
+
render_company_primer(
|
| 749 |
+
display_profile,
|
| 750 |
+
ticker,
|
| 751 |
+
market=market,
|
| 752 |
+
news_items=news_items,
|
| 753 |
+
news_error=news_error,
|
| 754 |
+
)
|
| 755 |
+
if market_error:
|
| 756 |
+
st.caption(market_error)
|
| 757 |
+
|
| 758 |
+
|
| 759 |
gen_error = st.session_state.pop("_gen_error", None)
|
| 760 |
if gen_error:
|
| 761 |
err_ticker, err_msg = gen_error
|
| 762 |
st.error(f"{t('brief_error_prefix')} **{err_ticker}**: {err_msg}")
|
| 763 |
|
| 764 |
gen = st.session_state["gen"]
|
| 765 |
+
active_key = st.session_state.get("nav_key", "company")
|
| 766 |
|
| 767 |
if gen["running"] or gen.get("trace"):
|
| 768 |
_live_trace_fragment()
|
|
|
|
| 786 |
if brief:
|
| 787 |
_render_ticker_bar(ticker_input, brief)
|
| 788 |
|
| 789 |
+
if active_key == "company":
|
| 790 |
+
_render_company_route(ticker_input, run_cfg)
|
| 791 |
+
elif active_key == "verdict":
|
| 792 |
if brief:
|
| 793 |
from dashboard.verdict import render as render_verdict
|
| 794 |
render_verdict(brief)
|
dashboard/company_primer.py
ADDED
|
@@ -0,0 +1,542 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""Company Primer — compact company-onboarding surface."""
|
| 2 |
+
from __future__ import annotations
|
| 3 |
+
|
| 4 |
+
from datetime import datetime
|
| 5 |
+
from html import escape
|
| 6 |
+
|
| 7 |
+
import streamlit as st
|
| 8 |
+
|
| 9 |
+
from dashboard.components import section_header
|
| 10 |
+
from dashboard.i18n import t
|
| 11 |
+
from dashboard.theme import (
|
| 12 |
+
AI_BG,
|
| 13 |
+
AI_BORDER,
|
| 14 |
+
AI_COLOR,
|
| 15 |
+
BLUE,
|
| 16 |
+
BORDER,
|
| 17 |
+
BRAND_GREEN,
|
| 18 |
+
FS_BODY,
|
| 19 |
+
FS_META,
|
| 20 |
+
FS_PAGE,
|
| 21 |
+
TEXT,
|
| 22 |
+
TEXT_MUTED,
|
| 23 |
+
)
|
| 24 |
+
|
| 25 |
+
|
| 26 |
+
def _fact_text(value) -> str:
|
| 27 |
+
if not isinstance(value, dict):
|
| 28 |
+
return ""
|
| 29 |
+
if value.get("verification_status") != "VERIFIED":
|
| 30 |
+
return ""
|
| 31 |
+
return str(value.get("text") or "")
|
| 32 |
+
|
| 33 |
+
|
| 34 |
+
def _source_meta(fact: dict | None) -> str:
|
| 35 |
+
if not isinstance(fact, dict):
|
| 36 |
+
return ""
|
| 37 |
+
source = escape(str(fact.get("source") or ""))
|
| 38 |
+
ref = fact.get("evidence_ref") or {}
|
| 39 |
+
url = escape(str(ref.get("source_url") or ""))
|
| 40 |
+
period = escape(str(ref.get("as_of") or ""))
|
| 41 |
+
label = " · ".join(value for value in (source, period) if value)
|
| 42 |
+
if url:
|
| 43 |
+
return (
|
| 44 |
+
f'<a href="{url}" target="_blank" style="color:{TEXT_MUTED};'
|
| 45 |
+
f'text-decoration:none;border-bottom:1px solid {BORDER};">{label}</a>'
|
| 46 |
+
)
|
| 47 |
+
return f'<span style="color:{TEXT_MUTED};">{label}</span>'
|
| 48 |
+
|
| 49 |
+
|
| 50 |
+
def _page_header(profile: dict, market: dict | None) -> None:
|
| 51 |
+
company = escape(str(profile.get("company_name") or profile.get("ticker") or ""))
|
| 52 |
+
ticker = escape(str(profile.get("ticker") or ""))
|
| 53 |
+
sector = escape(str((market or {}).get("sector") or ""))
|
| 54 |
+
industry = escape(str((market or {}).get("industry") or ""))
|
| 55 |
+
as_of = escape(str(profile.get("data_as_of") or ""))
|
| 56 |
+
meta = " · ".join(value for value in (ticker, sector, industry, as_of) if value)
|
| 57 |
+
st.markdown(
|
| 58 |
+
f'<div style="margin-bottom:18px;">'
|
| 59 |
+
f'<div style="font-size:{FS_PAGE};font-weight:700;color:{TEXT};'
|
| 60 |
+
f'letter-spacing:-0.02em;">{company}</div>'
|
| 61 |
+
f'<div style="font-size:{FS_META};color:{TEXT_MUTED};margin-top:3px;">{meta}</div>'
|
| 62 |
+
f'</div>',
|
| 63 |
+
unsafe_allow_html=True,
|
| 64 |
+
)
|
| 65 |
+
|
| 66 |
+
|
| 67 |
+
def _identity(profile: dict) -> None:
|
| 68 |
+
identity = profile.get("identity") or {}
|
| 69 |
+
one_liner = _fact_text(identity.get("one_liner"))
|
| 70 |
+
business_model = _fact_text(identity.get("business_model"))
|
| 71 |
+
if not one_liner and not business_model:
|
| 72 |
+
st.info(t("primer_identity_unavailable"))
|
| 73 |
+
return
|
| 74 |
+
source = identity.get("one_liner") or identity.get("business_model")
|
| 75 |
+
body = "".join(
|
| 76 |
+
(
|
| 77 |
+
f'<div style="font-size:1.02rem;font-weight:600;line-height:1.55;'
|
| 78 |
+
f'color:{TEXT};margin-bottom:8px;">{escape(one_liner)}</div>'
|
| 79 |
+
if one_liner
|
| 80 |
+
else "",
|
| 81 |
+
f'<div style="font-size:{FS_BODY};line-height:1.6;color:{TEXT_MUTED};">'
|
| 82 |
+
f'{escape(business_model)}</div>'
|
| 83 |
+
if business_model
|
| 84 |
+
else "",
|
| 85 |
+
)
|
| 86 |
+
)
|
| 87 |
+
st.markdown(
|
| 88 |
+
f'<div style="border-left:4px solid {BRAND_GREEN};padding:2px 0 2px 16px;">'
|
| 89 |
+
f'{body}<div style="font-size:{FS_META};margin-top:8px;">{_source_meta(source)}</div>'
|
| 90 |
+
f'</div>',
|
| 91 |
+
unsafe_allow_html=True,
|
| 92 |
+
)
|
| 93 |
+
|
| 94 |
+
|
| 95 |
+
def _business(profile: dict) -> None:
|
| 96 |
+
st.markdown(section_header(t("primer_business"), accent=BRAND_GREEN), unsafe_allow_html=True)
|
| 97 |
+
lines = profile.get("business_lines") or []
|
| 98 |
+
if not lines:
|
| 99 |
+
st.info(t("primer_business_unavailable"))
|
| 100 |
+
return
|
| 101 |
+
|
| 102 |
+
shares = [
|
| 103 |
+
(str(item.get("name") or ""), item.get("revenue_share_pct"))
|
| 104 |
+
for item in lines
|
| 105 |
+
if item.get("revenue_share_pct") is not None
|
| 106 |
+
]
|
| 107 |
+
total = sum(float(value) for _, value in shares)
|
| 108 |
+
if len(shares) >= 2 and total <= 100.5:
|
| 109 |
+
import plotly.graph_objects as go
|
| 110 |
+
|
| 111 |
+
fig = go.Figure()
|
| 112 |
+
palette = ["#10b981", "#3b82f6", "#8b5cf6", "#f59e0b", "#64748b"]
|
| 113 |
+
for index, (name, value) in enumerate(shares):
|
| 114 |
+
fig.add_bar(
|
| 115 |
+
y=["Revenue mix"],
|
| 116 |
+
x=[float(value)],
|
| 117 |
+
name=name,
|
| 118 |
+
orientation="h",
|
| 119 |
+
text=f"{name} {float(value):.0f}%",
|
| 120 |
+
textposition="inside",
|
| 121 |
+
marker_color=palette[index % len(palette)],
|
| 122 |
+
hovertemplate=f"{escape(name)}: {float(value):.1f}%<extra></extra>",
|
| 123 |
+
)
|
| 124 |
+
if total < 99.5:
|
| 125 |
+
fig.add_bar(
|
| 126 |
+
y=["Revenue mix"],
|
| 127 |
+
x=[100 - total],
|
| 128 |
+
name=t("primer_other"),
|
| 129 |
+
orientation="h",
|
| 130 |
+
marker_color="#d1d5db",
|
| 131 |
+
hovertemplate=f"{t('primer_other')}: {100-total:.1f}%<extra></extra>",
|
| 132 |
+
)
|
| 133 |
+
fig.update_layout(
|
| 134 |
+
barmode="stack",
|
| 135 |
+
height=105,
|
| 136 |
+
margin=dict(l=0, r=0, t=5, b=5),
|
| 137 |
+
paper_bgcolor="rgba(0,0,0,0)",
|
| 138 |
+
plot_bgcolor="rgba(0,0,0,0)",
|
| 139 |
+
showlegend=False,
|
| 140 |
+
xaxis=dict(range=[0, 100], visible=False),
|
| 141 |
+
yaxis=dict(visible=False),
|
| 142 |
+
font=dict(family="Inter"),
|
| 143 |
+
)
|
| 144 |
+
st.plotly_chart(fig, use_container_width=True, config={"displayModeBar": False})
|
| 145 |
+
|
| 146 |
+
rows = []
|
| 147 |
+
for item in lines[:5]:
|
| 148 |
+
fact = item.get("description") or {}
|
| 149 |
+
framing = str(item.get("economics") or "")
|
| 150 |
+
if framing:
|
| 151 |
+
framing = f"AI · {framing}"
|
| 152 |
+
rows.append(
|
| 153 |
+
{
|
| 154 |
+
t("primer_engine"): item.get("name") or "",
|
| 155 |
+
t("primer_role"): _fact_text(fact),
|
| 156 |
+
t("primer_share"): (
|
| 157 |
+
f"{float(item['revenue_share_pct']):.1f}% · {item.get('share_period') or ''}"
|
| 158 |
+
if item.get("revenue_share_pct") is not None
|
| 159 |
+
else t("primer_not_disclosed")
|
| 160 |
+
),
|
| 161 |
+
t("primer_trend"): t(
|
| 162 |
+
f"primer_trend_{item.get('trend') or 'not_disclosed'}"
|
| 163 |
+
),
|
| 164 |
+
t("primer_framing"): framing,
|
| 165 |
+
}
|
| 166 |
+
)
|
| 167 |
+
st.dataframe(rows, use_container_width=True, hide_index=True)
|
| 168 |
+
|
| 169 |
+
|
| 170 |
+
def _geography(profile: dict) -> None:
|
| 171 |
+
st.markdown(section_header(t("primer_geography"), accent=BLUE), unsafe_allow_html=True)
|
| 172 |
+
exposures = profile.get("geographic_exposures") or []
|
| 173 |
+
if not exposures:
|
| 174 |
+
st.info(t("primer_geography_unavailable"))
|
| 175 |
+
return
|
| 176 |
+
|
| 177 |
+
quantitative = [
|
| 178 |
+
item for item in exposures if item.get("revenue_share_pct") is not None
|
| 179 |
+
]
|
| 180 |
+
total = sum(float(item["revenue_share_pct"]) for item in quantitative)
|
| 181 |
+
if len(quantitative) >= 2 and total <= 100.5:
|
| 182 |
+
import plotly.graph_objects as go
|
| 183 |
+
|
| 184 |
+
fig = go.Figure()
|
| 185 |
+
palette = ["#3b82f6", "#10b981", "#8b5cf6", "#f59e0b", "#64748b"]
|
| 186 |
+
for index, item in enumerate(quantitative):
|
| 187 |
+
value = float(item["revenue_share_pct"])
|
| 188 |
+
name = str(item.get("name") or "")
|
| 189 |
+
fig.add_bar(
|
| 190 |
+
y=[t("primer_revenue_geography")],
|
| 191 |
+
x=[value],
|
| 192 |
+
name=name,
|
| 193 |
+
orientation="h",
|
| 194 |
+
text=f"{name} {value:.0f}%",
|
| 195 |
+
textposition="inside",
|
| 196 |
+
marker_color=palette[index % len(palette)],
|
| 197 |
+
hovertemplate=f"{escape(name)}: {value:.1f}%<extra></extra>",
|
| 198 |
+
)
|
| 199 |
+
fig.update_layout(
|
| 200 |
+
barmode="stack",
|
| 201 |
+
height=105,
|
| 202 |
+
margin=dict(l=0, r=0, t=5, b=5),
|
| 203 |
+
paper_bgcolor="rgba(0,0,0,0)",
|
| 204 |
+
plot_bgcolor="rgba(0,0,0,0)",
|
| 205 |
+
showlegend=False,
|
| 206 |
+
xaxis=dict(range=[0, max(100, total)], visible=False),
|
| 207 |
+
yaxis=dict(visible=False),
|
| 208 |
+
font=dict(family="Inter"),
|
| 209 |
+
)
|
| 210 |
+
st.plotly_chart(fig, use_container_width=True, config={"displayModeBar": False})
|
| 211 |
+
|
| 212 |
+
rows = [
|
| 213 |
+
{
|
| 214 |
+
t("primer_country_region"): item.get("name") or "",
|
| 215 |
+
t("primer_exposure"): ", ".join(
|
| 216 |
+
t(f"primer_exposure_{kind}")
|
| 217 |
+
for kind in (item.get("exposure_types") or [])
|
| 218 |
+
),
|
| 219 |
+
t("primer_description"): _fact_text(item.get("description")),
|
| 220 |
+
t("primer_share"): (
|
| 221 |
+
f"{float(item['revenue_share_pct']):.1f}% · {item.get('period') or ''}"
|
| 222 |
+
if item.get("revenue_share_pct") is not None
|
| 223 |
+
else t("primer_qualitative")
|
| 224 |
+
),
|
| 225 |
+
}
|
| 226 |
+
for item in exposures[:8]
|
| 227 |
+
]
|
| 228 |
+
st.dataframe(rows, use_container_width=True, hide_index=True)
|
| 229 |
+
|
| 230 |
+
|
| 231 |
+
def _evolution(profile: dict) -> None:
|
| 232 |
+
st.markdown(section_header(t("primer_evolution"), accent=BRAND_GREEN), unsafe_allow_html=True)
|
| 233 |
+
trends = profile.get("annual_trends") or []
|
| 234 |
+
changes = profile.get("strategic_changes") or []
|
| 235 |
+
if not trends and not changes:
|
| 236 |
+
st.info(t("primer_evolution_unavailable"))
|
| 237 |
+
return
|
| 238 |
+
if trends:
|
| 239 |
+
from plotly.subplots import make_subplots
|
| 240 |
+
import plotly.graph_objects as go
|
| 241 |
+
|
| 242 |
+
periods = [str(item.get("period") or "") for item in trends]
|
| 243 |
+
specs = [
|
| 244 |
+
("revenue_bn", t("primer_revenue"), "$B", 1),
|
| 245 |
+
("operating_margin", t("primer_op_margin"), "%", 2),
|
| 246 |
+
("free_cash_flow_bn", t("primer_fcf"), "$B", 3),
|
| 247 |
+
("eps", "EPS", "$", 4),
|
| 248 |
+
]
|
| 249 |
+
fig = make_subplots(rows=2, cols=2, subplot_titles=[item[1] for item in specs])
|
| 250 |
+
for key, _label, unit, col in specs:
|
| 251 |
+
values = [item.get(key) for item in trends]
|
| 252 |
+
display_values = [
|
| 253 |
+
value * 100 if key == "operating_margin" and value is not None else value
|
| 254 |
+
for value in values
|
| 255 |
+
]
|
| 256 |
+
fig.add_trace(
|
| 257 |
+
go.Scatter(
|
| 258 |
+
x=periods,
|
| 259 |
+
y=display_values,
|
| 260 |
+
mode="lines+markers+text",
|
| 261 |
+
text=[
|
| 262 |
+
f"{value:.1f}{unit}" if value is not None else ""
|
| 263 |
+
for value in display_values
|
| 264 |
+
],
|
| 265 |
+
textposition="top center",
|
| 266 |
+
line=dict(color=BRAND_GREEN, width=2),
|
| 267 |
+
marker=dict(size=6),
|
| 268 |
+
hovertemplate="%{x}: %{y:.2f}<extra></extra>",
|
| 269 |
+
showlegend=False,
|
| 270 |
+
),
|
| 271 |
+
row=1 if col <= 2 else 2,
|
| 272 |
+
col=col if col <= 2 else col - 2,
|
| 273 |
+
)
|
| 274 |
+
fig.update_layout(
|
| 275 |
+
height=430,
|
| 276 |
+
margin=dict(l=20, r=20, t=45, b=20),
|
| 277 |
+
paper_bgcolor="rgba(0,0,0,0)",
|
| 278 |
+
plot_bgcolor="rgba(0,0,0,0)",
|
| 279 |
+
font=dict(family="Inter", size=11),
|
| 280 |
+
)
|
| 281 |
+
fig.update_xaxes(showgrid=False)
|
| 282 |
+
fig.update_yaxes(showgrid=True, gridcolor="#e5e7eb", zeroline=False)
|
| 283 |
+
st.plotly_chart(fig, use_container_width=True, config={"displayModeBar": False})
|
| 284 |
+
|
| 285 |
+
if changes:
|
| 286 |
+
rows = [
|
| 287 |
+
{
|
| 288 |
+
t("primer_period"): " → ".join(
|
| 289 |
+
value for value in (item.get("period_from"), item.get("period_to")) if value
|
| 290 |
+
),
|
| 291 |
+
t("primer_change"): _fact_text(item.get("change")),
|
| 292 |
+
t("primer_implication"): f"AI · {item.get('implication') or ''}",
|
| 293 |
+
}
|
| 294 |
+
for item in changes[:3]
|
| 295 |
+
]
|
| 296 |
+
st.dataframe(rows, use_container_width=True, hide_index=True)
|
| 297 |
+
|
| 298 |
+
|
| 299 |
+
def _attention(profile: dict) -> None:
|
| 300 |
+
st.markdown(section_header(t("primer_attention"), accent=AI_COLOR), unsafe_allow_html=True)
|
| 301 |
+
themes = profile.get("attention_themes") or []
|
| 302 |
+
if not themes:
|
| 303 |
+
st.info(t("primer_attention_unavailable"))
|
| 304 |
+
return
|
| 305 |
+
for item in themes[:3]:
|
| 306 |
+
mentions = item.get("call_mentions") or []
|
| 307 |
+
trajectory = " → ".join(
|
| 308 |
+
f"{point.get('period')}: {point.get('mentions', 0)}" for point in mentions
|
| 309 |
+
) or t("primer_insufficient")
|
| 310 |
+
evidence = item.get("evidence") or {}
|
| 311 |
+
direction_label = t(
|
| 312 |
+
"primer_direction_" + str(item.get("direction") or "insufficient")
|
| 313 |
+
)
|
| 314 |
+
st.markdown(
|
| 315 |
+
f'<div style="border-bottom:1px solid {BORDER};padding:12px 0;">'
|
| 316 |
+
f'<div style="display:flex;justify-content:space-between;gap:10px;flex-wrap:wrap;">'
|
| 317 |
+
f'<div style="font-size:{FS_BODY};font-weight:600;color:{TEXT};">'
|
| 318 |
+
f'{escape(str(item.get("theme") or ""))}</div>'
|
| 319 |
+
f'<span style="background:{AI_BG};border:1px solid {AI_BORDER};color:{AI_COLOR};'
|
| 320 |
+
f'border-radius:4px;padding:1px 7px;font-size:0.65rem;font-weight:600;">'
|
| 321 |
+
f'AI · experimental</span></div>'
|
| 322 |
+
f'<div style="font-size:{FS_BODY};color:{TEXT_MUTED};margin-top:4px;">'
|
| 323 |
+
f'{escape(str(item.get("why_it_matters") or ""))}</div>'
|
| 324 |
+
f'<div style="font-size:{FS_META};color:{TEXT_MUTED};margin-top:7px;">'
|
| 325 |
+
f'Q&A: {escape(trajectory)} · {t("primer_news_count").format(count=int(item.get("retrieved_news_count") or 0))}'
|
| 326 |
+
f' · {escape(direction_label)}</div>'
|
| 327 |
+
f'<div style="font-size:{FS_META};margin-top:4px;">{_source_meta(evidence)}</div>'
|
| 328 |
+
f'</div>',
|
| 329 |
+
unsafe_allow_html=True,
|
| 330 |
+
)
|
| 331 |
+
|
| 332 |
+
|
| 333 |
+
def _market(profile: dict, market: dict | None) -> None:
|
| 334 |
+
st.markdown(section_header(t("primer_price"), accent=BLUE), unsafe_allow_html=True)
|
| 335 |
+
if not market or not market.get("history"):
|
| 336 |
+
st.info(t("primer_price_unavailable"))
|
| 337 |
+
return
|
| 338 |
+
import plotly.graph_objects as go
|
| 339 |
+
|
| 340 |
+
history = market["history"]
|
| 341 |
+
ticker = str(market.get("ticker") or profile.get("ticker") or "")
|
| 342 |
+
benchmark = str(market.get("benchmark") or "SPY")
|
| 343 |
+
series = [ticker, "SPY"] + ([] if benchmark == "SPY" else [benchmark])
|
| 344 |
+
colors = {ticker: BRAND_GREEN, "SPY": "#64748b", benchmark: "#3b82f6"}
|
| 345 |
+
fig = go.Figure()
|
| 346 |
+
for symbol in series:
|
| 347 |
+
fig.add_trace(
|
| 348 |
+
go.Scatter(
|
| 349 |
+
x=[item["date"] for item in history],
|
| 350 |
+
y=[item.get(symbol) for item in history],
|
| 351 |
+
mode="lines",
|
| 352 |
+
name=symbol,
|
| 353 |
+
line=dict(color=colors.get(symbol, "#8b5cf6"), width=2 if symbol == ticker else 1.5),
|
| 354 |
+
hovertemplate=f"{symbol} %{{y:.1f}}<extra></extra>",
|
| 355 |
+
)
|
| 356 |
+
)
|
| 357 |
+
event_dates = []
|
| 358 |
+
event_values = []
|
| 359 |
+
event_symbols = []
|
| 360 |
+
event_colors = []
|
| 361 |
+
event_hover = []
|
| 362 |
+
ticker_points = [
|
| 363 |
+
item for item in history if item.get("date") and item.get(ticker) is not None
|
| 364 |
+
]
|
| 365 |
+
for event in market.get("events") or []:
|
| 366 |
+
try:
|
| 367 |
+
event_day = datetime.fromisoformat(str(event.get("date") or "")[:10])
|
| 368 |
+
nearest = min(
|
| 369 |
+
ticker_points,
|
| 370 |
+
key=lambda point: abs(
|
| 371 |
+
(
|
| 372 |
+
datetime.fromisoformat(str(point["date"])[:10])
|
| 373 |
+
- event_day
|
| 374 |
+
).days
|
| 375 |
+
),
|
| 376 |
+
)
|
| 377 |
+
except (TypeError, ValueError):
|
| 378 |
+
continue
|
| 379 |
+
excess = float(event.get("sector_excess_pct") or 0)
|
| 380 |
+
event_dates.append(nearest["date"])
|
| 381 |
+
event_values.append(nearest[ticker])
|
| 382 |
+
event_symbols.append("triangle-up" if excess >= 0 else "triangle-down")
|
| 383 |
+
event_colors.append(BRAND_GREEN if excess >= 0 else "#dc2626")
|
| 384 |
+
headline_text = (
|
| 385 |
+
t("primer_no_verified_catalyst")
|
| 386 |
+
if event.get("attribution_status") == "NO_VERIFIED_CATALYST"
|
| 387 |
+
else str(event.get("headline") or t("primer_no_verified_catalyst"))
|
| 388 |
+
)
|
| 389 |
+
headline = escape(headline_text)
|
| 390 |
+
event_hover.append(
|
| 391 |
+
f"{escape(str(event.get('date') or ''))}<br>"
|
| 392 |
+
f"{escape(t('primer_relative_move'))}: {excess:+.1f}%<br>"
|
| 393 |
+
f"{headline}"
|
| 394 |
+
)
|
| 395 |
+
if event_dates:
|
| 396 |
+
fig.add_trace(
|
| 397 |
+
go.Scatter(
|
| 398 |
+
x=event_dates,
|
| 399 |
+
y=event_values,
|
| 400 |
+
mode="markers",
|
| 401 |
+
name=t("primer_price_events"),
|
| 402 |
+
marker=dict(
|
| 403 |
+
size=10,
|
| 404 |
+
symbol=event_symbols,
|
| 405 |
+
color=event_colors,
|
| 406 |
+
line=dict(color=BORDER, width=1),
|
| 407 |
+
),
|
| 408 |
+
text=event_hover,
|
| 409 |
+
hovertemplate="%{text}<extra></extra>",
|
| 410 |
+
)
|
| 411 |
+
)
|
| 412 |
+
fig.update_layout(
|
| 413 |
+
height=330,
|
| 414 |
+
margin=dict(l=20, r=20, t=10, b=20),
|
| 415 |
+
paper_bgcolor="rgba(0,0,0,0)",
|
| 416 |
+
plot_bgcolor="rgba(0,0,0,0)",
|
| 417 |
+
font=dict(family="Inter"),
|
| 418 |
+
legend=dict(orientation="h", y=1.05, x=0),
|
| 419 |
+
yaxis_title=t("primer_indexed_100"),
|
| 420 |
+
hovermode="x unified",
|
| 421 |
+
)
|
| 422 |
+
fig.update_xaxes(showgrid=False)
|
| 423 |
+
fig.update_yaxes(gridcolor="#e5e7eb", zeroline=False)
|
| 424 |
+
st.plotly_chart(fig, use_container_width=True, config={"displayModeBar": False})
|
| 425 |
+
st.caption(t("primer_price_caption").format(benchmark=benchmark))
|
| 426 |
+
|
| 427 |
+
events = market.get("events") or []
|
| 428 |
+
if events:
|
| 429 |
+
for event in events:
|
| 430 |
+
direction = "+" if float(event.get("sector_excess_pct") or 0) >= 0 else ""
|
| 431 |
+
headline_text = (
|
| 432 |
+
t("primer_no_verified_catalyst")
|
| 433 |
+
if event.get("attribution_status") == "NO_VERIFIED_CATALYST"
|
| 434 |
+
else str(event.get("headline") or "")
|
| 435 |
+
)
|
| 436 |
+
headline = escape(headline_text)
|
| 437 |
+
url = escape(str(event.get("url") or ""))
|
| 438 |
+
headline_html = (
|
| 439 |
+
f'<a href="{url}" target="_blank" style="color:{TEXT};text-decoration:none;'
|
| 440 |
+
f'border-bottom:1px solid {BORDER};">{headline}</a>'
|
| 441 |
+
if url else f'<span style="color:{TEXT_MUTED};">{headline}</span>'
|
| 442 |
+
)
|
| 443 |
+
st.markdown(
|
| 444 |
+
f'<div style="display:grid;grid-template-columns:86px 90px 1fr;gap:12px;'
|
| 445 |
+
f'padding:8px 0;border-bottom:1px solid {BORDER};font-size:{FS_META};">'
|
| 446 |
+
f'<span>{escape(str(event.get("date") or ""))}</span>'
|
| 447 |
+
f'<strong>{direction}{float(event.get("sector_excess_pct") or 0):.1f}%</strong>'
|
| 448 |
+
f'<span>{headline_html}</span></div>',
|
| 449 |
+
unsafe_allow_html=True,
|
| 450 |
+
)
|
| 451 |
+
|
| 452 |
+
|
| 453 |
+
def _news(news_items: list[dict], news_error: str | None) -> None:
|
| 454 |
+
st.markdown(section_header(t("primer_news"), accent=BLUE), unsafe_allow_html=True)
|
| 455 |
+
if not news_items:
|
| 456 |
+
st.info(news_error or t("primer_news_unavailable"))
|
| 457 |
+
return
|
| 458 |
+
for item in news_items[:5]:
|
| 459 |
+
title = escape(str(item.get("title") or ""))
|
| 460 |
+
url = escape(str(item.get("url") or ""))
|
| 461 |
+
date = escape(str(item.get("published_date") or "")[:10])
|
| 462 |
+
st.markdown(
|
| 463 |
+
f'<div style="padding:9px 0;border-bottom:1px solid {BORDER};">'
|
| 464 |
+
f'<div style="font-size:{FS_META};color:{TEXT_MUTED};">{date}</div>'
|
| 465 |
+
f'<a href="{url}" target="_blank" style="font-size:{FS_BODY};font-weight:600;'
|
| 466 |
+
f'color:{TEXT};text-decoration:none;">{title}</a></div>',
|
| 467 |
+
unsafe_allow_html=True,
|
| 468 |
+
)
|
| 469 |
+
|
| 470 |
+
|
| 471 |
+
def _watch(profile: dict) -> None:
|
| 472 |
+
st.markdown(section_header(t("primer_watch"), accent="#f59e0b"), unsafe_allow_html=True)
|
| 473 |
+
values = profile.get("watch_variables") or []
|
| 474 |
+
if not values:
|
| 475 |
+
st.info(t("primer_watch_unavailable"))
|
| 476 |
+
return
|
| 477 |
+
rows = [
|
| 478 |
+
{
|
| 479 |
+
t("primer_variable"): item.get("variable") or "",
|
| 480 |
+
t("primer_why"): f"AI · {item.get('why_it_matters') or ''}",
|
| 481 |
+
t("primer_next_datapoint"): item.get("next_datapoint") or "",
|
| 482 |
+
t("primer_alert"): item.get("alert_signal") or "",
|
| 483 |
+
}
|
| 484 |
+
for item in values[:4]
|
| 485 |
+
]
|
| 486 |
+
st.dataframe(rows, use_container_width=True, hide_index=True)
|
| 487 |
+
|
| 488 |
+
|
| 489 |
+
def render(
|
| 490 |
+
profile: dict,
|
| 491 |
+
ticker: str,
|
| 492 |
+
*,
|
| 493 |
+
market: dict | None = None,
|
| 494 |
+
news_items: list[dict] | None = None,
|
| 495 |
+
news_error: str | None = None,
|
| 496 |
+
) -> None:
|
| 497 |
+
_page_header(profile, market)
|
| 498 |
+
_identity(profile)
|
| 499 |
+
_business(profile)
|
| 500 |
+
_geography(profile)
|
| 501 |
+
_evolution(profile)
|
| 502 |
+
_attention(profile)
|
| 503 |
+
_market(profile, market)
|
| 504 |
+
_news(news_items or [], news_error)
|
| 505 |
+
_watch(profile)
|
| 506 |
+
|
| 507 |
+
coverage = profile.get("source_coverage") or {}
|
| 508 |
+
with st.expander(t("primer_sources"), expanded=False):
|
| 509 |
+
st.json(
|
| 510 |
+
{
|
| 511 |
+
"profile_status": profile.get("status"),
|
| 512 |
+
"data_as_of": profile.get("data_as_of"),
|
| 513 |
+
"source_coverage": coverage,
|
| 514 |
+
"evidence_coverage": profile.get("evidence_coverage") or {},
|
| 515 |
+
"price_source": (market or {}).get("price_source"),
|
| 516 |
+
"price_as_of": (market or {}).get("as_of"),
|
| 517 |
+
}
|
| 518 |
+
)
|
| 519 |
+
evidence_rows = []
|
| 520 |
+
|
| 521 |
+
def _walk(value):
|
| 522 |
+
if isinstance(value, dict):
|
| 523 |
+
if value.get("verification_status") == "VERIFIED" and value.get("evidence_snippet"):
|
| 524 |
+
ref = value.get("evidence_ref") or {}
|
| 525 |
+
evidence_rows.append(
|
| 526 |
+
{
|
| 527 |
+
"Claim": value.get("text") or "",
|
| 528 |
+
"Source": value.get("source") or "",
|
| 529 |
+
"As of": ref.get("as_of") or "",
|
| 530 |
+
"Evidence": value.get("evidence_snippet") or "",
|
| 531 |
+
"URL": ref.get("source_url") or "",
|
| 532 |
+
}
|
| 533 |
+
)
|
| 534 |
+
for nested in value.values():
|
| 535 |
+
_walk(nested)
|
| 536 |
+
elif isinstance(value, list):
|
| 537 |
+
for nested in value:
|
| 538 |
+
_walk(nested)
|
| 539 |
+
|
| 540 |
+
_walk(profile)
|
| 541 |
+
if evidence_rows:
|
| 542 |
+
st.dataframe(evidence_rows, use_container_width=True, hide_index=True)
|
dashboard/i18n.py
CHANGED
|
@@ -279,6 +279,7 @@ STRINGS: dict[str, dict[str, str]] = {
|
|
| 279 |
},
|
| 280 |
|
| 281 |
# ── Flat navigation (Decision Stack) ───────────────────────────────────
|
|
|
|
| 282 |
"nav_verdict": {"en": "Verdict", "fr": "Verdict", "es": "Veredicto", "de": "Fazit"},
|
| 283 |
"nav_chat": {"en": "Ask AI", "fr": "Demander à l'IA", "es": "Preguntar a la IA", "de": "KI fragen"},
|
| 284 |
"nav_signals": {"en": "Signals", "fr": "Signaux", "es": "Señales", "de": "Signale"},
|
|
@@ -543,6 +544,73 @@ STRINGS: dict[str, dict[str, str]] = {
|
|
| 543 |
"band_strong_bear": {"en": "Strongly Bearish", "fr": "Très baissier", "es": "Muy bajista", "de": "Stark bärisch"},
|
| 544 |
|
| 545 |
# ── Financials screen ──────────────────────────────────────────────────
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 546 |
"fin_kpi": {"en": "Latest quarter", "fr": "Dernier trimestre", "es": "Último trimestre", "de": "Letztes Quartal"},
|
| 547 |
"fin_next_guidance": {"en": "Next period guidance", "fr": "Guidance du prochain trimestre", "es": "Guidance del próximo período", "de": "Guidance nächste Periode"},
|
| 548 |
"fin_tab_revenue": {"en": "Revenue & Growth", "fr": "CA & croissance", "es": "Ingresos y crecimiento", "de": "Umsatz & Wachstum"},
|
|
|
|
| 279 |
},
|
| 280 |
|
| 281 |
# ── Flat navigation (Decision Stack) ───────────────────────────────────
|
| 282 |
+
"nav_company": {"en": "Company Primer", "fr": "Profil société", "es": "Perfil de empresa", "de": "Unternehmensprofil"},
|
| 283 |
"nav_verdict": {"en": "Verdict", "fr": "Verdict", "es": "Veredicto", "de": "Fazit"},
|
| 284 |
"nav_chat": {"en": "Ask AI", "fr": "Demander à l'IA", "es": "Preguntar a la IA", "de": "KI fragen"},
|
| 285 |
"nav_signals": {"en": "Signals", "fr": "Signaux", "es": "Señales", "de": "Signale"},
|
|
|
|
| 544 |
"band_strong_bear": {"en": "Strongly Bearish", "fr": "Très baissier", "es": "Muy bajista", "de": "Stark bärisch"},
|
| 545 |
|
| 546 |
# ── Financials screen ──────────────────────────────────────────────────
|
| 547 |
+
# Company Primer
|
| 548 |
+
"primer_generating": {"en": "Building the Company Primer", "fr": "Création du profil société", "es": "Creando el perfil de empresa", "de": "Unternehmensprofil wird erstellt"},
|
| 549 |
+
"primer_retry": {"en": "Retry Company Primer", "fr": "Relancer le profil société", "es": "Reintentar el perfil de empresa", "de": "Unternehmensprofil erneut erstellen"},
|
| 550 |
+
"primer_refresh": {"en": "Refresh profile", "fr": "Actualiser le profil", "es": "Actualizar perfil", "de": "Profil aktualisieren"},
|
| 551 |
+
"primer_refresh_failed": {"en": "Profile refresh failed; the cached version remains available.", "fr": "L’actualisation du profil a échoué ; la version en cache reste disponible.", "es": "La actualización del perfil falló; la versión en caché sigue disponible.", "de": "Die Profilaktualisierung ist fehlgeschlagen; die zwischengespeicherte Version bleibt verfügbar."},
|
| 552 |
+
"primer_loading_market": {"en": "Loading current market context…", "fr": "Chargement du contexte de marché actuel…", "es": "Cargando el contexto actual de mercado…", "de": "Aktueller Marktkontext wird geladen…"},
|
| 553 |
+
"primer_backfill_warning": {"en": "Company-profile source coverage is incomplete. Run `python ingest.py {ticker}` to backfill the 10-K Business and segment/geography sections.", "fr": "La couverture des sources du profil est incomplète. Lancez `python ingest.py {ticker}` pour récupérer les sections Business et segments/géographies du 10-K.", "es": "La cobertura de fuentes del perfil está incompleta. Ejecute `python ingest.py {ticker}` para recuperar las secciones Business y segmentos/geografía del 10-K.", "de": "Die Quellenabdeckung des Unternehmensprofils ist unvollständig. Führen Sie `python ingest.py {ticker}` aus, um die 10-K-Abschnitte Business und Segmente/Geografie nachzuladen."},
|
| 554 |
+
"primer_identity_unavailable": {"en": "The business description is not yet available from verified sources.", "fr": "La description de l’activité n’est pas encore disponible dans les sources vérifiées.", "es": "La descripción del negocio aún no está disponible en fuentes verificadas.", "de": "Die Geschäftsbeschreibung ist aus verifizierten Quellen noch nicht verfügbar."},
|
| 555 |
+
"primer_business": {"en": "How it makes money", "fr": "Comment l’entreprise gagne de l’argent", "es": "Cómo gana dinero", "de": "Wie das Unternehmen Geld verdient"},
|
| 556 |
+
"primer_business_unavailable": {"en": "No verified business-line disclosure is available.", "fr": "Aucune ventilation vérifiée des activités n’est disponible.", "es": "No hay desglose verificado de líneas de negocio.", "de": "Keine verifizierte Aufschlüsselung der Geschäftsbereiche verfügbar."},
|
| 557 |
+
"primer_geography": {"en": "Where it is exposed", "fr": "Où l’entreprise est exposée", "es": "Dónde está expuesta", "de": "Geografische Exponierung"},
|
| 558 |
+
"primer_geography_unavailable": {"en": "No verified geographic disclosure is available.", "fr": "Aucune information géographique vérifiée n’est disponible.", "es": "No hay información geográfica verificada.", "de": "Keine verifizierten geografischen Angaben verfügbar."},
|
| 559 |
+
"primer_evolution": {"en": "Three-year evolution", "fr": "Évolution sur trois ans", "es": "Evolución a tres años", "de": "Dreijahresentwicklung"},
|
| 560 |
+
"primer_evolution_unavailable": {"en": "Verified multi-year history is unavailable.", "fr": "L’historique pluriannuel vérifié est indisponible.", "es": "El historial plurianual verificado no está disponible.", "de": "Eine verifizierte Mehrjahreshistorie ist nicht verfügbar."},
|
| 561 |
+
"primer_attention": {"en": "What commands attention", "fr": "Ce qui concentre l’attention", "es": "Lo que concentra la atención", "de": "Was Aufmerksamkeit bindet"},
|
| 562 |
+
"primer_attention_unavailable": {"en": "Not enough verified call evidence to rank attention themes.", "fr": "Les calls vérifiés sont insuffisants pour classer les thèmes d’attention.", "es": "No hay suficiente evidencia verificada de calls para clasificar temas.", "de": "Nicht genügend verifizierte Call-Evidenz für eine Themenrangfolge."},
|
| 563 |
+
"primer_price": {"en": "Price and associated events", "fr": "Cours et événements associés", "es": "Precio y eventos asociados", "de": "Kurs und zugeordnete Ereignisse"},
|
| 564 |
+
"primer_price_unavailable": {"en": "Relative price history is unavailable.", "fr": "L’historique de cours relatif est indisponible.", "es": "El historial de precio relativo no está disponible.", "de": "Relative Kurshistorie ist nicht verfügbar."},
|
| 565 |
+
"primer_price_caption": {"en": "Three-year adjusted prices indexed to 100. Events are date associations, not proven causes. Sector benchmark: {benchmark}.", "fr": "Cours ajustés sur trois ans, base 100. Les événements sont des associations de dates, pas des causalités prouvées. Benchmark sectoriel : {benchmark}.", "es": "Precios ajustados a tres años, base 100. Los eventos son asociaciones de fechas, no causas probadas. Benchmark sectorial: {benchmark}.", "de": "Dreijährige adjustierte Kurse, indexiert auf 100. Ereignisse sind zeitliche Zuordnungen, keine bewiesenen Ursachen. Sektor-Benchmark: {benchmark}."},
|
| 566 |
+
"primer_price_events": {"en": "Relative-move events", "fr": "Événements de performance relative", "es": "Eventos de movimiento relativo", "de": "Ereignisse relativer Kursbewegungen"},
|
| 567 |
+
"primer_relative_move": {"en": "Relative move", "fr": "Mouvement relatif", "es": "Movimiento relativo", "de": "Relative Bewegung"},
|
| 568 |
+
"primer_indexed_100": {"en": "Indexed to 100", "fr": "Base 100", "es": "Base 100", "de": "Indexiert auf 100"},
|
| 569 |
+
"primer_no_verified_catalyst": {"en": "No verified catalyst found", "fr": "Aucun catalyseur vérifié trouvé", "es": "No se encontró un catalizador verificado", "de": "Kein verifizierter Auslöser gefunden"},
|
| 570 |
+
"primer_news": {"en": "Recent news", "fr": "Actualités récentes", "es": "Noticias recientes", "de": "Aktuelle Nachrichten"},
|
| 571 |
+
"primer_news_unavailable": {"en": "Recent news is unavailable.", "fr": "Les actualités récentes sont indisponibles.", "es": "Las noticias recientes no están disponibles.", "de": "Aktuelle Nachrichten sind nicht verfügbar."},
|
| 572 |
+
"primer_watch": {"en": "What to watch", "fr": "À surveiller", "es": "Qué vigilar", "de": "Worauf zu achten ist"},
|
| 573 |
+
"primer_watch_unavailable": {"en": "No verified monitoring variables are available.", "fr": "Aucune variable de suivi vérifiée n’est disponible.", "es": "No hay variables de seguimiento verificadas.", "de": "Keine verifizierten Beobachtungsvariablen verfügbar."},
|
| 574 |
+
"primer_sources": {"en": "Coverage and sources", "fr": "Couverture et sources", "es": "Cobertura y fuentes", "de": "Abdeckung und Quellen"},
|
| 575 |
+
"primer_engine": {"en": "Engine", "fr": "Moteur", "es": "Motor", "de": "Treiber"},
|
| 576 |
+
"primer_role": {"en": "Economic role", "fr": "Rôle économique", "es": "Función económica", "de": "Wirtschaftliche Rolle"},
|
| 577 |
+
"primer_share": {"en": "Revenue share", "fr": "Part du CA", "es": "Parte de ingresos", "de": "Umsatzanteil"},
|
| 578 |
+
"primer_trend": {"en": "Trend", "fr": "Tendance", "es": "Tendencia", "de": "Trend"},
|
| 579 |
+
"primer_trend_growing": {"en": "Growing", "fr": "En croissance", "es": "En crecimiento", "de": "Wachsend"},
|
| 580 |
+
"primer_trend_stable": {"en": "Stable", "fr": "Stable", "es": "Estable", "de": "Stabil"},
|
| 581 |
+
"primer_trend_declining": {"en": "Declining", "fr": "En baisse", "es": "En descenso", "de": "Rückläufig"},
|
| 582 |
+
"primer_trend_mixed": {"en": "Mixed", "fr": "Contrastée", "es": "Mixta", "de": "Gemischt"},
|
| 583 |
+
"primer_trend_not_disclosed": {"en": "Not disclosed", "fr": "Non communiqué", "es": "No publicado", "de": "Nicht offengelegt"},
|
| 584 |
+
"primer_framing": {"en": "Analyst framing", "fr": "Lecture analyste", "es": "Lectura del analista", "de": "Analysteneinordnung"},
|
| 585 |
+
"primer_other": {"en": "Other / undisclosed", "fr": "Autres / non communiqué", "es": "Otros / no publicado", "de": "Sonstige / nicht offengelegt"},
|
| 586 |
+
"primer_not_disclosed": {"en": "Not disclosed", "fr": "Non communiqué", "es": "No publicado", "de": "Nicht offengelegt"},
|
| 587 |
+
"primer_revenue_geography": {"en": "Revenue geography", "fr": "Répartition géographique du CA", "es": "Geografía de ingresos", "de": "Umsatzgeografie"},
|
| 588 |
+
"primer_country_region": {"en": "Country / region", "fr": "Pays / région", "es": "País / región", "de": "Land / Region"},
|
| 589 |
+
"primer_exposure": {"en": "Exposure", "fr": "Exposition", "es": "Exposición", "de": "Exponierung"},
|
| 590 |
+
"primer_exposure_revenue": {"en": "Revenue", "fr": "Chiffre d’affaires", "es": "Ingresos", "de": "Umsatz"},
|
| 591 |
+
"primer_exposure_operations": {"en": "Operations", "fr": "Opérations", "es": "Operaciones", "de": "Betrieb"},
|
| 592 |
+
"primer_exposure_supply_chain": {"en": "Supply chain", "fr": "Chaîne d’approvisionnement", "es": "Cadena de suministro", "de": "Lieferkette"},
|
| 593 |
+
"primer_exposure_regulation": {"en": "Regulation", "fr": "Réglementation", "es": "Regulación", "de": "Regulierung"},
|
| 594 |
+
"primer_description": {"en": "Description", "fr": "Description", "es": "Descripción", "de": "Beschreibung"},
|
| 595 |
+
"primer_qualitative": {"en": "Qualitative", "fr": "Qualitatif", "es": "Cualitativo", "de": "Qualitativ"},
|
| 596 |
+
"primer_revenue": {"en": "Revenue", "fr": "Chiffre d’affaires", "es": "Ingresos", "de": "Umsatz"},
|
| 597 |
+
"primer_op_margin": {"en": "Operating margin", "fr": "Marge opérationnelle", "es": "Margen operativo", "de": "Operative Marge"},
|
| 598 |
+
"primer_fcf": {"en": "Free cash flow", "fr": "Free cash flow", "es": "Flujo de caja libre", "de": "Free Cashflow"},
|
| 599 |
+
"primer_period": {"en": "Period", "fr": "Période", "es": "Período", "de": "Zeitraum"},
|
| 600 |
+
"primer_change": {"en": "What changed", "fr": "Ce qui a changé", "es": "Qué cambió", "de": "Was sich geändert hat"},
|
| 601 |
+
"primer_implication": {"en": "Implication", "fr": "Implication", "es": "Implicación", "de": "Implikation"},
|
| 602 |
+
"primer_insufficient": {"en": "Insufficient history", "fr": "Historique insuffisant", "es": "Historial insuficiente", "de": "Unzureichende Historie"},
|
| 603 |
+
"primer_news_count": {"en": "{count} retrieved news item(s)", "fr": "{count} actualité(s) récupérée(s)", "es": "{count} noticia(s) recuperada(s)", "de": "{count} abgerufene Nachricht(en)"},
|
| 604 |
+
"primer_direction_rising": {"en": "Rising", "fr": "En hausse", "es": "En aumento", "de": "Zunehmend"},
|
| 605 |
+
"primer_direction_stable": {"en": "Stable", "fr": "Stable", "es": "Estable", "de": "Stabil"},
|
| 606 |
+
"primer_direction_falling": {"en": "Falling", "fr": "En baisse", "es": "En descenso", "de": "Abnehmend"},
|
| 607 |
+
"primer_direction_new": {"en": "New", "fr": "Nouveau", "es": "Nuevo", "de": "Neu"},
|
| 608 |
+
"primer_direction_insufficient": {"en": "Insufficient history", "fr": "Historique insuffisant", "es": "Historial insuficiente", "de": "Unzureichende Historie"},
|
| 609 |
+
"primer_variable": {"en": "Variable", "fr": "Variable", "es": "Variable", "de": "Variable"},
|
| 610 |
+
"primer_why": {"en": "Why it matters", "fr": "Pourquoi c’est important", "es": "Por qué importa", "de": "Warum relevant"},
|
| 611 |
+
"primer_next_datapoint": {"en": "Next datapoint", "fr": "Prochain datapoint", "es": "Próximo dato", "de": "Nächster Datenpunkt"},
|
| 612 |
+
"primer_alert": {"en": "Alert signal", "fr": "Signal d’alerte", "es": "Señal de alerta", "de": "Warnsignal"},
|
| 613 |
+
|
| 614 |
"fin_kpi": {"en": "Latest quarter", "fr": "Dernier trimestre", "es": "Último trimestre", "de": "Letztes Quartal"},
|
| 615 |
"fin_next_guidance": {"en": "Next period guidance", "fr": "Guidance du prochain trimestre", "es": "Guidance del próximo período", "de": "Guidance nächste Periode"},
|
| 616 |
"fin_tab_revenue": {"en": "Revenue & Growth", "fr": "CA & croissance", "es": "Ingresos y crecimiento", "de": "Umsatz & Wachstum"},
|
dashboard/nav.py
CHANGED
|
@@ -1,7 +1,7 @@
|
|
| 1 |
"""Data-driven sidebar navigation for Amplegest — institutional style.
|
| 2 |
|
| 3 |
-
|
| 4 |
-
PM Flash → Ask AI → Evidence & Deltas → Financials
|
| 5 |
|
| 6 |
Each item has a stable ``key`` decoupled from its display label,
|
| 7 |
so renaming a section never breaks routing.
|
|
@@ -12,10 +12,11 @@ from dashboard.i18n import t
|
|
| 12 |
|
| 13 |
# ── Per-section color palette (accent, pastel bg, pastel border) ──────────────
|
| 14 |
# Deliberately narrow: emerald is the single brand/active accent shared by the
|
| 15 |
-
#
|
| 16 |
# exclusively for the AI tool (Chat), so a glance at the sidebar tells you
|
| 17 |
# "data screen" from "AI screen" without six competing hues to parse.
|
| 18 |
SECTION_COLORS: dict[str, tuple[str, str, str]] = {
|
|
|
|
| 19 |
"verdict": ("#10b981", "#ecfdf5", "#a7f3d0"), # emerald (brand)
|
| 20 |
"signals": ("#10b981", "#ecfdf5", "#a7f3d0"), # emerald (brand)
|
| 21 |
"financials": ("#10b981", "#ecfdf5", "#a7f3d0"), # emerald (brand)
|
|
@@ -23,9 +24,10 @@ SECTION_COLORS: dict[str, tuple[str, str, str]] = {
|
|
| 23 |
}
|
| 24 |
|
| 25 |
# ── Navigation items ──────────────────────────────────────────────────────────
|
| 26 |
-
#
|
| 27 |
-
#
|
| 28 |
NAV_ITEMS: list[dict] = [
|
|
|
|
| 29 |
{"key": "verdict", "i18n_key": "nav_verdict", "display_label": "PM Flash"},
|
| 30 |
{"key": "chat", "i18n_key": "nav_chat"},
|
| 31 |
{"key": "signals", "i18n_key": "nav_signals", "display_label": "Evidence & Deltas"},
|
|
@@ -45,6 +47,7 @@ LEGACY_NAV: dict[str, str] = {
|
|
| 45 |
"earnings_call": "signals",
|
| 46 |
"quality_tone": "signals",
|
| 47 |
"portfolio": "verdict",
|
|
|
|
| 48 |
}
|
| 49 |
|
| 50 |
|
|
@@ -56,6 +59,7 @@ def section_state(brief: dict | None) -> dict[str, str]:
|
|
| 56 |
"""
|
| 57 |
has_brief = bool(brief)
|
| 58 |
return {
|
|
|
|
| 59 |
"verdict": "ready" if has_brief else "—",
|
| 60 |
"signals": "ready" if has_brief else "—",
|
| 61 |
"financials": "ready", # reads SQLite directly, brief optional
|
|
|
|
| 1 |
"""Data-driven sidebar navigation for Amplegest — institutional style.
|
| 2 |
|
| 3 |
+
Five primary destinations, from company onboarding to decision and audit:
|
| 4 |
+
Company Primer → PM Flash → Ask AI → Evidence & Deltas → Financials
|
| 5 |
|
| 6 |
Each item has a stable ``key`` decoupled from its display label,
|
| 7 |
so renaming a section never breaks routing.
|
|
|
|
| 12 |
|
| 13 |
# ── Per-section color palette (accent, pastel bg, pastel border) ──────────────
|
| 14 |
# Deliberately narrow: emerald is the single brand/active accent shared by the
|
| 15 |
+
# four data screens (Company, Verdict, Signals, Financials); violet is reserved
|
| 16 |
# exclusively for the AI tool (Chat), so a glance at the sidebar tells you
|
| 17 |
# "data screen" from "AI screen" without six competing hues to parse.
|
| 18 |
SECTION_COLORS: dict[str, tuple[str, str, str]] = {
|
| 19 |
+
"company": ("#10b981", "#ecfdf5", "#a7f3d0"), # emerald (brand)
|
| 20 |
"verdict": ("#10b981", "#ecfdf5", "#a7f3d0"), # emerald (brand)
|
| 21 |
"signals": ("#10b981", "#ecfdf5", "#a7f3d0"), # emerald (brand)
|
| 22 |
"financials": ("#10b981", "#ecfdf5", "#a7f3d0"), # emerald (brand)
|
|
|
|
| 24 |
}
|
| 25 |
|
| 26 |
# ── Navigation items ──────────────────────────────────────────────────────────
|
| 27 |
+
# Company Primer onboards the analyst; Chat stays immediately after PM Flash
|
| 28 |
+
# for fast interrogation of the decision evidence.
|
| 29 |
NAV_ITEMS: list[dict] = [
|
| 30 |
+
{"key": "company", "i18n_key": "nav_company"},
|
| 31 |
{"key": "verdict", "i18n_key": "nav_verdict", "display_label": "PM Flash"},
|
| 32 |
{"key": "chat", "i18n_key": "nav_chat"},
|
| 33 |
{"key": "signals", "i18n_key": "nav_signals", "display_label": "Evidence & Deltas"},
|
|
|
|
| 47 |
"earnings_call": "signals",
|
| 48 |
"quality_tone": "signals",
|
| 49 |
"portfolio": "verdict",
|
| 50 |
+
"overview": "company",
|
| 51 |
}
|
| 52 |
|
| 53 |
|
|
|
|
| 59 |
"""
|
| 60 |
has_brief = bool(brief)
|
| 61 |
return {
|
| 62 |
+
"company": "ready",
|
| 63 |
"verdict": "ready" if has_brief else "—",
|
| 64 |
"signals": "ready" if has_brief else "—",
|
| 65 |
"financials": "ready", # reads SQLite directly, brief optional
|
ingest.py
CHANGED
|
@@ -72,7 +72,18 @@ def ingest(ticker: str, full: bool = False) -> None:
|
|
| 72 |
# --- Delta check ---
|
| 73 |
if not full:
|
| 74 |
transcript_resolved = get_section(edgar.ticker, edgar.period, "transcript")
|
| 75 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 76 |
print(f"[ingest] SKIP {label} (already ingested)")
|
| 77 |
skipped += 1
|
| 78 |
continue
|
|
@@ -148,6 +159,17 @@ def ingest(ticker: str, full: bool = False) -> None:
|
|
| 148 |
upsert_section(edgar.ticker, edgar.period, edgar.form_type, "mda", edgar.mda_text)
|
| 149 |
if edgar.risk_factors_text:
|
| 150 |
upsert_section(edgar.ticker, edgar.period, edgar.form_type, "risk_factors", edgar.risk_factors_text)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 151 |
|
| 152 |
embed_and_store_filing(
|
| 153 |
ticker=edgar.ticker,
|
|
@@ -159,6 +181,8 @@ def ingest(ticker: str, full: bool = False) -> None:
|
|
| 159 |
form_type=edgar.form_type,
|
| 160 |
accession=edgar.accession,
|
| 161 |
source_url=edgar.source_url,
|
|
|
|
|
|
|
| 162 |
)
|
| 163 |
|
| 164 |
# Always mark transcript resolution — even empty — so future delta runs skip the AV call.
|
|
|
|
| 72 |
# --- Delta check ---
|
| 73 |
if not full:
|
| 74 |
transcript_resolved = get_section(edgar.ticker, edgar.period, "transcript")
|
| 75 |
+
profile_sections_resolved = (
|
| 76 |
+
edgar.form_type != "10-K"
|
| 77 |
+
or (
|
| 78 |
+
get_section(edgar.ticker, edgar.period, "business") is not None
|
| 79 |
+
and get_section(edgar.ticker, edgar.period, "segments_geography") is not None
|
| 80 |
+
)
|
| 81 |
+
)
|
| 82 |
+
if (
|
| 83 |
+
edgar.period in existing_metric_periods
|
| 84 |
+
and transcript_resolved is not None
|
| 85 |
+
and profile_sections_resolved
|
| 86 |
+
):
|
| 87 |
print(f"[ingest] SKIP {label} (already ingested)")
|
| 88 |
skipped += 1
|
| 89 |
continue
|
|
|
|
| 159 |
upsert_section(edgar.ticker, edgar.period, edgar.form_type, "mda", edgar.mda_text)
|
| 160 |
if edgar.risk_factors_text:
|
| 161 |
upsert_section(edgar.ticker, edgar.period, edgar.form_type, "risk_factors", edgar.risk_factors_text)
|
| 162 |
+
if edgar.form_type == "10-K":
|
| 163 |
+
# Store empty results too: this records that the profile sections
|
| 164 |
+
# were attempted and prevents endless delta backfills.
|
| 165 |
+
upsert_section(edgar.ticker, edgar.period, edgar.form_type, "business", edgar.business_text)
|
| 166 |
+
upsert_section(
|
| 167 |
+
edgar.ticker,
|
| 168 |
+
edgar.period,
|
| 169 |
+
edgar.form_type,
|
| 170 |
+
"segments_geography",
|
| 171 |
+
edgar.segments_geography_text,
|
| 172 |
+
)
|
| 173 |
|
| 174 |
embed_and_store_filing(
|
| 175 |
ticker=edgar.ticker,
|
|
|
|
| 181 |
form_type=edgar.form_type,
|
| 182 |
accession=edgar.accession,
|
| 183 |
source_url=edgar.source_url,
|
| 184 |
+
business_text=edgar.business_text,
|
| 185 |
+
segments_geography_text=edgar.segments_geography_text,
|
| 186 |
)
|
| 187 |
|
| 188 |
# Always mark transcript resolution — even empty — so future delta runs skip the AV call.
|
ingestion/edgar.py
CHANGED
|
@@ -24,6 +24,8 @@ class EdgarData:
|
|
| 24 |
free_cash_flow: Optional[float]
|
| 25 |
mda_text: str
|
| 26 |
risk_factors_text: str
|
|
|
|
|
|
|
| 27 |
shares_diluted: Optional[float] = None
|
| 28 |
effective_tax_rate: Optional[float] = None
|
| 29 |
interest_expense: Optional[float] = None
|
|
@@ -703,15 +705,29 @@ def compute_metrics_for_accn(
|
|
| 703 |
|
| 704 |
_ITEM_7 = re.compile(r'^\s*item\s+7\b(?!\s*a)', re.IGNORECASE | re.MULTILINE)
|
| 705 |
_ITEM_7A = re.compile(r'^\s*item\s+7\s*a\b', re.IGNORECASE | re.MULTILINE)
|
|
|
|
| 706 |
_ITEM_1A = re.compile(r'^\s*item\s+1\s*a\b', re.IGNORECASE | re.MULTILINE)
|
| 707 |
_ITEM_1B = re.compile(r'^\s*item\s+1\s*b\b', re.IGNORECASE | re.MULTILINE)
|
| 708 |
_ITEM_2_Q = re.compile(r'^\s*item\s+2\b', re.IGNORECASE | re.MULTILINE)
|
| 709 |
_ITEM_3_Q = re.compile(r'^\s*item\s+3\b', re.IGNORECASE | re.MULTILINE)
|
|
|
|
|
|
|
| 710 |
|
| 711 |
_MDA_CAP = 80_000
|
| 712 |
_RISK_CAP = 40_000
|
|
|
|
|
|
|
|
|
|
| 713 |
_FALLBACK_WIN = 60_000
|
| 714 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 715 |
|
| 716 |
def _extract_item_section(text: str, start_re: re.Pattern, end_re: re.Pattern, cap: int) -> str:
|
| 717 |
"""Extract text from the last occurrence of start_re up to the first end_re after it."""
|
|
@@ -737,8 +753,65 @@ def _extract_risk_factors(text: str, form_type: str = "10-K") -> str:
|
|
| 737 |
return _extract_item_section(text, _ITEM_1A, end_re, _RISK_CAP)
|
| 738 |
|
| 739 |
|
| 740 |
-
def
|
| 741 |
-
"""
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 742 |
accn_clean = accn.replace("-", "")
|
| 743 |
cik_int = int(cik)
|
| 744 |
doc_url = f"https://www.sec.gov/Archives/edgar/data/{cik_int}/{accn_clean}/{primary_doc}"
|
|
@@ -747,18 +820,46 @@ def get_filing_text_for_accn(cik: str, accn: str, primary_doc: str, form_type: s
|
|
| 747 |
resp.raise_for_status()
|
| 748 |
except (httpx.HTTPError, httpx.TimeoutException) as e:
|
| 749 |
print(f"WARNING: filing fetch failed cik={cik} accn={accn}: {e}")
|
| 750 |
-
return
|
| 751 |
-
|
| 752 |
-
|
| 753 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
| 754 |
mda = _extract_mda(text, form_type)
|
| 755 |
risk = _extract_risk_factors(text, form_type)
|
|
|
|
|
|
|
| 756 |
if not mda:
|
| 757 |
print(f"WARNING: MD&A section not found in {accn} ({form_type})")
|
| 758 |
if not risk:
|
| 759 |
print(f"WARNING: Risk Factors section not found in {accn} ({form_type})")
|
| 760 |
-
|
| 761 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 762 |
|
| 763 |
|
| 764 |
def _extract_guidance(mda_text: str) -> Optional[str]:
|
|
@@ -816,7 +917,9 @@ def fetch_all_edgar_data(
|
|
| 816 |
report_date=acc.get("report_date", ""),
|
| 817 |
filing_date=acc.get("filing_date", ""),
|
| 818 |
)
|
| 819 |
-
|
|
|
|
|
|
|
| 820 |
results.append(EdgarData(
|
| 821 |
ticker=ticker.upper(),
|
| 822 |
company_name=company_name,
|
|
@@ -829,8 +932,10 @@ def fetch_all_edgar_data(
|
|
| 829 |
gross_margin=metrics["gross_margin"],
|
| 830 |
operating_margin=metrics["operating_margin"],
|
| 831 |
free_cash_flow=metrics["free_cash_flow"],
|
| 832 |
-
mda_text=
|
| 833 |
-
risk_factors_text=
|
|
|
|
|
|
|
| 834 |
shares_diluted=metrics["shares_diluted"],
|
| 835 |
effective_tax_rate=metrics["effective_tax_rate"],
|
| 836 |
interest_expense=metrics["interest_expense"],
|
|
|
|
| 24 |
free_cash_flow: Optional[float]
|
| 25 |
mda_text: str
|
| 26 |
risk_factors_text: str
|
| 27 |
+
business_text: str = ""
|
| 28 |
+
segments_geography_text: str = ""
|
| 29 |
shares_diluted: Optional[float] = None
|
| 30 |
effective_tax_rate: Optional[float] = None
|
| 31 |
interest_expense: Optional[float] = None
|
|
|
|
| 705 |
|
| 706 |
_ITEM_7 = re.compile(r'^\s*item\s+7\b(?!\s*a)', re.IGNORECASE | re.MULTILINE)
|
| 707 |
_ITEM_7A = re.compile(r'^\s*item\s+7\s*a\b', re.IGNORECASE | re.MULTILINE)
|
| 708 |
+
_ITEM_1 = re.compile(r'^\s*item\s+1\b(?!\s*[a-z])', re.IGNORECASE | re.MULTILINE)
|
| 709 |
_ITEM_1A = re.compile(r'^\s*item\s+1\s*a\b', re.IGNORECASE | re.MULTILINE)
|
| 710 |
_ITEM_1B = re.compile(r'^\s*item\s+1\s*b\b', re.IGNORECASE | re.MULTILINE)
|
| 711 |
_ITEM_2_Q = re.compile(r'^\s*item\s+2\b', re.IGNORECASE | re.MULTILINE)
|
| 712 |
_ITEM_3_Q = re.compile(r'^\s*item\s+3\b', re.IGNORECASE | re.MULTILINE)
|
| 713 |
+
_ITEM_8 = re.compile(r'^\s*item\s+8\b', re.IGNORECASE | re.MULTILINE)
|
| 714 |
+
_ITEM_9 = re.compile(r'^\s*item\s+9\b(?!\s*[a-z])', re.IGNORECASE | re.MULTILINE)
|
| 715 |
|
| 716 |
_MDA_CAP = 80_000
|
| 717 |
_RISK_CAP = 40_000
|
| 718 |
+
_BUSINESS_CAP = 80_000
|
| 719 |
+
_SEGMENT_CAP = 60_000
|
| 720 |
+
_ITEM_8_CAP = 240_000
|
| 721 |
_FALLBACK_WIN = 60_000
|
| 722 |
|
| 723 |
+
_SEGMENT_GEO_HEADING = re.compile(
|
| 724 |
+
r"(?im)^\s*(?:note\s+\d+\s*[-–—.:]?\s*)?"
|
| 725 |
+
r"(?:segment(?:s| information| reporting)?|reportable segments|"
|
| 726 |
+
r"geographic(?:al)? information|revenue by geographic(?: area)?|"
|
| 727 |
+
r"net sales by (?:reportable segment|geographic(?: area)?)|"
|
| 728 |
+
r"long-lived assets by geographic(?: area)?)\s*$"
|
| 729 |
+
)
|
| 730 |
+
|
| 731 |
|
| 732 |
def _extract_item_section(text: str, start_re: re.Pattern, end_re: re.Pattern, cap: int) -> str:
|
| 733 |
"""Extract text from the last occurrence of start_re up to the first end_re after it."""
|
|
|
|
| 753 |
return _extract_item_section(text, _ITEM_1A, end_re, _RISK_CAP)
|
| 754 |
|
| 755 |
|
| 756 |
+
def _extract_business(text: str, form_type: str = "10-K") -> str:
|
| 757 |
+
"""Extract Item 1 Business from an annual filing."""
|
| 758 |
+
if form_type != "10-K":
|
| 759 |
+
return ""
|
| 760 |
+
return _extract_item_section(text, _ITEM_1, _ITEM_1A, _BUSINESS_CAP)
|
| 761 |
+
|
| 762 |
+
|
| 763 |
+
def _extract_segments_geography(text: str, form_type: str = "10-K") -> str:
|
| 764 |
+
"""Extract targeted segment/geography disclosures from Item 8."""
|
| 765 |
+
if form_type != "10-K":
|
| 766 |
+
return ""
|
| 767 |
+
item8 = _extract_item_section(text, _ITEM_8, _ITEM_9, _ITEM_8_CAP)
|
| 768 |
+
search_text = item8 or text
|
| 769 |
+
matches = list(_SEGMENT_GEO_HEADING.finditer(search_text))
|
| 770 |
+
if not matches:
|
| 771 |
+
return ""
|
| 772 |
+
|
| 773 |
+
windows: list[str] = []
|
| 774 |
+
last_end = -1
|
| 775 |
+
for match in matches:
|
| 776 |
+
start = max(0, match.start() - 300)
|
| 777 |
+
end = min(len(search_text), match.start() + 12_000)
|
| 778 |
+
if start <= last_end and windows:
|
| 779 |
+
windows[-1] = (windows[-1] + search_text[last_end:end])[:_SEGMENT_CAP]
|
| 780 |
+
last_end = max(last_end, end)
|
| 781 |
+
continue
|
| 782 |
+
windows.append(search_text[start:end].strip())
|
| 783 |
+
last_end = end
|
| 784 |
+
if sum(len(window) for window in windows) >= _SEGMENT_CAP:
|
| 785 |
+
break
|
| 786 |
+
return "\n\n---\n\n".join(windows)[:_SEGMENT_CAP]
|
| 787 |
+
|
| 788 |
+
|
| 789 |
+
def _html_to_filing_text(html: str) -> str:
|
| 790 |
+
"""Flatten filing HTML while preserving table rows as pipe-delimited text."""
|
| 791 |
+
from bs4 import BeautifulSoup
|
| 792 |
+
|
| 793 |
+
soup = BeautifulSoup(html, "html.parser")
|
| 794 |
+
for table in soup.find_all("table"):
|
| 795 |
+
rows: list[str] = []
|
| 796 |
+
for row in table.find_all("tr"):
|
| 797 |
+
cells = [
|
| 798 |
+
" ".join(cell.get_text(" ", strip=True).split())
|
| 799 |
+
for cell in row.find_all(["th", "td"])
|
| 800 |
+
]
|
| 801 |
+
if any(cells):
|
| 802 |
+
rows.append(" | ".join(cells))
|
| 803 |
+
if rows:
|
| 804 |
+
table.replace_with("\n" + "\n".join(rows) + "\n")
|
| 805 |
+
return soup.get_text(separator="\n", strip=True)
|
| 806 |
+
|
| 807 |
+
|
| 808 |
+
def get_filing_sections_for_accn(
|
| 809 |
+
cik: str,
|
| 810 |
+
accn: str,
|
| 811 |
+
primary_doc: str,
|
| 812 |
+
form_type: str = "10-Q",
|
| 813 |
+
) -> dict[str, str]:
|
| 814 |
+
"""Return all filing sections needed by the research surfaces."""
|
| 815 |
accn_clean = accn.replace("-", "")
|
| 816 |
cik_int = int(cik)
|
| 817 |
doc_url = f"https://www.sec.gov/Archives/edgar/data/{cik_int}/{accn_clean}/{primary_doc}"
|
|
|
|
| 820 |
resp.raise_for_status()
|
| 821 |
except (httpx.HTTPError, httpx.TimeoutException) as e:
|
| 822 |
print(f"WARNING: filing fetch failed cik={cik} accn={accn}: {e}")
|
| 823 |
+
return {
|
| 824 |
+
"mda": "",
|
| 825 |
+
"risk_factors": "",
|
| 826 |
+
"business": "",
|
| 827 |
+
"segments_geography": "",
|
| 828 |
+
}
|
| 829 |
+
|
| 830 |
+
text = _html_to_filing_text(resp.text)
|
| 831 |
mda = _extract_mda(text, form_type)
|
| 832 |
risk = _extract_risk_factors(text, form_type)
|
| 833 |
+
business = _extract_business(text, form_type)
|
| 834 |
+
segments_geography = _extract_segments_geography(text, form_type)
|
| 835 |
if not mda:
|
| 836 |
print(f"WARNING: MD&A section not found in {accn} ({form_type})")
|
| 837 |
if not risk:
|
| 838 |
print(f"WARNING: Risk Factors section not found in {accn} ({form_type})")
|
| 839 |
+
if form_type == "10-K" and not business:
|
| 840 |
+
print(f"WARNING: Business section not found in {accn} ({form_type})")
|
| 841 |
+
print(
|
| 842 |
+
f"INFO: filing {accn} ({form_type}) extracted "
|
| 843 |
+
f"MD&A={len(mda)} chars, Risk={len(risk)} chars, "
|
| 844 |
+
f"Business={len(business)} chars, Segment/Geo={len(segments_geography)} chars"
|
| 845 |
+
)
|
| 846 |
+
return {
|
| 847 |
+
"mda": mda,
|
| 848 |
+
"risk_factors": risk,
|
| 849 |
+
"business": business,
|
| 850 |
+
"segments_geography": segments_geography,
|
| 851 |
+
}
|
| 852 |
+
|
| 853 |
+
|
| 854 |
+
def get_filing_text_for_accn(
|
| 855 |
+
cik: str,
|
| 856 |
+
accn: str,
|
| 857 |
+
primary_doc: str,
|
| 858 |
+
form_type: str = "10-Q",
|
| 859 |
+
) -> tuple[str, str]:
|
| 860 |
+
"""Backward-compatible wrapper returning only (MD&A, risk factors)."""
|
| 861 |
+
sections = get_filing_sections_for_accn(cik, accn, primary_doc, form_type)
|
| 862 |
+
return sections["mda"], sections["risk_factors"]
|
| 863 |
|
| 864 |
|
| 865 |
def _extract_guidance(mda_text: str) -> Optional[str]:
|
|
|
|
| 917 |
report_date=acc.get("report_date", ""),
|
| 918 |
filing_date=acc.get("filing_date", ""),
|
| 919 |
)
|
| 920 |
+
filing_sections = get_filing_sections_for_accn(
|
| 921 |
+
cik, acc["accn"], acc["primary_doc"], form_type
|
| 922 |
+
)
|
| 923 |
results.append(EdgarData(
|
| 924 |
ticker=ticker.upper(),
|
| 925 |
company_name=company_name,
|
|
|
|
| 932 |
gross_margin=metrics["gross_margin"],
|
| 933 |
operating_margin=metrics["operating_margin"],
|
| 934 |
free_cash_flow=metrics["free_cash_flow"],
|
| 935 |
+
mda_text=filing_sections["mda"],
|
| 936 |
+
risk_factors_text=filing_sections["risk_factors"],
|
| 937 |
+
business_text=filing_sections["business"],
|
| 938 |
+
segments_geography_text=filing_sections["segments_geography"],
|
| 939 |
shares_diluted=metrics["shares_diluted"],
|
| 940 |
effective_tax_rate=metrics["effective_tax_rate"],
|
| 941 |
interest_expense=metrics["interest_expense"],
|
ingestion/embedder.py
CHANGED
|
@@ -30,16 +30,23 @@ def embed_and_store_filing(
|
|
| 30 |
accession: str = "",
|
| 31 |
source_url: str = "",
|
| 32 |
document_id: str = "",
|
|
|
|
|
|
|
| 33 |
) -> None:
|
| 34 |
ticker = ticker.upper()
|
| 35 |
filing_document_id = document_id or f"sec:{ticker}:{accession or f'{form_type}:{period}:{filing_date}'}"
|
| 36 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 37 |
if not text.strip():
|
| 38 |
print(f"WARNING: empty section '{section}' for {ticker} {period} ({form_type}) — Chroma chunk not written")
|
| 39 |
continue
|
| 40 |
chunks = chunk_text(text)
|
| 41 |
total = len(chunks)
|
| 42 |
-
section_slug = "mda" if section == "MD&A" else "risk_factors"
|
| 43 |
chunk_ids = [f"{section_slug}:{i}" for i in range(total)]
|
| 44 |
metadatas = [
|
| 45 |
{
|
|
|
|
| 30 |
accession: str = "",
|
| 31 |
source_url: str = "",
|
| 32 |
document_id: str = "",
|
| 33 |
+
business_text: str = "",
|
| 34 |
+
segments_geography_text: str = "",
|
| 35 |
) -> None:
|
| 36 |
ticker = ticker.upper()
|
| 37 |
filing_document_id = document_id or f"sec:{ticker}:{accession or f'{form_type}:{period}:{filing_date}'}"
|
| 38 |
+
sections = [
|
| 39 |
+
("MD&A", "mda", mda_text),
|
| 40 |
+
("Risk Factors", "risk_factors", risk_text),
|
| 41 |
+
("Business", "business", business_text),
|
| 42 |
+
("Segments & Geography", "segments_geography", segments_geography_text),
|
| 43 |
+
]
|
| 44 |
+
for section, section_slug, text in sections:
|
| 45 |
if not text.strip():
|
| 46 |
print(f"WARNING: empty section '{section}' for {ticker} {period} ({form_type}) — Chroma chunk not written")
|
| 47 |
continue
|
| 48 |
chunks = chunk_text(text)
|
| 49 |
total = len(chunks)
|
|
|
|
| 50 |
chunk_ids = [f"{section_slug}:{i}" for i in range(total)]
|
| 51 |
metadatas = [
|
| 52 |
{
|
storage/company_profiles.py
ADDED
|
@@ -0,0 +1,172 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""Append-only persistence for Company Primer profiles and translations."""
|
| 2 |
+
from __future__ import annotations
|
| 3 |
+
|
| 4 |
+
import hashlib
|
| 5 |
+
import json
|
| 6 |
+
import sqlite3
|
| 7 |
+
from datetime import datetime, timezone
|
| 8 |
+
from pathlib import Path
|
| 9 |
+
from typing import Optional
|
| 10 |
+
|
| 11 |
+
DB_PATH = Path("data/company_profiles.db")
|
| 12 |
+
SCHEMA_VERSION = "company_profile.v1"
|
| 13 |
+
|
| 14 |
+
|
| 15 |
+
def init_db() -> None:
|
| 16 |
+
DB_PATH.parent.mkdir(parents=True, exist_ok=True)
|
| 17 |
+
with sqlite3.connect(DB_PATH) as conn:
|
| 18 |
+
conn.execute(
|
| 19 |
+
"""
|
| 20 |
+
CREATE TABLE IF NOT EXISTS company_profiles (
|
| 21 |
+
profile_id TEXT PRIMARY KEY,
|
| 22 |
+
ticker TEXT NOT NULL,
|
| 23 |
+
source_fingerprint TEXT NOT NULL,
|
| 24 |
+
schema_version TEXT NOT NULL,
|
| 25 |
+
generated_at TEXT NOT NULL,
|
| 26 |
+
data_as_of TEXT,
|
| 27 |
+
language TEXT NOT NULL,
|
| 28 |
+
status TEXT NOT NULL,
|
| 29 |
+
profile_json TEXT NOT NULL
|
| 30 |
+
)
|
| 31 |
+
"""
|
| 32 |
+
)
|
| 33 |
+
conn.execute(
|
| 34 |
+
"""
|
| 35 |
+
CREATE INDEX IF NOT EXISTS idx_company_profiles_latest
|
| 36 |
+
ON company_profiles (ticker, generated_at DESC)
|
| 37 |
+
"""
|
| 38 |
+
)
|
| 39 |
+
conn.execute(
|
| 40 |
+
"""
|
| 41 |
+
CREATE TABLE IF NOT EXISTS company_profile_translations (
|
| 42 |
+
profile_id TEXT NOT NULL,
|
| 43 |
+
language TEXT NOT NULL,
|
| 44 |
+
profile_json TEXT NOT NULL,
|
| 45 |
+
saved_at TEXT NOT NULL,
|
| 46 |
+
PRIMARY KEY (profile_id, language)
|
| 47 |
+
)
|
| 48 |
+
"""
|
| 49 |
+
)
|
| 50 |
+
|
| 51 |
+
|
| 52 |
+
def _profile_id(profile: dict) -> str:
|
| 53 |
+
identity = "\x1f".join(
|
| 54 |
+
(
|
| 55 |
+
str(profile.get("ticker") or "").upper(),
|
| 56 |
+
str(profile.get("source_fingerprint") or ""),
|
| 57 |
+
str(profile.get("schema_version") or SCHEMA_VERSION),
|
| 58 |
+
str(profile.get("generated_at") or ""),
|
| 59 |
+
str(profile.get("language") or "English"),
|
| 60 |
+
)
|
| 61 |
+
)
|
| 62 |
+
return hashlib.sha256(identity.encode("utf-8")).hexdigest()[:32]
|
| 63 |
+
|
| 64 |
+
|
| 65 |
+
def save_profile(ticker: str, profile: dict) -> str:
|
| 66 |
+
init_db()
|
| 67 |
+
payload = dict(profile)
|
| 68 |
+
payload["ticker"] = ticker.upper()
|
| 69 |
+
payload.setdefault("schema_version", SCHEMA_VERSION)
|
| 70 |
+
payload.setdefault("language", "English")
|
| 71 |
+
payload.setdefault("generated_at", datetime.now(timezone.utc).isoformat())
|
| 72 |
+
payload.setdefault("status", "PARTIAL")
|
| 73 |
+
profile_id = _profile_id(payload)
|
| 74 |
+
with sqlite3.connect(DB_PATH) as conn:
|
| 75 |
+
conn.execute(
|
| 76 |
+
"""
|
| 77 |
+
INSERT OR IGNORE INTO company_profiles (
|
| 78 |
+
profile_id, ticker, source_fingerprint, schema_version,
|
| 79 |
+
generated_at, data_as_of, language, status, profile_json
|
| 80 |
+
) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?)
|
| 81 |
+
""",
|
| 82 |
+
(
|
| 83 |
+
profile_id,
|
| 84 |
+
payload["ticker"],
|
| 85 |
+
payload.get("source_fingerprint") or "",
|
| 86 |
+
payload["schema_version"],
|
| 87 |
+
payload["generated_at"],
|
| 88 |
+
payload.get("data_as_of"),
|
| 89 |
+
payload["language"],
|
| 90 |
+
payload["status"],
|
| 91 |
+
json.dumps(payload, ensure_ascii=False),
|
| 92 |
+
),
|
| 93 |
+
)
|
| 94 |
+
return profile_id
|
| 95 |
+
|
| 96 |
+
|
| 97 |
+
def get_latest_profile(
|
| 98 |
+
ticker: str,
|
| 99 |
+
*,
|
| 100 |
+
source_fingerprint: Optional[str] = None,
|
| 101 |
+
language: str = "English",
|
| 102 |
+
) -> Optional[dict]:
|
| 103 |
+
if not DB_PATH.exists():
|
| 104 |
+
return None
|
| 105 |
+
init_db()
|
| 106 |
+
clauses = ["ticker = ?", "language = ?"]
|
| 107 |
+
params: list[str] = [ticker.upper(), language]
|
| 108 |
+
if source_fingerprint is not None:
|
| 109 |
+
clauses.append("source_fingerprint = ?")
|
| 110 |
+
params.append(source_fingerprint)
|
| 111 |
+
with sqlite3.connect(DB_PATH) as conn:
|
| 112 |
+
row = conn.execute(
|
| 113 |
+
"""
|
| 114 |
+
SELECT profile_id, profile_json
|
| 115 |
+
FROM company_profiles
|
| 116 |
+
WHERE """
|
| 117 |
+
+ " AND ".join(clauses)
|
| 118 |
+
+ " ORDER BY generated_at DESC LIMIT 1",
|
| 119 |
+
params,
|
| 120 |
+
).fetchone()
|
| 121 |
+
if row is None:
|
| 122 |
+
return None
|
| 123 |
+
try:
|
| 124 |
+
result = json.loads(row[1])
|
| 125 |
+
result["_profile_id"] = row[0]
|
| 126 |
+
return result
|
| 127 |
+
except json.JSONDecodeError:
|
| 128 |
+
return None
|
| 129 |
+
|
| 130 |
+
|
| 131 |
+
def get_translation(profile_id: str, language: str) -> Optional[dict]:
|
| 132 |
+
if not DB_PATH.exists():
|
| 133 |
+
return None
|
| 134 |
+
init_db()
|
| 135 |
+
with sqlite3.connect(DB_PATH) as conn:
|
| 136 |
+
row = conn.execute(
|
| 137 |
+
"""
|
| 138 |
+
SELECT profile_json
|
| 139 |
+
FROM company_profile_translations
|
| 140 |
+
WHERE profile_id = ? AND language = ?
|
| 141 |
+
""",
|
| 142 |
+
(profile_id, language),
|
| 143 |
+
).fetchone()
|
| 144 |
+
if row is None:
|
| 145 |
+
return None
|
| 146 |
+
try:
|
| 147 |
+
return json.loads(row[0])
|
| 148 |
+
except json.JSONDecodeError:
|
| 149 |
+
return None
|
| 150 |
+
|
| 151 |
+
|
| 152 |
+
def save_translation(profile_id: str, language: str, profile: dict) -> None:
|
| 153 |
+
init_db()
|
| 154 |
+
payload = dict(profile)
|
| 155 |
+
payload["language"] = language
|
| 156 |
+
with sqlite3.connect(DB_PATH) as conn:
|
| 157 |
+
conn.execute(
|
| 158 |
+
"""
|
| 159 |
+
INSERT INTO company_profile_translations (
|
| 160 |
+
profile_id, language, profile_json, saved_at
|
| 161 |
+
) VALUES (?, ?, ?, ?)
|
| 162 |
+
ON CONFLICT(profile_id, language) DO UPDATE SET
|
| 163 |
+
profile_json = excluded.profile_json,
|
| 164 |
+
saved_at = excluded.saved_at
|
| 165 |
+
""",
|
| 166 |
+
(
|
| 167 |
+
profile_id,
|
| 168 |
+
language,
|
| 169 |
+
json.dumps(payload, ensure_ascii=False),
|
| 170 |
+
datetime.now(timezone.utc).isoformat(),
|
| 171 |
+
),
|
| 172 |
+
)
|
storage/sections_db.py
CHANGED
|
@@ -1,8 +1,8 @@
|
|
| 1 |
"""storage/sections_db.py — persist raw section text across filing periods.
|
| 2 |
|
| 3 |
-
Stores verbatim MD&A, Risk Factors, and
|
| 4 |
-
(ticker, period, section). Used by
|
| 5 |
-
|
| 6 |
|
| 7 |
This is append-only at ingest time; textdiff reads it at runtime.
|
| 8 |
"""
|
|
@@ -40,7 +40,7 @@ def upsert_section(
|
|
| 40 |
section: str,
|
| 41 |
text: str,
|
| 42 |
) -> None:
|
| 43 |
-
"""Write
|
| 44 |
from datetime import datetime, timezone
|
| 45 |
SECTIONS_DB_PATH.parent.mkdir(parents=True, exist_ok=True)
|
| 46 |
with sqlite3.connect(SECTIONS_DB_PATH) as conn:
|
|
|
|
| 1 |
"""storage/sections_db.py — persist raw section text across filing periods.
|
| 2 |
|
| 3 |
+
Stores verbatim Business, Segments/Geography, MD&A, Risk Factors, and
|
| 4 |
+
transcript text keyed by (ticker, period, section). Used by company-profile
|
| 5 |
+
generation and analysis/textdiff.py without re-fetching source documents.
|
| 6 |
|
| 7 |
This is append-only at ingest time; textdiff reads it at runtime.
|
| 8 |
"""
|
|
|
|
| 40 |
section: str,
|
| 41 |
text: str,
|
| 42 |
) -> None:
|
| 43 |
+
"""Write or overwrite a supported filing or transcript section."""
|
| 44 |
from datetime import datetime, timezone
|
| 45 |
SECTIONS_DB_PATH.parent.mkdir(parents=True, exist_ok=True)
|
| 46 |
with sqlite3.connect(SECTIONS_DB_PATH) as conn:
|
tests/test_company_profile.py
ADDED
|
@@ -0,0 +1,298 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
from types import SimpleNamespace
|
| 2 |
+
import ast
|
| 3 |
+
from pathlib import Path
|
| 4 |
+
|
| 5 |
+
import pytest
|
| 6 |
+
|
| 7 |
+
from agent.company_profile import (
|
| 8 |
+
_collect_translation_segments,
|
| 9 |
+
_prune_unverified,
|
| 10 |
+
_quarantine_unsupported_shares,
|
| 11 |
+
profile_source_coverage,
|
| 12 |
+
source_fingerprint,
|
| 13 |
+
)
|
| 14 |
+
from agent.evidence import make_evidence_record
|
| 15 |
+
from agent.company_profile_schemas import CompanyProfile
|
| 16 |
+
from analysis.company_attention import attach_attention_stats, cluster_questions
|
| 17 |
+
from analytics.company_market import (
|
| 18 |
+
_parse_published_date,
|
| 19 |
+
sector_etf,
|
| 20 |
+
select_relative_events,
|
| 21 |
+
)
|
| 22 |
+
from storage import company_profiles
|
| 23 |
+
|
| 24 |
+
|
| 25 |
+
@pytest.mark.parametrize(
|
| 26 |
+
"relative_path",
|
| 27 |
+
[
|
| 28 |
+
"app.py",
|
| 29 |
+
"dashboard/company_primer.py",
|
| 30 |
+
"dashboard/i18n.py",
|
| 31 |
+
"dashboard/nav.py",
|
| 32 |
+
"agent/company_profile.py",
|
| 33 |
+
"analytics/company_market.py",
|
| 34 |
+
],
|
| 35 |
+
)
|
| 36 |
+
def test_company_primer_files_parse(relative_path):
|
| 37 |
+
source = Path(relative_path).read_text(encoding="utf-8")
|
| 38 |
+
ast.parse(source, filename=relative_path)
|
| 39 |
+
|
| 40 |
+
|
| 41 |
+
def _fact(status="VERIFIED"):
|
| 42 |
+
return {
|
| 43 |
+
"text": "Verified claim.",
|
| 44 |
+
"source": "10-K",
|
| 45 |
+
"reliability": "HIGH",
|
| 46 |
+
"evidence_snippet": "Verified claim.",
|
| 47 |
+
"verification_status": status,
|
| 48 |
+
}
|
| 49 |
+
|
| 50 |
+
|
| 51 |
+
def test_company_profile_schema_accepts_partial_payload():
|
| 52 |
+
profile = CompanyProfile(ticker="AAPL", company_name="Apple Inc.")
|
| 53 |
+
assert profile.status == "PARTIAL"
|
| 54 |
+
assert profile.schema_version == "company_profile.v1"
|
| 55 |
+
assert profile.business_lines == []
|
| 56 |
+
|
| 57 |
+
|
| 58 |
+
def test_prune_unverified_removes_nested_profile_claims():
|
| 59 |
+
profile = {
|
| 60 |
+
"identity": {
|
| 61 |
+
"one_liner": _fact("FAILED"),
|
| 62 |
+
"business_model": _fact(),
|
| 63 |
+
"customer_types": [_fact(), _fact("UNVERIFIED")],
|
| 64 |
+
},
|
| 65 |
+
"business_lines": [
|
| 66 |
+
{"name": "Good", "description": _fact()},
|
| 67 |
+
{"name": "Bad", "description": _fact("FAILED")},
|
| 68 |
+
],
|
| 69 |
+
"geographic_exposures": [],
|
| 70 |
+
"strategic_changes": [],
|
| 71 |
+
"attention_themes": [],
|
| 72 |
+
"watch_variables": [],
|
| 73 |
+
}
|
| 74 |
+
removed = _prune_unverified(profile)
|
| 75 |
+
assert removed == 3
|
| 76 |
+
assert profile["identity"]["one_liner"] is None
|
| 77 |
+
assert len(profile["identity"]["customer_types"]) == 1
|
| 78 |
+
assert [item["name"] for item in profile["business_lines"]] == ["Good"]
|
| 79 |
+
|
| 80 |
+
|
| 81 |
+
def test_translation_segments_skip_verbatim_evidence():
|
| 82 |
+
profile = {
|
| 83 |
+
"identity": {
|
| 84 |
+
"one_liner": {
|
| 85 |
+
"text": "Designs chips.",
|
| 86 |
+
"evidence_snippet": "designs semiconductor chips",
|
| 87 |
+
"source": "10-K",
|
| 88 |
+
}
|
| 89 |
+
},
|
| 90 |
+
"watch_variables": [
|
| 91 |
+
{"variable": "Gross margin", "next_datapoint": "Next earnings"}
|
| 92 |
+
],
|
| 93 |
+
}
|
| 94 |
+
segments = _collect_translation_segments(profile)
|
| 95 |
+
paths = [path for path, _ in segments]
|
| 96 |
+
assert ("identity", "one_liner", "text") in paths
|
| 97 |
+
assert ("identity", "one_liner", "evidence_snippet") not in paths
|
| 98 |
+
assert ("watch_variables", 0, "next_datapoint") in paths
|
| 99 |
+
|
| 100 |
+
|
| 101 |
+
def test_unsupported_parent_revenue_share_is_hidden():
|
| 102 |
+
record = make_evidence_record(
|
| 103 |
+
source="10-K",
|
| 104 |
+
content="The Compute segment serves data-center customers.",
|
| 105 |
+
document_id="sec:test",
|
| 106 |
+
chunk_id="business:1",
|
| 107 |
+
)
|
| 108 |
+
fact = {
|
| 109 |
+
"text": "Compute serves data-center customers.",
|
| 110 |
+
"source": "10-K",
|
| 111 |
+
"reliability": "HIGH",
|
| 112 |
+
"evidence_snippet": "Compute segment serves data-center customers",
|
| 113 |
+
"evidence_ref": record.ref.model_dump(mode="json"),
|
| 114 |
+
"verification_status": "VERIFIED",
|
| 115 |
+
}
|
| 116 |
+
profile = {
|
| 117 |
+
"business_lines": [
|
| 118 |
+
{
|
| 119 |
+
"name": "Compute",
|
| 120 |
+
"description": fact,
|
| 121 |
+
"revenue_share_pct": 80,
|
| 122 |
+
"share_period": "FY2025",
|
| 123 |
+
}
|
| 124 |
+
],
|
| 125 |
+
"geographic_exposures": [],
|
| 126 |
+
}
|
| 127 |
+
removed = _quarantine_unsupported_shares(profile, [record])
|
| 128 |
+
assert removed == 1
|
| 129 |
+
assert profile["business_lines"][0]["revenue_share_pct"] is None
|
| 130 |
+
assert profile["business_lines"][0]["share_period"] is None
|
| 131 |
+
|
| 132 |
+
|
| 133 |
+
def test_source_fingerprint_changes_when_business_backfill_changes(monkeypatch):
|
| 134 |
+
rows = [
|
| 135 |
+
{"form_type": "10-K", "accession": "K1", "filing_date": "2025-01-01"},
|
| 136 |
+
{"form_type": "10-Q", "accession": "Q1", "filing_date": "2025-04-01"},
|
| 137 |
+
]
|
| 138 |
+
monkeypatch.setattr("agent.company_profile.metrics_db.get_all_metrics", lambda _ticker: rows)
|
| 139 |
+
monkeypatch.setattr(
|
| 140 |
+
"agent.company_profile.get_recent_transcripts",
|
| 141 |
+
lambda _ticker, n=1: [("Q12025", "transcript")],
|
| 142 |
+
)
|
| 143 |
+
monkeypatch.setattr(
|
| 144 |
+
"agent.company_profile.get_periods_for_ticker",
|
| 145 |
+
lambda _ticker, form_type=None: ["FY2025"],
|
| 146 |
+
)
|
| 147 |
+
sections = {
|
| 148 |
+
("FY2025", "business"): "",
|
| 149 |
+
("FY2025", "segments_geography"): "",
|
| 150 |
+
}
|
| 151 |
+
monkeypatch.setattr(
|
| 152 |
+
"agent.company_profile.get_section",
|
| 153 |
+
lambda _ticker, period, section: sections[(period, section)],
|
| 154 |
+
)
|
| 155 |
+
before = source_fingerprint("AAPL")
|
| 156 |
+
sections[("FY2025", "business")] = "Item 1. Business content"
|
| 157 |
+
after = source_fingerprint("AAPL")
|
| 158 |
+
assert before != after
|
| 159 |
+
|
| 160 |
+
|
| 161 |
+
def test_profile_coverage_requires_business_geography_and_call_history(monkeypatch):
|
| 162 |
+
monkeypatch.setattr(
|
| 163 |
+
"agent.company_profile.get_periods_for_ticker",
|
| 164 |
+
lambda _ticker, form_type=None: ["FY2025"],
|
| 165 |
+
)
|
| 166 |
+
sections = {
|
| 167 |
+
"business": "Business disclosure",
|
| 168 |
+
"segments_geography": "Geographic disclosure",
|
| 169 |
+
}
|
| 170 |
+
monkeypatch.setattr(
|
| 171 |
+
"agent.company_profile.get_section",
|
| 172 |
+
lambda _ticker, _period, section: sections[section],
|
| 173 |
+
)
|
| 174 |
+
monkeypatch.setattr(
|
| 175 |
+
"agent.company_profile.get_recent_transcripts",
|
| 176 |
+
lambda _ticker, n=4: [("Q42024", "a"), ("Q12025", "b")],
|
| 177 |
+
)
|
| 178 |
+
|
| 179 |
+
complete = profile_source_coverage("AAPL")
|
| 180 |
+
assert complete["status"] == "COMPLETE"
|
| 181 |
+
assert complete["segments_geography_sections_attempted"] == 1
|
| 182 |
+
sections["segments_geography"] = ""
|
| 183 |
+
partial = profile_source_coverage("AAPL")
|
| 184 |
+
assert partial["status"] == "PARTIAL"
|
| 185 |
+
assert partial["segments_geography_sections_attempted"] == 1
|
| 186 |
+
|
| 187 |
+
|
| 188 |
+
def test_company_profile_storage_round_trip(tmp_path, monkeypatch):
|
| 189 |
+
monkeypatch.setattr(company_profiles, "DB_PATH", tmp_path / "profiles.db")
|
| 190 |
+
profile = {
|
| 191 |
+
"ticker": "AAPL",
|
| 192 |
+
"company_name": "Apple Inc.",
|
| 193 |
+
"source_fingerprint": "abc",
|
| 194 |
+
"schema_version": "company_profile.v1",
|
| 195 |
+
"generated_at": "2026-01-01T00:00:00+00:00",
|
| 196 |
+
"data_as_of": "2025-12-31",
|
| 197 |
+
"language": "English",
|
| 198 |
+
"status": "PARTIAL",
|
| 199 |
+
}
|
| 200 |
+
profile_id = company_profiles.save_profile("AAPL", profile)
|
| 201 |
+
loaded = company_profiles.get_latest_profile(
|
| 202 |
+
"AAPL", source_fingerprint="abc", language="English"
|
| 203 |
+
)
|
| 204 |
+
assert loaded["_profile_id"] == profile_id
|
| 205 |
+
company_profiles.save_translation(profile_id, "French", {**profile, "language": "French"})
|
| 206 |
+
assert company_profiles.get_translation(profile_id, "French")["language"] == "French"
|
| 207 |
+
|
| 208 |
+
|
| 209 |
+
def test_attention_cluster_counts_questions(monkeypatch):
|
| 210 |
+
parsed = {
|
| 211 |
+
"Q1": SimpleNamespace(
|
| 212 |
+
qa=[
|
| 213 |
+
SimpleNamespace(question="Can you discuss China demand and China inventory?"),
|
| 214 |
+
SimpleNamespace(question="How should we think about pricing and margins?"),
|
| 215 |
+
]
|
| 216 |
+
),
|
| 217 |
+
"Q2": SimpleNamespace(
|
| 218 |
+
qa=[
|
| 219 |
+
SimpleNamespace(question="What changed in China demand and channel inventory?"),
|
| 220 |
+
SimpleNamespace(question="Can you discuss China inventory and demand trends?"),
|
| 221 |
+
]
|
| 222 |
+
),
|
| 223 |
+
}
|
| 224 |
+
monkeypatch.setattr(
|
| 225 |
+
"analysis.company_attention.parse_call",
|
| 226 |
+
lambda period, _raw: parsed[period],
|
| 227 |
+
)
|
| 228 |
+
clusters = cluster_questions([("Q1", "a"), ("Q2", "b")])
|
| 229 |
+
assert clusters
|
| 230 |
+
assert clusters[0]["question_count"] == 3
|
| 231 |
+
assert clusters[0]["mentions"] == [
|
| 232 |
+
{"period": "Q1", "mentions": 1},
|
| 233 |
+
{"period": "Q2", "mentions": 2},
|
| 234 |
+
]
|
| 235 |
+
|
| 236 |
+
|
| 237 |
+
def test_attach_attention_stats_adds_news_count(monkeypatch):
|
| 238 |
+
monkeypatch.setattr(
|
| 239 |
+
"analysis.company_attention.get_attention_clusters",
|
| 240 |
+
lambda _ticker: [
|
| 241 |
+
{
|
| 242 |
+
"tokens": ["china", "demand"],
|
| 243 |
+
"mentions": [
|
| 244 |
+
{"period": "Q1", "mentions": 0},
|
| 245 |
+
{"period": "Q2", "mentions": 2},
|
| 246 |
+
],
|
| 247 |
+
}
|
| 248 |
+
],
|
| 249 |
+
)
|
| 250 |
+
profile = {
|
| 251 |
+
"attention_themes": [
|
| 252 |
+
{
|
| 253 |
+
"theme": "China demand",
|
| 254 |
+
"evidence": {"text": "China demand remains volatile."},
|
| 255 |
+
}
|
| 256 |
+
]
|
| 257 |
+
}
|
| 258 |
+
attach_attention_stats(
|
| 259 |
+
profile,
|
| 260 |
+
"AAPL",
|
| 261 |
+
[{"title": "China demand improves", "snippet": ""}, {"title": "Dividend", "snippet": ""}],
|
| 262 |
+
)
|
| 263 |
+
theme = profile["attention_themes"][0]
|
| 264 |
+
assert theme["direction"] == "new"
|
| 265 |
+
assert theme["retrieved_news_count"] == 1
|
| 266 |
+
|
| 267 |
+
|
| 268 |
+
def test_sector_etf_mapping_and_fallback():
|
| 269 |
+
assert sector_etf("Technology") == "XLK"
|
| 270 |
+
assert sector_etf("Consumer Cyclical") == "XLY"
|
| 271 |
+
assert sector_etf("Unknown") == "SPY"
|
| 272 |
+
|
| 273 |
+
|
| 274 |
+
def test_market_news_dates_accept_iso_and_rfc_formats():
|
| 275 |
+
assert _parse_published_date("2026-07-29T08:30:00Z").isoformat() == "2026-07-29"
|
| 276 |
+
assert (
|
| 277 |
+
_parse_published_date("Wed, 29 Jul 2026 08:30:00 GMT").isoformat()
|
| 278 |
+
== "2026-07-29"
|
| 279 |
+
)
|
| 280 |
+
assert _parse_published_date("not a date") is None
|
| 281 |
+
|
| 282 |
+
|
| 283 |
+
def test_relative_events_are_balanced_thresholded_and_spaced():
|
| 284 |
+
dates = [f"2026-01-{day:02d}" for day in range(1, 13)]
|
| 285 |
+
company = [100, 105, 106, 107, 108, 109, 102, 101, 100, 99, 106, 107]
|
| 286 |
+
sector = [100] * len(company)
|
| 287 |
+
events = select_relative_events(
|
| 288 |
+
dates,
|
| 289 |
+
company,
|
| 290 |
+
sector,
|
| 291 |
+
threshold_pct=2,
|
| 292 |
+
per_direction=2,
|
| 293 |
+
min_spacing=3,
|
| 294 |
+
)
|
| 295 |
+
assert any(item["sector_excess_pct"] > 0 for item in events)
|
| 296 |
+
assert any(item["sector_excess_pct"] < 0 for item in events)
|
| 297 |
+
indices = [item["index"] for item in events]
|
| 298 |
+
assert all(abs(a - b) >= 3 for i, a in enumerate(indices) for b in indices[i + 1 :])
|
tests/test_edgar.py
CHANGED
|
@@ -1,6 +1,16 @@
|
|
| 1 |
import pytest
|
| 2 |
from unittest.mock import patch, MagicMock
|
| 3 |
-
from ingestion.edgar import
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 4 |
|
| 5 |
FAKE_TICKERS = {
|
| 6 |
"0": {"cik_str": 320193, "ticker": "AAPL", "title": "Apple Inc."}
|
|
@@ -240,6 +250,57 @@ def test_extract_risk_factors_returns_empty_when_not_found():
|
|
| 240 |
assert result == ""
|
| 241 |
|
| 242 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 243 |
# ── XBRL concept expansion ───────────────────────────────────────────────────
|
| 244 |
|
| 245 |
def test_compute_metrics_uses_sales_revenue_net_concept():
|
|
|
|
| 1 |
import pytest
|
| 2 |
from unittest.mock import patch, MagicMock
|
| 3 |
+
from ingestion.edgar import (
|
| 4 |
+
get_cik,
|
| 5 |
+
compute_metrics_for_accn,
|
| 6 |
+
get_all_xbrl_facts,
|
| 7 |
+
_extract_business,
|
| 8 |
+
_extract_guidance,
|
| 9 |
+
_extract_mda,
|
| 10 |
+
_extract_risk_factors,
|
| 11 |
+
_extract_segments_geography,
|
| 12 |
+
_html_to_filing_text,
|
| 13 |
+
)
|
| 14 |
|
| 15 |
FAKE_TICKERS = {
|
| 16 |
"0": {"cik_str": 320193, "ticker": "AAPL", "title": "Apple Inc."}
|
|
|
|
| 250 |
assert result == ""
|
| 251 |
|
| 252 |
|
| 253 |
+
def test_extract_business_uses_item_1_and_stops_before_risks():
|
| 254 |
+
text = """Table of Contents
|
| 255 |
+
Item 1. Business
|
| 256 |
+
Item 1A. Risk Factors
|
| 257 |
+
|
| 258 |
+
Item 1. Business
|
| 259 |
+
We design accelerated computing platforms for data centers.
|
| 260 |
+
Our customers include cloud service providers.
|
| 261 |
+
|
| 262 |
+
Item 1A. Risk Factors
|
| 263 |
+
Export controls may affect sales.
|
| 264 |
+
"""
|
| 265 |
+
result = _extract_business(text, "10-K")
|
| 266 |
+
assert "accelerated computing platforms" in result
|
| 267 |
+
assert "Export controls" not in result
|
| 268 |
+
|
| 269 |
+
|
| 270 |
+
def test_extract_business_returns_empty_for_10q():
|
| 271 |
+
assert _extract_business("Item 1. Business\nText\nItem 1A.", "10-Q") == ""
|
| 272 |
+
|
| 273 |
+
|
| 274 |
+
def test_extract_segments_geography_preserves_disclosed_table_text():
|
| 275 |
+
text = """Item 8. Financial Statements and Supplementary Data
|
| 276 |
+
|
| 277 |
+
Note 12 - Segment Information
|
| 278 |
+
Compute | 80 | 70
|
| 279 |
+
Networking | 20 | 30
|
| 280 |
+
|
| 281 |
+
Geographic Information
|
| 282 |
+
United States | 60%
|
| 283 |
+
International | 40%
|
| 284 |
+
|
| 285 |
+
Item 9. Changes in and Disagreements with Accountants
|
| 286 |
+
"""
|
| 287 |
+
result = _extract_segments_geography(text, "10-K")
|
| 288 |
+
assert "Compute | 80 | 70" in result
|
| 289 |
+
assert "United States | 60%" in result
|
| 290 |
+
assert "Changes in and Disagreements" not in result
|
| 291 |
+
|
| 292 |
+
|
| 293 |
+
def test_html_to_filing_text_keeps_table_rows():
|
| 294 |
+
html = """
|
| 295 |
+
<html><body><h2>Geographic Information</h2>
|
| 296 |
+
<table><tr><th>Region</th><th>Revenue</th></tr>
|
| 297 |
+
<tr><td>Europe</td><td>25%</td></tr></table></body></html>
|
| 298 |
+
"""
|
| 299 |
+
result = _html_to_filing_text(html)
|
| 300 |
+
assert "Region | Revenue" in result
|
| 301 |
+
assert "Europe | 25%" in result
|
| 302 |
+
|
| 303 |
+
|
| 304 |
# ── XBRL concept expansion ───────────────────────────────────────────────────
|
| 305 |
|
| 306 |
def test_compute_metrics_uses_sales_revenue_net_concept():
|
tests/test_embedder.py
CHANGED
|
@@ -47,6 +47,36 @@ def test_embed_filing_skips_empty_sections(mock_vs):
|
|
| 47 |
assert mock_vs.add_chunks.call_count == 1
|
| 48 |
|
| 49 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 50 |
@patch("ingestion.embedder.vector_store")
|
| 51 |
def test_embed_filing_chunk_context_in_metadata(mock_vs):
|
| 52 |
embed_and_store_filing(
|
|
|
|
| 47 |
assert mock_vs.add_chunks.call_count == 1
|
| 48 |
|
| 49 |
|
| 50 |
+
@patch("ingestion.embedder.vector_store")
|
| 51 |
+
def test_embed_annual_filing_indexes_company_profile_sections(mock_vs):
|
| 52 |
+
embed_and_store_filing(
|
| 53 |
+
ticker="AAPL",
|
| 54 |
+
company_name="Apple Inc.",
|
| 55 |
+
mda_text="Revenue grew 5%.",
|
| 56 |
+
risk_text="Competition remains a risk.",
|
| 57 |
+
business_text="The company designs and sells devices and services.",
|
| 58 |
+
segments_geography_text="Net sales are reported by geography.",
|
| 59 |
+
filing_date="2024-11-01",
|
| 60 |
+
period="FY2024",
|
| 61 |
+
form_type="10-K",
|
| 62 |
+
)
|
| 63 |
+
|
| 64 |
+
assert mock_vs.add_chunks.call_count == 4
|
| 65 |
+
metadata = [call.args[2][0] for call in mock_vs.add_chunks.call_args_list]
|
| 66 |
+
assert [item["section"] for item in metadata] == [
|
| 67 |
+
"MD&A",
|
| 68 |
+
"Risk Factors",
|
| 69 |
+
"Business",
|
| 70 |
+
"Segments & Geography",
|
| 71 |
+
]
|
| 72 |
+
assert [item["chunk_id"] for item in metadata] == [
|
| 73 |
+
"mda:0",
|
| 74 |
+
"risk_factors:0",
|
| 75 |
+
"business:0",
|
| 76 |
+
"segments_geography:0",
|
| 77 |
+
]
|
| 78 |
+
|
| 79 |
+
|
| 80 |
@patch("ingestion.embedder.vector_store")
|
| 81 |
def test_embed_filing_chunk_context_in_metadata(mock_vs):
|
| 82 |
embed_and_store_filing(
|
tests/test_ingest.py
CHANGED
|
@@ -55,6 +55,8 @@ def test_10k_fetches_q4_transcript():
|
|
| 55 |
patch("ingest.prune_old_metrics"),
|
| 56 |
patch("ingest.init_db"),
|
| 57 |
patch("ingest.init_sections_db"),
|
|
|
|
|
|
|
| 58 |
patch("ingest.fetch_transcript", return_value="Q4 transcript text") as mock_transcript,
|
| 59 |
):
|
| 60 |
ingest("NVDA")
|
|
@@ -81,8 +83,62 @@ def test_10q_fetches_quarterly_transcript():
|
|
| 81 |
patch("ingest.prune_old_metrics"),
|
| 82 |
patch("ingest.init_db"),
|
| 83 |
patch("ingest.init_sections_db"),
|
|
|
|
|
|
|
| 84 |
patch("ingest.fetch_transcript", return_value="Q1 transcript text") as mock_transcript,
|
| 85 |
):
|
| 86 |
ingest("NVDA")
|
| 87 |
|
| 88 |
mock_transcript.assert_called_once_with("NVDA", "2025Q1")
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 55 |
patch("ingest.prune_old_metrics"),
|
| 56 |
patch("ingest.init_db"),
|
| 57 |
patch("ingest.init_sections_db"),
|
| 58 |
+
patch("ingest.get_all_metrics", return_value=[]),
|
| 59 |
+
patch("ingest.get_section", return_value=None),
|
| 60 |
patch("ingest.fetch_transcript", return_value="Q4 transcript text") as mock_transcript,
|
| 61 |
):
|
| 62 |
ingest("NVDA")
|
|
|
|
| 83 |
patch("ingest.prune_old_metrics"),
|
| 84 |
patch("ingest.init_db"),
|
| 85 |
patch("ingest.init_sections_db"),
|
| 86 |
+
patch("ingest.get_all_metrics", return_value=[]),
|
| 87 |
+
patch("ingest.get_section", return_value=None),
|
| 88 |
patch("ingest.fetch_transcript", return_value="Q1 transcript text") as mock_transcript,
|
| 89 |
):
|
| 90 |
ingest("NVDA")
|
| 91 |
|
| 92 |
mock_transcript.assert_called_once_with("NVDA", "2025Q1")
|
| 93 |
+
|
| 94 |
+
|
| 95 |
+
def test_delta_backfills_missing_company_profile_sections_without_refetching_transcript():
|
| 96 |
+
"""A previously ingested 10-K must be revisited when its profile sections are absent."""
|
| 97 |
+
from ingest import ingest
|
| 98 |
+
|
| 99 |
+
edgar_10k = _make_edgar("FY2025", "10-K")
|
| 100 |
+
edgar_10k.business_text = "NVIDIA designs accelerated computing platforms."
|
| 101 |
+
edgar_10k.segments_geography_text = "Revenue is disclosed by reportable segment and geography."
|
| 102 |
+
|
| 103 |
+
def stored_section(_ticker, _period, section):
|
| 104 |
+
return "cached transcript" if section == "transcript" else None
|
| 105 |
+
|
| 106 |
+
with (
|
| 107 |
+
patch("ingest.fetch_all_edgar_data", return_value=[edgar_10k]),
|
| 108 |
+
patch("ingest.fill_missing_metrics", side_effect=lambda x: x),
|
| 109 |
+
patch("ingest.get_all_metrics", return_value=[{"period": "FY2025"}]),
|
| 110 |
+
patch("ingest.get_section", side_effect=stored_section),
|
| 111 |
+
patch("ingest._extract_guidance", return_value=None),
|
| 112 |
+
patch("ingest.parse_guidance", return_value={}),
|
| 113 |
+
patch("ingest.upsert_metrics"),
|
| 114 |
+
patch("ingest.upsert_section") as mock_upsert_section,
|
| 115 |
+
patch("ingest.embed_and_store_filing") as mock_embed_filing,
|
| 116 |
+
patch("ingest.embed_and_store_transcript"),
|
| 117 |
+
patch("ingest.prune_old_metrics"),
|
| 118 |
+
patch("ingest.init_db"),
|
| 119 |
+
patch("ingest.init_sections_db"),
|
| 120 |
+
patch("ingest.fetch_transcript") as mock_fetch_transcript,
|
| 121 |
+
):
|
| 122 |
+
ingest("NVDA")
|
| 123 |
+
|
| 124 |
+
mock_fetch_transcript.assert_not_called()
|
| 125 |
+
mock_embed_filing.assert_called_once()
|
| 126 |
+
assert mock_embed_filing.call_args.kwargs["business_text"] == edgar_10k.business_text
|
| 127 |
+
assert (
|
| 128 |
+
mock_embed_filing.call_args.kwargs["segments_geography_text"]
|
| 129 |
+
== edgar_10k.segments_geography_text
|
| 130 |
+
)
|
| 131 |
+
assert call(
|
| 132 |
+
"NVDA",
|
| 133 |
+
"FY2025",
|
| 134 |
+
"10-K",
|
| 135 |
+
"business",
|
| 136 |
+
edgar_10k.business_text,
|
| 137 |
+
) in mock_upsert_section.call_args_list
|
| 138 |
+
assert call(
|
| 139 |
+
"NVDA",
|
| 140 |
+
"FY2025",
|
| 141 |
+
"10-K",
|
| 142 |
+
"segments_geography",
|
| 143 |
+
edgar_10k.segments_geography_text,
|
| 144 |
+
) in mock_upsert_section.call_args_list
|