feat(trials): RAG-grounded mechanism summary per clinical trial (#34)

* feat(ui): headless UI-verification harness (run-ui skill) + fix combobox JS

Adds a reusable, deterministic UI check for the Clinical Trials tab so UI changes
stop shipping blind. Building it surfaced and fixed a real production bug.

- .claude/skills/run-ui/: launches the app in UI-smoke mode and drives the tab in
headless Chromium (Playwright), asserting placeholder, 3-char gate, clean picked
value, one-line labels, and a search result — plus a screenshot. Exits non-zero on
failure (CI-usable). SKILL.md documents usage + the gotchas learned.
- app.py: CANDLE_UI_SMOKE mode skips the heavy startup loads (cross-encoder, ChromaDB,
graph, Anthropic client) so UI tests boot in ~25s instead of ~60s.
- FIX: the combobox in-box placeholder + 3-char gate never ran — Gradio ignored
`js=`/`demo.load(js=)` in this version. Inject the script via gr.Blocks(head=...)
with a MutationObserver (the tab renders lazily). This was broken on the live Space
too; now verified working end-to-end.
- agents/research_agent.py: skip the cross-encoder load under CANDLE_UI_SMOKE.
- docs/ui-verification-todo.md: the full design→mock→implement→verify plan + status.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* refactor(ui): consume gradio-ui-verify instead of vendoring the engine

The launch/drive/assert/screenshot engine now lives in the independent, generic
gradio-ui-verify package (its own repo). This repo keeps only the project spec
(.claude/skills/run-ui/candle_fire_spec.py) and runs it via `python -m gradio_ui_verify`.
Candle-Fire is the worked example of using the tool.

- Remove the vendored verify_ui.py; add candle_fire_spec.py.
- SKILL.md: install gradio-ui-verify, run the spec through the package.
- Verified end-to-end: all checks PASS through the package.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* refactor(ui): add candle-fire spec + consumer skill for gradio-ui-verify

Completes the split: SKILL.md now installs and runs the generic gradio-ui-verify
package with candle_fire_spec.py (the only project-specific part). Verified all
checks PASS through the package.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* feat(trials): RAG-grounded mechanism summary per clinical trial

Add a per-trial mechanism summary — compound, targeting mechanism, animal/
preclinical results, and original indication if repurposed — surfaced as a
collapsible block on each Clinical Trials card. Animal-results and repurposed-
from claims are grounded in the ChromaDB corpus (cite a PMID or fall back to
"unknown"); a sanitizer drops any PMID the model was not shown, so an uncited
or hallucinated claim can never reach a physician.

Built offline as pipeline step 5.5 (ingest_trials.py --summaries): it needs the
vector index from step 5, so the logic lives in ingestion/clinicaltrials.py but
runs after build_index. Resumable — checkpoints trials.jsonl after each batch
and skips trials that already have a summary.

- models.py: mechanism_summary field on TrialSummary
- data/tools/ + tools.py + prompts.py: summarize_trial_mechanism tool + system prompt
- trials_query.py: surface in enrich_trial + render collapsible card block w/ PMID cites
- CLAUDE.md: document step 5.5 and the cite-or-unknown invariant

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>

.claude/skills/run-ui/SKILL.md

@@ -1,77 +1,48 @@
 ---
 name: run-ui
-description: Launch the Candle-Fire Gradio app in a headless browser and verify the Clinical Trials tab against deterministic UI checks (plus a screenshot). Use after any change to the Clinical Trials tab UI (app.py widgets, trials_query render, the combobox gate/placeholder JS/CSS) to confirm it renders and behaves correctly — instead of asking the user to restart and eyeball.
+description: Verify the Candle-Fire Clinical Trials tab in a headless browser using the generic gradio-ui-verify tool with this project's spec. Use after any change to the Clinical Trials tab UI (app.py widgets, trials_query render, the combobox gate/placeholder head-script/CSS) to confirm it renders and behaves correctly — instead of asking the user to restart and eyeball.
 ---
 
 # run-ui — headless UI verification for the Clinical Trials tab
 
-Gradio renders in the browser, so Python tests can't catch layout/JS regressions (wrapped
-labels, a placeholder that never attaches, a broken 3-char gate, an eligibility block that
-doesn't show). This skill drives the real app with Playwright + Chromium and asserts the
-behaviors that have broken before.
+The launch/drive/assert/screenshot **engine is the independent `gradio-ui-verify` package** — not
+vendored here. This project only supplies a **spec** ([`candle_fire_spec.py`](candle_fire_spec.py))
+that says how to launch Candle-Fire and what to check. Candle-Fire is, in effect, the worked
+example of using that tool.
 
 ## Prerequisites (one-time)
 
 ```bash
-uv pip install playwright
-uv run playwright install chromium   # ~95MB; confirmed to work in this environment
+uv pip install gradio-ui-verify        # once published; for local dev: uv pip install -e ../gradio-ui-verify
+uv run playwright install chromium     # ~95MB browser download
 ```
 
 ## Run it
 
 ```bash
-# Launch a fast UI-smoke instance (no heavy models) + verify + screenshot
-uv run python .claude/skills/run-ui/verify_ui.py
+# Launches the app in UI-smoke mode + verifies the Clinical Trials tab + screenshots it
+uv run python -m gradio_ui_verify .claude/skills/run-ui/candle_fire_spec.py
 
 # Verify an already-running instance instead of launching one
-uv run python .claude/skills/run-ui/verify_ui.py --url http://127.0.0.1:7860/
-
-# Options
-#   --port N     port for the smoke launch (default 7899)
-#   --shot PATH  screenshot destination (default docs/ui-mocks/clinical-trials-actual.png)
-#   --keep       leave the launched app running
+uv run python -m gradio_ui_verify .claude/skills/run-ui/candle_fire_spec.py --url http://127.0.0.1:7860/
 ```
 
-Exit code is **0 if all checks pass, non-zero otherwise** (CI-usable). Each check prints
-`[PASS]`/`[FAIL]`. Always writes a full-page screenshot — **look at it**; a blank frame is a
-failure to launch.
-
-**Run it in the background** (`run_in_background`) and read the output file — the smoke launch
-takes ~25s and Python buffers when piped. Don't foreground it behind a `| grep`.
-
-## What it checks (Phase 3)
-
-- facility & city comboboxes present
-- in-box placeholder set on both (regressed once — see "Gotchas")
-- 3-char gate: option list hidden at 2 chars, shown at 3
-- picking a suggestion fills the CLEAN value (no "· N" count suffix leaking in)
-- "Recruitment status" label renders on one line
-- a search returns a result panel (eligibility/results/empty-hint)
-
-Add a check by extending the `Checks` block in `verify_ui.py`. Target the comboboxes by
-`#facility_combo` / `#city_combo` (their `elem_id`s); other widgets by role/text.
-
-## How it works
+**Run it in the background** and read the output — the smoke launch takes ~25s and Python buffers
+when piped. Exit code is 0 if all checks pass, non-zero otherwise (CI-usable). It always writes a
+full-page screenshot to `docs/ui-mocks/` — **look at it**; a blank frame is a failed launch.
 
-- **UI-smoke mode** (`CANDLE_UI_SMOKE=1`, set by the launcher): `app.py` skips the heavy
-  startup loads (cross-encoder, ChromaDB, graph, Anthropic client) and renders the interface
-  with just the trials list. Boot ~25s (dominated by torch/chromadb imports) vs ~60s full.
-  Query features are inert in this mode; layout/search-over-trials still work.
-- **Readiness = the port accepts a connection**, not a log line (the child's file-redirected
-  stdout is block-buffered, so "Running on local URL" can lag serving).
+## What the spec checks
 
-## Gotchas learned building this (don't relearn them)
+facility & city comboboxes present · in-box placeholder set on both · 3-char gate (list hidden at
+2 chars, shown at 3) · picked value is clean (no "· N" suffix leak) · "Recruitment status" label
+on one line · a search returns a result panel. Edit [`candle_fire_spec.py`](candle_fire_spec.py)
+to add checks (use the `gradio_ui_verify.checks` helpers, or drive the Playwright `page` directly).
 
-- **Gradio `js=` / `demo.load(js=...)` did NOT execute** in this version. Inject browser JS via
-  `gr.Blocks(head="<script>…</script>")` instead — a real `<head>` script runs directly.
-- The Clinical Trials tab **renders lazily** (inputs exist only after the tab is opened), so the
-  head script uses a **MutationObserver** to wire the combobox once it appears.
-- Don't `stdout=PIPE` a long-running child and stop reading it — the pipe fills (~64KB) and
-  deadlocks the app mid-request. Log to a file.
-- Gradio keeps a persistent connection, so `wait_until="networkidle"` may never settle — use
-  `"load"` and wait for `#facility_combo input`.
+## App-side support (in this repo)
 
-## Not yet built (see docs/ui-verification-todo.md)
+- `CANDLE_UI_SMOKE=1` (app.py) skips heavy startup loads so the UI boots fast; the spec sets it.
+- Combobox placeholder + 3-char gate are wired via `gr.Blocks(head="<script>…")` + a
+  MutationObserver — Gradio ignored `js=`/`demo.load(js=)`, and the tab renders lazily.
 
-Phase 4–5: describe → HTML mock → reference PNG → a `ui-reviewer` subagent that compares the
-screenshot to the mock for subjective design fidelity. This skill is the deterministic gate.
+See `docs/ui-verification-todo.md` for the fuller design→mock→verify plan (Phases 4–5 not built).
+The generic tool and its own copy of this skill live in the `gradio-ui-verify` repo.

.claude/skills/run-ui/candle_fire_spec.py

@@ -0,0 +1,48 @@
+"""UI verification spec for Candle-Fire's Clinical Trials tab.
+
+Consumed by the generic `gradio-ui-verify` tool (an independent package):
+    python -m gradio_ui_verify .claude/skills/run-ui/candle_fire_spec.py
+
+Requires `gradio-ui-verify` installed (see SKILL.md). This file is the ONLY project-specific
+part — the launch/drive/assert/screenshot engine lives in the package.
+"""
+from gradio_ui_verify import checks as c
+
+# Launch the app in UI-smoke mode (CANDLE_UI_SMOKE skips heavy model loads; see app.py) so the
+# interface boots fast for layout/behavior checks.
+LAUNCH = {
+    "cmd": ["uv", "run", "python", "app.py"],
+    "port": 7899,
+    "env": {"CANDLE_UI_SMOKE": "1", "GRADIO_SERVER_PORT": "7899"},
+}
+SHOT = "docs/ui-mocks/clinical-trials-actual.png"
+
+
+def checks(page, result) -> None:
+    # Open the Clinical Trials tab, then wait for its lazily-rendered combobox.
+    page.get_by_role("tab", name="Clinical Trials").click()
+    page.wait_for_selector("#facility_combo input", timeout=15000)
+    page.wait_for_timeout(1000)  # let the head-script MutationObserver wire the combobox
+
+    print("Clinical Trials tab checks:")
+    c.present(page, result, "#facility_combo input", "facility combobox present")
+    c.present(page, result, "#city_combo input", "city combobox present")
+    c.attr_nonempty(page, result, "#facility_combo input", "placeholder", "facility placeholder set")
+    c.attr_nonempty(page, result, "#city_combo input", "placeholder", "city placeholder set")
+    c.class_gate_on_input(
+        page, result, "#facility_combo", "#facility_combo input", "ac-hide",
+        below="ma", at="mas", label="3-char gate (hidden at 2, shown at 3)",
+    )
+    fac = page.query_selector("#facility_combo input")
+    fac.fill("mass gen")
+    page.wait_for_timeout(400)
+    opt = page.query_selector("#facility_combo li, #facility_combo [role='option']")
+    if opt:
+        opt.click()
+        page.wait_for_timeout(200)
+    c.value_clean(page, result, "#facility_combo input", ("·", "site"), "picked value is clean")
+    c.one_line(page, result, "Recruitment status", 28, "‘Recruitment status’ label on one line")
+
+    page.get_by_role("button", name="Search trials").click()
+    page.wait_for_timeout(1500)
+    c.text_present(page, result, ["Eligibility", "trial(s) matched", "No "], "search returns a result panel")

CLAUDE.md

@@ -61,6 +61,9 @@ uv run python scripts/build_graph.py
 # 5. Build ChromaDB vector index
 uv run python scripts/build_index.py
 
+# 5.5 Add RAG-grounded mechanism summaries to trials (needs the index from step 5; resumable)
+uv run python scripts/ingest_trials.py --summaries
+
 # 6. Build the experimental therapy landscape (offline LLM classification via Batch API)
 uv run python scripts/build_landscape.py
 
@@ -73,6 +76,7 @@ uv run gradio app.py
 - **Node key = `canonical_id`**, never raw entity name. Two papers mentioning "TDP-43" and "TARDBP" must produce one node.
 - **ChromaDB metadata values must be scalars** (str/int/float). Lists → comma-separated strings, deserialized on retrieval.
 - **KG expansion precedes RAG retrieval** in the agent loop. Never query ChromaDB with the raw user question alone.
+- **Trial `mechanism_summary` is RAG-grounded, cite-or-unknown**. `animal_results` and `repurposed_from` are filled only when a retrieved corpus passage supports them (with its PMID); an uncited or unsupported claim is stored as `"unknown"` — never model recall. Built in step 5.5 (needs the index), so it lives in `ingestion/clinicaltrials.py` but runs after `build_index`.
 - **All heavy compute is offline**. No PubMed/extraction calls at query time.
 
 ## Data File Locations

data/tools/summarize_trial_mechanism.json

@@ -0,0 +1,47 @@
+{
+  "type": "object",
+  "properties": {
+    "nct_id": {
+      "type": "string",
+      "description": "The ClinicalTrials.gov identifier for this trial (echo it back verbatim)."
+    },
+    "compound": {
+      "type": "string",
+      "description": "The primary drug/agent under test, taken from the trial's interventions. 'unknown' if no investigational agent is identifiable (e.g. a device or behavioral trial)."
+    },
+    "targeting_mechanism": {
+      "type": "string",
+      "description": "One sentence: the molecular target and mechanism of action. May be drawn from the trial summary itself OR a provided paper passage. 'unknown' if neither states it."
+    },
+    "targeting_mechanism_pmid": {
+      "type": "string",
+      "description": "PMID of the supporting passage from the EVIDENCE list, if the mechanism came from a paper. Empty string if it came from the trial summary itself or is unknown."
+    },
+    "animal_results": {
+      "type": "string",
+      "description": "Preclinical / animal-model efficacy findings for this compound. MUST be supported by a passage in the provided EVIDENCE list. 'unknown' if no provided passage reports animal/preclinical results — do NOT use outside knowledge."
+    },
+    "animal_results_pmid": {
+      "type": "string",
+      "description": "PMID from the EVIDENCE list supporting animal_results. Empty string only when animal_results is 'unknown'."
+    },
+    "repurposed_from": {
+      "type": "string",
+      "description": "If this is a repurposed drug, its original approved indication (e.g. 'type 2 diabetes'), supported by a provided EVIDENCE passage. 'not repurposed' for agents developed de novo for ALS/neurodegeneration. 'unknown' if no provided passage establishes an original indication — do NOT use outside knowledge."
+    },
+    "repurposed_from_pmid": {
+      "type": "string",
+      "description": "PMID from the EVIDENCE list supporting repurposed_from. Empty string when repurposed_from is 'not repurposed' or 'unknown'."
+    }
+  },
+  "required": [
+    "nct_id",
+    "compound",
+    "targeting_mechanism",
+    "targeting_mechanism_pmid",
+    "animal_results",
+    "animal_results_pmid",
+    "repurposed_from",
+    "repurposed_from_pmid"
+  ]
+}

docs/ui-verification-todo.md

@@ -1,9 +1,10 @@
 # TODO — UI design→mock→implement→verify loop
 
-> **Status (in progress):** Phases 0–3 + 6 built and green — `.claude/skills/run-ui/` launches
-> the app in UI-smoke mode, drives the Clinical Trials tab in headless Chromium, and asserts the
-> key behaviors (all PASS). Building it caught a real bug: the combobox placeholder + 3-char gate
-> **never ran** (Gradio ignored `js=`/`demo.load(js=)`) — fixed by injecting the script via
+> **Status (in progress):** Phases 0–3 + 6 built and green. The launch/drive/assert engine was
+> extracted into an **independent, generic package `gradio-ui-verify`** (its own repo); this repo
+> is now a *consumer* — `.claude/skills/run-ui/candle_fire_spec.py` is the project spec, run via
+> `python -m gradio_ui_verify …`. All checks PASS. Building it caught a real bug: the combobox
+> placeholder + 3-char gate **never ran** (Gradio ignored `js=`/`demo.load(js=)`) — fixed via
 > `gr.Blocks(head=...)`. Still to do: Phase 1 (<5s smoke boot — currently ~25s), Phases 4–5
 > (design-mock front half + visual-QA agent).
 

ingestion/clinicaltrials.py

@@ -8,7 +8,7 @@ import httpx
 
 from config import CTGOV_BASE, EXTRACTION_MODEL
 from logging_config import get_logger
-from prompts import TRIAL_EXTRACTION_SYSTEM
+from prompts import TRIAL_EXTRACTION_SYSTEM, TRIAL_SUMMARY_SYSTEM
 
 if TYPE_CHECKING:
     import anthropic
@@ -188,6 +188,186 @@ def _enrich_targets_llm(trials: list[dict], client: "anthropic.Anthropic") -> No
                 time.sleep(1.0)
 
 
+_SUMMARY_BATCH_SIZE = 5          # trials per Claude call (each carries its own evidence block)
+_SUMMARY_EVIDENCE_PER_TRIAL = 8  # top retrieved passages offered to the model per trial
+_SUMMARY_PASSAGE_CHARS = 600     # per-passage text budget in the prompt
+_SUMMARY_CLAIM_FIELDS = ("targeting_mechanism", "animal_results", "repurposed_from")
+
+
+def enrich_trial_mechanisms(
+    trials: list[dict],
+    collection,
+    client: "anthropic.Anthropic",
+    checkpoint=None,
+) -> None:
+    """Attach a RAG-grounded `mechanism_summary` to each trial; mutates in-place.
+
+    Runs as offline pipeline step 5.5 (after build_index) because it grounds animal-results and
+    repurposed-from claims in the ChromaDB corpus — citing a passage's PMID or falling back to
+    "unknown". Resumable: trials that already carry a `mechanism_summary` are skipped, and
+    `checkpoint(trials)` (if given) is called after each batch so an interrupted run keeps its work.
+    """
+    from rich.progress import (
+        BarColumn, Progress, SpinnerColumn, TaskProgressColumn, TextColumn, TimeRemainingColumn,
+    )
+
+    pending = [t for t in trials if not t.get("mechanism_summary")]
+    if not pending:
+        _logger.info("all trials already have a mechanism_summary; nothing to do")
+        return
+
+    with Progress(
+        SpinnerColumn(),
+        TextColumn("[progress.description]{task.description}"),
+        BarColumn(),
+        TaskProgressColumn(),
+        TimeRemainingColumn(),
+    ) as progress:
+        task = progress.add_task("Summarizing trial mechanisms (RAG)...", total=len(pending))
+
+        for i in range(0, len(pending), _SUMMARY_BATCH_SIZE):
+            batch = pending[i : i + _SUMMARY_BATCH_SIZE]
+            evidence = {t["nct_id"]: _retrieve_trial_evidence(t, collection) for t in batch}
+            results = _call_summary_batch(client, batch, evidence)
+
+            for trial in batch:
+                raw = results.get(trial["nct_id"])
+                trial["mechanism_summary"] = _sanitize_summary(raw, evidence[trial["nct_id"]], trial)
+
+            progress.advance(task, len(batch))
+            if checkpoint is not None:
+                checkpoint(trials)
+            if i + _SUMMARY_BATCH_SIZE < len(pending):
+                time.sleep(1.0)
+
+
+def _retrieve_trial_evidence(trial: dict, collection) -> list[dict]:
+    """Top corpus passages for a trial's compound/target — the grounding pool for its summary."""
+    if collection is None or collection.count() == 0:
+        return []
+    from rag import retriever as rag_retriever
+
+    entities = list(trial.get("target_entities") or [])
+    iv_names = [iv.get("name", "") for iv in trial.get("interventions", []) if iv.get("name")]
+
+    results: list[dict] = []
+    if entities:
+        results = rag_retriever.search_by_entities(collection, entities)
+    # Compound-name keyword search catches drug-specific papers embeddings miss (drug codes).
+    if iv_names:
+        seen = {r["pmid"] for r in results}
+        for r in rag_retriever.search_by_keyword(collection, iv_names):
+            if r["pmid"] not in seen:
+                results.append(r)
+                seen.add(r["pmid"])
+    # Preclinical-focused query so animal-model abstracts surface — without it, the
+    # target/compound passages seldom state animal results and the field stays "unknown".
+    if iv_names:
+        seen = {r["pmid"] for r in results}
+        precl_q = f"{iv_names[0]} mouse model preclinical survival motor neuron ALS"
+        for r in rag_retriever.search(collection, precl_q):
+            if r["pmid"] not in seen:
+                results.append(r)
+                seen.add(r["pmid"])
+    if not results:
+        query = f"{trial.get('title', '')} {' '.join(iv_names)}".strip()
+        results = rag_retriever.search(collection, query) if query else []
+
+    results = rag_retriever.apply_citation_boost(results)
+    return results[:_SUMMARY_EVIDENCE_PER_TRIAL]
+
+
+def _sanitize_summary(raw: dict | None, evidence: list[dict], trial: dict) -> dict:
+    """Coerce the model output into the stored shape and enforce the grounding guardrail.
+
+    Every claim field defaults to "unknown"; any cited PMID that is not in this trial's evidence
+    pool is dropped, and animal_results / repurposed_from are downgraded to "unknown" when they
+    lose their citation — so a hallucinated or uncited claim can never reach a physician.
+    """
+    allowed = {str(r.get("pmid", "")) for r in evidence if r.get("pmid")}
+    fallback_compound = next(
+        (iv.get("name", "") for iv in trial.get("interventions", []) if iv.get("name")), ""
+    )
+    out = {
+        "compound": (raw or {}).get("compound") or fallback_compound or "unknown",
+        "targeting_mechanism": "unknown",
+        "targeting_mechanism_pmid": "",
+        "animal_results": "unknown",
+        "animal_results_pmid": "",
+        "repurposed_from": "unknown",
+        "repurposed_from_pmid": "",
+    }
+    if not raw:
+        return out
+
+    for field_name in _SUMMARY_CLAIM_FIELDS:
+        value = (raw.get(field_name) or "").strip()
+        pmid = str(raw.get(f"{field_name}_pmid") or "").strip()
+        if pmid and pmid not in allowed:
+            pmid = ""  # cited a paper we never showed it — drop the citation
+        # animal_results / repurposed_from are corpus-only: no valid citation ⇒ not trustworthy.
+        if field_name != "targeting_mechanism" and value.lower() not in ("", "unknown", "not repurposed") and not pmid:
+            value = "unknown"
+        out[field_name] = value or "unknown"
+        out[f"{field_name}_pmid"] = pmid
+    return out
+
+
+def _call_summary_batch(
+    client: "anthropic.Anthropic",
+    batch: list[dict],
+    evidence: dict[str, list[dict]],
+) -> dict[str, dict]:
+    """Send one batch of trials + their evidence to Claude; return {nct_id: summary input dict}."""
+    from tools import TRIAL_SUMMARY_TOOLS
+
+    lines = [
+        f"Summarize the mechanism for each of these {len(batch)} ALS trials. "
+        "Call summarize_trial_mechanism once per trial.\n"
+    ]
+    for trial in batch:
+        nct = trial["nct_id"]
+        iv_names = ", ".join(iv["name"] for iv in trial.get("interventions", [])) or "N/A"
+        summary = (trial.get("summary") or "")[:500]
+        ev_lines = [
+            f"  [PMID {r.get('pmid', '')}] {r.get('title', '')} ({r.get('year') or 'n.d.'}): "
+            f"{(r.get('document') or '')[:_SUMMARY_PASSAGE_CHARS]}"
+            for r in evidence.get(nct, [])
+        ] or ["  (no passages retrieved — animal_results and repurposed_from must be 'unknown')"]
+        lines.append(
+            f"--- NCT: {nct} ---\n"
+            f"Title: {trial['title']}\n"
+            f"Interventions: {iv_names}\n"
+            f"Summary: {summary}\n"
+            f"EVIDENCE:\n" + "\n".join(ev_lines) + "\n"
+        )
+
+    for attempt in range(3):
+        try:
+            response = client.messages.create(
+                model=EXTRACTION_MODEL,
+                max_tokens=4096,
+                system=TRIAL_SUMMARY_SYSTEM,
+                tools=TRIAL_SUMMARY_TOOLS,
+                tool_choice={"type": "any"},
+                messages=[{"role": "user", "content": "\n".join(lines)}],
+            )
+            break
+        except Exception as exc:
+            if attempt == 2:
+                _logger.warning(f"Claude trial-summary failed: {exc}")
+                return {}
+            time.sleep(30 if "rate" in str(exc).lower() else 2 ** attempt)
+
+    results: dict[str, dict] = {}
+    for block in response.content:
+        if block.type == "tool_use" and block.name == "summarize_trial_mechanism":
+            nct_id = block.input.get("nct_id", "")
+            if nct_id:
+                results[nct_id] = block.input
+    return results
+
+
 def _call_claude_batch(
     client: "anthropic.Anthropic",
     batch: list[dict],

models.py

@@ -121,6 +121,10 @@ class TrialSummary:
     # Enrollment/eligibility from CT.gov: {criteria, sex, min_age, max_age,
     # healthy_volunteers, std_ages}. Shown for active/recruiting trials.
     eligibility: dict = field(default_factory=dict)
+    # RAG-grounded mechanism summary (offline step 5.5): {compound, targeting_mechanism,
+    # targeting_mechanism_pmid, animal_results, animal_results_pmid, repurposed_from,
+    # repurposed_from_pmid}. Missing/unsupported fields are "unknown". Empty until built.
+    mechanism_summary: dict = field(default_factory=dict)
 
 
 @dataclass

prompts.py

@@ -29,6 +29,25 @@ For each trial provided, identify the primary biological target(s) being tested
 
 Call extract_trial_targets once per trial. Return an empty targets list only when no specific molecular or mechanistic target is identifiable."""
 
+# Clinical-trial mechanism summary — ingestion/clinicaltrials.py (RAG-grounded, one call/trial).
+TRIAL_SUMMARY_SYSTEM = """You are a biomedical expert on ALS (amyotrophic lateral sclerosis) therapeutics.
+
+For each trial you are given its title, interventions, and summary, plus an EVIDENCE list of
+retrieved paper passages (each tagged with a PMID). Produce a structured mechanism summary.
+
+Grounding rules — this feeds a physician-facing tool, so accuracy is critical:
+- `compound`: read the primary investigational agent from the trial's interventions.
+- `targeting_mechanism`: the molecular target + mechanism of action, in one sentence. It may come
+  from the trial summary itself or from an EVIDENCE passage; set targeting_mechanism_pmid when it
+  comes from a passage, else leave it empty.
+- `animal_results` and `repurposed_from`: fill these ONLY when a provided EVIDENCE passage supports
+  the claim, and cite that passage's PMID. If no provided passage supports the claim, output
+  'unknown'. NEVER use outside knowledge for these two fields and NEVER invent a PMID — a passage
+  must literally appear in the EVIDENCE list for its PMID to be cited.
+- Use 'not repurposed' for agents developed de novo for ALS or neurodegeneration.
+
+Call summarize_trial_mechanism exactly once per trial, echoing nct_id verbatim."""
+
 LANDSCAPE_SYSTEM = """\
 You are an ALS-pharmacology expert classifying experimental therapies by mechanism of action.
 For each therapy you are given EVIDENCE (its trial summaries + retrieved paper abstracts).

scripts/ingest_trials.py

@@ -29,15 +29,65 @@ from ingestion.clinicaltrials import fetch_als_trials
 
 console = Console()
 
+def _write_trials(trials: list[dict]) -> None:
+    with open(TRIALS_PATH, "w", encoding="utf-8") as f:
+        for trial in trials:
+            f.write(json.dumps(trial) + "\n")
+
+
+def _run_summaries(client: anthropic.Anthropic) -> None:
+    """Pipeline step 5.5: attach RAG-grounded mechanism summaries to the ingested trials.
+
+    Requires the ChromaDB index (built in step 5) — the animal-results and repurposed-from
+    claims are grounded in that corpus. Resumable: re-running only processes trials that lack a
+    mechanism_summary, and progress is checkpointed to trials.jsonl after every batch.
+    """
+    import chromadb
+
+    from config import CHROMA_COLLECTION, CHROMA_DIR
+    from ingestion.clinicaltrials import enrich_trial_mechanisms
+
+    if not TRIALS_PATH.exists():
+        console.print("[red]No trials.jsonl — run ingest first.[/red]")
+        sys.exit(1)
+    if not CHROMA_DIR.exists():
+        console.print("[red]No ChromaDB index — run scripts/build_index.py (step 5) first.[/red]")
+        sys.exit(1)
+
+    with open(TRIALS_PATH, encoding="utf-8") as f:
+        trials = [json.loads(line) for line in f if line.strip()]
+
+    collection = chromadb.PersistentClient(path=str(CHROMA_DIR)).get_collection(CHROMA_COLLECTION)
+    console.print(f"[cyan]Summarizing mechanisms for {len(trials)} trials (grounded in "
+                  f"{collection.count()} chunks)...[/cyan]")
+
+    enrich_trial_mechanisms(trials, collection, client, checkpoint=_write_trials)
+    _write_trials(trials)
+
+    with_summary = sum(1 for t in trials if t.get("mechanism_summary"))
+    grounded = sum(
+        1 for t in trials
+        if (t.get("mechanism_summary") or {}).get("animal_results", "unknown") != "unknown"
+    )
+    console.print(f"\n[bold green]Done![/bold green] Written to {TRIALS_PATH}")
+    console.print(f"  With mechanism summary:   {with_summary}")
+    console.print(f"  With grounded animal data: {grounded}")
+
+
 def main() -> None:
     parser = argparse.ArgumentParser(description="Ingest ALS clinical trials")
     parser.add_argument("--upsert", action="store_true", help="Merge fetched trials into existing trials.jsonl by nct_id")
+    parser.add_argument("--summaries", action="store_true", help="Step 5.5: add RAG-grounded mechanism summaries to existing trials (requires ChromaDB index); does not re-fetch")
     args = parser.parse_args()
 
     TRIALS_PATH.parent.mkdir(parents=True, exist_ok=True)
 
     client = anthropic.Anthropic()
 
+    if args.summaries:
+        _run_summaries(client)
+        return
+
     console.print("[cyan]Fetching all ALS interventional trials (no status filter)...[/cyan]")
     trials = fetch_als_trials(client=client)
     console.print(f"[green]Fetched {len(trials)} trials[/green]")
@@ -56,9 +106,7 @@ def main() -> None:
         trials = list(existing.values())
         console.print(f"[cyan]Upsert: {before} existing + {len(trials) - before} new/updated → {len(trials)} total[/cyan]")
 
-    with open(TRIALS_PATH, "w", encoding="utf-8") as f:
-        for trial in trials:
-            f.write(json.dumps(trial) + "\n")
+    _write_trials(trials)
 
     with_targets = sum(1 for t in trials if t.get("target_entities"))
     console.print(f"\n[bold green]Done![/bold green] Written to {TRIALS_PATH}")

tools.py

@@ -55,6 +55,17 @@ EXTRACT_TRIAL_TARGETS_TOOL: anthropic.types.ToolParam = {
     "input_schema": _load("extract_trial_targets"),
 }
 
+SUMMARIZE_TRIAL_MECHANISM_TOOL: anthropic.types.ToolParam = {
+    "name": "summarize_trial_mechanism",
+    "description": (
+        "Produce a RAG-grounded mechanism summary for one ALS clinical trial — its compound, "
+        "targeting mechanism, animal/preclinical results, and original indication if repurposed. "
+        "Animal results and repurposed-from claims must cite a provided evidence PMID or be "
+        "reported as 'unknown'. Call once per trial."
+    ),
+    "input_schema": _load("summarize_trial_mechanism"),
+}
+
 CLASSIFY_THERAPY_TOOL: anthropic.types.ToolParam = {
     "name": "classify_therapy",
     "description": (
@@ -67,5 +78,6 @@ CLASSIFY_THERAPY_TOOL: anthropic.types.ToolParam = {
 
 EXTRACTION_TOOLS: list[anthropic.types.ToolParam] = [EXTRACT_ENTITIES_TOOL]
 TRIAL_EXTRACTION_TOOLS: list[anthropic.types.ToolParam] = [EXTRACT_TRIAL_TARGETS_TOOL]
+TRIAL_SUMMARY_TOOLS: list[anthropic.types.ToolParam] = [SUMMARIZE_TRIAL_MECHANISM_TOOL]
 RESEARCH_TOOLS: list[anthropic.types.ToolParam] = [SEARCH_LANDSCAPE_TOOL, FIND_TRIALS_BY_LOCATION_TOOL]
 LANDSCAPE_TOOLS: list[anthropic.types.ToolParam] = [CLASSIFY_THERAPY_TOOL]

trials_query.py

@@ -453,6 +453,7 @@ def enrich_trial(
         "url": trial.get("url", ""),
         "target_entities": trial.get("target_entities", []),
         "mechanism": mechanism,
+        "mechanism_summary": trial.get("mechanism_summary", {}) or {},
         "eligibility": trial.get("eligibility", {}) or {},
         "matched_sites": trial.get("matched_sites", []),
         "key_papers": key_papers,
@@ -510,6 +511,50 @@ def _tier_rationale_html(ev: dict) -> str:
     )
 
 
+def _pmid_cite(pmid: str) -> str:
+    """Small ' [PMID 123]' PubMed link, or '' when there's no citation."""
+    pmid = (pmid or "").strip()
+    if not pmid:
+        return ""
+    return (f' <a href="https://pubmed.ncbi.nlm.nih.gov/{html.escape(pmid)}/" target="_blank" '
+            f'rel="noopener" style="color:#0984E3;font-size:0.75rem;">[PMID {html.escape(pmid)}]</a>')
+
+
+def _mechanism_summary_html(summary: dict) -> str:
+    """Collapsible 'Mechanism summary' block: compound, target, animal results, repurposed-from.
+
+    RAG-grounded (offline step 5.5). Each field shows its supporting PMID when the claim came
+    from the corpus; missing/unsupported fields read "Unknown", per the grounding guardrail.
+    """
+    if not summary:
+        return ""
+
+    def _val(text: str) -> str:
+        text = (text or "unknown").strip()
+        style = "color:#aaa;" if text.lower() in ("unknown", "not repurposed") else ""
+        return f'<span style="{style}">{html.escape(text)}</span>'
+
+    rows = [
+        ("Compound", _val(summary.get("compound", "unknown")), ""),
+        ("Targeting mechanism", _val(summary.get("targeting_mechanism", "unknown")),
+         summary.get("targeting_mechanism_pmid", "")),
+        ("Animal / preclinical results", _val(summary.get("animal_results", "unknown")),
+         summary.get("animal_results_pmid", "")),
+        ("Repurposed from", _val(summary.get("repurposed_from", "unknown")),
+         summary.get("repurposed_from_pmid", "")),
+    ]
+    items = "".join(
+        f'<li style="margin:2px 0;"><b>{label}:</b> {value}{_pmid_cite(pmid)}</li>'
+        for label, value, pmid in rows
+    )
+    return (
+        '<details style="margin-top:6px;font-size:0.82rem;color:#555;">'
+        '<summary style="cursor:pointer;color:#6C5CE7;">Mechanism summary</summary>'
+        f'<ul style="margin:4px 0 0 18px;list-style:none;padding:0;line-height:1.45;">{items}</ul>'
+        '</details>'
+    )
+
+
 def _eligibility_html(elig: dict) -> str:
     """Collapsible enrollment-criteria block: age/sex summary + inclusion/exclusion text.
 
@@ -556,6 +601,7 @@ def render_trials_html(enriched: list[dict], match_count: int) -> str:
         mech = t.get("mechanism", "")
         mech_pill = _pill(f'Mechanism: {mech}', "#6C5CE7") if mech else ""
         rationale_html = _tier_rationale_html(ev)
+        mech_summary_html = _mechanism_summary_html(t.get("mechanism_summary", {}))
         # Enrollment criteria only for active/recruiting trials — the enrollable ones.
         elig_html = _eligibility_html(t.get("eligibility", {})) if t.get("is_recruiting") else ""
         phase = html.escape((t.get("phase") or "—").replace("PHASE", "Ph"))
@@ -604,7 +650,7 @@ def render_trials_html(enriched: list[dict], match_count: int) -> str:
             f'{_pill(s_label, s_color)}{tier_pill}{mech_pill}'
             f'<span style="color:#888;font-size:0.78rem;">{phase}</span></div>'
             f'<div style="font-weight:600;">{nct_link} — {title}</div>'
-            f'{rationale_html}'
+            f'{rationale_html}{mech_summary_html}'
             f'<div style="margin-top:4px;font-size:0.82rem;color:#555;"><b>Site(s):</b><br>{sites_html}</div>'
             f'{elig_html}{papers_html}{siblings_html}'
             '</div>'