KevinIsInCoding Claude Opus 4.8 commited on
Commit
f77eaaf
Β·
unverified Β·
1 Parent(s): 3e62ca2

feat(trials): facility & geography search + targeted mechanism & explainable evidence tier (#22)

Browse files

* feat(trials): facility & geography trial search with evidence enrichment

Physicians can now find ALS trials by facility ("trials at Mass General
Hospital") or geography ("trials in NY"), with each result enriched by
surrounding research evidence.

- ingestion: _flatten_trial() parses contactsLocationsModule so each trial
persists a locations list (facility/city/state/country/status/lat/lon) plus
central contact phone/email; TrialSummary gains matching fields.
- trials_query.py (new): search_trials_by_location() (state abbrev→name,
token-substring facility match, recruiting filter) and enrich_trial()
(key papers via the RAG retriever + citation boost, sibling trials for the
same compound, heuristic Strong/Moderate/Emerging evidence tier), plus an
HTML renderer.
- agent: new find_trials_by_location tool wired into RESEARCH_TOOLS and
dispatched in research_agent; prompt guidance added for when to use it.
- app: dedicated "Clinical Trials" tab (facility/state/city/status inputs β†’
enriched cards); _ensure_data() also fetches trials.jsonl for the Space.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* feat(trials): targeted mechanism + explainable evidence tier

Reverse-index each trial's dominant mechanism class from the offline
landscape (no LLM at query time) and surface it as a "Mechanism" pill.
Feed the mechanism's KG hub node into _kg_paper_count so a compound whose
only extracted target is its own name still gets credit for its
mechanism's literature instead of scoring 0.

Make the evidence tier explain itself: _evidence_tier now returns a
per-factor rationale (points + physician-readable reason) and the KG
node that supplied the paper count, flagging when credit came from the
mechanism vs. the drug target. Rendered as a collapsible "Why <tier>?"
disclosure on each trial card; the agent is told to justify tiers from
evidence.rationale.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>

agents/research_agent.py CHANGED
@@ -26,6 +26,7 @@ from normalization.drug_vocab import build_drug_vocab, suggest_drug_term
26
  from prompts import SYNTHESIS_SYSTEM
27
  from rag import retriever as rag_retriever
28
  from tools import RESEARCH_TOOLS
 
29
 
30
  _logger = get_logger("agents.research_agent")
31
 
@@ -135,6 +136,9 @@ def stream_research_agent(
135
  if tool_call["name"] == "search_research_landscape":
136
  result = _handle_search(tool_call["input"], collection, trials, _graph)
137
  is_error = False
 
 
 
138
  else:
139
  result = {"error": f"Unknown tool: {tool_call['name']}"}
140
  is_error = True
@@ -453,3 +457,68 @@ def _handle_search(
453
  "kg_expansion_active": graph is not None,
454
  "grounding_note": grounding_note,
455
  }
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
26
  from prompts import SYNTHESIS_SYSTEM
27
  from rag import retriever as rag_retriever
28
  from tools import RESEARCH_TOOLS
29
+ import trials_query
30
 
31
  _logger = get_logger("agents.research_agent")
32
 
 
136
  if tool_call["name"] == "search_research_landscape":
137
  result = _handle_search(tool_call["input"], collection, trials, _graph)
138
  is_error = False
139
+ elif tool_call["name"] == "find_trials_by_location":
140
+ result = _handle_trials_by_location(tool_call["input"], trials, collection, _graph)
141
+ is_error = False
142
  else:
143
  result = {"error": f"Unknown tool: {tool_call['name']}"}
144
  is_error = True
 
457
  "kg_expansion_active": graph is not None,
458
  "grounding_note": grounding_note,
459
  }
460
+
461
+
462
+ # Cap enrichment work per location query β€” each trial triggers RAG retrieval, so bound it.
463
+ _LOCATION_ENRICH_CAP = 25
464
+
465
+
466
+ def _handle_trials_by_location(
467
+ tool_input: dict,
468
+ trials: list[dict],
469
+ collection: chromadb.Collection | None = None,
470
+ graph: nx.DiGraph | None = None,
471
+ ) -> dict:
472
+ """Filter trials by facility/geography and enrich each with research evidence."""
473
+ facility = tool_input.get("facility") or None
474
+ city = tool_input.get("city") or None
475
+ state = tool_input.get("state") or None
476
+ country = tool_input.get("country") or None
477
+ status = tool_input.get("status") or None
478
+
479
+ matches = trials_query.search_trials_by_location(
480
+ trials, facility=facility, city=city, state=state, country=country, status=status
481
+ )
482
+
483
+ enriched = [
484
+ trials_query.enrich_trial(t, collection, graph, trials)
485
+ for t in matches[:_LOCATION_ENRICH_CAP]
486
+ ]
487
+
488
+ where_parts = [p for p in (facility, city, state, country) if p]
489
+ where = ", ".join(where_parts) or "the requested location"
490
+
491
+ _logger.info(
492
+ "location trial search",
493
+ extra={"data": {
494
+ "facility": facility, "city": city, "state": state, "country": country,
495
+ "status": status, "matches": len(matches), "enriched": len(enriched),
496
+ }},
497
+ )
498
+
499
+ if not enriched:
500
+ grounding_note = (
501
+ f"No ALS clinical trials in this database have a study site matching {where}. "
502
+ "State that plainly; do not invent trials or sites."
503
+ )
504
+ else:
505
+ grounding_note = (
506
+ f"{len(matches)} ALS trial(s) have a study site matching {where} "
507
+ f"(showing {len(enriched)}). Report them grouped by recruiting status (recruiting "
508
+ "first). For each, give the NCT ID, phase, matched facility/city/state, the targeted "
509
+ "mechanism (when present), and evidence tier. When you state a tier, justify it briefly "
510
+ "from evidence.rationale (the scoring factors and points behind it, incl. which KG node "
511
+ "supplied the paper count). Cite key_papers by PMID only for claims "
512
+ "their titles support. Mention "
513
+ "sibling_trials as related trials for the same compound. Do not add trials not listed here."
514
+ )
515
+
516
+ return {
517
+ "location_query": {
518
+ "facility": facility, "city": city, "state": state,
519
+ "country": country, "status": status,
520
+ },
521
+ "match_count": len(matches),
522
+ "trials": enriched,
523
+ "grounding_note": grounding_note,
524
+ }
app.py CHANGED
@@ -56,7 +56,8 @@ def _ensure_data() -> None:
56
  """
57
  need_chroma = not (CHROMA_DIR / "chroma.sqlite3").exists()
58
  need_graph = not GRAPH_PICKLE_PATH.exists()
59
- if not need_chroma and not need_graph:
 
60
  return
61
  try:
62
  from huggingface_hub import hf_hub_download, snapshot_download
@@ -82,6 +83,14 @@ def _ensure_data() -> None:
82
  filename="graph/als_graph.pkl", local_dir=str(GRAPH_PICKLE_PATH.parent.parent),
83
  )
84
  _logger.info("Graph download complete")
 
 
 
 
 
 
 
 
85
  except Exception as e:
86
  _logger.warning(f"Failed to download data from HF dataset: {e}")
87
 
@@ -192,6 +201,32 @@ def _compound_change(label: str):
192
  )
193
 
194
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
195
  with gr.Blocks(title="Candle-Fire β€” ALS Research Intelligence") as demo:
196
 
197
  with gr.Tabs():
@@ -307,6 +342,39 @@ with gr.Blocks(title="Candle-Fire β€” ALS Research Intelligence") as demo:
307
 
308
  gr.HTML(_DISCLAIMER_MD)
309
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
310
  submit_kwargs = dict(
311
  fn=respond,
312
  inputs=[msg_box, chatbot],
 
56
  """
57
  need_chroma = not (CHROMA_DIR / "chroma.sqlite3").exists()
58
  need_graph = not GRAPH_PICKLE_PATH.exists()
59
+ need_trials = not TRIALS_PATH.exists()
60
+ if not need_chroma and not need_graph and not need_trials:
61
  return
62
  try:
63
  from huggingface_hub import hf_hub_download, snapshot_download
 
83
  filename="graph/als_graph.pkl", local_dir=str(GRAPH_PICKLE_PATH.parent.parent),
84
  )
85
  _logger.info("Graph download complete")
86
+ if need_trials:
87
+ _logger.info("Downloading trials from HF dataset...")
88
+ TRIALS_PATH.parent.mkdir(parents=True, exist_ok=True)
89
+ hf_hub_download(
90
+ repo_id=_HF_DATASET, repo_type="dataset",
91
+ filename="trials/trials.jsonl", local_dir=str(TRIALS_PATH.parent.parent),
92
+ )
93
+ _logger.info("Trials download complete")
94
  except Exception as e:
95
  _logger.warning(f"Failed to download data from HF dataset: {e}")
96
 
 
201
  )
202
 
203
 
204
+ # ── Clinical Trials tab (facility / geography search) ─────────────────────────
205
+
206
+ import trials_query
207
+
208
+ _TRIAL_ENRICH_CAP = 25
209
+ _US_STATES = ["All"] + sorted(set(trials_query._STATE_ABBREV.values()))
210
+
211
+
212
+ def _search_trials(facility: str, state: str, city: str, status: str) -> str:
213
+ facility = (facility or "").strip() or None
214
+ city = (city or "").strip() or None
215
+ state = None if (not state or state == "All") else state
216
+
217
+ if not any([facility, city, state]):
218
+ return '<div style="color:#888;padding:12px 0;">Enter a facility, state, or city to search.</div>'
219
+
220
+ matches = trials_query.search_trials_by_location(
221
+ _trials, facility=facility, city=city, state=state, status=status
222
+ )
223
+ enriched = [
224
+ trials_query.enrich_trial(t, _collection, _graph, _trials)
225
+ for t in matches[:_TRIAL_ENRICH_CAP]
226
+ ]
227
+ return trials_query.render_trials_html(enriched, len(matches))
228
+
229
+
230
  with gr.Blocks(title="Candle-Fire β€” ALS Research Intelligence") as demo:
231
 
232
  with gr.Tabs():
 
342
 
343
  gr.HTML(_DISCLAIMER_MD)
344
 
345
+ with gr.Tab("πŸ₯ Clinical Trials"):
346
+ with gr.Column(elem_classes="container"):
347
+ gr.Markdown(
348
+ "### πŸ₯ Find ALS Trials by Facility or Geography\n"
349
+ "Search the trial database by **facility** (e.g. *Mass General Hospital*) or "
350
+ "**location** (state / city). Each result is enriched with recruiting status, an "
351
+ "evidence-strength tier, key supporting papers, and related trials for the same compound."
352
+ )
353
+ with gr.Row():
354
+ facility_tb = gr.Textbox(
355
+ label="Facility / institution", scale=2,
356
+ placeholder="e.g. Mass General Hospital",
357
+ )
358
+ state_dd = gr.Dropdown(
359
+ choices=_US_STATES, value="All", label="State", scale=1,
360
+ )
361
+ city_tb = gr.Textbox(label="City", scale=1, placeholder="e.g. Boston")
362
+ trial_status_dd = gr.Dropdown(
363
+ choices=["All", "Recruiting", "Not recruiting"],
364
+ value="All", label="Recruitment status", scale=1,
365
+ )
366
+ search_btn = gr.Button("Search trials", variant="primary")
367
+ trial_results = gr.HTML(
368
+ '<div style="color:#888;padding:12px 0;">Enter a facility, state, or city to search.</div>'
369
+ )
370
+
371
+ _trial_search_inputs = [facility_tb, state_dd, city_tb, trial_status_dd]
372
+ search_btn.click(_search_trials, inputs=_trial_search_inputs, outputs=[trial_results])
373
+ facility_tb.submit(_search_trials, inputs=_trial_search_inputs, outputs=[trial_results])
374
+ city_tb.submit(_search_trials, inputs=_trial_search_inputs, outputs=[trial_results])
375
+
376
+ gr.HTML(_DISCLAIMER_MD)
377
+
378
  submit_kwargs = dict(
379
  fn=respond,
380
  inputs=[msg_box, chatbot],
data/tools/find_trials_by_location.json ADDED
@@ -0,0 +1,26 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "type": "object",
3
+ "properties": {
4
+ "facility": {
5
+ "type": "string",
6
+ "description": "Hospital, institution, or research-site name mentioned in the question (e.g. 'Mass General Hospital', 'Mayo Clinic'). Provide the name as written; partial names are matched."
7
+ },
8
+ "city": {
9
+ "type": "string",
10
+ "description": "City name mentioned in the question (e.g. 'Boston')."
11
+ },
12
+ "state": {
13
+ "type": "string",
14
+ "description": "US state mentioned in the question, as a two-letter abbreviation or full name (e.g. 'NY' or 'New York')."
15
+ },
16
+ "country": {
17
+ "type": "string",
18
+ "description": "Country mentioned in the question (e.g. 'United States'). Only for non-US or explicitly international queries."
19
+ },
20
+ "status": {
21
+ "type": "string",
22
+ "enum": ["Recruiting", "Not recruiting", "All"],
23
+ "description": "Recruitment filter. Use 'Recruiting' when the physician asks for open/enrolling trials; otherwise 'All'."
24
+ }
25
+ }
26
+ }
ingestion/clinicaltrials.py CHANGED
@@ -87,6 +87,7 @@ def _flatten_trial(study: dict) -> dict:
87
  sponsor_mod = proto.get("sponsorCollaboratorsModule", {})
88
  arms_mod = proto.get("armsInterventionsModule", {})
89
  status_mod = proto.get("statusModule", {})
 
90
 
91
  nct_id = id_mod.get("nctId", "")
92
  interventions = [
@@ -97,6 +98,26 @@ def _flatten_trial(study: dict) -> dict:
97
  study_type = design_mod.get("studyType", "")
98
  is_eap = study_type == "EXPANDED_ACCESS"
99
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
100
  return {
101
  "nct_id": nct_id,
102
  "title": id_mod.get("briefTitle", ""),
@@ -111,6 +132,9 @@ def _flatten_trial(study: dict) -> dict:
111
  "start_date": status_mod.get("startDateStruct", {}).get("date", ""),
112
  "url": f"https://clinicaltrials.gov/study/{nct_id}" if nct_id else "",
113
  "target_entities": [],
 
 
 
114
  }
115
 
116
 
 
87
  sponsor_mod = proto.get("sponsorCollaboratorsModule", {})
88
  arms_mod = proto.get("armsInterventionsModule", {})
89
  status_mod = proto.get("statusModule", {})
90
+ contacts_mod = proto.get("contactsLocationsModule", {})
91
 
92
  nct_id = id_mod.get("nctId", "")
93
  interventions = [
 
98
  study_type = design_mod.get("studyType", "")
99
  is_eap = study_type == "EXPANDED_ACCESS"
100
 
101
+ # Site locations β€” facility, address, per-site recruiting status, and geo point.
102
+ # Physicians search trials by facility ("Mass General") or place ("in NY"), so this
103
+ # address data must be persisted in the trials DB (offline; no fetch at query time).
104
+ locations = [
105
+ {
106
+ "facility": (loc.get("facility") or "").strip(),
107
+ "city": loc.get("city", ""),
108
+ "state": loc.get("state", ""),
109
+ "country": loc.get("country", ""),
110
+ "status": loc.get("status", ""), # per-site recruiting status
111
+ "lat": (loc.get("geoPoint") or {}).get("lat"),
112
+ "lon": (loc.get("geoPoint") or {}).get("lon"),
113
+ }
114
+ for loc in contacts_mod.get("locations", [])
115
+ ]
116
+
117
+ central_contacts = contacts_mod.get("centralContacts", [])
118
+ contact_phone = next((c.get("phone", "") for c in central_contacts if c.get("phone")), "")
119
+ contact_email = next((c.get("email", "") for c in central_contacts if c.get("email")), "")
120
+
121
  return {
122
  "nct_id": nct_id,
123
  "title": id_mod.get("briefTitle", ""),
 
132
  "start_date": status_mod.get("startDateStruct", {}).get("date", ""),
133
  "url": f"https://clinicaltrials.gov/study/{nct_id}" if nct_id else "",
134
  "target_entities": [],
135
+ "locations": locations,
136
+ "contact_phone": contact_phone,
137
+ "contact_email": contact_email,
138
  }
139
 
140
 
models.py CHANGED
@@ -113,6 +113,11 @@ class TrialSummary:
113
  sponsor: str = ""
114
  start_date: str = ""
115
  url: str = ""
 
 
 
 
 
116
 
117
 
118
  @dataclass
 
113
  sponsor: str = ""
114
  start_date: str = ""
115
  url: str = ""
116
+ # Site locations: {facility, city, state, country, status, lat, lon}. Enables
117
+ # facility/geography trial search (e.g. "trials at Mass General", "trials in NY").
118
+ locations: list[dict] = field(default_factory=list)
119
+ contact_phone: str = ""
120
+ contact_email: str = ""
121
 
122
 
123
  @dataclass
prompts.py CHANGED
@@ -98,6 +98,10 @@ Any relevant ALS clinical trials linked to the topic, with NCT ID and status.
98
  *Research synthesis tool. Always verify with primary sources and current clinical evidence.
99
  Not a substitute for clinical judgment.*
100
 
 
 
 
 
101
  Guidelines:
102
  - Begin directly with the structured response β€” no preamble, no "let me search", no narration of your reasoning steps
103
  - GROUNDING RULE (non-negotiable): Every factual claim must be directly supported by text in the retrieved excerpt for the PMID you cite. Before citing a PMID, verify the claim actually appears in that paper's excerpt. NEVER cite a PMID because it is topically adjacent β€” a citation asserts that specific paper supports that specific claim.
 
98
  *Research synthesis tool. Always verify with primary sources and current clinical evidence.
99
  Not a substitute for clinical judgment.*
100
 
101
+ Tool selection:
102
+ - Use `search_research_landscape` for questions about ALS biology, drug targets, mechanisms, or a compound's evidence β€” this is the default.
103
+ - Use `find_trials_by_location` when the question names a hospital/research site (e.g. "trials at Mass General Hospital") or a place (state/city/country, e.g. "trials in NY"). Pass the facility/city/state/country you identified and set `status` to "Recruiting" only if the physician asked for open/enrolling trials. When it returns trials, present them grouped by recruiting status (recruiting first); for each give the NCT ID (as a link when a url is provided), phase, the matched facility/city/state, the evidence-strength tier, key supporting papers (cite PMIDs only for claims their titles support), and any sibling_trials as related trials for the same compound.
104
+
105
  Guidelines:
106
  - Begin directly with the structured response β€” no preamble, no "let me search", no narration of your reasoning steps
107
  - GROUNDING RULE (non-negotiable): Every factual claim must be directly supported by text in the retrieved excerpt for the PMID you cite. Before citing a PMID, verify the claim actually appears in that paper's excerpt. NEVER cite a PMID because it is topically adjacent β€” a citation asserts that specific paper supports that specific claim.
tools.py CHANGED
@@ -34,6 +34,18 @@ SEARCH_LANDSCAPE_TOOL: anthropic.types.ToolParam = {
34
  "input_schema": _load("search_landscape"),
35
  }
36
 
 
 
 
 
 
 
 
 
 
 
 
 
37
  EXTRACT_TRIAL_TARGETS_TOOL: anthropic.types.ToolParam = {
38
  "name": "extract_trial_targets",
39
  "description": (
@@ -55,5 +67,5 @@ CLASSIFY_THERAPY_TOOL: anthropic.types.ToolParam = {
55
 
56
  EXTRACTION_TOOLS: list[anthropic.types.ToolParam] = [EXTRACT_ENTITIES_TOOL]
57
  TRIAL_EXTRACTION_TOOLS: list[anthropic.types.ToolParam] = [EXTRACT_TRIAL_TARGETS_TOOL]
58
- RESEARCH_TOOLS: list[anthropic.types.ToolParam] = [SEARCH_LANDSCAPE_TOOL]
59
  LANDSCAPE_TOOLS: list[anthropic.types.ToolParam] = [CLASSIFY_THERAPY_TOOL]
 
34
  "input_schema": _load("search_landscape"),
35
  }
36
 
37
+ FIND_TRIALS_BY_LOCATION_TOOL: anthropic.types.ToolParam = {
38
+ "name": "find_trials_by_location",
39
+ "description": (
40
+ "Find ALS clinical trials by the facility/institution or the geography (state, city, "
41
+ "or country) where they are conducted. Use this when the physician's question names a "
42
+ "hospital or research site (e.g. 'trials at Mass General Hospital') or a place (e.g. "
43
+ "'trials in NY'). Returns matching trials enriched with recruiting status, an evidence-"
44
+ "strength tier, key supporting papers, and related trials for the same compound."
45
+ ),
46
+ "input_schema": _load("find_trials_by_location"),
47
+ }
48
+
49
  EXTRACT_TRIAL_TARGETS_TOOL: anthropic.types.ToolParam = {
50
  "name": "extract_trial_targets",
51
  "description": (
 
67
 
68
  EXTRACTION_TOOLS: list[anthropic.types.ToolParam] = [EXTRACT_ENTITIES_TOOL]
69
  TRIAL_EXTRACTION_TOOLS: list[anthropic.types.ToolParam] = [EXTRACT_TRIAL_TARGETS_TOOL]
70
+ RESEARCH_TOOLS: list[anthropic.types.ToolParam] = [SEARCH_LANDSCAPE_TOOL, FIND_TRIALS_BY_LOCATION_TOOL]
71
  LANDSCAPE_TOOLS: list[anthropic.types.ToolParam] = [CLASSIFY_THERAPY_TOOL]
trials_query.py ADDED
@@ -0,0 +1,512 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """Facility / geography trial search + research-evidence enrichment.
2
+
3
+ Physicians ask "what trials are at Mass General Hospital" or "what trials are in NY".
4
+ This module filters the offline-ingested trials list by facility/city/state/country
5
+ (in-memory β€” no fetch at query time) and enriches each hit with surrounding research
6
+ evidence: key papers, sibling trials for the same compound, a heuristic evidence-strength
7
+ tier, and recruiting status.
8
+ """
9
+ from __future__ import annotations
10
+
11
+ import html
12
+ import json
13
+ from functools import lru_cache
14
+ from typing import TYPE_CHECKING
15
+
16
+ from logging_config import get_logger
17
+
18
+ if TYPE_CHECKING:
19
+ import chromadb
20
+ import networkx as nx
21
+
22
+ _logger = get_logger("trials_query")
23
+
24
+ # Site statuses that count as "recruiting / open to enrollment".
25
+ RECRUITING_STATUSES = {
26
+ "RECRUITING", "NOT_YET_RECRUITING", "ENROLLING_BY_INVITATION", "AVAILABLE",
27
+ }
28
+
29
+ # Trial status display order β€” open first, closed/unavailable last (mirrors research_agent).
30
+ _TRIAL_STATUS_RANK = {
31
+ "AVAILABLE": 0, "RECRUITING": 1, "NOT_YET_RECRUITING": 2, "ENROLLING_BY_INVITATION": 3,
32
+ "ACTIVE_NOT_RECRUITING": 4, "TEMPORARILY_NOT_AVAILABLE": 5, "COMPLETED": 6,
33
+ "SUSPENDED": 7, "TERMINATED": 8, "WITHDRAWN": 9, "NO_LONGER_AVAILABLE": 10,
34
+ }
35
+
36
+ # US state abbreviation β†’ full name. ClinicalTrials.gov stores the full state name, so
37
+ # "NY" must be normalized to "New York" before matching.
38
+ _STATE_ABBREV = {
39
+ "AL": "Alabama", "AK": "Alaska", "AZ": "Arizona", "AR": "Arkansas", "CA": "California",
40
+ "CO": "Colorado", "CT": "Connecticut", "DE": "Delaware", "FL": "Florida", "GA": "Georgia",
41
+ "HI": "Hawaii", "ID": "Idaho", "IL": "Illinois", "IN": "Indiana", "IA": "Iowa",
42
+ "KS": "Kansas", "KY": "Kentucky", "LA": "Louisiana", "ME": "Maine", "MD": "Maryland",
43
+ "MA": "Massachusetts", "MI": "Michigan", "MN": "Minnesota", "MS": "Mississippi",
44
+ "MO": "Missouri", "MT": "Montana", "NE": "Nebraska", "NV": "Nevada", "NH": "New Hampshire",
45
+ "NJ": "New Jersey", "NM": "New Mexico", "NY": "New York", "NC": "North Carolina",
46
+ "ND": "North Dakota", "OH": "Ohio", "OK": "Oklahoma", "OR": "Oregon", "PA": "Pennsylvania",
47
+ "RI": "Rhode Island", "SC": "South Carolina", "SD": "South Dakota", "TN": "Tennessee",
48
+ "TX": "Texas", "UT": "Utah", "VT": "Vermont", "VA": "Virginia", "WA": "Washington",
49
+ "WV": "West Virginia", "WI": "Wisconsin", "WY": "Wyoming", "DC": "District of Columbia",
50
+ }
51
+
52
+ # Generic facility words that carry no discriminating signal on their own.
53
+ _FACILITY_STOPWORDS = {"of", "the", "and", "at", "for"}
54
+
55
+
56
+ def _norm_alnum(s: str) -> str:
57
+ """Lowercase, alphanumeric-only form so 'CNM-Au8', 'CNMAu8', 'cnm_au8' all unify."""
58
+ return "".join(c for c in s.lower() if c.isalnum())
59
+
60
+
61
+ def _tokens(s: str) -> list[str]:
62
+ """Lowercased alphanumeric word tokens, minus stopwords."""
63
+ words = "".join(c if c.isalnum() else " " for c in s.lower()).split()
64
+ return [w for w in words if w not in _FACILITY_STOPWORDS]
65
+
66
+
67
+ def _normalize_state(state: str) -> str:
68
+ """Map a state abbreviation to its full name; pass full names through unchanged."""
69
+ s = state.strip()
70
+ return _STATE_ABBREV.get(s.upper(), s)
71
+
72
+
73
+ def _facility_matches(query: str, site_facility: str) -> bool:
74
+ """True if every query token is a substring of some site-facility token.
75
+
76
+ Token-level substring matching lets 'Mass General' match 'Massachusetts General
77
+ Hospital' ('mass' βŠ‚ 'massachusetts') without a fuzzy library.
78
+ """
79
+ q_tokens = _tokens(query)
80
+ if not q_tokens:
81
+ return False
82
+ site_tokens = _tokens(site_facility)
83
+ return all(any(q in st for st in site_tokens) for q in q_tokens)
84
+
85
+
86
+ def _site_matches(
87
+ site: dict,
88
+ facility: str | None,
89
+ city: str | None,
90
+ state: str | None,
91
+ country: str | None,
92
+ ) -> bool:
93
+ """True if a single trial site satisfies every provided location filter."""
94
+ if facility and not _facility_matches(facility, site.get("facility", "")):
95
+ return False
96
+ if city and city.strip().lower() not in (site.get("city", "") or "").lower():
97
+ return False
98
+ if state:
99
+ want = _normalize_state(state).lower()
100
+ if want != (site.get("state", "") or "").lower():
101
+ return False
102
+ if country and country.strip().lower() not in (site.get("country", "") or "").lower():
103
+ return False
104
+ return True
105
+
106
+
107
+ def search_trials_by_location(
108
+ trials: list[dict],
109
+ *,
110
+ facility: str | None = None,
111
+ city: str | None = None,
112
+ state: str | None = None,
113
+ country: str | None = None,
114
+ status: str | None = None,
115
+ ) -> list[dict]:
116
+ """Return trials with at least one site matching the location filters.
117
+
118
+ `status`: "Recruiting" keeps only trials with an open overall status; "Not recruiting"
119
+ keeps only closed ones; anything else (None / "All") keeps all. Each returned trial is a
120
+ shallow copy with `matched_sites` attached; recruiting trials are ranked first.
121
+ """
122
+ if not any([facility, city, state, country]):
123
+ return []
124
+
125
+ status_filter = (status or "").strip().lower()
126
+ results: list[dict] = []
127
+ for trial in trials:
128
+ matched_sites = [
129
+ s for s in trial.get("locations", [])
130
+ if _site_matches(s, facility, city, state, country)
131
+ ]
132
+ if not matched_sites:
133
+ continue
134
+
135
+ is_recruiting = trial.get("status", "") in RECRUITING_STATUSES
136
+ if status_filter == "recruiting" and not is_recruiting:
137
+ continue
138
+ if status_filter == "not recruiting" and is_recruiting:
139
+ continue
140
+
141
+ hit = dict(trial)
142
+ hit["matched_sites"] = matched_sites
143
+ results.append(hit)
144
+
145
+ results.sort(key=lambda t: _TRIAL_STATUS_RANK.get(t.get("status", ""), 99))
146
+ return results
147
+
148
+
149
+ def _find_supporting_papers(
150
+ trial: dict,
151
+ collection: "chromadb.Collection | None",
152
+ top_n: int = 3,
153
+ ) -> list[dict]:
154
+ """Retrieve the top research papers relevant to this trial's compound/target."""
155
+ if collection is None or collection.count() == 0:
156
+ return []
157
+
158
+ from rag import retriever as rag_retriever
159
+
160
+ entities = list(trial.get("target_entities") or [])
161
+ if entities:
162
+ results = rag_retriever.search_by_entities(collection, entities)
163
+ else:
164
+ # No extracted target β€” fall back to the trial title + intervention names.
165
+ iv_names = " ".join(iv.get("name", "") for iv in trial.get("interventions", []))
166
+ query = f"{trial.get('title', '')} {iv_names}".strip()
167
+ results = rag_retriever.search(collection, query) if query else []
168
+
169
+ results = rag_retriever.apply_citation_boost(results)
170
+ return [
171
+ {
172
+ "pmid": r["pmid"],
173
+ "title": r["title"],
174
+ "year": r["year"],
175
+ "citation_count": r["citation_count"],
176
+ }
177
+ for r in results[:top_n]
178
+ ]
179
+
180
+
181
+ def _find_sibling_trials(
182
+ trial: dict,
183
+ all_trials: list[dict],
184
+ top_n: int = 5,
185
+ ) -> list[dict]:
186
+ """Other trials testing the same compound/target (prior + concurrent), excluding self."""
187
+ self_nct = trial.get("nct_id", "")
188
+ self_targets = {t.lower() for t in trial.get("target_entities", [])}
189
+ self_ivs = {_norm_alnum(iv.get("name", "")) for iv in trial.get("interventions", [])}
190
+ self_ivs.discard("")
191
+
192
+ siblings: list[dict] = []
193
+ for other in all_trials:
194
+ if other.get("nct_id", "") == self_nct:
195
+ continue
196
+ other_targets = {t.lower() for t in other.get("target_entities", [])}
197
+ other_ivs = {_norm_alnum(iv.get("name", "")) for iv in other.get("interventions", [])}
198
+ if (self_targets & other_targets) or (self_ivs & other_ivs):
199
+ siblings.append({
200
+ "nct_id": other.get("nct_id", ""),
201
+ "title": other.get("title", ""),
202
+ "phase": other.get("phase", ""),
203
+ "status": other.get("status", ""),
204
+ "url": other.get("url", ""),
205
+ })
206
+
207
+ siblings.sort(key=lambda t: _TRIAL_STATUS_RANK.get(t.get("status", ""), 99))
208
+ return siblings[:top_n]
209
+
210
+
211
+ # ── Targeted mechanism (reverse-indexed from the offline therapy landscape) ───
212
+ # A trial's "targeted mechanism" is the dominant mechanism class of the compound it
213
+ # tests, reused from the already-built landscape.json (no LLM at query time). Only
214
+ # landscape-classified compounds get one; unclassified drugs (e.g. brand-new agents
215
+ # whose only extracted target is their own name) map to "" and show no mechanism.
216
+
217
+ def _primary_class(therapy: dict) -> str:
218
+ """A compound's dominant mechanism class: primary role, else highest confidence.
219
+
220
+ Mirrors landscape._primary_class. Returns "" when the compound has no mechanism
221
+ (landscape left it unclassified).
222
+ """
223
+ mechs = therapy.get("mechanisms") or []
224
+ if not mechs:
225
+ return ""
226
+ pool = [m for m in mechs if m.get("role") == "primary"] or mechs
227
+ best = max(pool, key=lambda m: m.get("confidence", 0) or 0)
228
+ return best.get("class", "") or ""
229
+
230
+
231
+ @lru_cache(maxsize=1)
232
+ def _mechanism_index() -> dict[str, str]:
233
+ """Map NCT ID β†’ targeted-mechanism class name, reverse-indexed from landscape.json.
234
+
235
+ Classified compounds take precedence over unclassified ones; compounds with no
236
+ mechanism are skipped. Returns {} when the landscape artifact is absent.
237
+ """
238
+ from config import LANDSCAPE_PATH
239
+
240
+ if not LANDSCAPE_PATH.exists():
241
+ return {}
242
+ try:
243
+ landscape = json.loads(LANDSCAPE_PATH.read_text())
244
+ except Exception:
245
+ _logger.warning("could not load landscape for mechanism index", exc_info=True)
246
+ return {}
247
+
248
+ index: dict[str, str] = {}
249
+ # classifications first so a classified compound's mechanism wins over unclassified
250
+ for group in (landscape.get("classifications", []) + landscape.get("unclassified", [])):
251
+ therapies = group.get("therapies", [group]) if "therapies" in group else [group]
252
+ for therapy in therapies:
253
+ mechanism = _primary_class(therapy)
254
+ if not mechanism:
255
+ continue
256
+ for tr in therapy.get("trials", []):
257
+ index.setdefault(tr.get("nct_id", ""), mechanism)
258
+ index.pop("", None)
259
+ return index
260
+
261
+
262
+ def _kg_paper_count(
263
+ trial: dict, graph: "nx.DiGraph | None", mechanism: str = ""
264
+ ) -> tuple[int, str]:
265
+ """Max supporting-paper count in the KG, and the node label that supplied it.
266
+
267
+ Considers this trial's target entities plus its targeted `mechanism` hub node, so a
268
+ compound whose only extracted target is its own name (no KG node) still gets credit
269
+ for its mechanism's literature instead of scoring 0. The returned label lets the tier
270
+ show a physician *which* node the count came from (e.g. the mechanism vs. the target).
271
+ """
272
+ if graph is None:
273
+ return 0, ""
274
+ from graph.query import _find_node
275
+
276
+ # (name, is_mechanism) β€” mechanism last so a real target wins ties.
277
+ names = [(n, False) for n in trial.get("target_entities", [])]
278
+ if mechanism:
279
+ names.append((mechanism, True))
280
+
281
+ best, best_label = 0, ""
282
+ for name, is_mech in names:
283
+ for node_id in _find_node(graph, name):
284
+ count = graph.nodes[node_id].get("paper_count", 0)
285
+ if count > best:
286
+ best = count
287
+ display = graph.nodes[node_id].get("display_name", name)
288
+ best_label = f"{display} (mechanism)" if is_mech else display
289
+ return best, best_label
290
+
291
+
292
+ def _evidence_tier(
293
+ trial: dict,
294
+ key_papers: list[dict],
295
+ kg_paper_count: int,
296
+ kg_source: str = "",
297
+ ) -> dict:
298
+ """Heuristic evidence-strength tier from paper count, citations, KG breadth, and phase.
299
+
300
+ Strong / Moderate / Emerging β€” a fully offline signal (no LLM call). Returns a
301
+ `rationale`: one entry per scoring factor with the points it earned and a physician-
302
+ readable reason (including *which* KG node supplied the paper count), so the tier can
303
+ explain itself rather than presenting a bare label.
304
+ """
305
+ n_papers = len(key_papers)
306
+ max_cit = max((p.get("citation_count", 0) for p in key_papers), default=0)
307
+ phase = (trial.get("phase", "") or "").upper()
308
+
309
+ rationale: list[dict] = []
310
+
311
+ def factor(label: str, pts: int, detail: str) -> None:
312
+ rationale.append({"factor": label, "points": pts, "detail": detail})
313
+
314
+ if n_papers >= 5:
315
+ factor("Supporting papers", 2, f"{n_papers} supporting papers in the database (β‰₯5)")
316
+ elif n_papers >= 2:
317
+ factor("Supporting papers", 1, f"{n_papers} supporting papers in the database (2–4)")
318
+ else:
319
+ factor("Supporting papers", 0, f"{n_papers} supporting paper(s) in the database (<2)")
320
+
321
+ if max_cit >= 100:
322
+ factor("Citation impact", 2, f"top paper cited {max_cit}Γ— (β‰₯100)")
323
+ elif max_cit >= 20:
324
+ factor("Citation impact", 1, f"top paper cited {max_cit}Γ— (20–99)")
325
+ else:
326
+ factor("Citation impact", 0, f"top paper cited {max_cit}Γ— (<20)")
327
+
328
+ if kg_paper_count >= 5:
329
+ src = f" via {kg_source}" if kg_source else ""
330
+ factor("KG breadth", 1, f"{kg_paper_count} papers on the target/mechanism node{src} (β‰₯5)")
331
+ else:
332
+ src = f" (best: {kg_source})" if kg_source else ""
333
+ factor("KG breadth", 0, f"{kg_paper_count} papers on the target/mechanism node{src} (<5)")
334
+
335
+ is_phase3 = "PHASE3" in phase.replace(" ", "") or "3" in phase
336
+ if is_phase3:
337
+ factor("Trial phase", 1, f"reached Phase 3 ({trial.get('phase', '')})")
338
+ else:
339
+ factor("Trial phase", 0, f"not yet Phase 3 ({trial.get('phase', '') or 'phase unknown'})")
340
+
341
+ points = sum(f["points"] for f in rationale)
342
+
343
+ if points >= 4:
344
+ tier = "Strong"
345
+ elif points >= 2:
346
+ tier = "Moderate"
347
+ else:
348
+ tier = "Emerging"
349
+
350
+ return {
351
+ "tier": tier,
352
+ "points": points,
353
+ "n_papers": n_papers,
354
+ "max_citations": max_cit,
355
+ "kg_paper_count": kg_paper_count,
356
+ "kg_source": kg_source,
357
+ "rationale": rationale,
358
+ }
359
+
360
+
361
+ def enrich_trial(
362
+ trial: dict,
363
+ collection: "chromadb.Collection | None" = None,
364
+ graph: "nx.DiGraph | None" = None,
365
+ all_trials: list[dict] | None = None,
366
+ mechanism_index: dict[str, str] | None = None,
367
+ ) -> dict:
368
+ """Attach research-evidence context to a trial: targeted mechanism, key papers,
369
+ sibling trials, evidence tier."""
370
+ index = _mechanism_index() if mechanism_index is None else mechanism_index
371
+ mechanism = index.get(trial.get("nct_id", ""), "")
372
+
373
+ key_papers = _find_supporting_papers(trial, collection)
374
+ sibling_trials = _find_sibling_trials(trial, all_trials or [])
375
+ kg_paper_count, kg_source = _kg_paper_count(trial, graph, mechanism)
376
+ evidence = _evidence_tier(trial, key_papers, kg_paper_count, kg_source)
377
+
378
+ return {
379
+ "nct_id": trial.get("nct_id", ""),
380
+ "title": trial.get("title", ""),
381
+ "phase": trial.get("phase", ""),
382
+ "status": trial.get("status", ""),
383
+ "is_recruiting": trial.get("status", "") in RECRUITING_STATUSES,
384
+ "sponsor": trial.get("sponsor", ""),
385
+ "url": trial.get("url", ""),
386
+ "target_entities": trial.get("target_entities", []),
387
+ "mechanism": mechanism,
388
+ "matched_sites": trial.get("matched_sites", []),
389
+ "key_papers": key_papers,
390
+ "sibling_trials": sibling_trials,
391
+ "evidence": evidence,
392
+ }
393
+
394
+
395
+ # ── HTML rendering for the Clinical Trials tab ────────────────────────────────
396
+
397
+ def _status_badge(status: str) -> tuple[str, str]:
398
+ """(color, label) for a ClinicalTrials.gov overall-status value."""
399
+ s = (status or "").upper()
400
+ if s in RECRUITING_STATUSES:
401
+ return "#00B894", "Recruiting" if s == "RECRUITING" else status.replace("_", " ").title()
402
+ if s == "ACTIVE_NOT_RECRUITING":
403
+ return "#0984E3", "Active"
404
+ if s == "COMPLETED":
405
+ return "#636E72", "Completed"
406
+ if s in {"TERMINATED", "WITHDRAWN", "SUSPENDED"}:
407
+ return "#D63031", status.title()
408
+ return "#B2BEC3", (status or "Unknown").replace("_", " ").title()
409
+
410
+
411
+ _TIER_COLOR = {"Strong": "#00B894", "Moderate": "#E17055", "Emerging": "#B2BEC3"}
412
+
413
+
414
+ def _pill(text: str, color: str) -> str:
415
+ return (f'<span style="background:{color};color:#fff;border-radius:10px;'
416
+ f'padding:1px 8px;font-size:0.72rem;white-space:nowrap;">{html.escape(text)}</span>')
417
+
418
+
419
+ def _tier_rationale_html(ev: dict) -> str:
420
+ """A collapsible 'why this tier' breakdown: each scoring factor, its points, and reason.
421
+
422
+ Uses a native <details> disclosure (no JS) so a physician can audit the label without
423
+ it crowding the card by default.
424
+ """
425
+ rationale = ev.get("rationale") or []
426
+ if not rationale:
427
+ return ""
428
+ rows = "".join(
429
+ f'<li style="margin:1px 0;{"" if f["points"] else "color:#aaa;"}">'
430
+ f'<b>+{f["points"]}</b> {html.escape(f["factor"])} β€” {html.escape(f["detail"])}</li>'
431
+ for f in rationale
432
+ )
433
+ total = ev.get("points", sum(f["points"] for f in rationale))
434
+ summary = (f'Why {html.escape(ev.get("tier", ""))}? ({total} pts β€” '
435
+ "β‰₯4 Strong Β· 2–3 Moderate Β· &lt;2 Emerging)")
436
+ return (
437
+ '<details style="margin-top:4px;font-size:0.8rem;color:#555;">'
438
+ f'<summary style="cursor:pointer;color:#6C5CE7;">{summary}</summary>'
439
+ f'<ul style="margin:4px 0 0 18px;list-style:none;padding:0;">{rows}</ul>'
440
+ '</details>'
441
+ )
442
+
443
+
444
+ def render_trials_html(enriched: list[dict], match_count: int) -> str:
445
+ """Render enriched location-search results as an HTML card list."""
446
+ if not enriched:
447
+ return '<div style="color:#888;padding:12px 0;">No trials found for this location.</div>'
448
+
449
+ caption = (f'<div style="font-size:0.85rem;color:#666;margin:6px 0;">'
450
+ f'{match_count} trial(s) matched β€” showing {len(enriched)}, recruiting first.</div>')
451
+
452
+ cards = []
453
+ for t in enriched:
454
+ s_color, s_label = _status_badge(t["status"])
455
+ ev = t["evidence"]
456
+ tier_pill = _pill(f'Evidence: {ev["tier"]}', _TIER_COLOR.get(ev["tier"], "#B2BEC3"))
457
+ mech = t.get("mechanism", "")
458
+ mech_pill = _pill(f'Mechanism: {mech}', "#6C5CE7") if mech else ""
459
+ rationale_html = _tier_rationale_html(ev)
460
+ phase = html.escape((t.get("phase") or "β€”").replace("PHASE", "Ph"))
461
+ title = html.escape(t.get("title", "")[:140])
462
+ nct = html.escape(t.get("nct_id", ""))
463
+ url = html.escape(t.get("url", ""))
464
+ nct_link = f'<a href="{url}" target="_blank" rel="noopener">{nct}</a>' if url else nct
465
+
466
+ sites = t.get("matched_sites", [])
467
+ site_bits = []
468
+ for st in sites[:3]:
469
+ loc = ", ".join(x for x in (st.get("facility", ""), st.get("city", ""), st.get("state", "")) if x)
470
+ if loc:
471
+ site_bits.append(html.escape(loc))
472
+ sites_html = "<br>".join(site_bits)
473
+ if len(sites) > 3:
474
+ sites_html += f'<br><span style="color:#aaa;">+{len(sites) - 3} more site(s)</span>'
475
+
476
+ papers = t.get("key_papers", [])
477
+ if papers:
478
+ paper_items = "".join(
479
+ f'<li>{html.escape(p.get("title", "")[:120])} '
480
+ f'({p.get("year") or "n.d."}) β€” '
481
+ f'<a href="https://pubmed.ncbi.nlm.nih.gov/{html.escape(str(p.get("pmid", "")))}/" '
482
+ f'target="_blank" rel="noopener">PMID {html.escape(str(p.get("pmid", "")))}</a>, '
483
+ f'{p.get("citation_count", 0)} citations</li>'
484
+ for p in papers
485
+ )
486
+ papers_html = f'<div style="margin-top:6px;font-size:0.82rem;color:#555;"><b>Key papers:</b><ul style="margin:2px 0 0 18px;">{paper_items}</ul></div>'
487
+ else:
488
+ papers_html = '<div style="margin-top:6px;font-size:0.82rem;color:#aaa;">No supporting papers found in the database.</div>'
489
+
490
+ siblings = t.get("sibling_trials", [])
491
+ if siblings:
492
+ sib_links = ", ".join(
493
+ f'<a href="{html.escape(s.get("url", ""))}" target="_blank" rel="noopener">{html.escape(s.get("nct_id", ""))}</a>'
494
+ for s in siblings
495
+ )
496
+ siblings_html = f'<div style="margin-top:4px;font-size:0.82rem;color:#555;"><b>Related trials (same compound):</b> {sib_links}</div>'
497
+ else:
498
+ siblings_html = ""
499
+
500
+ cards.append(
501
+ '<div style="border:1px solid #e3e3e3;border-radius:8px;padding:10px 12px;margin:8px 0;">'
502
+ f'<div style="display:flex;gap:8px;align-items:center;flex-wrap:wrap;margin-bottom:4px;">'
503
+ f'{_pill(s_label, s_color)}{tier_pill}{mech_pill}'
504
+ f'<span style="color:#888;font-size:0.78rem;">{phase}</span></div>'
505
+ f'<div style="font-weight:600;">{nct_link} β€” {title}</div>'
506
+ f'{rationale_html}'
507
+ f'<div style="margin-top:4px;font-size:0.82rem;color:#555;"><b>Site(s):</b><br>{sites_html}</div>'
508
+ f'{papers_html}{siblings_html}'
509
+ '</div>'
510
+ )
511
+
512
+ return caption + "".join(cards)