KevinIsInCoding Claude Opus 4.8 commited on
Commit
3b2b5d9
·
unverified ·
1 Parent(s): f77eaaf

refactor(prompts): consolidate system prompts + docs/lockfile housekeeping (#23)

Browse files

* refactor(prompts): consolidate all system prompts into prompts.py

extraction/extractor.py and ingestion/clinicaltrials.py each defined their
own local system prompt, and prompts.py held a stale, unused EXTRACTION_SYSTEM
that had drifted from the one the extractor actually ran. Make prompts.py the
single source of truth: move the live extraction and trial-target prompts there
(byte-identical to what ran, so extraction output is unchanged) and import them.

Now all four system prompts (EXTRACTION_SYSTEM, TRIAL_EXTRACTION_SYSTEM,
LANDSCAPE_SYSTEM, SYNTHESIS_SYSTEM) live in prompts.py, matching CLAUDE.md.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* chore: doc paper count 500→10,000 and sync uv.lock to pinned gradio

CLAUDE.md described the corpus as ~500 papers; it is now ~10,000.
uv.lock had drifted to gradio 6.20.0 while pyproject pins gradio==6.14.0 —
re-lock so the lockfile matches the committed pin.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>

Files changed (5) hide show
  1. CLAUDE.md +2 -2
  2. extraction/extractor.py +2 -12
  3. ingestion/clinicaltrials.py +2 -11
  4. prompts.py +27 -22
  5. uv.lock +4 -4
CLAUDE.md CHANGED
@@ -2,7 +2,7 @@
2
 
3
  ## What This Is
4
 
5
- Candle-fire is a physician-facing ALS research intelligence tool. A physician asks a free-text question ("What's the evidence for tofersen targeting SOD1?") and gets a synthesized, cited answer grounded in ~500 curated ALS papers, enriched by a knowledge graph.
6
 
7
  **Sibling project**: beacon (patient-facing clinical trial finder at `../beacon`). Follow the same conventions.
8
 
@@ -10,7 +10,7 @@ Candle-fire is a physician-facing ALS research intelligence tool. A physician as
10
 
11
  1. **Knowledge Graph (KG)**: NetworkX DiGraph linking Gene → Protein → Compound → Pathway → Phenotype → ClinicalTrial. Used to expand query entities before retrieval (e.g., "tofersen" → SOD1 → oxidative stress → related compounds).
12
 
13
- 2. **RAG (Vector Search)**: ChromaDB collection of ~500 ALS paper abstracts/full-text. Citation-count-weighted re-ranking. Used to retrieve evidence passages for synthesis.
14
 
15
  **Query pipeline**: KG expansion first, then RAG retrieval with expanded entity context, then Claude synthesis.
16
 
 
2
 
3
  ## What This Is
4
 
5
+ Candle-fire is a physician-facing ALS research intelligence tool. A physician asks a free-text question ("What's the evidence for tofersen targeting SOD1?") and gets a synthesized, cited answer grounded in ~10,000 curated ALS papers, enriched by a knowledge graph.
6
 
7
  **Sibling project**: beacon (patient-facing clinical trial finder at `../beacon`). Follow the same conventions.
8
 
 
10
 
11
  1. **Knowledge Graph (KG)**: NetworkX DiGraph linking Gene → Protein → Compound → Pathway → Phenotype → ClinicalTrial. Used to expand query entities before retrieval (e.g., "tofersen" → SOD1 → oxidative stress → related compounds).
12
 
13
+ 2. **RAG (Vector Search)**: ChromaDB collection of ~10,000 ALS paper abstracts/full-text. Citation-count-weighted re-ranking. Used to retrieve evidence passages for synthesis.
14
 
15
  **Query pipeline**: KG expansion first, then RAG retrieval with expanded entity context, then Claude synthesis.
16
 
extraction/extractor.py CHANGED
@@ -30,6 +30,7 @@ from config import (
30
  from extraction.normalizer import CanonicalRegistry, guess_entity_type, normalize_entity
31
  from logging_config import get_logger
32
  from models import ALSPaper, ExtractedEntity, EntityRelationship, PaperExtractionResult
 
33
  from tools import EXTRACTION_TOOLS
34
 
35
  _logger = get_logger("extraction.extractor")
@@ -38,17 +39,6 @@ _logger = get_logger("extraction.extractor")
38
  # hour; the ceiling is 24h.
39
  _POLL_INTERVAL_S = 30
40
 
41
- _EXTRACTION_SYSTEM = """\
42
- You are a biomedical NLP expert specializing in ALS (amyotrophic lateral sclerosis).
43
- Extract entities and relationships from each paper using the extract_entities tool.
44
- Call it once per paper. Use the full text when provided — it is richer than the abstract alone.
45
-
46
- Entity types: Gene, Protein, Compound, Pathway, Phenotype, Mechanism.
47
- Relationship types: BINDS, INHIBITS, ASSOCIATED_WITH, TESTED_IN, EXPRESSED_IN, CO_OCCURS.
48
-
49
- Be precise. Only extract entities explicitly mentioned. Return pmid exactly as given.
50
- """
51
-
52
 
53
  def extract_all(
54
  papers_path: Path = PAPERS_PATH,
@@ -224,7 +214,7 @@ def _build_params(batch: list[ALSPaper]) -> MessageCreateParamsNonStreaming:
224
  return MessageCreateParamsNonStreaming(
225
  model=EXTRACTION_MODEL,
226
  max_tokens=8192,
227
- system=_EXTRACTION_SYSTEM,
228
  tools=list(EXTRACTION_TOOLS),
229
  tool_choice={"type": "any"},
230
  messages=[{"role": "user", "content": _format_batch(batch)}],
 
30
  from extraction.normalizer import CanonicalRegistry, guess_entity_type, normalize_entity
31
  from logging_config import get_logger
32
  from models import ALSPaper, ExtractedEntity, EntityRelationship, PaperExtractionResult
33
+ from prompts import EXTRACTION_SYSTEM
34
  from tools import EXTRACTION_TOOLS
35
 
36
  _logger = get_logger("extraction.extractor")
 
39
  # hour; the ceiling is 24h.
40
  _POLL_INTERVAL_S = 30
41
 
 
 
 
 
 
 
 
 
 
 
 
42
 
43
  def extract_all(
44
  papers_path: Path = PAPERS_PATH,
 
214
  return MessageCreateParamsNonStreaming(
215
  model=EXTRACTION_MODEL,
216
  max_tokens=8192,
217
+ system=EXTRACTION_SYSTEM,
218
  tools=list(EXTRACTION_TOOLS),
219
  tool_choice={"type": "any"},
220
  messages=[{"role": "user", "content": _format_batch(batch)}],
ingestion/clinicaltrials.py CHANGED
@@ -8,6 +8,7 @@ import httpx
8
 
9
  from config import CTGOV_BASE, EXTRACTION_MODEL
10
  from logging_config import get_logger
 
11
 
12
  if TYPE_CHECKING:
13
  import anthropic
@@ -16,16 +17,6 @@ _logger = get_logger("ingestion.clinicaltrials")
16
 
17
  _TRIAL_BATCH_SIZE = 10
18
 
19
- _TRIAL_EXTRACTION_SYSTEM = """You are a biomedical NLP expert specializing in ALS (amyotrophic lateral sclerosis) clinical trials.
20
-
21
- For each trial provided, identify the primary biological target(s) being tested or modulated:
22
- - Genes silenced or corrected (e.g., SOD1, TARDBP, FUS, C9orf72, NEK1, VCP, TBK1)
23
- - Proteins targeted (use canonical gene symbol, e.g. TARDBP for TDP-43 protein)
24
- - Compounds/drugs — report the molecular or pathway target, not the drug name (e.g., a trial of AMX0114 targets TARDBP)
25
- - Mechanisms (e.g., neuroinflammation, oxidative stress, glutamate excitotoxicity)
26
-
27
- Call extract_trial_targets once per trial. Return an empty targets list only when no specific molecular or mechanistic target is identifiable."""
28
-
29
 
30
  def fetch_als_trials(
31
  client: "anthropic.Anthropic | None" = None,
@@ -204,7 +195,7 @@ def _call_claude_batch(
204
  response = client.messages.create(
205
  model=EXTRACTION_MODEL,
206
  max_tokens=4096,
207
- system=_TRIAL_EXTRACTION_SYSTEM,
208
  tools=tools,
209
  tool_choice={"type": "any"},
210
  messages=[{"role": "user", "content": "\n".join(lines)}],
 
8
 
9
  from config import CTGOV_BASE, EXTRACTION_MODEL
10
  from logging_config import get_logger
11
+ from prompts import TRIAL_EXTRACTION_SYSTEM
12
 
13
  if TYPE_CHECKING:
14
  import anthropic
 
17
 
18
  _TRIAL_BATCH_SIZE = 10
19
 
 
 
 
 
 
 
 
 
 
 
20
 
21
  def fetch_als_trials(
22
  client: "anthropic.Anthropic | None" = None,
 
195
  response = client.messages.create(
196
  model=EXTRACTION_MODEL,
197
  max_tokens=4096,
198
+ system=TRIAL_EXTRACTION_SYSTEM,
199
  tools=tools,
200
  tool_choice={"type": "any"},
201
  messages=[{"role": "user", "content": "\n".join(lines)}],
prompts.py CHANGED
@@ -1,29 +1,34 @@
1
- """System prompts for extraction and synthesis agents."""
2
 
 
 
 
 
 
 
 
3
  EXTRACTION_SYSTEM = """\
4
- You are a biomedical NLP expert specializing in ALS (amyotrophic lateral sclerosis) research.
5
- Your task is to extract biomedical entities and relationships from ALS paper abstracts.
6
-
7
- Entity types to extract:
8
- - Gene: genetic loci (e.g., SOD1, TARDBP, FUS, C9orf72)
9
- - Protein: protein products (e.g., TDP-43, FUS protein, SOD1 protein)
10
- - Compound: drugs, small molecules, biologics (e.g., riluzole, tofersen, AMX0035)
11
- - Pathway: biological pathways or processes (e.g., glutamate excitotoxicity, autophagy)
12
- - Phenotype: disease features or clinical observations (e.g., bulbar onset, respiratory failure)
13
- - Mechanism: molecular or cellular mechanisms (e.g., protein aggregation, oxidative stress)
14
-
15
- Relationship types to extract:
16
- - BINDS: compound/protein binds to a target
17
- - INHIBITS: compound/gene inhibits a target
18
- - ASSOCIATED_WITH: entity is associated with a disease phenotype or another entity
19
- - TESTED_IN: compound is tested in a clinical trial or animal model
20
- - EXPRESSED_IN: gene/protein is expressed in a tissue or cell type
21
- - CO_OCCURS: entities frequently co-occur in ALS context (weakest relationship)
22
-
23
- Be precise. Only extract entities explicitly mentioned. Confidence reflects how clearly
24
- the entity is identified in the text (1.0 = unambiguous, 0.5 = inferred, 0.3 = uncertain).
25
  """
26
 
 
 
 
 
 
 
 
 
 
 
 
27
  LANDSCAPE_SYSTEM = """\
28
  You are an ALS-pharmacology expert classifying experimental therapies by mechanism of action.
29
  For each therapy you are given EVIDENCE (its trial summaries + retrieved paper abstracts).
 
1
+ """System prompts for every Claude call in candle-fire.
2
 
3
+ Single source of truth: entity extraction (extraction/extractor.py), trial-target
4
+ extraction (ingestion/clinicaltrials.py), therapy-landscape classification
5
+ (scripts/build_landscape.py), and query synthesis (agents/research_agent.py) all import
6
+ their system prompt from here. Add or modify prompts in this file only.
7
+ """
8
+
9
+ # Paper entity/relationship extraction — extraction/extractor.py (Batch API, one call/paper).
10
  EXTRACTION_SYSTEM = """\
11
+ You are a biomedical NLP expert specializing in ALS (amyotrophic lateral sclerosis).
12
+ Extract entities and relationships from each paper using the extract_entities tool.
13
+ Call it once per paper. Use the full text when provided — it is richer than the abstract alone.
14
+
15
+ Entity types: Gene, Protein, Compound, Pathway, Phenotype, Mechanism.
16
+ Relationship types: BINDS, INHIBITS, ASSOCIATED_WITH, TESTED_IN, EXPRESSED_IN, CO_OCCURS.
17
+
18
+ Be precise. Only extract entities explicitly mentioned. Return pmid exactly as given.
 
 
 
 
 
 
 
 
 
 
 
 
 
19
  """
20
 
21
+ # Clinical-trial target extraction — ingestion/clinicaltrials.py (one call/trial).
22
+ TRIAL_EXTRACTION_SYSTEM = """You are a biomedical NLP expert specializing in ALS (amyotrophic lateral sclerosis) clinical trials.
23
+
24
+ For each trial provided, identify the primary biological target(s) being tested or modulated:
25
+ - Genes silenced or corrected (e.g., SOD1, TARDBP, FUS, C9orf72, NEK1, VCP, TBK1)
26
+ - Proteins targeted (use canonical gene symbol, e.g. TARDBP for TDP-43 protein)
27
+ - Compounds/drugs — report the molecular or pathway target, not the drug name (e.g., a trial of AMX0114 targets TARDBP)
28
+ - Mechanisms (e.g., neuroinflammation, oxidative stress, glutamate excitotoxicity)
29
+
30
+ Call extract_trial_targets once per trial. Return an empty targets list only when no specific molecular or mechanistic target is identifiable."""
31
+
32
  LANDSCAPE_SYSTEM = """\
33
  You are an ALS-pharmacology expert classifying experimental therapies by mechanism of action.
34
  For each therapy you are given EVIDENCE (its trial summaries + retrieved paper abstracts).
uv.lock CHANGED
@@ -472,7 +472,7 @@ requires-dist = [
472
  { name = "anthropic", specifier = ">=0.50.0" },
473
  { name = "biopython", specifier = ">=1.84" },
474
  { name = "chromadb", specifier = ">=0.5.0" },
475
- { name = "gradio", specifier = ">=6.14.0,<7.0.0" },
476
  { name = "httpx", specifier = ">=0.27.0" },
477
  { name = "networkx", specifier = ">=3.3" },
478
  { name = "openai", specifier = ">=2.44.0" },
@@ -980,7 +980,7 @@ wheels = [
980
 
981
  [[package]]
982
  name = "gradio"
983
- version = "6.20.0"
984
  source = { registry = "https://pypi.org/simple" }
985
  dependencies = [
986
  { name = "anyio" },
@@ -1013,9 +1013,9 @@ dependencies = [
1013
  { name = "typing-extensions" },
1014
  { name = "uvicorn" },
1015
  ]
1016
- sdist = { url = "https://files.pythonhosted.org/packages/f8/96/3618f8a189180ab3d4e5f8b258fd92e419eda9f49b6ab4dd50c6782cda33/gradio-6.20.0.tar.gz", hash = "sha256:8672866e9225a1f8297325a6742bb525a3a0ddc8aa4d75d99beb10a27ea2915f", size = 44844340, upload-time = "2026-07-07T18:53:57.402Z" }
1017
  wheels = [
1018
- { url = "https://files.pythonhosted.org/packages/d7/f4/7dc4540836e326391a9d1ba20193d795d5a4582839b60f5c10397f5120bb/gradio-6.20.0-py3-none-any.whl", hash = "sha256:91d2ec29c37917d55b18dba87aa15bb15c4796d84dce9a9ef4ec7d87453a7fee", size = 30964820, upload-time = "2026-07-07T18:53:52.403Z" },
1019
  ]
1020
 
1021
  [[package]]
 
472
  { name = "anthropic", specifier = ">=0.50.0" },
473
  { name = "biopython", specifier = ">=1.84" },
474
  { name = "chromadb", specifier = ">=0.5.0" },
475
+ { name = "gradio", specifier = "==6.14.0" },
476
  { name = "httpx", specifier = ">=0.27.0" },
477
  { name = "networkx", specifier = ">=3.3" },
478
  { name = "openai", specifier = ">=2.44.0" },
 
980
 
981
  [[package]]
982
  name = "gradio"
983
+ version = "6.14.0"
984
  source = { registry = "https://pypi.org/simple" }
985
  dependencies = [
986
  { name = "anyio" },
 
1013
  { name = "typing-extensions" },
1014
  { name = "uvicorn" },
1015
  ]
1016
+ sdist = { url = "https://files.pythonhosted.org/packages/de/bd/7d1544571de4566138e50c868b91bb79e38c998266896d38fed3d3a77898/gradio-6.14.0.tar.gz", hash = "sha256:4972ef7d01ac57472772624eb4e095767b6c8f3cd4846b7fea648e8034cda9f8", size = 36026409, upload-time = "2026-04-30T16:50:31.698Z" }
1017
  wheels = [
1018
+ { url = "https://files.pythonhosted.org/packages/7e/e2/41f991b2e212b7afda3a9927676ebf8af302e5a2c632b330bd70bf2cf2c1/gradio-6.14.0-py3-none-any.whl", hash = "sha256:bb702f5ab643510d167bae54269ad6e985c2185174d388fe542cc5957f51f4fd", size = 19687959, upload-time = "2026-04-30T16:50:26.914Z" },
1019
  ]
1020
 
1021
  [[package]]