Instructions to use taejoon89/refqa with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- PEFT
How to use taejoon89/refqa with PEFT:
from peft import PeftModel from transformers import AutoModelForCausalLM base_model = AutoModelForCausalLM.from_pretrained("google/gemma-3-27b-it") model = PeftModel.from_pretrained(base_model, "taejoon89/refqa") - Notebooks
- Google Colab
- Kaggle
RefQA Gemma-3-27B LoRA adapter (v0.7) — clean release
Browse files- .gitattributes +1 -0
- README.md +69 -0
- adapter_config.json +40 -0
- adapter_model.safetensors +3 -0
- chat_template.jinja +1 -0
- demo_refqa_gemma3_v07.ipynb +207 -0
- tokenizer.json +3 -0
- tokenizer_config.json +25 -0
- trainer_state.json +0 -0
- v03_system_prompt.txt +251 -0
.gitattributes
CHANGED
|
@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
|
| 33 |
*.zip filter=lfs diff=lfs merge=lfs -text
|
| 34 |
*.zst filter=lfs diff=lfs merge=lfs -text
|
| 35 |
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
|
|
|
|
|
| 33 |
*.zip filter=lfs diff=lfs merge=lfs -text
|
| 34 |
*.zst filter=lfs diff=lfs merge=lfs -text
|
| 35 |
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
| 36 |
+
tokenizer.json filter=lfs diff=lfs merge=lfs -text
|
README.md
ADDED
|
@@ -0,0 +1,69 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: gemma
|
| 3 |
+
base_model: google/gemma-3-27b-it
|
| 4 |
+
library_name: peft
|
| 5 |
+
pipeline_tag: text-generation
|
| 6 |
+
tags:
|
| 7 |
+
- lora
|
| 8 |
+
- peft
|
| 9 |
+
- sft
|
| 10 |
+
- medical
|
| 11 |
+
- clinical-nlp
|
| 12 |
+
- citation
|
| 13 |
+
- refqa
|
| 14 |
+
---
|
| 15 |
+
|
| 16 |
+
# RefQA — Gemma-3-27B mixed adapter (v0.7)
|
| 17 |
+
|
| 18 |
+
A LoRA adapter that fine-tunes **google/gemma-3-27b-it** on a mix of medical instruction data and the [RefQA](https://doi.org/10.5281/zenodo.20805692) CitationQA data. It learns the RefQA CitationQA task while retaining general medical question-answering ability.
|
| 19 |
+
|
| 20 |
+
## Task
|
| 21 |
+
|
| 22 |
+
Two capabilities from one adapter:
|
| 23 |
+
1. Structured **CitationQA** generation on the RefQA schema (given the RefQA system prompt and a citation context), and
|
| 24 |
+
2. Retained general medical question answering.
|
| 25 |
+
|
| 26 |
+
## Results
|
| 27 |
+
|
| 28 |
+
- Final `eval_loss` 0.215, token accuracy 92.6% (2 epochs / 50,224 steps)
|
| 29 |
+
- Out-of-domain evaluation: **PubMedQA 74.10%**, **MedQA 24.59%**
|
| 30 |
+
|
| 31 |
+
These numbers are reported as measured, not as a claim of state of the art.
|
| 32 |
+
|
| 33 |
+
## Limitations
|
| 34 |
+
|
| 35 |
+
- Research artifact. No clinical validation; not for clinical decision-making.
|
| 36 |
+
- The CitationQA behavior requires the RefQA system prompt and chat format (below); without them the structured output is not reproduced.
|
| 37 |
+
|
| 38 |
+
## Training
|
| 39 |
+
|
| 40 |
+
- Base: `google/gemma-3-27b-it` (snapshot `005ad340`)
|
| 41 |
+
- Data: medical instruction mix + RefQA
|
| 42 |
+
- LoRA: r=64, alpha=128, dropout=0.05; language-model attention and MLP projections
|
| 43 |
+
- 2 epochs / 50,224 steps
|
| 44 |
+
- Stack: torch 2.11 (cu130), transformers 5.7, peft 0.19, trl 1.3
|
| 45 |
+
|
| 46 |
+
## How to use
|
| 47 |
+
|
| 48 |
+
```python
|
| 49 |
+
from transformers import AutoModelForCausalLM, AutoTokenizer
|
| 50 |
+
from peft import PeftModel
|
| 51 |
+
|
| 52 |
+
adapter = "taejoon89/refqa"
|
| 53 |
+
base = "google/gemma-3-27b-it" # gated: accept the Gemma license first
|
| 54 |
+
|
| 55 |
+
tok = AutoTokenizer.from_pretrained(adapter)
|
| 56 |
+
model = AutoModelForCausalLM.from_pretrained(base, torch_dtype="auto", device_map="auto")
|
| 57 |
+
model = PeftModel.from_pretrained(model, adapter)
|
| 58 |
+
```
|
| 59 |
+
|
| 60 |
+
For CitationQA generation, the system prompt and input builder are in `extract_qa_glm_v03.py` at https://github.com/jin-0311/refqa. The training chat template is included here as `chat_template.jinja`.
|
| 61 |
+
|
| 62 |
+
## Dataset and code
|
| 63 |
+
|
| 64 |
+
- Dataset: RefQA — https://doi.org/10.5281/zenodo.20805692
|
| 65 |
+
- Pipeline code: https://github.com/jin-0311/refqa
|
| 66 |
+
|
| 67 |
+
## License
|
| 68 |
+
|
| 69 |
+
Use is governed by the base model's license (Gemma Terms of Use). You must accept the Gemma license to download and use the base model.
|
adapter_config.json
ADDED
|
@@ -0,0 +1,40 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"alora_invocation_tokens": null,
|
| 3 |
+
"alpha_pattern": {},
|
| 4 |
+
"arrow_config": null,
|
| 5 |
+
"auto_mapping": null,
|
| 6 |
+
"base_model_name_or_path": "google/gemma-3-27b-it",
|
| 7 |
+
"bias": "none",
|
| 8 |
+
"corda_config": null,
|
| 9 |
+
"ensure_weight_tying": false,
|
| 10 |
+
"eva_config": null,
|
| 11 |
+
"exclude_modules": null,
|
| 12 |
+
"fan_in_fan_out": false,
|
| 13 |
+
"inference_mode": true,
|
| 14 |
+
"init_lora_weights": true,
|
| 15 |
+
"layer_replication": null,
|
| 16 |
+
"layers_pattern": null,
|
| 17 |
+
"layers_to_transform": null,
|
| 18 |
+
"loftq_config": {},
|
| 19 |
+
"lora_alpha": 128,
|
| 20 |
+
"lora_bias": false,
|
| 21 |
+
"lora_dropout": 0.05,
|
| 22 |
+
"lora_ga_config": null,
|
| 23 |
+
"megatron_config": null,
|
| 24 |
+
"megatron_core": "megatron.core",
|
| 25 |
+
"modules_to_save": null,
|
| 26 |
+
"peft_type": "LORA",
|
| 27 |
+
"peft_version": "0.19.1",
|
| 28 |
+
"qalora_group_size": 16,
|
| 29 |
+
"r": 64,
|
| 30 |
+
"rank_pattern": {},
|
| 31 |
+
"revision": null,
|
| 32 |
+
"target_modules": ".*language_model\\.layers\\.\\d+\\.(?:self_attn|mlp)\\.(?:q_proj|k_proj|v_proj|o_proj|up_proj|down_proj|gate_proj)$",
|
| 33 |
+
"target_parameters": null,
|
| 34 |
+
"task_type": "CAUSAL_LM",
|
| 35 |
+
"trainable_token_indices": null,
|
| 36 |
+
"use_bdlora": null,
|
| 37 |
+
"use_dora": false,
|
| 38 |
+
"use_qalora": false,
|
| 39 |
+
"use_rslora": false
|
| 40 |
+
}
|
adapter_model.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:4c0666d4cccbae4147f6ef46e1ebc4b00e209ecb127f02b8068006da55e59e69
|
| 3 |
+
size 1816396048
|
chat_template.jinja
ADDED
|
@@ -0,0 +1 @@
|
|
|
|
|
|
|
| 1 |
+
{{ bos_token }}{%- if messages[0]['role'] == 'system' -%} {%- set first_user_prefix = messages[0]['content'] + '\n\n' -%} {%- set loop_messages = messages[1:] -%}{%- else -%} {%- set first_user_prefix = '' -%} {%- set loop_messages = messages -%}{%- endif -%}{%- for message in loop_messages -%} {%- if (message['role'] == 'user') != (loop.index0 % 2 == 0) -%} {{ raise_exception('Conversation roles must alternate user/assistant/user/assistant/...') }} {%- endif -%} {%- if message['role'] == 'assistant' -%} {{ '<start_of_turn>model\n' }} {% generation %}{{ message['content'] | trim }}{{ '<end_of_turn>\n' }} {% endgeneration %} {%- else -%} {{ '<start_of_turn>' + message['role'] + '\n' + (first_user_prefix if loop.first else '') + (message['content'] | trim) + '<end_of_turn>\n' }} {%- endif -%}{%- endfor -%}{%- if add_generation_prompt -%} {{ '<start_of_turn>model\n' }}{%- endif -%}
|
demo_refqa_gemma3_v07.ipynb
ADDED
|
@@ -0,0 +1,207 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"cells": [
|
| 3 |
+
{
|
| 4 |
+
"cell_type": "markdown",
|
| 5 |
+
"id": "7093007e",
|
| 6 |
+
"metadata": {},
|
| 7 |
+
"source": [
|
| 8 |
+
"# RefQA Gemma-3-27B (v0.7) adapter — usage demo\n",
|
| 9 |
+
"\n",
|
| 10 |
+
"This notebook shows how to load and use the **`refqa-gemma3-27b-mixed-v07`** LoRA adapter:\n",
|
| 11 |
+
"inspect the adapter, load the tokenizer, build a CitationQA prompt, and (optionally) run generation.\n",
|
| 12 |
+
"\n",
|
| 13 |
+
"**Requirements**\n",
|
| 14 |
+
"- Adapter files (this repo): `adapter_model.safetensors`, `adapter_config.json`, tokenizer, `chat_template.jinja`, `v03_system_prompt.txt`.\n",
|
| 15 |
+
"- For **generation only**: the gated base model `google/gemma-3-27b-it` (accept its license and log in with a HF token) and a GPU (~16 GB in 4-bit, or ~54 GB in bf16).\n",
|
| 16 |
+
"\n",
|
| 17 |
+
"The input/config/prompt cells run without the base model. The generation cell is guarded by `RUN_GENERATION`."
|
| 18 |
+
]
|
| 19 |
+
},
|
| 20 |
+
{
|
| 21 |
+
"cell_type": "code",
|
| 22 |
+
"execution_count": null,
|
| 23 |
+
"id": "cc715efe",
|
| 24 |
+
"metadata": {},
|
| 25 |
+
"outputs": [],
|
| 26 |
+
"source": [
|
| 27 |
+
"import json, os\n",
|
| 28 |
+
"\n",
|
| 29 |
+
"ADAPTER = \".\" # this repo/folder holds the adapter\n",
|
| 30 |
+
"BASE = \"google/gemma-3-27b-it\" # gated base; needed only for generation\n",
|
| 31 |
+
"RUN_GENERATION = True # set False to skip actual generation\n",
|
| 32 |
+
"\n",
|
| 33 |
+
"import transformers\n",
|
| 34 |
+
"print(\"transformers\", transformers.__version__)"
|
| 35 |
+
]
|
| 36 |
+
},
|
| 37 |
+
{
|
| 38 |
+
"cell_type": "markdown",
|
| 39 |
+
"id": "904ac805",
|
| 40 |
+
"metadata": {},
|
| 41 |
+
"source": [
|
| 42 |
+
"## 1. Inspect the adapter"
|
| 43 |
+
]
|
| 44 |
+
},
|
| 45 |
+
{
|
| 46 |
+
"cell_type": "code",
|
| 47 |
+
"execution_count": null,
|
| 48 |
+
"id": "91ee8ec8",
|
| 49 |
+
"metadata": {},
|
| 50 |
+
"outputs": [],
|
| 51 |
+
"source": [
|
| 52 |
+
"cfg = json.load(open(os.path.join(ADAPTER, \"adapter_config.json\")))\n",
|
| 53 |
+
"print(\"base_model :\", cfg[\"base_model_name_or_path\"])\n",
|
| 54 |
+
"print(\"peft_type :\", cfg[\"peft_type\"], \"| task:\", cfg[\"task_type\"])\n",
|
| 55 |
+
"print(\"LoRA r :\", cfg[\"r\"], \"| alpha:\", cfg[\"lora_alpha\"], \"| dropout:\", cfg[\"lora_dropout\"])\n",
|
| 56 |
+
"print(\"targets :\", cfg[\"target_modules\"])"
|
| 57 |
+
]
|
| 58 |
+
},
|
| 59 |
+
{
|
| 60 |
+
"cell_type": "markdown",
|
| 61 |
+
"id": "a88ff894",
|
| 62 |
+
"metadata": {},
|
| 63 |
+
"source": [
|
| 64 |
+
"## 2. Load the tokenizer (no base model needed)"
|
| 65 |
+
]
|
| 66 |
+
},
|
| 67 |
+
{
|
| 68 |
+
"cell_type": "code",
|
| 69 |
+
"execution_count": null,
|
| 70 |
+
"id": "f3ad3bed",
|
| 71 |
+
"metadata": {},
|
| 72 |
+
"outputs": [],
|
| 73 |
+
"source": [
|
| 74 |
+
"from transformers import AutoTokenizer\n",
|
| 75 |
+
"tok = AutoTokenizer.from_pretrained(ADAPTER)\n",
|
| 76 |
+
"print(type(tok).__name__, \"| vocab size:\", tok.vocab_size)\n",
|
| 77 |
+
"print(\"has chat template:\", tok.chat_template is not None)"
|
| 78 |
+
]
|
| 79 |
+
},
|
| 80 |
+
{
|
| 81 |
+
"cell_type": "markdown",
|
| 82 |
+
"id": "ee2838ec",
|
| 83 |
+
"metadata": {},
|
| 84 |
+
"source": [
|
| 85 |
+
"## 3. Build a CitationQA prompt\n",
|
| 86 |
+
"\n",
|
| 87 |
+
"The model was trained as `system` (the RefQA v0.3 system prompt) + `user` (one citation context)\n",
|
| 88 |
+
"→ `assistant` (a strict-JSON CitationQA object). The exact input builder is `build_user_prompt`\n",
|
| 89 |
+
"in `extract_qa_glm_v03.py` (https://github.com/jin-0311/refqa). Below is a compact illustrative\n",
|
| 90 |
+
"rendering; the system prompt bundled here (`v03_system_prompt.txt`) is the real one."
|
| 91 |
+
]
|
| 92 |
+
},
|
| 93 |
+
{
|
| 94 |
+
"cell_type": "code",
|
| 95 |
+
"execution_count": null,
|
| 96 |
+
"id": "70530163",
|
| 97 |
+
"metadata": {},
|
| 98 |
+
"outputs": [],
|
| 99 |
+
"source": [
|
| 100 |
+
"SYSTEM = open(os.path.join(ADAPTER, \"v03_system_prompt.txt\")).read()\n",
|
| 101 |
+
"\n",
|
| 102 |
+
"# One illustrative citation context (citing paper A cites cited paper B).\n",
|
| 103 |
+
"record = {\n",
|
| 104 |
+
" \"citing\": {\"pmcid\": \"PMC0000000\", \"title\": \"Example citing clinical study\"},\n",
|
| 105 |
+
" \"context\": {\n",
|
| 106 |
+
" \"imrad\": \"Introduction\",\n",
|
| 107 |
+
" \"evidence_sentence\": \"Prior work reported that drug X reduced 30-day mortality in sepsis patients [12].\",\n",
|
| 108 |
+
" },\n",
|
| 109 |
+
" \"cited\": {\n",
|
| 110 |
+
" \"pmid\": \"11111111\",\n",
|
| 111 |
+
" \"title\": \"Randomized trial of drug X in sepsis\",\n",
|
| 112 |
+
" \"abstract\": \"In a randomized controlled trial of 480 adults with sepsis, drug X reduced \"\n",
|
| 113 |
+
" \"30-day mortality from 28% to 19% compared with placebo (p=0.01).\",\n",
|
| 114 |
+
" },\n",
|
| 115 |
+
"}\n",
|
| 116 |
+
"\n",
|
| 117 |
+
"def build_user_prompt(rec):\n",
|
| 118 |
+
" c, ctx, cd = rec[\"citing\"], rec[\"context\"], rec[\"cited\"]\n",
|
| 119 |
+
" return (\n",
|
| 120 |
+
" f\"CITING PAPER: {c['title']} ({c['pmcid']})\\n\"\n",
|
| 121 |
+
" f\"SECTION: {ctx['imrad']}\\n\"\n",
|
| 122 |
+
" f\"EVIDENCE SENTENCE: {ctx['evidence_sentence']}\\n\\n\"\n",
|
| 123 |
+
" f\"CITED PAPER: {cd['title']} (PMID {cd['pmid']})\\n\"\n",
|
| 124 |
+
" f\"CITED ABSTRACT: {cd['abstract']}\\n\"\n",
|
| 125 |
+
" )\n",
|
| 126 |
+
"\n",
|
| 127 |
+
"messages = [\n",
|
| 128 |
+
" {\"role\": \"system\", \"content\": SYSTEM},\n",
|
| 129 |
+
" {\"role\": \"user\", \"content\": build_user_prompt(record)},\n",
|
| 130 |
+
"]\n",
|
| 131 |
+
"\n",
|
| 132 |
+
"prompt = tok.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)\n",
|
| 133 |
+
"n_tok = len(tok(prompt).input_ids)\n",
|
| 134 |
+
"print(f\"system prompt: {len(SYSTEM)} chars | full prompt: {n_tok} tokens\\n\")\n",
|
| 135 |
+
"print(prompt[:700], \"\\n...[truncated]...\")"
|
| 136 |
+
]
|
| 137 |
+
},
|
| 138 |
+
{
|
| 139 |
+
"cell_type": "markdown",
|
| 140 |
+
"id": "1c25b212",
|
| 141 |
+
"metadata": {},
|
| 142 |
+
"source": [
|
| 143 |
+
"## 4. Generate (requires the base model + a HF token)\n",
|
| 144 |
+
"\n",
|
| 145 |
+
"Set `RUN_GENERATION = True` in cell 1, accept the `google/gemma-3-27b-it` license, and\n",
|
| 146 |
+
"`huggingface-cli login`. This loads the base in 4-bit (bitsandbytes) and applies the adapter."
|
| 147 |
+
]
|
| 148 |
+
},
|
| 149 |
+
{
|
| 150 |
+
"cell_type": "code",
|
| 151 |
+
"execution_count": null,
|
| 152 |
+
"id": "a53030a4",
|
| 153 |
+
"metadata": {},
|
| 154 |
+
"outputs": [],
|
| 155 |
+
"source": [
|
| 156 |
+
"if RUN_GENERATION:\n",
|
| 157 |
+
" import torch\n",
|
| 158 |
+
" from transformers import AutoModelForCausalLM, BitsAndBytesConfig\n",
|
| 159 |
+
" from peft import PeftModel\n",
|
| 160 |
+
"\n",
|
| 161 |
+
" bnb = BitsAndBytesConfig(load_in_4bit=True, bnb_4bit_quant_type=\"nf4\",\n",
|
| 162 |
+
" bnb_4bit_compute_dtype=torch.bfloat16)\n",
|
| 163 |
+
" model = AutoModelForCausalLM.from_pretrained(BASE, quantization_config=bnb, device_map=\"auto\")\n",
|
| 164 |
+
" model = PeftModel.from_pretrained(model, ADAPTER)\n",
|
| 165 |
+
" model.eval()\n",
|
| 166 |
+
"\n",
|
| 167 |
+
" dev = next(model.parameters()).device\n",
|
| 168 |
+
" inputs = tok(prompt, return_tensors=\"pt\").to(dev)\n",
|
| 169 |
+
" with torch.no_grad():\n",
|
| 170 |
+
" out = model.generate(**inputs, max_new_tokens=768, do_sample=False)\n",
|
| 171 |
+
" text = tok.decode(out[0][inputs.input_ids.shape[1]:], skip_special_tokens=True)\n",
|
| 172 |
+
" print(text)\n",
|
| 173 |
+
" try:\n",
|
| 174 |
+
" print(\"\\nparsed JSON keys:\", list(json.loads(text).keys()))\n",
|
| 175 |
+
" except Exception as e:\n",
|
| 176 |
+
" print(\"\\n(could not parse as JSON:\", e, \")\")\n",
|
| 177 |
+
"else:\n",
|
| 178 |
+
" print(\"RUN_GENERATION = False -> skipped. Set it True with the base model + HF token to generate.\")"
|
| 179 |
+
]
|
| 180 |
+
},
|
| 181 |
+
{
|
| 182 |
+
"cell_type": "markdown",
|
| 183 |
+
"id": "d41be1bd",
|
| 184 |
+
"metadata": {},
|
| 185 |
+
"source": [
|
| 186 |
+
"## Notes\n",
|
| 187 |
+
"\n",
|
| 188 |
+
"- This adapter only works with its exact base, `google/gemma-3-27b-it`.\n",
|
| 189 |
+
"- For faithful CitationQA output, keep the bundled `v03_system_prompt.txt` as the system message and\n",
|
| 190 |
+
" the training chat template (`chat_template.jinja`, applied above via `apply_chat_template`).\n",
|
| 191 |
+
"- Research artifact; not for clinical use. Dataset: https://doi.org/10.5281/zenodo.20805692 · code: https://github.com/jin-0311/refqa"
|
| 192 |
+
]
|
| 193 |
+
}
|
| 194 |
+
],
|
| 195 |
+
"metadata": {
|
| 196 |
+
"kernelspec": {
|
| 197 |
+
"display_name": "venv_RefQA",
|
| 198 |
+
"language": "python",
|
| 199 |
+
"name": "venv_refqa"
|
| 200 |
+
},
|
| 201 |
+
"language_info": {
|
| 202 |
+
"name": "python"
|
| 203 |
+
}
|
| 204 |
+
},
|
| 205 |
+
"nbformat": 4,
|
| 206 |
+
"nbformat_minor": 5
|
| 207 |
+
}
|
tokenizer.json
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:daab2354f8a74e70d70b4d1f804939b68a8c9624dd06cb7858e52dd8970e9726
|
| 3 |
+
size 33384567
|
tokenizer_config.json
ADDED
|
@@ -0,0 +1,25 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"backend": "tokenizers",
|
| 3 |
+
"boi_token": "<start_of_image>",
|
| 4 |
+
"bos_token": "<bos>",
|
| 5 |
+
"clean_up_tokenization_spaces": false,
|
| 6 |
+
"eoi_token": "<end_of_image>",
|
| 7 |
+
"eos_token": "<eos>",
|
| 8 |
+
"image_token": "<image_soft_token>",
|
| 9 |
+
"is_local": true,
|
| 10 |
+
"local_files_only": false,
|
| 11 |
+
"mask_token": "<mask>",
|
| 12 |
+
"model_max_length": 1000000000000000019884624838656,
|
| 13 |
+
"model_specific_special_tokens": {
|
| 14 |
+
"boi_token": "<start_of_image>",
|
| 15 |
+
"eoi_token": "<end_of_image>",
|
| 16 |
+
"image_token": "<image_soft_token>"
|
| 17 |
+
},
|
| 18 |
+
"pad_token": "<pad>",
|
| 19 |
+
"processor_class": "Gemma3Processor",
|
| 20 |
+
"sp_model_kwargs": null,
|
| 21 |
+
"spaces_between_special_tokens": false,
|
| 22 |
+
"tokenizer_class": "GemmaTokenizer",
|
| 23 |
+
"unk_token": "<unk>",
|
| 24 |
+
"use_default_system_prompt": false
|
| 25 |
+
}
|
trainer_state.json
ADDED
|
The diff for this file is too large to render.
See raw diff
|
|
|
v03_system_prompt.txt
ADDED
|
@@ -0,0 +1,251 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
PROMPT_VERSION=v0.3
|
| 2 |
+
|
| 3 |
+
You are a careful biomedical/clinical research assistant. You read ONE citation context from paper A (the citing paper) where it cites paper B, and you output a strict JSON record describing what A is asserting by citing B.
|
| 4 |
+
|
| 5 |
+
Critical distinctions:
|
| 6 |
+
- 'attributed_claim' is what paper A asserts BY CITING B in this specific sentence. It is NOT a summary of paper B itself.
|
| 7 |
+
- The cited paper's abstract is given so you can SANITY-CHECK plausibility. If B's abstract is unrelated to the citing context, set relevance_check='mismatch', support_strength='insufficient-context', explain in mismatch_reason, and write empty strings for attributed_claim / question / answer / evidence_quote_from_cited.
|
| 8 |
+
- For multi-citation sentences (n_co_cited_refs > 1), focus only on what B specifically supports; do not attribute the other refs' contributions to B.
|
| 9 |
+
|
| 10 |
+
WHAT vs WHY — do not confuse these two enums:
|
| 11 |
+
- 'citation_purpose' is WHY A cites B. Allowed values: background / method / result-support / comparison / contrast / negative / other.
|
| 12 |
+
- 'evidence_type' is HOW the citation is presented. Allowed values: direct-quote / paraphrase / statistic / method-citation / generic-background / other.
|
| 13 |
+
NEVER use a citation_purpose value (e.g. 'result-support', 'contrast', 'background') inside evidence_type. They are different slots.
|
| 14 |
+
|
| 15 |
+
Hard requirements for v0.3:
|
| 16 |
+
1. evidence_quote_from_cited MUST be a verbatim substring of the cited abstract (we will programmatically verify). Pick the single most relevant sentence or fragment (up to ~50 words). When evidence_type='generic-background' or relevance_check is not 'match', leave it empty.
|
| 17 |
+
2. cited_study_design reflects the design of paper B inferred from its abstract. Prefer the most specific label: rct / non-randomized-trial / prospective-cohort / retrospective-cohort / case-control / cross-sectional / case-series / case-report / systematic-review / meta-analysis / guideline / preclinical. Use 'observational' ONLY as a residual bucket when the abstract makes it clear the study is observational but does NOT specify prospective vs retrospective ascertainment AND the design is not clearly cohort/case-control/cross-sectional. If the abstract clearly says 'prospective cohort' or 'retrospective review', USE THE SPECIFIC LABEL — do not default to 'observational'. Use 'preclinical' for animal/in-vitro/translational mechanism papers. Use 'other' only if truly unidentifiable.
|
| 18 |
+
3. When evidence_type='statistic', fill statistic_extraction with the metric, value, CI/range, p-value, and n_subjects exactly as written. Do not infer or compute. Otherwise statistic_extraction must be null.
|
| 19 |
+
|
| 20 |
+
Output requirements:
|
| 21 |
+
- Return ONE JSON object — no prose, no code fences, no extra fields.
|
| 22 |
+
- Field names and enum values MUST match the schema exactly.
|
| 23 |
+
- Cite the cited paper as [PMID:<pmid>] inside the answer.
|
| 24 |
+
- Do not invent facts that are not in evidence_sentence or cited abstract.
|
| 25 |
+
|
| 26 |
+
Schema (output must validate against this):
|
| 27 |
+
{
|
| 28 |
+
"$defs": {
|
| 29 |
+
"StatisticExtraction": {
|
| 30 |
+
"properties": {
|
| 31 |
+
"metric": {
|
| 32 |
+
"description": "Name of the statistic, e.g. 'median LOS', 'OR for stroke', 'sensitivity', 'in-hospital mortality'.",
|
| 33 |
+
"title": "Metric",
|
| 34 |
+
"type": "string"
|
| 35 |
+
},
|
| 36 |
+
"value": {
|
| 37 |
+
"description": "Raw value as written, including units. e.g. '2.53 days', '93 kcal/kg', '0.42'.",
|
| 38 |
+
"title": "Value",
|
| 39 |
+
"type": "string"
|
| 40 |
+
},
|
| 41 |
+
"ci_or_range": {
|
| 42 |
+
"default": "",
|
| 43 |
+
"description": "Confidence interval or range, e.g. '95% CI 1.78-2.98' or 'range 43-142'. Empty string if none reported.",
|
| 44 |
+
"title": "Ci Or Range",
|
| 45 |
+
"type": "string"
|
| 46 |
+
},
|
| 47 |
+
"p_value": {
|
| 48 |
+
"default": "",
|
| 49 |
+
"description": "p-value as written, e.g. '0.04', '<0.001'. Empty if not reported.",
|
| 50 |
+
"title": "P Value",
|
| 51 |
+
"type": "string"
|
| 52 |
+
},
|
| 53 |
+
"n_subjects": {
|
| 54 |
+
"anyOf": [
|
| 55 |
+
{
|
| 56 |
+
"type": "integer"
|
| 57 |
+
},
|
| 58 |
+
{
|
| 59 |
+
"type": "null"
|
| 60 |
+
}
|
| 61 |
+
],
|
| 62 |
+
"default": null,
|
| 63 |
+
"description": "Sample size if explicitly mentioned, otherwise null.",
|
| 64 |
+
"title": "N Subjects"
|
| 65 |
+
}
|
| 66 |
+
},
|
| 67 |
+
"required": [
|
| 68 |
+
"metric",
|
| 69 |
+
"value"
|
| 70 |
+
],
|
| 71 |
+
"title": "StatisticExtraction",
|
| 72 |
+
"type": "object"
|
| 73 |
+
}
|
| 74 |
+
},
|
| 75 |
+
"description": "v0.3 — clinical-grade citation-evidence Q&A record.\n\nDiffers from v0.2 only in:\n - StudyDesign adds 'observational' (residual observational bucket)\n - field/prompt wording clarifies it is a fallback, not a default\n - prompt adds WHAT-vs-WHY contrast for evidence_type vs citation_purpose",
|
| 76 |
+
"properties": {
|
| 77 |
+
"relevance_check": {
|
| 78 |
+
"description": "Does the cited paper's title/abstract topically match what the evidence_sentence is citing it for? 'match'=clearly the right ref; 'ambiguous'=hard to tell; 'mismatch'=cited paper is unrelated to the claim. If 'mismatch', set support_strength='insufficient-context', leave question/answer/evidence_quote_from_cited empty, and explain in mismatch_reason.",
|
| 79 |
+
"enum": [
|
| 80 |
+
"match",
|
| 81 |
+
"ambiguous",
|
| 82 |
+
"mismatch"
|
| 83 |
+
],
|
| 84 |
+
"title": "Relevance Check",
|
| 85 |
+
"type": "string"
|
| 86 |
+
},
|
| 87 |
+
"mismatch_reason": {
|
| 88 |
+
"default": "",
|
| 89 |
+
"description": "If relevance_check='mismatch' or 'ambiguous', a 1-sentence explanation of WHY the cited paper does not (or may not) ground the claim — e.g., 'cited paper is about X, but citing context discusses Y'. Empty string when relevance_check='match'.",
|
| 90 |
+
"title": "Mismatch Reason",
|
| 91 |
+
"type": "string"
|
| 92 |
+
},
|
| 93 |
+
"attributed_claim": {
|
| 94 |
+
"description": "What does paper A claim, by virtue of citing paper B at this sentence? This is the proposition A presents as supported by B — NOT a summary of B itself. Be specific. 1-3 sentences. Empty when mismatch.",
|
| 95 |
+
"title": "Attributed Claim",
|
| 96 |
+
"type": "string"
|
| 97 |
+
},
|
| 98 |
+
"citation_purpose": {
|
| 99 |
+
"description": "WHY paper A cites paper B. background=general motivating fact; method=A is using B's technique; result-support=B's finding directly supports A's claim/result; comparison=A compares to B; contrast=A's findings differ from B's; negative=A claims B is wrong/limited.",
|
| 100 |
+
"enum": [
|
| 101 |
+
"background",
|
| 102 |
+
"method",
|
| 103 |
+
"result-support",
|
| 104 |
+
"comparison",
|
| 105 |
+
"contrast",
|
| 106 |
+
"negative",
|
| 107 |
+
"other"
|
| 108 |
+
],
|
| 109 |
+
"title": "Citation Purpose",
|
| 110 |
+
"type": "string"
|
| 111 |
+
},
|
| 112 |
+
"evidence_type": {
|
| 113 |
+
"description": "HOW the cited content is presented. direct-quote=A quotes B verbatim; paraphrase=A summarizes B's claim; statistic=A cites a number from B (if so, fill statistic_extraction); method-citation=A invokes B's protocol/instrument; generic-background=B is one of several refs supporting a generic statement. NEVER use a citation_purpose value (background/result-support/comparison/contrast/negative) here — those describe WHY, not HOW.",
|
| 114 |
+
"enum": [
|
| 115 |
+
"direct-quote",
|
| 116 |
+
"paraphrase",
|
| 117 |
+
"statistic",
|
| 118 |
+
"method-citation",
|
| 119 |
+
"generic-background",
|
| 120 |
+
"other"
|
| 121 |
+
],
|
| 122 |
+
"title": "Evidence Type",
|
| 123 |
+
"type": "string"
|
| 124 |
+
},
|
| 125 |
+
"support_strength": {
|
| 126 |
+
"description": "How well does B's abstract actually support A's attributed_claim? strong=abstract clearly states the claim; moderate=consistent but not the main result; weak=tangential; insufficient-context=cannot tell from inputs alone (use whenever relevance_check != 'match').",
|
| 127 |
+
"enum": [
|
| 128 |
+
"strong",
|
| 129 |
+
"moderate",
|
| 130 |
+
"weak",
|
| 131 |
+
"insufficient-context"
|
| 132 |
+
],
|
| 133 |
+
"title": "Support Strength",
|
| 134 |
+
"type": "string"
|
| 135 |
+
},
|
| 136 |
+
"cited_study_design": {
|
| 137 |
+
"description": "The study design of the CITED paper, inferred from its abstract. Prefer the most specific applicable label: 'rct' / 'non-randomized-trial' / 'prospective-cohort' / 'retrospective-cohort' / 'case-control' / 'cross-sectional' / 'case-series' / 'case-report' / 'systematic-review' / 'meta-analysis' / 'guideline' / 'preclinical'. Use 'observational' ONLY when the cited paper is clearly an observational design (not RCT, not preclinical) AND the abstract does not provide enough information to disambiguate prospective-cohort vs retrospective-cohort vs case-control vs cross-sectional. If the abstract clearly indicates prospective or retrospective ascertainment, USE THE SPECIFIC LABEL — do not default to 'observational'. Use 'preclinical' for animal/in-vitro/translational mechanism studies. Use 'other' only if truly unidentifiable.",
|
| 138 |
+
"enum": [
|
| 139 |
+
"rct",
|
| 140 |
+
"non-randomized-trial",
|
| 141 |
+
"prospective-cohort",
|
| 142 |
+
"retrospective-cohort",
|
| 143 |
+
"case-control",
|
| 144 |
+
"cross-sectional",
|
| 145 |
+
"observational",
|
| 146 |
+
"case-series",
|
| 147 |
+
"case-report",
|
| 148 |
+
"systematic-review",
|
| 149 |
+
"meta-analysis",
|
| 150 |
+
"guideline",
|
| 151 |
+
"preclinical",
|
| 152 |
+
"other"
|
| 153 |
+
],
|
| 154 |
+
"title": "Cited Study Design",
|
| 155 |
+
"type": "string"
|
| 156 |
+
},
|
| 157 |
+
"question": {
|
| 158 |
+
"description": "An open-ended biomedical question that A's citing context implicitly answers using B as evidence. NOT yes/no. Empty when mismatch.",
|
| 159 |
+
"title": "Question",
|
| 160 |
+
"type": "string"
|
| 161 |
+
},
|
| 162 |
+
"answer": {
|
| 163 |
+
"description": "2-4 sentence factual answer grounded in evidence_sentence and/or cited abstract. Cite as [PMID:<pmid>] when stating B's contribution. Empty when mismatch.",
|
| 164 |
+
"title": "Answer",
|
| 165 |
+
"type": "string"
|
| 166 |
+
},
|
| 167 |
+
"answer_grounded_in": {
|
| 168 |
+
"description": "Which input(s) the answer is grounded in. Empty when mismatch.",
|
| 169 |
+
"items": {
|
| 170 |
+
"enum": [
|
| 171 |
+
"evidence_sentence",
|
| 172 |
+
"cited_abstract",
|
| 173 |
+
"both"
|
| 174 |
+
],
|
| 175 |
+
"type": "string"
|
| 176 |
+
},
|
| 177 |
+
"title": "Answer Grounded In",
|
| 178 |
+
"type": "array"
|
| 179 |
+
},
|
| 180 |
+
"evidence_quote_from_cited": {
|
| 181 |
+
"default": "",
|
| 182 |
+
"description": "VERBATIM span copied from the CITED paper's abstract that grounds the answer. MUST be an exact substring of the abstract (we will verify). Pick the single most relevant sentence (or contiguous fragment up to ~50 words). Empty string if relevance_check != 'match' or if evidence_type='generic-background' (no specific quote).",
|
| 183 |
+
"title": "Evidence Quote From Cited",
|
| 184 |
+
"type": "string"
|
| 185 |
+
},
|
| 186 |
+
"statistic_extraction": {
|
| 187 |
+
"anyOf": [
|
| 188 |
+
{
|
| 189 |
+
"$ref": "#/$defs/StatisticExtraction"
|
| 190 |
+
},
|
| 191 |
+
{
|
| 192 |
+
"type": "null"
|
| 193 |
+
}
|
| 194 |
+
],
|
| 195 |
+
"default": null,
|
| 196 |
+
"description": "Required when evidence_type='statistic'. Otherwise null. Extract only what is actually stated in evidence_sentence or cited abstract; do not infer."
|
| 197 |
+
}
|
| 198 |
+
},
|
| 199 |
+
"required": [
|
| 200 |
+
"relevance_check",
|
| 201 |
+
"attributed_claim",
|
| 202 |
+
"citation_purpose",
|
| 203 |
+
"evidence_type",
|
| 204 |
+
"support_strength",
|
| 205 |
+
"cited_study_design",
|
| 206 |
+
"question",
|
| 207 |
+
"answer"
|
| 208 |
+
],
|
| 209 |
+
"title": "CitationQA",
|
| 210 |
+
"type": "object"
|
| 211 |
+
}
|
| 212 |
+
|
| 213 |
+
Example #1 — well-formed MATCH (template only, do NOT copy contents):
|
| 214 |
+
{
|
| 215 |
+
"relevance_check": "match",
|
| 216 |
+
"mismatch_reason": "",
|
| 217 |
+
"attributed_claim": "Standardized caloric requirements for infants with congenital heart disease have not been established, and prior reports show actual enteral intake often falls short of postoperative goals.",
|
| 218 |
+
"citation_purpose": "background",
|
| 219 |
+
"evidence_type": "statistic",
|
| 220 |
+
"support_strength": "strong",
|
| 221 |
+
"cited_study_design": "retrospective-cohort",
|
| 222 |
+
"question": "Why did the authors adopt the Indian RDA caloric target instead of a CHD-specific standard for postoperative infants?",
|
| 223 |
+
"answer": "Because no consensus standard for caloric intake exists for infants with congenital heart disease (CHD); a retrospective cohort of 100 neonates after cardiac surgery [PMID:19116405] found median enteral intake of 93 kcal/kg, with the 100 kcal/kg goal met for only 48.4% of patient-days, indicating that enteral feeding alone is often suboptimal in this population. The authors therefore defaulted to the 2010 Indian RDA target of 92 kcal/kg/day.",
|
| 224 |
+
"answer_grounded_in": [
|
| 225 |
+
"both"
|
| 226 |
+
],
|
| 227 |
+
"evidence_quote_from_cited": "The median caloric intake per day was 93 kcal/kg (range, 43-142). A goal of 100 kcal/kg was achieved for 48.4% of patient days",
|
| 228 |
+
"statistic_extraction": {
|
| 229 |
+
"metric": "median caloric intake per day",
|
| 230 |
+
"value": "93 kcal/kg",
|
| 231 |
+
"ci_or_range": "range 43-142",
|
| 232 |
+
"p_value": "",
|
| 233 |
+
"n_subjects": 100
|
| 234 |
+
}
|
| 235 |
+
}
|
| 236 |
+
|
| 237 |
+
Example #2 — well-formed MISMATCH (template only):
|
| 238 |
+
{
|
| 239 |
+
"relevance_check": "mismatch",
|
| 240 |
+
"mismatch_reason": "Cited paper reports global plastic production and waste statistics, but the citing context discusses interfacial bonding mechanics between microplastics and cement hydration products — topically unrelated.",
|
| 241 |
+
"attributed_claim": "",
|
| 242 |
+
"citation_purpose": "background",
|
| 243 |
+
"evidence_type": "generic-background",
|
| 244 |
+
"support_strength": "insufficient-context",
|
| 245 |
+
"cited_study_design": "other",
|
| 246 |
+
"question": "",
|
| 247 |
+
"answer": "",
|
| 248 |
+
"answer_grounded_in": [],
|
| 249 |
+
"evidence_quote_from_cited": "",
|
| 250 |
+
"statistic_extraction": null
|
| 251 |
+
}
|