Context blindness in small clinical models: same patient, different world

1. At a primary care facility, I saw a woman who had arrived in Nigeria from the United States two weeks earlier. She had a high fever, weakness, headache, nausea and vomiting. Many illnesses can cause these symptoms. But she had recently travelled, had no previous malaria exposure, and malaria is common in Nigeria, so it was our main concern. We started intravenous artesunate before the test results came back, because she was vomiting and could not reliably take tablets, and the tests confirmed malaria. The symptoms alone did not lead to the diagnosis; her context did. I wondered whether AI models adjust their answers to context in the same way. My view is that they often answer for the textbook patient, not the patient in front of them. In my earlier research, I tested models on clinician-validated pairs of cases where only the geographical and economic setting changed. Some errors shrank as models got larger, but others remained, including under-prioritizing treatable diseases in lower-income settings. Context should change the answer: travel made malaria more likely for my patient. Standard benchmarks check whether an answer is correct, not whether it changes correctly when the context changes.

2. I wrote 80 paired test cases. In each pair the medical facts are the same and only one sentence of context differs, so any change in the answer comes from the context. I tested Qwen3-4B-Instruct, and MedGemma-4B (a medical model) for comparison, on three types of context:

   • Resources: 12 emergencies, in a well-equipped hospital and in a rural Nigerian hospital without CT, ICU or blood cultures.
   • Exposure: 8 fever cases, in a Lagos resident, a returned traveller, and a person with no exposure.
   • Athletes: 16 ECG findings, in a professional athlete and a non-athlete, judged against the 2017 international criteria for athletes.

   I reviewed and approved every case and expected answer. Each case was run 11 times, and I report 95% confidence intervals.

   Results:

   • In the rural hospital, 13% of Qwen's plans and 22% of MedGemma's used something stated to be unavailable, such as clot-busting drugs without a brain scan.
   • The models recognized malaria after travel more often than in people who live with it. For a pregnant woman with fever, MedGemma listed malaria in its top three 91% of the time if she had travelled from Lagos, and 9% if she lived there.
   • Both models almost never considered Lassa fever.
   • On athlete ECGs the two models made opposite mistakes. Qwen called a normal athlete finding abnormal every time, and a dangerous long QT interval normal 73% of the time. MedGemma called two warning signs of heart muscle disease normal every time.
   • Medical fine-tuning changed the mistakes but did not remove them.

   The number of cases is small, so these are consistent patterns, not exact rates.

3. I would fix this in three ways:

   • Training data: train on the same kind of pairs used here, each labelled with the correct answer for its context from clinical guidelines and checked by clinicians. Pairs teach the model what difference the context should make.
   • Training rewards: reward the model for answers that are correct in context, using rules that can be checked automatically. For example: does the plan use only the resources available, is the endemic disease in the top three, is the athlete's ECG classified correctly? The same paired test then checks the result.
   • Context as an input: give the model the setting, local disease patterns and patient group as a required structured input, and reject plans that need unavailable resources unless they recommend referral.

   Benchmarks should also report how well models handle these pairs, not just overall accuracy.

Results table

Each case was run with greedy decoding plus 10 samples at temperature 0.7. Brackets are 95% confidence intervals from a bootstrap over case pairs; n is the number of case pairs.

Files

• data/probes_v2.jsonl: the 80 reviewed test cases, with expected answers and scoring rules.
• jobs/run_blindspot_job.py: runs the models and scores the answers.
• results/v2/, results/v2_medgemma/: every model answer and its score, plus summary.json.
• data/probes.jsonl, results/v1/: an earlier run on the test cases before my review. Kept for transparency; all reported numbers come from v2. The v1 results agree with v2 within the confidence intervals.
• Models: Qwen/Qwen3-4B-Instruct-2507 and google/medgemma-4b-it. Linked, not copied.