Score it on your dataRun a second seed

medgemma-decide-4b-v2

A typed decision model continued from medgemma-decide-4b (v1). Same interface as v1: it reads a medical text and a set of typed questions, and returns a probability for every option in one forward pass. It doesn't generate text.

Not a medical device. Not for clinical decisions, diagnosis, triage or patient care. Research use only, under the HAI-DEF terms (see NOTICE).

What changed from v1

Base	v1's released merged checkpoint (Amaljith/medgemma-decide-4b)
Method	Fresh LoRA r=16, alpha=16, one epoch, lr 5e-5 (head 1e-4), batch 16x2, seed 3407
Data	The same 22,132-row train split plus 1,156 oversampled rows: minority classes of the five skewed yes/no questions oversampled to ~35% (medline/adverse, group_infants, group_children, group_adolescents, drug_therapy)
Temperatures	Refit per question on the calib split (2963 decisions); shipped in decision_format.json
Training	728 steps in 51.7 min on an A100-80GB, Unsloth 2026.10.8, Gemma turn format
Eval	Test split (15,320 records, 19,480 decisions) scored with eval.py from Amaljith/medical-typed-decisions

Motivation: v1 failed its release gate on medline/adverse — 92% of its training rows answer "no", so the model rarely said "yes" and balanced accuracy (0.625) fell below the TF-IDF baseline (0.721). Oversampling the minority class was the fix; balanced accuracy now clears the floor.

Results on the medical-typed-decisions test set

Balanced accuracy uses the prior-adjusted decision (training answer shares divided out), the same rule as the v1 card. v1 numbers are the 4-bit adapter scores from the v1 card.

Question	n	Balanced acc. v1	Balanced acc. v2	Accuracy v2	ECE v2	Gate
medline/adverse	1500	0.625	0.825	0.903	0.026	pass
medline/animals	1500	0.896	0.918	0.927	0.021	pass
medline/design	605	0.796	0.876	0.945	0.037	(not gated)
medline/disease_area	314	0.952	0.572	0.943	0.028	(not gated)
medline/drug_therapy	1500	0.791	0.886	0.914	0.017	pass
medline/group_adolescents	356	0.668	0.774	0.862	0.049	pass
medline/group_adults	313	0.823	0.833	0.818	0.051	pass
medline/group_children	322	0.881	0.917	0.935	0.050	pass
medline/group_female	304	0.828	0.822	0.826	0.060	pass
medline/group_infants	308	0.750	0.940	0.984	0.023	(not gated)
medline/group_male	335	0.869	0.866	0.866	0.060	pass
medline/group_older_adults	316	0.835	0.831	0.864	0.077	pass
medline/humans	1500	0.912	0.925	0.932	0.011	pass
medmcqa/answer	4162	0.508	0.507	0.509	0.080	pass
medmcqa/subject	4160	0.677	0.717	0.687	0.027	(not gated)
medqa/answer	1273	0.602	0.609	0.611	0.052	pass
pubmedqa/answer	500	0.578	0.488	0.738	0.054	pass
symptoms/diagnosis	212	0.855	0.876	0.877	0.066	(not gated)

Macro over sources (balanced accuracy): 0.686 (v1: 0.689). Release gate: PASS (v1: FAILED on medline/adverse).

Honest regressions

• medline/disease_area balanced accuracy fell 0.952 → 0.572 while accuracy barely moved (0.949 → 0.943). The question is not gated (several of its ~19 classes have 1-4 test rows), and the prior-adjusted decision swings on those tiny classes. Treat v2 as equal to v1 there, not worse.
• pubmedqa/answer fell 0.578 → 0.488 (plain-argmax balanced accuracy 0.561, close to v1's 0.578; the prior-adjusted rule hurts this question in v2). Still passes the gate.
• Exam knowledge did not move (medmcqa/answer 0.507, medqa/answer 0.609): one continued epoch does not add medical knowledge, it fixes decision balance. For the exam questions, a fresh multi-epoch run or a bigger data mixture would be the next lever.

Use

Same layout as v1: merged weights and the joint schema head at the root, loaded with transformers by joint_schema_model.py; the scored adapter with predict.py and gemma_format.py in adapter/.

import importlib.util, sys
from huggingface_hub import hf_hub_download
spec = importlib.util.spec_from_file_location("predict", hf_hub_download("Amaljith/medgemma-decide-4b-v2", "adapter/predict.py"))
predict = importlib.util.module_from_spec(spec); spec.loader.exec_module(predict)
model, tokenizer, fmt = predict.load("Amaljith/medgemma-decide-4b-v2")
predict.predict(model, tokenizer, fmt, state, questions, source="medqa")

predict.py applies the prompt wrapper the model was trained with (Gemma turn format) and the per-question temperatures in decision_format.json (this run's, refit on the calib split).

Limits

Inherits all of v1's limits (see the v1 card): MeSH labels are not clinical ground truth, exam sources are old public sets, text only, English, and the model always splits probability across the options given — include the right answer among them. Seed-to-seed variance is still unmeasured (one seed per run; "Run a second seed" above starts that).