Card: reviewed release text. HealthBench Hard 39.7 (not 39.8); MedCalc 86.6% is a separate calculator-enabled configuration, not MIA 1; charts from the release package
Browse files- .gitattributes +3 -0
- README.md +89 -26
- charts/mia1-healthbench-hard.png +3 -0
- charts/mia1-healthbench-professional.png +3 -0
- charts/mia1-medcalc-bench.png +3 -0
- mia1-results-16x9.png +0 -0
.gitattributes
CHANGED
|
@@ -34,3 +34,6 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
|
| 34 |
*.zst filter=lfs diff=lfs merge=lfs -text
|
| 35 |
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
| 36 |
mia1-hero-16x9.png filter=lfs diff=lfs merge=lfs -text
|
|
|
|
|
|
|
|
|
|
|
|
| 34 |
*.zst filter=lfs diff=lfs merge=lfs -text
|
| 35 |
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
| 36 |
mia1-hero-16x9.png filter=lfs diff=lfs merge=lfs -text
|
| 37 |
+
charts/mia1-healthbench-professional.png filter=lfs diff=lfs merge=lfs -text
|
| 38 |
+
charts/mia1-healthbench-hard.png filter=lfs diff=lfs merge=lfs -text
|
| 39 |
+
charts/mia1-medcalc-bench.png filter=lfs diff=lfs merge=lfs -text
|
README.md
CHANGED
|
@@ -1,54 +1,117 @@
|
|
| 1 |
---
|
| 2 |
-
license: other
|
| 3 |
-
license_name: mia-1-research-release
|
| 4 |
-
pipeline_tag: text-generation
|
| 5 |
language:
|
| 6 |
- en
|
| 7 |
tags:
|
| 8 |
- medical
|
| 9 |
- clinical-reasoning
|
| 10 |
-
-
|
| 11 |
-
- medcalc-bench
|
| 12 |
-
- qwen3.8
|
| 13 |
---
|
| 14 |
|
| 15 |
# MIA 1, Medical Intelligent Agent
|
| 16 |
|
| 17 |
**Dr. Max Health** · Research release · October 2026
|
| 18 |
|
| 19 |
-
|
| 20 |
|
| 21 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 22 |
|
| 23 |
## Highlights
|
| 24 |
|
| 25 |
-
- **
|
| 26 |
-
|
| 27 |
-
- **
|
| 28 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 29 |
|
| 30 |
-
|
|
|
|
| 31 |
|
| 32 |
-
|
|
|
|
| 33 |
|
| 34 |
-
|
| 35 |
-
|---|---|---|---|
|
| 36 |
-
| HealthBench Professional | **54.1** | 41.3 | 65.1 |
|
| 37 |
-
| HealthBench Hard (100-question sample) | **39.8** | 37.6 | 43.8 |
|
| 38 |
-
| MedCalc-Bench (accuracy %) | **86.6** (with calculator tools) | 75.7 | 88.2 |
|
| 39 |
|
| 40 |
-
|
|
|
|
|
|
|
|
|
|
| 41 |
|
| 42 |
-
|
| 43 |
|
| 44 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 45 |
|
| 46 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 47 |
|
| 48 |
## Access
|
| 49 |
|
| 50 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 51 |
|
| 52 |
-
##
|
| 53 |
|
| 54 |
-
|
|
|
|
| 1 |
---
|
|
|
|
|
|
|
|
|
|
| 2 |
language:
|
| 3 |
- en
|
| 4 |
tags:
|
| 5 |
- medical
|
| 6 |
- clinical-reasoning
|
| 7 |
+
- evaluation
|
|
|
|
|
|
|
| 8 |
---
|
| 9 |
|
| 10 |
# MIA 1, Medical Intelligent Agent
|
| 11 |
|
| 12 |
**Dr. Max Health** · Research release · October 2026
|
| 13 |
|
| 14 |
+
MIA 1 is a research model, not a medical device; not for diagnosis or treatment decisions.
|
| 15 |
|
| 16 |
+
## What is released
|
| 17 |
+
|
| 18 |
+
This page is a model card only. **No model weights, adapters or other model files are published here.**
|
| 19 |
+
|
| 20 |
+
MIA 1 is not a fine-tuned checkpoint. It runs the unmodified open Qwen3.8-27B model with Dr. Max Health's clinical
|
| 21 |
+
reasoning harness, applied at inference time:
|
| 22 |
+
|
| 23 |
+
- instructions written for clinicians, and
|
| 24 |
+
- an answer step that drafts, checks the draft against what a complete answer needs, and tightens it.
|
| 25 |
+
|
| 26 |
+
The model's weights are not changed. The harness is not part of this release.
|
| 27 |
|
| 28 |
## Highlights
|
| 29 |
|
| 30 |
+
- **Built for clinicians.** 54.1 on HealthBench Professional, up from 41.3 for Qwen3.8-27B on its own, with answers
|
| 31 |
+
less than half as long on average. Claude Opus 5.5, run by us on the same questions, scores 65.1.
|
| 32 |
+
- **Hard cases.** 39.7 on a 100-question sample of HealthBench Hard, against 37.6 for Qwen3.8-27B and 43.8 for
|
| 33 |
+
Claude Opus 5.5.
|
| 34 |
+
- **Compact.** 27 billion parameters.
|
| 35 |
+
- **Clinical calculations.** Separately, our Qwen-based calculator-enabled configuration scored 86.6% on MedCalc-Bench
|
| 36 |
+
Verified (75.7% without tools; Claude Opus 5.5 88.2% without tools). It is a different configuration from the MIA 1
|
| 37 |
+
setup evaluated on HealthBench.
|
| 38 |
+
|
| 39 |
+
## Intended use
|
| 40 |
|
| 41 |
+
MIA 1 is released for research and evaluation. It is not yet a certified medical device; we are working toward
|
| 42 |
+
clinical decision support that meets medical-device requirements.
|
| 43 |
|
| 44 |
+
Intended users are researchers and partner institutions evaluating clinical reasoning systems together with
|
| 45 |
+
Dr. Max Health.
|
| 46 |
|
| 47 |
+
## Out-of-scope use
|
|
|
|
|
|
|
|
|
|
|
|
|
| 48 |
|
| 49 |
+
- Diagnosis, treatment, triage or any other decision about a real patient.
|
| 50 |
+
- Medical advice to patients or the public.
|
| 51 |
+
- Use as, or inside, a medical device or a clinical decision support system in patient care.
|
| 52 |
+
- Emergencies, or any setting where an unchecked answer could harm a person.
|
| 53 |
|
| 54 |
+
## Evaluation
|
| 55 |
|
| 56 |
+
All models were run by Dr. Max Health on the same questions.
|
| 57 |
+
|
| 58 |
+
**HealthBench** (length-adjusted score, 0-100; mean of two model graders)
|
| 59 |
+
|
| 60 |
+
| Model | HealthBench Professional (325 held out) | HealthBench Hard (100-question sample) |
|
| 61 |
+
|---|---|---|
|
| 62 |
+
| MIA 1 | 54.1 | 39.7 |
|
| 63 |
+
| Claude Opus 5.5 | 65.1 | 43.8 |
|
| 64 |
+
| Qwen3.8-27B | 41.3 | 37.6 |
|
| 65 |
|
| 66 |
+
**MedCalc-Bench Verified** (accuracy on 1,100 test questions, one scoring rule for every model). The first row is a
|
| 67 |
+
separate calculator-enabled configuration, not the MIA 1 setup evaluated on HealthBench.
|
| 68 |
+
|
| 69 |
+
| Configuration | MedCalc-Bench Verified |
|
| 70 |
+
|---|---|
|
| 71 |
+
| Qwen3.8-27B + calculator tools | 86.6% |
|
| 72 |
+
| Claude Opus 5.5, no tools | 88.2% |
|
| 73 |
+
| Qwen3.8-27B, no tools | 75.7% |
|
| 74 |
+
|
| 75 |
+

|
| 76 |
+
|
| 77 |
+

|
| 78 |
+
|
| 79 |
+

|
| 80 |
+
|
| 81 |
+
### How we evaluated
|
| 82 |
+
|
| 83 |
+
HealthBench Professional (325 examples held out from our development selection) and HealthBench Hard (100-question
|
| 84 |
+
sample): scores are length-adjusted following the HealthBench Professional paper and are the mean of two blind model
|
| 85 |
+
graders (Qwen 27B and Claude Opus 5.5) using the official rubrics; per-grader and raw scores will be in the technical
|
| 86 |
+
report. MedCalc-Bench Verified (1,100 test items): one pre-registered scoring rule for every model; the
|
| 87 |
+
calculator-enabled Qwen3.8-27B configuration uses calculator tools, and its score and the score of Qwen3.8-27B without
|
| 88 |
+
tools are averages over 8 answers per question; Claude Opus 5.5 answers once per question, without tools. All models
|
| 89 |
+
were run by Dr. Max Health on the same questions. These internal results are not comparable to official HealthBench
|
| 90 |
+
or MedCalc-Bench numbers.
|
| 91 |
+
|
| 92 |
+
## Limitations
|
| 93 |
+
|
| 94 |
+
- These are benchmark results, not clinical validation. MIA 1 has not been clinically validated.
|
| 95 |
+
- Claude Opus 5.5 scores higher than MIA 1 on both HealthBench benchmarks (mean of the two graders).
|
| 96 |
+
- HealthBench Hard is a 100-question sample, so small differences there are within noise.
|
| 97 |
+
- The two graders are models, not clinicians. One grader shares MIA 1's base model; the other is Claude Opus 5.5,
|
| 98 |
+
which is also one of the compared models.
|
| 99 |
+
- MIA 1 inherits the limits of Qwen3.8-27B: it can give confident wrong answers, miss recent guidance and reflect
|
| 100 |
+
biases in its training data. Shorter answers can leave out details a clinician would want.
|
| 101 |
+
- We have not evaluated MIA 1 outside the benchmarks listed here.
|
| 102 |
|
| 103 |
## Access
|
| 104 |
|
| 105 |
+
No weights are published. Hospitals, clinics and research partners: contact us via drmaxhealth.de.
|
| 106 |
+
|
| 107 |
+
## Licence and attribution
|
| 108 |
+
|
| 109 |
+
Built on Qwen3.8-27B.
|
| 110 |
+
|
| 111 |
+
Qwen3.8-27B is released by the Qwen team under the Apache License 2.0
|
| 112 |
+
([Qwen/Qwen3.8-27B](https://huggingface.co/Qwen/Qwen3.8-27B)). This page redistributes no Qwen3.8-27B weights or
|
| 113 |
+
files. The Qwen name identifies the base model only and does not imply endorsement by the Qwen team.
|
| 114 |
|
| 115 |
+
## Technical report
|
| 116 |
|
| 117 |
+
A technical report will follow.
|
charts/mia1-healthbench-hard.png
ADDED
|
Git LFS Details
|
charts/mia1-healthbench-professional.png
ADDED
|
Git LFS Details
|
charts/mia1-medcalc-bench.png
ADDED
|
Git LFS Details
|
mia1-results-16x9.png
DELETED
|
Binary file (79.6 kB)
|
|
|