DrMaxHealth commited on
Commit
4eace5a
·
verified ·
1 Parent(s): 7c62cf2

Card: reviewed release text. HealthBench Hard 39.7 (not 39.8); MedCalc 86.6% is a separate calculator-enabled configuration, not MIA 1; charts from the release package

Browse files
.gitattributes CHANGED
@@ -34,3 +34,6 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
36
  mia1-hero-16x9.png filter=lfs diff=lfs merge=lfs -text
 
 
 
 
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
36
  mia1-hero-16x9.png filter=lfs diff=lfs merge=lfs -text
37
+ charts/mia1-healthbench-professional.png filter=lfs diff=lfs merge=lfs -text
38
+ charts/mia1-healthbench-hard.png filter=lfs diff=lfs merge=lfs -text
39
+ charts/mia1-medcalc-bench.png filter=lfs diff=lfs merge=lfs -text
README.md CHANGED
@@ -1,54 +1,117 @@
1
  ---
2
- license: other
3
- license_name: mia-1-research-release
4
- pipeline_tag: text-generation
5
  language:
6
  - en
7
  tags:
8
  - medical
9
  - clinical-reasoning
10
- - healthbench
11
- - medcalc-bench
12
- - qwen3.8
13
  ---
14
 
15
  # MIA 1, Medical Intelligent Agent
16
 
17
  **Dr. Max Health** · Research release · October 2026
18
 
19
- ![MIA 1](mia1-hero-16x9.png)
20
 
21
- MIA 1 is a 27B clinical reasoning model built on Qwen3.8-27B with Dr. Max Health's clinical reasoning harness and medical tools. It is our first step toward AI that reasons through real clinical cases at the level of an experienced doctor.
 
 
 
 
 
 
 
 
 
 
22
 
23
  ## Highlights
24
 
25
- - **Clinical calculations:** 86.6% on MedCalc-Bench with its medical tools, within 2 points of Claude Opus 5.5.
26
- - **Hard cases:** within 4 points of Claude Opus 5.5 on HealthBench Hard.
27
- - **Built for clinicians:** 54.1 vs 41.3 for its base model on HealthBench Professional, with answers half as long.
28
- - **Efficient:** 27B parameters, running on a single GPU node.
 
 
 
 
 
 
29
 
30
- ## Results
 
31
 
32
- ![MIA 1 results](mia1-results-16x9.png)
 
33
 
34
- | Benchmark | MIA 1 | Qwen3.8-27B | Claude Opus 5.5 |
35
- |---|---|---|---|
36
- | HealthBench Professional | **54.1** | 41.3 | 65.1 |
37
- | HealthBench Hard (100-question sample) | **39.8** | 37.6 | 43.8 |
38
- | MedCalc-Bench (accuracy %) | **86.6** (with calculator tools) | 75.7 | 88.2 |
39
 
40
- ### How we evaluated
 
 
 
41
 
42
- HealthBench Professional (325 held-out examples) and HealthBench Hard (100-question sample): scores are length-adjusted following the HealthBench Professional paper and are the mean of two blind model graders (Qwen 27B and Claude Opus 5.5) using the official rubrics; per-grader and raw scores are in the technical report. MedCalc-Bench (1,100 test items): scored by exact answer with the same rule for every model; MIA 1 uses its calculator tools and its score is the average over 8 answers per question, Claude Opus 5.5 and Qwen3.8-27B answer without tools (Opus 1 answer per question). All models were run by Dr. Max Health on the same questions. These internal results are not comparable to official HealthBench leaderboard numbers. Technical report to follow.
43
 
44
- ## Intended use
 
 
 
 
 
 
 
 
45
 
46
- MIA 1 is released for research and evaluation. It is not yet a certified medical device; we are working toward clinical decision support that meets medical-device requirements.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
47
 
48
  ## Access
49
 
50
- Weights are not public. For research or clinical partnerships, contact us via [drmaxhealth.de](https://drmaxhealth.de).
 
 
 
 
 
 
 
 
51
 
52
- ## Attribution
53
 
54
- Built on Qwen3.8-27B by the Qwen team (Alibaba Cloud), used under its license.
 
1
  ---
 
 
 
2
  language:
3
  - en
4
  tags:
5
  - medical
6
  - clinical-reasoning
7
+ - evaluation
 
 
8
  ---
9
 
10
  # MIA 1, Medical Intelligent Agent
11
 
12
  **Dr. Max Health** · Research release · October 2026
13
 
14
+ MIA 1 is a research model, not a medical device; not for diagnosis or treatment decisions.
15
 
16
+ ## What is released
17
+
18
+ This page is a model card only. **No model weights, adapters or other model files are published here.**
19
+
20
+ MIA 1 is not a fine-tuned checkpoint. It runs the unmodified open Qwen3.8-27B model with Dr. Max Health's clinical
21
+ reasoning harness, applied at inference time:
22
+
23
+ - instructions written for clinicians, and
24
+ - an answer step that drafts, checks the draft against what a complete answer needs, and tightens it.
25
+
26
+ The model's weights are not changed. The harness is not part of this release.
27
 
28
  ## Highlights
29
 
30
+ - **Built for clinicians.** 54.1 on HealthBench Professional, up from 41.3 for Qwen3.8-27B on its own, with answers
31
+ less than half as long on average. Claude Opus 5.5, run by us on the same questions, scores 65.1.
32
+ - **Hard cases.** 39.7 on a 100-question sample of HealthBench Hard, against 37.6 for Qwen3.8-27B and 43.8 for
33
+ Claude Opus 5.5.
34
+ - **Compact.** 27 billion parameters.
35
+ - **Clinical calculations.** Separately, our Qwen-based calculator-enabled configuration scored 86.6% on MedCalc-Bench
36
+ Verified (75.7% without tools; Claude Opus 5.5 88.2% without tools). It is a different configuration from the MIA 1
37
+ setup evaluated on HealthBench.
38
+
39
+ ## Intended use
40
 
41
+ MIA 1 is released for research and evaluation. It is not yet a certified medical device; we are working toward
42
+ clinical decision support that meets medical-device requirements.
43
 
44
+ Intended users are researchers and partner institutions evaluating clinical reasoning systems together with
45
+ Dr. Max Health.
46
 
47
+ ## Out-of-scope use
 
 
 
 
48
 
49
+ - Diagnosis, treatment, triage or any other decision about a real patient.
50
+ - Medical advice to patients or the public.
51
+ - Use as, or inside, a medical device or a clinical decision support system in patient care.
52
+ - Emergencies, or any setting where an unchecked answer could harm a person.
53
 
54
+ ## Evaluation
55
 
56
+ All models were run by Dr. Max Health on the same questions.
57
+
58
+ **HealthBench** (length-adjusted score, 0-100; mean of two model graders)
59
+
60
+ | Model | HealthBench Professional (325 held out) | HealthBench Hard (100-question sample) |
61
+ |---|---|---|
62
+ | MIA 1 | 54.1 | 39.7 |
63
+ | Claude Opus 5.5 | 65.1 | 43.8 |
64
+ | Qwen3.8-27B | 41.3 | 37.6 |
65
 
66
+ **MedCalc-Bench Verified** (accuracy on 1,100 test questions, one scoring rule for every model). The first row is a
67
+ separate calculator-enabled configuration, not the MIA 1 setup evaluated on HealthBench.
68
+
69
+ | Configuration | MedCalc-Bench Verified |
70
+ |---|---|
71
+ | Qwen3.8-27B + calculator tools | 86.6% |
72
+ | Claude Opus 5.5, no tools | 88.2% |
73
+ | Qwen3.8-27B, no tools | 75.7% |
74
+
75
+ ![HealthBench Professional, length-adjusted score on 325 questions held out from development, scale 0-100: MIA 1 54.1, Claude Opus 5.5 65.1, Qwen3.8-27B 41.3.](charts/mia1-healthbench-professional.png)
76
+
77
+ ![HealthBench Hard, length-adjusted score on a 100-question sample, scale 0-100: MIA 1 39.7, Claude Opus 5.5 43.8, Qwen3.8-27B 37.6.](charts/mia1-healthbench-hard.png)
78
+
79
+ ![MedCalc-Bench Verified, accuracy on 1,100 test questions under one pre-registered scoring rule, scale 0-100 %: Qwen3.8-27B + calculator tools 86.6 %, Claude Opus 5.5 without tools 88.2 %, Qwen3.8-27B without tools 75.7 %.](charts/mia1-medcalc-bench.png)
80
+
81
+ ### How we evaluated
82
+
83
+ HealthBench Professional (325 examples held out from our development selection) and HealthBench Hard (100-question
84
+ sample): scores are length-adjusted following the HealthBench Professional paper and are the mean of two blind model
85
+ graders (Qwen 27B and Claude Opus 5.5) using the official rubrics; per-grader and raw scores will be in the technical
86
+ report. MedCalc-Bench Verified (1,100 test items): one pre-registered scoring rule for every model; the
87
+ calculator-enabled Qwen3.8-27B configuration uses calculator tools, and its score and the score of Qwen3.8-27B without
88
+ tools are averages over 8 answers per question; Claude Opus 5.5 answers once per question, without tools. All models
89
+ were run by Dr. Max Health on the same questions. These internal results are not comparable to official HealthBench
90
+ or MedCalc-Bench numbers.
91
+
92
+ ## Limitations
93
+
94
+ - These are benchmark results, not clinical validation. MIA 1 has not been clinically validated.
95
+ - Claude Opus 5.5 scores higher than MIA 1 on both HealthBench benchmarks (mean of the two graders).
96
+ - HealthBench Hard is a 100-question sample, so small differences there are within noise.
97
+ - The two graders are models, not clinicians. One grader shares MIA 1's base model; the other is Claude Opus 5.5,
98
+ which is also one of the compared models.
99
+ - MIA 1 inherits the limits of Qwen3.8-27B: it can give confident wrong answers, miss recent guidance and reflect
100
+ biases in its training data. Shorter answers can leave out details a clinician would want.
101
+ - We have not evaluated MIA 1 outside the benchmarks listed here.
102
 
103
  ## Access
104
 
105
+ No weights are published. Hospitals, clinics and research partners: contact us via drmaxhealth.de.
106
+
107
+ ## Licence and attribution
108
+
109
+ Built on Qwen3.8-27B.
110
+
111
+ Qwen3.8-27B is released by the Qwen team under the Apache License 2.0
112
+ ([Qwen/Qwen3.8-27B](https://huggingface.co/Qwen/Qwen3.8-27B)). This page redistributes no Qwen3.8-27B weights or
113
+ files. The Qwen name identifies the base model only and does not imply endorsement by the Qwen team.
114
 
115
+ ## Technical report
116
 
117
+ A technical report will follow.
charts/mia1-healthbench-hard.png ADDED

Git LFS Details

  • SHA256: 22b553b5066728ce4f22575089db1731257cded943c6da3241a69299f4b6b54d
  • Pointer size: 131 Bytes
  • Size of remote file: 206 kB
charts/mia1-healthbench-professional.png ADDED

Git LFS Details

  • SHA256: 2abf3eef9037d2e850996f54d1d4154ce55c62ab51fa268916c2340193e9be29
  • Pointer size: 131 Bytes
  • Size of remote file: 211 kB
charts/mia1-medcalc-bench.png ADDED

Git LFS Details

  • SHA256: 3b991b762b4e1cf13d1021c0f69e4dde4d414225133f09142f582fb44aebf1e7
  • Pointer size: 131 Bytes
  • Size of remote file: 251 kB
mia1-results-16x9.png DELETED
Binary file (79.6 kB)