Image-Text-to-Text
Transformers
Safetensors
qwen3_5
openmed
medical
clinical
medqa
weight-space-programming
conversational
Eval Results (legacy)
Instructions to use OpenMed/Qwen3.5-0.8B-WSP-Math-Clinical with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use OpenMed/Qwen3.5-0.8B-WSP-Math-Clinical with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("image-text-to-text", model="OpenMed/Qwen3.5-0.8B-WSP-Math-Clinical") messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] pipe(text=messages)# pip install -U transformers accelerate # Load model directly from transformers import AutoProcessor, AutoModelForMultimodalLM processor = AutoProcessor.from_pretrained("OpenMed/Qwen3.5-0.8B-WSP-Math-Clinical") model = AutoModelForMultimodalLM.from_pretrained("OpenMed/Qwen3.5-0.8B-WSP-Math-Clinical", device_map="auto") messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] inputs = processor.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=256) print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use OpenMed/Qwen3.5-0.8B-WSP-Math-Clinical with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "OpenMed/Qwen3.5-0.8B-WSP-Math-Clinical" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "OpenMed/Qwen3.5-0.8B-WSP-Math-Clinical", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker
docker model run hf.co/OpenMed/Qwen3.5-0.8B-WSP-Math-Clinical
- SGLang
How to use OpenMed/Qwen3.5-0.8B-WSP-Math-Clinical with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "OpenMed/Qwen3.5-0.8B-WSP-Math-Clinical" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "OpenMed/Qwen3.5-0.8B-WSP-Math-Clinical", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "OpenMed/Qwen3.5-0.8B-WSP-Math-Clinical" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "OpenMed/Qwen3.5-0.8B-WSP-Math-Clinical", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }' - Docker Model Runner
How to use OpenMed/Qwen3.5-0.8B-WSP-Math-Clinical with Docker Model Runner:
docker model run hf.co/OpenMed/Qwen3.5-0.8B-WSP-Math-Clinical
|
Download README.md from OpenMed/Qwen3.5-0.8B-WSP-Math-Clinical: direct link, hf CLI and curl.
- Browser
- Download file 15.9 kB
-
https://huggingface.co/OpenMed/Qwen3.5-0.8B-WSP-Math-Clinical/resolve/main/README.md
- Command line
-
hf download hf://OpenMed/Qwen3.5-0.8B-WSP-Math-Clinical/README.md
-
curl -L -o README.md https://huggingface.co/OpenMed/Qwen3.5-0.8B-WSP-Math-Clinical/resolve/main/README.md
15.9 kB
| license: apache-2.0 | |
| base_model: Qwen/Qwen3.5-0.8B | |
| library_name: transformers | |
| tags: | |
| - openmed | |
| - medical | |
| - clinical | |
| - medqa | |
| - weight-space-programming | |
| - qwen3_5 | |
| datasets: | |
| - GBaker/MedQA-USMLE-4-options | |
| - openlifescienceai/medmcqa | |
| - cais/mmlu | |
| model-index: | |
| - name: Qwen3.5-0.8B-WSP-Math-Clinical | |
| results: | |
| - task: | |
| type: question-answering | |
| name: Clinical multiple-choice questions | |
| dataset: | |
| name: MedQA-USMLE (4 options) | |
| type: GBaker/MedQA-USMLE-4-options | |
| split: test | |
| metrics: | |
| - type: accuracy | |
| value: 45.4 | |
| name: Accuracy (thinking on, one sample, seed 0) | |
| <p align="center"> | |
| <a href="https://huggingface.co/OpenMed"><img src="https://cdn-avatars.huggingface.co/v1/production/uploads/5fd5e18a90b6dc4633f6d292/qWeFUOCMgN4q-ET102lRZ.png" alt="OpenMed" width="110"></a> | |
| </p> | |
| <h1 align="center">OpenMed Qwen3.5-0.8B WSP Math + Clinical</h1> | |
| <p align="center"> | |
| <a href="https://huggingface.co/OpenMed"><img alt="OpenMed on Hugging Face" src="https://img.shields.io/badge/OpenMed-open%20models-ff9d00"></a> | |
| <a href="https://github.com/maziyarpanahi/openmed"><img alt="OpenMed SDK on GitHub" src="https://img.shields.io/github/stars/maziyarpanahi/openmed?label=OpenMed%20SDK&logo=github"></a> | |
| <img alt="License Apache-2.0" src="https://img.shields.io/badge/license-Apache--2.0-blue"> | |
| <img alt="No training" src="https://img.shields.io/badge/training-none-3DDC97"> | |
| </p> | |
| **OpenMed's clinical weight edit of Qwen3.5-0.8B: two weight edits, no training, and the model answers medical exam questions in the format they ask for.** On the 1,273 MedQA-USMLE test questions it scores **45.4%**, against | |
| 24.4% for Qwen3.5-0.8B as released (paired difference +21.0 points, 95% CI | |
| +18.1 to +23.8, McNemar p = 1.6e-42). No gradient descent, no | |
| fine-tuning, no training data: a frontier model read the weights in a Python kernel and changed them by hand. It is Qwen3.5-0.8B with the published math edit from "Sparks of Weight-Space Programming" (which stops its thinking loops) plus one OpenMed clinical edit to a single attention head. | |
| | Developed by | [OpenMed](https://huggingface.co/OpenMed) | | |
| |---|---| | |
| | Base model | [Qwen/Qwen3.5-0.8B](https://huggingface.co/Qwen/Qwen3.5-0.8B) @ `2fc06364715b967f1860aea9cf38778875588b17` | | |
| | What changed | Three tensors: the paper's math edit (MLP down_proj of layers 15 and 19, 64 output columns each, 131,072 weights) and OpenMed's clinical edit (the 256 output columns of attention head 4 in layer 23, o_proj, 262,144 weights) | | |
| | Method | Weight-space programming (Takehi, Teoh, Zhang and Khattab, MIT CSAIL, 2026): a frontier LLM reads the weights in a Python kernel and edits them directly | | |
| | Editor of the clinical edit | a frontier LLM, 21 forward passes, 148 Python runs, 206 turns | | |
| | Practice data the editor could use | training splits only: MedQA-USMLE train (10,178) and 10,000 MedMCQA train questions | | |
| | License | Apache-2.0 (see Credits for the inputs and their licenses) | | |
| | Intended use | research on model editing and evaluation; **not for clinical use** | | |
| ## About OpenMed | |
| OpenMed builds open models and open-source tools for clinical and biomedical text that run on your own hardware. | |
| - **[Open models on Hugging Face](https://huggingface.co/OpenMed)** for clinical NER, de-identification and more. | |
| - **[The OpenMed Python SDK](https://github.com/maziyarpanahi/openmed)**, Apache-2.0: `pip install openmed`. | |
| - Follow [OpenMed on Hugging Face](https://huggingface.co/OpenMed) and [@OpenMed_AI on X](https://x.com/OpenMed_AI). | |
| ## Results | |
| Same model architecture, tokenizer, chat template and sampling for every row; only weight values differ. One sample per | |
| question with seed 0 (MedQA also with seeds 1 and 2). Thinking on, temperature 0.6, top_p 0.95, top_k 20, presence | |
| penalty 1.5, at most 32,768 new tokens (the paper's evaluation protocol). The prompt is OpenMed's fixed MedQA exam prompt | |
| (below), and an answer is graded from the text after `</think>`: its last `Answer: X` line, X in A to D. Answers cut at the | |
| token limit and answers with no gradable line count as wrong, as in the paper. | |
| | Checkpoint | MedQA (seed 0) | MedQA, median of 3 seeds (range) | MedMCQA | MMLU medical | MedQA answers cut at 32,768 tokens | MedQA answers with no gradable line | | |
| |---|---|---|---|---|---|---| | |
| | **OpenMed WSP Math + Clinical 0.8B** (this model) | 45.4% | 45.4% (44.1% to 47.0%) | 38.8% | 60.2% | 0.2% | 0.3% | | |
| | Qwen3.5-0.8B + the paper's math edit (before OpenMed's clinical edit) | 32.9% | 32.9% (32.8% to 33.6%) | 33.3% | 52.4% | 0.1% | 28.8% | | |
| | Random patch, same size, random direction (seed 1) | 32.4% | n/a | 33.7% | 50.3% | 0.1% | 31.7% | | |
| | Random patch, same size, random direction (seed 2) | 34.0% | n/a | 33.1% | 50.9% | 0.1% | 29.1% | | |
| | Qwen3.5-0.8B as released | 24.4% | 23.1% (22.1% to 24.4%) | 30.6% | 43.4% | 18.3% | 32.9% | | |
| Paired tests on the same questions (seed 0; exact McNemar on the questions only one checkpoint got right; bootstrap 95% | |
| CI over questions, 10,000 resamples): | |
| | Comparison | Benchmark | n | Before | After | Difference (95% CI) | Only before right / only after right | McNemar p | | |
| |---|---|---|---|---|---|---|---| | |
| | Qwen3.5-0.8B -> WSP Math + Clinical 0.8B | MedQA-USMLE (4 options) | 1,273 | 24.4% | 45.4% | +21.0 pts (+18.1 to +23.8) | 72 / 339 | 1.6e-42 | | |
| | Qwen3.5-0.8B -> WSP Math + Clinical 0.8B | MedMCQA | 1,000 | 30.6% | 38.8% | +8.2 pts (+5.0 to +11.4) | 98 / 180 | 1.0e-06 | | |
| | Qwen3.5-0.8B -> WSP Math + Clinical 0.8B | MMLU | 1,089 | 43.4% | 60.2% | +16.7 pts (+13.6 to +19.8) | 72 / 254 | 6.1e-25 | | |
| | + math edit -> WSP Math + Clinical 0.8B | MedQA-USMLE (4 options) | 1,273 | 32.9% | 45.4% | +12.5 pts (+9.7 to +15.3) | 102 / 261 | 3.4e-17 | | |
| | + math edit -> WSP Math + Clinical 0.8B | MedMCQA | 1,000 | 33.3% | 38.8% | +5.5 pts (+2.4 to +8.5) | 98 / 153 | 6.2e-04 | | |
| | + math edit -> WSP Math + Clinical 0.8B | MMLU | 1,089 | 52.4% | 60.2% | +7.7 pts (+5.0 to +10.5) | 76 / 160 | 4.8e-08 | | |
| | Qwen3.5-0.8B -> + math edit | MedQA-USMLE (4 options) | 1,273 | 24.4% | 32.9% | +8.5 pts (+5.7 to +11.2) | 115 / 223 | 4.4e-09 | | |
| | Qwen3.5-0.8B -> + math edit | MedMCQA | 1,000 | 30.6% | 33.3% | +2.7 pts (-0.5 to +5.8) | 116 / 143 | 0.106 | | |
| | Qwen3.5-0.8B -> + math edit | MMLU | 1,089 | 43.4% | 52.4% | +9.0 pts (+6.0 to +11.9) | 101 / 199 | 1.6e-08 | | |
| | random patch 1 -> WSP Math + Clinical 0.8B | MedQA-USMLE (4 options) | 1,273 | 32.4% | 45.4% | +13.0 pts (+10.2 to +15.9) | 98 / 264 | 1.0e-18 | | |
| | random patch 1 -> WSP Math + Clinical 0.8B | MedMCQA | 1,000 | 33.7% | 38.8% | +5.1 pts (+2.0 to +8.1) | 99 / 150 | 0.001 | | |
| | random patch 1 -> WSP Math + Clinical 0.8B | MMLU | 1,089 | 50.3% | 60.2% | +9.8 pts (+7.1 to +12.6) | 70 / 177 | 7.2e-12 | | |
| | random patch 2 -> WSP Math + Clinical 0.8B | MedQA-USMLE (4 options) | 1,273 | 34.0% | 45.4% | +11.4 pts (+8.6 to +14.3) | 108 / 253 | 1.5e-14 | | |
| | random patch 2 -> WSP Math + Clinical 0.8B | MedMCQA | 1,000 | 33.1% | 38.8% | +5.7 pts (+2.6 to +8.7) | 97 / 154 | 3.9e-04 | | |
| | random patch 2 -> WSP Math + Clinical 0.8B | MMLU | 1,089 | 50.9% | 60.2% | +9.3 pts (+6.4 to +12.0) | 77 / 178 | 2.2e-10 | | |
| ### Format-agnostic score (secondary) | |
| Also accepting `Answer: <B>`, `Answer: (b)`, `Answer: <letter>B` and `\boxed{B}` after `</think>`. The gap between this | |
| and the main score shows how much of a checkpoint's score is lost to the format of its final line rather than to the | |
| choice itself. | |
| | Checkpoint | MedQA | MedMCQA | MMLU medical | | |
| |---|---|---|---| | |
| | **OpenMed WSP Math + Clinical 0.8B** (this model) | 45.4% | 38.8% | 60.2% | | |
| | Qwen3.5-0.8B + the paper's math edit (before OpenMed's clinical edit) | 39.4% | 37.5% | 56.4% | | |
| | Random patch, same size, random direction (seed 1) | 39.8% | 37.5% | 55.7% | | |
| | Random patch, same size, random direction (seed 2) | 39.5% | 35.8% | 55.7% | | |
| | Qwen3.5-0.8B as released | 32.5% | 34.8% | 50.0% | | |
| ### Nothing else broke (general benchmarks, one sample, seed 0) | |
| | Checkpoint | MATH-500 | MATH-500 cut at 32,768 tokens | IFEval (strict, prompt level) | | |
| |---|---|---|---| | |
| | **OpenMed WSP Math + Clinical 0.8B** (this model) | 51.2% | 31.0% | 62.5% | | |
| | Qwen3.5-0.8B + the paper's math edit (before OpenMed's clinical edit) | 50.6% | 30.2% | 61.4% | | |
| | Qwen3.5-0.8B as released | 7.4% | 93.6% | 20.2% | | |
| The paper reports 48.1% on MATH-500 and 62.3% on IFEval for its math edit (4 and 2 samples per problem). | |
| ## What the edit does | |
| The fixed exam prompt ends with "finish with one last line exactly like: Answer: <letter>". Under the paper's sampling the model often copies the placeholder instead of filling it in, ending with `Answer: <letter>` or `Answer: <B>`, which no grader reads as a letter. The editor traced this to attention head 4 of layer 23, the last layer: at the final `Answer:` position it attends to the prompt's `<` and writes the direction that raises the logit of the ` <` token. The presence penalty makes it worse, because the four letters have usually appeared in the thinking already while ` <` has not. | |
| The edit removes that one direction from the head's output: with u the unit vector of (final-norm weight x embedding of ` <`), the 256 columns of `o_proj` that belong to head 4 become `W - u u^T W`. Nothing else in the model changes. On 2,560 practice questions from the training splits (ten fresh sets, never used to design the edit) the editor measured a paired gain of +9.65 +/- 1.01 points with the residual ungradable answers down to 0.3%. | |
| | Tensor | Shape | bfloat16 entries changed | Relative change | sha256 (edited, bfloat16 bytes) | | |
| |---|---|---|---|---| | |
| | `model.language_model.layers.23.self_attn.o_proj.weight` | 1024 x 2048 | 234,183 | 0.0333 | `8fac195c18d739da8a38cc2b56577813617655d145912bd2352dce9d750f0951` | | |
| 234,183 bfloat16 entries differ from the starting checkpoint (the math-edited checkpoint ML-Intern-lab/Qwen3.5-0.8B-weight-space-programming @ f4a5dd43, whose two edited tensors match the paper authors' published diff bit for bit). `edit/manifest.json` and | |
| `edit/edit.npz` hold the edit as an exact diff (indices and bfloat16 bits), with the sha256 of every tensor before and | |
| after; rebuilding the tensors from the diff reproduces these hashes. | |
| ## Quick start | |
| ```python | |
| from vllm import LLM, SamplingParams | |
| from transformers import AutoTokenizer | |
| def main(): | |
| repo = "OpenMed/Qwen3.5-0.8B-WSP-Math-Clinical" | |
| tok = AutoTokenizer.from_pretrained(repo) | |
| llm = LLM(model=repo, dtype="bfloat16", max_model_len=34816, language_model_only=True, generation_config="vllm") | |
| # a MedQA-USMLE practice question (train split, id medqa_train_4541; the key is B) | |
| question = ("A 23-year-old male with a homozygous CCR5 mutation is found to be immune to HIV infection. " | |
| "The patient's CCR5 mutation interferes with the function of which viral protein?") | |
| options = {"A": "Reverse transcriptase", "B": "gp120", "C": "gp41", "D": "pp17"} | |
| user = ("Which option is the best answer to this clinical question?\n\n" + question + "\n\n" | |
| + "\n".join(f"{k}. {v}" for k, v in options.items()) | |
| + "\n\nReason as much as you need, then finish with one last line exactly like: Answer: <letter>") | |
| prompt = tok.apply_chat_template([{"role": "user", "content": user}], tokenize=False, | |
| add_generation_prompt=True, enable_thinking=True) | |
| sp = SamplingParams(temperature=0.6, top_p=0.95, top_k=20, presence_penalty=1.5, max_tokens=32768, seed=0) | |
| text = llm.generate([prompt], sp)[0].outputs[0].text | |
| print(text.split("</think>")[-1].strip()) | |
| if __name__ == "__main__": # vLLM starts worker processes | |
| main() | |
| ``` | |
| ## How it was made | |
| 1. **The paper's math edit, verified.** We started from Hugging Face's ML Intern reproduction of the paper's 50-pass edit and checked that its two edited tensors are byte-identical to the authors' published diff (`RikiyaT/lm_understand-logs`, `exp8_math/runs/p50c/edit/`) and that every other tensor equals `Qwen/Qwen3.5-0.8B`. On medicine it already helps (table above): the thinking loops that hit the 32,768-token limit almost disappear. | |
| 2. **The clinical edit.** We rebuilt the paper's harness, whose prompts and rules are published but whose code is not: a persistent CPU Python kernel over the weights (no GPU, no internet, no gradients; `save_edit` refuses the final norm, the embeddings and whole-tensor rescaling) and a forward-pass tool (generations with the evaluation's sampling, paired original vs edited with fixed seeds, teacher-forced reads of hidden states, MLP activations and attention). The task text is the paper's, with the target changed from math to clinical questions. The editor could use only training-split practice questions; the MedQA, MedMCQA and MMLU test sets above were never shown to it. The editor is a frontier LLM. The run was stopped after 21 of its 50 forward passes; the editor had settled on this edit by its 6th pass and spent the rest confirming it. | |
| 3. **Evaluation.** Every checkpoint ran with the same vLLM 0.33.0 server and settings on one GPU. The random-patch rows apply a change of the same size to the same 256 columns of the math-edited model (same singular values, random directions; two seeds): the direction matters, not the size of the change. | |
| ## Files | |
| - The weights, config, tokenizer and chat template: Qwen3.5-0.8B's release files with the edited tensors (every | |
| other tensor is unchanged). | |
| - `edit/`: the clinical edit as an exact, hash-checked diff from the starting checkpoint. | |
| - `eval/`: per-question results of every checkpoint on every benchmark (id, finish reason, tokens, extracted answer, | |
| grade) with no question text, the scores, the first 20 generated texts per benchmark for this model, and | |
| `receipt.json` (forward passes, Python runs, turns and wall time of the edit run). | |
| ## Limitations | |
| - One 0.8B model and one sample per question (MedQA with three seeds). Gains are relative to the same model and sampling; they say nothing about clinical safety. | |
| - Most of the gain is the model now stating a letter where it used to copy the placeholder; the format-agnostic table shows how much of it is format and how much is choice. | |
| - The fixed prompt literally says `Answer: <letter>`; with a prompt that does not show a placeholder the clinical edit has less to fix. | |
| - The edit lowers the probability of ` <` everywhere, not only in the answer line (the editor measured a median change of 0.0008 nats per thinking token). | |
| - Research only. **Not for clinical use**, diagnosis or treatment. | |
| ## Credits | |
| - **Method:** "Sparks of Weight-Space Programming: LLMs can natively read & write neural network weights" by Rikiya Takehi, Jayden Teoh, Alex Zhang and Omar Khattab (MIT CSAIL, 2 Nov 2026), https://rikiyat.github.io/blog/weight-space-programming/. Their logs and diffs: https://github.com/RikiyaT/lm_understand-logs (logs, data and networks CC BY 4.0, scripts MIT). The math edit inside this model is theirs (p50c), used under CC BY 4.0. | |
| - **Reproduction we built on:** Hugging Face's ML Intern rebuilt and verified the math edit ([ML-Intern-lab/Qwen3.5-0.8B-weight-space-programming](https://huggingface.co/ML-Intern-lab/Qwen3.5-0.8B-weight-space-programming)). ML Intern did not take part in OpenMed's clinical edit or evaluation. | |
| - **Base model:** [Qwen/Qwen3.5-0.8B](https://huggingface.co/Qwen/Qwen3.5-0.8B) by the Qwen team, Apache-2.0. | |
| - **Data:** MedQA-USMLE (GBaker/MedQA-USMLE-4-options, CC-BY-4.0), MedMCQA (openlifescienceai/medmcqa, Apache-2.0), MMLU (cais/mmlu, MIT), MATH-500 (HuggingFaceH4/MATH-500), IFEval (google/IFEval, Apache-2.0). | |
| ## Citation | |
| ```bibtex | |
| @misc{takehi2026weightspace, | |
| title = {Sparks of Weight-Space Programming: LLMs can natively read \& write neural network weights}, | |
| author = {Rikiya Takehi and Jayden Teoh and Alex Zhang and Omar Khattab}, | |
| year = {2026}, | |
| month = {nov}, | |
| url = {https://rikiyat.github.io/blog/weight-space-programming/} | |
| } | |
| ``` | |