💊 Medical Reasoning
Collection
Small models for local use by xtools-at, fine-tuned for medical reasoning and diagnosis • 6 items • Updated • 1
SmolLM3-3B Medical Reasoning
LoRA fine-tune of SmolLM3-3B for medical reasoning.
- Finetuned from model: K0D3IN/SmolLM3-3B-Instruct-heretic
- Dataset used: FreedomIntelligence/medical-o1-reasoning-SFT
Training params
- 16bit LoRA fine-tune on Tesla T4 16GB GPU
- 1.1 epochs on 2000 random entries of the dataset + 2% eval set
- 4096 context length; learning rate 2e-4
- rank: 16; alpha: 32; dropout: 0.01; targeting all modules
- optimizer: AdamW 8-bit; LR scheduler: linear
- batch size: 4; grad accum: 8, weight decay: 0.001
- result: training loss 1.36; evaluation loss 1.44
Example use
System prompt:
You're an expert at medical reasoning and diagnosis.
User prompt:
A 48-year-old woman has been experiencing intermittent nausea, decreased appetite, and increasing weakness and fatigue. She has a history of kidney stones and an elevated calcium level of 12.9 mg/dL, along with elevated nocturnal blood pressure. Given these symptoms and lab findings, what abnormality is most likely to be found upon further hormonal evaluation?
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support
Model tree for xtools-at/SmolLM3-3B-Medical-Reasoning
Base model
HuggingFaceTB/SmolLM3-3B-Base Finetuned
HuggingFaceTB/SmolLM3-3B Finetuned
K0D3IN/SmolLM3-3B-Instruct-heretic