Morpho-1.5B-Distill

A 1.5B student distilled from Morpho-180B — the 180.5B collective whose merged teacher folds 64 converged domain experts into one dense 72.7B backbone. Same recipe as Morpho-7B-Distill, smaller student.

Fact	Value
Teacher	bluemorpholimited/Morpho-180B-Merged, 4-bit NF4
Student	1.5B dense open-weights checkpoint, full fine-tune
Objective	L = 0.5·T²·KL(s/T ‖ t/T) + 0.5·CE, T = 2, answer-span masked
Schedule	2,500 steps, max length 512
Best loss	1.5750

Evaluation

Metric	Value
Perplexity (10-doc holdout)	13.16
GSM8K (10 items)	2/10
Domain audit, 8 sectors	on-topic; 6/8 loop under pure greedy
Domain audit with repetition_penalty=1.1	8/8 loop-free

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer

repo = "bluemorpholimited/Morpho-1.5B-Distill"
tok = AutoTokenizer.from_pretrained(repo)
model = AutoModelForCausalLM.from_pretrained(repo, torch_dtype="bfloat16", device_map="auto")

ids = tok.apply_chat_template([{"role": "user", "content": "What is stare decisis?"}],
                              add_generation_prompt=True, return_tensors="pt").to(model.device)
out = model.generate(ids, max_new_tokens=200, do_sample=False, repetition_penalty=1.1)
print(tok.decode(out[0][ids.shape[1]:], skip_special_tokens=True))

Scope

This is a demo-sized model: it demonstrates the distillation pipeline end to end and produces fluent, on-topic short answers with repetition_penalty=1.1. It has little factual depth and should not be used where correctness matters. Use the 72.7B for real work, or the 7B student for a middle ground.

Licence

Apache 2.0 for this model's weights and training recipe. See LICENSE.

---

Release map

Stage	Artifact
Pipeline, gate history, benchmarks	Morpho-180B
4 · 64 converged SFT adapters	Checkpoints
5 · 31 DPO-aligned adapters	DPO
6 · Merged 72B weights	Merged
8 · Quantized builds	GGUF
9 · Distilled students	7B · 1.5B
Data and pipeline code	Morpho-180B-Data