bluemorpholimited/Morpho-180B-Checkpoints

Every earned weight, sealed the moment it converged. This vault holds the per-expert best adapters of MORPHO-180B — all 64 of them, each header-verified, each one releasable on its own. Training boxes die; the vault does not.

flowchart LR
    A["SFT<br/>64/64 converged"] --> B["VAULT<br/>sealed bests"]
    B --> C["DPO<br/>31/31 aligned"]
    C --> D["Merge<br/>72.7B dense"]
    D --> E["GGUF + students<br/>published"]
    style B fill:#ff2d78,stroke:#b967ff,color:#fff
    style E fill:#39ff88,stroke:#00f0ff,color:#000

Naming note: the display brand is Morpho-180B — 180,500,471,808 parameters across a 72.7B dense backbone and 64 expert adapters. HF repo slugs (...-MoE-Checkpoints, ...-MoE-Data, .../Morpho-180B) stay frozen.

Per-expert QLoRA adapters (rank 128, alpha 256) for Morpho-180B SFT. Each expert trained to convergence (patience 2,000 steps) on its own domain slice of the 822,745-example corpus.

Status — COMPLETE

Fact	Value
Experts converged	64 / 64
Parameters per adapter	1,684,275,200
Tensors per adapter	1,120 (80 layers x 14 modules)
Adapter storage	404.2 GB (56 fp32 @ 6,737,255,616 B · 8 bf16 @ 3,368,705,936 B)
Collective size	180,500,471,808 (72,706,859,008 backbone + 64 x 1,684,275,200)
Best loss range	3.97e-08 to 0.6756 (median 0.0943)
Best step range	375 to 13689 (median 4094)
DPO refresh	31 experts, published in Morpho-180B-DPO
Downstream	merged 72.7B, 4 GGUF builds, 2 distilled students — all published

Every adapter below was read back from this repo: adapter_model.safetensors present, safetensors header parsed, 1,120 tensors, r=128 / alpha=256, 7 target modules (q,k,v,o,gate,up,down). No row is copied from a training log.

The 64 adapters

Expert	Sector	Specialisation	Best loss	Best step	Weights
00	Finance	Trading & Markets	0.0547	8391	yes fp32
01	Finance	Investing & Portfolio	0.2746	1898	yes fp32
02	Finance	Banking & Lending	0.1307	5333	yes fp32
03	Finance	Insurance & Risk	0.4501	1917	yes fp32
04	Finance	Accounting & Auditing	0.1500	3003	yes fp32
05	Finance	Taxation & Compliance	0.0754	7771	yes fp32
06	Finance	Real Estate Finance	0.4678	1721	yes fp32
07	Finance	Financial Engineering	0.2545	3817	yes fp32
08	Law	Constitutional Law	0.0692	6660	yes fp32
09	Law	Criminal Law	0.2838	3176	yes fp32
10	Law	Civil Law	0.0955	4983	yes fp32
11	Law	Corporate Law	0.4710	989	yes fp32
12	Law	Intellectual Property	0.3093	2215	yes fp32
13	Law	International Law	0.3621	1054	yes fp32
14	Law	Tax Law	0.0086	6023	yes fp32
15	Law	Regulatory Compliance	0.5570	375	yes fp32
16	Medicine	Clinical Medicine	0.1744	1036	yes fp32
17	Medicine	Pharmacology	0.0966	1159	yes fp32
18	Medicine	Diagnostics & Imaging	0.0015	7293	yes fp32
19	Medicine	Surgery & Procedures	0.0035	4953	yes fp32
20	Medicine	Neurology	0.2621	2864	yes fp32
21	Medicine	Cardiology	0.3481	498	yes fp32
22	Medicine	Oncology	0.5547	1753	yes fp32
23	Medicine	Public Health	0.1529	4614	yes fp32
24	Technology	Machine Learning & AI	0.0002	5103	yes fp32
25	Technology	Systems & Infrastructure	0.0016	5524	yes fp32
26	Technology	Cybersecurity	0.0781	1214	yes fp32
27	Technology	Databases & SQL	0.1779	2050	yes fp32
28	Technology	Networking & Cloud	0.2036	3632	yes fp32
29	Technology	Web Development	0.0003	6426	yes fp32
30	Technology	DevOps & MLOps	5.94e-05	9631	yes fp32
31	Technology	Embedded & IoT	0.1429	5898	yes bf16
32	Science	Physics	0.0025	4836	yes fp32
33	Science	Chemistry	5.96e-07	835	yes fp32
34	Science	Biology & Life Sciences	0.1974	8949	yes fp32
35	Science	Mathematics	0.3517	2268	yes fp32
36	Science	Mechanical Engineering	0.1394	4281	yes fp32
37	Science	Electrical Engineering	0.0123	7769	yes fp32
38	Science	Civil Engineering	0.1018	13689	yes fp32
39	Science	Environmental Science	0.6756	1799	yes fp32
40	Business	Strategic Management	0.0318	10033	yes fp32
41	Business	Marketing & Sales	0.0003	7435	yes fp32
42	Business	Human Resources	0.0021	11129	yes fp32
43	Business	Operations & Logistics	3.97e-05	5896	yes fp32
44	Business	Supply Chain	8.59e-05	5982	yes fp32
45	Business	Project Management	4.00e-05	6819	yes fp32
46	Business	Entrepreneurship	0.0931	4173	yes fp32
47	Business	Consulting	0.2452	3271	yes fp32
48	Arts	Creative Writing	0.2855	2114	yes fp32
49	Arts	Graphic Design	0.0003	3071	yes fp32
50	Arts	Music & Audio	0.0014	9092	yes fp32
51	Arts	Film & Video	0.0006	7928	yes fp32
52	Arts	Translation & Localization	0.0003	5525	yes bf16
53	Arts	Linguistics	0.0138	9731	yes fp32
54	Arts	Storytelling & Narrative	0.0014	8118	yes bf16
55	Arts	Content Creation	0.0003	3508	yes fp32
56	Humanities	Pedagogy	0.4577	4016	yes fp32
57	Humanities	Psychology	0.5472	1708	yes fp32
58	Humanities	Sociology	8.34e-06	2704	yes fp32
59	Humanities	Economics	0.0001	6783	yes bf16
60	Humanities	Philosophy	0.4240	673	yes bf16
61	Humanities	History	0.3632	376	yes bf16
62	Humanities	Political Science	3.97e-08	2211	yes bf16
63	Humanities	Critical Analysis & Reasoning	2.29e-05	2967	yes bf16

fp32 = 6,737,255,616 bytes · bf16 = 3,368,705,936 bytes. Both carry the same config and the same 1,120 tensors; the bf16 group was saved half-precision and is numerically equivalent within bf16 tolerance. The merge loads each adapter with its own adapter_config.json, so mixed storage never affected Stage 6.

Layout

sft/expert_XX/best/
  adapter_model.safetensors   trained LoRA weights, 1,120 tensors
  adapter_config.json         PEFT config (r=128, alpha=256, dropout 0.05)
  best_loss.txt               converged training loss
  step.txt                    global step of the best checkpoint

Pipeline outcomes

Stage	Artifact
4 · SFT	these 64 adapters
5 · DPO	Morpho-180B-DPO — 31/31 aligned experts
6 · Merge	Morpho-180B-Merged — PPL 7.69 to 7.83 (x1.02), gate PASS
7 · Verify	GSM8K 76.7% · PPL 7.68 · 19.5 tok/s
8 · GGUF	Morpho-180B-GGUF — 324.5 GB, four formats
9 · Distill	Morpho-7B-Distill · Morpho-1.5B-Distill

Loading one expert

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

# The backbone checkpoint is identified in NOTICE. Load it 4-bit or bf16.
BASE = "<72.7B backbone checkpoint>"

base = AutoModelForCausalLM.from_pretrained(BASE, device_map="auto",
                                            torch_dtype="bfloat16")
tok = AutoTokenizer.from_pretrained(BASE)

model = PeftModel.from_pretrained(base, "bluemorpholimited/Morpho-180B-Checkpoints",
                                  subfolder="sft/expert_00/best")

Provenance

• 2026-09-21: best/ weights for 00–14 deleted in a cleanup error.
• 2026-09-22: 12 files recovered from git history, later header-audited and found truncated; replaced by genuine retrains. Repo recreated to release LFS quota.
• 2026-09-23: Master Plan v2; fresh SFT passes for the lost experts.
• 2026-10-24: 64/64 converged and verified. Merge gate passed, GGUF and both distilled students published. Pipeline closed.

Resume policy (historical)

Training resolved LOCAL-first, HF-second; best/ is authoritative. step_* dirs were ephemeral (latest 5 kept on the box). optimizer.pt and step dirs are banned from this repo — the box disk was the working store, this repo the release channel. With all 64 experts converged, the resume path is now only needed for reruns.

Licence

Adapter weights: Apache 2.0. They are trained against the upstream backbone and distributed for use with it under its terms. The required notice and a full copy of the agreement are in NOTICE and LICENSE.

---

Part of MORPHO-180B — model card · training data.

THE VAULT DOES NOT FORGET