README.md

metadata

title: README
colorFrom: gray
colorTo: gray
sdk: static
pinned: false

OpensourceWTF

Model packs for Apple silicon: streaming EXL3 packs for mlx-serve, whose experts stream from SSD so a 128 GB Mac can serve models larger than its memory, the Qwen3.8 EXL3 packs, and MTPLX packs. Each group below is a collection.

EXL3 streaming packs

Packs whose routed experts stream from SSD, for mlx-serve.

Family	Repo	Runtime	Branches	Status
DeepSeek-V4.1-Flash-1017	DeepSeek-V4.1-Flash-1017-streaming-repack-exl3	mlx-serve + turboexl3_ds41 (2.81bpw), mlx-stream (2.9bpw)	2.81bpw, 2.9bpw; 1.68bpw, 2.25bpw, 2.68bpw (testing)	live
DeepSeek-V4.1-Flash (earlier base)	DeepSeek-V4.1-Flash-streaming-repack-exl3	mlx-serve + mlx-stream	3.0bpw	live

Qwen3.8-Flash-Next · mlx-serve EXL3

turboderp's EXL3 quants in mlx-serve's layout; the experts load into memory.

Family	Repo	Runtime	Branches	Status
Qwen3.8-Flash-Next	Qwen3.8-Flash-Next-exl3	mlx-serve + turboexl3	5.05bpw, 4.05bpw, 3.05bpw, 2.05bpw	live

GLM-5.x · streaming EXL3

GLM-5.3 and GLM-5.3-Flash EXL3 packs for mlx-serve + mlx-stream.

Family	Repo	Runtime	Branches	Status
GLM-5.3-Flash	GLM-5.3-Flash-streaming-repack-exl3-3.05bpw	mlx-serve + mlx-stream	main	live
GLM-5.3	GLM-5.3-streaming-repack-exl3-3.0bpw	mlx-serve + mlx-stream	main	live
GLM-5.3	GLM-5.3-streaming-repack-exl3-3.42bpw	mlx-serve + mlx-stream	main	live

MTPLX streaming packs

Packs for MTPLX on Apple silicon.

Family	Repo	Runtime	Branches	Status
DeepSeek-V4.1-Flash	DeepSeek-V4.1-Flash-MTPLX-streaming-q2	MTPLX	main	live
Hunyuan 3.0	Hy3-oQ2e-MTPLX-streaming	MTPLX	main	live
Kimi K3	Kimi-K3-Q2_K-t158-MTPLX-streaming	MTPLX	main	live
Qwen3.8-Flash-Next	Qwen3.8-Flash-Next-MTPLX-oQ4-MTP	MTPLX	main	live
GLM-5.2	GLM-5.2-t158-MTPLX-streaming	MTPLX	main	live

mlx-stream and other

Other packs.

Family	Repo	Runtime	Branches	Status
Beam 1.0	Beam-1.0-501B-A23B-NVFP4-q8-mlx-stream	mlx-serve + mlx-stream	main	live
MiMo-V2.6-Flash-MOPD	MiMo-V2.6-Flash-MOPD-fp8res-mlx-stream	mlx-serve + mlx-stream	main	live
Muse-Glimmer-30B	Muse-Glimmer-30B-MLX-q4	MTPLX	main	live
DeepSeek-V4.1-Flash-1017	DeepSeek-V4.1-Flash-1017-exl3-2.81bpw	exllamav3 (CUDA), the plain layout	main	live

Moved

• Qwen3.8-Flash-Next-EXL3-5.05-mlx-serve, -4.05-mlx-serve, -3.05-mlx-serve and -2.05-mlx-serve → the 5.05bpw, 4.05bpw, 3.05bpw and 2.05bpw branches of Qwen3.8-Flash-Next-exl3. Each old repo is a stub.
• DeepSeek-V4.1-Flash-1017-streaming-repack-exl3-2.9bpw → the 2.9bpw branch of DeepSeek-V4.1-Flash-1017-streaming-repack-exl3. The old repo is a stub.
• Kimi-K3-Q2_K-t158-MTPLX (the fully resident serialization) was removed to free storage; the supported checkpoint is Kimi-K3-Q2_K-t158-MTPLX-streaming. The old repo is a stub.
• Renamed (the old names redirect): DeepSeek-V4.1-Flash-1017-exl3 → DeepSeek-V4.1-Flash-1017-streaming-repack-exl3; DeepSeek-V4.1-Flash-streaming-repack-exl3-3.0bpw → DeepSeek-V4.1-Flash-streaming-repack-exl3 (branch 3.0bpw).