MLX / Apple Silicon conversions (working MTP + vision), plus a reconstructed bf16
Thank you for this tune — it's genuinely excellent, and the MTP quants were the
reason I went down this road.
Since the repo is GGUF-only, I reconstructed HF-format safetensors from your Q8_0 MTP
export and built MLX versions for Apple Silicon. The MTP head works — these run with real
speculative decoding under MTPLX, and they also load as plain mlx_lm
checkpoints:
- Qwen3.6-27B-Fable-Fusion-711-MTPLX — 6-bit, 23.6 GB
- ...-MTPLX-8bit — 8-bit, 30.3 GB
- ...-MTPLX-4bit — 4-bit, 16 GB
- ...-bf16 — the reconstructed safetensors, 56 GB
Measured on an M5 Max 128 GB (numbers and methodology are on each card): 2.2–2.9x decode
speedup from MTP, 5/5 on a 5-problem AIME run and 8/8 on an executable coding eval for all
three quant levels, and wikitext-2 perplexity for the 6- and 8-bit builds that is
statistically indistinguishable from the bf16 reconstruction. The vision tower is included
too (grafted from mmproj-BF16.gguf, verified on real images).
Two honest caveats I've put on the cards:
- The bf16 repo is a dequantization of your Q8_0, not the original training checkpoint.
If you ever publish the true BF16 safetensors, that would be strictly better and I'd
happily rebuild everything from it. - NEO-Imatrix doesn't carry over — it shapes your sub-8-bit GGUF quants, and the Q8_0
source doesn't use it. So my 4-bit is plain affine, and your 4-bit NEO GGUFs may well beat
it at the same size.
Everything is Apache-2.0 with credit to you, and every card links back here for your settings
guide and class notes. If you'd rather I take any of them down, rename them, or change how
they're attributed, just say so and I'll do it.
One thing that may be useful to others attempting this: llama.cpp's Qwen3.5/3.6 converter
applies three transforms that have to be inverted, and getting them wrong produces a model
that degenerates while still passing consistency checks — the weight + 1 RMSNorm baking,
the Gated-DeltaNet value-head reorder (grouped to tiled, including out_proj columns),
and ssm_a = -exp(A_log). All three are documented on the cards.
I also made the F32 version of the model, available here. It might work better for lower quants like NVFP4
https://huggingface.co/nightmedia/Qwen3.6-27B-Architect-Polaris2-Fable-B-F451-F32
Thank you — that's genuinely useful, and it let me verify the conversion against a source I hadn't used.
I compared the F32 repo tensor-by-tensor against my reconstruction (which came out of the Q8_0 MTP GGUF):
mtp.fc.weight— cosine 0.99997model.language_model.norm.weight— cosine 1.00000model.language_model.layers.0.mlp.gate_proj.weight— cosine 0.99998
Identical structure too: 1199 tensors, 15 mtp.*, 333 vision. The residual is exactly the Q8_0 step my copy passed through, so this independently confirms the reverse-conversion — the weight + 1 RMSNorm unshift, the Gated-DeltaNet value-head reorder (including out_proj columns), ssm_a = -exp(A_log), and the nextn.* → mtp.* mapping all came out right.
I've updated all four model cards to credit the F32 repo and to point anyone building their own quants at it rather than my reconstruction, since it never went through a quantizer. Agreed it should serve lower quants like NVFP4 better.
One question if you know: does the NM in the GGUF's internal name (Qwen3.6 27B Architect Polaris2 Fable B F451 NM) mark a step after this merge, or is it just a naming tag? If there's a stage between the F32 merge and the GGUF, I'd want to say so on the cards.
Here are the formulas side by side: the only thing I changed was the precision.
The drift in metrics is as expected, and F32 provides more resolution to cook fine-tuned quants.
The mxfp4 metrics are identical between BF16 and F32.
models:
- model: Qwen3.6-27B-Architect-Polaris2-Fable-B
parameters:
weight: 1.4
- model: Qwen3.6-27B-Architect-Polaris-Fable-F451
parameters:
weight: 0.6
merge_method: nuslerp
dtype: bfloat16
name: Qwen3.6-27B-Architect-Polaris2-Fable-B-F451
mxfp8 0.711,0.879,0.910,0.790,0.514,0.823,0.763
qx86-hi 0.696,0.876,0.912,0.791,0.518,0.824,0.760
qx64-hi 0.702,0.873,0.909,0.794,0.514,0.822,0.750
mxfp4 0.701,0.873,0.909,0.786,0.488,0.813,0.759
nvfp4 0.701,0.866,0.907
Quant Perplexity Peak Memory Tokens/sec
mxfp8 3.783 ± 0.023 34.74 GB 203
qx86-hi 3.735 ± 0.023 33.25 GB 183
qx64-hi 3.747 ± 0.023 27.03 GB 194
mxfp4 3.854 ± 0.024 21.30 GB 197
Qwen3.6-27B-Architect-Polaris2-Fable-B
mxfp8 0.706,0.875,0.911,0.788,0.516,0.821,0.769
Qwen3.6-27B-Architect-Polaris-Fable-F451
mxfp8 0.699,0.869,0.911,0.789,0.498,0.818,0.772
models:
- model: Qwen3.6-27B-Architect-Polaris2-Fable-B
parameters:
weight: 1.4
- model: Qwen3.6-27B-Architect-Polaris-Fable-F451
parameters:
weight: 0.6
merge_method: nuslerp
dtype: float32
name: Qwen3.6-27B-Architect-Polaris2-Fable-B-F451-F32
mxfp8 0.709,0.880,0.909
qx64-hi 0.706,0.873,0.908
nvfp4 0.704,0.868,0.908
mxfp4 0.701,0.873,0.909