qwen36-reap-2k-mlx-q8

The same model from the story — the one that grew up on the filtered public internet, pruned but whole, with one chapter of structured-reasoning traces folded into its final layer — here pressed into a lighter form.

Where the bf16 release carries every weight at full precision, this is the 8-bit reading: the text and mixture-of-experts weights quantized to 8-bit affine (group size 64), the embeddings and output projection kept at full bf16 so the vocabulary stays sharp, and the vision tower preserved exactly as in the source — bit for bit, untouched by the compression. It thinks the same thoughts; it just carries them in a smaller satchel.

Built by fusing the rank-four LoRA into the base and quantizing in a single pass, so no precision was lost to an intermediate round-trip. Higher fidelity than the 4-bit release, lighter than full precision — for when you want the model's best foot forward without paying for the whole library.

---

Technical details

• Base: RangerX-Qwen3.6-35B-REAP-Pruned-ratio-0.5 — Qwen3.6 35B MoE (128 experts, 8 active), REAP-pruned at ratio 0.5, multimodal (text + vision)
• Precision: text/MoE weights 8-bit affine (group_size 64); embed_tokens and lm_head kept at bf16; vision tower bf16 (preserved bit-identical from source)
• ~8.9 bits per weight overall
• Adapter: LoRA, rank 4, scale 8.0, last full-attention layer only (layer 39), 300 iterations at lr 1e-5
• Merge: adapter fused via mlx_lm fuse(), then quantized in the same pass; vision tower re-attached verbatim
• Architecture: 40 layers (every 4th is full_attention, rest linear_attention), hidden size 2048, 16 attention heads, MRoPE

Verification

• Loads strictly under mlx_lm 0.31.3 with zero missing or unexpected weights (full multimodal: text + vision)
• Built via the same fuse + quantize pipeline as the verified-working Q4 release

Note on generation testing

Generation was not verified on the build machine (24 GB unified memory): the Q8 weights (~20 GB) plus vision tower exceed available GPU memory at runtime. The model strict-loads cleanly and the weights are produced by the same proven fuse + quantize path as the Q4 (which generates correctly). Verify generation on a host with ≥ 32 GB unified memory.