ToPo-ToPo/Ornith-1.5-122B-MTP-bf16

MTP (multi-token prediction) drafter for speculative decoding with ornith-ai/Ornith-1.5-122B on Apple Silicon (mlx-vlm).

This is not a standalone chat model โ€” it only works bound to an Ornith-1.5-122B target model. The official MLX conversions (MLX, MLX-8bit, MLX-4bit) do not include the MTP weights.

Provenance

  • Split from the official checkpoint's built-in mtp.* weights (20 tensors) with mlx-vlm 0.7.6 (mlx_vlm.speculative.drafters.qwen3_5_mtp.split, mlx 0.32.3)
  • Precision: bf16 (unquantized, split as-is)
  • Size: 5.0 GB, block_size: 3, model_type: qwen3_5_mtp

Usage

mlx_vlm.generate --model ornith-ai/Ornith-1.5-122B-MLX-8bit \
  --draft-model ToPo-ToPo/Ornith-1.5-122B-MTP-bf16 --draft-kind mtp \
  --prompt "..." --max-tokens 400

Tested bound to an 8bit (affine, group size 64) conversion of Ornith-1.5-122B, served through mlx-vlm 0.7.9. Speedup depends on hardware, target quantization, workload and the mlx-vlm version โ€” measure it yourself.

Downloads last month
133
Safetensors
Model size
3B params
Tensor type
BF16
ยท
MLX
Hardware compatibility
Log In to add your hardware

Quantized

Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for ToPo-ToPo/Ornith-1.5-122B-MTP-bf16

Finetuned
(9)
this model