hermitdave/Agnes-3.0-Flash-MLX-oQ4e-MTP

Agnes-3.0-Flash — MLX oQ4e + MTP

oQ4e (mixed-precision) MLX quantization of Agnes-AI/Agnes-3.0-Flash (Apache-2.0), grafted with Qwen3.8's vision tower and Agnes's native MTP head.

• oQ4e mixed-precision quantization — sensitive layers kept at higher precision, rest at 4-bit
• ~19 GB across 5 shards
• 262,144-token context, thinking on/off via the original chat template
• MTP head grafted: Agnes native MTP head with FFN padded 17408→19456 (zero-pad, mathematically identical), quantized to 4-bit
• Vision tower grafted from Qwen3.8 (333 bf16 tensors, byte-identical to Agnes original)

Architecture

Agnes-3.0-Flash is a 33B hybrid attention model:

• 54 recurrent (Gated DeltaNet) layers + 18 global attention layers
• GQA 6:1 (24q/4kv heads, dim 256), 3-axis RoPE, causal conv kernel 4
• Hidden size 5120, intermediate size 17408 (Agnes native)
• MTP head intermediate size 19456 (Qwen3.8 compatible, padded from 17408)

What changed

Converted from the original Agnes format to standard Qwen3.5 architecture:

• Folded parallel FFN into main MLP via concatenation (intermediate_size: 19456)
• Renamed delta_attn → linear_attn, global_attn → self_attn
• Converted one-centered RMSNorm to standard format
• Cast bf16 → fp16 for serialization compatibility

MTP graft:

• Padded Agnes native MTP head FFN layers (gate_proj, up_proj: +2048 rows; down_proj: +2048 cols) to match Qwen3.8's expected 19456
• Quantized MTP weights to 4-bit affine (group_size=64)
• Grafted 333 vision tensors from Qwen3.8 (byte-identical to Agnes original)

Benchmark

Measured on M3 Max 64GB, oMLX, temperature 0:

Metric	Baseline (no MTP)	With MTP	Speedup
Throughput	15.0 tok/s	21.7 tok/s	1.44×
Acceptance rate	—	85.7%	—
Tokens per cycle	—	2.46	—

Usage

Place under ~/.omlx/models/hermitdave/ and enable mtp_enabled + vlm_mtp_enabled in model settings.

Attribution

• Original model: Agnes-AI
• Vision tower: Qwen3.8-27B (byte-identical)
• Converted with Hermes Agent + mlx_lm
• Quantized with oMLX