File size: 5,569 Bytes
33e706d c7203c9 33e706d c7203c9 33e706d c7203c9 33e706d c7203c9 33e706d c7203c9 cadf8c9 33e706d c7203c9 33e706d c7203c9 33e706d 01dc492 33e706d | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 | ---
license: mit
library_name: transformers
pipeline_tag: text-generation
tags:
- speculative-decoding
- eagle3
- draft-model
- minimax-m3
- amd-quark
- vllm
base_model:
- amd/MiniMax-M3-MXFP4
---
# MiniMax-M3-EAGLE3.1
## Model Overview
* **Model Architecture:** `LlamaForCausalLMEagle3` (EAGLE3 speculative-decoding draft model)
* **Input:** Text
* **Output:** Text
* **Draft variant:** EAGLE3.1 (single decoder layer, full vocabulary, BF16)
* **Target model:** [amd/MiniMax-M3-MXFP4](https://huggingface.co/amd/MiniMax-M3-MXFP4)
* **Supported Hardware Microarchitecture:** AMD Instinct MI350X / MI355X
* **Inference Engine:** vLLM (ROCm)
* **Trained by:** the AMD Quark team
**MiniMax-M3-EAGLE3.1** is an EAGLE3 draft model for accelerating inference of
[amd/MiniMax-M3-MXFP4](https://huggingface.co/amd/MiniMax-M3-MXFP4) with speculative
decoding. It was trained **from scratch (cold-start)** entirely on AMD Instinct
MI350X GPUs with a vLLM-centric pipeline (on-policy data generated by the target,
hidden-state extraction from the target, and in-loop serve-evaluation). Speculative
decoding is **lossless** — every draft token is verified by the target, so the
target's output distribution is preserved.
## Intended Use
This model is intended to be used as an EAGLE3 draft model for speculative decoding
with [amd/MiniMax-M3-MXFP4](https://huggingface.co/amd/MiniMax-M3-MXFP4) as the
target model. It reuses the target's tokenizer, so no tokenizer files are shipped
with the draft.
## Acceptance Length
Evaluated on the official **SPEED-Bench** harness (`specdec_bench`) at TP=8 on
vLLM ROCm v0.27.1 with native AITER MXFP4 MoE kernels, using
`num_speculative_tokens=3` and temperature 0. Qualitative results are the mean of
two complete runs; each fixed-context Throughput split is one complete run.
Acceptance length (AL) is the mean number of tokens emitted per target-model
verification step (AL = 1 means no speedup); higher is better.
### Acceptance length by domain (SPEED-Bench qualitative)
| Domain | Acceptance length (AL) |
| ------------- | ---------------------- |
| Coding | 3.33 |
| Math | 3.17 |
| RAG | 3.14 |
| Multilingual | 3.07 |
| Reasoning | 2.91 |
| Summarization | 2.88 |
| STEM | 2.84 |
| Humanities | 2.70 |
| QA | 2.55 |
| Writing | 2.33 |
| Roleplay | 2.02 |
| **Overall** | **2.81** |
### Acceptance length by context length
| Context length | Acceptance length (AL) |
| -------------- | ---------------------- |
| 1K | 2.71 |
| 8K | 2.71 |
| 16K | 2.72 |
| 32K | 2.71 |
Acceptance length stays essentially flat at 2.71–2.72 from 1K to 32K context.
## Native AITER Serving Results
On SPEED-Bench Qualitative at concurrency 48, EAGLE3 improves output throughput
and end-to-end latency over a matched non-speculative baseline:
| Metric | EAGLE3 average | Non-speculative | Change |
| --------------------- | -------------- | --------------- | -------------- |
| Output throughput/GPU | 415.2 tok/s | 281.7 tok/s | 1.47× |
| Median E2E latency | 9.68 s | 13.15 s | 1.36× faster |
| Median TTFT | 154 ms | 130 ms | +18.5% latency |
The EAGLE3 values are means of two complete runs. Both configurations use vLLM
ROCm v0.27.1, native AITER MXFP4, TP=8, temperature 0, a 12,288-token maximum
model length, and a 2,048-token output cap. The TTFT result shows the first-token
latency trade-off.
Across concurrency 1–128, EAGLE3 improves per-GPU output throughput at every
tested point, scaling from 29 to 652 tok/s/GPU versus 16 to 465 tok/s/GPU for
the non-speculative baseline. The relative gain narrows from 1.77× at
concurrency 1 to 1.40× at concurrency 128 as the baseline approaches saturation,
while the absolute throughput advantage grows from approximately 13 to
187 tok/s/GPU. Figure labels are rounded; speedup ratios use the underlying
unrounded benchmark values.
## Serving with vLLM
Serve the [amd/MiniMax-M3-MXFP4](https://huggingface.co/amd/MiniMax-M3-MXFP4) target
with this draft as the EAGLE3 speculative model at TP=8. See the
[target model card](https://huggingface.co/amd/MiniMax-M3-MXFP4) for the
ROCm/vLLM runtime image and setup.
```bash
export VLLM_ROCM_USE_AITER=1
vllm serve amd/MiniMax-M3-MXFP4 --trust-remote-code --tensor-parallel-size 8 \
--block-size 128 --attention-backend TRITON_ATTN --moe-backend aiter \
--speculative-config '{"method":"eagle3","model":"amd/MiniMax-M3-EAGLE3.1","num_speculative_tokens":3,"attention_backend":"TRITON_ATTN"}'
```
## Citation and Acknowledgements
Trained by the AMD Quark team as the EAGLE3 draft for
[amd/MiniMax-M3-MXFP4](https://huggingface.co/amd/MiniMax-M3-MXFP4). Please validate
quality and acceptance length in your own serving stack.
## License
This draft targets MiniMax-M3; see the bundled `MiniMax M3 LICENSE.txt` (MiniMax Community License) for the terms that apply to the target model and its derivatives.
Modifications Copyright (c) 2026 Advanced Micro Devices, Inc. All rights reserved.
|