---
license: mit
library_name: transformers
pipeline_tag: text-generation
tags:
- speculative-decoding
- eagle3
- draft-model
- minimax-m3
- amd-quark
- vllm
base_model:
- amd/MiniMax-M3-MXFP4
---

# MiniMax-M3-EAGLE3.1

## Model Overview

* **Model Architecture:** `LlamaForCausalLMEagle3` (EAGLE3 speculative-decoding draft model)
  * **Input:** Text
  * **Output:** Text
* **Draft variant:** EAGLE3.1 (single decoder layer, full vocabulary, BF16)
* **Target model:** [amd/MiniMax-M3-MXFP4](https://huggingface.co/amd/MiniMax-M3-MXFP4)
* **Supported Hardware Microarchitecture:** AMD Instinct MI350X / MI355X
* **Inference Engine:** vLLM (ROCm)
* **Trained by:** the AMD Quark team

**MiniMax-M3-EAGLE3.1** is an EAGLE3 draft model for accelerating inference of
[amd/MiniMax-M3-MXFP4](https://huggingface.co/amd/MiniMax-M3-MXFP4) with speculative
decoding. It was trained **from scratch (cold-start)** entirely on AMD Instinct
MI350X GPUs with a vLLM-centric pipeline (on-policy data generated by the target,
hidden-state extraction from the target, and in-loop serve-evaluation). Speculative
decoding is **lossless** — every draft token is verified by the target, so the
target's output distribution is preserved.

## Intended Use

This model is intended to be used as an EAGLE3 draft model for speculative decoding
with [amd/MiniMax-M3-MXFP4](https://huggingface.co/amd/MiniMax-M3-MXFP4) as the
target model. It reuses the target's tokenizer, so no tokenizer files are shipped
with the draft.

## Acceptance Length

Evaluated on the official **SPEED-Bench** harness (`specdec_bench`) at TP=8 on
vLLM ROCm v0.27.1 with native AITER MXFP4 MoE kernels, using
`num_speculative_tokens=3` and temperature 0. Qualitative results are the mean of
two complete runs; each fixed-context Throughput split is one complete run.
Acceptance length (AL) is the mean number of tokens emitted per target-model
verification step (AL = 1 means no speedup); higher is better.

### Acceptance length by domain (SPEED-Bench qualitative)

| Domain        | Acceptance length (AL) |
| ------------- | ---------------------- |
| Coding        | 3.33                   |
| Math          | 3.17                   |
| RAG           | 3.14                   |
| Multilingual  | 3.07                   |
| Reasoning     | 2.91                   |
| Summarization | 2.88                   |
| STEM          | 2.84                   |
| Humanities    | 2.70                   |
| QA            | 2.55                   |
| Writing       | 2.33                   |
| Roleplay      | 2.02                   |
| **Overall**   | **2.81**               |

### Acceptance length by context length

| Context length | Acceptance length (AL) |
| -------------- | ---------------------- |
| 1K             | 2.71                   |
| 8K             | 2.71                   |
| 16K            | 2.72                   |
| 32K            | 2.71                   |

Acceptance length stays essentially flat at 2.71–2.72 from 1K to 32K context.

## Native AITER Serving Results

On SPEED-Bench Qualitative at concurrency 48, EAGLE3 improves output throughput
and end-to-end latency over a matched non-speculative baseline:

| Metric                | EAGLE3 average | Non-speculative | Change         |
| --------------------- | -------------- | --------------- | -------------- |
| Output throughput/GPU | 415.2 tok/s    | 281.7 tok/s     | 1.47×          |
| Median E2E latency    | 9.68 s         | 13.15 s         | 1.36× faster   |
| Median TTFT           | 154 ms         | 130 ms          | +18.5% latency |

The EAGLE3 values are means of two complete runs. Both configurations use vLLM
ROCm v0.27.1, native AITER MXFP4, TP=8, temperature 0, a 12,288-token maximum
model length, and a 2,048-token output cap. The TTFT result shows the first-token
latency trade-off.

![Native AITER per-GPU output throughput across concurrency on SPEED-Bench Qualitative](assets/eagle3-throughput-concurrency.png)

Across concurrency 1–128, EAGLE3 improves per-GPU output throughput at every
tested point, scaling from 29 to 652 tok/s/GPU versus 16 to 465 tok/s/GPU for
the non-speculative baseline. The relative gain narrows from 1.77× at
concurrency 1 to 1.40× at concurrency 128 as the baseline approaches saturation,
while the absolute throughput advantage grows from approximately 13 to
187 tok/s/GPU. Figure labels are rounded; speedup ratios use the underlying
unrounded benchmark values.

## Serving with vLLM

Serve the [amd/MiniMax-M3-MXFP4](https://huggingface.co/amd/MiniMax-M3-MXFP4) target
with this draft as the EAGLE3 speculative model at TP=8. See the
[target model card](https://huggingface.co/amd/MiniMax-M3-MXFP4) for the
ROCm/vLLM runtime image and setup.

```bash
export VLLM_ROCM_USE_AITER=1
vllm serve amd/MiniMax-M3-MXFP4 --trust-remote-code --tensor-parallel-size 8 \
    --block-size 128 --attention-backend TRITON_ATTN --moe-backend aiter \
    --speculative-config '{"method":"eagle3","model":"amd/MiniMax-M3-EAGLE3.1","num_speculative_tokens":3,"attention_backend":"TRITON_ATTN"}'
```

## Citation and Acknowledgements

Trained by the AMD Quark team as the EAGLE3 draft for
[amd/MiniMax-M3-MXFP4](https://huggingface.co/amd/MiniMax-M3-MXFP4). Please validate
quality and acceptance length in your own serving stack.

## License

This draft targets MiniMax-M3; see the bundled `MiniMax M3 LICENSE.txt` (MiniMax Community License) for the terms that apply to the target model and its derivatives.

Modifications Copyright (c) 2026 Advanced Micro Devices, Inc. All rights reserved.