Transformers
Safetensors
llama
speculative-decoding
eagle3
draft-model
kimi-k2.5
fp8
amd-quark
quantized
no-lm-head-quantization
text-generation-inference
quark
Instructions to use amd/Kimi-K2.5-Eagle3-FP8 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use amd/Kimi-K2.5-Eagle3-FP8 with Transformers:
# pip install -U transformers accelerate # Load model directly from transformers import AutoTokenizer, LlamaForCausalLMEagle3 tokenizer = AutoTokenizer.from_pretrained("amd/Kimi-K2.5-Eagle3-FP8") model = LlamaForCausalLMEagle3.from_pretrained("amd/Kimi-K2.5-Eagle3-FP8", device_map="auto") - Notebooks
- Google Colab
- Kaggle
Commit History
-
Capitalize model name, simplify Quark version, and add shared auxiliary files (#6)
- Capitalize model name, simplify Quark version, and add shared auxiliary files (e1bc3631105dd1ea684d6a1aa59fe7495c73f906) - Align Model Overview with AMD template (version pins, layers) for model-card-verify (e2ad6c57270cffdc187b59a11ce43ada1990d683) - Remove figures/demo_video.mp4 (binary media not needed for FP8 draft card) (23769bc03e13ce9a88bf2eeb12207017f33354ef) - Remove figures/kimi-logo.png (binary media not needed for FP8 draft card) (6333286a83cfb3f95e2bdaee1cc5581fcbd44718) - Remove images/benchmark_results.png (binary media not needed for FP8 draft card) (7654ea4ceca98d10d6b502ec3620b93516f00a31) - Remove images/eval_metrics_phase1.png (binary media not needed for FP8 draft card) (7f5094fbe33d990a59a1144f7a7e538a08ebaa95) - Remove images/eval_metrics_phase2.png (binary media not needed for FP8 draft card) (fa338e3fae70596fe0176feb2413fb904fafd0e4) - Restore images/benchmark_results.png (27da617b8c1b06d7ec27f63c3543602afefffaca) - Restore images/eval_metrics_phase1.png (1ce…d42b020