# Mistral-Small-24B-Instruct-2501-FP8-Dynamic
## Model Overview
- **Model Architecture:** Mistral-Small-24B-Instruct-2501
- **Input:** Text
- **Output:** Text
- **Model Optimizations:**
- **Weight quantization:** FP8
- **Activation quantization:** FP8
- **Release Date:** 3/1/2025
- **Version:** 1.0
- **Model Developers:** Neural Magic
Quantized version of [Mistral-Small-24B-Instruct-2501](https://huggingface.co/mistralai/Mistral-Small-24B-Instruct-2501).
It achieves a flexible-extract filter score of 0.9030 on the evaluated on [GSM8k](https://github.com/EleutherAI/lm-evaluation-harness/tree/main/lm_eval/tasks/gsm8k) task, where as the unquantized model achieves a flexible-extract filter score of 0.9060.
### Model Optimizations
This model was obtained by quantizing the weights and activations to FP8 data type, ready for inference with vLLM >= 0.5.2.
This optimization reduces the number of bits per parameter from 16 to 8, reducing the disk size and GPU memory requirements by …
