Post
79
Mistral Large 4 scores 48% on the Artificial Analysis Coding Agent Index, effectively tied with GLM-5.3, the leading open-weight model, while using 47% fewer tokens per task. @mistralai has released the weights.
It is a 1T-parameter MoE with 49B parameters active per token. It launched with official vLLM and SGLang Docker images, transformers support and llama.cpp GGUFs. You can self-host it as a coding agent today, but only on a multi-GPU node or a 512 GB Mac.
🔍 What's inside
• 128 routed experts (4 active) plus 1 shared expert, 72 layers, 1M context
• MLA attention with a DeepSeek Sparse Attention indexer that picks up to 2,048 keys per query
• Reasoning effort is low, medium or high, and reasoning cannot be turned off ("none" runs as high)
• A DSpark draft head ships in the checkpoint: up to 1.9x throughput with unchanged outputs
📊 What self-hosting takes
• Q2_K on one 512 GB M3 Ultra: about 15 tok/s generation
• 8x H200, FP8, vLLM: about 65 tok/s for a single request, 120-138 tok/s on chat prompts when DSpark drafts are accepted
• IQ2_XXS on 4x RTX6000 Pro: 30 tok/s at short context, 3 tok/s at long context
⚠️ Caveats
• Running a Model-as-a-Service business needs a separate commercial license
• The HF blog reports 49.8% on the index and 28.3% on Terminal-Bench 4.0, while Mistral reports 48% and 41%
• No quality numbers yet for the 2-bit and 4-bit quants
To use it with Mistral Vibe, add your vLLM server to ~/.vibe/config.toml. The config is on the model card.
mistralai/Mistral-Large-4-1T-A52B
https://huggingface.co/blog/mistral-large4
Could the 47% token saving make an 8-GPU node cheaper than API tokens for your agent workloads?
• Medium: https://medium.com/p/7b7860a6fdcf
• Substack: https://sajjadpakdamansavoji.substack.com/p/mistral-large-4-ties-the-best-open
• X: https://x.com/saavoji/status/2115876124865728631
• Website: https://sajjadpsavoji.github.io/blog#mistral-large
It is a 1T-parameter MoE with 49B parameters active per token. It launched with official vLLM and SGLang Docker images, transformers support and llama.cpp GGUFs. You can self-host it as a coding agent today, but only on a multi-GPU node or a 512 GB Mac.
🔍 What's inside
• 128 routed experts (4 active) plus 1 shared expert, 72 layers, 1M context
• MLA attention with a DeepSeek Sparse Attention indexer that picks up to 2,048 keys per query
• Reasoning effort is low, medium or high, and reasoning cannot be turned off ("none" runs as high)
• A DSpark draft head ships in the checkpoint: up to 1.9x throughput with unchanged outputs
📊 What self-hosting takes
• Q2_K on one 512 GB M3 Ultra: about 15 tok/s generation
• 8x H200, FP8, vLLM: about 65 tok/s for a single request, 120-138 tok/s on chat prompts when DSpark drafts are accepted
• IQ2_XXS on 4x RTX6000 Pro: 30 tok/s at short context, 3 tok/s at long context
⚠️ Caveats
• Running a Model-as-a-Service business needs a separate commercial license
• The HF blog reports 49.8% on the index and 28.3% on Terminal-Bench 4.0, while Mistral reports 48% and 41%
• No quality numbers yet for the 2-bit and 4-bit quants
To use it with Mistral Vibe, add your vLLM server to ~/.vibe/config.toml. The config is on the model card.
mistralai/Mistral-Large-4-1T-A52B
https://huggingface.co/blog/mistral-large4
Could the 47% token saving make an 8-GPU node cheaper than API tokens for your agent workloads?
• Medium: https://medium.com/p/7b7860a6fdcf
• Substack: https://sajjadpakdamansavoji.substack.com/p/mistral-large-4-ties-the-best-open
• X: https://x.com/saavoji/status/2115876124865728631
• Website: https://sajjadpsavoji.github.io/blog#mistral-large