README.md

---
license: mit
library_name: transformers
base_model: deepseek-ai/DeepSeek-V4-Flash-0731
base_model_relation: finetune
pipeline_tag: text-generation
tags:
- agentic
- code
- smaug
- abacusai
---
<div align="center">
  <picture>
      <img src="https://cdn-avatars.huggingface.co/v1/production/uploads/63128dd099791aa61d180c72/dJ6uR23m09M-YEafBpmea.png" width="20%" alt="Smaug-Flash">
  </picture>
</div>
<hr>
<div align="center" style="line-height:1">
  <a href="https://abacus.ai" target="_blank"><img alt="Homepage" src="https://img.shields.io/badge/Homepage-Abacus.AI-white?color=1783ff&logoColor=white"/></a>
  <a href="https://huggingface.co/abacusai" target="_blank"><img alt="Hugging Face" src="https://img.shields.io/badge/%F0%9F%A4%97%20Hugging%20Face-Abacus.AI-ffc107?color=ffc107&logoColor=white"/></a>
</div>
<div align="center" style="line-height: 1;">
  <a href="LICENSE"><img alt="License" src="https://img.shields.io/badge/License-MIT-f5de53?&color=f5de53"/></a>
</div>

## 1. Model Introduction

Smaug-Flash is an agentic-coding finetune of [DeepSeek-V4-Flash-0731](https://huggingface.co/deepseek-ai/DeepSeek-V4-Flash-0731), finetuned by Abacus.AI. It improves end-to-end agentic performance by a wide margin (+14.3 on LiveBench agentic-coding, +10.1 on Terminal Bench 2.1 under the DeepSeek harness, +13.7 on AutomationBench, +19 on NL2Repo-Bench) while improving general capability (+3.2 overall LiveBench). The weights load exactly like the official release: same layout, same quantization formats, same context length (1M), same reasoning interface.

This card describes the training approach and the evaluation results. Dataset contents are not disclosed; training data consists of curated agentic loop traces from various sources - public repositories, traces synthesized from real world agentic product use cases, augmented with synthetic data variations.

## 2. Model Summary

<div align="center">
<table>
<tbody>
<tr><td align="center"><strong>Architecture</strong></td><td align="center">Mixture-of-Experts (MoE)</td></tr>
<tr><td align="center"><strong>Number of Layers</strong></td><td align="center">43</td></tr>
<tr><td align="center"><strong>Attention Mechanism</strong></td><td align="center">MLA (low-rank q/o factors) with sparse token indexer</td></tr>
<tr><td align="center"><strong>Attention Hidden Dimension</strong></td><td align="center">4096</td></tr>
<tr><td align="center"><strong>Number of Attention Heads</strong></td><td align="center">64</td></tr>
<tr><td align="center"><strong>Number of Routed Experts</strong></td><td align="center">256</td></tr>
<tr><td align="center"><strong>Selected Experts per Token</strong></td><td align="center">6</td></tr>
<tr><td align="center"><strong>Number of Shared Experts</strong></td><td align="center">1</td></tr>
<tr><td align="center"><strong>MoE Hidden Dimension (per Expert)</strong></td><td align="center">2048</td></tr>
<tr><td align="center"><strong>Vocabulary Size</strong></td><td align="center">~129K</td></tr>
<tr><td align="center"><strong>Context Length</strong></td><td align="center">1,048,576</td></tr>
<tr><td align="center"><strong>Speculative Decoding</strong></td><td align="center">DSpark multi-token module (inherited, fully functional)</td></tr>
<tr><td align="center"><strong>Quantization</strong></td><td align="center">Block-FP8 attention (e4m3, 128×128) / packed-FP4 experts</td></tr>
<tr><td align="center"><strong>Modality</strong></td><td align="center">Text</td></tr>
<tr><td align="center"><strong>Base Model</strong></td><td align="center"><code>deepseek-ai/DeepSeek-V4-Flash-0731</code></td></tr>
<tr><td align="center"><strong>Adaptation</strong></td><td align="center">SFT + preference LoRA adapters, merged as full deltas (attention-only)</td></tr>
</tbody>
</table>
</div>

## 3. Evaluation

Measured on fresh 2026 task generations, self-served, with agentic benchmarks run via their public harnesses and the base model run through identical harnesses for every paired comparison.

### Smaug-Flash vs its base

*Higher is better. Unstarred rows are paired runs under the same harness; in starred rows the base score is the vendor-reported number for the identical public task set.*

<div align="center">
  <img src="figures/smaug_flash_vs_base_bars.png" width="100%" alt="Smaug-Flash vs DeepSeek-V4-Flash-0731 across agentic benchmarks (bar chart)">
</div>


### LiveBench category profile

*Scores 0–100; overall = mean of the seven category averages. Same harness, same questions (LiveBench 2026-06-25 release), paired against the base model.*

<div align="center">
  <img src="figures/smaug_flash_livebench_categories.png" width="100%" alt="Smaug-Flash LiveBench category profile vs base">
</div>



## 4. Training Approach

Smaug-Flash was trained to make long context agentic loops faster and less prone to spins and stalls in max reasoning mode. We curated trajectories from real world use cases along with generated synthetic variants to address various issues around agentic tool use. 

The model is the composition of three LoRA adapters trained in sequence and merged as full deltas - 2 SFT steps and a KTO step. 
Only the MLA attention factor matrices (129 in total) are adapted; experts, router, embeddings, and the speculative-decoding module remain byte-identical to the official release, packed FP4 included. Adapter deltas are applied in full (no rank truncation) and touched matrices are requantized to the checkpoint-native block-FP8 format. Because only attention changes, any serving stack that runs the official release runs Smaug-Flash unmodified — FP4 expert kernels and speculative decoding paths included. One caution: serve in the checkpoint-native format; requantizing merged weights into a different scheme at load time measurably degrades constrained instruction-following.

## 5. Known Behaviors and Limitations

Smaug-Flash is more decisive than its base: it finishes tasks in fewer, denser turns and submits sooner. Under tight wall-clock budgets this is an asset (it wins several timeout-bound tasks the base loses); under hidden-test grading it can submit at ~98% spec coverage without spending remaining budget on self-verification. If your use case rewards exhaustive verification over decisiveness, prompt for explicit self-testing before completion. 

## 6. Deployment

- SGLang ≥ 0.5.15 on Blackwell: tp=4/8 with `--enable-dp-attention`, `--moe-runner-backend flashinfer_mxfp4` (required for the packed-FP4 experts), fp8 KV cache, full 1M context.
- Reasoning effort **max**; sampling temperature 1.0, top-p 0.95.
- For agentic harnesses, note that wall-clock task budgets are effectively *inference-compute* budgets: serving throughput changes measured scores on timeout-bounded benchmarks. Benchmark on serving comparable to what you deploy.

### Chat template and encoding

Like the base release, this repository ships no Jinja chat template. The [`encoding`](encoding/README.md) folder contains Python scripts and test cases for encoding OpenAI-format messages into model input strings and parsing model output. The `reasoning_effort` parameter supports `low`, `high`, and `max`.

```python
from encoding_dsv4 import encode_messages, parse_message_from_completion_text

messages = [
    {"role": "user", "content": "hello"},
    {"role": "assistant", "content": "Hello!", "reasoning_content": "thinking..."},
    {"role": "user", "content": "1+1=?"}
]

prompt = encode_messages(messages, thinking_mode="thinking", reasoning_effort="max")

import transformers
tokenizer = transformers.AutoTokenizer.from_pretrained("abacusai/Smaug-Flash")
tokens = tokenizer.encode(prompt)
```

### Running with vLLM

Because only attention matrices differ from the official release (§4), any stack that serves DeepSeek-V4-Flash-0731 serves Smaug-Flash unmodified, DSpark speculative decoding included. Example, single 4×GB300 node (see the base model's [vLLM recipe](https://recipes.vllm.ai/deepseek-ai/DeepSeek-V4-Flash?hardware=b300&features=tool_calling,reasoning) for other hardware):

```bash
vllm serve abacusai/Smaug-Flash \
  --trust-remote-code --kv-cache-dtype fp8 --block-size 256 \
  --data-parallel-size 4 --enable-expert-parallel \
  --moe-backend deep_gemm_mega_moe \
  --attention-config '{"use_fp4_indexer_cache": true}' \
  --speculative-config '{"method":"dspark","num_speculative_tokens":7,"draft_sample_method":"greedy"}'
```

### Running with SGLang

```bash
sglang serve \
  --trust-remote-code \
  --model-path abacusai/Smaug-Flash \
  --tp 4 \
  --enable-dp-attention \
  --moe-runner-backend flashinfer_mxfp4 \
  --speculative-algorithm DSPARK \
  --mem-fraction-static 0.90 \
  --chunked-prefill-size 4096 \
  --swa-full-tokens-ratio 0.1
```

For the `high` and `max` reasoning effort levels we recommend a maximum output length of **384K** tokens. Local weight conversion and interactive demos: see the [`inference`](inference/README.md) folder.

## 7. License

This repository and the model weights are licensed under the [MIT License](LICENSE), matching the base release.

## 8. Citation

```bibtex
@misc{abacusai2026smaugflash,
  title  = {Smaug-Flash},
  author = {Abacus.AI},
  year   = {2026},
  note   = {Agentic fine-tune of deepseek-ai/DeepSeek-V4-Flash-0731},
  url    = {https://huggingface.co/abacusai/Smaug-Flash}
}
```

Smaug-Flash builds on DeepSeek-V4-Flash-0731 by DeepSeek-AI:

```bibtex
@misc{deepseekai2026deepseekv4,
      title={DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligence},
      author={DeepSeek-AI},
      year={2026},
}
```

## 9. Contact Us

If you have any questions, please reach out at [Abacus.AI](https://abacus.ai).