--- license: apache-2.0 library_name: transformers base_model: Qwen/Qwen3.8-27B base_model_relation: finetune pipeline_tag: image-text-to-text tags: - agentic - smaug - abacusai ---
Smaug-Mini
Homepage Hugging Face
License
## 1. Model Introduction Smaug-Mini is an agentic finetune of [Qwen3.8-27B](https://huggingface.co/Qwen/Qwen3.8-27B), finetuned by Abacus.AI. It improves end-to-end agentic tool use and automation performance (+4.5 on AutomationBench, +17.1 on JobBench, +2.0 overall on LiveBench, +2.5 on IFBench) while holding the base model's general capabilities at parity (GPQA-diamond at the official number, LiveBench coding and agentic-coding at board parity). The weights load exactly like the official release: same layout, same `Qwen3_5ForConditionalGeneration` architecture with the vision tower intact, same 262,144-token context, same reasoning interface with `xhigh` / `medium` / `low` effort levels. This card describes the training approach and the evaluation results. Dataset contents are not disclosed; training data consists of multi-turn, tool-using automation episodes generated and verified against executable environments. ## 2. Model Summary
ArchitectureDense hybrid-attention transformer + vision tower
Total Parameters27B
Number of Layers64 (48 linear-attention + 16 full-attention, 3:1 interleave)
Attention MechanismGated linear attention & full attention (GQA)
Hidden Dimension5120
Number of Attention Heads24 (4 KV heads)
Vision Encoder27-layer ViT, patch 16
Vocabulary Size~248K
Context Length262,144
Multi-Token Prediction1-layer MTP head (inherited; leave speculative decoding off)
Precisionbfloat16
ModalityText, Image
Base ModelQwen/Qwen3.8-27B
AdaptationOn-policy RL (GRPO), LoRA merged as full delta (language trunk only)
## 3. Evaluation ### Smaug-Mini vs its base *Higher is better. Base points from the Qwen3.8-27B model card and livebench.ai.*
Smaug-Mini Qwen3.8-27B
(base)
 Qwen3.6-27B Qwen3.7-Plus Opus4.6 Max
Agentic
AutomationBench 41.8 37.3 — 20.4 25.5
JobBench 50.5 33.4 21.8 27.6 36.9
LiveBench agentic coding 60.8 61.4 39.3 — 49.0
NL2Repo-Bench 55.8 42.3 36.2 41.1 47.6
Reasoning, knowledge & instruction following
GPQA-diamond 89.4 89.2 87.8 90.3 91.3
HLE 34.2 30.8 24.0 34.7 40.0
IFBench 82.0 79.5 69.1 79.1 62.5
LiveBench overall 76.9 75.3 64.0 — 74.5
Vision
MMMU-Pro 75.6 76.3 75 80 75
Bold = best score in the row. Shading is relative within each row (darker = higher). Qwen3.6-27B, Qwen3.7-Plus, and Opus4.6 Max scores are as reported on the Qwen3.8-27B model card unless noted; the Opus 4.6 AutomationBench score is from our own harness run and its JobBench score is a reported number, not our run; LiveBench rows use the public livebench.ai leaderboard (its Opus 4.6 entry runs at high effort); the Qwen3.7-Plus AutomationBench score and all MMMU-Pro comparison scores are from Artificial Analysis (its Opus 4.6 MMMU-Pro entry runs at max effort); — = not reported.
### LiveBench category profile *Scores 0–100; overall = mean of the seven category averages. Both models' scores are the published [livebench.ai](https://livebench.ai) leaderboard entries (Smaug-Mini listed under the finetunes filter).*
Smaug-Mini LiveBench category profile vs Qwen3.8-27B board entry
Notes All Smaug-Mini numbers were produced at temperature 1.0, top_p 0.95, reasoning effort `xhigh`, with generation budgets large enough that no score is truncation-bound. Base points: JobBench, IFBench, and GPQA-diamond are the scores published on the Qwen3.8-27B model card; the AutomationBench base score is from our own run of the base model. JobBench was run under the official protocol with the OpenCode scaffold and an LLM judge.
## 4. Training Approach Smaug-Mini was trained to make multi-turn tool use and automation episodes more reliable end to end. It is trained with on-policy reinforcement learning (GRPO) over multi-turn, tool-using automation episodes with verified, outcome-based rewards - an episode is rewarded for measurably completing its task in the environment. Training is a LoRA adapter over the language trunk only, merged into the base weights as a full delta; the vision tower and multimodal projection are bitwise-identical to the base release. One deployment note: the multi-token-prediction (MTP) head is inherited from the base model and was not retrained against the updated trunk. Speculative decoding via MTP should be left off; standard decoding is unaffected. ## 5. Known Behaviors and Limitations Smaug-Mini redistributes deliberation rather than adding it: more reasoning per action but fewer actions per episode, finishing passing episodes about three steps sooner at essentially unchanged total reasoning volume. The clearest shift is in failure behavior — episodes that burn the entire step budget without finishing drop from 3.4% to 1.0%. Refusals are zero for both models, and the general-capability benchmarks in §3 show the deliberation shift does not come at the cost of base skills. ## 6. Deployment Smaug-Mini serves as a drop-in replacement for Qwen3.8-27B on any stack that supports the base model. Reference vLLM invocation: ```bash vllm serve abacusai/Smaug-Mini \ --max-model-len 262144 \ --reasoning-parser qwen3 \ --enable-auto-tool-choice --tool-call-parser qwen3_coder ``` Recommended sampling for agentic use: temperature 1.0, top_p 0.95, reasoning effort `xhigh` (the default). Recent vLLM versions return the chain-of-thought in the `reasoning` field of the message; when replaying conversation history, pass reasoning back under both `reasoning` and `reasoning_content` keys for portability across serving versions. Leave MTP-based speculative decoding off (§4). ## 7. License Apache 2.0, inherited from Qwen3.8-27B. ## 8. Citation ```bibtex @misc{abacusai2026smaugmini, title = {Smaug-Mini}, author = {Abacus.AI}, year = {2026}, note = {Agentic fine-tune of Qwen/Qwen3.8-27B}, url = {https://huggingface.co/abacusai/Smaug-Mini} } ``` The Smaug line and the DPO-Positive method behind it: ```bibtex @article{pal2024smaug, title={Smaug: Fixing Failure Modes of Preference Optimisation with DPO-Positive}, author={Pal, Arka and Karkhanis, Deep and Dooley, Samuel and Roberts, Manley and Naidu, Siddartha and White, Colin}, journal={arXiv preprint arXiv:2402.13228}, year={2024} } ``` ## 9. Contact Us If you have any questions, please reach out at [Abacus.AI](https://abacus.ai).