Add vs-base benchmark figure, Abacus header, tags, citation
Browse files- README.md +44 -2
- figures/smaug_mini_vs_base.png +0 -0
README.md
CHANGED
|
@@ -7,7 +7,21 @@ pipeline_tag: image-text-to-text
|
|
| 7 |
tags:
|
| 8 |
- agentic
|
| 9 |
- smaug
|
|
|
|
| 10 |
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 11 |
|
| 12 |
# Smaug-Mini
|
| 13 |
|
|
@@ -31,6 +45,10 @@ One deployment note: the multi-token-prediction (MTP) head is inherited from the
|
|
| 31 |
|
| 32 |
## 3. Evaluation
|
| 33 |
|
|
|
|
|
|
|
|
|
|
|
|
|
| 34 |
All Smaug-Mini numbers below were produced at temperature 1.0, top_p 0.95, reasoning effort `xhigh`, with generation budgets large enough that no score is truncation-bound. Base-model comparison points are the published Qwen3.8-27B model-card numbers and the public [livebench.ai](https://livebench.ai) leaderboard entries, never our own reruns of the base.
|
| 35 |
|
| 36 |
### Agentic
|
|
@@ -93,10 +111,34 @@ vllm serve abacusai/Smaug-Mini \
|
|
| 93 |
|
| 94 |
Recommended sampling for agentic use: temperature 1.0, top_p 0.95, reasoning effort `xhigh` (the default). Recent vLLM versions return the chain-of-thought in the `reasoning` field of the message; when replaying conversation history, pass reasoning back under both `reasoning` and `reasoning_content` keys for portability across serving versions. Leave MTP-based speculative decoding off (§2).
|
| 95 |
|
| 96 |
-
## License
|
| 97 |
|
| 98 |
Apache 2.0, inherited from Qwen3.8-27B.
|
| 99 |
|
| 100 |
-
##
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 101 |
|
| 102 |
For questions about Smaug-Mini, reach out at [https://abacus.ai](https://abacus.ai).
|
|
|
|
| 7 |
tags:
|
| 8 |
- agentic
|
| 9 |
- smaug
|
| 10 |
+
- abacusai
|
| 11 |
---
|
| 12 |
+
<div align="center">
|
| 13 |
+
<picture>
|
| 14 |
+
<img src="https://cdn-avatars.huggingface.co/v1/production/uploads/63128dd099791aa61d180c72/dJ6uR23m09M-YEafBpmea.png" width="20%" alt="Smaug-Mini">
|
| 15 |
+
</picture>
|
| 16 |
+
</div>
|
| 17 |
+
<hr>
|
| 18 |
+
<div align="center" style="line-height:1">
|
| 19 |
+
<a href="https://abacus.ai" target="_blank"><img alt="Homepage" src="https://img.shields.io/badge/Homepage-Abacus.AI-white?color=1783ff&logoColor=white"/></a>
|
| 20 |
+
<a href="https://huggingface.co/abacusai" target="_blank"><img alt="Hugging Face" src="https://img.shields.io/badge/%F0%9F%A4%97%20Hugging%20Face-Abacus.AI-ffc107?color=ffc107&logoColor=white"/></a>
|
| 21 |
+
</div>
|
| 22 |
+
<div align="center" style="line-height: 1;">
|
| 23 |
+
<a href="LICENSE"><img alt="License" src="https://img.shields.io/badge/License-Apache_2.0-f5de53?&color=f5de53"/></a>
|
| 24 |
+
</div>
|
| 25 |
|
| 26 |
# Smaug-Mini
|
| 27 |
|
|
|
|
| 45 |
|
| 46 |
## 3. Evaluation
|
| 47 |
|
| 48 |
+
<div align="center">
|
| 49 |
+
<img src="figures/smaug_mini_vs_base.png" width="90%" alt="Smaug-Mini vs its base across benchmarks">
|
| 50 |
+
</div>
|
| 51 |
+
|
| 52 |
All Smaug-Mini numbers below were produced at temperature 1.0, top_p 0.95, reasoning effort `xhigh`, with generation budgets large enough that no score is truncation-bound. Base-model comparison points are the published Qwen3.8-27B model-card numbers and the public [livebench.ai](https://livebench.ai) leaderboard entries, never our own reruns of the base.
|
| 53 |
|
| 54 |
### Agentic
|
|
|
|
| 111 |
|
| 112 |
Recommended sampling for agentic use: temperature 1.0, top_p 0.95, reasoning effort `xhigh` (the default). Recent vLLM versions return the chain-of-thought in the `reasoning` field of the message; when replaying conversation history, pass reasoning back under both `reasoning` and `reasoning_content` keys for portability across serving versions. Leave MTP-based speculative decoding off (§2).
|
| 113 |
|
| 114 |
+
## 6. License
|
| 115 |
|
| 116 |
Apache 2.0, inherited from Qwen3.8-27B.
|
| 117 |
|
| 118 |
+
## 7. Citation
|
| 119 |
+
|
| 120 |
+
```bibtex
|
| 121 |
+
@misc{abacusai2026smaugmini,
|
| 122 |
+
title = {Smaug-Mini},
|
| 123 |
+
author = {Abacus.AI},
|
| 124 |
+
year = {2026},
|
| 125 |
+
note = {Agentic fine-tune of Qwen/Qwen3.8-27B},
|
| 126 |
+
url = {https://huggingface.co/abacusai/Smaug-Mini}
|
| 127 |
+
}
|
| 128 |
+
```
|
| 129 |
+
|
| 130 |
+
The Smaug line and the DPO-Positive method behind it:
|
| 131 |
+
|
| 132 |
+
```bibtex
|
| 133 |
+
@article{pal2024smaug,
|
| 134 |
+
title={Smaug: Fixing Failure Modes of Preference Optimisation with DPO-Positive},
|
| 135 |
+
author={Pal, Arka and Karkhanis, Deep and Dooley, Samuel and
|
| 136 |
+
Roberts, Manley and Naidu, Siddartha and White, Colin},
|
| 137 |
+
journal={arXiv preprint arXiv:2402.13228},
|
| 138 |
+
year={2024}
|
| 139 |
+
}
|
| 140 |
+
```
|
| 141 |
+
|
| 142 |
+
## 8. Contact us
|
| 143 |
|
| 144 |
For questions about Smaug-Mini, reach out at [https://abacus.ai](https://abacus.ai).
|
figures/smaug_mini_vs_base.png
ADDED
|