---
license: apache-2.0
language: [en]
library_name: transformers
pipeline_tag: text-generation
tags: [auroragpt, small-language-model, distillation, tool-calling, math, on-device, gguf]
---

# AuroraGPT-Math (700M)

A **707M-parameter** chat model by **UltraLabs**, continuing from [AuroraGPT-Qwen-Distill](https://huggingface.co/SmallAICreator/AuroraGPT-Qwen-Distill) with a dedicated push on **real arithmetic** and **multi-turn context use**, while keeping the same chat feel and tool-calling.

## What's new vs AuroraGPT-Qwen-Distill
Continued full-parameter SFT (2 epochs, sequence-packed) adding:
- **~250k procedurally generated, correct-by-construction chain-of-thought math examples** (addition/subtraction/multiplication/division/percentages/fractions/order-of-operations/word problems) — every answer computed in Python, so labels are guaranteed correct, teaching the model to follow the *algorithm* (e.g. distributive-breakdown multiplication) rather than memorize answers.
- **~8k multi-turn context-use conversations**: summarize-that, explain-simpler, compare-two-things, recall-your-own-first-question, incremental list continuation, name/topic correction, topic-switch-and-return.
- The **full original distilled-chat + tool-call + identity/greeting-fix dataset** mixed back in for balance, so the new skills don't come at the cost of chat feel.

## Measured results (local benchmark vs the prior flagship, 16 closed-book arithmetic Qs)
- **Math: 5/16 → 10/16** (roughly doubled)
- **Tool-calling: 3/3 → 3/3** (zero regression — web_search / calculator / fetch_url all still fire correctly)
- **Chat feel / identity / instruction-following:** essentially unchanged

## Honest limitations
- **Harder arithmetic is still fragile** — multi-digit subtraction with borrowing and 2-digit×2-digit multiplication are still error-prone. Use the `calculator` tool for anything you need to be exactly right.
- **False-premise correction is unchanged** (not targeted this round) — it can still confidently agree with a popular myth rather than catching it.
- 700M-scale limits still apply: closed-book knowledge is limited by design — pair with `web_search`/`fetch_url` for facts outside its training.

## Chat format (NOT ChatML)
```
<|system|>{system}<|end|><|user|>{user}<|end|><|assistant|>{reply}<|end|>
```
Tool call (model emits): `<tool_call>\n{"name": "...", "arguments": {...}}\n</tool_call>`
Tool result (feed back as a user turn): `<|user|><tool_response>\n{result}\n</tool_response><|end|>`

## Usage (transformers)
```python
from transformers import AutoModelForCausalLM, AutoTokenizer
tok = AutoTokenizer.from_pretrained("SmallAICreator/AuroraGPT-Math")
model = AutoModelForCausalLM.from_pretrained("SmallAICreator/AuroraGPT-Math")
msgs = [{"role": "user", "content": "What is 47 times 23? Show your work."}]
ids = tok.apply_chat_template(msgs, add_generation_prompt=True, return_tensors="pt")
print(tok.decode(model.generate(ids, max_new_tokens=100)[0][ids.shape[1]:], skip_special_tokens=True))
```

## On-device (llama.cpp / GGUF)
A ready-to-run **Q8_0 GGUF** (`AuroraGPT-Math.Q8_0.gguf`, 753MB) is included, with a tool-capable chat template embedded so mobile GGUF apps show the tool picker.

---
Made by **UltraLabs**. EOS token is `<|end|>`.