---
license: apache-2.0
language: [en]
library_name: transformers
pipeline_tag: text-generation
base_model: udgamlabs/Laghu-Base-32K
tags: [udgamlabs, small-language-model, trained-from-scratch, research-release, gemma4]
---

![Laghu · Udgam Labs](./banner.png)

**UDGAM LABS** · Language model · 152M parameters, fully open

# Laghu

Laghu is a 152M-parameter English chat model trained from scratch by Udgam Labs for $146 of pretraining compute.
It thinks before it answers (on/off), holds multi-turn conversations and chats, in 163 MB that runs in your browser.
It is level with SmolLM2-135M-Instruct on chat quality and multi-turn conversation, ahead of it on maths with thinking,
and level with our own 384M model Gargi on chat quality and multi-turn at 40% of the size. Its base model predicts new
text (published September-October 2026) better than SmolLM2-135M after reading about 28 times fewer tokens (0.919 vs
0.941 bits per byte). Falcon-H1-Tiny-90M-Instruct, trained with about 12 times our compute and a preference-tuning
stage, is ahead on all three of our chat tests, and Laghu's maths (GSM8K 7.9%) and instruction following (IFEval 26.1)
are weak. It is a fully open research release: weights, surviving checkpoints, code, the full recipe, evaluation
outputs item by item, failures and cost.

*Laghu* (लघु) is Sanskrit for "small, light".

## Run it

| Where | How |
|---|---|
| **Your browser**, no install | [udgamlabs.com/models/laghu](https://udgamlabs.com/models/laghu): the 163 MB model downloads once and runs on your device; your messages never leave it |
| **Hugging Face Space** | [spaces/udgamlabs/Laghu](https://huggingface.co/spaces/udgamlabs/Laghu) |
| **Ollama** | `ollama run hf.co/udgamlabs/Laghu-GGUF:Q8_0` |
| **LM Studio** | search for `udgamlabs/Laghu-GGUF` in LM Studio, or "Use this model" → LM Studio on [Laghu-GGUF](https://huggingface.co/udgamlabs/Laghu-GGUF) |
| **llama.cpp** | `llama-server -hf udgamlabs/Laghu-GGUF:Q8_0 --jinja --reasoning off -c 16384` |
| **MLX** on a Mac | `mlx_lm.generate --model udgamlabs/Laghu-MLX-8bit --prompt "Who are you?" --chat-template-config '{"enable_thinking": false}'` |
| **transformers** | below |

**Thinking** is off by default in transformers and on by default in Ollama, LM Studio, llama.cpp and the MLX server. It
helps a little on short arithmetic and hurts most other tasks, so for normal chat turn it off: Ollama `--think=false`
(or `/set nothink` in a chat), LM Studio API `"reasoning_effort": "none"`, llama.cpp `--reasoning off`, MLX server
`--chat-template-args '{"enable_thinking":false}'`. On a Mac mini (Apple M6) Laghu generates about 330 tokens per second
in llama.cpp and 456 in MLX, and llama-server needs about 154 MiB of memory at a 4K context. The code is on
[GitHub](https://github.com/udgamlabs/laghu).

### transformers (5.18 or newer)


```python
from transformers import AutoTokenizer, AutoModelForCausalLM

tok = AutoTokenizer.from_pretrained("udgamlabs/Laghu")
model = AutoModelForCausalLM.from_pretrained("udgamlabs/Laghu")

messages = [{"role": "user", "content": "A pen costs 12 rupees. How much do 7 pens cost?"}]
inputs = tok.apply_chat_template(messages, add_generation_prompt=True, enable_thinking=True,
                                 return_tensors="pt", return_dict=True)
out = model.generate(**inputs, max_new_tokens=512, do_sample=True, temperature=0.7, top_p=0.95, top_k=64)
text = tok.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=False)
answer = text.rpartition("<channel|>")[2]               # the answer follows the thinking block, if there is one
print(answer.replace("<turn|>", "").strip())
```

`enable_thinking=False` (the default) answers directly. Decode with `skip_special_tokens=False` and take the text after the last
`<channel|>` as above: with `skip_special_tokens=True` the thinking text stays in front of the answer.

Sampling defaults (in `generation_config.json`): temperature 0.7, top-p 0.95, top-k 64. Greedy decoding loops more
(see below).

## Results

Measured by us on 2026-10-19 with one harness for every model (vLLM, bf16, greedy, the same items), as fixed in our
[pre-registration](https://github.com/udgamlabs/laghu/blob/main/udgam3/PREREGISTRATION.md) before Laghu was measured.

| Category (metric) | Laghu | SmolLM2-135M-Instruct | Falcon-H1-Tiny-90M-Instruct | MobileLLM-R1.5-140M |
|---|---|---|---|---|
| Maths with thinking (GSM8K test, 1,319 items, strict: a reply cut off by the length limit counts as wrong) | 7.9% | 1.8% | **23.3%** | 0.0% |
| Multi-turn (second answer right, 144 checkable follow-ups) | 41.7% | 37.5% | **54.9%** | 3.5% |
| General chat quality (Laghu's length-controlled win rate against the rival; 324 held-out prompts, both orders; judge DeepSeek-V4.1-Flash) | — | 51.0% [46.5, 55.5] | 28.7% [24.0, 33.6] | 57.3% [27.0, 76.2] |

After a Holm correction over the nine comparisons, Laghu is ahead of SmolLM2-135M-Instruct and MobileLLM-R1.5 on maths
and of MobileLLM-R1.5 on multi-turn, and level with SmolLM2-135M-Instruct on multi-turn and chat quality.
Falcon-H1-Tiny-90M-Instruct, trained with about 12 times our compute and a preference-tuning stage, is ahead in all
three categories. We fixed these categories, rivals and statistics in advance and publish every result as measured.

MobileLLM-R1.5-140M results are reported for noncommercial research under the FAIR Noncommercial Research License; we acknowledge Meta's materials.

Against our own models: chat quality vs Gargi (384M) 45.9% win rate, p = 0.06 (no significant difference); multi-turn
41.7% vs Gargi 49.3% (p = 0.12) and vs Udgam-384M-Chat 40.3% (p = 0.89).

Other measurements (reported, not claimed):

| Check | Laghu, thinking off | Laghu, thinking on | Falcon-H1-Tiny-90M-Instruct | Gargi (best mode) |
|---|---|---|---|---|
| IFEval, prompt-level strict | 26.1 | 17.7 | 60.8 | 26.4 |
| GSM8K (last number) | 4.8 | 8.0 | 23.3 | 35.6 |
| MATH-500 levels 1-3 (238 items) | 5.5 | 6.3 | 23.9 | 19.3 |
| Greedy replies that loop to the 1,024-token limit (324 prompts) | 8.3% | 12.0% | 0.3% | 11.7% |
| Same, with the shipped sampling settings | 1.2% | 1.9% | 0.0% | 2.5% |

Every number with confidence intervals, the other 11 models we measured, and the per-item outputs of every model are in
the [technical report](https://udgamlabs.com/research/laghu), on GitHub and in
[udgamlabs/laghu-evals](https://huggingface.co/datasets/udgamlabs/laghu-evals).

## Where it falls short

- **Maths:** 7.9% on GSM8K with thinking. The thinking is often wrong even when the answer is right. Example (greedy):
  for "A pen costs 12 rupees. How much do 7 pens cost?" it thinks "Convert the price to dollars: 12 ÷ 100 = 0.12 rupees
  per pen. Multiply by 7 pens: 0.12 × 7 = 0.84 rupees" and then answers "7 pens cost 84 rupees."
- **Instructions:** IFEval 26.1, far below Falcon-H1-Tiny-90M-Instruct (60.8). Thinking on makes it worse (17.7).
- **Knowledge:** a 152M model knows little and states wrong facts confidently. Do not rely on it for facts.
- **Loops:** with greedy decoding 8.3% of replies repeat until the length limit (12.0% with thinking on). Use the
  shipped sampling settings (1-2%).
- **Long context:** use up to 16K tokens. A hidden 5-digit code is found 96% of the time at 16K and never at 24K or 32K.
- **Thinking mode** helps maths a little (GSM8K 4.8% → 8.0%) but lowers multi-turn (41.7% → 32.6%) and IFEval.
- **Compute gap:** Falcon-H1-Tiny-90M-Instruct, which beats it everywhere we measured, trained on 800B tokens with a
  preference-tuning stage, about 12× Laghu's training compute (from TII's published description; our estimate).

## How it was trained

| Stage | What | Compute |
|---|---|---|
| Pretraining | 40.0B token positions at 2,048 context from 14 public sources (FineWeb-Edu, DCLM-Edu, FinePhrase, Ultra-FineWeb, FineMath, SYNTH, Wikipedia, permissive GitHub code, NVIDIA Nemotron datasets); Muon; token bags for the first 25% of steps (about 70B tokens read in all); cool-down data from 80%; weight EMA | 8× A100 80GB, 13.2 h, $146.26 |
| 32K stretch | 996M tokens at 32,768 context: long documents (books, papers, government text) plus replay | 1× RTX PRO 6000, ~4 h |
| Supervised fine-tuning | 41,467 conversations (34.5% multi-turn): DeepSeek-V4.1-Flash answers to real-user-style prompts, gpt-oss-120b multi-turn conversations, verified maths, logic and code data, identity, safety; 2 passes, chosen by a rule written before measuring | 1× RTX PRO 6000, < 1 h |

Training compute ≈ 3.6 × 10¹⁹ FLOPs (6 × parameters × token positions). All-in cash cost about $194, including data
generation, test runs and the judge. The training data was decontaminated against the evaluation sets (13-gram).
Reinforcement learning (GRPO with distillation from Gargi) was tried and failed its gate, so this is the SFT model.
Full recipe, data sources and licences, and every failure: [technical report](https://udgamlabs.com/research/laghu) and
[GitHub](https://github.com/udgamlabs/laghu).

**Checkpoints:** [Laghu-Base](https://huggingface.co/udgamlabs/Laghu-Base) (end of pretraining),
[Laghu-Base-32K](https://huggingface.co/udgamlabs/Laghu-Base-32K) (start of SFT) and
[Laghu-Checkpoints](https://huggingface.co/udgamlabs/Laghu-Checkpoints) (raw training states with optimiser, SFT 1/2/3
passes, the failed RL run). The checkpoints saved every 2,000 pretraining steps were deleted by a storage clean-up on
2026-10-21, our mistake; only the late states survive.

## Intended use and limitations

Research on small language models: training dynamics, fine-tuning, evaluation, distillation, on-device experiments, and
teaching. It is not suitable for factual answers, advice (medical, legal, financial) or any decision that matters. It
was trained on English only. Its safety training is light; it can produce wrong, biased or harmful text.

## Licence

Apache-2.0. Copyright 2026 Udgam Labs LLP. The chat template is Google's Gemma 4 template (Apache-2.0). Training-data
credits and licences: [DATA_LICENSES](https://github.com/udgamlabs/laghu/blob/main/docs/DATA_LICENSES.md).

## Citation

```bibtex
@misc{udgamlabs2026laghu,
  title  = {Laghu: a fully open 152M chat model trained from scratch},
  author = {Prasad, Jitesh K},
  year   = {2026},
  howpublished = {Udgam Labs},
  url    = {https://huggingface.co/udgamlabs/Laghu}
}
```

---

**About Udgam Labs.** Udgam Labs is an independent AI research lab. We work from first principles: before we build, we understand why things work, whether we are training a model, fine-tuning one or applying AI to a new problem. Udgam (उद्गम) is Sanskrit for origin. [udgamlabs.com](https://udgamlabs.com)