Instructions to use udgamlabs/Laghu-Base with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use udgamlabs/Laghu-Base with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="udgamlabs/Laghu-Base")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("udgamlabs/Laghu-Base")
model = AutoModelForCausalLM.from_pretrained("udgamlabs/Laghu-Base", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use udgamlabs/Laghu-Base with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "udgamlabs/Laghu-Base"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "udgamlabs/Laghu-Base",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/udgamlabs/Laghu-Base

SGLang

How to use udgamlabs/Laghu-Base with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "udgamlabs/Laghu-Base" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "udgamlabs/Laghu-Base",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "udgamlabs/Laghu-Base" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "udgamlabs/Laghu-Base",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use udgamlabs/Laghu-Base with Docker Model Runner:
```
docker model run hf.co/udgamlabs/Laghu-Base
```

Laghu-Base / README.md

jiteshkprasad

Card: dated rank (private staging)

479e692 verified 11 days ago

preview code

Raw History Blame Contribute Delete

6.28 kB

---
license: apache-2.0
language: [en]
library_name: transformers
pipeline_tag: text-generation
tags: [udgamlabs, small-language-model, trained-from-scratch, research-release, base-model, gemma4]
---
![Laghu-Base · Udgam Labs](./banner.png)

**UDGAM LABS** · Language model · 152M parameters, base

# Laghu-Base

Laghu-Base is the pretrained base model behind [Laghu](https://huggingface.co/udgamlabs/Laghu): 152M parameters,
trained from scratch on 40.0B token positions of English web, educational, maths, code and synthetic text at a
2,048-token context. These are the exponential-moving-average (EMA) weights at the last step, 76,294. On text published in
September and October 2026 it scores 0.919 bits per byte, better than SmolLM2-135M (0.941, pretrained on 2T tokens) and
Falcon-H1-Tiny-90M-Base (0.979), after a $146 pretraining run. It is a text completer, not a chat model. Part of a fully
open research release: code, recipe, checkpoints, evaluations and cost are all public.

**Try it:** [the chat model](https://huggingface.co/udgamlabs/Laghu) · [code on GitHub](https://github.com/udgamlabs/laghu) · [udgamlabs.com/models/laghu](https://udgamlabs.com/models/laghu)

## Quick start

```python
from transformers import AutoTokenizer, AutoModelForCausalLM
tok = AutoTokenizer.from_pretrained("udgamlabs/Laghu-Base")
model = AutoModelForCausalLM.from_pretrained("udgamlabs/Laghu-Base")
inputs = tok("The water cycle begins when", return_tensors="pt")   # the tokenizer adds <bos>
out = model.generate(**inputs, max_new_tokens=60, do_sample=True, temperature=0.8, top_p=0.95)
print(tok.decode(out[0], skip_special_tokens=True))
```

Keep prompts and outputs within 2,048 tokens: the model was trained at that length and fails at longer ones (the config
allows 8,192 positions, but recall at 4K is 0%). For long inputs use [Laghu-Base-32K](https://huggingface.co/udgamlabs/Laghu-Base-32K).
The raw (non-EMA) weights of the same step are in the `raw-final/` folder:
`AutoModelForCausalLM.from_pretrained("udgamlabs/Laghu-Base", subfolder="raw-final")`.

## Results

**Zero-shot benchmarks** (lm-eval 0.4.13, bf16, `<bos>` added, CUDA; measured by us for every model, 2026-10-19):

| Model | Params | ARC-E | ARC-C | HellaSwag | PIQA | WinoGrande | 5-task avg | MMLU | GSM8K 5-shot |
|---|---|---|---|---|---|---|---|---|---|
| **Laghu-Base** | 152M | 60.6 | 30.1 | 40.9 | 65.7 | 52.2 | **49.9** | 25.3 | 4.2 |
| GPT-X3-150M | 149M | 63.0 | 31.4 | 45.9 | 71.3 | 53.6 | 53.0 | 24.6 | 4.2 |
| SmolLM2-135M | 135M | 64.5 | 30.0 | 43.0 | 68.3 | 53.0 | 51.8 | 24.5 | 2.0 |
| Falcon-H1-Tiny-90M-Base | 91M | 54.3 | 28.5 | 37.7 | 64.8 | 50.8 | 47.2 | 32.4 | 2.7 |
| Gargi-Base (ours) | 384M | 63.0 | 32.5 | 41.5 | 66.9 | 54.5 | 51.7 | 25.3 | 5.7 |

ARC-C, HellaSwag and PIQA use length-normalised accuracy. Laghu-Base is 5th of the 19 models up to 200M parameters we
measured on 19 October on the 5-task average; the full table is in the technical report.

**Bits per byte on held-out text** (lower is better; one script and the same documents for every model):

| Model | New text (1,695 docs: Wikipedia pages created and arXiv abstracts submitted Sep-Oct 2026) | Web (580) | Reasoning (1,179) |
|---|---|---|---|
| **Laghu-Base** | **0.919** | 0.858 | 0.693 |
| SmolLM2-135M | 0.941 | 0.858 | 0.752 |
| Falcon-H1-Tiny-90M-Base | 0.979 | 0.910 | 0.769 |
| Gargi-Base (ours, 384M) | 0.914 | 0.850 | 0.686 |
| SmolLM2-360M | 0.850 | 0.788 | 0.678 |

## Where it falls short

- Below SmolLM2-135M (51.8) and GPT-X3-150M (53.0) on the commonsense average; near chance on MMLU (25.3).
- Slightly behind our 384M Gargi-Base on all three held-out text sets (the differences are significant).
- Our scaling projection before the run expected 0.900-0.912 bits per byte on new text after a 32K stretch; the measured
  value was 0.921, so the run came in worse than planned.
- 2,048-token context only.
- Not instruction-tuned: it continues text and does not follow requests.

## How it was trained

- **Data:** a 52.6B-token pool from 14 public sources (DCLM-Edu and FineWeb-Edu ≈ 35%, FinePhrase ≈ 23%, Ultra-FineWeb
  ≈ 14.5%, FineMath ≈ 9%, SYNTH ≈ 8.5%, Wikipedia ≈ 3.4%, NVIDIA Nemotron chat and STEM ≈ 5%, permissive GitHub code ≈
  1.7%), 13-gram decontaminated against 10 evaluation sets (21,427 documents removed). Shares are the builder's own token
  estimates.
- **Run:** 76,294 steps × 524,288 tokens = 40.0B token positions; Muon; token bags (4 tokens per position) for the first
  25% of steps, so about 70B tokens were read; cool-down data mix from 80%; learning rate warm-down over the last 65% to 5%
  of peak; EMA from 90% (β 0.998); bf16.
- **Compute and cost:** 8× A100 80GB for 13.2 hours, $146.26. About 3.6 × 10¹⁹ FLOPs.
- **Architecture:** Gemma-4 text layout, 28 layers, width 640, 10 query / 2 key-value heads, 512-token sliding window with
  every 4th layer global, tied embeddings, our own 32,768-token tokenizer.

Every setting, the data builder and the launch scripts: [GitHub](https://github.com/udgamlabs/laghu). The raw training
states with optimiser at steps 76,000 and 76,294 are in [Laghu-Checkpoints](https://huggingface.co/udgamlabs/Laghu-Checkpoints);
the checkpoints from steps 2,000-74,000 were deleted by a storage clean-up on 2026-10-21, our mistake.

## Intended use and limitations

A base for research: fine-tuning, continued pretraining, interpretability and training-dynamics studies. It has no
safety tuning and continues any text, including harmful or false text.

## Licence

Apache-2.0. Copyright 2026 Udgam Labs LLP. Training-data credits:
[DATA_LICENSES](https://github.com/udgamlabs/laghu/blob/main/docs/DATA_LICENSES.md).

## Citation

```bibtex
@misc{udgamlabs2026laghu,
  title  = {Laghu: a fully open 152M chat model trained from scratch},
  author = {Prasad, Jitesh K},
  year   = {2026},
  howpublished = {Udgam Labs},
  url    = {https://huggingface.co/udgamlabs/Laghu}
}
```

---

**About Udgam Labs.** Udgam Labs is an independent AI research lab. We work from first principles: before we build, we understand why things work, whether we are training a model, fine-tuning one or applying AI to a new problem. Udgam (उद्गम) is Sanskrit for origin. [udgamlabs.com](https://udgamlabs.com)

	---
	license: apache-2.0
	language: [en]
	library_name: transformers
	pipeline_tag: text-generation
	tags: [udgamlabs, small-language-model, trained-from-scratch, research-release, base-model, gemma4]
	---

	![Laghu-Base · Udgam Labs](./banner.png)

	UDGAM LABS · Language model · 152M parameters, base

	# Laghu-Base

	Laghu-Base is the pretrained base model behind [Laghu](https://huggingface.co/udgamlabs/Laghu): 152M parameters,
	trained from scratch on 40.0B token positions of English web, educational, maths, code and synthetic text at a
	2,048-token context. These are the exponential-moving-average (EMA) weights at the last step, 76,294. On text published in
	September and October 2026 it scores 0.919 bits per byte, better than SmolLM2-135M (0.941, pretrained on 2T tokens) and
	Falcon-H1-Tiny-90M-Base (0.979), after a $146 pretraining run. It is a text completer, not a chat model. Part of a fully
	open research release: code, recipe, checkpoints, evaluations and cost are all public.

	Try it: [the chat model](https://huggingface.co/udgamlabs/Laghu) · [code on GitHub](https://github.com/udgamlabs/laghu) · [udgamlabs.com/models/laghu](https://udgamlabs.com/models/laghu)

	## Quick start

	```python
	from transformers import AutoTokenizer, AutoModelForCausalLM

	tok = AutoTokenizer.from_pretrained("udgamlabs/Laghu-Base")
	model = AutoModelForCausalLM.from_pretrained("udgamlabs/Laghu-Base")
	inputs = tok("The water cycle begins when", return_tensors="pt") # the tokenizer adds <bos>
	out = model.generate(**inputs, max_new_tokens=60, do_sample=True, temperature=0.8, top_p=0.95)
	print(tok.decode(out[0], skip_special_tokens=True))
	```

	Keep prompts and outputs within 2,048 tokens: the model was trained at that length and fails at longer ones (the config
	allows 8,192 positions, but recall at 4K is 0%). For long inputs use [Laghu-Base-32K](https://huggingface.co/udgamlabs/Laghu-Base-32K).
	The raw (non-EMA) weights of the same step are in the `raw-final/` folder:
	`AutoModelForCausalLM.from_pretrained("udgamlabs/Laghu-Base", subfolder="raw-final")`.

	## Results

	Zero-shot benchmarks (lm-eval 0.4.13, bf16, `<bos>` added, CUDA; measured by us for every model, 2026-10-19):

	\| Model \| Params \| ARC-E \| ARC-C \| HellaSwag \| PIQA \| WinoGrande \| 5-task avg \| MMLU \| GSM8K 5-shot \|
	\|---\|---\|---\|---\|---\|---\|---\|---\|---\|---\|
	\| Laghu-Base \| 152M \| 60.6 \| 30.1 \| 40.9 \| 65.7 \| 52.2 \| 49.9 \| 25.3 \| 4.2 \|
	\| GPT-X3-150M \| 149M \| 63.0 \| 31.4 \| 45.9 \| 71.3 \| 53.6 \| 53.0 \| 24.6 \| 4.2 \|
	\| SmolLM2-135M \| 135M \| 64.5 \| 30.0 \| 43.0 \| 68.3 \| 53.0 \| 51.8 \| 24.5 \| 2.0 \|
	\| Falcon-H1-Tiny-90M-Base \| 91M \| 54.3 \| 28.5 \| 37.7 \| 64.8 \| 50.8 \| 47.2 \| 32.4 \| 2.7 \|
	\| Gargi-Base (ours) \| 384M \| 63.0 \| 32.5 \| 41.5 \| 66.9 \| 54.5 \| 51.7 \| 25.3 \| 5.7 \|

	ARC-C, HellaSwag and PIQA use length-normalised accuracy. Laghu-Base is 5th of the 19 models up to 200M parameters we
	measured on 19 October on the 5-task average; the full table is in the technical report.

	Bits per byte on held-out text (lower is better; one script and the same documents for every model):

	\| Model \| New text (1,695 docs: Wikipedia pages created and arXiv abstracts submitted Sep-Oct 2026) \| Web (580) \| Reasoning (1,179) \|
	\|---\|---\|---\|---\|
	\| Laghu-Base \| 0.919 \| 0.858 \| 0.693 \|
	\| SmolLM2-135M \| 0.941 \| 0.858 \| 0.752 \|
	\| Falcon-H1-Tiny-90M-Base \| 0.979 \| 0.910 \| 0.769 \|
	\| Gargi-Base (ours, 384M) \| 0.914 \| 0.850 \| 0.686 \|
	\| SmolLM2-360M \| 0.850 \| 0.788 \| 0.678 \|

	## Where it falls short

	- Below SmolLM2-135M (51.8) and GPT-X3-150M (53.0) on the commonsense average; near chance on MMLU (25.3).
	- Slightly behind our 384M Gargi-Base on all three held-out text sets (the differences are significant).
	- Our scaling projection before the run expected 0.900-0.912 bits per byte on new text after a 32K stretch; the measured
	value was 0.921, so the run came in worse than planned.
	- 2,048-token context only.
	- Not instruction-tuned: it continues text and does not follow requests.

	## How it was trained

	- Data: a 52.6B-token pool from 14 public sources (DCLM-Edu and FineWeb-Edu ≈ 35%, FinePhrase ≈ 23%, Ultra-FineWeb
	≈ 14.5%, FineMath ≈ 9%, SYNTH ≈ 8.5%, Wikipedia ≈ 3.4%, NVIDIA Nemotron chat and STEM ≈ 5%, permissive GitHub code ≈
	1.7%), 13-gram decontaminated against 10 evaluation sets (21,427 documents removed). Shares are the builder's own token
	estimates.
	- Run: 76,294 steps × 524,288 tokens = 40.0B token positions; Muon; token bags (4 tokens per position) for the first
	25% of steps, so about 70B tokens were read; cool-down data mix from 80%; learning rate warm-down over the last 65% to 5%
	of peak; EMA from 90% (β 0.998); bf16.
	- Compute and cost: 8× A100 80GB for 13.2 hours, $146.26. About 3.6 × 10¹⁹ FLOPs.
	- Architecture: Gemma-4 text layout, 28 layers, width 640, 10 query / 2 key-value heads, 512-token sliding window with
	every 4th layer global, tied embeddings, our own 32,768-token tokenizer.

	Every setting, the data builder and the launch scripts: [GitHub](https://github.com/udgamlabs/laghu). The raw training
	states with optimiser at steps 76,000 and 76,294 are in [Laghu-Checkpoints](https://huggingface.co/udgamlabs/Laghu-Checkpoints);
	the checkpoints from steps 2,000-74,000 were deleted by a storage clean-up on 2026-10-21, our mistake.

	## Intended use and limitations

	A base for research: fine-tuning, continued pretraining, interpretability and training-dynamics studies. It has no
	safety tuning and continues any text, including harmful or false text.

	## Licence

	Apache-2.0. Copyright 2026 Udgam Labs LLP. Training-data credits:
	[DATA_LICENSES](https://github.com/udgamlabs/laghu/blob/main/docs/DATA_LICENSES.md).

	## Citation

	```bibtex
	@misc{udgamlabs2026laghu,
	title = {Laghu: a fully open 152M chat model trained from scratch},
	author = {Prasad, Jitesh K},
	year = {2026},
	howpublished = {Udgam Labs},
	url = {https://huggingface.co/udgamlabs/Laghu}
	}
	```

	---

	About Udgam Labs. Udgam Labs is an independent AI research lab. We work from first principles: before we build, we understand why things work, whether we are training a model, fine-tuning one or applying AI to a new problem. Udgam (उद्गम) is Sanskrit for origin. [udgamlabs.com](https://udgamlabs.com)