Model card for release (27 Oct 2026)

884fee1 verified 3 days ago

2.73 kB

---
license: apache-2.0
language: [te]
library_name: transformers
tags: [udgamlabs, trained-from-scratch, base-model, telugu, checkpoints, gemma4]
---
![Vemana · Udgam Labs](./banner.png)

**UDGAM LABS** · Training checkpoints · Vemana, 278M parameters, Telugu

# Vemana-Checkpoints

Twelve checkpoints from the training run of [Vemana-Base](https://huggingface.co/udgamlabs/Vemana-Base), a 278M Telugu
language model trained from scratch on 12.0B token positions (one H100, about 16 hours, 26-27 October 2026, live at
[udgamlabs.com/live/vemana](https://udgamlabs.com/live/vemana)). Each folder is a complete Hugging Face model
(`Gemma4ForCausalLM`, float32) with the tokenizer, so any of them loads on its own:

```python
from transformers import AutoTokenizer, AutoModelForCausalLM
tok = AutoTokenizer.from_pretrained("udgamlabs/Vemana-Checkpoints", subfolder="m050-step-011444")
model = AutoModelForCausalLM.from_pretrained("udgamlabs/Vemana-Checkpoints", subfolder="m050-step-011444")
```

| Folder | Step | Token positions | Fresh-text bits per byte (416 pages, Sep-Oct 2026) |
|---|---|---|---|
| `m010-step-002289` | 2,289 | 1.2B | 0.420 |
| `m020-step-004578` | 4,578 | 2.4B | 0.414 |
| `stable-025-step-005722` | 5,722 | 3.0B | 0.405 |
| `gate-step-005722` | 5,722 | 3.0B | 0.383 (a copy cooled down from step 4,578, scored as a finished model) |
| `m030-step-006866` | 6,866 | 3.6B | 0.406 |
| `m040-step-009155` | 9,155 | 4.8B | 0.400 |
| `m050-step-011444` | 11,444 | 6.0B | 0.398 |
| `m060-step-013733` | 13,733 | 7.2B | 0.398 |
| `m070-step-016022` | 16,022 | 8.4B | 0.398 |
| `m080-step-018310` | 18,310 | 9.6B | 0.393 (start of the final cool-down) |
| `m090-step-020599` | 20,599 | 10.8B | 0.371 |
| `final-step-022888` | 22,888 | 12.0B | 0.361 (= Vemana-Base) |

The scores use the same script and pages as the [Vemana-Base](https://huggingface.co/udgamlabs/Vemana-Base) card
(lower is better). Most of the gain comes in the cool-down: the stable checkpoints level off near 0.398 between 40% and
70% of the run, and the last 20% brings the score to 0.361.

Each folder also holds `fresh.json` (the per-page scores behind the table) and `train_state.json` (the run's step,
schedule settings and the text it wrote at that point).

## Licence

Apache-2.0. Copyright 2026 Udgam Labs LLP. Training data credits are in the
[Vemana-Base](https://huggingface.co/udgamlabs/Vemana-Base) `NOTICE`.

---

**About Udgam Labs.** Udgam Labs is an independent AI research lab. We work from first principles: before we build, we understand why things work, whether we are training a model, fine-tuning one or applying AI to a new problem. Udgam (उद्गम) is Sanskrit for origin. [udgamlabs.com](https://udgamlabs.com)

	---
	license: apache-2.0
	language: [te]
	library_name: transformers
	tags: [udgamlabs, trained-from-scratch, base-model, telugu, checkpoints, gemma4]
	---

	![Vemana · Udgam Labs](./banner.png)

	UDGAM LABS · Training checkpoints · Vemana, 278M parameters, Telugu

	# Vemana-Checkpoints

	Twelve checkpoints from the training run of [Vemana-Base](https://huggingface.co/udgamlabs/Vemana-Base), a 278M Telugu
	language model trained from scratch on 12.0B token positions (one H100, about 16 hours, 26-27 October 2026, live at
	[udgamlabs.com/live/vemana](https://udgamlabs.com/live/vemana)). Each folder is a complete Hugging Face model
	(`Gemma4ForCausalLM`, float32) with the tokenizer, so any of them loads on its own:

	```python
	from transformers import AutoTokenizer, AutoModelForCausalLM

	tok = AutoTokenizer.from_pretrained("udgamlabs/Vemana-Checkpoints", subfolder="m050-step-011444")
	model = AutoModelForCausalLM.from_pretrained("udgamlabs/Vemana-Checkpoints", subfolder="m050-step-011444")
	```

	\| Folder \| Step \| Token positions \| Fresh-text bits per byte (416 pages, Sep-Oct 2026) \|
	\|---\|---\|---\|---\|
	\| `m010-step-002289` \| 2,289 \| 1.2B \| 0.420 \|
	\| `m020-step-004578` \| 4,578 \| 2.4B \| 0.414 \|
	\| `stable-025-step-005722` \| 5,722 \| 3.0B \| 0.405 \|
	\| `gate-step-005722` \| 5,722 \| 3.0B \| 0.383 (a copy cooled down from step 4,578, scored as a finished model) \|
	\| `m030-step-006866` \| 6,866 \| 3.6B \| 0.406 \|
	\| `m040-step-009155` \| 9,155 \| 4.8B \| 0.400 \|
	\| `m050-step-011444` \| 11,444 \| 6.0B \| 0.398 \|
	\| `m060-step-013733` \| 13,733 \| 7.2B \| 0.398 \|
	\| `m070-step-016022` \| 16,022 \| 8.4B \| 0.398 \|
	\| `m080-step-018310` \| 18,310 \| 9.6B \| 0.393 (start of the final cool-down) \|
	\| `m090-step-020599` \| 20,599 \| 10.8B \| 0.371 \|
	\| `final-step-022888` \| 22,888 \| 12.0B \| 0.361 (= Vemana-Base) \|

	The scores use the same script and pages as the [Vemana-Base](https://huggingface.co/udgamlabs/Vemana-Base) card
	(lower is better). Most of the gain comes in the cool-down: the stable checkpoints level off near 0.398 between 40% and
	70% of the run, and the last 20% brings the score to 0.361.

	Each folder also holds `fresh.json` (the per-page scores behind the table) and `train_state.json` (the run's step,
	schedule settings and the text it wrote at that point).

	## Licence

	Apache-2.0. Copyright 2026 Udgam Labs LLP. Training data credits are in the
	[Vemana-Base](https://huggingface.co/udgamlabs/Vemana-Base) `NOTICE`.

	---

	About Udgam Labs. Udgam Labs is an independent AI research lab. We work from first principles: before we build, we understand why things work, whether we are training a model, fine-tuning one or applying AI to a new problem. Udgam (उद्गम) is Sanskrit for origin. [udgamlabs.com](https://udgamlabs.com)