UDGAM LABS · Training checkpoints · Vemana, 278M parameters, Telugu

Vemana-Checkpoints

Twelve checkpoints from the training run of Vemana-Base, a 278M Telugu language model trained from scratch on 12.0B token positions (one H100, about 16 hours, 26-27 October 2026, live at udgamlabs.com/live/vemana). Each folder is a complete Hugging Face model (Gemma4ForCausalLM, float32) with the tokenizer, so any of them loads on its own:

from transformers import AutoTokenizer, AutoModelForCausalLM

tok = AutoTokenizer.from_pretrained("udgamlabs/Vemana-Checkpoints", subfolder="m050-step-011444")
model = AutoModelForCausalLM.from_pretrained("udgamlabs/Vemana-Checkpoints", subfolder="m050-step-011444")

Folder	Step	Token positions	Fresh-text bits per byte (416 pages, Sep-Oct 2026)
m010-step-002289	2,289	1.2B	0.420
m020-step-004578	4,578	2.4B	0.414
stable-025-step-005722	5,722	3.0B	0.405
gate-step-005722	5,722	3.0B	0.383 (a copy cooled down from step 4,578, scored as a finished model)
m030-step-006866	6,866	3.6B	0.406
m040-step-009155	9,155	4.8B	0.400
m050-step-011444	11,444	6.0B	0.398
m060-step-013733	13,733	7.2B	0.398
m070-step-016022	16,022	8.4B	0.398
m080-step-018310	18,310	9.6B	0.393 (start of the final cool-down)
m090-step-020599	20,599	10.8B	0.371
final-step-022888	22,888	12.0B	0.361 (= Vemana-Base)

The scores use the same script and pages as the Vemana-Base card (lower is better). Most of the gain comes in the cool-down: the stable checkpoints level off near 0.398 between 40% and 70% of the run, and the last 20% brings the score to 0.361.

Each folder also holds fresh.json (the per-page scores behind the table) and train_state.json (the run's step, schedule settings and the text it wrote at that point).

Licence

Apache-2.0. Copyright 2026 Udgam Labs LLP. Training data credits are in the Vemana-Base NOTICE.

---

About Udgam Labs. Udgam Labs is an independent AI research lab. We work from first principles: before we build, we understand why things work, whether we are training a model, fine-tuning one or applying AI to a new problem. Udgam (उद्गम) is Sanskrit for origin. udgamlabs.com