---
language: [bn, en]
license: cc-by-sa-4.0
pipeline_tag: text-generation
library_name: transformers
tags: [llama, bengali, tagore, poetry, small-model, gguf]
---

# TagoreGPT 29M

A small (29 million parameter) model that writes **poems, songs, stories, letters, essays and plays in the style of
Rabindranath Tagore (1861–1941)**, in **Bengali or English**. Trained from scratch on a laptop.

Sir Rabindranath Tagore (Gurudev) was the first non-European to win the Nobel Prize in Literature (1913, for
*Gitanjali*) and wrote the national anthems of India and Bangladesh.

## How to ask

Ask in plain words. The app/chat template wraps it as `<|request|>…<|response|>`:

- `write a poem about the river in Bengali`
- `write a song about the rain`
- `একটা কবিতা লেখো বৃষ্টি নিয়ে`
- `write a letter to a friend in English`

Settings that work well: temperature 0.8, top-k 40, **repeat penalty 1.15** (last 256 tokens), up to ~400 new tokens.
Without a repeat penalty the model can loop (LM Studio: set "Repeat Penalty" to 1.15).

### LM Studio / llama.cpp / Ollama

- File: `tagoregpt-29m-f16.gguf` (Llama layout, 58 MB).
- llama.cpp: `llama-cli -m tagoregpt-29m-f16.gguf -cnv --temp 0.8 --top-k 40`
- Ollama: `ollama create tagoregpt -f Modelfile` then `ollama run tagoregpt "write a poem about the river"`

### Transformers

```python
from transformers import AutoTokenizer, AutoModelForCausalLM
tok = AutoTokenizer.from_pretrained("PATH_OR_REPO")
model = AutoModelForCausalLM.from_pretrained("PATH_OR_REPO")
ids = tok.apply_chat_template([{"role": "user", "content": "write a poem about the river"}], return_tensors="pt")
out = model.generate(ids, max_new_tokens=300)
print(tok.decode(out[0][ids.shape[1]:], skip_special_tokens=True))
```

## What it cannot do

- It is not a chatbot: it does not answer questions, explain, or translate. It only writes.
- Topics given in English for a Bengali piece are not translated outside the TagoreGPT app.
- It sees about 1,024 word pieces (~3,300 letters) at a time; longer pieces can wander. (The TagoreGPT app also
  keeps the title and opening paragraph in view; plain llama.cpp / LM Studio do not.)
- It is small: lines can be beautiful or nonsensical. It is a style imitation, never Tagore's real words.

## Training

- Llama layout: 8 layers, width 512, 8 heads, SwiGLU 1344, rotary positions, tied embeddings. 8,000 SentencePiece word pieces (Bengali + English).
- Phase 1 (warm-up): simple practice stories (TinyStories, CDLA-Sharing-1.0; Regional-TinyStories Bengali, Apache-2.0),
  other Bengali writers (Bengali Wikisource) + 19th-century English (Project Gutenberg) + Tagore.
- Phase 2: Tagore only (Bengali scans from the Digital Library of India / Internet Archive, English from Project Gutenberg).
- Every training piece starts with hidden labels (language, type, author); its title and opening paragraph stay in view, which keeps long pieces on their hero and topic.
- Weights in this release: tagore step -1. Unseen-text loss per word piece: tagore_en: 3.586, tagore_bn: 3.832.

## Data and licence

Tagore's works are in the public domain. The Bengali warm-up text comes from **Bengali Wikisource**
(CC BY-SA 4.0), so this model is released under **CC BY-SA 4.0**. English text from Project Gutenberg (public domain).