dbishwarup/TagoreGPT-29M

TagoreGPT 29M

A small (29 million parameter) model that writes poems, songs, stories, letters, essays and plays in the style of Rabindranath Tagore (1861–1941), in Bengali or English. Trained from scratch on a laptop.

Sir Rabindranath Tagore (Gurudev) was the first non-European to win the Nobel Prize in Literature (1913, for Gitanjali) and wrote the national anthems of India and Bangladesh.

How to ask

Ask in plain words. The app/chat template wraps it as <|request|>…<|response|>:

• write a poem about the river in Bengali
• write a song about the rain
• একটা কবিতা লেখো বৃষ্টি নিয়ে
• write a letter to a friend in English

Settings that work well: temperature 0.8, top-k 40, repeat penalty 1.15 (last 256 tokens), up to ~400 new tokens. Without a repeat penalty the model can loop (LM Studio: set "Repeat Penalty" to 1.15).

LM Studio / llama.cpp / Ollama

• File: tagoregpt-29m-f16.gguf (Llama layout, 58 MB).
• llama.cpp: llama-cli -m tagoregpt-29m-f16.gguf -cnv --temp 0.8 --top-k 40
• Ollama: ollama create tagoregpt -f Modelfile then ollama run tagoregpt "write a poem about the river"

Transformers

from transformers import AutoTokenizer, AutoModelForCausalLM
tok = AutoTokenizer.from_pretrained("PATH_OR_REPO")
model = AutoModelForCausalLM.from_pretrained("PATH_OR_REPO")
ids = tok.apply_chat_template([{"role": "user", "content": "write a poem about the river"}], return_tensors="pt")
out = model.generate(ids, max_new_tokens=300)
print(tok.decode(out[0][ids.shape[1]:], skip_special_tokens=True))

What it cannot do

• It is not a chatbot: it does not answer questions, explain, or translate. It only writes.
• Topics given in English for a Bengali piece are not translated outside the TagoreGPT app.
• It sees about 1,024 word pieces (~3,300 letters) at a time; longer pieces can wander. (The TagoreGPT app also keeps the title and opening paragraph in view; plain llama.cpp / LM Studio do not.)
• It is small: lines can be beautiful or nonsensical. It is a style imitation, never Tagore's real words.

Training

• Llama layout: 8 layers, width 512, 8 heads, SwiGLU 1344, rotary positions, tied embeddings. 8,000 SentencePiece word pieces (Bengali + English).
• Phase 1 (warm-up): simple practice stories (TinyStories, CDLA-Sharing-1.0; Regional-TinyStories Bengali, Apache-2.0), other Bengali writers (Bengali Wikisource) + 19th-century English (Project Gutenberg) + Tagore.
• Phase 2: Tagore only (Bengali scans from the Digital Library of India / Internet Archive, English from Project Gutenberg).
• Every training piece starts with hidden labels (language, type, author); its title and opening paragraph stay in view, which keeps long pieces on their hero and topic.
• Weights in this release: tagore step 4999. Unseen-text loss per word piece: tagore_en: 3.426, tagore_bn: 3.912.

Data and licence

Tagore's works are in the public domain. The Bengali warm-up text comes from Bengali Wikisource (CC BY-SA 4.0), so this model is released under CC BY-SA 4.0. English text from Project Gutenberg (public domain).