Upload README.md with huggingface_hub

README.md

@@ -0,0 +1,80 @@
+---
+language:
+  - hi
+  - en
+  - bn
+  - ta
+  - te
+  - mr
+  - gu
+  - kn
+  - ml
+  - pa
+  - or
+library_name: transformers
+pipeline_tag: text-generation
+tags:
+  - indic
+  - small-language-model
+  - qwen3
+---
+
+# NextToken-model-1
+
+A 272.7M-parameter base language model pretrained from scratch on 11 Indic
+languages + English (Sangraha corpus). **This is a base next-token model,
+not instruction-tuned** — it continues text, it does not follow
+instructions or answer questions in a chat format. Instruction-tuning for
+a specific downstream task has not been done yet for this model; this
+checkpoint will be updated in place as that work progresses.
+
+## Architecture
+
+Built from custom composable primitives, structurally equivalent to Qwen3
+(confirmed by direct source comparison during HF conversion — see
+"Conversion" below) and saved in that format for standard `transformers`
+loading:
+
+| | |
+|---|---|
+| Parameters | 272.7M (69.7M embedding, tied with output head; 203.0M transformer) |
+| Layers | 16 |
+| Hidden size | 1024 |
+| Attention | GQA — 16 query heads, 4 KV heads |
+| Normalization | RMSNorm (pre-norm) + per-head QK-norm before RoPE |
+| FFN | SwiGLU, intermediate size 3277 |
+| Positional encoding | RoPE (θ=10000) |
+| Context length | 2048 |
+| Tokenizer | [`sarvamai/sarvam-1`](https://huggingface.co/sarvamai/sarvam-1), vocab 68,096 |
+
+## Training
+
+Pretrained on the Sangraha corpus across 11 Indic languages + English.
+Trained on a mix of H100 and V100 GPUs across two phases (~5.7B tokens
+per phase). This checkpoint is from the best validation loss during the
+11-language phase (step 23,500, loss 2.9318).
+
+## Usage
+
+```python
+from transformers import AutoModelForCausalLM, AutoTokenizer
+
+model = AutoModelForCausalLM.from_pretrained("somasekhar-dev/NextToken-model-1")
+tokenizer = AutoTokenizer.from_pretrained("somasekhar-dev/NextToken-model-1")
+
+inputs = tokenizer("भारत एक ऐसा देश है जहाँ", return_tensors="pt")
+out = model.generate(**inputs, max_new_tokens=50, do_sample=True, temperature=0.8)
+print(tokenizer.decode(out[0], skip_special_tokens=True))
+```
+
+## Status and limitations
+
+- **Base model only** — no instruction-tuning, no chat template, no
+  downstream task specialization. Expect raw text continuation, not
+  question-answering or instruction-following behavior.
+- Conversion from the original training format to this `Qwen3ForCausalLM`
+  checkpoint was verified numerically (100% argmax agreement between the
+  original and converted model on random inputs, plus a full
+  `save_pretrained`/`from_pretrained` round-trip) — not just "it loads."
+- This repository will be updated in place as fine-tuning work proceeds;
+  check back for updates rather than assuming this snapshot is final.