Updated README 🚀

README.md

@@ -5,4 +5,62 @@ datasets:
 - openai/gsm8k
 language:
 - en
----
+---
+
+# NanoMath: A 136M Parameter Math-Reasoning Language Model
+
+## 🚀 Model Overview
+
+NanoMath is a custom-built, GPT-style Large Language Model explicitly designed to solve mathematical problems using Chain-of-Thought (CoT) reasoning. Built entirely from scratch in PyTorch, the primary goal of this model is to demonstrate an end-to-end language modeling pipeline, helping to understand transformer architectures, training optimizations, and data processing bottlenecks.
+
+## 🛠️ Model Details
+
+* **Architecture:** Decoder-only Transformer (GPT-style) built from scratch.
+* **Parameters:** 136 Million.
+* **Training Compute:** Trained for 12 hours on a single NVIDIA T4 GPU via Kaggle.
+* **Tokenizer:** Custom Byte-Pair Encoding (BPE) tokenizer trained using `sentencepiece` to explicitly recognize mathematical operators and digits efficiently.
+
+## 📚 Training Data
+
+The model was trained on approximately 1 Billion tokens. The dataset is a combination of high-quality math datasets and procedural generations:
+
+* **Synthetic Basic Math:** Hundreds of thousands of synthetic arithmetic problems generated via a custom Python script.
+* **GSM8K:** Gold-standard grade school math word problems.
+* **NuminaMath:** Large-scale math reasoning problems.
+
+To teach the model step-by-step reasoning, the data enforces a strict `<|thinking|>` and `<|answer|>` formatting structure.
+
+## 🧠 Performance and Capabilities
+
+NanoMath was trained to "think before it speaks". It successfully generates intermediate reasoning steps and performs well on basic arithmetic tasks.
+
+**Successes:**
+
+* The model successfully executes multi-step algorithmic logic like partial products.
+* It correctly utilizes long division from left to right.
+
+## ⚠️ Limitations
+
+Given its small scale (136M parameters) and heavily constrained training time (12 hours), the model exhibits standard LLM hallucinations on complex operations.
+
+* **Multi-Step Borrowing:** It sometimes struggles to carry values perfectly across complex mathematical borrowing steps.
+* **Logic Routing:** It can lose track of logical routing in word problems, leading to hallucinated results.
+
+## 💻 How to Use
+
+Because this is a custom model architecture built from scratch, you cannot use the standard HuggingFace `transformers` library out-of-the-box. Instead, download the weights and run inference using the codebase provided in the linked GitHub repository.
+
+1. Download the `model_weights.pth` and `token.model` files from this repository.
+2. Clone the [NanoMath GitHub repository](https://github.com/agmada-asa/NanoMath) and install the dependencies:
+```bash
+pip install -r requirements.txt
+
+```
+
+
+3. Place the downloaded `.pth` and `.model` files into the `build/` directory of the cloned repository.
+4. Run the interactive chat CLI:
+```bash
+python chat.py
+
+```