| --- |
| datasets: |
| - openai/gsm8k |
| language: |
| - en |
| base_model: |
| - HuggingFaceTB/SmolLM2-135M-Instruct |
| pipeline_tag: text-generation |
| tags: |
| - safetensors |
| - smollm |
| - smollm2 |
| - math |
| license: apache-2.0 |
| --- |
| # SmolLM2 Math |
| ## SmolLM2 but fine-tuned on math data! |
| I just made a fine-tune of [SmolLM2 135M](https://huggingface.co/HuggingFaceTB/SmolLM2-135M-Instruct) on the [GSM8K](https://huggingface.co/datasets/openai/gsm8k) dataset |
| and it does improve math sometimes. |
| ## Evaluation Results |
| | Metric | Value | |
| | :----- | :--------: | |
| | **Loss** | 1.284519 | |
| | **Steps** | 2805 | |
| ## How to Use |
| This code is by Gemini 3 Flash: |
| ```python |
| from transformers import AutoModelForCausalLM, AutoTokenizer |
| |
| model_name = "MihaiPopa-1/SmolLM2-135M-Math" # Replace with your repo path |
| tokenizer = AutoTokenizer.from_pretrained(model_name) |
| model = AutoModelForCausalLM.from_pretrained(model_name) |
| |
| prompt = "Question: If John has 5 apples and eats 2, then buys 4 more, how many does he have?\nAnswer:" |
| inputs = tokenizer(prompt, return_tensors="pt") |
| outputs = model.generate(**inputs, max_new_tokens=50) |
| |
| print(tokenizer.decode(outputs[0], skip_special_tokens=True)) |
| ``` |
| ## Comparision |
| ### First Question |
| We both give the same question: "I have a number, 96. If the result is 24 and we divide it, what's the second part of the equation?" |
|
|
| [SmolLM2 Math](https://huggingface.co/MihaiPopa-1/SmolLM2-135M-Math) gave: |
| ``` |
| Question: I have a number, 96. If the result is 24 and we divide it, what's the second part of the equation? |
| Answer: The second part of the equation is 96 / 24 = 4. |
| #### 4 |
| The answer is: 4 |
| ``` |
| It correctly gave a answer. |
|
|
| [SmolLM2 Base](https://huggingface.co/HuggingFaceTB/SmolLM2-135M-Instruct) gave: |
| ``` |
| Question: I have a number, 96. If the result is 24 and we divide it, what's the second part of the equation? |
| Answer: |
| ``` |
| It produced NOTHING! |
| ### Second Question |
| Also, Gemini 2.5 Flash thought another question: "If 5 apples cost $2.50, how much does 1 apple cost?" |
|
|
| So we put this question in both models. |
|
|
| [SmolLM2 Math](https://huggingface.co/MihaiPopa-1/SmolLM2-135M-Math) gave: |
| ``` |
| Question: If 5 apples cost $2.50, how much does 1 apple cost? |
| Answer: 1 apple = 100 apples |
| 2 apples = 100 * 2 = 200 apples |
| 1 apple = 100 * 2 = 200 apples |
| #### 200 |
| The answer is: 200 |
| ``` |
| Wrong answer. |
|
|
| Second time: |
| ``` |
| Question: If 5 apples cost $2.50, how much does 1 apple cost? Think carefully. |
| Solution: 1/5 * $2.50 = $0.50 |
| Therefore, 1 apple costs $0.50. |
| #### 0.50 |
| The answer is: 0.50 |
| ``` |
| Correct answer! |
|
|
| [SmolLM2 Base](https://huggingface.co/HuggingFaceTB/SmolLM2-135M-Instruct) gave: |
| ``` |
| Question: If 5 apples cost $2.50, how much does 1 apple cost? |
| Answer: 1 apple = $0.10 |
| |
| Reasoning: The answer is $0.10. |
| ``` |
| Wrong answer too. |
