---
license: apache-2.0
language:
- en
library_name: transformers
pipeline_tag: text-generation
tags:
- small-language-model
- instruct
- smoltalk
---

# flame-27m-instruct

A **27.1M-parameter** English instruction-tuned model: [flame-27m-base](https://huggingface.co/ifx-pse-sys-ml/flame-27m-base)
(pretrained on FineWeb-Edu-dedup + Cosmopedia-v2 + ClimbMix + FineMath) fine-tuned on a ~1.18M
conversation instruction mixture. ~5× smaller than SmolLM-135M-Instruct.

- **Architecture:** Llama-style decoder — hidden 512, 8 layers, 8 heads / 2 KV heads (GQA),
  intermediate 1280, RoPE (θ=1e6), context 2048, vocab 12000 (English BPE).
- **Training:** base → SFT (5 epochs, EMA weights) on
  [SmolTalk](https://huggingface.co/datasets/HuggingFaceTB/smoltalk) +
  [Tulu-3-Persona-IF](https://huggingface.co/datasets/allenai/tulu-3-sft-personas-instruction-following) +
  [No-Robots](https://huggingface.co/datasets/HuggingFaceH4/no_robots) +
  [WildChat](https://huggingface.co/datasets/allenai/WildChat-1M) (English, non-toxic).

## Usage (chat)

```python
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("ifx-pse-sys-ml/flame-27m-instruct", trust_remote_code=True)
tok = AutoTokenizer.from_pretrained("ifx-pse-sys-ml/flame-27m-instruct")

messages = [{"role": "user", "content": "Tell me about the moon in one sentence."}]
ids = tok.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt")
out = model.generate(ids, max_new_tokens=64, do_sample=True, temperature=0.7, top_p=0.9)
print(tok.decode(out[0][ids.shape[1]:], skip_special_tokens=True))
```

The decoder also accepts `inputs_embeds` (exactly one of `input_ids` / `inputs_embeds`), so a
vision projector can splice visual tokens in — usable as a small VLM text backbone. A raw
PyTorch checkpoint (`pytorch_model.pth`) is included alongside the safetensors weights.

## Generation parameters

The model ships its own `generate()` (loaded via `trust_remote_code`). Two things to know:

- **It does not read `generation_config.json`.** A bare `model.generate(ids)` uses the built-in
  defaults — `temperature 0.85, top_p 0.85, top_k 50, repetition_penalty 1.0` — so pass the
  settings you want explicitly.
- **Greedy is `do_sample=False`**, not `temperature=0` (which divides by zero).

Measured presets, from a 28-prompt VLM-style sweep over 13 decoding configs:

| Use | Settings | Measured |
|---|---|---|
| **Answering about a given scene / image (VQA-style)** | `do_sample=False, repetition_penalty=1.3, no_repeat_ngram_size=3, penalize_prompt=False` | Best overall: 59% grounded accuracy, no looping, shortest answers |
| Captions / open-ended description | `do_sample=True, temperature=0.7, top_p=0.9, repetition_penalty=1.3, no_repeat_ngram_size=3, penalize_prompt=False` | Varied and loop-free |

| Parameter | Default | What it does / when to change |
|---|---|---|
| `repetition_penalty` | `1.0` | Penalises tokens already seen. `1.3` stops this model's multi-sentence looping. |
| `no_repeat_ngram_size` | `0` | Blocks any n-gram from repeating (`3` works well). Never blocks the end-of-turn token. |
| `penalize_prompt` | `True` | **Set `False` for anything grounded in the prompt.** By default (Hugging Face semantics) both penalties also cover the prompt, which pushes the model *away* from copying the answer out of the context ("a red car", "Answer yes or no"). Limiting them to the reply raised grounded accuracy from 47% → 59% (greedy) and 27% → 39% (sampled). |
| `temperature` | `0.85` | Lower = more focused. Must be > 0. |
| `top_p` / `top_k` | `0.85` / `50` | Nucleus / top-k cutoffs. |
| `max_new_tokens` | `8192` (capped at the 2048-token context) | Always set it — the model is at its best in short replies. |

```python
# VQA-style: answer from the context, deterministically
out = model.generate(ids, max_new_tokens=64, do_sample=False,
                     repetition_penalty=1.3, no_repeat_ngram_size=3, penalize_prompt=False)

# Captions / open-ended
out = model.generate(ids, max_new_tokens=120, do_sample=True, temperature=0.7, top_p=0.9,
                     repetition_penalty=1.3, no_repeat_ngram_size=3, penalize_prompt=False)
```

## Benchmarks

Accuracy (%) via [lm-evaluation-harness](https://github.com/EleutherAI/lm-evaluation-harness)
0.4, **same harness and shots for every model**, so columns are directly comparable.

| Benchmark | chance | **flame-27m-instruct** | SmolLM-135M-Instruct |
|---|---|---|---|
| hellaswag | 25 | 29.2 | 41.9 |
| arc_easy | 25 | 37.3 | 43.9 |
| arc_challenge | 25 | 22.6 | 27.4 |
| piqa | 50 | 59.3 | 67.0 |
| winogrande | 50 | 51.7 | 51.3 |
| openbookqa | 25 | 27.4 | 33.6 |
| commonsense_qa | 20 | 19.7 | 20.3 |
| mmlu | 25 | 24.9 | 24.4 |
| **average** | — | **34.0** | **38.7** |

These academic benchmarks measure *base knowledge*, which SFT cannot add. The richer instruction
mixture was chosen to improve instruction-following and response quality, which it does (held-out
assistant-token loss 1.34 → 1.23 vs a SmolTalk-only SFT, and cleaner format adherence) — at a
small cost on the knowledge probes above. Benchmarks are the wrong lens for an instruct model's
quality; they are shown only for comparability with the base and SmolLM.

## Honest limitations

At **27M parameters** this is near random chance on knowledge/reasoning benchmarks; the gap to
SmolLM-135M is capacity, not data or tuning. Instruction-tuning adds response *format*, not
facts. It follows simple instructions but cannot reliably satisfy hard multi-constraint prompts
(IFEval ≈ 0) — that capability is bound by the 27M base, not the SFT data. A research/prototyping
instruct model and a lightweight decoder, not a knowledge model. English only. Trained with the
Nexus codebase.