Instructions to use ifx-pse-sys-ml/flame-27m-instruct with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use ifx-pse-sys-ml/flame-27m-instruct with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="ifx-pse-sys-ml/flame-27m-instruct", trust_remote_code=True)
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained("ifx-pse-sys-ml/flame-27m-instruct", trust_remote_code=True, device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use ifx-pse-sys-ml/flame-27m-instruct with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "ifx-pse-sys-ml/flame-27m-instruct"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ifx-pse-sys-ml/flame-27m-instruct",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/ifx-pse-sys-ml/flame-27m-instruct

SGLang

How to use ifx-pse-sys-ml/flame-27m-instruct with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "ifx-pse-sys-ml/flame-27m-instruct" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ifx-pse-sys-ml/flame-27m-instruct",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "ifx-pse-sys-ml/flame-27m-instruct" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ifx-pse-sys-ml/flame-27m-instruct",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use ifx-pse-sys-ml/flame-27m-instruct with Docker Model Runner:
```
docker model run hf.co/ifx-pse-sys-ml/flame-27m-instruct
```

flame-27m-instruct / README.md

lyonsn4

Fix generate(): implement no_repeat_ngram_size, add penalize_prompt; correct the generation docs

33250fc verified about 2 months ago

preview code

Raw History Blame Contribute Delete

5.82 kB

---
license: apache-2.0
language:
- en
library_name: transformers
pipeline_tag: text-generation
tags:
- small-language-model
- instruct
- smoltalk
---
# flame-27m-instruct

A **27.1M-parameter** English instruction-tuned model: [flame-27m-base](https://huggingface.co/ifx-pse-sys-ml/flame-27m-base)
(pretrained on FineWeb-Edu-dedup + Cosmopedia-v2 + ClimbMix + FineMath) fine-tuned on a ~1.18M
conversation instruction mixture. ~5× smaller than SmolLM-135M-Instruct.

- **Architecture:** Llama-style decoder — hidden 512, 8 layers, 8 heads / 2 KV heads (GQA),
  intermediate 1280, RoPE (θ=1e6), context 2048, vocab 12000 (English BPE).
- **Training:** base → SFT (5 epochs, EMA weights) on
  [SmolTalk](https://huggingface.co/datasets/HuggingFaceTB/smoltalk) +
  [Tulu-3-Persona-IF](https://huggingface.co/datasets/allenai/tulu-3-sft-personas-instruction-following) +
  [No-Robots](https://huggingface.co/datasets/HuggingFaceH4/no_robots) +
  [WildChat](https://huggingface.co/datasets/allenai/WildChat-1M) (English, non-toxic).

## Usage (chat)

```python
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
model = AutoModelForCausalLM.from_pretrained("ifx-pse-sys-ml/flame-27m-instruct", trust_remote_code=True)
tok = AutoTokenizer.from_pretrained("ifx-pse-sys-ml/flame-27m-instruct")
messages = [{"role": "user", "content": "Tell me about the moon in one sentence."}]
ids = tok.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt")
out = model.generate(ids, max_new_tokens=64, do_sample=True, temperature=0.7, top_p=0.9)
print(tok.decode(out[0][ids.shape[1]:], skip_special_tokens=True))
```

The decoder also accepts `inputs_embeds` (exactly one of `input_ids` / `inputs_embeds`), so a
vision projector can splice visual tokens in — usable as a small VLM text backbone. A raw
PyTorch checkpoint (`pytorch_model.pth`) is included alongside the safetensors weights.

## Generation parameters

The model ships its own `generate()` (loaded via `trust_remote_code`). Two things to know:

- **It does not read `generation_config.json`.** A bare `model.generate(ids)` uses the built-in
  defaults — `temperature 0.85, top_p 0.85, top_k 50, repetition_penalty 1.0` — so pass the
  settings you want explicitly.
- **Greedy is `do_sample=False`**, not `temperature=0` (which divides by zero).
Measured presets, from a 28-prompt VLM-style sweep over 13 decoding configs:
| Use | Settings | Measured |
|---|---|---|
| **Answering about a given scene / image (VQA-style)** | `do_sample=False, repetition_penalty=1.3, no_repeat_ngram_size=3, penalize_prompt=False` | Best overall: 59% grounded accuracy, no looping, shortest answers |
| Captions / open-ended description | `do_sample=True, temperature=0.7, top_p=0.9, repetition_penalty=1.3, no_repeat_ngram_size=3, penalize_prompt=False` | Varied and loop-free |

| Parameter | Default | What it does / when to change |
|---|---|---|
| `repetition_penalty` | `1.0` | Penalises tokens already seen. `1.3` stops this model's multi-sentence looping. |
| `no_repeat_ngram_size` | `0` | Blocks any n-gram from repeating (`3` works well). Never blocks the end-of-turn token. |
| `penalize_prompt` | `True` | **Set `False` for anything grounded in the prompt.** By default (Hugging Face semantics) both penalties also cover the prompt, which pushes the model *away* from copying the answer out of the context ("a red car", "Answer yes or no"). Limiting them to the reply raised grounded accuracy from 47% → 59% (greedy) and 27% → 39% (sampled). |
| `temperature` | `0.85` | Lower = more focused. Must be > 0. |
| `top_p` / `top_k` | `0.85` / `50` | Nucleus / top-k cutoffs. |
| `max_new_tokens` | `8192` (capped at the 2048-token context) | Always set it — the model is at its best in short replies. |

```python
# VQA-style: answer from the context, deterministically
out = model.generate(ids, max_new_tokens=64, do_sample=False,
                     repetition_penalty=1.3, no_repeat_ngram_size=3, penalize_prompt=False)
# Captions / open-ended
out = model.generate(ids, max_new_tokens=120, do_sample=True, temperature=0.7, top_p=0.9,
                     repetition_penalty=1.3, no_repeat_ngram_size=3, penalize_prompt=False)
```

## Benchmarks

Accuracy (%) via [lm-evaluation-harness](https://github.com/EleutherAI/lm-evaluation-harness)
0.4, **same harness and shots for every model**, so columns are directly comparable.

| Benchmark | chance | **flame-27m-instruct** | SmolLM-135M-Instruct |
|---|---|---|---|
| hellaswag | 25 | 29.2 | 41.9 |
| arc_easy | 25 | 37.3 | 43.9 |
| arc_challenge | 25 | 22.6 | 27.4 |
| piqa | 50 | 59.3 | 67.0 |
| winogrande | 50 | 51.7 | 51.3 |
| openbookqa | 25 | 27.4 | 33.6 |
| commonsense_qa | 20 | 19.7 | 20.3 |
| mmlu | 25 | 24.9 | 24.4 |
| **average** | — | **34.0** | **38.7** |
These academic benchmarks measure *base knowledge*, which SFT cannot add. The richer instruction
mixture was chosen to improve instruction-following and response quality, which it does (held-out
assistant-token loss 1.34 → 1.23 vs a SmolTalk-only SFT, and cleaner format adherence) — at a
small cost on the knowledge probes above. Benchmarks are the wrong lens for an instruct model's
quality; they are shown only for comparability with the base and SmolLM.
## Honest limitations
At **27M parameters** this is near random chance on knowledge/reasoning benchmarks; the gap to
SmolLM-135M is capacity, not data or tuning. Instruction-tuning adds response *format*, not
facts. It follows simple instructions but cannot reliably satisfy hard multi-constraint prompts
(IFEval ≈ 0) — that capability is bound by the 27M base, not the SFT data. A research/prototyping
instruct model and a lightweight decoder, not a knowledge model. English only. Trained with the
Nexus codebase.

	---
	license: apache-2.0
	language:
	- en
	library_name: transformers
	pipeline_tag: text-generation
	tags:
	- small-language-model
	- instruct
	- smoltalk
	---

	# flame-27m-instruct

	A 27.1M-parameter English instruction-tuned model: [flame-27m-base](https://huggingface.co/ifx-pse-sys-ml/flame-27m-base)
	(pretrained on FineWeb-Edu-dedup + Cosmopedia-v2 + ClimbMix + FineMath) fine-tuned on a ~1.18M
	conversation instruction mixture. ~5× smaller than SmolLM-135M-Instruct.

	- Architecture: Llama-style decoder — hidden 512, 8 layers, 8 heads / 2 KV heads (GQA),
	intermediate 1280, RoPE (θ=1e6), context 2048, vocab 12000 (English BPE).
	- Training: base → SFT (5 epochs, EMA weights) on
	[SmolTalk](https://huggingface.co/datasets/HuggingFaceTB/smoltalk) +
	[Tulu-3-Persona-IF](https://huggingface.co/datasets/allenai/tulu-3-sft-personas-instruction-following) +
	[No-Robots](https://huggingface.co/datasets/HuggingFaceH4/no_robots) +
	[WildChat](https://huggingface.co/datasets/allenai/WildChat-1M) (English, non-toxic).

	## Usage (chat)

	```python
	import torch
	from transformers import AutoModelForCausalLM, AutoTokenizer

	model = AutoModelForCausalLM.from_pretrained("ifx-pse-sys-ml/flame-27m-instruct", trust_remote_code=True)
	tok = AutoTokenizer.from_pretrained("ifx-pse-sys-ml/flame-27m-instruct")

	messages = [{"role": "user", "content": "Tell me about the moon in one sentence."}]
	ids = tok.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt")
	out = model.generate(ids, max_new_tokens=64, do_sample=True, temperature=0.7, top_p=0.9)
	print(tok.decode(out[0][ids.shape[1]:], skip_special_tokens=True))
	```

	The decoder also accepts `inputs_embeds` (exactly one of `input_ids` / `inputs_embeds`), so a
	vision projector can splice visual tokens in — usable as a small VLM text backbone. A raw
	PyTorch checkpoint (`pytorch_model.pth`) is included alongside the safetensors weights.

	## Generation parameters

	The model ships its own `generate()` (loaded via `trust_remote_code`). Two things to know:

	- It does not read `generation_config.json`. A bare `model.generate(ids)` uses the built-in
	defaults — `temperature 0.85, top_p 0.85, top_k 50, repetition_penalty 1.0` — so pass the
	settings you want explicitly.
	- Greedy is `do_sample=False`, not `temperature=0` (which divides by zero).

	Measured presets, from a 28-prompt VLM-style sweep over 13 decoding configs:

	\| Use \| Settings \| Measured \|
	\|---\|---\|---\|
	\| Answering about a given scene / image (VQA-style) \| `do_sample=False, repetition_penalty=1.3, no_repeat_ngram_size=3, penalize_prompt=False` \| Best overall: 59% grounded accuracy, no looping, shortest answers \|
	\| Captions / open-ended description \| `do_sample=True, temperature=0.7, top_p=0.9, repetition_penalty=1.3, no_repeat_ngram_size=3, penalize_prompt=False` \| Varied and loop-free \|

	\| Parameter \| Default \| What it does / when to change \|
	\|---\|---\|---\|
	\| `repetition_penalty` \| `1.0` \| Penalises tokens already seen. `1.3` stops this model's multi-sentence looping. \|
	\| `no_repeat_ngram_size` \| `0` \| Blocks any n-gram from repeating (`3` works well). Never blocks the end-of-turn token. \|
	\| `penalize_prompt` \| `True` \| Set `False` for anything grounded in the prompt. By default (Hugging Face semantics) both penalties also cover the prompt, which pushes the model away from copying the answer out of the context ("a red car", "Answer yes or no"). Limiting them to the reply raised grounded accuracy from 47% → 59% (greedy) and 27% → 39% (sampled). \|
	\| `temperature` \| `0.85` \| Lower = more focused. Must be > 0. \|
	\| `top_p` / `top_k` \| `0.85` / `50` \| Nucleus / top-k cutoffs. \|
	\| `max_new_tokens` \| `8192` (capped at the 2048-token context) \| Always set it — the model is at its best in short replies. \|

	```python
	# VQA-style: answer from the context, deterministically
	out = model.generate(ids, max_new_tokens=64, do_sample=False,
	repetition_penalty=1.3, no_repeat_ngram_size=3, penalize_prompt=False)

	# Captions / open-ended
	out = model.generate(ids, max_new_tokens=120, do_sample=True, temperature=0.7, top_p=0.9,
	repetition_penalty=1.3, no_repeat_ngram_size=3, penalize_prompt=False)
	```

	## Benchmarks

	Accuracy (%) via [lm-evaluation-harness](https://github.com/EleutherAI/lm-evaluation-harness)
	0.4, same harness and shots for every model, so columns are directly comparable.

	\| Benchmark \| chance \| flame-27m-instruct \| SmolLM-135M-Instruct \|
	\|---\|---\|---\|---\|
	\| hellaswag \| 25 \| 29.2 \| 41.9 \|
	\| arc_easy \| 25 \| 37.3 \| 43.9 \|
	\| arc_challenge \| 25 \| 22.6 \| 27.4 \|
	\| piqa \| 50 \| 59.3 \| 67.0 \|
	\| winogrande \| 50 \| 51.7 \| 51.3 \|
	\| openbookqa \| 25 \| 27.4 \| 33.6 \|
	\| commonsense_qa \| 20 \| 19.7 \| 20.3 \|
	\| mmlu \| 25 \| 24.9 \| 24.4 \|
	\| average \| — \| 34.0 \| 38.7 \|

	These academic benchmarks measure base knowledge, which SFT cannot add. The richer instruction
	mixture was chosen to improve instruction-following and response quality, which it does (held-out
	assistant-token loss 1.34 → 1.23 vs a SmolTalk-only SFT, and cleaner format adherence) — at a
	small cost on the knowledge probes above. Benchmarks are the wrong lens for an instruct model's
	quality; they are shown only for comparability with the base and SmolLM.

	## Honest limitations

	At 27M parameters this is near random chance on knowledge/reasoning benchmarks; the gap to
	SmolLM-135M is capacity, not data or tuning. Instruction-tuning adds response format, not
	facts. It follows simple instructions but cannot reliably satisfy hard multi-constraint prompts
	(IFEval ≈ 0) — that capability is bound by the 27M base, not the SFT data. A research/prototyping
	instruct model and a lightweight decoder, not a knowledge model. English only. Trained with the
	Nexus codebase.