---
language:
- hu
license: apache-2.0
library_name: transformers
base_model: utter-project/EuroLLM-9B
pipeline_tag: text-generation
tags:
- hungarian
- emese
- eurollm
- instruct
- chatml
---

# Emese-Patak (9.15B)

**Patak** ("stream") is the balanced, desktop-class tier of the [Emese](https://emese.tech) Hungarian
model family — [EuroLLM-9B](https://huggingface.co/utter-project/EuroLLM-9B) continued-pretrained on
Hungarian, instruction-tuned, and DPO-aligned. This is the family's best-benchmarked model.

| | |
|---|---|
| **Parameters** | 9.15B |
| **Base** | EuroLLM-9B |
| **Architecture** | LLaMA-style (RoPE θ=1,000,000, GQA 32Q/8KV, SwiGLU, RMSNorm) |
| **Hidden / layers / heads** | 4096 / 42 / 32 (8 KV heads) |
| **Vocabulary** | 128,000 (EuroLLM multilingual SentencePiece) |
| **Max context length** | **32,768 tokens** (EuroLLM-9B's native `max_position_embeddings`) |
| **Precision** | bfloat16 (this repo); MLX q8 also available (`patak-mlx/`) |
| **License** | Apache-2.0 |

## Formats in this release

| Folder | Format | Size | Notes |
|---|---|---|---|
| `patak/` (this repo) | bf16, standard HF `safetensors` | ~17 GB | loads directly with `transformers` **and** `mlx_lm`; dequantized from the q8-native training chain (near-lossless, see Training) |
| `patak-mlx/` | MLX q8 | ~9.1 GB | `mlx_lm`-only, the native training/serving precision for this model |

## Usage (transformers)

```python
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch
tok = AutoTokenizer.from_pretrained("emese-tech/patak")
model = AutoModelForCausalLM.from_pretrained("emese-tech/patak", dtype=torch.bfloat16, device_map="auto")
msgs = [{"role": "user", "content": "Mi Magyarország fővárosa?"}]
prompt = tok.apply_chat_template(msgs, tokenize=False, add_generation_prompt=True)
ids = tok(prompt, return_tensors="pt").to(model.device)
out = model.generate(**ids, max_new_tokens=256, do_sample=True, temperature=0.2, eos_token_id=[2, 4])
print(tok.decode(out[0][ids.input_ids.shape[1]:], skip_special_tokens=True))
```

## Usage (MLX)

```python
from mlx_lm import load, generate
from mlx_lm.sample_utils import make_sampler
model, tok = load("emese-tech/patak-mlx")  # q8
p = tok.apply_chat_template([{"role": "user", "content": "Mi Magyarország fővárosa?"}],
                            tokenize=False, add_generation_prompt=True)
print(generate(model, tok, prompt=p, max_tokens=256, sampler=make_sampler(temp=0.2)))
```

**Decode:** temperature `0.2`, no repetition penalty, **eos `{2, 4}`** (`</s>` and `<|im_end|>` — omitting
id 4 causes non-stopping generation), ChatML template. For multi-turn conversations, **always pass the
full history**, not just the latest message.

## Training

- **CPT** — light continued pretraining of EuroLLM-9B on the Emese Hungarian corpus: **5.1M tokens over
  5,000 iterations** (val loss 1.857). This is a known-light budget relative to the ~3.7B-token corpus
  available on disk (`corpus/cpt/README.md`) — a deeper CPT pass is a documented candidate for a future
  release (see `instruct/V16_PATAK_CPT.md`), but was not required to reach this release's benchmark result.
- **SFT** — 1 epoch (4,914 iterations) on the `instruct_v18b` corpus (4,914 rows: persona, safety, code +
  code-debug, hedging/anti-confabulation, multi-step reasoning, compound constraints, multi-turn
  refinement, anti-repetition). LoRA rank 16 / scale 32, dropout 0.1, lr 1.5e-5, gradient-checkpointed, all
  42 layers + `lm_head` trained.
- **DPO** — 120 iterations of DPO-lite ("alfa") on 36 hand-written preference pairs (persona
  identity-defense + anti-repetition), fixing residual persona/anti-repetition/honesty-calibration issues
  the SFT pass alone left behind. LoRA rank 16 / scale 32, lr 5e-6, trained on top of the quantized (q8)
  SFT model — this repo's bf16 weights are dequantized from that q8-native chain, which this project's own
  testing found near-lossless (MLX bf16 and q8 score within 1 point of each other on full benchmark runs).

## Benchmarks

| | Ultimate Bench (0-250) | BlindSpot Bench (0-376) |
|---|---|---|
| **This release (DPO alfa)** | **218/250 (87%)** — family all-time record | **302/376** |

Safety: 100% refusal rate. Weak points: multi-turn *refinement* (revising an answer under a new
constraint), occasional factual confabulation on obscure names/dates.

**emese-bench v1 (500 pts, consolidated Ultimate+BlindSpot, MLX q8): 413/500 (83%)** — by far the
family's strongest result on the new unified benchmark. Near-perfect on longform, reading, code,
safety, honesty, and English; the only real weak spots are multi-step math, spatial estimation, and
code-debugging. See `emese-bench/results/patak-mlx.md` for the full category breakdown and
`emese-bench/README.md` for the benchmark's design.

## Limitations

- Can hallucinate specific facts (dates, attributions) — verify critical details.
- Multi-turn refinement is a known weak spot relative to single-turn quality.
- Hungarian-first; other-language quality inherited from EuroLLM-9B.
- **Do not convert this model to GGUF without re-validating first** — this project's own testing found a
  consistent ~14-27 point Ultimate Bench regression when serving via llama.cpp/GGUF, regardless of source
  precision or quantization level, root-caused to llama.cpp's inference path itself (not this repo's
  weights). Use the `transformers`/`mlx_lm` paths above.