---
language:
- hu
license: apache-2.0
library_name: mlx
base_model: utter-project/EuroLLM-22B
pipeline_tag: text-generation
tags:
- hungarian
- emese
- eurollm
- instruct
- chatml
- mlx
- apple-silicon
- quantized
---

# Emese-Folyó (22B) — MLX q8

**MLX q8** — the **native training/serving precision** for Folyó (CPT, SFT, and DPO were all trained on
top of a q8-quantized base). See the `folyo/` repo's README for full architecture, CPT/SFT/DPO training
details, and benchmarks — this file covers only the q8-specific notes.

| | |
|---|---|
| **Quantization** | q8, group size 64 |
| **Size on disk** | ~22 GB (vs. ~42 GB bf16) |
| **Quality** | This *is* the model's native precision — the `folyo/` bf16 repo is dequantized *from* this, not the other way around. |
| **Max context length** | 32,768 tokens (EuroLLM-22B's native context) |

## Usage

```python
from mlx_lm import load, generate
from mlx_lm.sample_utils import make_sampler
model, tok = load("emese-tech/folyo-mlx")
p = tok.apply_chat_template([{"role": "user", "content": "Mi Magyarország fővárosa?"}],
                            tokenize=False, add_generation_prompt=True)
print(generate(model, tok, prompt=p, max_tokens=256, sampler=make_sampler(temp=0.2)))
```

**Decode:** temperature `0.2`, no repetition penalty, eos `{2, 4}`, ChatML template.

⚠️ **This repo is `mlx_lm`-only** — MLX's q8 quantization packs weights into `uint32` + per-group
`scales`/`biases` tensors with a `quantization` block in `config.json` that plain `transformers` does not
understand. Use the `folyo/` (bf16) repo for `transformers`/vLLM/TGI.

## Training

This is the primary artifact of the CPT+SFT+DPO training chain — see `folyo/README.md` for the full
q8-native CPT (~6M tokens/6,000 iters, rank64), SFT (`instruct_v18b`, 1 epoch, rank16/scale16/lr5e-6,
iteration 3,600 selected), and DPO (36 alfa pairs, 120 iters, rank16/scale16/lr2e-6) recipe, plus the
211/250 Ultimate · 310/376 BlindSpot benchmark results.

## Benchmarks

**This exact q8 artifact scored 410/500 (82%) on emese-bench v1** (200 questions + 10 chat sessions,
consolidated Ultimate+BlindSpot) — a close second to Patak (413/500) and far ahead of Csermely
(211/500). Strongest on safety, reading, translation, and code; weakest on multi-step math and logic
puzzles. Notably the best of the three models on in-context memory recall and roleplay (both 10/10),
but it failed to stay in English during the English-chat session, answering in Hungarian throughout.
See `emese-bench/results/folyo-mlx.md` for the full transcript and category breakdown.