| --- |
| language: |
| - hu |
| license: apache-2.0 |
| library_name: transformers |
| base_model: utter-project/EuroLLM-9B |
| pipeline_tag: text-generation |
| tags: |
| - hungarian |
| - emese |
| - eurollm |
| - instruct |
| - chatml |
| --- |
| |
| # Emese-Patak (9.15B) |
|
|
| **Patak** ("stream") is the balanced, desktop-class tier of the [Emese](https://emese.tech) Hungarian |
| model family — [EuroLLM-9B](https://huggingface.co/utter-project/EuroLLM-9B) continued-pretrained on |
| Hungarian, instruction-tuned, and DPO-aligned. This is the family's best-benchmarked model. |
|
|
| | | | |
| |---|---| |
| | **Parameters** | 9.15B | |
| | **Base** | EuroLLM-9B | |
| | **Architecture** | LLaMA-style (RoPE θ=1,000,000, GQA 32Q/8KV, SwiGLU, RMSNorm) | |
| | **Hidden / layers / heads** | 4096 / 42 / 32 (8 KV heads) | |
| | **Vocabulary** | 128,000 (EuroLLM multilingual SentencePiece) | |
| | **Max context length** | **32,768 tokens** (EuroLLM-9B's native `max_position_embeddings`) | |
| | **Precision** | bfloat16 (this repo); MLX q8 also available (`patak-mlx/`) | |
| | **License** | Apache-2.0 | |
|
|
| ## Formats in this release |
|
|
| | Folder | Format | Size | Notes | |
| |---|---|---|---| |
| | `patak/` (this repo) | bf16, standard HF `safetensors` | ~17 GB | loads directly with `transformers` **and** `mlx_lm`; dequantized from the q8-native training chain (near-lossless, see Training) | |
| | `patak-mlx/` | MLX q8 | ~9.1 GB | `mlx_lm`-only, the native training/serving precision for this model | |
|
|
| ## Usage (transformers) |
|
|
| ```python |
| from transformers import AutoModelForCausalLM, AutoTokenizer |
| import torch |
| tok = AutoTokenizer.from_pretrained("emese-tech/patak") |
| model = AutoModelForCausalLM.from_pretrained("emese-tech/patak", dtype=torch.bfloat16, device_map="auto") |
| msgs = [{"role": "user", "content": "Mi Magyarország fővárosa?"}] |
| prompt = tok.apply_chat_template(msgs, tokenize=False, add_generation_prompt=True) |
| ids = tok(prompt, return_tensors="pt").to(model.device) |
| out = model.generate(**ids, max_new_tokens=256, do_sample=True, temperature=0.2, eos_token_id=[2, 4]) |
| print(tok.decode(out[0][ids.input_ids.shape[1]:], skip_special_tokens=True)) |
| ``` |
|
|
| ## Usage (MLX) |
|
|
| ```python |
| from mlx_lm import load, generate |
| from mlx_lm.sample_utils import make_sampler |
| model, tok = load("emese-tech/patak-mlx") # q8 |
| p = tok.apply_chat_template([{"role": "user", "content": "Mi Magyarország fővárosa?"}], |
| tokenize=False, add_generation_prompt=True) |
| print(generate(model, tok, prompt=p, max_tokens=256, sampler=make_sampler(temp=0.2))) |
| ``` |
|
|
| **Decode:** temperature `0.2`, no repetition penalty, **eos `{2, 4}`** (`</s>` and `<|im_end|>` — omitting |
| id 4 causes non-stopping generation), ChatML template. For multi-turn conversations, **always pass the |
| full history**, not just the latest message. |
|
|
| ## Training |
|
|
| - **CPT** — light continued pretraining of EuroLLM-9B on the Emese Hungarian corpus: **5.1M tokens over |
| 5,000 iterations** (val loss 1.857). This is a known-light budget relative to the ~3.7B-token corpus |
| available on disk (`corpus/cpt/README.md`) — a deeper CPT pass is a documented candidate for a future |
| release (see `instruct/V16_PATAK_CPT.md`), but was not required to reach this release's benchmark result. |
| - **SFT** — 1 epoch (4,914 iterations) on the `instruct_v18b` corpus (4,914 rows: persona, safety, code + |
| code-debug, hedging/anti-confabulation, multi-step reasoning, compound constraints, multi-turn |
| refinement, anti-repetition). LoRA rank 16 / scale 32, dropout 0.1, lr 1.5e-5, gradient-checkpointed, all |
| 42 layers + `lm_head` trained. |
| - **DPO** — 120 iterations of DPO-lite ("alfa") on 36 hand-written preference pairs (persona |
| identity-defense + anti-repetition), fixing residual persona/anti-repetition/honesty-calibration issues |
| the SFT pass alone left behind. LoRA rank 16 / scale 32, lr 5e-6, trained on top of the quantized (q8) |
| SFT model — this repo's bf16 weights are dequantized from that q8-native chain, which this project's own |
| testing found near-lossless (MLX bf16 and q8 score within 1 point of each other on full benchmark runs). |
|
|
| ## Benchmarks |
|
|
| | | Ultimate Bench (0-250) | BlindSpot Bench (0-376) | |
| |---|---|---| |
| | **This release (DPO alfa)** | **218/250 (87%)** — family all-time record | **302/376** | |
|
|
| Safety: 100% refusal rate. Weak points: multi-turn *refinement* (revising an answer under a new |
| constraint), occasional factual confabulation on obscure names/dates. |
|
|
| **emese-bench v1 (500 pts, consolidated Ultimate+BlindSpot, MLX q8): 413/500 (83%)** — by far the |
| family's strongest result on the new unified benchmark. Near-perfect on longform, reading, code, |
| safety, honesty, and English; the only real weak spots are multi-step math, spatial estimation, and |
| code-debugging. See `emese-bench/results/patak-mlx.md` for the full category breakdown and |
| `emese-bench/README.md` for the benchmark's design. |
|
|
| ## Limitations |
|
|
| - Can hallucinate specific facts (dates, attributions) — verify critical details. |
| - Multi-turn refinement is a known weak spot relative to single-turn quality. |
| - Hungarian-first; other-language quality inherited from EuroLLM-9B. |
| - **Do not convert this model to GGUF without re-validating first** — this project's own testing found a |
| consistent ~14-27 point Ultimate Bench regression when serving via llama.cpp/GGUF, regardless of source |
| precision or quantization level, root-caused to llama.cpp's inference path itself (not this repo's |
| weights). Use the `transformers`/`mlx_lm` paths above. |
|
|
