Upload folder using huggingface_hub

README.md

@@ -0,0 +1,88 @@
+---
+language:
+- hu
+license: mit
+library_name: mlx
+tags:
+- hungarian
+- emese
+- from-scratch
+- llama
+- base-model
+- mlx
+- apple-silicon
+pipeline_tag: text-generation
+---
+
+# Emese-Ér (517M)
+
+**Ér** ("brook/rill" — the smallest flowing water in the family name scheme) is the edge/embedded/research
+tier of the [Emese](https://emese.tech) Hungarian model family. Unlike the other three tiers, Ér is
+**not** built on EuroLLM — it's a from-scratch Hungarian foundation model, pretrained on ~4.5B tokens of
+Hungarian text with a custom 32k SentencePiece tokenizer.
+
+| | |
+|---|---|
+| **Parameters** | 516.8M |
+| **Base** | none — from-scratch pretraining |
+| **Architecture** | LLaMA-style decoder (RoPE + YaRN, RMSNorm, SwiGLU, tied embeddings) |
+| **Hidden / layers / heads** | 1280 / 24 / 20 |
+| **FFN size** | 3456 |
+| **Vocabulary** | 32,000 (custom Hungarian SentencePiece Unigram) |
+| **Max context length** | **32,768 tokens** (trained at 1,024, YaRN-extended to 8,192 during CPT, RoPE scale 4.0 rated to 32,768) |
+| **Precision** | bfloat16 |
+| **License** | MIT |
+
+## Formats in this release
+
+| Folder | Format | Notes |
+|---|---|---|
+| `er/` (this repo) | bf16, MLX-native safetensors | **not** a standard `transformers` repo — see below |
+| `er-mlx/` | MLX q8 | `mlx_lm`-loadable, quantized |
+
+## ⚠️ Not `transformers`-compatible out of the box
+
+Ér predates the EuroLLM/Llama-arch pipeline used for Csermely/Patak/Folyó. Its `config.json` uses a
+custom schema (`d_model`, `n_layers`, `n_heads`, `d_ff` — not HF's `hidden_size`/`num_hidden_layers`/...)
+and its weight tensor names are custom (`embed.weight`, `layers.N.attn.wq.weight`, ...), not the standard
+`model.embed_tokens.weight` / `model.layers.N.self_attn.q_proj.weight` naming. **`AutoModelForCausalLM.
+from_pretrained` will fail on this repo.** Use the project's own `model.py` (an MLX `Emese` class) or load
+`er-mlx/` via `mlx_lm`, which does understand this checkpoint despite the non-standard config.
+
+## Usage (MLX, via mlx_lm)
+
+```python
+from mlx_lm import load, generate
+model, tok = load("er-mlx")  # or "er" — both load via mlx_lm
+print(generate(model, tok, prompt="A magyar nyelv", max_tokens=100))
+```
+
+## Usage (project's own loader)
+
+```bash
+python generate.py --model models-release/er/ --prompt "A magyar nyelv"
+```
+
+## Training
+
+- **Pretraining (from scratch):** ~4.5B tokens of Hungarian text (the same corpus family documented in
+  `corpus/cpt/README.md`: Wikipedia + HPLT web text, quality-filtered). No separate "CPT" stage — this
+  *is* the base pretraining, there is no upstream foundation model underneath it.
+
+## Benchmarks
+
+**Not applicable** — `emese-bench` (and its Ultimate/BlindSpot predecessors) are chat-instruction
+benchmarks (ChatML turns, persona/safety/instruction-following categories); Ér has no chat template or
+instruction-tuning, so scoring it on those benchmarks would not be a meaningful comparison against the
+other three tiers. During development it was evaluated on HuCoLA (Hungarian grammatical-acceptability
+classification) instead, not on chat-style benchmarks.
+- **No SFT / no DPO.** Ér ships as a **base/research model only** — it has no chat template, no persona
+  training, no instruction-tuning. It is not conversational; treat it as a research/embedding artifact for
+  downstream fine-tuning experiments, not an assistant.
+
+## Limitations
+
+- Base model only — expects raw text continuation, not chat-formatted prompts.
+- Smallest tier by a wide margin (517M vs. 1.7B+ for the rest of the family) — limited world knowledge and
+  reasoning capacity even relative to Csermely.
+- Requires custom loading code (`model.py`) for anything beyond `mlx_lm`.

config.json

@@ -0,0 +1,19 @@
+{
+  "model_type": "llama",
+  "vocab_size": 32000,
+  "d_model": 1280,
+  "n_layers": 24,
+  "n_heads": 20,
+  "d_ff": 3456,
+  "max_seq_len": 8192,
+  "rope_theta": 10000.0,
+  "rope_yarn": true,
+  "rope_yarn_scale": 4.0,
+  "rms_norm_eps": 1e-6,
+  "tie_word_embeddings": true,
+  "torch_dtype": "bfloat16",
+  "repetition_penalty": 1.2,
+  "bos_token_id": 2,
+  "eos_token_id": 3,
+  "pad_token_id": 1
+}

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fbfd0c6df6e03db5f30f3dcff2332c7153ef4234acfeff75c8c05e7d1deb2076
+size 1033649911

special_tokens_map.json

@@ -0,0 +1,30 @@
+{
+  "bos_token": {
+    "content": "<bos>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "<eos>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "unk_token": {
+    "content": "<unk>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "<pad>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

tokenizer.model

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:244bb6facd57b3890990261b9932ab50d79630d5b35058c902a5c96c32fa2950
+size 845117

tokenizer_config.json

@@ -0,0 +1,12 @@
+{
+  "bos_token": "<bos>",
+  "eos_token": "<eos>",
+  "unk_token": "<unk>",
+  "pad_token": "<pad>",
+  "sp_model_kwargs": {},
+  "add_bos_token": true,
+  "add_eos_token": false,
+  "model_max_length": 8192,
+  "tokenizer_class": "LlamaTokenizer",
+  "clean_up_tokenization_spaces": false
+}