Upload Morpheus v2 Mamba-2 (91M, step 74K, best checkpoint)

README.md

@@ -0,0 +1,66 @@
+---
+library_name: transformers
+license: apache-2.0
+language:
+  - eu
+  - en
+  - es
+tags:
+  - mamba-2
+  - basque
+  - autocomplete
+  - on-device
+  - low-resource
+pipeline_tag: text-generation
+---
+
+# Morpheus v2 (Mamba-2) — Basque Autocomplete
+
+A 91M-parameter Mamba-2 language model for on-device Basque (Euskara) text autocompletion.
+
+## Model Details
+
+- **Architecture:** Mamba-2 (State Space Model)
+- **Parameters:** 91M
+- **Embedding vocab:** 4,000 (Unigram SentencePiece)
+- **Hidden dimension:** 768
+- **Layers:** 24
+- **State dimension:** 64
+- **Head dimension:** 64
+- **Inner dimension:** 1,536
+- **Sequence length:** 1,024
+- **Training tokens:** ~10 billion
+- **Training steps:** 76,000 (best checkpoint at 74,000)
+- **Held-out PPL:** 7.13
+- **Trained without BOS token**
+
+## Tokenizer
+
+A 4K Unigram SentencePiece tokenizer trained on the cleaned Basque corpus. The small vocabulary size was chosen based on evidence that lower vocab sizes achieve lower downstream perplexity for agglutinative low-resource languages (cf. QuechuaTok).
+
+- `add_bos_token: false` (the model was trained without a BOS token)
+- EOS token: `</s>` (id=2)
+- UNK token: `<unk>` (id=0)
+
+## Intended Use
+
+On-device Basque text autocomplete and predictive keyboard input. The model is small enough to run on CPU via llama.cpp (see the GGUF quantized versions at [itzune/morpheus-gguf](https://huggingface.co/itzune/morpheus-gguf)).
+
+## Training Data
+
+Trained on a ~22 GB cleaned Basque text corpus comprising Wikipedia, news (Berria), literature, and other web-crawled sources. The corpus underwent a multi-stage cleaning pipeline (deduplication, language filtering, quality auditing).
+
+## Quantized Versions
+
+GGUF quantized models (Q4_K_M, Q5_K_M) for llama.cpp inference are available at:
+[itzune/morpheus-gguf](https://huggingface.co/itzune/morpheus-gguf)
+
+## Citation
+
+```bibtex
+@misc{morpheus_v2_mamba,
+  author = {Xabier Ezpeleta},
+  title = {Morpheus v2: On-Device Basque Autocompletion with Mamba-2},
+  year = {2026},
+}
+```

config.json

@@ -0,0 +1,20 @@
+{
+  "d_model": 768,
+  "n_layer": 24,
+  "vocab_size": 4000,
+  "d_conv": 4,
+  "expand": 2,
+  "d_state": 64,
+  "head_dim": 64,
+  "d_inner": 1536,
+  "n_heads": 24,
+  "rms_norm_eps": 1e-05,
+  "pad_vocab_size_multiple": 16,
+  "tie_word_embeddings": true,
+  "model_type": "mamba2",
+  "architectures": [
+    "Mamba2ForCausalLM"
+  ],
+  "torch_dtype": "float16",
+  "transformers_version": "4.40.0"
+}

generation_config.json

@@ -0,0 +1,4 @@
+{
+  "eos_token_id": 2,
+  "transformers_version": "4.40.0"
+}

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:60eeda065f795d4e0596b02eb47e2d2efcb036312fd665208a2d1d164a0a890d
+size 188298360

tokenizer.model

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:72c5d481f9875ff63edb8b9d33c8a6ab46ccb3fcc71d1238100f7483ec18705f
+size 305450

tokenizer_config.json

@@ -0,0 +1,11 @@
+{
+  "tokenizer_class": "LlamaTokenizer",
+  "model_max_length": 1000000000000000019884624838656,
+  "add_bos_token": false,
+  "add_eos_token": false,
+  "bos_token": null,
+  "eos_token": "</s>",
+  "unk_token": "<unk>",
+  "pad_token": null,
+  "sp_model_kwargs": {}
+}