Upload folder using huggingface_hub

.gitattributes

@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+tokenizer.json filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,109 @@
+---
+language:
+- hu
+license: apache-2.0
+library_name: transformers
+base_model: utter-project/EuroLLM-22B
+pipeline_tag: text-generation
+tags:
+- hungarian
+- emese
+- eurollm
+- instruct
+- chatml
+---
+
+# Emese-Folyó (22B)
+
+**Folyó** ("river") is the flagship, server-class tier of the [Emese](https://emese.tech) Hungarian model
+family — [EuroLLM-22B](https://huggingface.co/utter-project/EuroLLM-22B) continued-pretrained on
+Hungarian, instruction-tuned, and DPO-aligned.
+
+| | |
+|---|---|
+| **Parameters** | 22B |
+| **Base** | EuroLLM-22B |
+| **Architecture** | LLaMA-style (RoPE θ=1,000,000, GQA 48Q/8KV, SwiGLU, RMSNorm) |
+| **Hidden / layers / heads** | 6144 / 54 / 48 (8 KV heads) |
+| **Vocabulary** | 128,000 (EuroLLM multilingual SentencePiece) |
+| **Max context length** | **32,768 tokens** (EuroLLM-22B's native `max_position_embeddings`) |
+| **Precision** | bfloat16 (this repo, dequantized for `transformers` compatibility); MLX q8 also available (`folyo-mlx/`) |
+| **License** | Apache-2.0 |
+
+## Formats in this release
+
+| Folder | Format | Size | Notes |
+|---|---|---|---|
+| `folyo/` (this repo) | bf16, standard HF `safetensors` | ~42 GB | loads directly with `transformers` **and** `mlx_lm`; dequantized from the q8-native training chain (near-lossless) |
+| `folyo-mlx/` | MLX q8 | ~22 GB | `mlx_lm`-only, the native training/serving precision for this model |
+
+## Usage (transformers)
+
+```python
+from transformers import AutoModelForCausalLM, AutoTokenizer
+import torch
+tok = AutoTokenizer.from_pretrained("emese-tech/folyo")
+model = AutoModelForCausalLM.from_pretrained("emese-tech/folyo", dtype=torch.bfloat16, device_map="auto")
+msgs = [{"role": "user", "content": "Mi Magyarország fővárosa?"}]
+prompt = tok.apply_chat_template(msgs, tokenize=False, add_generation_prompt=True)
+ids = tok(prompt, return_tensors="pt").to(model.device)
+out = model.generate(**ids, max_new_tokens=256, do_sample=True, temperature=0.2, eos_token_id=[2, 4])
+print(tok.decode(out[0][ids.input_ids.shape[1]:], skip_special_tokens=True))
+```
+
+## Usage (MLX)
+
+```python
+from mlx_lm import load, generate
+from mlx_lm.sample_utils import make_sampler
+model, tok = load("emese-tech/folyo-mlx")  # q8
+p = tok.apply_chat_template([{"role": "user", "content": "Mi Magyarország fővárosa?"}],
+                            tokenize=False, add_generation_prompt=True)
+print(generate(model, tok, prompt=p, max_tokens=256, sampler=make_sampler(temp=0.2)))
+```
+
+**Decode:** temperature `0.2`, no repetition penalty, **eos `{2, 4}`** (`</s>` and `<|im_end|>` — omitting
+id 4 causes non-stopping generation), ChatML template. For multi-turn conversations, **always pass the
+full history**, not just the latest message.
+
+## Training
+
+- **CPT** — q8-native continued pretraining of EuroLLM-22B on the Emese Hungarian corpus: **~6M tokens
+  over 6,000 iterations**, LoRA rank 64, then fused and **requantized to q8** (rather than kept in bf16)
+  to avoid a q4→bf16 fuse-precision mismatch discovered earlier in this project's history. The resulting
+  q8-native CPT base (`eurollm-22b-cpt-v2-q8`) is what every downstream SFT/DPO stage built on.
+- **SFT** — 1 epoch (4,914 iterations) on the `instruct_v18b` corpus (4,914 rows: persona, safety, code +
+  code-debug, hedging/anti-confabulation, multi-step reasoning, compound constraints, multi-turn
+  refinement, anti-repetition — same corpus as Patak/Csermely). LoRA rank 16 / scale 16, dropout 0.1,
+  lr 5e-6, bottom 14 of 54 layers frozen (`num_layers: 40`). Checkpoint-swept across the run; **iteration
+  3,600 (not the final iteration) was selected** as the best-behaving checkpoint.
+- **DPO** — 120 iterations of DPO-lite ("alfa") on 36 hand-written preference pairs (persona
+  identity-defense + anti-repetition — same pair bank as Patak/Csermely). LoRA rank 16 / scale 16,
+  lr 2e-6 — the lowest DPO learning rate in the family, since this checkpoint was already the peak-swept
+  best and had the least margin to spare.
+
+## Benchmarks
+
+| | Ultimate Bench (0-250) | BlindSpot Bench (0-376) |
+|---|---|---|
+| **This release (DPO alfa)** | 211/250 (84%) | **310/376** — family-best BlindSpot score |
+
+**emese-bench v1 (500 pts, consolidated Ultimate+BlindSpot, MLX q8): 410/500 (82%)** — the current
+unified reference benchmark going forward, just behind Patak (413/500) and far ahead of Csermely
+(211/500). Near-perfect on safety, safety_edge, honesty_calibration, persona_selfaware, reading,
+translation, and code-writing, with no repetition loops or stop-token leakage anywhere in the
+transcript. Weakest on multi-step math, logic puzzles, and two confidently-fabricated Hungarian
+scientist biographies; the standout chat-session finding is a perfect 10/10 on both in-context memory
+recall and roleplay (best in the family so far), offset by a surprising 1/10 on the English-chat
+session — the model answered every turn in Hungarian despite the user writing in English. See
+`emese-bench/results/folyo-mlx.md` and `emese-bench/README.md` for full category-level detail.
+
+## Limitations
+
+- Can hallucinate specific facts (dates, attributions) — verify critical details.
+- Hungarian-first; other-language quality inherited from EuroLLM-22B.
+- Large — 42 GB bf16 / 22 GB q8; needs a machine with enough RAM/VRAM for practical serving.
+- **Do not convert this model to GGUF without re-validating first** — this project's own testing found a
+  consistent ~14-27 point Ultimate Bench regression when serving via llama.cpp/GGUF (tested on Patak,
+  same architecture family), regardless of source precision or quantization level, root-caused to
+  llama.cpp's inference path itself. Use the `transformers`/`mlx_lm` paths above.

chat_template.jinja

@@ -0,0 +1,4 @@
+{% for message in messages %}<|im_start|>{{ message['role'] }}
+{{ message['content'] }}<|im_end|>
+{% endfor %}{% if add_generation_prompt %}<|im_start|>assistant
+{% endif %}

config.json

@@ -0,0 +1,32 @@
+{
+    "architectures": [
+        "LlamaForCausalLM"
+    ],
+    "attention_bias": false,
+    "attention_dropout": 0.0,
+    "bos_token_id": 1,
+    "dtype": "bfloat16",
+    "eos_token_id": [
+        2,
+        4
+    ],
+    "head_dim": 128,
+    "hidden_act": "silu",
+    "hidden_size": 6144,
+    "initializer_range": 0.02,
+    "intermediate_size": 16384,
+    "max_position_embeddings": 32768,
+    "mlp_bias": false,
+    "model_type": "llama",
+    "num_attention_heads": 48,
+    "num_hidden_layers": 54,
+    "num_key_value_heads": 8,
+    "pretraining_tp": 1,
+    "rms_norm_eps": 1e-05,
+    "rope_scaling": null,
+    "rope_theta": 1000000,
+    "tie_word_embeddings": false,
+    "transformers_version": "4.56.1",
+    "use_cache": true,
+    "vocab_size": 128000
+}

generation_config.json

@@ -0,0 +1,9 @@
+{
+  "_from_model_config": true,
+  "bos_token_id": 1,
+  "eos_token_id": [
+    2,
+    4
+  ],
+  "transformers_version": "4.56.1"
+}

model-00001-of-00009.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9c35cc6bcd936d6ec1baefbeac8368e4b15d26247b65ae79674c86111bd4be55
+size 5272343418

model-00002-of-00009.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9e89eaeaeba06a692f82630725d0f2e145f70e298c607383419f579dc0bbb4b1
+size 5259836455

model-00003-of-00009.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d2ec0ca7be6d5d492857a985aa925abf6950dbb9eaf6e84cad96423aef5f87ee
+size 5259836497

model-00004-of-00009.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1fd67acc3e7836e0f2ed326f0724bfd4b3a4ef9b3eec0c437cf2810cd46c6885
+size 5360499553

model-00005-of-00009.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5185e0bc08575af4271681dbbab4a5184ca7f9e3aa66acc782172341e2788df3
+size 5184314104

model-00006-of-00009.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:984bf07097191710ccd6fecf59b3536b452bc54c1a1b93fc7644b5270f21c101
+size 5259836497

model-00007-of-00009.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2da7777293cfb30d3723440a4538deecba0f864af8978be101e4a3e797915bdc
+size 5259836493

model-00008-of-00009.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8a0d4f3c287dfcc7e8e941020d6aa183206af4d11c7112247193ed01e9426af5
+size 5360499561

model-00009-of-00009.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:cf3f5c67814ad6a4762dfcdb7942fe2d8212a02bf42c7d093865b8fa0f5e5eb4
+size 3057711258

model.safetensors.index.json

@@ -0,0 +1,497 @@
+{
+    "metadata": {
+        "total_size": 45274656768,
+        "total_parameters": 22637328384
+    },
+    "weight_map": {
+        "lm_head.weight": "model-00009-of-00009.safetensors",
+        "model.embed_tokens.weight": "model-00001-of-00009.safetensors",
+        "model.layers.0.input_layernorm.weight": "model-00001-of-00009.safetensors",
+        "model.layers.0.mlp.down_proj.weight": "model-00001-of-00009.safetensors",
+        "model.layers.0.mlp.gate_proj.weight": "model-00001-of-00009.safetensors",
+        "model.layers.0.mlp.up_proj.weight": "model-00001-of-00009.safetensors",
+        "model.layers.0.post_attention_layernorm.weight": "model-00001-of-00009.safetensors",
+        "model.layers.0.self_attn.k_proj.weight": "model-00001-of-00009.safetensors",
+        "model.layers.0.self_attn.o_proj.weight": "model-00001-of-00009.safetensors",
+        "model.layers.0.self_attn.q_proj.weight": "model-00001-of-00009.safetensors",
+        "model.layers.0.self_attn.v_proj.weight": "model-00001-of-00009.safetensors",
+        "model.layers.1.input_layernorm.weight": "model-00001-of-00009.safetensors",
+        "model.layers.1.mlp.down_proj.weight": "model-00001-of-00009.safetensors",
+        "model.layers.1.mlp.gate_proj.weight": "model-00001-of-00009.safetensors",
+        "model.layers.1.mlp.up_proj.weight": "model-00001-of-00009.safetensors",
+        "model.layers.1.post_attention_layernorm.weight": "model-00001-of-00009.safetensors",
+        "model.layers.1.self_attn.k_proj.weight": "model-00001-of-00009.safetensors",
+        "model.layers.1.self_attn.o_proj.weight": "model-00001-of-00009.safetensors",
+        "model.layers.1.self_attn.q_proj.weight": "model-00001-of-00009.safetensors",
+        "model.layers.1.self_attn.v_proj.weight": "model-00001-of-00009.safetensors",
+        "model.layers.10.input_layernorm.weight": "model-00002-of-00009.safetensors",
+        "model.layers.10.mlp.down_proj.weight": "model-00002-of-00009.safetensors",
+        "model.layers.10.mlp.gate_proj.weight": "model-00002-of-00009.safetensors",
+        "model.layers.10.mlp.up_proj.weight": "model-00002-of-00009.safetensors",
+        "model.layers.10.post_attention_layernorm.weight": "model-00002-of-00009.safetensors",
+        "model.layers.10.self_attn.k_proj.weight": "model-00002-of-00009.safetensors",
+        "model.layers.10.self_attn.o_proj.weight": "model-00002-of-00009.safetensors",
+        "model.layers.10.self_attn.q_proj.weight": "model-00002-of-00009.safetensors",
+        "model.layers.10.self_attn.v_proj.weight": "model-00002-of-00009.safetensors",
+        "model.layers.11.input_layernorm.weight": "model-00003-of-00009.safetensors",
+        "model.layers.11.mlp.down_proj.weight": "model-00003-of-00009.safetensors",
+        "model.layers.11.mlp.gate_proj.weight": "model-00002-of-00009.safetensors",
+        "model.layers.11.mlp.up_proj.weight": "model-00003-of-00009.safetensors",
+        "model.layers.11.post_attention_layernorm.weight": "model-00003-of-00009.safetensors",
+        "model.layers.11.self_attn.k_proj.weight": "model-00002-of-00009.safetensors",
+        "model.layers.11.self_attn.o_proj.weight": "model-00002-of-00009.safetensors",
+        "model.layers.11.self_attn.q_proj.weight": "model-00002-of-00009.safetensors",
+        "model.layers.11.self_attn.v_proj.weight": "model-00002-of-00009.safetensors",
+        "model.layers.12.input_layernorm.weight": "model-00003-of-00009.safetensors",
+        "model.layers.12.mlp.down_proj.weight": "model-00003-of-00009.safetensors",
+        "model.layers.12.mlp.gate_proj.weight": "model-00003-of-00009.safetensors",
+        "model.layers.12.mlp.up_proj.weight": "model-00003-of-00009.safetensors",
+        "model.layers.12.post_attention_layernorm.weight": "model-00003-of-00009.safetensors",
+        "model.layers.12.self_attn.k_proj.weight": "model-00003-of-00009.safetensors",
+        "model.layers.12.self_attn.o_proj.weight": "model-00003-of-00009.safetensors",
+        "model.layers.12.self_attn.q_proj.weight": "model-00003-of-00009.safetensors",
+        "model.layers.12.self_attn.v_proj.weight": "model-00003-of-00009.safetensors",
+        "model.layers.13.input_layernorm.weight": "model-00003-of-00009.safetensors",
+        "model.layers.13.mlp.down_proj.weight": "model-00003-of-00009.safetensors",
+        "model.layers.13.mlp.gate_proj.weight": "model-00003-of-00009.safetensors",
+        "model.layers.13.mlp.up_proj.weight": "model-00003-of-00009.safetensors",
+        "model.layers.13.post_attention_layernorm.weight": "model-00003-of-00009.safetensors",
+        "model.layers.13.self_attn.k_proj.weight": "model-00003-of-00009.safetensors",
+        "model.layers.13.self_attn.o_proj.weight": "model-00003-of-00009.safetensors",
+        "model.layers.13.self_attn.q_proj.weight": "model-00003-of-00009.safetensors",
+        "model.layers.13.self_attn.v_proj.weight": "model-00003-of-00009.safetensors",
+        "model.layers.14.input_layernorm.weight": "model-00003-of-00009.safetensors",
+        "model.layers.14.mlp.down_proj.weight": "model-00003-of-00009.safetensors",
+        "model.layers.14.mlp.gate_proj.weight": "model-00003-of-00009.safetensors",
+        "model.layers.14.mlp.up_proj.weight": "model-00003-of-00009.safetensors",
+        "model.layers.14.post_attention_layernorm.weight": "model-00003-of-00009.safetensors",
+        "model.layers.14.self_attn.k_proj.weight": "model-00003-of-00009.safetensors",
+        "model.layers.14.self_attn.o_proj.weight": "model-00003-of-00009.safetensors",
+        "model.layers.14.self_attn.q_proj.weight": "model-00003-of-00009.safetensors",
+        "model.layers.14.self_attn.v_proj.weight": "model-00003-of-00009.safetensors",
+        "model.layers.15.input_layernorm.weight": "model-00003-of-00009.safetensors",
+        "model.layers.15.mlp.down_proj.weight": "model-00003-of-00009.safetensors",
+        "model.layers.15.mlp.gate_proj.weight": "model-00003-of-00009.safetensors",
+        "model.layers.15.mlp.up_proj.weight": "model-00003-of-00009.safetensors",
+        "model.layers.15.post_attention_layernorm.weight": "model-00003-of-00009.safetensors",
+        "model.layers.15.self_attn.k_proj.weight": "model-00003-of-00009.safetensors",
+        "model.layers.15.self_attn.o_proj.weight": "model-00003-of-00009.safetensors",
+        "model.layers.15.self_attn.q_proj.weight": "model-00003-of-00009.safetensors",
+        "model.layers.15.self_attn.v_proj.weight": "model-00003-of-00009.safetensors",
+        "model.layers.16.input_layernorm.weight": "model-00003-of-00009.safetensors",
+        "model.layers.16.mlp.down_proj.weight": "model-00003-of-00009.safetensors",
+        "model.layers.16.mlp.gate_proj.weight": "model-00003-of-00009.safetensors",
+        "model.layers.16.mlp.up_proj.weight": "model-00003-of-00009.safetensors",
+        "model.layers.16.post_attention_layernorm.weight": "model-00003-of-00009.safetensors",
+        "model.layers.16.self_attn.k_proj.weight": "model-00003-of-00009.safetensors",
+        "model.layers.16.self_attn.o_proj.weight": "model-00003-of-00009.safetensors",
+        "model.layers.16.self_attn.q_proj.weight": "model-00003-of-00009.safetensors",
+        "model.layers.16.self_attn.v_proj.weight": "model-00003-of-00009.safetensors",
+        "model.layers.17.input_layernorm.weight": "model-00003-of-00009.safetensors",
+        "model.layers.17.mlp.down_proj.weight": "model-00003-of-00009.safetensors",
+        "model.layers.17.mlp.gate_proj.weight": "model-00003-of-00009.safetensors",
+        "model.layers.17.mlp.up_proj.weight": "model-00003-of-00009.safetensors",
+        "model.layers.17.post_attention_layernorm.weight": "model-00003-of-00009.safetensors",
+        "model.layers.17.self_attn.k_proj.weight": "model-00003-of-00009.safetensors",
+        "model.layers.17.self_attn.o_proj.weight": "model-00003-of-00009.safetensors",
+        "model.layers.17.self_attn.q_proj.weight": "model-00003-of-00009.safetensors",
+        "model.layers.17.self_attn.v_proj.weight": "model-00003-of-00009.safetensors",
+        "model.layers.18.input_layernorm.weight": "model-00004-of-00009.safetensors",
+        "model.layers.18.mlp.down_proj.weight": "model-00004-of-00009.safetensors",
+        "model.layers.18.mlp.gate_proj.weight": "model-00004-of-00009.safetensors",
+        "model.layers.18.mlp.up_proj.weight": "model-00004-of-00009.safetensors",
+        "model.layers.18.post_attention_layernorm.weight": "model-00004-of-00009.safetensors",
+        "model.layers.18.self_attn.k_proj.weight": "model-00003-of-00009.safetensors",
+        "model.layers.18.self_attn.o_proj.weight": "model-00003-of-00009.safetensors",
+        "model.layers.18.self_attn.q_proj.weight": "model-00003-of-00009.safetensors",
+        "model.layers.18.self_attn.v_proj.weight": "model-00003-of-00009.safetensors",
+        "model.layers.19.input_layernorm.weight": "model-00004-of-00009.safetensors",
+        "model.layers.19.mlp.down_proj.weight": "model-00004-of-00009.safetensors",
+        "model.layers.19.mlp.gate_proj.weight": "model-00004-of-00009.safetensors",
+        "model.layers.19.mlp.up_proj.weight": "model-00004-of-00009.safetensors",
+        "model.layers.19.post_attention_layernorm.weight": "model-00004-of-00009.safetensors",
+        "model.layers.19.self_attn.k_proj.weight": "model-00004-of-00009.safetensors",
+        "model.layers.19.self_attn.o_proj.weight": "model-00004-of-00009.safetensors",
+        "model.layers.19.self_attn.q_proj.weight": "model-00004-of-00009.safetensors",
+        "model.layers.19.self_attn.v_proj.weight": "model-00004-of-00009.safetensors",
+        "model.layers.2.input_layernorm.weight": "model-00001-of-00009.safetensors",
+        "model.layers.2.mlp.down_proj.weight": "model-00001-of-00009.safetensors",
+        "model.layers.2.mlp.gate_proj.weight": "model-00001-of-00009.safetensors",
+        "model.layers.2.mlp.up_proj.weight": "model-00001-of-00009.safetensors",
+        "model.layers.2.post_attention_layernorm.weight": "model-00001-of-00009.safetensors",
+        "model.layers.2.self_attn.k_proj.weight": "model-00001-of-00009.safetensors",
+        "model.layers.2.self_attn.o_proj.weight": "model-00001-of-00009.safetensors",
+        "model.layers.2.self_attn.q_proj.weight": "model-00001-of-00009.safetensors",
+        "model.layers.2.self_attn.v_proj.weight": "model-00001-of-00009.safetensors",
+        "model.layers.20.input_layernorm.weight": "model-00004-of-00009.safetensors",
+        "model.layers.20.mlp.down_proj.weight": "model-00004-of-00009.safetensors",
+        "model.layers.20.mlp.gate_proj.weight": "model-00004-of-00009.safetensors",
+        "model.layers.20.mlp.up_proj.weight": "model-00004-of-00009.safetensors",
+        "model.layers.20.post_attention_layernorm.weight": "model-00004-of-00009.safetensors",
+        "model.layers.20.self_attn.k_proj.weight": "model-00004-of-00009.safetensors",
+        "model.layers.20.self_attn.o_proj.weight": "model-00004-of-00009.safetensors",
+        "model.layers.20.self_attn.q_proj.weight": "model-00004-of-00009.safetensors",
+        "model.layers.20.self_attn.v_proj.weight": "model-00004-of-00009.safetensors",
+        "model.layers.21.input_layernorm.weight": "model-00004-of-00009.safetensors",
+        "model.layers.21.mlp.down_proj.weight": "model-00004-of-00009.safetensors",
+        "model.layers.21.mlp.gate_proj.weight": "model-00004-of-00009.safetensors",
+        "model.layers.21.mlp.up_proj.weight": "model-00004-of-00009.safetensors",
+        "model.layers.21.post_attention_layernorm.weight": "model-00004-of-00009.safetensors",
+        "model.layers.21.self_attn.k_proj.weight": "model-00004-of-00009.safetensors",
+        "model.layers.21.self_attn.o_proj.weight": "model-00004-of-00009.safetensors",
+        "model.layers.21.self_attn.q_proj.weight": "model-00004-of-00009.safetensors",
+        "model.layers.21.self_attn.v_proj.weight": "model-00004-of-00009.safetensors",
+        "model.layers.22.input_layernorm.weight": "model-00004-of-00009.safetensors",
+        "model.layers.22.mlp.down_proj.weight": "model-00004-of-00009.safetensors",
+        "model.layers.22.mlp.gate_proj.weight": "model-00004-of-00009.safetensors",
+        "model.layers.22.mlp.up_proj.weight": "model-00004-of-00009.safetensors",
+        "model.layers.22.post_attention_layernorm.weight": "model-00004-of-00009.safetensors",
+        "model.layers.22.self_attn.k_proj.weight": "model-00004-of-00009.safetensors",
+        "model.layers.22.self_attn.o_proj.weight": "model-00004-of-00009.safetensors",
+        "model.layers.22.self_attn.q_proj.weight": "model-00004-of-00009.safetensors",
+        "model.layers.22.self_attn.v_proj.weight": "model-00004-of-00009.safetensors",
+        "model.layers.23.input_layernorm.weight": "model-00004-of-00009.safetensors",
+        "model.layers.23.mlp.down_proj.weight": "model-00004-of-00009.safetensors",
+        "model.layers.23.mlp.gate_proj.weight": "model-00004-of-00009.safetensors",
+        "model.layers.23.mlp.up_proj.weight": "model-00004-of-00009.safetensors",
+        "model.layers.23.post_attention_layernorm.weight": "model-00004-of-00009.safetensors",
+        "model.layers.23.self_attn.k_proj.weight": "model-00004-of-00009.safetensors",
+        "model.layers.23.self_attn.o_proj.weight": "model-00004-of-00009.safetensors",
+        "model.layers.23.self_attn.q_proj.weight": "model-00004-of-00009.safetensors",
+        "model.layers.23.self_attn.v_proj.weight": "model-00004-of-00009.safetensors",
+        "model.layers.24.input_layernorm.weight": "model-00004-of-00009.safetensors",
+        "model.layers.24.mlp.down_proj.weight": "model-00004-of-00009.safetensors",
+        "model.layers.24.mlp.gate_proj.weight": "model-00004-of-00009.safetensors",
+        "model.layers.24.mlp.up_proj.weight": "model-00004-of-00009.safetensors",
+        "model.layers.24.post_attention_layernorm.weight": "model-00004-of-00009.safetensors",
+        "model.layers.24.self_attn.k_proj.weight": "model-00004-of-00009.safetensors",
+        "model.layers.24.self_attn.o_proj.weight": "model-00004-of-00009.safetensors",
+        "model.layers.24.self_attn.q_proj.weight": "model-00004-of-00009.safetensors",
+        "model.layers.24.self_attn.v_proj.weight": "model-00004-of-00009.safetensors",
+        "model.layers.25.input_layernorm.weight": "model-00005-of-00009.safetensors",
+        "model.layers.25.mlp.down_proj.weight": "model-00005-of-00009.safetensors",
+        "model.layers.25.mlp.gate_proj.weight": "model-00005-of-00009.safetensors",
+        "model.layers.25.mlp.up_proj.weight": "model-00005-of-00009.safetensors",
+        "model.layers.25.post_attention_layernorm.weight": "model-00005-of-00009.safetensors",
+        "model.layers.25.self_attn.k_proj.weight": "model-00005-of-00009.safetensors",
+        "model.layers.25.self_attn.o_proj.weight": "model-00005-of-00009.safetensors",
+        "model.layers.25.self_attn.q_proj.weight": "model-00004-of-00009.safetensors",
+        "model.layers.25.self_attn.v_proj.weight": "model-00005-of-00009.safetensors",
+        "model.layers.26.input_layernorm.weight": "model-00005-of-00009.safetensors",
+        "model.layers.26.mlp.down_proj.weight": "model-00005-of-00009.safetensors",
+        "model.layers.26.mlp.gate_proj.weight": "model-00005-of-00009.safetensors",
+        "model.layers.26.mlp.up_proj.weight": "model-00005-of-00009.safetensors",
+        "model.layers.26.post_attention_layernorm.weight": "model-00005-of-00009.safetensors",
+        "model.layers.26.self_attn.k_proj.weight": "model-00005-of-00009.safetensors",
+        "model.layers.26.self_attn.o_proj.weight": "model-00005-of-00009.safetensors",
+        "model.layers.26.self_attn.q_proj.weight": "model-00005-of-00009.safetensors",
+        "model.layers.26.self_attn.v_proj.weight": "model-00005-of-00009.safetensors",
+        "model.layers.27.input_layernorm.weight": "model-00005-of-00009.safetensors",
+        "model.layers.27.mlp.down_proj.weight": "model-00005-of-00009.safetensors",
+        "model.layers.27.mlp.gate_proj.weight": "model-00005-of-00009.safetensors",
+        "model.layers.27.mlp.up_proj.weight": "model-00005-of-00009.safetensors",
+        "model.layers.27.post_attention_layernorm.weight": "model-00005-of-00009.safetensors",
+        "model.layers.27.self_attn.k_proj.weight": "model-00005-of-00009.safetensors",
+        "model.layers.27.self_attn.o_proj.weight": "model-00005-of-00009.safetensors",
+        "model.layers.27.self_attn.q_proj.weight": "model-00005-of-00009.safetensors",
+        "model.layers.27.self_attn.v_proj.weight": "model-00005-of-00009.safetensors",
+        "model.layers.28.input_layernorm.weight": "model-00005-of-00009.safetensors",
+        "model.layers.28.mlp.down_proj.weight": "model-00005-of-00009.safetensors",
+        "model.layers.28.mlp.gate_proj.weight": "model-00005-of-00009.safetensors",
+        "model.layers.28.mlp.up_proj.weight": "model-00005-of-00009.safetensors",
+        "model.layers.28.post_attention_layernorm.weight": "model-00005-of-00009.safetensors",
+        "model.layers.28.self_attn.k_proj.weight": "model-00005-of-00009.safetensors",
+        "model.layers.28.self_attn.o_proj.weight": "model-00005-of-00009.safetensors",
+        "model.layers.28.self_attn.q_proj.weight": "model-00005-of-00009.safetensors",
+        "model.layers.28.self_attn.v_proj.weight": "model-00005-of-00009.safetensors",
+        "model.layers.29.input_layernorm.weight": "model-00005-of-00009.safetensors",
+        "model.layers.29.mlp.down_proj.weight": "model-00005-of-00009.safetensors",
+        "model.layers.29.mlp.gate_proj.weight": "model-00005-of-00009.safetensors",
+        "model.layers.29.mlp.up_proj.weight": "model-00005-of-00009.safetensors",
+        "model.layers.29.post_attention_layernorm.weight": "model-00005-of-00009.safetensors",
+        "model.layers.29.self_attn.k_proj.weight": "model-00005-of-00009.safetensors",
+        "model.layers.29.self_attn.o_proj.weight": "model-00005-of-00009.safetensors",
+        "model.layers.29.self_attn.q_proj.weight": "model-00005-of-00009.safetensors",
+        "model.layers.29.self_attn.v_proj.weight": "model-00005-of-00009.safetensors",
+        "model.layers.3.input_layernorm.weight": "model-00001-of-00009.safetensors",
+        "model.layers.3.mlp.down_proj.weight": "model-00001-of-00009.safetensors",
+        "model.layers.3.mlp.gate_proj.weight": "model-00001-of-00009.safetensors",
+        "model.layers.3.mlp.up_proj.weight": "model-00001-of-00009.safetensors",
+        "model.layers.3.post_attention_layernorm.weight": "model-00001-of-00009.safetensors",
+        "model.layers.3.self_attn.k_proj.weight": "model-00001-of-00009.safetensors",
+        "model.layers.3.self_attn.o_proj.weight": "model-00001-of-00009.safetensors",
+        "model.layers.3.self_attn.q_proj.weight": "model-00001-of-00009.safetensors",
+        "model.layers.3.self_attn.v_proj.weight": "model-00001-of-00009.safetensors",
+        "model.layers.30.input_layernorm.weight": "model-00005-of-00009.safetensors",
+        "model.layers.30.mlp.down_proj.weight": "model-00005-of-00009.safetensors",
+        "model.layers.30.mlp.gate_proj.weight": "model-00005-of-00009.safetensors",
+        "model.layers.30.mlp.up_proj.weight": "model-00005-of-00009.safetensors",
+        "model.layers.30.post_attention_layernorm.weight": "model-00005-of-00009.safetensors",
+        "model.layers.30.self_attn.k_proj.weight": "model-00005-of-00009.safetensors",
+        "model.layers.30.self_attn.o_proj.weight": "model-00005-of-00009.safetensors",
+        "model.layers.30.self_attn.q_proj.weight": "model-00005-of-00009.safetensors",
+        "model.layers.30.self_attn.v_proj.weight": "model-00005-of-00009.safetensors",
+        "model.layers.31.input_layernorm.weight": "model-00006-of-00009.safetensors",
+        "model.layers.31.mlp.down_proj.weight": "model-00005-of-00009.safetensors",
+        "model.layers.31.mlp.gate_proj.weight": "model-00005-of-00009.safetensors",
+        "model.layers.31.mlp.up_proj.weight": "model-00006-of-00009.safetensors",
+        "model.layers.31.post_attention_layernorm.weight": "model-00006-of-00009.safetensors",
+        "model.layers.31.self_attn.k_proj.weight": "model-00005-of-00009.safetensors",
+        "model.layers.31.self_attn.o_proj.weight": "model-00005-of-00009.safetensors",
+        "model.layers.31.self_attn.q_proj.weight": "model-00005-of-00009.safetensors",
+        "model.layers.31.self_attn.v_proj.weight": "model-00005-of-00009.safetensors",
+        "model.layers.32.input_layernorm.weight": "model-00006-of-00009.safetensors",
+        "model.layers.32.mlp.down_proj.weight": "model-00006-of-00009.safetensors",
+        "model.layers.32.mlp.gate_proj.weight": "model-00006-of-00009.safetensors",
+        "model.layers.32.mlp.up_proj.weight": "model-00006-of-00009.safetensors",
+        "model.layers.32.post_attention_layernorm.weight": "model-00006-of-00009.safetensors",
+        "model.layers.32.self_attn.k_proj.weight": "model-00006-of-00009.safetensors",
+        "model.layers.32.self_attn.o_proj.weight": "model-00006-of-00009.safetensors",
+        "model.layers.32.self_attn.q_proj.weight": "model-00006-of-00009.safetensors",
+        "model.layers.32.self_attn.v_proj.weight": "model-00006-of-00009.safetensors",
+        "model.layers.33.input_layernorm.weight": "model-00006-of-00009.safetensors",
+        "model.layers.33.mlp.down_proj.weight": "model-00006-of-00009.safetensors",
+        "model.layers.33.mlp.gate_proj.weight": "model-00006-of-00009.safetensors",
+        "model.layers.33.mlp.up_proj.weight": "model-00006-of-00009.safetensors",
+        "model.layers.33.post_attention_layernorm.weight": "model-00006-of-00009.safetensors",
+        "model.layers.33.self_attn.k_proj.weight": "model-00006-of-00009.safetensors",
+        "model.layers.33.self_attn.o_proj.weight": "model-00006-of-00009.safetensors",
+        "model.layers.33.self_attn.q_proj.weight": "model-00006-of-00009.safetensors",
+        "model.layers.33.self_attn.v_proj.weight": "model-00006-of-00009.safetensors",
+        "model.layers.34.input_layernorm.weight": "model-00006-of-00009.safetensors",
+        "model.layers.34.mlp.down_proj.weight": "model-00006-of-00009.safetensors",
+        "model.layers.34.mlp.gate_proj.weight": "model-00006-of-00009.safetensors",
+        "model.layers.34.mlp.up_proj.weight": "model-00006-of-00009.safetensors",
+        "model.layers.34.post_attention_layernorm.weight": "model-00006-of-00009.safetensors",
+        "model.layers.34.self_attn.k_proj.weight": "model-00006-of-00009.safetensors",
+        "model.layers.34.self_attn.o_proj.weight": "model-00006-of-00009.safetensors",
+        "model.layers.34.self_attn.q_proj.weight": "model-00006-of-00009.safetensors",
+        "model.layers.34.self_attn.v_proj.weight": "model-00006-of-00009.safetensors",
+        "model.layers.35.input_layernorm.weight": "model-00006-of-00009.safetensors",
+        "model.layers.35.mlp.down_proj.weight": "model-00006-of-00009.safetensors",
+        "model.layers.35.mlp.gate_proj.weight": "model-00006-of-00009.safetensors",
+        "model.layers.35.mlp.up_proj.weight": "model-00006-of-00009.safetensors",
+        "model.layers.35.post_attention_layernorm.weight": "model-00006-of-00009.safetensors",
+        "model.layers.35.self_attn.k_proj.weight": "model-00006-of-00009.safetensors",
+        "model.layers.35.self_attn.o_proj.weight": "model-00006-of-00009.safetensors",
+        "model.layers.35.self_attn.q_proj.weight": "model-00006-of-00009.safetensors",
+        "model.layers.35.self_attn.v_proj.weight": "model-00006-of-00009.safetensors",
+        "model.layers.36.input_layernorm.weight": "model-00006-of-00009.safetensors",
+        "model.layers.36.mlp.down_proj.weight": "model-00006-of-00009.safetensors",
+        "model.layers.36.mlp.gate_proj.weight": "model-00006-of-00009.safetensors",
+        "model.layers.36.mlp.up_proj.weight": "model-00006-of-00009.safetensors",
+        "model.layers.36.post_attention_layernorm.weight": "model-00006-of-00009.safetensors",
+        "model.layers.36.self_attn.k_proj.weight": "model-00006-of-00009.safetensors",
+        "model.layers.36.self_attn.o_proj.weight": "model-00006-of-00009.safetensors",
+        "model.layers.36.self_attn.q_proj.weight": "model-00006-of-00009.safetensors",
+        "model.layers.36.self_attn.v_proj.weight": "model-00006-of-00009.safetensors",
+        "model.layers.37.input_layernorm.weight": "model-00006-of-00009.safetensors",
+        "model.layers.37.mlp.down_proj.weight": "model-00006-of-00009.safetensors",
+        "model.layers.37.mlp.gate_proj.weight": "model-00006-of-00009.safetensors",
+        "model.layers.37.mlp.up_proj.weight": "model-00006-of-00009.safetensors",
+        "model.layers.37.post_attention_layernorm.weight": "model-00006-of-00009.safetensors",
+        "model.layers.37.self_attn.k_proj.weight": "model-00006-of-00009.safetensors",
+        "model.layers.37.self_attn.o_proj.weight": "model-00006-of-00009.safetensors",
+        "model.layers.37.self_attn.q_proj.weight": "model-00006-of-00009.safetensors",
+        "model.layers.37.self_attn.v_proj.weight": "model-00006-of-00009.safetensors",
+        "model.layers.38.input_layernorm.weight": "model-00007-of-00009.safetensors",
+        "model.layers.38.mlp.down_proj.weight": "model-00007-of-00009.safetensors",
+        "model.layers.38.mlp.gate_proj.weight": "model-00006-of-00009.safetensors",
+        "model.layers.38.mlp.up_proj.weight": "model-00007-of-00009.safetensors",
+        "model.layers.38.post_attention_layernorm.weight": "model-00007-of-00009.safetensors",
+        "model.layers.38.self_attn.k_proj.weight": "model-00006-of-00009.safetensors",
+        "model.layers.38.self_attn.o_proj.weight": "model-00006-of-00009.safetensors",
+        "model.layers.38.self_attn.q_proj.weight": "model-00006-of-00009.safetensors",
+        "model.layers.38.self_attn.v_proj.weight": "model-00006-of-00009.safetensors",
+        "model.layers.39.input_layernorm.weight": "model-00007-of-00009.safetensors",
+        "model.layers.39.mlp.down_proj.weight": "model-00007-of-00009.safetensors",
+        "model.layers.39.mlp.gate_proj.weight": "model-00007-of-00009.safetensors",
+        "model.layers.39.mlp.up_proj.weight": "model-00007-of-00009.safetensors",
+        "model.layers.39.post_attention_layernorm.weight": "model-00007-of-00009.safetensors",
+        "model.layers.39.self_attn.k_proj.weight": "model-00007-of-00009.safetensors",
+        "model.layers.39.self_attn.o_proj.weight": "model-00007-of-00009.safetensors",
+        "model.layers.39.self_attn.q_proj.weight": "model-00007-of-00009.safetensors",
+        "model.layers.39.self_attn.v_proj.weight": "model-00007-of-00009.safetensors",
+        "model.layers.4.input_layernorm.weight": "model-00002-of-00009.safetensors",
+        "model.layers.4.mlp.down_proj.weight": "model-00001-of-00009.safetensors",
+        "model.layers.4.mlp.gate_proj.weight": "model-00001-of-00009.safetensors",
+        "model.layers.4.mlp.up_proj.weight": "model-00002-of-00009.safetensors",
+        "model.layers.4.post_attention_layernorm.weight": "model-00002-of-00009.safetensors",
+        "model.layers.4.self_attn.k_proj.weight": "model-00001-of-00009.safetensors",
+        "model.layers.4.self_attn.o_proj.weight": "model-00001-of-00009.safetensors",
+        "model.layers.4.self_attn.q_proj.weight": "model-00001-of-00009.safetensors",
+        "model.layers.4.self_attn.v_proj.weight": "model-00001-of-00009.safetensors",
+        "model.layers.40.input_layernorm.weight": "model-00007-of-00009.safetensors",
+        "model.layers.40.mlp.down_proj.weight": "model-00007-of-00009.safetensors",
+        "model.layers.40.mlp.gate_proj.weight": "model-00007-of-00009.safetensors",
+        "model.layers.40.mlp.up_proj.weight": "model-00007-of-00009.safetensors",
+        "model.layers.40.post_attention_layernorm.weight": "model-00007-of-00009.safetensors",
+        "model.layers.40.self_attn.k_proj.weight": "model-00007-of-00009.safetensors",
+        "model.layers.40.self_attn.o_proj.weight": "model-00007-of-00009.safetensors",
+        "model.layers.40.self_attn.q_proj.weight": "model-00007-of-00009.safetensors",
+        "model.layers.40.self_attn.v_proj.weight": "model-00007-of-00009.safetensors",
+        "model.layers.41.input_layernorm.weight": "model-00007-of-00009.safetensors",
+        "model.layers.41.mlp.down_proj.weight": "model-00007-of-00009.safetensors",
+        "model.layers.41.mlp.gate_proj.weight": "model-00007-of-00009.safetensors",
+        "model.layers.41.mlp.up_proj.weight": "model-00007-of-00009.safetensors",
+        "model.layers.41.post_attention_layernorm.weight": "model-00007-of-00009.safetensors",
+        "model.layers.41.self_attn.k_proj.weight": "model-00007-of-00009.safetensors",
+        "model.layers.41.self_attn.o_proj.weight": "model-00007-of-00009.safetensors",
+        "model.layers.41.self_attn.q_proj.weight": "model-00007-of-00009.safetensors",
+        "model.layers.41.self_attn.v_proj.weight": "model-00007-of-00009.safetensors",
+        "model.layers.42.input_layernorm.weight": "model-00007-of-00009.safetensors",
+        "model.layers.42.mlp.down_proj.weight": "model-00007-of-00009.safetensors",
+        "model.layers.42.mlp.gate_proj.weight": "model-00007-of-00009.safetensors",
+        "model.layers.42.mlp.up_proj.weight": "model-00007-of-00009.safetensors",
+        "model.layers.42.post_attention_layernorm.weight": "model-00007-of-00009.safetensors",
+        "model.layers.42.self_attn.k_proj.weight": "model-00007-of-00009.safetensors",
+        "model.layers.42.self_attn.o_proj.weight": "model-00007-of-00009.safetensors",
+        "model.layers.42.self_attn.q_proj.weight": "model-00007-of-00009.safetensors",
+        "model.layers.42.self_attn.v_proj.weight": "model-00007-of-00009.safetensors",
+        "model.layers.43.input_layernorm.weight": "model-00007-of-00009.safetensors",
+        "model.layers.43.mlp.down_proj.weight": "model-00007-of-00009.safetensors",
+        "model.layers.43.mlp.gate_proj.weight": "model-00007-of-00009.safetensors",
+        "model.layers.43.mlp.up_proj.weight": "model-00007-of-00009.safetensors",
+        "model.layers.43.post_attention_layernorm.weight": "model-00007-of-00009.safetensors",
+        "model.layers.43.self_attn.k_proj.weight": "model-00007-of-00009.safetensors",
+        "model.layers.43.self_attn.o_proj.weight": "model-00007-of-00009.safetensors",
+        "model.layers.43.self_attn.q_proj.weight": "model-00007-of-00009.safetensors",
+        "model.layers.43.self_attn.v_proj.weight": "model-00007-of-00009.safetensors",
+        "model.layers.44.input_layernorm.weight": "model-00007-of-00009.safetensors",
+        "model.layers.44.mlp.down_proj.weight": "model-00007-of-00009.safetensors",
+        "model.layers.44.mlp.gate_proj.weight": "model-00007-of-00009.safetensors",
+        "model.layers.44.mlp.up_proj.weight": "model-00007-of-00009.safetensors",
+        "model.layers.44.post_attention_layernorm.weight": "model-00007-of-00009.safetensors",
+        "model.layers.44.self_attn.k_proj.weight": "model-00007-of-00009.safetensors",
+        "model.layers.44.self_attn.o_proj.weight": "model-00007-of-00009.safetensors",
+        "model.layers.44.self_attn.q_proj.weight": "model-00007-of-00009.safetensors",
+        "model.layers.44.self_attn.v_proj.weight": "model-00007-of-00009.safetensors",
+        "model.layers.45.input_layernorm.weight": "model-00008-of-00009.safetensors",
+        "model.layers.45.mlp.down_proj.weight": "model-00008-of-00009.safetensors",
+        "model.layers.45.mlp.gate_proj.weight": "model-00008-of-00009.safetensors",
+        "model.layers.45.mlp.up_proj.weight": "model-00008-of-00009.safetensors",
+        "model.layers.45.post_attention_layernorm.weight": "model-00008-of-00009.safetensors",
+        "model.layers.45.self_attn.k_proj.weight": "model-00007-of-00009.safetensors",
+        "model.layers.45.self_attn.o_proj.weight": "model-00007-of-00009.safetensors",
+        "model.layers.45.self_attn.q_proj.weight": "model-00007-of-00009.safetensors",
+        "model.layers.45.self_attn.v_proj.weight": "model-00007-of-00009.safetensors",
+        "model.layers.46.input_layernorm.weight": "model-00008-of-00009.safetensors",
+        "model.layers.46.mlp.down_proj.weight": "model-00008-of-00009.safetensors",
+        "model.layers.46.mlp.gate_proj.weight": "model-00008-of-00009.safetensors",
+        "model.layers.46.mlp.up_proj.weight": "model-00008-of-00009.safetensors",
+        "model.layers.46.post_attention_layernorm.weight": "model-00008-of-00009.safetensors",
+        "model.layers.46.self_attn.k_proj.weight": "model-00008-of-00009.safetensors",
+        "model.layers.46.self_attn.o_proj.weight": "model-00008-of-00009.safetensors",
+        "model.layers.46.self_attn.q_proj.weight": "model-00008-of-00009.safetensors",
+        "model.layers.46.self_attn.v_proj.weight": "model-00008-of-00009.safetensors",
+        "model.layers.47.input_layernorm.weight": "model-00008-of-00009.safetensors",
+        "model.layers.47.mlp.down_proj.weight": "model-00008-of-00009.safetensors",
+        "model.layers.47.mlp.gate_proj.weight": "model-00008-of-00009.safetensors",
+        "model.layers.47.mlp.up_proj.weight": "model-00008-of-00009.safetensors",
+        "model.layers.47.post_attention_layernorm.weight": "model-00008-of-00009.safetensors",
+        "model.layers.47.self_attn.k_proj.weight": "model-00008-of-00009.safetensors",
+        "model.layers.47.self_attn.o_proj.weight": "model-00008-of-00009.safetensors",
+        "model.layers.47.self_attn.q_proj.weight": "model-00008-of-00009.safetensors",
+        "model.layers.47.self_attn.v_proj.weight": "model-00008-of-00009.safetensors",
+        "model.layers.48.input_layernorm.weight": "model-00008-of-00009.safetensors",
+        "model.layers.48.mlp.down_proj.weight": "model-00008-of-00009.safetensors",
+        "model.layers.48.mlp.gate_proj.weight": "model-00008-of-00009.safetensors",
+        "model.layers.48.mlp.up_proj.weight": "model-00008-of-00009.safetensors",
+        "model.layers.48.post_attention_layernorm.weight": "model-00008-of-00009.safetensors",
+        "model.layers.48.self_attn.k_proj.weight": "model-00008-of-00009.safetensors",
+        "model.layers.48.self_attn.o_proj.weight": "model-00008-of-00009.safetensors",
+        "model.layers.48.self_attn.q_proj.weight": "model-00008-of-00009.safetensors",
+        "model.layers.48.self_attn.v_proj.weight": "model-00008-of-00009.safetensors",
+        "model.layers.49.input_layernorm.weight": "model-00008-of-00009.safetensors",
+        "model.layers.49.mlp.down_proj.weight": "model-00008-of-00009.safetensors",
+        "model.layers.49.mlp.gate_proj.weight": "model-00008-of-00009.safetensors",
+        "model.layers.49.mlp.up_proj.weight": "model-00008-of-00009.safetensors",
+        "model.layers.49.post_attention_layernorm.weight": "model-00008-of-00009.safetensors",
+        "model.layers.49.self_attn.k_proj.weight": "model-00008-of-00009.safetensors",
+        "model.layers.49.self_attn.o_proj.weight": "model-00008-of-00009.safetensors",
+        "model.layers.49.self_attn.q_proj.weight": "model-00008-of-00009.safetensors",
+        "model.layers.49.self_attn.v_proj.weight": "model-00008-of-00009.safetensors",
+        "model.layers.5.input_layernorm.weight": "model-00002-of-00009.safetensors",
+        "model.layers.5.mlp.down_proj.weight": "model-00002-of-00009.safetensors",
+        "model.layers.5.mlp.gate_proj.weight": "model-00002-of-00009.safetensors",
+        "model.layers.5.mlp.up_proj.weight": "model-00002-of-00009.safetensors",
+        "model.layers.5.post_attention_layernorm.weight": "model-00002-of-00009.safetensors",
+        "model.layers.5.self_attn.k_proj.weight": "model-00002-of-00009.safetensors",
+        "model.layers.5.self_attn.o_proj.weight": "model-00002-of-00009.safetensors",
+        "model.layers.5.self_attn.q_proj.weight": "model-00002-of-00009.safetensors",
+        "model.layers.5.self_attn.v_proj.weight": "model-00002-of-00009.safetensors",
+        "model.layers.50.input_layernorm.weight": "model-00008-of-00009.safetensors",
+        "model.layers.50.mlp.down_proj.weight": "model-00008-of-00009.safetensors",
+        "model.layers.50.mlp.gate_proj.weight": "model-00008-of-00009.safetensors",
+        "model.layers.50.mlp.up_proj.weight": "model-00008-of-00009.safetensors",
+        "model.layers.50.post_attention_layernorm.weight": "model-00008-of-00009.safetensors",
+        "model.layers.50.self_attn.k_proj.weight": "model-00008-of-00009.safetensors",
+        "model.layers.50.self_attn.o_proj.weight": "model-00008-of-00009.safetensors",
+        "model.layers.50.self_attn.q_proj.weight": "model-00008-of-00009.safetensors",
+        "model.layers.50.self_attn.v_proj.weight": "model-00008-of-00009.safetensors",
+        "model.layers.51.input_layernorm.weight": "model-00008-of-00009.safetensors",
+        "model.layers.51.mlp.down_proj.weight": "model-00008-of-00009.safetensors",
+        "model.layers.51.mlp.gate_proj.weight": "model-00008-of-00009.safetensors",
+        "model.layers.51.mlp.up_proj.weight": "model-00008-of-00009.safetensors",
+        "model.layers.51.post_attention_layernorm.weight": "model-00008-of-00009.safetensors",
+        "model.layers.51.self_attn.k_proj.weight": "model-00008-of-00009.safetensors",
+        "model.layers.51.self_attn.o_proj.weight": "model-00008-of-00009.safetensors",
+        "model.layers.51.self_attn.q_proj.weight": "model-00008-of-00009.safetensors",
+        "model.layers.51.self_attn.v_proj.weight": "model-00008-of-00009.safetensors",
+        "model.layers.52.input_layernorm.weight": "model-00009-of-00009.safetensors",
+        "model.layers.52.mlp.down_proj.weight": "model-00009-of-00009.safetensors",
+        "model.layers.52.mlp.gate_proj.weight": "model-00009-of-00009.safetensors",
+        "model.layers.52.mlp.up_proj.weight": "model-00009-of-00009.safetensors",
+        "model.layers.52.post_attention_layernorm.weight": "model-00009-of-00009.safetensors",
+        "model.layers.52.self_attn.k_proj.weight": "model-00009-of-00009.safetensors",
+        "model.layers.52.self_attn.o_proj.weight": "model-00009-of-00009.safetensors",
+        "model.layers.52.self_attn.q_proj.weight": "model-00008-of-00009.safetensors",
+        "model.layers.52.self_attn.v_proj.weight": "model-00009-of-00009.safetensors",
+        "model.layers.53.input_layernorm.weight": "model-00009-of-00009.safetensors",
+        "model.layers.53.mlp.down_proj.weight": "model-00009-of-00009.safetensors",
+        "model.layers.53.mlp.gate_proj.weight": "model-00009-of-00009.safetensors",
+        "model.layers.53.mlp.up_proj.weight": "model-00009-of-00009.safetensors",
+        "model.layers.53.post_attention_layernorm.weight": "model-00009-of-00009.safetensors",
+        "model.layers.53.self_attn.k_proj.weight": "model-00009-of-00009.safetensors",
+        "model.layers.53.self_attn.o_proj.weight": "model-00009-of-00009.safetensors",
+        "model.layers.53.self_attn.q_proj.weight": "model-00009-of-00009.safetensors",
+        "model.layers.53.self_attn.v_proj.weight": "model-00009-of-00009.safetensors",
+        "model.layers.6.input_layernorm.weight": "model-00002-of-00009.safetensors",
+        "model.layers.6.mlp.down_proj.weight": "model-00002-of-00009.safetensors",
+        "model.layers.6.mlp.gate_proj.weight": "model-00002-of-00009.safetensors",
+        "model.layers.6.mlp.up_proj.weight": "model-00002-of-00009.safetensors",
+        "model.layers.6.post_attention_layernorm.weight": "model-00002-of-00009.safetensors",
+        "model.layers.6.self_attn.k_proj.weight": "model-00002-of-00009.safetensors",
+        "model.layers.6.self_attn.o_proj.weight": "model-00002-of-00009.safetensors",
+        "model.layers.6.self_attn.q_proj.weight": "model-00002-of-00009.safetensors",
+        "model.layers.6.self_attn.v_proj.weight": "model-00002-of-00009.safetensors",
+        "model.layers.7.input_layernorm.weight": "model-00002-of-00009.safetensors",
+        "model.layers.7.mlp.down_proj.weight": "model-00002-of-00009.safetensors",
+        "model.layers.7.mlp.gate_proj.weight": "model-00002-of-00009.safetensors",
+        "model.layers.7.mlp.up_proj.weight": "model-00002-of-00009.safetensors",
+        "model.layers.7.post_attention_layernorm.weight": "model-00002-of-00009.safetensors",
+        "model.layers.7.self_attn.k_proj.weight": "model-00002-of-00009.safetensors",
+        "model.layers.7.self_attn.o_proj.weight": "model-00002-of-00009.safetensors",
+        "model.layers.7.self_attn.q_proj.weight": "model-00002-of-00009.safetensors",
+        "model.layers.7.self_attn.v_proj.weight": "model-00002-of-00009.safetensors",
+        "model.layers.8.input_layernorm.weight": "model-00002-of-00009.safetensors",
+        "model.layers.8.mlp.down_proj.weight": "model-00002-of-00009.safetensors",
+        "model.layers.8.mlp.gate_proj.weight": "model-00002-of-00009.safetensors",
+        "model.layers.8.mlp.up_proj.weight": "model-00002-of-00009.safetensors",
+        "model.layers.8.post_attention_layernorm.weight": "model-00002-of-00009.safetensors",
+        "model.layers.8.self_attn.k_proj.weight": "model-00002-of-00009.safetensors",
+        "model.layers.8.self_attn.o_proj.weight": "model-00002-of-00009.safetensors",
+        "model.layers.8.self_attn.q_proj.weight": "model-00002-of-00009.safetensors",
+        "model.layers.8.self_attn.v_proj.weight": "model-00002-of-00009.safetensors",
+        "model.layers.9.input_layernorm.weight": "model-00002-of-00009.safetensors",
+        "model.layers.9.mlp.down_proj.weight": "model-00002-of-00009.safetensors",
+        "model.layers.9.mlp.gate_proj.weight": "model-00002-of-00009.safetensors",
+        "model.layers.9.mlp.up_proj.weight": "model-00002-of-00009.safetensors",
+        "model.layers.9.post_attention_layernorm.weight": "model-00002-of-00009.safetensors",
+        "model.layers.9.self_attn.k_proj.weight": "model-00002-of-00009.safetensors",
+        "model.layers.9.self_attn.o_proj.weight": "model-00002-of-00009.safetensors",
+        "model.layers.9.self_attn.q_proj.weight": "model-00002-of-00009.safetensors",
+        "model.layers.9.self_attn.v_proj.weight": "model-00002-of-00009.safetensors",
+        "model.norm.weight": "model-00009-of-00009.safetensors"
+    }
+}

tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f236d4eaa17b1b3f4f2aff62301eaa163a4176a74509874bd5514e8fb93ff32a
+size 15783361

tokenizer_config.json

@@ -0,0 +1,16 @@
+{
+  "add_prefix_space": true,
+  "backend": "tokenizers",
+  "bos_token": "<s>",
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "</s>",
+  "is_local": true,
+  "local_files_only": false,
+  "model_max_length": 1000000000000000019884624838656,
+  "pad_token": null,
+  "sp_model_kwargs": {},
+  "spaces_between_special_tokens": false,
+  "tokenizer_class": "LlamaTokenizer",
+  "unk_token": "<unk>",
+  "use_default_system_prompt": false
+}