Add LM Studio GGUF companion

.gitattributes

@@ -32,5 +32,6 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.xz filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
+*.gguf filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
 weights/model.q12 filter=lfs diff=lfs merge=lfs -text

EXLLM-0.005B-LMStudio-F16.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6f649e74fd006c4c487a141307a7598531a1ff1ce8ba5e45d45f9c590f399759
+size 10915168

README.md

@@ -2,15 +2,15 @@
 license: apache-2.0
 language: [ja]
 pipeline_tag: text-generation
-library_name: pytorch
-tags: [japanese, instruction-tuning, tiny-language-model, edge-ai, embedded-ai, ex-word, custom-code]
+library_name: gguf
+tags: [japanese, instruction-tuning, tiny-language-model, edge-ai, embedded-ai, ex-word, gguf, llama-cpp, lm-studio, custom-code]
 ---
 
 # EXLLM-0.005B-Instruct
 
-EXLLM-0.005B-Instruct is a **0.005377824B-parameter** Japanese instruction model for local inference on low-memory embedded hardware. The primary deployment target is the CASIO EX-word XD-B4800. A PyTorch reference runtime is provided for host-side inference and evaluation.
+EXLLM-0.005B-Instruct is a **0.005377824B-parameter** Japanese instruction model for local inference on low-memory embedded hardware. The primary deployment target is the CASIO EX-word XD-B4800. A PyTorch reference runtime is provided for host-side inference and evaluation. This repository also includes a separately trained **0.005441184B-parameter** Llama-compatible GGUF companion for LM Studio and llama.cpp.
 
-The model uses a custom decoder-only Transformer and a hybrid character/UTF-8-byte tokenizer. It is not compatible with the standard `transformers` auto classes, GGUF, llama.cpp, or LM Studio without an architecture-specific implementation.
+The embedded checkpoint uses a custom decoder-only Transformer and a hybrid character/UTF-8-byte tokenizer; it is not directly compatible with standard `transformers` auto classes or GGUF runtimes. Use `EXLLM-0.005B-LMStudio-F16.gguf` for LM Studio or llama.cpp. The GGUF is a companion model trained on the EXLLM project data, not a numerical conversion of the embedded checkpoint.
 
 日本語版の説明は[こちら](#日本語)です。
 
@@ -24,6 +24,7 @@ The model uses a custom decoder-only Transformer and a hybrid character/UTF-8-by
 - Reference inference at **239.082 token/s** on an RTX 3060
 - Integer inference at **0.52–0.55 token/s** on a 0.024184 GHz SH-4A-class CPU
 - Apache-2.0 weights and reference code
+- Ready-to-run **0.010915168 GB** F16 GGUF companion for LM Studio
 
 ## Model Details
 
@@ -117,6 +118,21 @@ These are internal regression tests for the intended application scope. They are
 
 ## Usage
 
+### LM Studio / llama.cpp
+
+In LM Studio, search for `ToTo-40417/EXLLM`, download `EXLLM-0.005B-LMStudio-F16.gguf`, and start a chat. The required single-turn chat template is embedded in the GGUF. Keep the context length at **128 tokens** and use greedy decoding or a low temperature; the model is intentionally tiny and is intended for short Japanese prompts.
+
+With llama.cpp:
+
+```bash
+llama-cli -m EXLLM-0.005B-LMStudio-F16.gguf \
+  -p "RAMとは何ですか？" -n 48 --temp 0 --ctx-size 128 --single-turn
+```
+
+The GGUF companion uses a Llama-compatible decoder, a 1,024-piece SentencePiece tokenizer with byte fallback, 6 layers, hidden size 288, intermediate size 608, 9 attention/KV heads, and a 128-token context. It was trained separately for five epochs (3,325 optimizer steps) on 85,005 project-data records; best validation loss was 0.0390303. Exact metadata is in [`lmstudio/training_manifest.json`](lmstudio/training_manifest.json).
+
+### Original EXLLM reference runtime
+
 ```bash
 git clone https://huggingface.co/ToTo-40417/EXLLM
 cd EXLLM
@@ -150,6 +166,7 @@ print(answer(model, tokenizer, "RAMとは何ですか？", temperature=0.0))
 | `weights/EXLLM-v1.1-5m-release3.pt` | 21,526,737 bytes (20.53 MiB; 0.021526737 GB) | `48ad00c6640689595a253fc949cad83f3000e14cccdf69275fcf245e85be3ceb` | fp32 PyTorch checkpoint |
 | `weights/EXLLM-v1.1-5m-int8.bin` | 5,443,105 bytes (5.19 MiB; 0.005443105 GB) | `3b2bf2d9f103cba714bc98976709c5ad34860c1ff55fed2fce91fc95afb5b34a` | EXLLM8 interchange artifact |
 | `weights/model.q12` | 5,443,105 bytes (5.19 MiB; 0.005443105 GB) | `d64037fde791e5c0e48101bc1a8ab366a3287f1f36b4464879e36495ea7e5a53` | EX-word EXQ12 artifact |
+| `EXLLM-0.005B-LMStudio-F16.gguf` | 10,915,168 bytes (10.41 MiB; 0.010915168 GB) | `6f649e74fd006c4c487a141307a7598531a1ff1ce8ba5e45d45f9c590f399759` | Separately trained Llama-compatible F16 companion for LM Studio / llama.cpp |
 
 EXLLM8 and EXQ12 have the same file size but are not interchangeable formats.
 
@@ -170,7 +187,7 @@ EXLLM8 and EXQ12 have the same file size but are not interchangeable formats.
 - No RLHF, DPO, tool use, retrieval, web access, or current-information source is included.
 - The deterministic calculator is runtime routing, not learned arithmetic capability.
 - The model must not be used as an authority for medical, legal, financial, safety-critical, or other high-stakes decisions.
-- Standard Hugging Face Inference Providers cannot serve this checkpoint without custom architecture support.
+- Standard Hugging Face Inference Providers cannot serve the original embedded checkpoint without custom architecture support. The bundled GGUF companion is intended for local LM Studio / llama.cpp execution.
 
 ## Links
 
@@ -180,9 +197,9 @@ EXLLM8 and EXQ12 have the same file size but are not interchangeable formats.
 
 ## 日本語
 
-EXLLM-0.005B-Instructは、低メモリ組込み機器でのローカル推論を目的とした、**0.005377824Bパラメータ**の日本語instruction modelです。主な実装対象はCASIO EX-word XD-B4800です。
+EXLLM-0.005B-Instructは、低メモリ組込み機器でのローカル推論を目的とした、**0.005377824Bパラメータ**の日本語instruction modelです。主な実装対象はCASIO EX-word XD-B4800です。同じプロジェクトデータから別途学習した、LM Studio / llama.cpp向けの**0.005441184Bパラメータ**GGUF companionも収録しています。
 
-独自のdecoder-only Transformerと、頻出日本語文字・UTF-8 byte fallbackを組み合わせたtokenizerを使用します。標準`transformers`、GGUF、llama.cpp、LM Studioとの直接互換性はありません。
+電子辞書用checkpointは独自のdecoder-only Transformerとtokenizerを使用するため、GGUFに直接変換したものではありません。LM Studioでは`ToTo-40417/EXLLM`を検索し、`EXLLM-0.005B-LMStudio-F16.gguf`を選択してください。チャット用templateはGGUFに内蔵済みで、contextは128 tokensです。このGGUFはプロジェクトデータで別途学習したPC用companionであり、電子辞書版と同一重みではありません。
 
 RTX 3060でのfp32参照実行は、40回のwarm-up後に120回測定し、TTFT中央値0.003990秒、生成速度239.082 token/sでした。EX-word整数runtimeでは、0.024184 GHzのSH-4A-class CPU上でTTFT中央値23.711秒、生成速度0.52–0.55 token/sでした。
 

lmstudio/training_manifest.json

@@ -0,0 +1,48 @@
+{
+  "name": "EXLLM-0.005B-LMStudio-GGUF-companion",
+  "relationship": "separately trained Llama-compatible companion; not converted EX-word weights",
+  "parameters": 5441184,
+  "architecture": {
+    "family": "LlamaForCausalLM",
+    "layers": 6,
+    "hidden_size": 288,
+    "intermediate_size": 608,
+    "attention_heads": 9,
+    "kv_heads": 9,
+    "context_length": 128,
+    "vocabulary_size": 1024
+  },
+  "training_records_including_overlaps": 85005,
+  "validation_records": 1403,
+  "epochs": 5,
+  "steps": 3325,
+  "best_validation_loss": 0.03903034566482638,
+  "seed": 20260926,
+  "history": [
+    {
+      "epoch": 1,
+      "valid_loss": 0.08999436360746654,
+      "seconds": 35.0497190952301
+    },
+    {
+      "epoch": 2,
+      "valid_loss": 0.06198247724884875,
+      "seconds": 68.68868088722229
+    },
+    {
+      "epoch": 3,
+      "valid_loss": 0.047818813336203816,
+      "seconds": 102.3995361328125
+    },
+    {
+      "epoch": 4,
+      "valid_loss": 0.04088344847093563,
+      "seconds": 136.1156406402588
+    },
+    {
+      "epoch": 5,
+      "valid_loss": 0.03903034566482638,
+      "seconds": 169.76451516151428
+    }
+  ]
+}