EXLLM-0.005B-Instruct

EXLLM-0.005B-Instruct is a 0.005377824B-parameter Japanese instruction model for local inference on low-memory embedded hardware. The primary deployment target is the CASIO EX-word XD-B4800. A PyTorch reference runtime is provided for host-side inference and evaluation. This repository also includes a separately trained 0.005441184B-parameter Llama-compatible GGUF companion for LM Studio and llama.cpp.

Here, LLM stands for Little Language Model.

The embedded checkpoint uses a custom decoder-only Transformer and a hybrid character/UTF-8-byte tokenizer; it is not directly compatible with standard transformers auto classes or GGUF runtimes. Use EXLLM-0.005B-LMStudio-F16.gguf for LM Studio or llama.cpp. The GGUF is a companion model trained on the EXLLM project data, not a numerical conversion of the embedded checkpoint.

日本語版の説明はこちらです。

Highlights

• 0.005377824B parameters
• 0.000868M-token vocabulary with UTF-8 byte fallback
• 0.000128M-token context length
• 0.021526737 GB fp32 checkpoint
• 0.005443105 GB int8 and EXQ12 deployment artifacts
• Reference inference at 239.082 token/s on an RTX 3060
• Integer inference at 0.52–0.55 token/s on a 0.024184 GHz SH-4A-class CPU
• Apache-2.0 weights and reference code
• Ready-to-run 0.010915168 GB F16 GGUF companion for LM Studio

Model Details

Field	Value
Model type	Instruction-tuned decoder-only Transformer
Parameters	0.005377824B
Language	Japanese
Context length	0.000128M tokens
Vocabulary	0.000868M tokens
Layers	6
Hidden size	288
Attention heads	9
Head dimension	32
FFN hidden size	896
Activation	ReLU
Normalization	Pre-norm RMSNorm, epsilon 1e-5
Position encoding	Learned absolute position embedding
Output head	Weight-tied token embedding
Tokenizer	Frequent Japanese characters + UTF-8 byte fallback, NFC normalization
License	Apache-2.0

The parameter count reports unique trainable parameters. The tied language-model head and token embedding refer to the same parameter tensor.

Training

EXLLM was trained only on project-generated Japanese instruction/response data. It does not use an external pretrained checkpoint or imported public text corpus. The EXLLM project originates from random initialization. The 0.005B release was expanded from an earlier in-project EXLLM checkpoint by transplanting dimension-compatible parameters, followed by staged training and corrective fine-tuning. No external pretrained checkpoint was used.

Item	Value
Published staged data	0.086408M JSONL records
External pretrained base	None
Optimizer	AdamW
Base learning rate	5e-4
Adam betas	0.9, 0.95
Base weight decay	0.03
Gradient clipping	1.0
LR schedule	Linear warm-up + cosine decay
Checkpoint global step	1,810
Final corrective stage	100 steps at 6e-6
Final corrective loss	0.03875 → 0.02420

The published staged files include overlapping base, robustness, recovery, and corrective sets; 0.086408M is a file-record total, not a deduplicated example count. Exact end-to-end training time and training-device telemetry were not retained in the release checkpoint and are therefore not reported.

Data scope and weight lineage are documented in DATA_PROVENANCE.md. The machine-readable release metadata is available in training/release-5m.json.

Inference Performance

All results use greedy decoding. TTFT means time to first generated token. Decode throughput excludes the first generated token.

Runtime	Compute	Model format	Memory	Load	TTFT	Decode throughput
PyTorch 2.11.0+cu130	RTX 3060, 12 GB VRAM	fp32	0.0307584 GB peak allocated	0.4645 s	0.003990 s	239.082 token/s
EX-word integer runtime	SH-4A-class, 1C/1T, 0.024184 GHz	int8 weights / Q12 activations	0.00786432 GB verified contiguous application heap	device storage	23.711 s	0.52–0.55 token/s

RTX 3060 methodology

• Driver 580.178.04; Python 3.12.13; CUDA 13.0.
• 40 warm-up inferences followed by 120 measured inferences.
• Median TTFT: 0.003990 s; observed range: 0.003913–0.004072 s.
• Median total generation time: 0.079589 s.
• Tukey 1.5 IQR filtering removed 0/120 observations for TTFT, total time, and decode throughput.
• The first cold warm-up inference recorded 0.193056 s TTFT.
• Raw per-run results are published in benchmarks/rtx3060-cuda-robust-20260925.json.

EX-word methodology

• CASIO EX-word XD-B4800 / DATAPLUS 6.
• SH7305-family SH-4A, 1 core / 1 thread; measured instruction clock 0.024184 GHz.
• Five prompts, Thinking disabled, model stored in internal storage.
• Application UI rendering and UTF-8 display handling are included.
• Median TTFT: 23.711 s; observed range: 12.705–28.440 s.
• Median total generation time: 54.145 s.

The two runtimes use the same architecture and training lineage but different numerical representations. The results are deployment measurements, not a direct hardware-compute comparison.

Evaluation

The release was evaluated with a project-specific regression suite after dequantizing the EXLLM8 artifact into the reference runtime.

Suite	Result
Semantic behavior	35 / 35
Deterministic calculator routing	300 / 300
Fuzz prompts	200 / 200
Sampling checks	48 / 48
Release gate	PASS

These are internal regression tests for the intended application scope. They are not standard language-model benchmarks and should not be interpreted as evidence of broad knowledge, general reasoning, or benchmark comparability with larger models.

Usage

LM Studio / llama.cpp

In LM Studio, search for ToTo-40417/EXLLM, download EXLLM-0.005B-LMStudio-F16.gguf, and start a chat. The required single-turn chat template is embedded in the GGUF. Set the loaded context length to 512 tokens and use greedy decoding or a low temperature; the model is intentionally tiny and is intended for short Japanese prompts. Training sequences were limited to 128 tokens; the larger runtime window reserves space for LM Studio's chat wrapper and short history.

With llama.cpp:

llama-cli -m EXLLM-0.005B-LMStudio-F16.gguf \
  -p "RAMとは何ですか？" -n 48 --temp 0 --ctx-size 512 --single-turn

The GGUF companion uses a Llama-compatible decoder, a 1,024-piece SentencePiece tokenizer with byte fallback, 6 layers, hidden size 288, intermediate size 608, 9 attention/KV heads, 128-token training sequences, and a 512-token runtime context. It was trained separately for five epochs (3,325 optimizer steps) on 85,005 project-data records; best validation loss was 0.0390303. Exact metadata is in lmstudio/training_manifest.json.

Original EXLLM reference runtime

git clone https://huggingface.co/ToTo-40417/EXLLM
cd EXLLM
python -m venv .venv
. .venv/bin/activate
pip install -r requirements.txt
python chat.py こんにちは

from src.loader import load_release_model
from src.runtime import answer

model, tokenizer = load_release_model()
print(answer(model, tokenizer, "RAMとは何ですか？", temperature=0.0))

Generation Examples

Prompt	Greedy output
こんにちは	こんにちは。ことばの意味や簡単な質問ならどうぞ。
あなたは何というモデルですか？	私はEXLLMです。
RAMとは何ですか？	RAMは、処理中のデータやプログラムを一時的に保持する記憶装置です。
オフラインとは何ですか？	ウェブ検索はできません。

Release Artifacts

Artifact	Size	SHA-256	Purpose
weights/EXLLM-v1.1-5m-release3.pt	21,526,737 bytes (20.53 MiB; 0.021526737 GB)	48ad00c6640689595a253fc949cad83f3000e14cccdf69275fcf245e85be3ceb	fp32 PyTorch checkpoint
weights/EXLLM-v1.1-5m-int8.bin	5,443,105 bytes (5.19 MiB; 0.005443105 GB)	3b2bf2d9f103cba714bc98976709c5ad34860c1ff55fed2fce91fc95afb5b34a	EXLLM8 interchange artifact
weights/model.q12	5,443,105 bytes (5.19 MiB; 0.005443105 GB)	d64037fde791e5c0e48101bc1a8ab366a3287f1f36b4464879e36495ea7e5a53	EX-word EXQ12 artifact
EXLLM-0.005B-LMStudio-F16.gguf	10,915,168 bytes (10.41 MiB; 0.010915168 GB)	0e3c519d4a044fac45be84dc607eb8a43de4a7c60083983bf246c5b9c8a43472	Separately trained Llama-compatible F16 companion for LM Studio / llama.cpp

EXLLM8 and EXQ12 have the same file size but are not interchangeable formats.

Intended Use

• Ultra-small language-model research
• Embedded and low-memory inference experiments
• Short Japanese UI responses and narrow-domain instruction handling
• Integer-runtime and quantization experiments
• Educational analysis of language models below 0.01B parameters

Limitations

• The model is not a general-purpose assistant.
• The original embedded checkpoint and the GGUF training sequences use 0.000128M tokens. The GGUF advertises a 0.000512M-token runtime window for LM Studio framing and short history; quality beyond the trained 128-token range is not guaranteed.
• Knowledge coverage is restricted to the project-generated training scope.
• Unseen concepts, long instructions, translation, and multi-step reasoning are unreliable.
• No RLHF, DPO, tool use, retrieval, web access, or current-information source is included.
• The deterministic calculator is runtime routing, not learned arithmetic capability.
• The model must not be used as an authority for medical, legal, financial, safety-critical, or other high-stakes decisions.
• Standard Hugging Face Inference Providers cannot serve the original embedded checkpoint without custom architecture support. The bundled GGUF companion is intended for local LM Studio / llama.cpp execution.

Links

• Source and training code: ToTo-40417/exllm
• EX-word runtime: ToTo-40417/exllm-exword
• Hardware probe: ToTo-40417/exword-hardware-dump

日本語

EXLLM-0.005B-Instructは、低メモリ組込み機器でのローカル推論を目的とした、0.005377824Bパラメータの日本語instruction modelです。主な実装対象はCASIO EX-word XD-B4800です。同じプロジェクトデータから別途学習した、LM Studio / llama.cpp向けの0.005441184BパラメータGGUF companionも収録しています。

LLMとは、Little Language Modelの略です。

電子辞書用checkpointは独自のdecoder-only Transformerとtokenizerを使用するため、GGUFに直接変換したものではありません。LM StudioではToTo-40417/EXLLMを検索し、EXLLM-0.005B-LMStudio-F16.ggufを選択してください。チャット用templateはGGUFに内蔵済みで、LM Studioのロード時contextは512 tokensに設定します。学習時の系列長は128 tokensであり、追加領域はチャット制御情報と短い履歴のための余白です。このGGUFはプロジェクトデータで別途学習したPC用companionであり、電子辞書版と同一重みではありません。

RTX 3060でのfp32参照実行は、40回のwarm-up後に120回測定し、TTFT中央値0.003990秒、生成速度239.082 token/sでした。EX-word整数runtimeでは、0.024184 GHzのSH-4A-class CPU上でTTFT中央値23.711秒、生成速度0.52–0.55 token/sでした。

用途は、0.01B未満の言語モデル、組込み推論、量子化、整数runtimeの研究です。汎用的な質問応答、長文処理、専門判断、最新情報の取得を目的としたモデルではありません。

コード、重み、付属データはApache License 2.0で公開します。

Citation

@software{toto_exllm_2026,
  author  = {ToTo},
  title   = {EXLLM-0.005B-Instruct},
  year    = {2026},
  url     = {https://huggingface.co/ToTo-40417/EXLLM}
}