Rahspide/MNN-Pocket-0.3B-RU-Chat

Status: v1 released. Instruction-tuned on 50 M tokens (18.8 M Russian + 6 M English replay) on one Kaggle T4 in ~6.5 h. Weights and GGUF builds are in this repository.

Russian instruction-following assistant, built by SFT on top of Rahspide/MNN-Pocket-0.3B-RU. It answers instead of continuing; that dialogue shape is what 50 M tokens of SFT bought.

Two models, two jobs

	Sister — MNN-Pocket-0.3B-RU	This model — MNN-Pocket-0.3B-RU-Chat
Job	continue text	follow instructions
Input	a prefix, often mid-sentence	a system + user message
Output	the plausible continuation	an answer
Trained with	continued pretraining on 1.2 B tokens of raw text	instruction tuning on top of that model
Choose it for	keyboard suggestions, autocomplete, prefix completion	chat, Q&A, short rewrites
Do not choose it for	assistant behaviour: it will continue your question instead of answering it	autocomplete: it will answer instead of continuing

The two tasks reward opposite behaviour. Trained on both jobs, one checkpoint does each badly: it answers when it should continue, and continues when it should answer.

Why start from the Russian continuation model

The base MNN-Pocket-0.3B is bilingual zh/en and has essentially no Russian; a direct SFT of the official instruct model would fight both problems at once (new language and new format). Starting from the continuation model means Russian is already paid for, and SFT only has to teach the dialogue shape.

Recipe (as run)

• Start point: Rahspide/MNN-Pocket-0.3B-RU (step 36,621).
• Chat template: the model's own ChatML (<|im_start|> / <|im_end|>), untouched.
• Loss only on assistant tokens; system and user turns are masked context.
• Multi-turn dialogue kept as one example; packing with explicit boundaries.
• One run at 50 M tokens (1,525 optimizer steps, 32 k tokens each), checkpoints every 300 steps.
• 85 / 15 Russian-to-English sampling per micro-batch; the English share keeps bilingual instruction following.
• lr 1e-4, cosine, 5% warmup, full fine-tuning, fp16 autocast on T4 with fp32 master weights.
• ~6.5 h on one Kaggle T4.

Data (as used)

Dataset	Role	Licence
IlyaGusev/saiga_scored	core Russian instructions, 18.8 M tokens / 23,518 dialogues	see dataset card
HuggingFaceH4/ultrachat_200k	English replay, 6.0 M tokens / 5,181 dialogues	MIT

Filters applied to saiga_scored: Russian only, dropped rows flagged is_bad_by_regex, 80–6,000 characters per dialogue, 2–8 turns. Roles bot / gpt are normalised to assistant; empty turns are dropped.

Quality spot-check

Greedy generation, Q4_K_M, CPU:

Prompt	Answer
«Объясни в двух предложениях, что такое инфляция.»	«Инфляция — это процесс роста цен на товары и услуги в стране или за ее пределами. Этот процесс может привести к у…»
«Напиши короткое поздравление с днём рождения коллеге.»	«Дорогой [Имя], С днём рождения! Желаю, чтобы ваша работа была не только эффективной, но и приятной для глаз…»
«Привет! Как дела?»	repeats the question — a known weak spot of 0.3 B on very short turns

The model answers instead of continuing the prompt. That is all this table proves.

Builds

Build	Size	Download	Note
Q8_0	310 MB	MNN-Pocket-0.3B-RU-Chat-Q8_0.gguf	near-lossless
Q4_K_M	223 MB	MNN-Pocket-0.3B-RU-Chat-Q4_K_M.gguf	phone size
MNN INT4	165 MB	mnn/MNN-Pocket-0.3B-RU-Chat-INT4	Android, MNN runtime

llama-cli -hf Rahspide/MNN-Pocket-0.3B-RU-Chat:Q4_K_M --jinja -p "Объясни, что такое инфляция" -n 60

The MNN build runs on MNN's runtime (llm_demo, built with -DMNN_BUILD_LLM=ON). Its llm_config.json keeps the ChatML template, so it answers out of the box:

cd mnn/MNN-Pocket-0.3B-RU-Chat-INT4 && llm_demo config.json prompt.txt

165 MB on disk, ~497 tok/s prefill and ~80 tok/s decode on 12 CPU threads. The answer is coherent Russian; the INT4 weights cost some precision against Q8_0, as expected.

~123 tok/s generation on Q4_K_M, ~102 tok/s on Q8_0 (16-thread x86 CPU, no GPU). Apply the model's own ChatML template (--jinja in llama.cpp, apply_chat_template in transformers). Without it the model continues text instead of answering and behaves like its sister.

Reproduce

# chat smoke test through the model's own template
python bench/chat_smoke.py --model Rahspide/MNN-Pocket-0.3B-RU-Chat --cpu

# throughput of the quantised build
LLAMA_CLI=/path/to/llama-cli bash bench/bench_cpu.sh gguf/MNN-Pocket-0.3B-RU-Chat-Q4_K_M.gguf --chat

bench/chat_smoke.py applies the chat template from tokenizer_config.json, generates greedily and asserts that the answer does not echo the prompt, which is a cheap check that the template and the checkpoint still match. bench/bench_cpu.sh --chat wraps the prompt in ChatML before timing, so the numbers are comparable with the table above.

Evaluation

60 held-out Russian prompts from the saiga_scored validation slice, greedy decoding, the model's own template:

Automatic check	Result
answers in Russian (>50% Cyrillic)	57 / 60
mean Cyrillic share of letters	0.93
empty answers	0
echoes the prompt	2
mean answer length, words	26.1
distinct-2 (higher = less repetition)	0.90

The sheet with all 60 prompts and answers is in review/blind_sheet.md.

A comparison against taobao-mnn/MNN-Pocket-0.3B was attempted on the same prompts and dropped. That checkpoint degenerates on Russian input, repeating the prompt in bold markdown fragments:

Q: Как стать популярным в яндекс дзен?
A: Конечный откорт: **"Популярный в яндекс дзен"** — это **не** в яндекс (в яндекс) — **в яндекс** — **в яндекс** — …

So it is not a baseline.

Limitations

• 0.36 B parameters: short answers, weak facts, no reliable multi-step reasoning.
• Russian SFT data is mostly machine-translated or model-generated; style is flatter than human chat.
• The sister's tokenizer cost applies here too: about 3.3 tokens per Russian word, so ~3× more decode steps per Russian sentence than for English. The vocabulary is byte-level, so rare and uppercase Cyrillic letters can end up split across tokens; a streaming decoder has to buffer partial UTF-8 sequences.
• Not a coding model, not a tool-calling model, no long-context dialogue.
• English answers are thinner than Russian ones. Replay keeps the ability alive; it does not polish it.

Releases

Date	Version	What changed
2026-11-10	v1.0	Assistant released: 50 M-token SFT (18.8 M ru + 6 M en), bf16 weights, GGUF Q8_0 / Q4_K_M, ChatML smoke test passing
2026-11-11	v1.1	MNN INT4 build for Android: 165 MB, ChatML template included, ~80 tok/s decode
2026-11-10	v1.0 (sister)	Continuation checkpoint released: 1.2 B-token CPT, same base

Licence and attribution

Weights: Apache-2.0, inherited through MNN-Pocket-0.3B-RU from taobao-mnn/MNN-Pocket-0.3B-Base.

@misc{mnn_pocket_03b,
  title  = {MNN-Pocket-0.3B},
  author = {Taobao MNN team},
  year   = {2026},
  url    = {https://huggingface.co/taobao-mnn/MNN-Pocket-0.3B}
}