Rahspide/MNN-Pocket-0.3B-RU

Status: v1 released. 1.2 B tokens, one epoch, weights in this repository, GGUF in gguf/. Training log, checkpoints and the evaluation script live in the private repo Rahspide/mnn-pocket-ru-run.

A continued-pretraining run of taobao-mnn/MNN-Pocket-0.3B-Base that teaches it Russian while keeping English and Chinese readable. The keyboard is the use case that set the size and latency budget.

It is trained as a plain causal LM on raw text, so it continues a typed prefix instead of answering it. For dialogue there is a separate checkpoint — Rahspide/MNN-Pocket-0.3B-RU-Chat.

Two models, two jobs

	This model — MNN-Pocket-0.3B-RU	Sister — MNN-Pocket-0.3B-RU-Chat
Job	continue text	follow instructions
Input	a prefix, often mid-sentence	a system + user message
Output	the plausible continuation	an answer
Trained with	continued pretraining, 1.2 B tokens of raw text	SFT on top of this model, 50 M tokens
Choose it for	keyboard suggestions, autocomplete, prefix completion	chat, Q&A, short rewrites
Do not choose it for	assistant behaviour: it continues your question instead of answering	autocomplete: it answers instead of continuing

The two tasks reward opposite behaviour. Trained on both jobs, one checkpoint does each badly: it answers when it should continue, and continues when it should answer.

Specs

Base	taobao-mnn/MNN-Pocket-0.3B-Base (Qwen3 architecture, 303 M non-embedding / 361 M total)
Tokenizer	65,536 tokens, byte-level BPE, `<
Training tokens	1.2 B — 0.9 B ru + 0.204 B en + 0.096 B zh
Method	full fine-tune, 1 epoch, no LoRA
Steps	36,621 × 32,768 tokens
Hardware	one Colab T4 16 GB, 4.8 k tok/s, ~69 h
Precision	fp32 master weights, fp16 autocast (T4 has no usable bf16)

Results

100 k held-out tokens per language, identical settings for both checkpoints:

Lang	ppl base → v1	bits/char base → v1	top-1 base → v1	top-5 base → v1
ru	10.42 → 5.05	1.625 → 1.123 (−31%)	46.2% → 61.0%	69.3% → 80.6%
en	21.85 → 41.16	1.016 → 1.225 (+21%)	39.9% → 35.5%	64.1% → 58.7%
zh	23.95 → 51.86	3.274 → 4.071 (+24%)	41.9% → 37.7%	62.0% → 55.6%

The published weights were re-downloaded and re-evaluated independently (CPU, fp32, 20 k tokens per language) and reproduce the same ordering: ru 1.022 against base 1.519, en 1.337 against 1.124, zh 4.053 against 3.393.

Russian landed past the target (1.123 against a 1.40 gate). English and Chinese regressed 21–24% instead of the ≤10% the recipe aimed for. a 25% replay mix is too thin at 0.36 B when the target language is tokenised this badly. A second pass with a 50/25/25 mix at lr 5e-5 was started, got through ~50 M of a planned 200 M tokens, and changed nothing measurable on English (1.337 → 1.337 on the same eval), so it is not published. Treat v1 as Russian-first, with working but measurably weaker English and Chinese.

Why the tokenizer shapes the recipe

The base vocabulary is a distilled 65 k set with almost no Cyrillic merges, and it is byte-level: the vocabulary stores UTF-8 bytes remapped to latin-1, so convert_ids_to_tokens prints С where the text says С. That is a display artifact; decode returns the word. The count is what matters:

Word	Characters	Tokens	Pieces
Спасибо	7	6	С · п · ас · и · б · о
привет	6	3	пр · ив · ет
покупать	8	4	п · ок · уп · ать
Ёжик	4	4	Ё lands in two separate byte tokens

Because the vocabulary is byte-level, a token can end in the middle of a character: in Ёжик and ЩУКА the uppercase letters are cut into single bytes. Decoding the whole sequence is still exact, but a streaming client must hold an incomplete UTF-8 sequence back instead of printing it. llama.cpp and transformers buffer it for you; a hand-rolled decoder has to do the same or the output shows replacement characters.

Tokenizer	RU chars/token	RU tokens per word	EN chars/token
MNN-Pocket / Qwen3-distilled (65,536)	1.75–1.85 (2.08 on dialogue text)	≈3.3	3.78
+8,192 Cyrillic merges (73,728)	3.10	≈1.8	—
+16,384 Cyrillic merges (81,920)	3.39	≈1.6	—
ruGPT3-small (50 k, Russian BPE)	4.2	≈1.3	—

Consequences that shaped the plan:

• the same 1.2 B-token budget buys 2.2 B Russian characters (340 M words), roughly 2.2× less text than a Russian-native tokenizer would fit;
• on a phone one suggested Russian word costs ~3 decode steps instead of ~1, which matters for a keyboard latency budget;
• vocabulary surgery was measured but kept out of v1, so v1 stays tokenizer-compatible with the base and drops into the existing MNN / GGUF toolchain. An extended-vocabulary artefact (model_ext, 73,728 rows, embeddings initialised as the mean of their child tokens) exists in the private run repo but is not part of this release.

Recipe

• Full fine-tune from MNN-Pocket-0.3B-Base, fp32 master weights with fp16 autocast.
• Sequence length 1024, random windows over a packed uint16 token stream, <|endoftext|> between documents.
• 32,768 tokens per optimizer step (micro-batch 4 × 1024, gradient accumulation 8).
• AdamW, β = (0.9, 0.95), weight decay 0.1, gradient clip 1.0, lr 2e-4 with 1% linear warmup and cosine decay.
• One epoch over 1.2 B tokens = 36,621 steps.
• Checkpoint every 500 steps to a private HF repo; training auto-resumes from the last checkpoint after a session restart.

Data

All sources are public Hugging Face datasets. * marks the validation split: the first 100 k tokens of each stream are held out before training starts, so validation text is never seen.

Language	Dataset	Config	Tokens	Share
ru	HuggingFaceFW/fineweb-2	rus_Cyrl	0.90 B (100 k held out*)	75%
en	HuggingFaceFW/fineweb	sample-100BT	0.204 B (100 k held out*)	17%
zh	HuggingFaceFW/fineweb-2	cmn_Hani	0.096 B (100 k held out*)	8%

Licences: FineWeb / FineWeb-2 are ODC-BY 1.0. No scraped Telegram dumps or other unlicensed chat corpora were used, so the resulting weights stay publishable.

Builds

Build	Size	Download	Note
Q8_0	310 MB	MNN-Pocket-0.3B-RU-Q8_0.gguf	near-lossless, for desktop
Q4_K_M	223 MB	MNN-Pocket-0.3B-RU-Q4_K_M.gguf	the size to ship on a phone
MNN INT4	165 MB	mnn/MNN-Pocket-0.3B-RU-INT4	MNN runtime, Android

Both files sit under gguf/ with the quant tag in the name, which is what llama.cpp matches on:

llama-cli -hf Rahspide/MNN-Pocket-0.3B-RU:Q4_K_M -p "Сегодня я" -n 40

Converted with llama.cpp convert_hf_to_gguf.py (f16) and quantised with llama-quantize. Measured on a 16-thread x86 CPU, no GPU involved:

Build	Prompt processing	Generation
Q4_K_M	~410 tok/s	~130 tok/s
Q8_0	~525 tok/s	~99 tok/s

Phone figures will be several times lower; these are desktop-CPU numbers.

The MNN build runs through MNN's own runtime. Build llm_demo from the MNN repository with -DMNN_BUILD_LLM=ON, then point it at the folder:

cd mnn/MNN-Pocket-0.3B-RU-INT4 && llm_demo config.json prompt.txt

165 MB on disk, ~354 tok/s prefill and ~77 tok/s decode on 12 CPU threads here. llm_config.json carries no chat template on purpose: this checkpoint continues the prompt, so the runtime should not wrap it in a dialogue format.

Two conversion quirks, both already fixed in this repository's tokenizer_config.json.

• the base pre-tokenizer does not hash-match in current llama.cpp, so the Qwen pre-tokenizer has to be pinned by hand (qwen2);
• extra_special_tokens must not be a list, or transformers v4 refuses to load the tokenizer.

Reproduce

Everything below runs on CPU. The numbers in this card came from these paths.

# bits/char, ppl and top-k against the base, 20k tokens per language
python bench/eval_bpc.py --model Rahspide/MNN-Pocket-0.3B-RU --base taobao-mnn/MNN-Pocket-0.3B-Base \
    --dataset HuggingFaceFW/fineweb-2 --config rus_Cyrl --tokens 20000

# throughput and a smoke generation on the quantised build
LLAMA_CLI=/path/to/llama-cli bash bench/bench_cpu.sh gguf/MNN-Pocket-0.3B-RU-Q4_K_M.gguf

bench/eval_bpc.py reports the same three metrics as the tables above (ppl, bits/char, top-1/top-5) and exits non-zero when the model loses more than a given --max-bpc-regression against the base, so it can run as a regression gate in CI. bench/bench_cpu.sh prints prompt and generation tok/s and checks that generation stops at <|endoftext|>.

Intended use

• On-device next-word and short-phrase suggestion in a keyboard or IME, Russian first, English and Chinese from the same checkpoint (a keyboard can pick the language from the current word or the layout).
• Text completion, prefix continuation and autocomplete for Russian anywhere a ~220 MB quantised model has to run locally: chat composers, note apps, search fields, terminal prompts.
• Any pipeline that needs a small causal LM fluent in Russian that can still read English and Chinese, for example tagging, short rewriting suggestions, or generating Russian seed text.

Not intended for: instruction following, factual question answering, code, long-form reasoning, or translation. This is a 0.36 B model trained for 1.2 B tokens; its knowledge is narrow by construction. The language mix is a deliberate trade: Russian is the target, English and Chinese are kept from degrading, neither is improved.

Limitations

• Russian is tokenised near character level (see above): slower decoding and a larger effective cost per Russian word than the same model pays for English.
• Token boundaries do not follow character boundaries. Rare and uppercase Cyrillic letters are split into single bytes, so a streaming decoder must buffer partial UTF-8 sequences; llama.cpp and transformers already do this.
• Web text only. Register is mostly written/informal blog Russian; no dialogue corpora, no post-training.
• Capacity is 0.36 B. Expect weak facts, weak arithmetic, and drift on long generations.
• English and Chinese regressed 21–24% against the base. The 25% replay mix limits the damage; it does not remove it.
• The ChatML template from the base is inherited but untrained here. Do not use this checkpoint in chat mode.

Releases

Date	Version	What changed
2026-11-10	v1.0	1.2 B-token continued pretraining released: bf16 weights, GGUF Q8_0 and Q4_K_M, measured ru/en/zh bits-per-char
2026-11-11	v1.1	MNN INT4 build for the MNN runtime: 165 MB, ~77 tok/s decode on 12 CPU threads
2026-11-10	v2.0 (sister)	Assistant checkpoint released: 50 M-token SFT on top of v1, weights and GGUF

Licence and attribution

Weights: Apache-2.0, inherited from the base model. Data: ODC-BY 1.0 (FineWeb, FineWeb-2). If you use this model, credit the base release:

@misc{mnn_pocket_03b,
  title  = {MNN-Pocket-0.3B},
  author = {Taobao MNN team},
  year   = {2026},
  url    = {https://huggingface.co/taobao-mnn/MNN-Pocket-0.3B}
}