How to use from
Docker Model Runner
docker model run hf.co/RusMiniLM/RusNanoLM-800M:F16
Quick Links

RusNanoLM-800M

English | ะ ัƒััะบะธะน


English Overview

RusNanoLM-800M is an open-weights 800M parameter bilingual (Russian / English) language model built on a hybrid Linear Attention architecture (Qwen3.5 with Gated Delta Rule layers). It is evaluated on MERA, the Russian language understanding benchmark suite: on RuMultiAR a 0.8B model outperforms Qwen2.5-7B and GigaChat-2-10B.

MERA Results (Pass@1, greedy, chat template)

Benchmark Tasks RusNanoLM-0.8B Qwen3.5-0.8B (baseline) ฮ”
SimpleAR 1000 83.2% 48.3% +34.9
RuMultiAR 1039 33.5% 8.1% +25.4 (ร—4.1)
MathLogicQA 680 38.2% 33.9% +4.4
MaMuRaMu 285 46.0% 43.6% +2.4
PARus 500 53.4% 53.6% -0.2
RWSD 810 49.9% 52.7% -2.8
Strict Constraints JSON 12 58.3% - -

Evaluation date: 2026-11-17. Baseline numbers โ€” official MERA submission of Qwen3.5-0.8B (submit id 16125).

RuMultiAR vs larger models

Model Params RuMultiAR
RuadaptQwen-32B 32B 43.3%
RusNanoLM-0.8B (ours) 0.8B 33.5%
GigaChat-2-10B 10B 31.5%
Qwen2.5-7B 7B 30.8%
Meta-Llama-3-8B 8B 24.8%
Llama-3.2-1B-Instruct 1B 16.2%
Qwen3.5-0.8B (baseline) 0.8B 8.1%

Visual Leaderboards

MERA Suite Summary

MERA Suite Summary

Baseline Progression

Baseline Progression

SimpleAR

SimpleAR

RuMultiAR

RuMultiAR

MathLogicQA

MathLogicQA

MaMuRaMu

MaMuRaMu


Quickstart & Inference

import torch
from transformers import AutoModelForImageTextToText, AutoTokenizer

MODEL_ID = "itismax0/RusNanoLM-800M"

tokenizer = AutoTokenizer.from_pretrained(MODEL_ID)
model = AutoModelForImageTextToText.from_pretrained(
    MODEL_ID,
    torch_dtype=torch.bfloat16,
    device_map="auto"
)
model.eval()

messages = [
    {"role": "user", "content": "ะ ะตัˆะธ ะทะฐะดะฐั‡ัƒ: ะฒ ะบะพั€ะทะธะฝะต 5 ัะฑะปะพะบ ะธ 7 ะณั€ัƒัˆ. ะกะบะพะปัŒะบะพ ะฒัะตะณะพ ั„ั€ัƒะบั‚ะพะฒ?"}
]

prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)

with torch.no_grad():
    outputs = model.generate(
        **inputs,
        max_new_tokens=512,
        do_sample=False,
        pad_token_id=tokenizer.eos_token_id
    )

response = tokenizer.decode(outputs[0][inputs.input_ids.shape[1]:], skip_special_tokens=True)
print(response)

Run with llama.cpp / Ollama (GGUF, full precision)

Two full-precision F16 GGUF files are included โ€” no quantization loss. They differ only in tensor naming (llama.cpp renamed some Qwen3.5 tensors between b12257 and b12413):

File Use with
RusNanoLM-800M-F16.gguf llama.cpp >= b12413 (current master)
RusNanoLM-800M-F16-ollama.gguf Ollama (all current versions) and other apps bundling llama.cpp <= b12257 (LM Studio, etc.)
# llama.cpp (fresh build)
llama-cli -hf itismax0/RusNanoLM-800M:RusNanoLM-800M-F16.gguf --temp 0 -p "ะ ะตัˆะธ: 13*7"

# Ollama โ€” direct pull
ollama run hf.co/itismax0/RusNanoLM-800M:RusNanoLM-800M-F16-ollama.gguf "ะ’ั‹ั‡ะธัะปะธ: 13*7"

Note: the model uses the Qwen3.5 hybrid linear-attention architecture. If you see tensor 'blk.0.ssm_beta.weight' not found in Ollama โ€” use the -ollama GGUF variant. If LM Studio refuses to load, update its llama.cpp runtime (Settings โ†’ Runtimes).


ะžะฑะทะพั€ ะฝะฐ ั€ัƒััะบะพะผ

RusNanoLM-800M โ€” ะพั‚ะบั€ั‹ั‚ะฐั ะดะฒัƒัะทั‹ั‡ะฝะฐั (ั€ัƒััะบะธะน ะธ ะฐะฝะณะปะธะนัะบะธะน) ะผะพะดะตะปัŒ ะฝะฐ 800 ะผะปะฝ ะฟะฐั€ะฐะผะตั‚ั€ะพะฒ, ะฟะพัั‚ั€ะพะตะฝะฝะฐั ะฝะฐ ะณะธะฑั€ะธะดะฝะพะน ะฐั€ั…ะธั‚ะตะบั‚ัƒั€ะต Qwen3.5 ั ะปะธะฝะตะนะฝั‹ะผ ะฒะฝะธะผะฐะฝะธะตะผ (Gated Delta Rule): 24 ัะปะพั, ะบะฐะถะดั‹ะน ั‡ะตั‚ะฒั‘ั€ั‚ั‹ะน โ€” ะฟะพะปะฝะพะต ะฒะฝะธะผะฐะฝะธะต. ะžั†ะตะฝะธะฒะฐะปะฐััŒ ะฝะฐ ั€ัƒััะบะพัะทั‹ั‡ะฝะพะผ ะฑะตะฝั‡ะผะฐั€ะบะต MERA (Pass@1, ะถะฐะดะฝะฐั ะณะตะฝะตั€ะฐั†ะธั, chat template).

ะ ะตะทัƒะปัŒั‚ะฐั‚ั‹ MERA

ะ‘ะตะฝั‡ะผะฐั€ะบ ะ—ะฐะดะฐั‡ RusNanoLM-0.8B Qwen3.5-0.8B (baseline) ฮ”
SimpleAR 1000 83.2% 48.3% +34.9
RuMultiAR 1039 33.5% 8.1% +25.4 (ร—4.1)
MathLogicQA 680 38.2% 33.9% +4.4
MaMuRaMu 285 46.0% 43.6% +2.4
PARus 500 53.4% 53.6% -0.2
RWSD 810 49.9% 52.7% -2.8
Strict Constraints JSON 12 58.3% - -

ะ“ะปะฐะฒะฝั‹ะน ั€ะตะทัƒะปัŒั‚ะฐั‚: RuMultiAR ะฟั€ะพั‚ะธะฒ ะฑะพะปัŒัˆะธั… ะผะพะดะตะปะตะน

ะะฐ ะฐั€ะธั„ะผะตั‚ะธั‡ะตัะบะธั… ั€ะฐัััƒะถะดะตะฝะธัั… (RuMultiAR) ะบะพะผะฟะฐะบั‚ะฝะฐั ะผะพะดะตะปัŒ 0.8B ะพะฑะณะพะฝัะตั‚ ะผะพะดะตะปะธ ะฒ 8-12 ั€ะฐะท ั‚ัะถะตะปะตะต:

  • 33.5% โ€” RusNanoLM-0.8B (ะฝะฐัˆะฐ)
  • 31.5% โ€” GigaChat-2-10B
  • 30.8% โ€” Qwen2.5-7B
  • 24.8% โ€” Meta-Llama-3-8B
  • 8.1% โ€” ะฑะฐะทะพะฒะฐั Qwen3.5-0.8B (ั€ะพัั‚ ะฒ 4.1 ั€ะฐะทะฐ)

ะงะตัั‚ะฝั‹ะต ะพะณั€ะฐะฝะธั‡ะตะฝะธั

  • PARus (-0.2%) ะธ RWSD (-2.8%) โ€” ะฝะฐ ัƒั€ะพะฒะฝะต ะธะปะธ ั‡ัƒั‚ัŒ ะฝะธะถะต baseline: ะฟั€ะธั€ะพัั‚ ะฟะพะปัƒั‡ะตะฝ ะทะฐ ัั‡ั‘ั‚ ะฐั€ะธั„ะผะตั‚ะธะบะธ ะธ ั€ะฐัััƒะถะดะตะฝะธะน, ะพะฑั‰ะตะต ะฟะพะฝะธะผะฐะฝะธะต ั‚ะตะบัั‚ะฐ ะฝะต ะดะตะณั€ะฐะดะธั€ะพะฒะฐะปะพ, ะฝะพ ะธ ะฝะต ะฒั‹ั€ะพัะปะพ.
  • Strict Constraints JSON โ€” ะฒัะตะณะพ 12 ะทะฐะดะฐั‡, ะฒั‹ะฑะพั€ะบะฐ ะผะฐะปะฐ ะดะปั ัะธะปัŒะฝั‹ั… ะฒั‹ะฒะพะดะพะฒ.

ะ“ั€ะฐั„ะธะบะธ

ะกะฒะพะดะบะฐ MERA

ะŸั€ะพะณั€ะตัั ะพั‚ะฝะพัะธั‚ะตะปัŒะฝะพ baseline

SimpleAR

RuMultiAR

MathLogicQA

MaMuRaMu

ะั€ั…ะธั‚ะตะบั‚ัƒั€ะฐ ะธ ะฟะฐั€ะฐะผะตั‚ั€ั‹

  • ะŸะฐั€ะฐะผะตั‚ั€ั‹: 800 ะผะปะฝ (0.8B)
  • ะ’ะฝะธะผะฐะฝะธะต: ะณะธะฑั€ะธะดะฝะพะต ะปะธะฝะตะนะฝะพะต ะฒะฝะธะผะฐะฝะธะต (Gated Delta Rule), ัะปะพะถะฝะพัั‚ัŒ ะฟะฐะผัั‚ะธ ัะพัั‚ะพัะฝะธั O(1)
  • ะกะปะพะธ: 24, ะบะฐะถะดั‹ะน ั‡ะตั‚ะฒั‘ั€ั‚ั‹ะน โ€” ะฟะพะปะฝะพะต ะฒะฝะธะผะฐะฝะธะต
  • ะขั€ะตะฑะพะฒะฐะฝะธั ะบ ะฒะธะดะตะพะฟะฐะผัั‚ะธ: ะพะบะพะปะพ 1.7 ะ“ะ‘ ะฒ ั„ะพั€ะผะฐั‚ะต BF16 (ะทะฐะฟัƒัะบะฐะตั‚ัั ะฝะฐ GPU ั 4 ะ“ะ‘ VRAM)
  • ะจะฐะฑะปะพะฝ ะดะธะฐะปะพะณะฐ: ChatML (ะฒัั‚ั€ะพะตะฝ ะฒ tokenizer_config)
  • ะ›ะธั†ะตะฝะทะธั: Apache 2.0
Downloads last month
194
Safetensors
Model size
0.9B params
Tensor type
F16
ยท
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for RusMiniLM/RusNanoLM-800M

Quantizations
1 model