Instructions to use loqira/Qwen3.5-0.8B-GEC-KAZ-RUS-ENG with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use loqira/Qwen3.5-0.8B-GEC-KAZ-RUS-ENG with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf loqira/Qwen3.5-0.8B-GEC-KAZ-RUS-ENG:Q4_0 # Run inference directly in the terminal: llama cli -hf loqira/Qwen3.5-0.8B-GEC-KAZ-RUS-ENG:Q4_0
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf loqira/Qwen3.5-0.8B-GEC-KAZ-RUS-ENG:Q4_0 # Run inference directly in the terminal: llama cli -hf loqira/Qwen3.5-0.8B-GEC-KAZ-RUS-ENG:Q4_0
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-app # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama serve -hf loqira/Qwen3.5-0.8B-GEC-KAZ-RUS-ENG:Q4_0 # Run inference directly in the terminal: ./build/bin/llama cli -hf loqira/Qwen3.5-0.8B-GEC-KAZ-RUS-ENG:Q4_0
Use Docker
docker model run hf.co/loqira/Qwen3.5-0.8B-GEC-KAZ-RUS-ENG:Q4_0
- Llama new
- LM Studio
- Jan
- vLLM
How to use loqira/Qwen3.5-0.8B-GEC-KAZ-RUS-ENG with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "loqira/Qwen3.5-0.8B-GEC-KAZ-RUS-ENG" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "loqira/Qwen3.5-0.8B-GEC-KAZ-RUS-ENG", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/loqira/Qwen3.5-0.8B-GEC-KAZ-RUS-ENG:Q4_0
- Ollama
How to use loqira/Qwen3.5-0.8B-GEC-KAZ-RUS-ENG with Ollama:
ollama run hf.co/loqira/Qwen3.5-0.8B-GEC-KAZ-RUS-ENG:Q4_0
- Unsloth Desktop
- Pi
How to use loqira/Qwen3.5-0.8B-GEC-KAZ-RUS-ENG with Pi:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf loqira/Qwen3.5-0.8B-GEC-KAZ-RUS-ENG:Q4_0
Configure the model in Pi
# Install Pi: npm install -g @earendil-works/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "llama-cpp": { "baseUrl": "http://localhost:9931/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "loqira/Qwen3.5-0.8B-GEC-KAZ-RUS-ENG:Q4_0" } ] } } }Run Pi
# Start Pi in your project directory: pi
- Docker Model Runner
How to use loqira/Qwen3.5-0.8B-GEC-KAZ-RUS-ENG with Docker Model Runner:
docker model run hf.co/loqira/Qwen3.5-0.8B-GEC-KAZ-RUS-ENG:Q4_0
- Lemonade
How to use loqira/Qwen3.5-0.8B-GEC-KAZ-RUS-ENG with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull loqira/Qwen3.5-0.8B-GEC-KAZ-RUS-ENG:Q4_0
Run and chat with the model
lemonade run user.Qwen3.5-0.8B-GEC-KAZ-RUS-ENG-Q4_0
List all available models
lemonade list
- Hermes Agent
How to use loqira/Qwen3.5-0.8B-GEC-KAZ-RUS-ENG with Hermes Agent:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf loqira/Qwen3.5-0.8B-GEC-KAZ-RUS-ENG:Q4_0
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:9931/v1 hermes config set model.default loqira/Qwen3.5-0.8B-GEC-KAZ-RUS-ENG:Q4_0
Run Hermes
hermes
- Atomic Chat
- OpenClaw
How to use loqira/Qwen3.5-0.8B-GEC-KAZ-RUS-ENG with OpenClaw:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf loqira/Qwen3.5-0.8B-GEC-KAZ-RUS-ENG:Q4_0
Configure OpenClaw
# Install OpenClaw: npm install -g openclaw@latest # Register the local server and set it as the default model: openclaw onboard --non-interactive --mode local \ --auth-choice custom-api-key \ --custom-base-url http://127.0.0.1:9931/v1 \ --custom-model-id "loqira/Qwen3.5-0.8B-GEC-KAZ-RUS-ENG:Q4_0" \ --custom-provider-id llama-cpp \ --custom-compatibility openai \ --custom-text-input \ --accept-risk \ --skip-health
Run OpenClaw
openclaw agent --local --agent main --message "Hello from Hugging Face"
Qwen3.5-0.8B GEC for Kazakh, Russian, and English
A compact 0.8B grammatical-error-correction / proofreading model for Kazakh, Russian, and English. It takes raw text โ voice-dictation output, chat messages, quick notes โ and cleans it up: restores punctuation, capitalization, and paragraph breaks, and fixes typos and spelling errors โ while preserving your words, meaning, language, and tone. It does not rewrite, paraphrase, translate, soften, or add anything of its own.
Published as GGUF (Q4_0) for CPU inference with llama.cpp, LM Studio, and Ollama. ~537 MB, ~44 tokens/s on CPU.
By Loqira Labs โ everyday AI you own, not rent. Runs on the hardware you already have โ no cloud, no frontier API.
โก TL;DR
- Task: grammar / spelling / punctuation / capitalization / paragraphing correction (GEC, proofreading).
- Word-preserving: keeps your exact words, meaning and tone; it corrects, it does not rewrite.
- Languages: Kazakh, Russian, English (incl. Kazakh special letters).
- One request โ one response. This is a single-turn model: use it for one-shot calls, not multi-turn chat.
- Must-have runtime settings: the exact system prompt below, thinking OFF, temperature 0.
- Format: GGUF Q4_0, ~537 MB, ~44 tok/s CPU.
What it does
- Restores punctuation, capitalization, and paragraph structure.
- Fixes typos and spelling errors.
- Word-preserving โ keeps your words, meaning, tone, and language intact.
- Adds nothing, invents nothing, and does not soften or rephrase.
โ ๏ธ Single-turn model โ one request, one response
This model is trained and meant to be used as a one-shot call: send one input, get one corrected output. It is not a conversational model โ do not build multi-turn dialogues, do not feed prior turns back in, and do not chain calls in a running chat context. Each correction is an independent request with a fresh context and the same system prompt.
Languages
- Explicitly trained and evaluated: Kazakh (kk), Russian (ru), English (en) โ including Kazakh-specific letters (า, ำฉ, าฑ, าฏ, ำ, ั, าฃ, า, าป), which the model restores.
- Other languages: the base model (Qwen3.5-0.8B) additionally covers 198+ languages. The correction behaviour may transfer to them, but this was not tested โ treat any non-KK/RU/EN use as experimental.
Model details
| Base model | Qwen/Qwen3.5-0.8B (hybrid: GatedDeltaNet + full attention, 24 layers, hidden 1024) |
| Relation | finetune (quantization-aware) |
| Method | QAT at 4/8/16-bit with train==deploy parity โ our int4/int8 codes are injected into the GGUF, not re-quantized |
| Format | GGUF, Q4_0 weights + Q8_0 / F32 (short-conv, norms) |
| Size | ~537 MB |
| Speed | ~44 tokens/s on CPU |
| Training context | 2048 tokens (โ1600 per sample on average) |
| Chat template + stop token | Both baked into the GGUF |
The stop token is <|im_end|> โ clients that read the GGUF metadata pick it up automatically.
Usage
1. System prompt โ required (Russian, verbatim)
The model was trained to always receive this exact system prompt. Pass it as the system message. It is a single unified prompt โ the same for every input and every language (Kazakh, Russian, and English input all use this one prompt).
ะััะพัะผะฐัะธััะน ัะตะบัั ะณะพะปะพัะพะฒะพะณะพ ะฒะฒะพะดะฐ: ัะฐัััะฐะฒั ะฟัะฝะบััะฐัะธั ะธ ะทะฐะณะปะฐะฒะฝัะต ะฑัะบะฒั, ัะฐะทะฑะตะน ะฝะฐ ะฐะฑะทะฐัั, ะธัะฟัะฐะฒั ะพะฟะตัะฐัะบะธ ะธ ะพััะพะณัะฐัะธัะตัะบะธะต ะพัะธะฑะบะธ. ะกะพั
ัะฐะฝะธ ัะทัะบ, ัะปะพะฒะฐ ะธ ัะผััะป, ะฝะธัะตะณะพ ะฝะต ะดะพะฑะฐะฒะปัะน ะพั ัะตะฑั.
The prompt is in Russian by design โ that is what the model was trained on, so it gives the best results. A version of the prompt translated into another language also works, but noticeably worse โ keep the Russian prompt unless you have a specific reason not to. Without a system prompt at all (or with an unrelated one) the model is out-of-distribution and quality drops sharply.
2. Thinking OFF โ required
The model was trained with an empty reasoning block (<think>\n\n</think>\n\n). A client with "thinking / reasoning" enabled will inject <think>\n and push the model out-of-distribution (garbled prefix, quality loss). Keep thinking OFF and use temperature 0 (greedy, deterministic).
3. llama.cpp (llama-cli)
llama-cli -m Qwen3.5-0.8B-GEC-KAZ-RUS-ENG.Q4_0.gguf \
-sys "ะััะพัะผะฐัะธััะน ัะตะบัั ะณะพะปะพัะพะฒะพะณะพ ะฒะฒะพะดะฐ: ัะฐัััะฐะฒั ะฟัะฝะบััะฐัะธั ะธ ะทะฐะณะปะฐะฒะฝัะต ะฑัะบะฒั, ัะฐะทะฑะตะน ะฝะฐ ะฐะฑะทะฐัั, ะธัะฟัะฐะฒั ะพะฟะตัะฐัะบะธ ะธ ะพััะพะณัะฐัะธัะตัะบะธะต ะพัะธะฑะบะธ. ะกะพั
ัะฐะฝะธ ัะทัะบ, ัะปะพะฒะฐ ะธ ัะผััะป, ะฝะธัะตะณะพ ะฝะต ะดะพะฑะฐะฒะปัะน ะพั ัะตะฑั." \
-p "ะฟัะธะฒัะต ะฒัะตะผ ัะตะณะดะฝั ั ั
ะพัั ัะฐัะบะฐะทะฐัั ะฟัะพ ะฝะฐั ะฝะพะฒัะน ะฟัะฐะตะบั ะฝะฐะด ะบะพัััะผ ะผั ัะฐะฑะพัะฐะตะผ ัะถะต ะฟะพััะธ ะฟะพะป ะณะพะดะฐ" \
-st -no-cnv --reasoning off --reasoning-format none \
--chat-template-kwargs "{\"enable_thinking\": false}" \
--temp 0 --top-k 1 --repeat-penalty 1.0 -n 512
For long inputs, raise -n (e.g. 1024โ2048) so the output isn't truncated.
Keep each request short. The model was trained on a 2048-token context window (โ1600 tokens per sample on average). It is a single-shot formatter, not a long-document processor โ feeding a much longer sequence in one call pushes it out-of-distribution and degrades quality. Split long text into chunks (e.g. by paragraph) and format each chunk in its own request, keeping prompt + output within ~2048 tokens.
4. llama-server (OpenAI-compatible API)
Start the server with --reasoning-format none. In each request send the system prompt above, temperature: 0, and "chat_template_kwargs": {"enable_thinking": false}. Strip any leading <think>...</think> from the returned content.
5. Ollama
A ready Modelfile is included โ it bakes in the correct chat template (think-off), the system prompt, and greedy decoding.
ollama create qwen3.5-gec -f Modelfile
ollama run qwen3.5-gec "ะฟัะธะฒัะต ะฒัะตะผ ัะตะณะดะฝั ั ั
ะพัั ัะฐัะบะฐะทะฐัั ะฟัะพ ะฝะฐั ะฝะพะฒัะน ะฟัะฐะตะบั"
6. LM Studio
Load the GGUF, disable "thinking / reasoning", set temperature 0, and set the system prompt above. Verify that the rendered template produces <think>\n\n</think>\n\n (an empty block), not <think>\n.
Compatibility
This model uses the Qwen3.5 architecture (qwen35 โ hybrid GatedDeltaNet + attention), supported in mainline llama.cpp and the apps built on it. Just use a recent llama.cpp / LM Studio / Ollama. If you ever see unknown model architecture 'qwen35', your app bundles an outdated llama.cpp โ update it. Verified on the CPU path; on a GPU backend, run the golden-probe examples below first.
Examples (temperature 0, thinking off)
Russian โ typos:
in : ะฟัะธะฒัะต ะฒัะตะผ ัะตะณะดะฝั ั ั
ะพัั ัะฐัะบะฐะทะฐัั ะฟัะพ ะฝะฐั ะฝะพะฒัะน ะฟัะฐะตะบั ะฝะฐะด ะบะพัััะผ ะผั ัะฐะฑะพัะฐะตะผ ัะถะต ะฟะพััะธ ะฟะพะป ะณะพะดะฐ
out: ะัะธะฒะตั ะฒัะตะผ! ะกะตะณะพะดะฝั ั ั
ะพัั ัะฐััะบะฐะทะฐัั ะฟัะพ ะฝะฐั ะฝะพะฒัะน ะฟัะพะตะบั, ะฝะฐะด ะบะพัะพััะผ ะผั ัะฐะฑะพัะฐะตะผ ัะถะต ะฟะพััะธ ะฟะพะป ะณะพะดะฐ.
English โ typos:
in : helo evryone i jsut wanted to tel you abuot the meting we have tomorow mornig at nine oclock
out: Hello everyone, I just wanted to tell you about the meeting we have tomorrow morning at nine o'clock.
Kazakh โ special letters restored (า / ำฉ / ั):
in : ะผะตะฝ ะบะฐะปะฐะณะฐ ะฑะฐัะดัะผ ะพะฝะดะฐ ะบะพะฟ ะฐะดะฐะผ ะฑะฐั ะตะดะธ ะบัั ะบะตะทะธ ัััะบ ะฑะพะปะดั
out: ะะตะฝ าะฐะปะฐาะฐ ะฑะฐัะดัะผ, ะะฝะดะฐ ะบำฉะฟ ะฐะดะฐะผ ะฑะฐั, ะะฝะดั าัั ะบะตะทั ัััา ะฑะพะปะดั.
Preserves tone โ does not soften or rewrite:
in : ะดะฐ ัั ะธะทะดะตะฒะฐะตัััั ััะพ ะปะธ ะพะฟััั ะฒัั ัะปะพะผะฐะป ั ะถะต ะณะพะฒะพัะธะป ะฝะต ััะพะณะฐะน ััะพั ะบะพะด
out: ะะฐ ัั ะธะทะดะตะฒะฐะตัััั ััะพ ะปะธ? ะะฟััั ะฒัั ัะปะพะผะฐะป. ะฏ ะถะต ะณะพะฒะพัะธะป: ยซะะต ััะพะณะฐะน ััะพั ะบะพะดยป.
You can use these four as a quick golden probe: if your client produces materially different output at temperature 0, the prompt or think-off setting is wrong โ check the system prompt and thinking-off first.
Known limitations
- Mixed script within one word (e.g. Cyrillic + Latin "ะฑะปin") is not corrected.
- Segmentation around URLs / @handles can be unstable (though URLs and handles themselves are preserved byte-for-byte).
- "HH MM" time without a separator may get an extra dot.
- ~10% of isolated in-word typos may remain (not systematic); rare Kazakh word substitutions.
- Short context (2048 tokens, ~1600 avg per sample). Process long text in chunks โ one request per chunk โ rather than a single large sequence; see Usage.
- 0.8B / 4-bit ceiling โ this is a small, quantized model built for speed and privacy, not a frontier LLM.
License
Apache-2.0. Base model: Qwen/Qwen3.5-0.8B.
About Loqira Labs
Loqira Labs builds capable AI that runs on the hardware you already have โ no cloud, no frontier API, no vendor to answer to. Everyday AI you own, not rent.
๐ loqira.tech
- Downloads last month
- 398
4-bit