bukva-0.8b

Anything in, one letter out. Bukva (Russian буква, "letter") is a 0.8B decision model: you give it a situation (text and/or a screenshot), a question and a list of options, and it picks one — in a single forward pass, by reading the next-token logits of the option letters A, B, C, …. No generation, no output parsing, no retries. About 36 ms per decision on an RTX 3060 (≈150 ms with a 1280×800 screenshot).

It is a set of LoRA adapters on Qwen3.5-0.8B-Base (the base model is frozen; the vision tower is untouched). Four decision types:

type	what you get
choice	one of up to 26 options + calibrated probabilities
noul	yes / no with a probability
score	an ordinal option (e.g. 1–5 stars) + the expected value
multi	every option that applies

What it is good at: routing intents, moderation (EN/RU toxicity, spam), topic / sentiment / rating, "does this passage answer the question", reading UI screenshots, and — in the agent builds — driving an app or a website step by step ("what do I tap next?", "is the goal reached?").

Which build?

All builds share the same code; pick one with build=.

build	use it for
v8-classifier	the best pure classifier (text): classic tasks 82.9 (300 items per task)
mix-v11v7	universal: classifier + app agent — phone test 40/40, classic tasks 81.1
mix-v5v6	classifier + web-browsing agent (browser 5/8 on text)
v11-classifier, v12-classifier-r64	later classifier passes (not better on average; v11 is best on review stars, v12 on the game probes)
mix-v8v7, mix-v5v7	earlier universal merges
v7-agent-app / v6-agent-web	agent specialists (raw, before merging)
v5-classifier, v4-vision	earlier stages, kept for comparison

mix-* builds are exact linear merges of LoRA deltas (no extra training): specialise, then merge — the agent skill stays and the classifier forgets much less.

Benchmarks

Text version: BENCHMARKS.md. Every test is held out — no test item was trained on:

• Phone agent — an unseen synthetic test app (switches, files, delete-with-confirm, profile), 40 multi-step episodes in EN and RU, screenshot only. Success = the goal state is reached and the model itself says it is done.
• Browser — a real Chrome on live Wikipedia (EN/RU) and python.org, 8 navigation tasks; the page is given as text or as a screenshot.
• Classic tasks — 300 items each (spam: 187, balanced): intent routing (clinc151, 16 options), SMS spam, Russian news topics, toxicity EN and RU (textdetox), review stars (Yelp), QNLI.
• Probes — small hand-written question sets (47 EN, 21 RU, 11 game situations) that are in no dataset. They are small: one question is 2–5 points.
• UI screenshots — 484 questions about synthetic UI screenshots (toggles, sliders, dialogs, tabs…).
• Vision tests — mean over 11 public sets: AI2D, BLINK, CountBench, CV-Bench, GameQA, MMBench EN/RU, MMStar, ruCLEVR, RealWorldQA, ScreenSpot (multiple-choice form, 400 items each).

Quick start

pip install torch transformers safetensors huggingface_hub pillow
# Qwen3.5 linear-attention layers run much faster with flash-linear-attention (optional):
pip install flash-linear-attention

from huggingface_hub import hf_hub_download
import importlib.util, sys
spec = importlib.util.spec_from_file_location("bukva", hf_hub_download("annelo/bukva-0.8b", "bukva.py"))
bukva = importlib.util.module_from_spec(spec); spec.loader.exec_module(bukva)

m = bukva.Bukva.from_pretrained("annelo/bukva-0.8b", build="v8-classifier")   # or "mix-v11v7" (universal)

m.decide("hey, i can't log into my account since this morning",
         "Which team should handle this?", ["billing", "tech support", "sales", "spam"])
# {'answer': 'tech support', 'index': 1, 'probs': [...], 'confidence': ...}

m.decide("Это просто ужас, а не сервис!", "Есть ли в тексте оскорбление?", ["да", "нет"], type="noul")
m.decide("Great coffee but we waited 40 minutes.", "How many stars?", ["1", "2", "3", "4", "5"], type="score")
# -> also "expected": ~3.1
m.decide("", "Is Wi-Fi turned on?", ["yes", "no"], image="settings.png", type="noul")

The prompt the model was trained on (the code builds it for you):

<state text>

Question: <question>
Options:
A. <option 1>
B. <option 2>
Answer:

The answer is read from the logits of A, B, … right after Answer:. Probabilities are temperature- calibrated (T is stored with each build). A useful rule: act on its own when max(probs) ≥ 0.8 (on the classic tasks it is right 93–100% of the time there), otherwise escalate.

More than 26 options: split them into groups of ≤ 20, pick a winner per group, then pick among winners.

Agent loop (app or web)

m = bukva.Bukva.from_pretrained("annelo/bukva-0.8b", build="mix-v11v7")
log = []
for step in range(12):
    shot = take_screenshot()                          # your code
    state = "Screenshot of the app.\nActions taken so far:\n" + "\n".join(log)
    done = m.decide(state, "Task: turn off Bluetooth.\nIs the task finished?", ["yes", "no"], image=shot, type="noul")
    if done["answer"] == "yes":
        break
    buttons = list_clickable_labels()                 # your code: what can be tapped right now
    a = m.decide(state, "Task: turn off Bluetooth. What is the next click?", buttons, image=shot)
    result = tap(a["answer"])                         # your code
    log.append("step %d: %s => %s" % (step + 1, a["answer"], result))

How it was trained

• Base: Qwen3.5-0.8B-Base, frozen. LoRA r=16 (α=32) on all attention / linear-attention / MLP projections of the language model (150 matrices, 10.2M parameters). The vision tower is frozen; image features were pre-computed once and cached.
• Readout: the tied embedding rows of the letters — no new head. Options are reshuffled every epoch so that position carries no signal. Temperature fitted on validation afterwards.
• Data: ~140 task types, EN and RU — NLI, QA, sentiment, topics, intents (MASSIVE, banking77, HWU64, SNIPS, ATIS, Bitext), spam (email and SMS), toxicity, ratings, reasoning, games (chess, tic-tac-toe), synthetic UI / game / chart screenshots, A-OKVQA, ScienceQA, GQA-ru, ScreenQA, ChartQA, and synthetic multi-step app and website trajectories (shortest-path expert, pop-ups, dead buttons, "already done" states). Every benchmark above was de-duplicated out of the training data.
• Stages: v4 vision → v5 classic-task gaps → v6/v7 agents → v8 classifier; each stage ~20–40 minutes on one RTX 5090. Before every run an independent reviewer model audited the data generator from scratch (labels, leaks, shortcuts).

What did not help (so you don't have to try)

• A second low-learning-rate LoRA pass (2e-5) on top of the merged universal model: on all 24 layers or on layers 16-23 only. It changed the scores by less than the noise.
• Task arithmetic, v8 + λ·(v7 − v5), instead of a plain merge: the two skills interfere. Classic tasks went down to 79–81.
• Averaging the answer over two option orders: no gain. The model has no position bias, since options are reshuffled in training.
• A rank-64 pass on top of v8 and a second classifier pass with 9.5k Amazon reviews: no average gain over v8. The reviews gave +2–3 on Yelp stars but cost other tasks.

Limitations and honest notes

• It is not a chat model and does not generate text. It chooses among the options you give it.
• EN and RU only were trained and tested.
• The app-agent builds recognise "done" much better than the web builds. On real websites the app agent tends to fall back to "go to the main page" when lost (browser 3/8 vs 5/8).
• Small test sets: the browser test has 8 tasks and the probes 11–47 questions, so treat 1–2 point differences as noise.
• The v5–v7 replay data contained, word for word, the question wording of two benchmarks: clinc151 routing and Yelp stars. Test items were never included. v8 rewords those questions.
• SMS spam test contamination in v8 / mix-v8v7 (found by our own audit, fixed for later builds): one of the three public spam corpora used for v8 contains UCI SMS messages with punctuation deleted, which slipped past the de-duplication. 3 of the 300 held-out SMS test rows (1 of the 100 scored items) were in v8's training data. Re-scored without that item, v8's spam score moves from 89.7 to 89.5, so the effect is negligible. Builds after v8 use a stricter letters-and-digits filter.
• The phone test app uses pill-style switches, a standard widget that also appears among the v7 training styles. Its labels, wording and history format differ from the training ones.

Files

bukva.py                          loader + decide() (no PEFT needed; merges the LoRA at load time)
builds/<build>/adapter.safetensors
builds/<build>/bukva.json         rank, alpha, temperature, multi-label calibration
BENCHMARKS.md, bench.png, header.png

License

See LICENSE.md. In short: bukva.py is Apache-2.0. The adapters are for research and non-commercial use, because some training datasets are non-commercial (e.g. ScienceQA, CC BY-NC-SA 4.0). The base model is Apache-2.0.