ProCreations/nib-4b-preview

Nib 4B (preview)

Nib is a small browser agent. It is Qwen3.5-4B post-trained to work websites through the tools of common agent harnesses (Playwright MCP, browser-use, agent-browser-style CLIs, Chrome DevTools MCP, Aside, computer-use loops and coding agents that drive a browser). It reads page snapshots and screenshots, calls tools, and reports what it found.

This is a preview: it is useful, but it still makes mistakes a bigger model would not, and it too often reports success on tasks it did not finish (see Limitations). Check its answers before you rely on them.

• About twice as good as its base on our browser tasks: 62% vs 29–30% on live web + synthetic tasks, and 80% vs 23% on self-hosted web apps.
• Close to models many times its size: on the combined set it is within noise of DeepSeek V4.1 Flash (high) and of the untrained Qwen3.8-27B (tied with thinking off, 3–6 points behind with thinking on), and it leads both on self-hosted apps.
• Fewer tokens: 1,300–2,500 output tokens per task, 17–41% fewer than its base with thinking on.
• Four reasoning efforts: thinking off, low, medium and xhigh, picked per request.

Benchmarks

Every model ran the same tasks with the same harnesses (Aside, Playwright MCP and an agent-browser-style bash CLI). These are our own task sets, not public benchmarks:

• Live web (120 tasks): real websites. Graded by one LLM judge with an evidence digest of the episode.
• Synthetic (108 tasks): generated web apps with planted facts and forms. Graded by code.
• Self-hosted web apps (62 tasks): fresh sandboxes of Gitea, WordPress, Odoo and similar apps. Graded by code against the app's state.

Charts plot success against mean output tokens per task; lines connect a model's reasoning efforts.

Success % on live web + synthetic / live web only / self-hosted apps, and mean output tokens per task (live + synthetic):

Model	Setting	Live + synthetic	Live web	Self-hosted apps	Tokens
Nib 4B	off	56.6	52.5	80.6	1,293
Nib 4B	low	61.8	53.3	82.3	1,783
Nib 4B	medium	59.2	49.2	79.0	1,955
Nib 4B	xhigh	62.3	55.8	80.6	2,537
Qwen3.5-4B (base, untrained)	off	30.3	37.5	22.6	1,807
Qwen3.5-4B (base, untrained)	thinking on	28.5	35.6	22.6	3,040
Qwen3.5-9B (untrained)	off	41.2	49.2	not run	1,926
Qwen3.5-9B (untrained)	thinking on	36.7	42.2	not run	2,983
Qwen3.8-27B (untrained)	off	56.1	55.0	64.5	1,444
Qwen3.8-27B (untrained)	low	65.8	65.0	75.8	2,727
Qwen3.8-27B (untrained)	medium	64.9	66.7	64.5	2,188
Qwen3.8-27B (untrained)	xhigh	65.8	66.7	77.4	3,771
Qwen3.8-Flash-Next	medium	74.6	70.8	not run	3,179
Mistral Large 4	high	63.6	67.5	79.0	2,884
DeepSeek V4.1 Flash	high	60.1	55.8	66.1	3,311
GLM 5.3	high	47.4	48.3	64.5	1,163

• 95% confidence intervals are about ±6 points (live + synthetic), ±9 (live web) and ±10 (apps).
• Qwen3.5-4B and Qwen3.5-9B have no effort levels (their chat template only turns thinking on or off); their "thinking on" row averages three runs.
• Nib trained on other tasks from the same synthetic generators and the same self-hosted apps (never the evaluation tasks), so part of its lead there is in-distribution. Live web is the fairest comparison.
• The other models ran through their providers' APIs at one setting, two days before Nib's runs. Live sites change; synthetic and app tasks do not.
• On a larger and harder live-web set (581 tasks across 8 harnesses), Nib scores about 45% against 56% for the untrained Qwen3.8-27B, using about half the tokens.

Using Nib

Nib uses the Qwen3.5 architecture (hybrid Gated DeltaNet + attention, with a vision encoder). We tested it with SGLang 0.5.21:

python -m sglang.launch_server \
  --model-path ProCreations/nib-4b-preview --served-model-name nib-4b \
  --context-length 131072 --reasoning-parser qwen3 --tool-call-parser qwen3_coder \
  --mamba-ssm-dtype bfloat16 --mamba-radix-cache-strategy extra_buffer_lazy --trust-remote-code

Pick the reasoning effort per request through the chat template:

from openai import OpenAI
client = OpenAI(base_url="http://127.0.0.1:30000/v1", api_key="none")
resp = client.chat.completions.create(
    model="nib-4b",
    messages=[{"role": "user", "content": "Find the opening hours of the Louvre today."}],
    tools=my_browser_tools,                     # your harness's tools
    temperature=1.0, top_p=0.95,                # what all our evaluations used
    extra_body={"chat_template_kwargs": {"enable_thinking": True, "reasoning_effort": "low"}},
)

Effort	chat_template_kwargs	When to use it
off	{"enable_thinking": false}	fastest and cheapest; fine for simple, well-specified tasks
low	{"enable_thinking": true, "reasoning_effort": "low"}	best balance in our tests
medium	{"enable_thinking": true, "reasoning_effort": "medium"}
xhigh	{"enable_thinking": true} (the default)	most careful, most tokens

Tips:

• Nib works best inside an agent harness that gives it browser tools and page snapshots. It was trained with Playwright MCP, browser-use, agent-browser-style CLIs, Chrome DevTools MCP, Aside, Qwen-style computer use, and coding agents (Pi, OpenCode) driving a browser.
• Screenshots help: it was trained with the images these harnesses send.
• Most training episodes fit in 32k tokens. Longer sessions work, but quality on very long contexts is untested.
• Not tested with vLLM, llama.cpp or MLX. They may work wherever Qwen3.5 is supported.

Quantized versions

Every quant keeps the vision encoder, embeddings, norms, Gated DeltaNet gates and MTP weights in their original precision. Agreement is measured against these BF16 weights on 5,070 held-out assistant tokens from agent episodes.

Repo	Format	Size	Mean KL	Top-1 agreement	Browser tasks (581-task live set)
this repo	BF16	9.3 GB	–	–	45.3%
nib-4b-preview-FP8	FP8 block 128×128 (SGLang)	5.8 GB	0.0028	98.9%	42.5% (within run-to-run noise)
nib-4b-preview-NVFP4	NVFP4 (SGLang, Blackwell)	5.0 GB	0.0147	97.0%	45.6%
nib-4b-preview-EXL3	EXL3 3.05–6.05 bpw (exllamav3 / TabbyAPI)	3.6–4.9 GB	0.0257–0.0007	96.3–99.3%	not run

How Nib was trained

1. Supervised fine-tuning on browser-agent episodes that were verified as successful, from larger teacher models (Qwen3.8-27B-class and Qwen3.8-Flash-Next), across about ten harnesses, with screenshots. It also included demonstrations of reporting plainly when a task could not be done.
2. On-policy distillation: Nib ran tasks itself, and larger teachers scored its own tokens, first a browser-tuned Qwen3.8-27B, then Qwen3.8-Flash-Next. This preview is the third update of the final round, picked by a held-out live evaluation. Later updates and an honesty-penalty round did not score higher.

Rewards and filters came from code checkers (synthetic tasks, app databases) and an LLM judge (live websites). The evaluation tasks were held out from training. The training data is not released.

Limitations

• It often claims success when it failed. In a held-out check, about 7 in 10 of its failed live-web tasks ended with a confident (wrong) answer rather than a plain "I could not do this". Verify important results.
• Behind larger models on the open web. On live websites it trails the untrained Qwen3.8-27B by 11–18 points with thinking on, and by about 11 points on our harder 581-task live set.
• Small-model knowledge. It can guess wrong URLs or facts that a bigger model would know, and should look things up instead.
• Training and evaluation tasks were almost all in English.
• These are our own benchmarks; live-web results drift as sites change.

License

Apache 2.0, the same as the base model Qwen/Qwen3.5-4B.