--- license: apache-2.0 license_link: https://huggingface.co/Qwen/Qwen3.5-4B/blob/main/LICENSE base_model: - Qwen/Qwen3.5-4B base_model_relation: finetune library_name: transformers pipeline_tag: image-text-to-text language: - en tags: - browser-agent - browser-use - computer-use - agent - tool-use - qwen3.5 --- # Nib 4B (preview) **Nib is a small browser agent.** It is [Qwen3.5-4B](https://huggingface.co/Qwen/Qwen3.5-4B) post-trained to work websites through the tools of common agent harnesses (Playwright MCP, browser-use, agent-browser-style CLIs, Chrome DevTools MCP, Aside, computer-use loops and coding agents that drive a browser). It reads page snapshots and screenshots, calls tools, and reports what it found. This is a **preview**: it is useful, but it still makes mistakes a bigger model would not, and it too often reports success on tasks it did not finish (see [Limitations](#limitations)). Check its answers before you rely on them. - **About twice as good as its base** on our browser tasks: 62% vs 29–30% on live web + synthetic tasks, and 80% vs 23% on self-hosted web apps. - **Close to models many times its size**: on the combined set it is within noise of DeepSeek V4.1 Flash (high) and of the untrained Qwen3.8-27B (tied with thinking off, 3–6 points behind with thinking on), and it leads both on self-hosted apps. - **Fewer tokens**: 1,300–2,500 output tokens per task, 17–41% fewer than its base with thinking on. - **Four reasoning efforts**: thinking off, low, medium and xhigh, picked per request. ## Benchmarks Every model ran the same tasks with the same harnesses (Aside, Playwright MCP and an agent-browser-style bash CLI). These are our own task sets, not public benchmarks: - **Live web (120 tasks):** real websites. Graded by one LLM judge with an evidence digest of the episode. - **Synthetic (108 tasks):** generated web apps with planted facts and forms. Graded by code. - **Self-hosted web apps (62 tasks):** fresh sandboxes of Gitea, WordPress, Odoo and similar apps. Graded by code against the app's state. Charts plot success against mean output tokens per task; lines connect a model's reasoning efforts. ![Live web + synthetic tasks](https://huggingface.co/ProCreations/nib-4b-preview/resolve/main/assets/bench_live_synth.png) ![Live web tasks](https://huggingface.co/ProCreations/nib-4b-preview/resolve/main/assets/bench_live.png) ![Self-hosted web-app tasks](https://huggingface.co/ProCreations/nib-4b-preview/resolve/main/assets/bench_apps.png) Success % on live web + synthetic / live web only / self-hosted apps, and mean output tokens per task (live + synthetic): | Model | Setting | Live + synthetic | Live web | Self-hosted apps | Tokens | |---|---|---:|---:|---:|---:| | **Nib 4B** | off | 56.6 | 52.5 | 80.6 | 1,293 | | **Nib 4B** | low | 61.8 | 53.3 | **82.3** | 1,783 | | **Nib 4B** | medium | 59.2 | 49.2 | 79.0 | 1,955 | | **Nib 4B** | xhigh | 62.3 | 55.8 | 80.6 | 2,537 | | Qwen3.5-4B (base, untrained) | off | 30.3 | 37.5 | 22.6 | 1,807 | | Qwen3.5-4B (base, untrained) | thinking on | 28.5 | 35.6 | 22.6 | 3,040 | | Qwen3.5-9B (untrained) | off | 41.2 | 49.2 | not run | 1,926 | | Qwen3.5-9B (untrained) | thinking on | 36.7 | 42.2 | not run | 2,983 | | Qwen3.8-27B (untrained) | off | 56.1 | 55.0 | 64.5 | 1,444 | | Qwen3.8-27B (untrained) | low | 65.8 | 65.0 | 75.8 | 2,727 | | Qwen3.8-27B (untrained) | medium | 64.9 | 66.7 | 64.5 | 2,188 | | Qwen3.8-27B (untrained) | xhigh | 65.8 | 66.7 | 77.4 | 3,771 | | Qwen3.8-Flash-Next | medium | 74.6 | 70.8 | not run | 3,179 | | Mistral Large 4 | high | 63.6 | 67.5 | 79.0 | 2,884 | | DeepSeek V4.1 Flash | high | 60.1 | 55.8 | 66.1 | 3,311 | | GLM 5.3 | high | 47.4 | 48.3 | 64.5 | 1,163 | - 95% confidence intervals are about ±6 points (live + synthetic), ±9 (live web) and ±10 (apps). - Qwen3.5-4B and Qwen3.5-9B have no effort levels (their chat template only turns thinking on or off); their "thinking on" row averages three runs. - Nib trained on other tasks from the same synthetic generators and the same self-hosted apps (never the evaluation tasks), so part of its lead there is in-distribution. Live web is the fairest comparison. - The other models ran through their providers' APIs at one setting, two days before Nib's runs. Live sites change; synthetic and app tasks do not. - On a larger and harder live-web set (581 tasks across 8 harnesses), Nib scores about 45% against 56% for the untrained Qwen3.8-27B, using about half the tokens. ## Using Nib Nib uses the Qwen3.5 architecture (hybrid Gated DeltaNet + attention, with a vision encoder). We tested it with **SGLang 0.5.21**: ```bash python -m sglang.launch_server \ --model-path ProCreations/nib-4b-preview --served-model-name nib-4b \ --context-length 131072 --reasoning-parser qwen3 --tool-call-parser qwen3_coder \ --mamba-ssm-dtype bfloat16 --mamba-radix-cache-strategy extra_buffer_lazy --trust-remote-code ``` Pick the reasoning effort per request through the chat template: ```python from openai import OpenAI client = OpenAI(base_url="http://127.0.0.1:30000/v1", api_key="none") resp = client.chat.completions.create( model="nib-4b", messages=[{"role": "user", "content": "Find the opening hours of the Louvre today."}], tools=my_browser_tools, # your harness's tools temperature=1.0, top_p=0.95, # what all our evaluations used extra_body={"chat_template_kwargs": {"enable_thinking": True, "reasoning_effort": "low"}}, ) ``` | Effort | `chat_template_kwargs` | When to use it | |---|---|---| | off | `{"enable_thinking": false}` | fastest and cheapest; fine for simple, well-specified tasks | | low | `{"enable_thinking": true, "reasoning_effort": "low"}` | best balance in our tests | | medium | `{"enable_thinking": true, "reasoning_effort": "medium"}` | | | xhigh | `{"enable_thinking": true}` (the default) | most careful, most tokens | Tips: - Nib works best inside an agent harness that gives it browser tools and page snapshots. It was trained with Playwright MCP, browser-use, agent-browser-style CLIs, Chrome DevTools MCP, Aside, Qwen-style computer use, and coding agents (Pi, OpenCode) driving a browser. - Screenshots help: it was trained with the images these harnesses send. - Most training episodes fit in 32k tokens. Longer sessions work, but quality on very long contexts is untested. - Not tested with vLLM, llama.cpp or MLX. They may work wherever Qwen3.5 is supported. ## Quantized versions Every quant keeps the vision encoder, embeddings, norms, Gated DeltaNet gates and MTP weights in their original precision. Agreement is measured against these BF16 weights on 5,070 held-out assistant tokens from agent episodes. | Repo | Format | Size | Mean KL | Top-1 agreement | Browser tasks (581-task live set) | |---|---|---:|---:|---:|---| | this repo | BF16 | 9.3 GB | – | – | 45.3% | | [nib-4b-preview-FP8](https://huggingface.co/ProCreations/nib-4b-preview-FP8) | FP8 block 128×128 (SGLang) | 5.8 GB | 0.0028 | 98.9% | 42.5% (within run-to-run noise) | | [nib-4b-preview-NVFP4](https://huggingface.co/ProCreations/nib-4b-preview-NVFP4) | NVFP4 (SGLang, Blackwell) | 5.0 GB | 0.0147 | 97.0% | 45.6% | | [nib-4b-preview-EXL3](https://huggingface.co/ProCreations/nib-4b-preview-EXL3) | EXL3 3.05–6.05 bpw (exllamav3 / TabbyAPI) | 3.6–4.9 GB | 0.0257–0.0007 | 96.3–99.3% | not run | ## How Nib was trained 1. **Supervised fine-tuning** on browser-agent episodes that were verified as successful, from larger teacher models (Qwen3.8-27B-class and Qwen3.8-Flash-Next), across about ten harnesses, with screenshots. It also included demonstrations of reporting plainly when a task could not be done. 2. **On-policy distillation**: Nib ran tasks itself, and larger teachers scored its own tokens, first a browser-tuned Qwen3.8-27B, then Qwen3.8-Flash-Next. This preview is the third update of the final round, picked by a held-out live evaluation. Later updates and an honesty-penalty round did not score higher. Rewards and filters came from code checkers (synthetic tasks, app databases) and an LLM judge (live websites). The evaluation tasks were held out from training. The training data is not released. ## Limitations - **It often claims success when it failed.** In a held-out check, about 7 in 10 of its failed live-web tasks ended with a confident (wrong) answer rather than a plain "I could not do this". Verify important results. - **Behind larger models on the open web.** On live websites it trails the untrained Qwen3.8-27B by 11–18 points with thinking on, and by about 11 points on our harder 581-task live set. - **Small-model knowledge.** It can guess wrong URLs or facts that a bigger model would know, and should look things up instead. - Training and evaluation tasks were almost all in English. - These are our own benchmarks; live-web results drift as sites change. ## License Apache 2.0, the same as the base model [Qwen/Qwen3.5-4B](https://huggingface.co/Qwen/Qwen3.5-4B).