Image-Text-to-Text
Transformers
Safetensors
English
qwen3_5
browser-agent
browser-use
computer-use
agent
tool-use
qwen3.5
conversational
Instructions to use ProCreations/nib-4b-preview with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use ProCreations/nib-4b-preview with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("image-text-to-text", model="ProCreations/nib-4b-preview") messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] pipe(text=messages)# pip install -U transformers accelerate # Load model directly from transformers import AutoProcessor, AutoModelForMultimodalLM processor = AutoProcessor.from_pretrained("ProCreations/nib-4b-preview") model = AutoModelForMultimodalLM.from_pretrained("ProCreations/nib-4b-preview", device_map="auto") messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] inputs = processor.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=256) print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use ProCreations/nib-4b-preview with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "ProCreations/nib-4b-preview" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "ProCreations/nib-4b-preview", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker
docker model run hf.co/ProCreations/nib-4b-preview
- SGLang
How to use ProCreations/nib-4b-preview with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "ProCreations/nib-4b-preview" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "ProCreations/nib-4b-preview", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "ProCreations/nib-4b-preview" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "ProCreations/nib-4b-preview", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }' - Docker Model Runner
How to use ProCreations/nib-4b-preview with Docker Model Runner:
docker model run hf.co/ProCreations/nib-4b-preview
|
Download README.md from ProCreations/nib-4b-preview: direct link, hf CLI and curl.
- Browser
- Download file 9.06 kB
-
https://huggingface.co/ProCreations/nib-4b-preview/resolve/main/README.md
- Command line
-
hf download hf://ProCreations/nib-4b-preview/README.md
-
curl -L -o README.md https://huggingface.co/ProCreations/nib-4b-preview/resolve/main/README.md
9.06 kB
| license: apache-2.0 | |
| license_link: https://huggingface.co/Qwen/Qwen3.5-4B/blob/main/LICENSE | |
| base_model: | |
| - Qwen/Qwen3.5-4B | |
| base_model_relation: finetune | |
| library_name: transformers | |
| pipeline_tag: image-text-to-text | |
| language: | |
| - en | |
| tags: | |
| - browser-agent | |
| - browser-use | |
| - computer-use | |
| - agent | |
| - tool-use | |
| - qwen3.5 | |
| # Nib 4B (preview) | |
| **Nib is a small browser agent.** It is [Qwen3.5-4B](https://huggingface.co/Qwen/Qwen3.5-4B) post-trained to work | |
| websites through the tools of common agent harnesses (Playwright MCP, browser-use, agent-browser-style CLIs, Chrome | |
| DevTools MCP, Aside, computer-use loops and coding agents that drive a browser). It reads page snapshots and | |
| screenshots, calls tools, and reports what it found. | |
| This is a **preview**: it is useful, but it still makes mistakes a bigger model would not, and it too often reports | |
| success on tasks it did not finish (see [Limitations](#limitations)). Check its answers before you rely on them. | |
| - **About twice as good as its base** on our browser tasks: 62% vs 29–30% on live web + synthetic tasks, and 80% vs 23% | |
| on self-hosted web apps. | |
| - **Close to models many times its size**: on the combined set it is within noise of DeepSeek V4.1 Flash (high) and of | |
| the untrained Qwen3.8-27B (tied with thinking off, 3–6 points behind with thinking on), and it leads both on | |
| self-hosted apps. | |
| - **Fewer tokens**: 1,300–2,500 output tokens per task, 17–41% fewer than its base with thinking on. | |
| - **Four reasoning efforts**: thinking off, low, medium and xhigh, picked per request. | |
| ## Benchmarks | |
| Every model ran the same tasks with the same harnesses (Aside, Playwright MCP and an agent-browser-style bash CLI). | |
| These are our own task sets, not public benchmarks: | |
| - **Live web (120 tasks):** real websites. Graded by one LLM judge with an evidence digest of the episode. | |
| - **Synthetic (108 tasks):** generated web apps with planted facts and forms. Graded by code. | |
| - **Self-hosted web apps (62 tasks):** fresh sandboxes of Gitea, WordPress, Odoo and similar apps. Graded by code against | |
| the app's state. | |
| Charts plot success against mean output tokens per task; lines connect a model's reasoning efforts. | |
|  | |
|  | |
|  | |
| Success % on live web + synthetic / live web only / self-hosted apps, and mean output tokens per task (live + synthetic): | |
| | Model | Setting | Live + synthetic | Live web | Self-hosted apps | Tokens | | |
| |---|---|---:|---:|---:|---:| | |
| | **Nib 4B** | off | 56.6 | 52.5 | 80.6 | 1,293 | | |
| | **Nib 4B** | low | 61.8 | 53.3 | **82.3** | 1,783 | | |
| | **Nib 4B** | medium | 59.2 | 49.2 | 79.0 | 1,955 | | |
| | **Nib 4B** | xhigh | 62.3 | 55.8 | 80.6 | 2,537 | | |
| | Qwen3.5-4B (base, untrained) | off | 30.3 | 37.5 | 22.6 | 1,807 | | |
| | Qwen3.5-4B (base, untrained) | thinking on | 28.5 | 35.6 | 22.6 | 3,040 | | |
| | Qwen3.5-9B (untrained) | off | 41.2 | 49.2 | not run | 1,926 | | |
| | Qwen3.5-9B (untrained) | thinking on | 36.7 | 42.2 | not run | 2,983 | | |
| | Qwen3.8-27B (untrained) | off | 56.1 | 55.0 | 64.5 | 1,444 | | |
| | Qwen3.8-27B (untrained) | low | 65.8 | 65.0 | 75.8 | 2,727 | | |
| | Qwen3.8-27B (untrained) | medium | 64.9 | 66.7 | 64.5 | 2,188 | | |
| | Qwen3.8-27B (untrained) | xhigh | 65.8 | 66.7 | 77.4 | 3,771 | | |
| | Qwen3.8-Flash-Next | medium | 74.6 | 70.8 | not run | 3,179 | | |
| | Mistral Large 4 | high | 63.6 | 67.5 | 79.0 | 2,884 | | |
| | DeepSeek V4.1 Flash | high | 60.1 | 55.8 | 66.1 | 3,311 | | |
| | GLM 5.3 | high | 47.4 | 48.3 | 64.5 | 1,163 | | |
| - 95% confidence intervals are about ±6 points (live + synthetic), ±9 (live web) and ±10 (apps). | |
| - Qwen3.5-4B and Qwen3.5-9B have no effort levels (their chat template only turns thinking on or off); their | |
| "thinking on" row averages three runs. | |
| - Nib trained on other tasks from the same synthetic generators and the same self-hosted apps (never the evaluation | |
| tasks), so part of its lead there is in-distribution. Live web is the fairest comparison. | |
| - The other models ran through their providers' APIs at one setting, two days before Nib's runs. Live sites change; | |
| synthetic and app tasks do not. | |
| - On a larger and harder live-web set (581 tasks across 8 harnesses), Nib scores about 45% against 56% for the | |
| untrained Qwen3.8-27B, using about half the tokens. | |
| ## Using Nib | |
| Nib uses the Qwen3.5 architecture (hybrid Gated DeltaNet + attention, with a vision encoder). We tested it with | |
| **SGLang 0.5.21**: | |
| ```bash | |
| python -m sglang.launch_server \ | |
| --model-path ProCreations/nib-4b-preview --served-model-name nib-4b \ | |
| --context-length 131072 --reasoning-parser qwen3 --tool-call-parser qwen3_coder \ | |
| --mamba-ssm-dtype bfloat16 --mamba-radix-cache-strategy extra_buffer_lazy --trust-remote-code | |
| ``` | |
| Pick the reasoning effort per request through the chat template: | |
| ```python | |
| from openai import OpenAI | |
| client = OpenAI(base_url="http://127.0.0.1:30000/v1", api_key="none") | |
| resp = client.chat.completions.create( | |
| model="nib-4b", | |
| messages=[{"role": "user", "content": "Find the opening hours of the Louvre today."}], | |
| tools=my_browser_tools, # your harness's tools | |
| temperature=1.0, top_p=0.95, # what all our evaluations used | |
| extra_body={"chat_template_kwargs": {"enable_thinking": True, "reasoning_effort": "low"}}, | |
| ) | |
| ``` | |
| | Effort | `chat_template_kwargs` | When to use it | | |
| |---|---|---| | |
| | off | `{"enable_thinking": false}` | fastest and cheapest; fine for simple, well-specified tasks | | |
| | low | `{"enable_thinking": true, "reasoning_effort": "low"}` | best balance in our tests | | |
| | medium | `{"enable_thinking": true, "reasoning_effort": "medium"}` | | | |
| | xhigh | `{"enable_thinking": true}` (the default) | most careful, most tokens | | |
| Tips: | |
| - Nib works best inside an agent harness that gives it browser tools and page snapshots. It was trained with | |
| Playwright MCP, browser-use, agent-browser-style CLIs, Chrome DevTools MCP, Aside, Qwen-style computer use, and coding | |
| agents (Pi, OpenCode) driving a browser. | |
| - Screenshots help: it was trained with the images these harnesses send. | |
| - Most training episodes fit in 32k tokens. Longer sessions work, but quality on very long contexts is untested. | |
| - Not tested with vLLM, llama.cpp or MLX. They may work wherever Qwen3.5 is supported. | |
| ## Quantized versions | |
| Every quant keeps the vision encoder, embeddings, norms, Gated DeltaNet gates and MTP weights in their original | |
| precision. Agreement is measured against these BF16 weights on 5,070 held-out assistant tokens from agent episodes. | |
| | Repo | Format | Size | Mean KL | Top-1 agreement | Browser tasks (581-task live set) | | |
| |---|---|---:|---:|---:|---| | |
| | this repo | BF16 | 9.3 GB | – | – | 45.3% | | |
| | [nib-4b-preview-FP8](https://huggingface.co/ProCreations/nib-4b-preview-FP8) | FP8 block 128×128 (SGLang) | 5.8 GB | 0.0028 | 98.9% | 42.5% (within run-to-run noise) | | |
| | [nib-4b-preview-NVFP4](https://huggingface.co/ProCreations/nib-4b-preview-NVFP4) | NVFP4 (SGLang, Blackwell) | 5.0 GB | 0.0147 | 97.0% | 45.6% | | |
| | [nib-4b-preview-EXL3](https://huggingface.co/ProCreations/nib-4b-preview-EXL3) | EXL3 3.05–6.05 bpw (exllamav3 / TabbyAPI) | 3.6–4.9 GB | 0.0257–0.0007 | 96.3–99.3% | not run | | |
| ## How Nib was trained | |
| 1. **Supervised fine-tuning** on browser-agent episodes that were verified as successful, from larger teacher models | |
| (Qwen3.8-27B-class and Qwen3.8-Flash-Next), across about ten harnesses, with screenshots. It also included | |
| demonstrations of reporting plainly when a task could not be done. | |
| 2. **On-policy distillation**: Nib ran tasks itself, and larger teachers scored its own tokens, first a browser-tuned | |
| Qwen3.8-27B, then Qwen3.8-Flash-Next. This preview is the third update of the final round, picked by a held-out live | |
| evaluation. Later updates and an honesty-penalty round did not score higher. | |
| Rewards and filters came from code checkers (synthetic tasks, app databases) and an LLM judge (live websites). The | |
| evaluation tasks were held out from training. The training data is not released. | |
| ## Limitations | |
| - **It often claims success when it failed.** In a held-out check, about 7 in 10 of its failed live-web tasks ended | |
| with a confident (wrong) answer rather than a plain "I could not do this". Verify important results. | |
| - **Behind larger models on the open web.** On live websites it trails the untrained Qwen3.8-27B by 11–18 points | |
| with thinking on, and by about 11 points on our harder 581-task live set. | |
| - **Small-model knowledge.** It can guess wrong URLs or facts that a bigger model would know, and should look things up | |
| instead. | |
| - Training and evaluation tasks were almost all in English. | |
| - These are our own benchmarks; live-web results drift as sites change. | |
| ## License | |
| Apache 2.0, the same as the base model [Qwen/Qwen3.5-4B](https://huggingface.co/Qwen/Qwen3.5-4B). | |