Instructions to use inclusionAI/ZwZ-4B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use inclusionAI/ZwZ-4B with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("image-text-to-text", model="inclusionAI/ZwZ-4B") messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] pipe(text=messages)# pip install -U transformers accelerate # Load model directly from transformers import AutoProcessor, AutoModelForMultimodalLM processor = AutoProcessor.from_pretrained("inclusionAI/ZwZ-4B") model = AutoModelForMultimodalLM.from_pretrained("inclusionAI/ZwZ-4B", device_map="auto") messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] inputs = processor.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=256) print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use inclusionAI/ZwZ-4B with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "inclusionAI/ZwZ-4B" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "inclusionAI/ZwZ-4B", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker
docker model run hf.co/inclusionAI/ZwZ-4B
- SGLang
How to use inclusionAI/ZwZ-4B with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "inclusionAI/ZwZ-4B" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "inclusionAI/ZwZ-4B", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "inclusionAI/ZwZ-4B" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "inclusionAI/ZwZ-4B", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }' - Docker Model Runner
How to use inclusionAI/ZwZ-4B with Docker Model Runner:
docker model run hf.co/inclusionAI/ZwZ-4B
Upload folder using huggingface_hub
Browse files
README.md
CHANGED
|
@@ -11,9 +11,14 @@
|
|
| 11 |
**ZwZ-4B** is a fine-grained multimodal perception model built upon [Qwen3-VL-4B](https://huggingface.co/Qwen/Qwen3-VL-4B). It is trained using **Region-to-Image Distillation (R2I)** combined with reinforcement learning, enabling superior fine-grained visual understanding in a single forward pass — no inference-time zooming or tool calling required. ZwZ-4B achieves state-of-the-art performance on fine-grained perception benchmarks among open-source models of comparable size.
|
| 12 |
|
| 13 |
<div align=center>
|
| 14 |
-
<img src="gp_avg_comparison.png" width="90%" alt="avg_comparison"/>
|
| 15 |
</div>
|
| 16 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 17 |
|
| 18 |
## How It Works
|
| 19 |
|
|
@@ -96,4 +101,4 @@ ZwZ-4B is trained on [inclusionAI/ZwZ-RL-VQA](https://huggingface.co/datasets/in
|
|
| 96 |
|
| 97 |
## License
|
| 98 |
|
| 99 |
-
This model follows the license of
|
|
|
|
| 11 |
**ZwZ-4B** is a fine-grained multimodal perception model built upon [Qwen3-VL-4B](https://huggingface.co/Qwen/Qwen3-VL-4B). It is trained using **Region-to-Image Distillation (R2I)** combined with reinforcement learning, enabling superior fine-grained visual understanding in a single forward pass — no inference-time zooming or tool calling required. ZwZ-4B achieves state-of-the-art performance on fine-grained perception benchmarks among open-source models of comparable size.
|
| 12 |
|
| 13 |
<div align=center>
|
| 14 |
+
<img src="assets/gp_avg_comparison.png" width="90%" alt="avg_comparison"/>
|
| 15 |
</div>
|
| 16 |
|
| 17 |
+
## Key Features
|
| 18 |
+
|
| 19 |
+
- **⚡ Single-Pass Efficiency**: Achieves fine-grained perception in one forward pass, eliminating inference-time tool-calling overhead
|
| 20 |
+
- **🎯 Superior Accuracy**: State-of-the-art on perception benchmarks among open-source models
|
| 21 |
+
- **📈 Broad Improvements**: Enhances not only perception benchmarks but also out-of-distribution generalization on visual reasoning, GUI agent, and AIGC detection
|
| 22 |
|
| 23 |
## How It Works
|
| 24 |
|
|
|
|
| 101 |
|
| 102 |
## License
|
| 103 |
|
| 104 |
+
This model follows the license of [Qwen3-VL-4B](https://huggingface.co/Qwen/Qwen3-VL-4B). Please refer to the base model's license for usage terms.
|