---
license: apache-2.0
base_model: tencent/WeVisOne-VL-4B
pipeline_tag: image-text-to-text
tags:
  - comfyui
  - custom-node
  - object-detection
  - referring-expression-comprehension
  - object-counting
  - point-grounding
  - ocr
  - image-matting
---

# ComfyUI-WeVisOne-VL

ComfyUI nodes for [tencent/WeVisOne-VL-4B](https://huggingface.co/tencent/WeVisOne-VL-4B): open-vocabulary detection, referring localization, pointing, counting, OCR, and promptable alpha matting.

The model is a Qwen3-VL-4B with 1010 extra tokens and its own `lm_head`, plus a 9.6M-parameter pixel decoder. The pack reuses ComfyUI's native Qwen3-VL implementation, and the decoder runs on ComfyUI ops.

## Examples

| | | |
| --- | --- | --- |
| ![](examples/01_detection_dogs.jpg) detection: `dog` | ![](examples/02_detection_livingroom.jpg) detection: `sofa` `pillow` `vase` `painting` | ![](examples/03_detection_boys.jpg) detection: `guitar` `drum` `microphone` |
| ![](examples/04_referring_boys.jpg) referring: `the boy wearing glasses` | ![](examples/05_count_point_chess.jpg) count_point: `white chess pieces` (15) | ![](examples/06_ocr_coffee_sign.jpg) ocr |
| ![](examples/07_matting_seabreeze.jpg) matting: `person` | ![](examples/08_matting_parasol.jpg) matting: `the red parasol` | ![](examples/09_matting_boys.jpg) matting: `the boy playing the drums` |

Input images are the demo images of [Tencent/WeVisOne-VL](https://github.com/Tencent/WeVisOne-VL). The overlays are drawn from the node's raw output; mattes are shown by dimming the background.

## Install

Clone the pack without the weights, then download the weights into `models/text_encoders`:

```bash
GIT_LFS_SKIP_SMUDGE=1 git clone https://huggingface.co/paralaif/ComfyUI-WeVisOne-VL custom_nodes/ComfyUI-WeVisOne-VL
hf download paralaif/ComfyUI-WeVisOne-VL wevisone_vl_4b_int8_convrot.safetensors --local-dir models/text_encoders
```

| file | size | |
| --- | --- | --- |
| `wevisone_vl_4b_int8_convrot.safetensors` | 5.7 GB | int8 + ConvRot language and vision linears (comfy-kitchen format), about 3× faster than bf16 on a 16 GB GPU; boxes within 2/1000 and mattes at 0.999 IoU of bf16 |
| `wevisone_vl_4b_bf16.safetensors` | 9.7 GB | the original bf16 weights |

Both files bundle the fp32 pixel decoder. To build the bf16 file yourself from the original release:

```bash
hf download tencent/WeVisOne-VL-4B --local-dir WeVisOne-VL-4B
python custom_nodes/ComfyUI-WeVisOne-VL/convert_wevisone.py WeVisOne-VL-4B models/text_encoders/wevisone_vl_4b_bf16.safetensors
```

## Nodes

- **WeVisOne-VL Loader** loads the packed file from `models/text_encoders` and outputs a `CLIP`. The core **Generate Text** node also works with it.
- **WeVisOne-VL Perceive** takes an image, a task and a query. It outputs:
  - the raw response;
  - `bboxes`, compatible with SAM3 Detect and Draw BBoxes;
  - `points` as JSON, usable as SAM3 point prompts;
  - `mask`: the alpha mattes for the `matting` task, or filled boxes for the other box tasks. It gives their union per image, or one mask per instance with `individual_masks`.

| task | query |
| --- | --- |
| detection, point | labels, one per line |
| referring, matting | a referring phrase |
| count_box, count_point | the object to count |
| ocr | — |
| ocr_grounding | the text to find |
| freeform | the full prompt |

Decoding is greedy with the reference repetition penalty of 1.05. The pixel decoder runs in fp32 on a 1024×1024 canvas and needs about 2 GB of VRAM per instance.

`example_workflows/WeVisOne-VL-Perceive.json` mattes a person and builds a transparent cutout; copy `example_workflows/wevisone_seabreeze.jpg` into `input/` to run it as is.

## License

Apache 2.0. The weights are converted from [tencent/WeVisOne-VL-4B](https://huggingface.co/tencent/WeVisOne-VL-4B) (Apache 2.0), and the pixel decoder is ported from the [reference implementation](https://github.com/Tencent/WeVisOne-VL).

```bibtex
@article{liang2026wevisone,
  title={WeVisOne-VL: Unified Visual Perception from Objects to Pixels},
  author={Liang, Tianming and Pan, Junyi and Li, Yuxi and Fu, Canmiao and Huang, Zhipeng and Sun, Chong and Li, Chen and Lyu, Jing},
  journal={arXiv preprint arXiv:2610.31329},
  year={2026}
}
```