Instructions to use mis2/ViSS-8B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use mis2/ViSS-8B with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="mis2/ViSS-8B")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("mis2/ViSS-8B")
model = AutoModelForMultimodalLM.from_pretrained("mis2/ViSS-8B", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use mis2/ViSS-8B with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "mis2/ViSS-8B"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "mis2/ViSS-8B",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/mis2/ViSS-8B

SGLang

How to use mis2/ViSS-8B with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "mis2/ViSS-8B" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "mis2/ViSS-8B",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "mis2/ViSS-8B" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "mis2/ViSS-8B",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use mis2/ViSS-8B with Docker Model Runner:
```
docker model run hf.co/mis2/ViSS-8B
```

ViSS-8B / code /docs /TRAINING.md

mis2

Release ViSS-8B weights, processors and source code

78ac017 verified 21 days ago

preview code

Raw History Blame Contribute Delete

1.79 kB

# Training ViSS

Install the training environment described in [Installation](INSTALL.md) and
prepare a sealed training release as described in [Data preparation](DATA.md).

## Paper recipe

The manuscript specifies one epoch, learning rate `2e-6`, cosine scheduling,
warmup ratio `0.03`, seed `59`, maximum length `8192`, maximum image pixels
`200704`, and a global batch of `12`. The vision tower is frozen; the language
model and multimodal projector are trainable.

For two GPUs, a microbatch of one and six accumulation steps give the stated
global batch:

```bash
torchrun --nproc_per_node=2 -m experiments.train_qwen3vl_sft \
  --model checkpoints/Qwen3-VL-8B-Instruct \
  --train data/viss_release/train.jsonl \
  --output checkpoints/viss \
  --epochs 1 \
  --lr 2e-6 \
  --lr-scheduler-type cosine \
  --warmup-ratio 0.03 \
  --per-device-batch-size 1 \
  --gradient-accumulation-steps 6 \
  --max-length 8192 \
  --max-pixels 200704 \
  --seed 59 \
  --train-order sequential \
  --freeze-vision-tower \
  --train-vision-projector \
  --attn-implementation flash_attention_2 \
  --deepspeed configs/training/deepspeed_zero3_bf16.json
```

Replace `train.jsonl` with the actual row file inside the sealed release.
The sequential setting preserves prepared row order. The paper preparation
uses a parent stage followed by capability rows, with stage shuffle seed 59;
that ordering must already be reflected in the input release.

Changing the GPU count requires adjusting accumulation if the global batch
is to remain fixed. Check the trainer's complete options with:

```bash
python -m experiments.train_qwen3vl_sft --help
```

The training entry point requires Transformers even for `--help`. Training
outputs and logs are generated at runtime and are not part of this archive.

	# Training ViSS

	Install the training environment described in [Installation](INSTALL.md) and
	prepare a sealed training release as described in [Data preparation](DATA.md).

	## Paper recipe

	The manuscript specifies one epoch, learning rate `2e-6`, cosine scheduling,
	warmup ratio `0.03`, seed `59`, maximum length `8192`, maximum image pixels
	`200704`, and a global batch of `12`. The vision tower is frozen; the language
	model and multimodal projector are trainable.

	For two GPUs, a microbatch of one and six accumulation steps give the stated
	global batch:

	```bash
	torchrun --nproc_per_node=2 -m experiments.train_qwen3vl_sft \
	--model checkpoints/Qwen3-VL-8B-Instruct \
	--train data/viss_release/train.jsonl \
	--output checkpoints/viss \
	--epochs 1 \
	--lr 2e-6 \
	--lr-scheduler-type cosine \
	--warmup-ratio 0.03 \
	--per-device-batch-size 1 \
	--gradient-accumulation-steps 6 \
	--max-length 8192 \
	--max-pixels 200704 \
	--seed 59 \
	--train-order sequential \
	--freeze-vision-tower \
	--train-vision-projector \
	--attn-implementation flash_attention_2 \
	--deepspeed configs/training/deepspeed_zero3_bf16.json
	```

	Replace `train.jsonl` with the actual row file inside the sealed release.
	The sequential setting preserves prepared row order. The paper preparation
	uses a parent stage followed by capability rows, with stage shuffle seed 59;
	that ordering must already be reflected in the input release.

	Changing the GPU count requires adjusting accumulation if the global batch
	is to remain fixed. Check the trainer's complete options with:

	```bash
	python -m experiments.train_qwen3vl_sft --help
	```

	The training entry point requires Transformers even for `--help`. Training
	outputs and logs are generated at runtime and are not part of this archive.