Instructions to use mis2/ViSS-8B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use mis2/ViSS-8B with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="mis2/ViSS-8B")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("mis2/ViSS-8B")
model = AutoModelForMultimodalLM.from_pretrained("mis2/ViSS-8B", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use mis2/ViSS-8B with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "mis2/ViSS-8B"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "mis2/ViSS-8B",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/mis2/ViSS-8B

SGLang

How to use mis2/ViSS-8B with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "mis2/ViSS-8B" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "mis2/ViSS-8B",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "mis2/ViSS-8B" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "mis2/ViSS-8B",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use mis2/ViSS-8B with Docker Model Runner:
```
docker model run hf.co/mis2/ViSS-8B
```

ViSS-8B / code /docs /DATA.md

mis2

Release ViSS-8B weights, processors and source code

78ac017 verified 21 days ago

preview code

Raw History Blame Contribute Delete

2.58 kB

# Data preparation

The code package distributes construction and validation code. Training
records, source media, benchmark exports, and calibration artifacts are
external inputs.

## Training records

The trainer consumes a sealed semantic release. Its validation checks the
manifest, source inventory, user/assistant message schema, media identity,
and permitted targets. See:

- `viss_core/data_v0/schema.py`
- `viss_core/data_v0/semantic_release.py`
- `viss_core/data_v0/trainer_adapter.py`

The source-aligned builder combines already prepared source releases:

```bash
python -m data_pipeline.temporal_scope.build_source_aligned_sft_release \
  --source data/prepared_source_a \
  --source data/prepared_source_b \
  --output data/viss_release \
  --release-id my_viss_release \
  --copy-mode copy
```

`prepared_source_a` and `prepared_source_b` are placeholders for valid
source releases, not raw-video directories. This generic example does not
reconstruct the paper's accepted training mixture by itself.

The paper describes the training mixture. Company training records and their
manifests are not distributed. The training guide assumes users have prepared
their own valid source releases, media, example weights, and ordering.

Set `VISS_UPSTREAM_MANIFEST_SHA256` to the trusted upstream identity for your
training inputs before starting the builder or trainer. The validator retains
the identity check, but the code package embeds no company-specific digest.
Private legacy source adapters are excluded; supply the standard semantic
release format instead.

## Evaluation assets

For each benchmark, prepare:

1. Official annotations and authorized video files.
2. A query export using the fields read by `viss_core/benchmarks.py`.
3. The benchmark's official scoring installation.
4. Model and processor files compatible with the locked evaluation protocol.

`configs/evaluation/recent4.example.json` describes the public sampling,
query-count, prompt, and processor conventions. It contains no run dates,
company file identities, or query records. Follow the evaluation guide to
create a local configuration bound to your own query exports and processor.
The generated configuration is excluded from publication.

## TVS calibration

The TVS policy consumes acquisition/commit calibration artifacts and aligned
view predictions from one model. Calibrate on source-disjoint data, then
freeze the thresholds for benchmark evaluation. The archive includes the
selection and analysis implementation, but not the paper's calibration
records or precomputed predictions.

	# Data preparation

	The code package distributes construction and validation code. Training
	records, source media, benchmark exports, and calibration artifacts are
	external inputs.

	## Training records

	The trainer consumes a sealed semantic release. Its validation checks the
	manifest, source inventory, user/assistant message schema, media identity,
	and permitted targets. See:

	- `viss_core/data_v0/schema.py`
	- `viss_core/data_v0/semantic_release.py`
	- `viss_core/data_v0/trainer_adapter.py`

	The source-aligned builder combines already prepared source releases:

	```bash
	python -m data_pipeline.temporal_scope.build_source_aligned_sft_release \
	--source data/prepared_source_a \
	--source data/prepared_source_b \
	--output data/viss_release \
	--release-id my_viss_release \
	--copy-mode copy
	```

	`prepared_source_a` and `prepared_source_b` are placeholders for valid
	source releases, not raw-video directories. This generic example does not
	reconstruct the paper's accepted training mixture by itself.

	The paper describes the training mixture. Company training records and their
	manifests are not distributed. The training guide assumes users have prepared
	their own valid source releases, media, example weights, and ordering.

	Set `VISS_UPSTREAM_MANIFEST_SHA256` to the trusted upstream identity for your
	training inputs before starting the builder or trainer. The validator retains
	the identity check, but the code package embeds no company-specific digest.
	Private legacy source adapters are excluded; supply the standard semantic
	release format instead.

	## Evaluation assets

	For each benchmark, prepare:

	1. Official annotations and authorized video files.
	2. A query export using the fields read by `viss_core/benchmarks.py`.
	3. The benchmark's official scoring installation.
	4. Model and processor files compatible with the locked evaluation protocol.

	`configs/evaluation/recent4.example.json` describes the public sampling,
	query-count, prompt, and processor conventions. It contains no run dates,
	company file identities, or query records. Follow the evaluation guide to
	create a local configuration bound to your own query exports and processor.
	The generated configuration is excluded from publication.

	## TVS calibration

	The TVS policy consumes acquisition/commit calibration artifacts and aligned
	view predictions from one model. Calibrate on source-disjoint data, then
	freeze the thresholds for benchmark evaluation. The archive includes the
	selection and analysis implementation, but not the paper's calibration
	records or precomputed predictions.