Instructions to use JiaerX/Streamo_3B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use JiaerX/Streamo_3B with Transformers:
# pip install -U transformers accelerate # Load model directly from transformers import AutoProcessor, AutoModelForMultimodalLM processor = AutoProcessor.from_pretrained("JiaerX/Streamo_3B") model = AutoModelForMultimodalLM.from_pretrained("JiaerX/Streamo_3B", device_map="auto") - Notebooks
- Google Colab
- Kaggle
Download README.md from JiaerX/Streamo_3B: direct link, hf CLI and curl.
- Browser
- Download file 5.82 kB
-
https://huggingface.co/JiaerX/Streamo_3B/resolve/main/README.md
- Command line
-
hf download hf://JiaerX/Streamo_3B/README.md
-
curl -L -o README.md https://huggingface.co/JiaerX/Streamo_3B/resolve/main/README.md
language:
- en
base_model:
- Qwen/Qwen2.5-VL-3B-Instruct
base_model_relation: finetune
pipeline_tag: video-text-to-text
library_name: transformers
tags:
- streamo
- streaming-video
- video-understanding
- multimodal
- qwen2_5_vl
- arxiv:2512.21334
datasets:
- maifoundations/Streamo-Instruct-465K
Streamo-3B
Streamo-3B is a streaming video language model based on Qwen2.5-VL-3B-Instruct, developed for Streaming Video Instruction Tuning. It processes incoming video frames with conversational context and decides when to remain silent, wait for more evidence, or respond.
Paper 路 Project page 路 Code 路 Streamo-Instruct-465K
Streamo targets streaming tasks such as event narration, action understanding, temporal grounding, and time-sensitive question answering. This repository contains the model weights, tokenizer, and processor files. The inference implementation is maintained in the Streamo code repository.
Model details
| Item | Value |
|---|---|
| Base model | Qwen/Qwen2.5-VL-3B-Instruct |
| Architecture | Qwen2_5_VLForConditionalGeneration |
| Checkpoint precision | BF16 |
| Weight format | Safetensors, two shards |
| Inputs | Video frames, timestamps, and text instructions |
| Outputs | Streaming state tokens and text responses |
Load the tokenizer and processor from this checkpoint to retain the additional streaming tokens.
Quick start
1. Install the dependencies
Use a Linux CUDA environment with compatible PyTorch and torchvision installations. Run the following commands in a dedicated Python environment:
git clone https://github.com/maifoundations/Streamo.git
cd Streamo
python -m pip install -r requirements.txt \
"transformers==4.57.1" \
"qwen-vl-utils>=0.0.6,<0.0.12" \
decord opencv-python huggingface_hub
The checkpoint configuration records Transformers 4.57.1, which is within the repository's supported version range. The bundled Qwen model loader requires qwen-vl-utils<0.0.12. Run inference from the cloned repository so that Python imports its bundled swift implementation.
2. Download the checkpoint
# Authenticate first if repository access requires it.
hf auth login
hf download JiaerX/Streamo_3B --local-dir ./checkpoints/Streamo_3B
Download the complete repository, including both weight shards and the tokenizer/processor files. See the Hugging Face CLI documentation for authentication and download options.
3. Configure and run inference
Edit the existing variables in the __main__ block of inference.py:
infer_backend = 'pt'
model = './checkpoints/Streamo_3B'
video_path = './demo/cook.mp4'
target_fps = 1.0
question = 'Detect and summarize each event sequence in the video.'
global_question = True
question_time = 0
max_rounds = 300
Then run:
python inference.py
The script uses configuration variables, not command-line flags. This example selects its PyTorch backend; the upstream default is vllm, which requires a separate vLLM installation compatible with your PyTorch, CUDA, and Transformers versions. GPU selection is set inside the script with CUDA_VISIBLE_DEVICES='0'; edit it there if needed.
Keep target_fps=1.0: the current frame indexing and timestamp labels advance in one-second steps. question_time specifies the question's insertion round. With global_question=True, the question is included from the first frame and retained when the history window slides. Replace video_path and question for your own task.
Streaming responses
| Token | Meaning |
|---|---|
</Silence> |
No relevant evidence yet. |
</Standby> |
Relevant activity is ongoing; more evidence is needed. |
</Response> |
Enough evidence is available; an answer follows. |
The script prints each round and appends one record per video run to test_sample_video.jsonl, containing video_path, target_fps, and an output mapping from round names to responses. Repeated runs append to the same file.
Usage notes
- The provided demo reads a local video file sequentially. A live camera application needs its own capture loop.
- Context and GPU memory usage grow with retained frames and response history. Reduce
max_roundsif necessary; a frame-count window alone does not guarantee that the prompt fits the backend's token limit. - The upstream vLLM configuration uses
max_model_len=32768. Treat this as an inference setting, not a measured long-video capability of this checkpoint. - Sampling at 1 FPS can miss brief events. Model responses and timing may be inaccurate. See the paper for the evaluation protocol and reported results.
License
The Streamo source code is licensed under Apache-2.0. The base model has its own Qwen Research License.
Citation
@inproceedings{xia2026streaming,
title={Streaming video instruction tuning},
author={Xia, Jiaer and Chen, Peixian and Zhang, Mengdan and Sun, Xing and Zhou, Kaiyang},
booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
pages={31219--31229},
year={2026}
}