--- language: - en base_model: - Qwen/Qwen2.5-VL-3B-Instruct base_model_relation: finetune pipeline_tag: video-text-to-text library_name: transformers tags: - streamo - streaming-video - video-understanding - multimodal - qwen2_5_vl - arxiv:2512.21334 datasets: - maifoundations/Streamo-Instruct-465K --- # Streamo-3B **Streamo-3B** is a streaming video language model based on [Qwen2.5-VL-3B-Instruct](https://huggingface.co/Qwen/Qwen2.5-VL-3B-Instruct), developed for **Streaming Video Instruction Tuning**. It processes incoming video frames with conversational context and decides when to remain silent, wait for more evidence, or respond. [Paper](https://arxiv.org/abs/2512.21334) · [Project page](https://jiaerxia.github.io/Streamo/) · [Code](https://github.com/maifoundations/Streamo) · [Streamo-Instruct-465K](https://huggingface.co/datasets/maifoundations/Streamo-Instruct-465K) Streamo targets streaming tasks such as event narration, action understanding, temporal grounding, and time-sensitive question answering. This repository contains the model weights, tokenizer, and processor files. The inference implementation is maintained in the [Streamo code repository](https://github.com/maifoundations/Streamo). ## Model details | Item | Value | | --- | --- | | Base model | Qwen/Qwen2.5-VL-3B-Instruct | | Architecture | `Qwen2_5_VLForConditionalGeneration` | | Checkpoint precision | BF16 | | Weight format | Safetensors, two shards | | Inputs | Video frames, timestamps, and text instructions | | Outputs | Streaming state tokens and text responses | Load the tokenizer and processor from **this checkpoint** to retain the additional streaming tokens. ## Quick start ### 1. Install the dependencies Use a Linux CUDA environment with compatible [PyTorch and torchvision](https://pytorch.org/get-started/locally/) installations. Run the following commands in a dedicated Python environment: ```bash git clone https://github.com/maifoundations/Streamo.git cd Streamo python -m pip install -r requirements.txt \ "transformers==4.57.1" \ "qwen-vl-utils>=0.0.6,<0.0.12" \ decord opencv-python huggingface_hub ``` The checkpoint configuration records Transformers `4.57.1`, which is within the repository's supported version range. The bundled [Qwen model loader](https://github.com/maifoundations/Streamo/blob/main/swift/llm/model/model/qwen.py) requires `qwen-vl-utils<0.0.12`. Run inference from the cloned repository so that Python imports its bundled `swift` implementation. ### 2. Download the checkpoint ```bash # Authenticate first if repository access requires it. hf auth login hf download JiaerX/Streamo_3B --local-dir ./checkpoints/Streamo_3B ``` Download the complete repository, including both weight shards and the tokenizer/processor files. See the [Hugging Face CLI documentation](https://huggingface.co/docs/huggingface_hub/guides/cli) for authentication and download options. ### 3. Configure and run inference Edit the existing variables in the `__main__` block of [`inference.py`](https://github.com/maifoundations/Streamo/blob/main/inference.py): ```python infer_backend = 'pt' model = './checkpoints/Streamo_3B' video_path = './demo/cook.mp4' target_fps = 1.0 question = 'Detect and summarize each event sequence in the video.' global_question = True question_time = 0 max_rounds = 300 ``` Then run: ```bash python inference.py ``` The script uses configuration variables, not command-line flags. This example selects its PyTorch backend; the upstream default is `vllm`, which requires a separate vLLM installation compatible with your PyTorch, CUDA, and Transformers versions. GPU selection is set inside the script with `CUDA_VISIBLE_DEVICES='0'`; edit it there if needed. Keep `target_fps=1.0`: the current frame indexing and timestamp labels advance in one-second steps. `question_time` specifies the question's insertion round. With `global_question=True`, the question is included from the first frame and retained when the history window slides. Replace `video_path` and `question` for your own task. ## Streaming responses | Token | Meaning | | --- | --- | | `` | No relevant evidence yet. | | `` | Relevant activity is ongoing; more evidence is needed. | | `` | Enough evidence is available; an answer follows. | The script prints each round and appends one record per video run to `test_sample_video.jsonl`, containing `video_path`, `target_fps`, and an `output` mapping from round names to responses. Repeated runs append to the same file. ## Usage notes - The provided demo reads a local video file sequentially. A live camera application needs its own capture loop. - Context and GPU memory usage grow with retained frames and response history. Reduce `max_rounds` if necessary; a frame-count window alone does not guarantee that the prompt fits the backend's token limit. - The upstream vLLM configuration uses `max_model_len=32768`. Treat this as an inference setting, not a measured long-video capability of this checkpoint. - Sampling at 1 FPS can miss brief events. Model responses and timing may be inaccurate. See the [paper](https://arxiv.org/abs/2512.21334) for the evaluation protocol and reported results. ## License The [Streamo source code](https://github.com/maifoundations/Streamo/blob/main/LICENSE) is licensed under Apache-2.0. The base model has its own [Qwen Research License](https://huggingface.co/Qwen/Qwen2.5-VL-3B-Instruct/blob/main/LICENSE). ## Citation ```bibtex @inproceedings{xia2026streaming, title={Streaming video instruction tuning}, author={Xia, Jiaer and Chen, Peixian and Zhang, Mengdan and Sun, Xing and Zhou, Kaiyang}, booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition}, pages={31219--31229}, year={2026} } ```