VideoJev-0.8B

VideoJev-0.8B adapts Qwen3.5-0.8B to make structured video decisions: localizing events in time and selecting answers to multiple-choice questions. It scores candidates with a shared decision head instead of generating answer tokens.

Paper · Homepage · Code · Usage guide

Benchmark Results

All values are percentages. Grounding uses mIoU; MLVU uses M-Avg; the other QA benchmarks use accuracy. Both columns use this checkpoint.

Benchmark Metric Cached Joint
Charades-STA mIoU 49.83 50.48
ActivityNet Captions mIoU 52.01 52.69
QVHighlights mIoU 64.05 63.98
LVBench accuracy 53.78 54.81
Video-MME accuracy 60.33 60.70
LongVideoBench accuracy 61.11 60.28
MLVU M-Avg 62.05 64.13

Quick start

Use Python 3.12, PyTorch 2.8 with CUDA 12.8, and the following packages. Install the PyTorch build for your CUDA environment first.

pip install transformers==5.14.1 accelerate safetensors numpy pillow av==18.1.0 huggingface_hub \
  flash-linear-attention==0.4.2 fla-core==0.4.2
pip install flash-attn causal-conv1d --no-build-isolation

The repository contains the complete visual encoder, language backbone, decision head, tokenizer, and processor. Weights follow the corresponding Qwen tensor dtypes; the decision head is stored in FP32. CUDA inference uses BF16 autocast and FP32 decision scoring. The example uses Joint inference. While this repository is private, authenticate with an account that has access before downloading.

import json
import torch
from huggingface_hub import hf_hub_download
from transformers import AutoModel, AutoProcessor

repo = "ZhuYuhan/VideoJev-0.8B"
video = hf_hub_download(
    repo_id="ZhuYuhan/VideoJev-Demo",
    filename="eating_spaghetti.mp4",
    repo_type="dataset",
    revision="7e2b9685bca592c3b929268f6fcf82c4ad74b267",
)
processor = AutoProcessor.from_pretrained(repo, trust_remote_code=True)
model = AutoModel.from_pretrained(
    repo, trust_remote_code=True, dtype=torch.bfloat16,
    device_map={"": "cuda:0"}, attn_implementation="flash_attention_2",
)
model.prepare_for_inference()

questions = [
    {"id": "event", "task": "grounding",
     "query": "The woman in green eats pasta."},
    {"id": "question", "task": "choice",
     "query": "What is the woman in green eating?",
     "options": ["A. Pasta.", "B. Rice.", "C. Soup."]},
]
results = model.predict_video(
    processor, video, questions, fps=2.0, max_frames=2048, context=8192,
)
print(json.dumps(results, indent=2))

The short demo uses 2 FPS and an 8K context. Use the task-specific settings above for benchmark evaluation. Long videos require more memory than this short example.

Output

Each question returns its id, task, and candidate probabilities. Grounding also returns an interval in seconds. QA returns a zero-based answer_index and the selected answer. Grounding scores are independent sigmoid probabilities; answer-choice scores use softmax.

See USAGE.md for input settings and evaluation_results.json for exact scores and sample counts.

Citation

@article{zhu2026videojev,
  title={VideoJev: System One Model for Video Understanding and Interaction},
  author={Zhu, Yuhan and Zeng, Xiangyu and Si, Qingyi and Yang, Chenxu and Li, Yifei and Qin, Luozheng and Zhou, Junhao and Yao, Dingyu and Zhang, Jun and Qin, Chuanyu and others},
  journal={arXiv preprint arXiv:2610.18615},
  year={2026}
}
Downloads last month
77
Safetensors
Model size
0.8B params
Tensor type
F32
·
BF16
·
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for ZhuYuhan/VideoJev-0.8B

Finetuned
(911)
this model

Paper for ZhuYuhan/VideoJev-0.8B