Commit History
2 days ago
-
Add LongShOTBench evaluation results
Verified signatureAdd the LongShOTBench (NeurIPS 2026 Evaluations & Datasets Track) benchmark authors' published results for VideoChat-R1_7B on overall, core_perception, reasoning, information and multimodal. Each value is a rubric-weighted score on a 0-100 scale; higher is better. LongShOTBench evaluates long-form omni-modal video understanding: 3,401 open-ended single- and multi-turn samples (4,882 QA turns) over 274 real-world videos (~41 min avg) that require combining visual, speech and ambient-audio evidence. Answers are graded by an LLM judge (Qwen3-14B) against human-verified, weighted criterion-level rubrics. Source: https://longshot.cvmbzuai.com/leaderboard Benchmark: https://huggingface.co/datasets/MBZUAI/longshot-bench Paper: https://arxiv.org/abs/2512.169788cc0d81
