base_model: Qwen/Qwen3-Omni-30B-A3B-Instruct
library_name: transformers
tags:
- qwen3-omni
- audio
- video
- multimodal
- temporal-grounding
- dense-video-captioning
- reinforcement-learning
AVTime-RL
AVTime-RL is a merged, bf16 model based on
Qwen/Qwen3-Omni-30B-A3B-Instruct for time-aware audio-video understanding,
including timestamped dense video captioning, temporal video grounding,
and segment captioning.
Milestone identity
| Field | Value |
|---|---|
| Release | AVTime-RL |
| Precision | bfloat16 |
| Serialization | 15 safetensors shards, about 66 GB total |
| Architecture | Qwen3OmniMoeForConditionalGeneration |
| Release date | 2026-08-11 |
This is a fully merged checkpoint; no LoRA adapter needs to be applied at
inference time. Use a recent transformers or vLLM build that supports
Qwen3-Omni MoE models and provide both video and audio when available.
LongVALE evaluation
The following results use the LongVALE test split (1,171 videos and 13,867 events) and the official metric implementation.
| Task | Metric | Score |
|---|---|---|
| Omni-TVG | R@0.3 | 26.13 |
| Omni-TVG | R@0.5 | 16.85 |
| Omni-TVG | R@0.7 | 8.71 |
| Omni-TVG | mIoU | 17.76 |
| Omni-SC | BLEU-4 | 7.79 |
| Omni-SC | ROUGE-L | 24.84 |
| Omni-SC | CIDEr | 31.28 |
| Omni-SC | METEOR | 13.20 |
On the clean 512-video DVC evaluation using the historical one-round generation protocol, the model achieves SODA-C 5.057, METEOR 7.033, and CIDEr 12.156.
For TVG, evaluation used constrained decoding to enforce LongVALE's official
From XX to YY response syntax. Fair comparisons should use the same output
constraint for every compared model; this checkpoint should not be interpreted
as a claim about unconstrained free-form timestamp formatting.
Limitations and use
This is a research checkpoint, not a safety-validated production system. Its timestamps and captions can be inaccurate, especially for subtle audio events, very long videos, or videos outside the training distribution. The checkpoint does not include any LongVALE or AVTime media or annotations. Users must comply with the terms of the upstream base model and any input datasets.