| --- |
| base_model: Qwen/Qwen3-Omni-30B-A3B-Instruct |
| library_name: transformers |
| tags: |
| - qwen3-omni |
| - audio |
| - video |
| - multimodal |
| - temporal-grounding |
| - dense-video-captioning |
| - reinforcement-learning |
| --- |
| |
| # AVTime-RL |
|
|
| AVTime-RL is a merged, bf16 model based on |
| `Qwen/Qwen3-Omni-30B-A3B-Instruct` for time-aware audio-video understanding, |
| including timestamped dense video captioning, temporal video grounding, |
| and segment captioning. |
|
|
| ## Milestone identity |
|
|
| | Field | Value | |
| | --- | --- | |
| | Release | AVTime-RL | |
| | Precision | bfloat16 | |
| | Serialization | 15 safetensors shards, about 66 GB total | |
| | Architecture | `Qwen3OmniMoeForConditionalGeneration` | |
| | Release date | 2026-08-11 | |
|
|
| This is a fully merged checkpoint; no LoRA adapter needs to be applied at |
| inference time. Use a recent `transformers` or vLLM build that supports |
| Qwen3-Omni MoE models and provide both video and audio when available. |
|
|
| ## LongVALE evaluation |
|
|
| The following results use the LongVALE test split (1,171 videos and 13,867 |
| events) and the official metric implementation. |
|
|
| | Task | Metric | Score | |
| | --- | ---: | ---: | |
| | Omni-TVG | R@0.3 | 26.13 | |
| | Omni-TVG | R@0.5 | 16.85 | |
| | Omni-TVG | R@0.7 | 8.71 | |
| | Omni-TVG | mIoU | 17.76 | |
| | Omni-SC | BLEU-4 | 7.79 | |
| | Omni-SC | ROUGE-L | 24.84 | |
| | Omni-SC | CIDEr | 31.28 | |
| | Omni-SC | METEOR | 13.20 | |
|
|
| On the clean 512-video DVC evaluation using the historical one-round |
| generation protocol, the model achieves SODA-C 5.057, METEOR 7.033, and |
| CIDEr 12.156. |
|
|
| For TVG, evaluation used constrained decoding to enforce LongVALE's official |
| `From XX to YY` response syntax. Fair comparisons should use the same output |
| constraint for every compared model; this checkpoint should not be interpreted |
| as a claim about unconstrained free-form timestamp formatting. |
|
|
| ## Limitations and use |
|
|
| This is a research checkpoint, not a safety-validated production system. Its |
| timestamps and captions can be inaccurate, especially for subtle audio events, |
| very long videos, or videos outside the training distribution. The checkpoint |
| does not include any LongVALE or AVTime media or annotations. Users must |
| comply with the terms of the upstream base model and any input datasets. |
|
|
|
|
