Video-Text-to-Text
Transformers
Safetensors
English
Chinese
moss_vl
feature-extraction
Realtime
Streaming
Video-Understanding
Image-Understanding
MOSS-VL
OpenMOSS
multimodal
video
vision-language
custom_code
Instructions to use OpenMOSS-Team/MOSS-VL-Realtime with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use OpenMOSS-Team/MOSS-VL-Realtime with Transformers:
# pip install -U transformers accelerate # Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("OpenMOSS-Team/MOSS-VL-Realtime", trust_remote_code=True, device_map="auto") - Notebooks
- Google Colab
- Kaggle
Fix query RoPE when reusing cached vision keys
Browse filesApply query RoPE on every cross-attention step, including cached-vision steps. Rotate new vision keys only once. Preserve the current NPU, idle-wait and text-cleanup code. No weights, configs, processors or tokenizer files changed. CPU fresh/cached regression passed.
- modeling_moss_vl.py +6 -4
modeling_moss_vl.py
CHANGED
|
@@ -644,6 +644,11 @@ class MossVLTextCrossAttention(nn.Module):
|
|
| 644 |
query_states = query_states.view(batch_size, seq_length, self.num_heads, self.head_dim).transpose(1, 2)
|
| 645 |
query_states = self.q_norm(query_states)
|
| 646 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 647 |
if cross_attention_states is not None:
|
| 648 |
# Key and Value from vision cross_attention_states
|
| 649 |
key_states = self.k_proj(cross_attention_states)
|
|
@@ -653,10 +658,7 @@ class MossVLTextCrossAttention(nn.Module):
|
|
| 653 |
key_states = self.k_norm(key_states)
|
| 654 |
value_states = value_states.view(batch_size, -1, self.num_key_value_heads, self.head_dim).transpose(1, 2)
|
| 655 |
|
| 656 |
-
#
|
| 657 |
-
if query_position_embeddings is not None:
|
| 658 |
-
cos, sin = query_position_embeddings
|
| 659 |
-
query_states = apply_rotary_pos_emb_cross_attention(query_states, cos, sin)
|
| 660 |
|
| 661 |
if vision_position_embeddings is not None:
|
| 662 |
vision_cos, vision_sin = vision_position_embeddings
|
|
|
|
| 644 |
query_states = query_states.view(batch_size, seq_length, self.num_heads, self.head_dim).transpose(1, 2)
|
| 645 |
query_states = self.q_norm(query_states)
|
| 646 |
|
| 647 |
+
# Every new query needs RoPE, including reads from the vision KV cache.
|
| 648 |
+
if query_position_embeddings is not None:
|
| 649 |
+
cos, sin = query_position_embeddings
|
| 650 |
+
query_states = apply_rotary_pos_emb_cross_attention(query_states, cos, sin)
|
| 651 |
+
|
| 652 |
if cross_attention_states is not None:
|
| 653 |
# Key and Value from vision cross_attention_states
|
| 654 |
key_states = self.k_proj(cross_attention_states)
|
|
|
|
| 658 |
key_states = self.k_norm(key_states)
|
| 659 |
value_states = value_states.view(batch_size, -1, self.num_key_value_heads, self.head_dim).transpose(1, 2)
|
| 660 |
|
| 661 |
+
# Rotate new keys once; cached keys already include vision RoPE.
|
|
|
|
|
|
|
|
|
|
| 662 |
|
| 663 |
if vision_position_embeddings is not None:
|
| 664 |
vision_cos, vision_sin = vision_position_embeddings
|