Video-Text-to-Text
Transformers
Safetensors
English
qwen2
text-generation
Action
Video
MQA
multimodal
VLM
LLaVAction
MLLMs
Eval Results (legacy)
text-generation-inference
Instructions to use MLAdaptiveIntelligence/LLaVAction-7B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use MLAdaptiveIntelligence/LLaVAction-7B with Transformers:
# Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("MLAdaptiveIntelligence/LLaVAction-7B") model = AutoModelForCausalLM.from_pretrained("MLAdaptiveIntelligence/LLaVAction-7B", device_map="auto") - Notebooks
- Google Colab
- Kaggle
Update README.md
Browse files
README.md
CHANGED
|
@@ -95,6 +95,24 @@ model-index:
|
|
| 95 |
|
| 96 |
# LLaVAction-7B
|
| 97 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 98 |
## Model Summary
|
| 99 |
The LLaVAction-7B model is trained on EPIC-KITCHENS-100-MQA, based on Qwen2 language model with a context window of 32K tokens.
|
| 100 |
This model supports at most 64 frames.
|
|
|
|
| 95 |
|
| 96 |
# LLaVAction-7B
|
| 97 |
|
| 98 |
+
<div align="center">
|
| 99 |
+
<h2>LLaVAction: evaluating and training multi-modal large language models for action recognition
|
| 100 |
+
</h2>
|
| 101 |
+
|
| 102 |
+
[Shaokai Ye](https://jwyang.github.io/)<sup>*</sup><sup>1</sup>
|
| 103 |
+
[Haozhe Qi](https://cs-people.bu.edu/rxtan/)<sup>*</sup><sup>1</sup>
|
| 104 |
+
|
| 105 |
+
[Alexander Mathis](https://qianhuiwu.github.io/)<sup>1</sup><sup>†</sup>
|
| 106 |
+
[Mackenzie Weygandt Mathis](https://ruijiezheng.com/)<sup>1</sup><sup>†</sup><sup>‡</sup>
|
| 107 |
+
|
| 108 |
+
<sup>1</sup> EPFL
|
| 109 |
+
|
| 110 |
+
<sup>*</sup> First authors <sup>†</sup> Senior Authors <sup>‡</sup> Corresponding Author
|
| 111 |
+
|
| 112 |
+
\[[arXiv Paper](https://www.arxiv.org/tbd)\] \[[Project Page](https://mmathislab.github.io/llavaction/)\] \[[Github Repo](https://github.com/AdaptiveMotorControlLab/LLaVAction)\]
|
| 113 |
+
|
| 114 |
+
</div>
|
| 115 |
+
|
| 116 |
## Model Summary
|
| 117 |
The LLaVAction-7B model is trained on EPIC-KITCHENS-100-MQA, based on Qwen2 language model with a context window of 32K tokens.
|
| 118 |
This model supports at most 64 frames.
|