Visual Question Answering
Transformers
English
videollama2_mixtral
text-generation
multimodal large language model
large video-language model
Instructions to use DAMO-NLP-SG/VideoLLaMA2-8x7B-Base with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use DAMO-NLP-SG/VideoLLaMA2-8x7B-Base with Transformers:
# Use a pipeline as a high-level helper # Warning: Pipeline type "visual-question-answering" is no longer supported in transformers v5. # You must load the model directly (see below) or downgrade to v4.x with: # pip install "transformers<5.0.0" from transformers import pipeline pipe = pipeline("visual-question-answering", model="DAMO-NLP-SG/VideoLLaMA2-8x7B-Base")# pip install -U transformers accelerate # Load model directly from transformers import AutoModelForCausalLM model = AutoModelForCausalLM.from_pretrained("DAMO-NLP-SG/VideoLLaMA2-8x7B-Base", device_map="auto") - Notebooks
- Google Colab
- Kaggle
Update README.md
Browse files
README.md
CHANGED
|
@@ -81,7 +81,7 @@ def inference():
|
|
| 81 |
# The woman in the image is wearing a black coat and sunglasses, and she is walking down a rain-soaked city street. The image feels vibrant and lively, with the bright city lights reflecting off the wet pavement, creating a visually appealing atmosphere. The woman's presence adds a sense of style and confidence to the scene, as she navigates the bustling urban environment.
|
| 82 |
modal_list = ['image']
|
| 83 |
# 1. Initialize the model.
|
| 84 |
-
model_path = 'DAMO-NLP-SG/VideoLLaMA2-
|
| 85 |
model_name = get_model_name_from_path(model_path)
|
| 86 |
tokenizer, model, processor, context_len = load_pretrained_model(model_path, None, model_name)
|
| 87 |
model = model.to('cuda:0')
|
|
|
|
| 81 |
# The woman in the image is wearing a black coat and sunglasses, and she is walking down a rain-soaked city street. The image feels vibrant and lively, with the bright city lights reflecting off the wet pavement, creating a visually appealing atmosphere. The woman's presence adds a sense of style and confidence to the scene, as she navigates the bustling urban environment.
|
| 82 |
modal_list = ['image']
|
| 83 |
# 1. Initialize the model.
|
| 84 |
+
model_path = 'DAMO-NLP-SG/VideoLLaMA2-8x7B-Base'
|
| 85 |
model_name = get_model_name_from_path(model_path)
|
| 86 |
tokenizer, model, processor, context_len = load_pretrained_model(model_path, None, model_name)
|
| 87 |
model = model.to('cuda:0')
|