liuhaotian/LLaVA-Pretrain
Preview • Updated • 6.71k • 227
How to use xtuner/llava-internlm-7b-pretrain with Transformers:
# Use a pipeline as a high-level helper
# Warning: Pipeline type "visual-question-answering" is no longer supported in transformers v5.
# You must load the model directly (see below) or downgrade to v4.x with:
# pip install "transformers<5.0.0"
from transformers import pipeline
pipe = pipeline("visual-question-answering", model="xtuner/llava-internlm-7b-pretrain") # pip install -U transformers accelerate
# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("xtuner/llava-internlm-7b-pretrain", device_map="auto")Configuration Parsing Warning:Invalid JSON for config file config.json
llava-internlm-7b-pretrain is a LLaVA projector pretrained with InternLM-Chat-7B and CLIP-ViT-Large-patch14-336 on LLaVA-Pretrain dataset by XTuner. The fine-tuned LLaVA model can be found on xtuner/llava-internlm-7b.
@misc{2023xtuner,
title={XTuner: A Toolkit for Efficiently Fine-tuning LLM},
author={XTuner Contributors},
howpublished = {\url{https://github.com/InternLM/xtuner}},
year={2023}
}