Instructions to use OpenFace-CQUPT/Human_LLaVA with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use OpenFace-CQUPT/Human_LLaVA with Transformers:
# Use a pipeline as a high-level helper # Warning: Pipeline type "visual-question-answering" is no longer supported in transformers v5. # You must load the model directly (see below) or downgrade to v4.x with: # pip install "transformers<5.0.0" from transformers import pipeline pipe = pipeline("visual-question-answering", model="OpenFace-CQUPT/Human_LLaVA")# pip install -U transformers accelerate # Load model directly from transformers import AutoProcessor, AutoModelForMultimodalLM processor = AutoProcessor.from_pretrained("OpenFace-CQUPT/Human_LLaVA") model = AutoModelForMultimodalLM.from_pretrained("OpenFace-CQUPT/Human_LLaVA", device_map="auto") - Notebooks
- Google Colab
- Kaggle
LoRA FineTuning
#2
by hcim - opened
"I find your work on this subject very interesting. Therefore, I would like to use it. Do you have any plans to include code related to LoRA Fine-Tuning or share a link to a related GitHub repository? If so, I would like to know when it might be available."
That will be released when the paper is accepted!
I am curious whether it is possible to attach LoRA to the HumanLLaVA that you have been trained on and perform retraining using LoRA fine-tuning.
yes, that is possible.