Text Generation
Transformers
PyTorch
Safetensors
English
idefics
image-text-to-text
multimodal
text
image
image-to-text
text-generation-inference
Instructions to use HuggingFaceM4/idefics-9b-instruct with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use HuggingFaceM4/idefics-9b-instruct with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="HuggingFaceM4/idefics-9b-instruct")# pip install -U transformers accelerate # Load model directly from transformers import AutoProcessor, AutoModelForMultimodalLM processor = AutoProcessor.from_pretrained("HuggingFaceM4/idefics-9b-instruct") model = AutoModelForMultimodalLM.from_pretrained("HuggingFaceM4/idefics-9b-instruct", device_map="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use HuggingFaceM4/idefics-9b-instruct with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "HuggingFaceM4/idefics-9b-instruct" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "HuggingFaceM4/idefics-9b-instruct", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker
docker model run hf.co/HuggingFaceM4/idefics-9b-instruct
- SGLang
How to use HuggingFaceM4/idefics-9b-instruct with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "HuggingFaceM4/idefics-9b-instruct" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "HuggingFaceM4/idefics-9b-instruct", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "HuggingFaceM4/idefics-9b-instruct" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "HuggingFaceM4/idefics-9b-instruct", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }' - Docker Model Runner
How to use HuggingFaceM4/idefics-9b-instruct with Docker Model Runner:
docker model run hf.co/HuggingFaceM4/idefics-9b-instruct
Commit History
-
set the correct vision_config.hidden_act
8e7a7cf
-
Update readme and doc from the 80b repo
53f0739 -
Update readme and doc from the 80b repo
31098a9
-
Update readme and doc from the 80b repo
08fe8b2 -
Update readme and doc from the 80b repo
ca56117 -
Update readme and doc from the 80b repo
bac0bbf -
Update readme and doc from the 80b repo
f5a6954
-
Update readme and doc from the 80b repo
757bf67
-
Update readme and doc from the 80b repo
d9566ea
-
Update README.md
7a17dc0 -
revert
978c967 -
adding tokenizer.json (for fast tokenizer)
edc31ee -
Update README from idefics-80b
dc19976 -
Update README from idefics-80b
5110da0
-
Create README.md
b07f202
-
add safetensors
hugo@huggingface.co625506c
-
fix because we added a 3rd additional token
30266cc -
add the new instruct model
4b3cca8 -
Update tokenizer_config.json
b99c897
-
Update config.json
aec559d
-
Update preprocessor_config.json
682a420 -
Create preprocessor_config.json
8e4929a -
Update config.json
ceb649f
-
First commit
5f31d1d -
initial commit
Victor Sanh5729d26