Instructions to use HuggingFaceTB/SmolVLM2-2.2B-Instruct with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use HuggingFaceTB/SmolVLM2-2.2B-Instruct with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("image-text-to-text", model="HuggingFaceTB/SmolVLM2-2.2B-Instruct") messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] pipe(text=messages)# pip install -U transformers accelerate # Load model directly from transformers import AutoProcessor, AutoModelForMultimodalLM processor = AutoProcessor.from_pretrained("HuggingFaceTB/SmolVLM2-2.2B-Instruct") model = AutoModelForMultimodalLM.from_pretrained("HuggingFaceTB/SmolVLM2-2.2B-Instruct", device_map="auto") messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] inputs = processor.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=256) print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use HuggingFaceTB/SmolVLM2-2.2B-Instruct with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "HuggingFaceTB/SmolVLM2-2.2B-Instruct" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "HuggingFaceTB/SmolVLM2-2.2B-Instruct", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker
docker model run hf.co/HuggingFaceTB/SmolVLM2-2.2B-Instruct
- SGLang
How to use HuggingFaceTB/SmolVLM2-2.2B-Instruct with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "HuggingFaceTB/SmolVLM2-2.2B-Instruct" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "HuggingFaceTB/SmolVLM2-2.2B-Instruct", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "HuggingFaceTB/SmolVLM2-2.2B-Instruct" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "HuggingFaceTB/SmolVLM2-2.2B-Instruct", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }' - Docker Model Runner
How to use HuggingFaceTB/SmolVLM2-2.2B-Instruct with Docker Model Runner:
docker model run hf.co/HuggingFaceTB/SmolVLM2-2.2B-Instruct
Commit History
-
add AIBOM
Verified signatureDear model owner(s), We are a group of researchers investigating the usefulness of sharing AIBOMs (Artificial Intelligence Bill of Materials) to document AI models – AIBOMs are machine-readable structured lists of components (e.g., datasets and models) used to enhance transparency in AI-model supply chains. To pursue the above-mentioned objective, we identified popular models on HuggingFace and, based on your model card (and some configuration information available in HuggingFace), we generated your AIBOM according to the CyclonDX (v1.6) standard (see https://cyclonedx.org/docs/1.6/json/). AIBOMs are generated as JSON files by using the following open-source supporting tool: https://github.com/MSR4SBOM/ALOHA (technical details are available in the research paper: https://github.com/MSR4SBOM/ALOHA/blob/main/ALOHA.pdf). The JSON file in this pull request is your AIBOM (see https://github.com/MSR4SBOM/ALOHA/blob/main/documentation.json for details on its structure). Clearly, the subm…fe87765
-
Update README.md
Verified signature482adb5
-
Update README.md
Verified signature274fe49
-
Update README.md
Verified signature42a4cc9 -
Update README.md
Verified signature83e8cba -
Update README.md
Verified signaturef83c1bb
-
Test video tag (#13)
Verified signature- Test video tag (24c3651146e7787db7ac8c1133cf0553ac0da08e) Co-authored-by: Pedro Cuenca <pcuenq@users.noreply.huggingface.co>e00155d -
update tag (#11)
Verified signature- update tag (d0d821c44384a65c294cd1a98271657cd957e507) Co-authored-by: Orr Zohar <orrzohar@users.noreply.huggingface.co>a9e1eea
-
Code snippets (#10)
Verified signature- Code snippets (c0a7af506d0f71a771f24216ade491dec52ff6c5) Co-authored-by: Pedro Cuenca <pcuenq@users.noreply.huggingface.co>8af4ecd
-
Fix code typo for Simple and Video Inference examples (#9)
Verified signature- Fix code typo for Simple and Video Inference examples (da907c3ab206b8d615c2a4b1b33ae7fd440b4c7d) Co-authored-by: Michal Kubista <kubistmi@users.noreply.huggingface.co>e92909e
-
Fix dtype processing in README example (#5)
Verified signature- Fix dtype processing in README example (5e12bdc378a11643e58065dada60fce07beee680) Co-authored-by: Orr Zohar <orrzohar@users.noreply.huggingface.co>41348c8 -
SmolVLM2-500M-Video -> SmolVLM2-2.2B (#3)
Verified signature- SmolVLM2-500M-Video -> SmolVLM2-2.2B (eccc2482edff325f6d227d57791385e8a196fb1d) Co-authored-by: Sai <SaisExperiments@users.noreply.huggingface.co>54cb403
-
Update README.md
Verified signature06ac4f8
-
Fix chat template (#1)
Verified signature- Fix chat template (73d6c973580e0922293b9d7c80f7dbc3f03d4c97) Co-authored-by: Pedro Cuenca <pcuenq@users.noreply.huggingface.co>5719134
-
Update README.md
Verified signature520a778 -
Update README.md
Verified signature920ac89
-
Upload processor
Verified signature8d39814
-
Upload 3 files
Verified signature70402a4
-
Upload SmolVLMForConditionalGeneration
Verified signature67b63ec
-
Upload processor
Verified signature8e96b17 -
Upload SmolVLMForConditionalGeneration
Verified signaturef33043d -
initial commit
Verified signature9fbd096