How to use mobilint/aya-vision-8b with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("image-text-to-text", model="mobilint/aya-vision-8b", trust_remote_code=True) messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] pipe(text=messages)
# pip install -U transformers accelerate # Load model directly from transformers import AutoModelForCausalLM model = AutoModelForCausalLM.from_pretrained("mobilint/aya-vision-8b", trust_remote_code=True, device_map="auto")
How to use mobilint/aya-vision-8b with Mobilint:
# pip install mblt-model-zoo from mblt_model_zoo.vision import MBLT_Engine model = MBLT_Engine( model_cls="aya-vision-8b", model_type="DEFAULT", model_path="", core_mode="global8", ) try: image = model.preprocess("path/to/image.jpg") output = model(image) result = model.postprocess(output) finally: model.dispose()
How to use mobilint/aya-vision-8b with vLLM:
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "mobilint/aya-vision-8b" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "mobilint/aya-vision-8b", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'
docker model run hf.co/mobilint/aya-vision-8b
How to use mobilint/aya-vision-8b with SGLang:
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "mobilint/aya-vision-8b" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "mobilint/aya-vision-8b", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "mobilint/aya-vision-8b" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "mobilint/aya-vision-8b", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'
How to use mobilint/aya-vision-8b with Docker Model Runner:
This repository provides a model compiled and optimized for Mobilint NPU hardware.The model is packaged for deployment on Mobilint’s acceleration stack and is intended to be used within that environment.
Chat template
Files info
Base model
