Instructions to use mjf-su/AutoVLA with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use mjf-su/AutoVLA with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="mjf-su/AutoVLA")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("mjf-su/AutoVLA")
model = AutoModelForMultimodalLM.from_pretrained("mjf-su/AutoVLA", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use mjf-su/AutoVLA with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "mjf-su/AutoVLA"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "mjf-su/AutoVLA",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/mjf-su/AutoVLA

SGLang

How to use mjf-su/AutoVLA with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "mjf-su/AutoVLA" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "mjf-su/AutoVLA",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "mjf-su/AutoVLA" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "mjf-su/AutoVLA",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use mjf-su/AutoVLA with Docker Model Runner:
```
docker model run hf.co/mjf-su/AutoVLA
```

mjf-su commited on May 19

Commit

ebe4e3f

verified ·

1 Parent(s): 1aebaf7

Add files using upload-large-folder tool

Browse files

Files changed (35) hide show

.gitattributes +4 -0
checkpoint-1000/model-00001-of-00002.safetensors +3 -0
checkpoint-1000/model-00002-of-00002.safetensors +3 -0
checkpoint-1000/optimizer.pt +3 -0
checkpoint-1000/rng_state_0.pth +3 -0
checkpoint-1000/rng_state_1.pth +3 -0
checkpoint-1000/rng_state_2.pth +3 -0
checkpoint-1000/rng_state_3.pth +3 -0
checkpoint-1000/scheduler.pt +3 -0
checkpoint-1000/tokenizer.json +3 -0
checkpoint-1000/training_args.bin +3 -0
checkpoint-1042/model-00001-of-00002.safetensors +3 -0
checkpoint-1042/model-00002-of-00002.safetensors +3 -0
checkpoint-1042/optimizer.pt +3 -0
checkpoint-1042/rng_state_0.pth +3 -0
checkpoint-1042/rng_state_1.pth +3 -0
checkpoint-1042/rng_state_2.pth +3 -0
checkpoint-1042/rng_state_3.pth +3 -0
checkpoint-1042/scheduler.pt +3 -0
checkpoint-1042/tokenizer.json +3 -0
checkpoint-1042/training_args.bin +3 -0
checkpoint-500/model-00001-of-00002.safetensors +3 -0
checkpoint-500/model-00002-of-00002.safetensors +3 -0
checkpoint-500/optimizer.pt +3 -0
checkpoint-500/rng_state_0.pth +3 -0
checkpoint-500/rng_state_1.pth +3 -0
checkpoint-500/rng_state_2.pth +3 -0
checkpoint-500/rng_state_3.pth +3 -0
checkpoint-500/scheduler.pt +3 -0
checkpoint-500/tokenizer.json +3 -0
checkpoint-500/training_args.bin +3 -0
model-00001-of-00002.safetensors +3 -0
model-00002-of-00002.safetensors +3 -0
tokenizer.json +3 -0
training_args.bin +3 -0

.gitattributes CHANGED Viewed

@@ -33,3 +33,7 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text

 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+checkpoint-1042/tokenizer.json filter=lfs diff=lfs merge=lfs -text
+checkpoint-1000/tokenizer.json filter=lfs diff=lfs merge=lfs -text
+checkpoint-500/tokenizer.json filter=lfs diff=lfs merge=lfs -text
+tokenizer.json filter=lfs diff=lfs merge=lfs -text

checkpoint-1000/model-00001-of-00002.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:c0b5e3e202889fe9de706cdea57fb462287488daf7a25f9400057f57e2a99681
+size 4990497880

checkpoint-1000/model-00002-of-00002.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:e989b68053f0279b6bd850bfd16de927610c996d8667dbb080cd254781a1e849
+size 3885221448

checkpoint-1000/optimizer.pt ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:ffe9f96432876b1f2e2dfe4cde7cdcdafc9b7620342a267ec2b8b7b09d021b67
+size 17751894342

checkpoint-1000/rng_state_0.pth ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:659b1cdee2219458dd84ce6a632a595465680b8080e5c44bd600ff97eca8d752
+size 15429

checkpoint-1000/rng_state_1.pth ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:86accf27064cdd503053e90476a6bd10de333d4ff0594535ad55ea13a473c91d
+size 15429

checkpoint-1000/rng_state_2.pth ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:18ca8d714ef40be035404c1957b5a4dee84e1f43980408393f8aa710552ee6f6
+size 15429

checkpoint-1000/rng_state_3.pth ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:2cfdebe99e40accc9c9d8f09c63136a14abda997d9b501969ec8e16e9d183179
+size 15429

checkpoint-1000/scheduler.pt ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:9315751c7c34cc902519ff6ea35377d514efd584513923188b2ad61f8a3abaf5
+size 1465

checkpoint-1000/tokenizer.json ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:f54b55fa0c3aba0c91ce09ea79ae4e62da24e2a1a630f96c4bae34aba25e234a
+size 11422822

checkpoint-1000/training_args.bin ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:4e3bc58a5aabd30db4680413c98dc67d8c4fa9401a48f678ab21c246aeb2a4a2
+size 6353

checkpoint-1042/model-00001-of-00002.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:9861937733323638216ca9bacf6b3751d7ae5de4d7ed41911ba4cdcd0fe2eca8
+size 4990497880

checkpoint-1042/model-00002-of-00002.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:620fce0ff53db1218b4f9e922d186a78ab5b00e2ea6106e42b46635a1da7a80a
+size 3885221448

checkpoint-1042/optimizer.pt ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:a105495e02d5287e7f5a3b5b013d4666f1420cd37d2a1aa221a3c40697c308d6
+size 17751894342

checkpoint-1042/rng_state_0.pth ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:659b1cdee2219458dd84ce6a632a595465680b8080e5c44bd600ff97eca8d752
+size 15429

checkpoint-1042/rng_state_1.pth ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:86accf27064cdd503053e90476a6bd10de333d4ff0594535ad55ea13a473c91d
+size 15429

checkpoint-1042/rng_state_2.pth ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:18ca8d714ef40be035404c1957b5a4dee84e1f43980408393f8aa710552ee6f6
+size 15429

checkpoint-1042/rng_state_3.pth ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:2cfdebe99e40accc9c9d8f09c63136a14abda997d9b501969ec8e16e9d183179
+size 15429

checkpoint-1042/scheduler.pt ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:f43ac548e362b7ca12e5b278076074ac3a1d580273026349ca00b2cb6b979c5e
+size 1465

checkpoint-1042/tokenizer.json ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:f54b55fa0c3aba0c91ce09ea79ae4e62da24e2a1a630f96c4bae34aba25e234a
+size 11422822

checkpoint-1042/training_args.bin ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:4e3bc58a5aabd30db4680413c98dc67d8c4fa9401a48f678ab21c246aeb2a4a2
+size 6353

checkpoint-500/model-00001-of-00002.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:107c473e1202f69335e5705b7fc09fe885ecc38e3fb4af67299bbd6bcc3b89ff
+size 4990497880

checkpoint-500/model-00002-of-00002.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:43f8b47878d72b1ce6a16e2bafc36ad717219e32aa549a4e581a5297e01a869c
+size 3885221448

checkpoint-500/optimizer.pt ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:f026e7b51991f9e22d0c018ccb499679a8655c8407ed2d5c5605a065fcd9adc5
+size 17751894342

checkpoint-500/rng_state_0.pth ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:659b1cdee2219458dd84ce6a632a595465680b8080e5c44bd600ff97eca8d752
+size 15429

checkpoint-500/rng_state_1.pth ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:86accf27064cdd503053e90476a6bd10de333d4ff0594535ad55ea13a473c91d
+size 15429

checkpoint-500/rng_state_2.pth ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:18ca8d714ef40be035404c1957b5a4dee84e1f43980408393f8aa710552ee6f6
+size 15429

checkpoint-500/rng_state_3.pth ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:2cfdebe99e40accc9c9d8f09c63136a14abda997d9b501969ec8e16e9d183179
+size 15429

checkpoint-500/scheduler.pt ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:914902cb68ec99f7a4b980d2c38a79de1b9564c204f9fed8ae59fdaa50c584a0
+size 1465

checkpoint-500/tokenizer.json ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:f54b55fa0c3aba0c91ce09ea79ae4e62da24e2a1a630f96c4bae34aba25e234a
+size 11422822

checkpoint-500/training_args.bin ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:4e3bc58a5aabd30db4680413c98dc67d8c4fa9401a48f678ab21c246aeb2a4a2
+size 6353

model-00001-of-00002.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:9861937733323638216ca9bacf6b3751d7ae5de4d7ed41911ba4cdcd0fe2eca8
+size 4990497880

model-00002-of-00002.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:620fce0ff53db1218b4f9e922d186a78ab5b00e2ea6106e42b46635a1da7a80a
+size 3885221448

tokenizer.json ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:f54b55fa0c3aba0c91ce09ea79ae4e62da24e2a1a630f96c4bae34aba25e234a
+size 11422822

training_args.bin ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:4e3bc58a5aabd30db4680413c98dc67d8c4fa9401a48f678ab21c246aeb2a4a2
+size 6353