Instructions to use IVC-liuyuan/M3Diff with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use IVC-liuyuan/M3Diff with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="IVC-liuyuan/M3Diff")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained("IVC-liuyuan/M3Diff", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use IVC-liuyuan/M3Diff with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "IVC-liuyuan/M3Diff"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "IVC-liuyuan/M3Diff",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/IVC-liuyuan/M3Diff

SGLang

How to use IVC-liuyuan/M3Diff with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "IVC-liuyuan/M3Diff" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "IVC-liuyuan/M3Diff",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "IVC-liuyuan/M3Diff" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "IVC-liuyuan/M3Diff",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use IVC-liuyuan/M3Diff with Docker Model Runner:
```
docker model run hf.co/IVC-liuyuan/M3Diff
```

Unable to load the released M3Diff checkpoint

by chaebyeol - opened Sep 3

Discussion

chaebyeol

Sep 3

Hello,

Thank you for releasing M3Diff.

I am currently having difficulty loading the released checkpoint. I checked all four safetensors shards, and bytes 0–15 appear to be zero for each file, resulting in a zero-length safetensors header.

The downloaded files match the LFS hashes reported by the Hugging Face Hub API. Could you please confirm whether the checkpoint files are complete, or let me know if there is a recommended loading method or alternative download link?

Thank you for your help.

IVC-liuyuan

Owner Sep 3

Dear Chaebyeol Lee,

Thank you for your interest in the M3Diff code and model, and for bringing the issue with the Hugging Face checkpoint files to my attention.

I will investigate the corrupted safetensors shards and ensure this problem is fixed within the next two weeks. I appreciate your patience and will let you know once the repository is updated or an alternative download link is available.

Best,
Yuan

IVC-liuyuan changed discussion status to closed Sep 3

IVC-liuyuan changed discussion status to open Sep 3

IVC-liuyuan

Owner Sep 3

Thank you for bringing this issue to our attention. We are actively working on a fix and expect to resolve it within the next two weeks.

Please be assured that it is being handled, so there is no need to send further reminders or reach out through other channels in the meantime. We appreciate your patience and understanding.

IVC-liuyuan changed discussion status to closed Sep 3

IVC-liuyuan changed discussion status to open Sep 3

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment