Instructions to use Sebesky/MiniMax-M3-W4A16-GPTQ with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Sebesky/MiniMax-M3-W4A16-GPTQ with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="Sebesky/MiniMax-M3-W4A16-GPTQ")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("Sebesky/MiniMax-M3-W4A16-GPTQ")
model = AutoModelForMultimodalLM.from_pretrained("Sebesky/MiniMax-M3-W4A16-GPTQ", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use Sebesky/MiniMax-M3-W4A16-GPTQ with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Sebesky/MiniMax-M3-W4A16-GPTQ"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Sebesky/MiniMax-M3-W4A16-GPTQ",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/Sebesky/MiniMax-M3-W4A16-GPTQ

SGLang

How to use Sebesky/MiniMax-M3-W4A16-GPTQ with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Sebesky/MiniMax-M3-W4A16-GPTQ" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Sebesky/MiniMax-M3-W4A16-GPTQ",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "Sebesky/MiniMax-M3-W4A16-GPTQ" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Sebesky/MiniMax-M3-W4A16-GPTQ",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use Sebesky/MiniMax-M3-W4A16-GPTQ with Docker Model Runner:
```
docker model run hf.co/Sebesky/MiniMax-M3-W4A16-GPTQ
```

Sebesky commited on Jun 26

Commit

efc052d

verified ·

1 Parent(s): a1334b6

Add files using upload-large-folder tool

Browse files

Files changed (34) hide show

model-00001-of-00059.safetensors +3 -0
model-00002-of-00059.safetensors +3 -0
model-00003-of-00059.safetensors +3 -0
model-00004-of-00059.safetensors +3 -0
model-00005-of-00059.safetensors +3 -0
model-00006-of-00059.safetensors +3 -0
model-00007-of-00059.safetensors +3 -0
model-00033-of-00059.safetensors +3 -0
model-00035-of-00059.safetensors +3 -0
model-00036-of-00059.safetensors +3 -0
model-00037-of-00059.safetensors +3 -0
model-00038-of-00059.safetensors +3 -0
model-00039-of-00059.safetensors +3 -0
model-00040-of-00059.safetensors +3 -0
model-00041-of-00059.safetensors +3 -0
model-00042-of-00059.safetensors +3 -0
model-00043-of-00059.safetensors +3 -0
model-00044-of-00059.safetensors +3 -0
model-00045-of-00059.safetensors +3 -0
model-00046-of-00059.safetensors +3 -0
model-00047-of-00059.safetensors +3 -0
model-00048-of-00059.safetensors +3 -0
model-00049-of-00059.safetensors +3 -0
model-00050-of-00059.safetensors +3 -0
model-00051-of-00059.safetensors +3 -0
model-00052-of-00059.safetensors +3 -0
model-00053-of-00059.safetensors +3 -0
model-00054-of-00059.safetensors +3 -0
model-00055-of-00059.safetensors +3 -0
model-00056-of-00059.safetensors +3 -0
model-00057-of-00059.safetensors +3 -0
model-00058-of-00059.safetensors +3 -0
model-00059-of-00059.safetensors +3 -0
model.safetensors.index.json +0 -0

model-00001-of-00059.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:a42f3add6a14c9fa34995302716e3942a1df19e2d42778d325ea3d450ad7eaa1
+size 3283370272

model-00002-of-00059.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:72023862b7b4cb8643062245b89cb8cc0378d8536900f30048d4ea43c88426f5
+size 2815991688

model-00003-of-00059.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:a19327ebf94ce2e4fa1ecffea20a7d6c003e337302b4d91e95b8d1981d876bb9
+size 4152283820

model-00004-of-00059.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:9019a94a63365d5dde3fe8603177a954465c48b824f76416dfeedff6281636d5
+size 4153856924

model-00005-of-00059.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:61bc546bf00d8289d86c579f39bbd74a60f278f674891437844acd9fc5917525
+size 4153856924

model-00006-of-00059.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:e8224857d94f2c62778ae0215a3decb1d052378c7321dbadb63355bf2fd1ac32
+size 4153856924

model-00007-of-00059.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:220efb932d056f291907e05d6308a473345eae04a85d0d9df92aa97c6599b041
+size 4153856924

model-00033-of-00059.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:085d611205d0eff5e277acbfb94a23a43a648857d3c7f108cf43dc1ada8a9121
+size 3832683364

model-00035-of-00059.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:8b1ca2e6645b2c049b3c744c196733599e1301a5840b45c907841b8d48c7928e
+size 3832683364

model-00036-of-00059.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:97ff8b3d575cdd41da648a5c83a50b98323b93aff44d508bf08d4bb8bfbcf737
+size 3832683364

model-00037-of-00059.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:3bd3fa59ab3fdb42ab5866409d8e44313291dbcd4d13c3f10f40af16e0ab655b
+size 3511508612

model-00038-of-00059.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:3e0cff0c79f0f85c316a31f65bd3d2ba862e2ea00467ddfbbc0e47f23757ecc9
+size 3511508612

model-00039-of-00059.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:0585222fd015b0e6918d8e105d763801798587625ac35dedf8131886f283bf27
+size 3511508612

model-00040-of-00059.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:c9bfb8d32dbee7ff90a84893ca2764d782f5c7c7133e5338d988f9f9cbcaa4dd
+size 3511508612

model-00041-of-00059.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:0848ce2fa29c26895f0486c56b36e9d6c571e49b9f67e4a73fe2c1e6b3750494
+size 3511508612

model-00042-of-00059.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:36bc0eb508f9c6f469fa88f9bef86fdd521e7d281ab25db5663cd4cb09cbd1de
+size 3511508612

model-00043-of-00059.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:280278d74a6d828f04e3256534144a3717be1ef420126480e073ca8232a16fce
+size 3511508612

model-00044-of-00059.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:932a80bbe05b10d4f254a6f8396d0aa3eb09a5b440dc8a5410cefde49aaa773a
+size 3511508612

model-00045-of-00059.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:f614b42204b81679a519fd133e2a62d0226b11dd30ee1d5cbd42f4021181c457
+size 3511508612

model-00046-of-00059.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:e466a8e15e183f6cebce6efe36cc46f698c7cada6ecfda0c2f416ccd1b33bd9d
+size 3511508612

model-00047-of-00059.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:de6c69043cd42e5012175ba2949dc5320376daa5dcc574e059efc796b37498b8
+size 3511508612

model-00048-of-00059.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:78129873587c9fe523fc76cf486135cd9537c7d3784464a27b24dd5e1054f62e
+size 3511508612

model-00049-of-00059.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:a588803038a861013019a0fe3ee98d87709c9775fc9ab9f61592b3fed7eabae3
+size 3511508612

model-00050-of-00059.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:3eba70e58d90dfee43f334a2413f4e92901fce898f125a169f213410bd3fea5d
+size 3511508612

model-00051-of-00059.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:31ffea3a11ba87a1c166a8ff9dd0d3200fd11f6725fe04bbfc1794b372a0c2aa
+size 3511508612

model-00052-of-00059.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:6ba931035b88df41463a669c599c4bf73d2d2f7808abb77a5699e37b4c4fbb29
+size 3511508612

model-00053-of-00059.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:6c858172df75d857ca62ea84ecbd9ff266264e06baaec8550cbdeac726adaeae
+size 3511508612

model-00054-of-00059.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:1566a28f8b1a75f5759f00dbc6f2c2d9cd17599038fcb11eec07c8edd62ed76f
+size 3511508612

model-00055-of-00059.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:ac0c29983b6dd60014ac486ce9e7fe63415668fe987eb246ed1de58f94d177a3
+size 3511508612

model-00056-of-00059.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:927745b53e13faff81c9e7042cf0542a092b42a99fdb6dd62f4a17979ac1398e
+size 3511508612

model-00057-of-00059.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:c9419d30bd9f08cae2eabf0ab2e64a5d99b465bc2d65e5e91adcff4365abaf8b
+size 3511508612

model-00058-of-00059.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:6a26761977d64f3d3051e5d3b75a52cdb2ef29e7c70845bc5b8337b11dccf11f
+size 3511508612

model-00059-of-00059.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:a798b94d8c3113b537f49a778755edc139aa31587cc689500b45096f67492ec8
+size 5095138796

model.safetensors.index.json CHANGED Viewed

The diff for this file is too large to render. See raw diff