Instructions to use axteam/Motif-3-W4A16 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use axteam/Motif-3-W4A16 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="axteam/Motif-3-W4A16", trust_remote_code=True)
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("axteam/Motif-3-W4A16", trust_remote_code=True, device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use axteam/Motif-3-W4A16 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "axteam/Motif-3-W4A16"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "axteam/Motif-3-W4A16",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/axteam/Motif-3-W4A16

SGLang

How to use axteam/Motif-3-W4A16 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "axteam/Motif-3-W4A16" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "axteam/Motif-3-W4A16",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "axteam/Motif-3-W4A16" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "axteam/Motif-3-W4A16",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use axteam/Motif-3-W4A16 with Docker Model Runner:
```
docker model run hf.co/axteam/Motif-3-W4A16
```

Motif-3 · INT4 W4A16 (RTN, data-free)

4-bit quantization of Motif-Technologies/Motif-3 for 🤗 Transformers (trust_remote_code). Quantized by axteam.


Base model	`Motif-Technologies/Motif-3`
Method	INT4 W4A16 (RTN, data-free)
Format	compressed-tensors `pack-quantized`
Weights / activations	W4A16 — INT4 group-128 weights, BF16 activations
Architecture	MoE (Motif arch · 384 experts + shared) · 53 layers · text-only
Size on disk	~154.8 GB
Calibration	None (data-free RTN, name-based `model_free_ptq`)
License	mit

Quantization details

Scheme — W4A16 — INT4 group-128 weights, BF16 activations, stored as compressed-tensors pack-quantized.
Kept in BF16 — token embeddings, the LM head, and all RMSNorms. Only attention & MLP projections are quantized. In addition, all MoE routers (router.gate) and attention gates (wq_b_gate) are kept in BF16.
Post-training quantization (no retraining). Tokenizer / chat_template identical to the original.
Modified modeling_motif.py — the original stores routed experts as fused 3D tensors (gate_up_proj [E, 2I, H], down_proj [E, H, I]). Quantization splits them per expert, so this repo's MotifExperts holds one set of nn.Linear layers per expert (experts.<e>.{gate,up,down}_proj). gate_proj / up_proj are the first / second half of the original gate_up_proj[e]; the computation is unchanged. With the original modeling file the expert weights would not load.

Usage (Transformers)

vLLM does not support the Motif architecture (checked with vLLM 0.32), so load it with Transformers and the bundled modeling code:

from transformers import AutoModelForCausalLM, AutoTokenizer
tok = AutoTokenizer.from_pretrained("axteam/Motif-3-W4A16")
model = AutoModelForCausalLM.from_pretrained("axteam/Motif-3-W4A16", trust_remote_code=True, dtype="auto",
                                             device_map="auto", attn_implementation="flash_attention_2")

License

Quantized derivative of Motif-Technologies/Motif-3 (mit).

한국어 요약

Motif-Technologies/Motif-3 를 INT4 W4A16 (RTN, data-free) 로 양자화한 서빙용(4-bit) 모델입니다. 작성: axteam.

방식 — INT4 128묶음 가중치 + 활성화 BF16, 데이터프리 RTN. 형식 compressed-tensors pack-quantized.
BF16 유지 — 임베딩·LM 헤드·모든 norm. attention·MLP projection 만 양자화. 추가로 MoE 라우터(router.gate)와 어텐션 게이트(wq_b_gate)도 BF16 유지.
PTQ(재학습 없음), 토크나이저·chat_template 원본 동일.
modeling_motif.py 수정 — 원본은 전문가를 3D 텐서(gate_up_proj, down_proj)로 두는데, 양자화하면서 전문가별로 나뉘어 이 리포의 MotifExperts 는 전문가마다 nn.Linear(experts.<e>.{gate,up,down}_proj)를 둡니다. 계산은 원본과 같고, 원본 modeling 파일로는 전문가 가중치가 실리지 않습니다.
사용 — Transformers AutoModelForCausalLM.from_pretrained("axteam/Motif-3-W4A16", trust_remote_code=True, attn_implementation="flash_attention_2") (vLLM 은 Motif 아키텍처 미지원). 라이선스 mit.

Downloads last month: -

Safetensors

Model size

315B params

Tensor type

I32

BF16

Model tree for axteam/Motif-3-W4A16

Base model

Motif-Technologies/Motif-3-Base

Finetuned

Motif-Technologies/Motif-3

Quantized

(14)

this model