Instructions to use UT-IFML/LLaDA-8B-Instruct-PIG with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use UT-IFML/LLaDA-8B-Instruct-PIG with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="UT-IFML/LLaDA-8B-Instruct-PIG", trust_remote_code=True)
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("UT-IFML/LLaDA-8B-Instruct-PIG", trust_remote_code=True, device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use UT-IFML/LLaDA-8B-Instruct-PIG with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "UT-IFML/LLaDA-8B-Instruct-PIG"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "UT-IFML/LLaDA-8B-Instruct-PIG",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/UT-IFML/LLaDA-8B-Instruct-PIG

SGLang

How to use UT-IFML/LLaDA-8B-Instruct-PIG with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "UT-IFML/LLaDA-8B-Instruct-PIG" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "UT-IFML/LLaDA-8B-Instruct-PIG",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "UT-IFML/LLaDA-8B-Instruct-PIG" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "UT-IFML/LLaDA-8B-Instruct-PIG",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use UT-IFML/LLaDA-8B-Instruct-PIG with Docker Model Runner:
```
docker model run hf.co/UT-IFML/LLaDA-8B-Instruct-PIG
```

LLaDA-8B-Instruct-PIG / config.json

qfruytier

Combined model: backbone + PIG adapter + tables, transformers 5.x; arXiv 2610.17375 (PIG-DLM-release 085b176)

6b9e4ee verified 4 days ago

Raw History Blame Contribute Delete

2.74 kB

	{
	"adapter": {
	"description": "Entropy Finite Differences Estimator for GSAI-ML/LLaDA-8B-Instruct",
	"epoch": "7",
	"package": "085b176",
	"parameters": 2165527,
	"source": "assets/llada/fde.safetensors",
	"val_edge_corr": "0.8268"
	},
	"architectures": [
	"PIGModel"
	],
	"auto_map": {
	"AutoConfig": "configuration_pig.PIGConfig",
	"AutoModel": "modeling_pig.PIGModel"
	},
	"backbone": "GSAI-ML/LLaDA-8B-Instruct",
	"backbone_config": {
	"activation_type": "silu",
	"alibi": false,
	"alibi_bias_max": 8.0,
	"attention_dropout": 0.0,
	"attention_layer_norm": false,
	"attention_layer_norm_with_affine": true,
	"bias_for_layer_norm": false,
	"block_group_size": 1,
	"block_type": "llama",
	"d_model": 4096,
	"embedding_dropout": 0.0,
	"embedding_size": 126464,
	"eos_token_id": 126081,
	"flash_attention": false,
	"include_bias": false,
	"include_qkv_bias": false,
	"init_cutoff_factor": null,
	"init_device": "meta",
	"init_fn": "mitchell",
	"init_std": 0.02,
	"input_emb_norm": false,
	"layer_norm_type": "rms",
	"layer_norm_with_affine": true,
	"mask_token_id": 126336,
	"max_sequence_length": 4096,
	"mlp_hidden_size": 12288,
	"mlp_ratio": 4,
	"model_type": "llada",
	"multi_query_attention": null,
	"n_heads": 32,
	"n_kv_heads": 32,
	"n_layers": 32,
	"pad_token_id": 126081,
	"precision": "amp_bf16",
	"residual_dropout": 0.0,
	"rms_norm_eps": 1e-05,
	"rope": true,
	"rope_full_precision": true,
	"rope_theta": 500000.0,
	"scale_logits": false,
	"use_cache": false,
	"vocab_size": 126464,
	"weight_tying": false
	},
	"backbone_revision": "08b83a6feb34df1a6011b80c3c00c7563e963b07",
	"defaults": {
	"block": 32,
	"canvas": 256,
	"floor": null,
	"geometry": "block",
	"mode": "pig.int",
	"nu": 0.9,
	"phi": 0.1,
	"theta": 0.5,
	"window": 16
	},
	"dtype": "bfloat16",
	"family": "llada",
	"model_type": "pig",
	"table_provenance": {
	"confidence": {
	"description": "LLaDA-8B-Instruct: P(parallel commit != sequential reference \| confidence, aggregate influence); fit on sequential replays of GSM8K-train and OpenCoder states",
	"mismatches": 268849,
	"rows": 792841,
	"source": "assets/llada/flip_confidence.npz"
	},
	"entropy": {
	"description": "LLaDA-8B-Instruct: P(parallel commit != sequential reference \| entropy, aggregate influence); fit on sequential replays of GSM8K-train and OpenCoder states",
	"mismatches": 132943,
	"rows": 395854,
	"source": "assets/llada/flip_entropy.npz"
	}
	},
	"tables": [
	"confidence",
	"entropy"
	],
	"transformers_version": "5.19.0"
	}