Instructions to use summerAIO/Diffusion-Qwen3.8-27B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use summerAIO/Diffusion-Qwen3.8-27B with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="summerAIO/Diffusion-Qwen3.8-27B")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("summerAIO/Diffusion-Qwen3.8-27B")
model = AutoModelForCausalLM.from_pretrained("summerAIO/Diffusion-Qwen3.8-27B", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use summerAIO/Diffusion-Qwen3.8-27B with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "summerAIO/Diffusion-Qwen3.8-27B"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "summerAIO/Diffusion-Qwen3.8-27B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/summerAIO/Diffusion-Qwen3.8-27B

SGLang

How to use summerAIO/Diffusion-Qwen3.8-27B with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "summerAIO/Diffusion-Qwen3.8-27B" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "summerAIO/Diffusion-Qwen3.8-27B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "summerAIO/Diffusion-Qwen3.8-27B" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "summerAIO/Diffusion-Qwen3.8-27B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use summerAIO/Diffusion-Qwen3.8-27B with Docker Model Runner:
```
docker model run hf.co/summerAIO/Diffusion-Qwen3.8-27B
```

Diffusion-Qwen3.8-27B

🧪 summerMC Diffusion series — autoregressive Qwen models converted into masked diffusion LMs. Experimental research checkpoint: not benchmarked, not for production use.

Model	Base	Weights
Diffusion-Qwen3.5-4B	Qwen/Qwen3.5-4B	merged weights + BART fusion
Diffusion-Qwen3.5-9B	Qwen/Qwen3.5-9B	merged weights + BART fusion
Diffusion-Qwen3.8-27B (this repo)	Qwen/Qwen3.8-27B	merged weights

How it works

Qwen/Qwen3.8-27B is converted into a masked diffusion language model (LLaDA / Dream style). It generates by iterative parallel unmasking, not left-to-right.

Full-attention layers: causal mask removed (bidirectional).
Gated DeltaNet layers: forward + reverse scan, out = orig(h) + g ⊙ flip(orig(flip(h))), with a per-channel rev_gate initialised to 0.
[MASK]: the <|fim_pad|> token with a learned embedding.
Loss: masked-token cross-entropy with 1/t weighting; shifted prediction (position j from hidden j−1) keeps the AR lm_head aligned.
Decoding: confidence-threshold parallel unmasking; text after the first EOS is dropped.

Files

File	Contents
`model-*.safetensors`	merged weights
`dllm_extras.pt`	mask_embed, rev_gates, mask_id, eos_id
`dllm_config.json`	base model and decoding type
`dqwen/`	loading, training and inference code (`dqwen.hub`, `dqwen.engine.BatchEngine`)

Plain AutoModelForCausalLM.from_pretrained gives a causal model, not a diffusion model. Apply the dqwen patch and restore rev_gates / mask_embed as shown below.

Usage

Every model in the series loads the same way through dqwen.hub (shipped in each repo).

pip install -U transformers accelerate huggingface_hub safetensors flash-linear-attention

import sys
from huggingface_hub import snapshot_download
path = snapshot_download("summerMC/Diffusion-Qwen3.8-27B")
sys.path.insert(0, path)
from dqwen.hub import load_pretrained, generate, stream

dlm, tok = load_pretrained(path)
print(generate(dlm, tok, "What is the capital of France?", gen_len=128, thr=0.9))

for ids, newly_committed, step in stream(dlm, tok, "Write a haiku about the sea."):
    pass                                     # undecided tokens are dlm.mask_id

thr is the confidence needed to commit a token at each step: lower is faster, higher is better quality.

⚡ Triton acceleration

load_pretrained enables a Triton fast path by default (dqwen/accel.py, dqwen/triton_kernels.py). Nothing changes in the API: generate, stream and any code calling dlm.hidden(...) get faster.

	What is fused / removed
Bidirectional Gated DeltaNet	input projections computed once (`in_proj(flip(h)) = flip(in_proj(h))`), forward+reverse conv in one kernel, both scans as one batch-2 `fla` call, `rev_gate` folded into a single `out_proj` GEMM
Norms / MLP / attention	residual add + (1+w) RMSNorm across layers, merged gate/up GEMM + SiLU·up, q/k-norm + partial RoPE, sigmoid output gate
Launch overhead	the whole forward is captured as a CUDA graph per sequence length
Vocab (248k)	`dqwen.accel.select` — fused lm_head + softmax max/argmax/LSE without materialising fp32 logits (for custom samplers)

Safety: before switching on, the fast path is checked against the original layers and the full forward pass (dlm._dqwen_fast_report). If it does not match, or for training / padding masks / BART fusion / batch > 1 / 4-bit / pre-Ampere GPUs, the original PyTorch path is used.

dlm, tok = load_pretrained(path)               # on (default)
dlm, tok = load_pretrained(path, fast=False)   # off
from dqwen import accel
accel.report(dlm)                              # validation numbers
accel.disable(dlm)                             # switch back at runtime

Environment: DQWEN_FAST=0 disables it, DQWEN_CUDA_GRAPH=0 keeps the Triton kernels but disables CUDA graphs. Requires an Ampere-or-newer GPU (A100, L4, RTX 30xx+, H100); flash-linear-attention is strongly recommended.

Training

Item	Value
Base model	Qwen/Qwen3.8-27B
LoRA	r=32, merged into the weights
Data	not recorded
Sequence length	not recorded
Steps	short run (exact count not recorded)

Requirements

Hardware: About 56 GB of GPU memory in bf16.
Packages: transformers (with Qwen3.5 support), accelerate, safetensors, huggingface_hub; optional flash-linear-attention.

Limitations

Not evaluated: no benchmark has been run; quality is expected to be below the base model.
No autoregressive generate(): the bidirectional patch runs without a KV/state cache.
Fragile to library updates: changes to the Qwen implementation in transformers may break the patch.
Safety: the diffusion adaptation may weaken the base model's safety tuning.
dqwen/engine.py also has BatchEngine for many prompts in one batched forward pass.
Slower per token than the AR base on a single prompt (no KV cache, extra reverse scan); throughput gains come from batching.

License

Weights follow the license of the base model(s): Qwen/Qwen3.8-27B. Check their model cards.

References

LLaDA: Large Language Diffusion Models (Nie et al., 2025)
Dream 7B: diffusion LM adapted from an AR model (shifted-logits alignment)
Gated DeltaNet / Qwen3.5 hybrid architecture

Optional block KV / state cache

All three DiffusionQwen models support opt-in block KV and bidirectional DeltaNet state caching. See KV_CACHE.md for usage, approximation trade-offs and validation limits.

from dqwen.kvcache import enable_kv_cache
enable_kv_cache(dlm, refresh=0)  # after load_pretrained; then call generate as usual

Downloads last month: 1,173

Safetensors

Model size

27B params

Tensor type

BF16

F32

Model tree for summerAIO/Diffusion-Qwen3.8-27B

Base model

Qwen/Qwen3.8-27B

Finetuned

(776)

this model

Collection including summerAIO/Diffusion-Qwen3.8-27B

DiffusionQwen

Collection

3 items • Updated 16 days ago