Instructions to use summerMC/Diffusion-Qwen3.5-4B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use summerMC/Diffusion-Qwen3.5-4B with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="summerMC/Diffusion-Qwen3.5-4B")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("summerMC/Diffusion-Qwen3.5-4B")
model = AutoModelForCausalLM.from_pretrained("summerMC/Diffusion-Qwen3.5-4B", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use summerMC/Diffusion-Qwen3.5-4B with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "summerMC/Diffusion-Qwen3.5-4B"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "summerMC/Diffusion-Qwen3.5-4B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/summerMC/Diffusion-Qwen3.5-4B

SGLang

How to use summerMC/Diffusion-Qwen3.5-4B with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "summerMC/Diffusion-Qwen3.5-4B" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "summerMC/Diffusion-Qwen3.5-4B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "summerMC/Diffusion-Qwen3.5-4B" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "summerMC/Diffusion-Qwen3.5-4B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use summerMC/Diffusion-Qwen3.5-4B with Docker Model Runner:
```
docker model run hf.co/summerMC/Diffusion-Qwen3.5-4B
```
Browse Quantizations to use this model in llama.cpp, Ollama, LM Studio, or any compatible app.

Diffusion-Qwen3.5-4B

🧪 summerMC Diffusion series — autoregressive Qwen models converted into masked diffusion LMs. Experimental research checkpoint: not benchmarked, not for production use.

Model	Base	Weights
Diffusion-Qwen3.5-4B (this repo)	Qwen/Qwen3.5-4B	merged weights + BART fusion
Diffusion-Qwen3.5-9B	Qwen/Qwen3.5-9B	merged weights + BART fusion
Diffusion-Qwen3.8-27B	Qwen/Qwen3.8-27B	merged weights

How it works

Qwen/Qwen3.5-4B is converted into a masked diffusion language model (LLaDA / Dream style). It generates by iterative parallel unmasking, not left-to-right.

Full-attention layers: causal mask removed (bidirectional).
Gated DeltaNet layers: forward + reverse scan, out = orig(h) + g ⊙ flip(orig(flip(h))), with a per-channel rev_gate initialised to 0.
[MASK]: the <|fim_pad|> token with a learned embedding.
Loss: masked-token cross-entropy with 1/t weighting; shifted prediction (position j from hidden j−1) keeps the AR lm_head aligned.
Decoding: block-wise low-confidence remasking (diffusion_generate).
BART fusion: facebook/bart-base (FP32, frozen) encodes the draft; a zero-gated cross-attention module injects it into the response embeddings for a refinement pass.

Files

File	Contents
`config.json, model*.safetensors`	merged weights (standard HF format)
`tokenizer files`	Qwen tokenizer
`diffusion_extras.pt`	mask_embed, mask_id, rev_gates
`diffusion_config.json`	base model, mask/EOS tokens and related settings
`dqwen/`	bidirectional patch, loss and sampler
`fusion.pt`	optional BART fusion module

Plain AutoModelForCausalLM.from_pretrained gives a causal model, not a diffusion model. Apply the dqwen patch and restore rev_gates / mask_embed as shown below.

Usage

Every model in the series loads the same way through dqwen.hub (shipped in each repo).

pip install -U transformers accelerate huggingface_hub safetensors flash-linear-attention

import sys
from huggingface_hub import snapshot_download
path = snapshot_download("summerMC/Diffusion-Qwen3.5-4B")
sys.path.insert(0, path)
from dqwen.hub import load_pretrained, generate, stream

dlm, tok = load_pretrained(path)
print(generate(dlm, tok, "What is the capital of France?", gen_len=128, thr=0.9))
# With fusion.pt present, BART refinement runs automatically; return_draft=True gives (draft, final).

for ids, newly_committed, step in stream(dlm, tok, "Write a haiku about the sea."):
    pass                                     # undecided tokens are dlm.mask_id

thr is the confidence needed to commit a token at each step: lower is faster, higher is better quality.

⚡ Triton acceleration

load_pretrained enables a Triton fast path by default (dqwen/accel.py, dqwen/triton_kernels.py). Nothing changes in the API: generate, stream and any code calling dlm.hidden(...) get faster.

	What is fused / removed
Bidirectional Gated DeltaNet	input projections computed once (`in_proj(flip(h)) = flip(in_proj(h))`), forward+reverse conv in one kernel, both scans as one batch-2 `fla` call, `rev_gate` folded into a single `out_proj` GEMM
Norms / MLP / attention	residual add + (1+w) RMSNorm across layers, merged gate/up GEMM + SiLU·up, q/k-norm + partial RoPE, sigmoid output gate
Launch overhead	the whole forward is captured as a CUDA graph per sequence length
Vocab (248k)	`dqwen.accel.select` — fused lm_head + softmax max/argmax/LSE without materialising fp32 logits (for custom samplers)

Safety: before switching on, the fast path is checked against the original layers and the full forward pass (dlm._dqwen_fast_report). If it does not match, or for training / padding masks / BART fusion / batch > 1 / 4-bit / pre-Ampere GPUs, the original PyTorch path is used.

dlm, tok = load_pretrained(path)               # on (default)
dlm, tok = load_pretrained(path, fast=False)   # off
from dqwen import accel
accel.report(dlm)                              # validation numbers
accel.disable(dlm)                             # switch back at runtime

Environment: DQWEN_FAST=0 disables it, DQWEN_CUDA_GRAPH=0 keeps the Triton kernels but disables CUDA graphs. Requires an Ampere-or-newer GPU (A100, L4, RTX 30xx+, H100); flash-linear-attention is strongly recommended.

Training

Item	Value
Base model	Qwen/Qwen3.5-4B
LoRA	merged into the weights
Data	not recorded
Sequence length	not recorded
Steps	demo scale (few steps)

Requirements

Hardware: One CUDA GPU in bf16. flash-linear-attention / causal-conv1d are optional speed-ups (PyTorch fallback otherwise).
Packages: transformers (with Qwen3.5 support), accelerate, safetensors, huggingface_hub; optional flash-linear-attention.

Limitations

Not evaluated: no benchmark has been run; quality is expected to be below the base model.
No autoregressive generate(): the bidirectional patch runs without a KV/state cache.
Fragile to library updates: changes to the Qwen implementation in transformers may break the patch.
Safety: the diffusion adaptation may weaken the base model's safety tuning.
The low-confidence remasking sampler tends to emit EOS too early; ban EOS for the first few tokens or defer EOS commits.
Quality drops as steps decreases (more tokens committed per step).