Instructions to use Diffuspace/Clover-35B-A3B-Preview with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Diffuspace/Clover-35B-A3B-Preview with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="Diffuspace/Clover-35B-A3B-Preview", trust_remote_code=True)
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained("Diffuspace/Clover-35B-A3B-Preview", trust_remote_code=True, device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use Diffuspace/Clover-35B-A3B-Preview with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Diffuspace/Clover-35B-A3B-Preview"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Diffuspace/Clover-35B-A3B-Preview",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/Diffuspace/Clover-35B-A3B-Preview

SGLang

How to use Diffuspace/Clover-35B-A3B-Preview with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Diffuspace/Clover-35B-A3B-Preview" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Diffuspace/Clover-35B-A3B-Preview",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "Diffuspace/Clover-35B-A3B-Preview" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Diffuspace/Clover-35B-A3B-Preview",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use Diffuspace/Clover-35B-A3B-Preview with Docker Model Runner:
```
docker model run hf.co/Diffuspace/Clover-35B-A3B-Preview
```

Clover in CMF format: runs with cortiq (Rust, Vulkan/CPU, 4-bit and 8-bit files)

by infosave - opened about 16 hours ago

Discussion

infosave

about 16 hours ago

Hi DiffuSpace team, thanks for releasing Clover!

We added Clover-35B-A3B-Preview to CMF, the single-file format of cortiq (a Rust inference engine: no Python, no PyTorch). Block diffusion runs natively from the file: LowConfidence over 32-token blocks at your threshold, bidirectional block previews, the Gated DeltaNet preview operator, the MoE block coreset, and your response-only penalties and stop / </think> ramps.

Files: https://huggingface.co/infosave/Clover-35B-A3B-Preview-cmf

file	size	wikitext-2 ppl
`clover-35b-a3b-preview-q4tp-v2.cmf` (4-bit experts, GPTQ, default)	19.36 GB	16.80
`clover-35b-a3b-preview-q4tp.cmf` (4-bit experts)	19.38 GB	18.22
`clover-35b-a3b-preview-q8_2f.cmf` (8-bit)	35.00 GB	15.86

Run:

cargo install cortiq-cli
hf download infosave/Clover-35B-A3B-Preview-cmf clover-35b-a3b-preview-q4tp-v2.cmf --local-dir .
cortiq run clover-35b-a3b-preview-q4tp-v2.cmf --prompt "Write a Python function that merges two sorted lists." --no-think --max-tokens 512

Prebuilt binaries for Linux, macOS and Windows: https://github.com/infosave2007/cmf/releases (0.8.40 or later).

Checked against your PyTorch implementation: identical prompt ids, and 30–32 of 32 rows agree on the top-1 token in the first denoising pass. On an RTX 5090 (Vulkan) the whole model runs on the card in 18.8 GB: a 32-row pass takes 34–36 ms, prose ~26 tok/s at the default 0.95 threshold, code 80–95 tok/s (124–150 at threshold 0.8).

For those asking about Vulkan and 4-bit in #1–#3: a Vulkan GPU with 24 GB+ holds the whole q4tp-v2 file; a smaller card keeps the leading layers on the GPU and runs the rest on the CPU.

jx25520

Diffuspace org about 15 hours ago

Thanks, will have to give it a try when everything else is done!

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment