Instructions to use PrimeIntellect/DeepSeek-V4.1-Flash-bf16 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use PrimeIntellect/DeepSeek-V4.1-Flash-bf16 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="PrimeIntellect/DeepSeek-V4.1-Flash-bf16")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained("PrimeIntellect/DeepSeek-V4.1-Flash-bf16", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use PrimeIntellect/DeepSeek-V4.1-Flash-bf16 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "PrimeIntellect/DeepSeek-V4.1-Flash-bf16"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "PrimeIntellect/DeepSeek-V4.1-Flash-bf16",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/PrimeIntellect/DeepSeek-V4.1-Flash-bf16

SGLang

How to use PrimeIntellect/DeepSeek-V4.1-Flash-bf16 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "PrimeIntellect/DeepSeek-V4.1-Flash-bf16" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "PrimeIntellect/DeepSeek-V4.1-Flash-bf16",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "PrimeIntellect/DeepSeek-V4.1-Flash-bf16" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "PrimeIntellect/DeepSeek-V4.1-Flash-bf16",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use PrimeIntellect/DeepSeek-V4.1-Flash-bf16 with Docker Model Runner:
```
docker model run hf.co/PrimeIntellect/DeepSeek-V4.1-Flash-bf16
```

DeepSeek-V4.1-Flash-bf16 / README.md

samsja

Add files using upload-large-folder tool

80f30f0 verified about 1 month ago

preview code

Raw History Blame Contribute Delete

1.72 kB

---
license: mit
base_model: deepseek-ai/DeepSeek-V4.1-Flash
library_name: transformers
---
# DeepSeek-V4.1-Flash-bf16

A bfloat16 re-export of [deepseek-ai/DeepSeek-V4.1-Flash](https://huggingface.co/deepseek-ai/DeepSeek-V4.1-Flash)
under the published tensor names. The dense linears (fp8 e4m3, ue8m0 scales over 32x32 blocks) and
the routed experts (MXFP4, ue8m0 scales over 1x32 groups) are dequantized to plain bfloat16. Every
fp8 / fp4 value times its power-of-two scale is representable in bfloat16, so the dequantization is
exact. 48 safetensors shards, 1.34 TB.

## Differences from upstream

- `quantization_config` removed from `config.json`. An engine that reads it allocates fp8 and MXFP4
  parameters, which bfloat16 weights cannot fill.
- The two engram tables (`layers.1.engram.embed` and `layers.14.engram.embed`, ~98B parameters
  each) are **not** dequantized: they keep the upstream fp8 e4m3 weight and 1x32 ue8m0
  `.scale`, because vLLM's engram is fp8-only. In bfloat16 they would add ~197 GB.
- Everything else is copied unchanged, including the vision tower, the MTP / DSpark weights, the
  tokenizer, `encoding/` and `inference/`.
- Not everything is bfloat16: mHC, attention-sink and bias tensors stay fp32. The export holds
  48,107 bf16, 387 fp32, 2 fp8 e4m3 and 2 e8m0 tensors.

## Use

- vLLM: serve it without `quantization_config`; pass `quantization = "fp8"` (online per-block
  fp8) to quantize the bf16 weights at load.
- [prime-rl](https://github.com/PrimeIntellect-ai/prime-rl): use it as `model.name`. The trainer
  dequantizes the engram tables and renames to its own layout on first load.

## License

MIT, inherited from upstream. The upstream `LICENSE` is included verbatim.

	---
	license: mit
	base_model: deepseek-ai/DeepSeek-V4.1-Flash
	library_name: transformers
	---

	# DeepSeek-V4.1-Flash-bf16

	A bfloat16 re-export of [deepseek-ai/DeepSeek-V4.1-Flash](https://huggingface.co/deepseek-ai/DeepSeek-V4.1-Flash)
	under the published tensor names. The dense linears (fp8 e4m3, ue8m0 scales over 32x32 blocks) and
	the routed experts (MXFP4, ue8m0 scales over 1x32 groups) are dequantized to plain bfloat16. Every
	fp8 / fp4 value times its power-of-two scale is representable in bfloat16, so the dequantization is
	exact. 48 safetensors shards, 1.34 TB.

	## Differences from upstream

	- `quantization_config` removed from `config.json`. An engine that reads it allocates fp8 and MXFP4
	parameters, which bfloat16 weights cannot fill.
	- The two engram tables (`layers.1.engram.embed` and `layers.14.engram.embed`, ~98B parameters
	each) are not dequantized: they keep the upstream fp8 e4m3 weight and 1x32 ue8m0
	`.scale`, because vLLM's engram is fp8-only. In bfloat16 they would add ~197 GB.
	- Everything else is copied unchanged, including the vision tower, the MTP / DSpark weights, the
	tokenizer, `encoding/` and `inference/`.
	- Not everything is bfloat16: mHC, attention-sink and bias tensors stay fp32. The export holds
	48,107 bf16, 387 fp32, 2 fp8 e4m3 and 2 e8m0 tensors.

	## Use

	- vLLM: serve it without `quantization_config`; pass `quantization = "fp8"` (online per-block
	fp8) to quantize the bf16 weights at load.
	- [prime-rl](https://github.com/PrimeIntellect-ai/prime-rl): use it as `model.name`. The trainer
	dequantizes the engram tables and renames to its own layout on first load.

	## License

	MIT, inherited from upstream. The upstream `LICENSE` is included verbatim.