Text Generation
Transformers
Safetensors
English
deepseek_v3
deepseek
unsloth
conversational
custom_code
text-generation-inference
Instructions to use unsloth/DeepSeek-R1-BF16 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use unsloth/DeepSeek-R1-BF16 with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="unsloth/DeepSeek-R1-BF16", trust_remote_code=True) messages = [ {"role": "user", "content": "Who are you?"}, ] pipe(messages)# pip install -U transformers accelerate # Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("unsloth/DeepSeek-R1-BF16", trust_remote_code=True) model = AutoModelForCausalLM.from_pretrained("unsloth/DeepSeek-R1-BF16", trust_remote_code=True, device_map="auto") messages = [ {"role": "user", "content": "Who are you?"}, ] inputs = tokenizer.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=256) print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use unsloth/DeepSeek-R1-BF16 with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "unsloth/DeepSeek-R1-BF16" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "unsloth/DeepSeek-R1-BF16", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/unsloth/DeepSeek-R1-BF16
- SGLang
How to use unsloth/DeepSeek-R1-BF16 with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "unsloth/DeepSeek-R1-BF16" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "unsloth/DeepSeek-R1-BF16", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "unsloth/DeepSeek-R1-BF16" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "unsloth/DeepSeek-R1-BF16", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }' - Unsloth Desktop
- Docker Model Runner
How to use unsloth/DeepSeek-R1-BF16 with Docker Model Runner:
docker model run hf.co/unsloth/DeepSeek-R1-BF16
Commit History
-
Fix chat_template crash when assistant message omits the `content` key
Verified signature## Fix `chat_template` to handle assistant messages without a `content` key **⚠️ This template will start crashing for every tool-calling user as soon as the next `transformers` release ships.** The upstream PR https://github.com/huggingface/transformers/pull/45422 normalizes message inputs by stripping `content=None` before rendering (`None` and absent are semantically identical, and `content=None` is exactly what the OpenAI API returns for tool-call-only messages). That normalization is correct, but it exposes a latent bug in this template: the `tool_calls` branch reads `message['content']` directly, which raises when the key is absent. Concretely, this code path is hit by **any tool-calling pipeline** (OpenAI-compatible servers, agent frameworks, function-calling demos) that produces assistant messages with `tool_calls` and no textual content. Today most of them happen to pass `content=None` explicitly and get away with it. After the transformers release, all of them break. ### …ef2f6c5
-
Upload tokenizer
Verified signature7d1258f
-
Add files using upload-large-folder tool
Verified signature5898ef6
-
Update README.md
Verified signatured0e8ab8 -
Create README.md
Verified signature13e2843 -
Add files using upload-large-folder tool
Verified signaturefb3ec6c
-
Upload model-00163-of-000163.safetensors with huggingface_hub
Verified signature8922d26 -
Upload model-00162-of-000163.safetensors with huggingface_hub
Verified signature6c67ade -
Upload model-00161-of-000163.safetensors with huggingface_hub
Verified signature0b0db15 -
Upload model-00160-of-000163.safetensors with huggingface_hub
Verified signatured510332 -
Upload model-00159-of-000163.safetensors with huggingface_hub
Verified signature088d712 -
Upload model-00158-of-000163.safetensors with huggingface_hub
Verified signature1cd250b -
Upload model-00157-of-000163.safetensors with huggingface_hub
Verified signature0868986 -
Upload model-00156-of-000163.safetensors with huggingface_hub
Verified signatureb047780 -
Upload model-00155-of-000163.safetensors with huggingface_hub
Verified signature4e1ae23 -
Upload model-00154-of-000163.safetensors with huggingface_hub
Verified signature69983cf -
Upload model-00153-of-000163.safetensors with huggingface_hub
Verified signatureaba402a -
Upload model-00152-of-000163.safetensors with huggingface_hub
Verified signature77693a8 -
Upload model-00151-of-000163.safetensors with huggingface_hub
Verified signature7b2cc5b -
Upload model-00150-of-000163.safetensors with huggingface_hub
Verified signature3e07587 -
Upload model-00149-of-000163.safetensors with huggingface_hub
Verified signature6588e6e -
Upload model-00148-of-000163.safetensors with huggingface_hub
Verified signature16f6941 -
Upload model-00147-of-000163.safetensors with huggingface_hub
Verified signatureeb23eae -
Upload model-00146-of-000163.safetensors with huggingface_hub
Verified signature15980d6 -
Upload model-00145-of-000163.safetensors with huggingface_hub
Verified signature4c9caaa -
Upload model-00144-of-000163.safetensors with huggingface_hub
Verified signature98e2882 -
Upload model-00143-of-000163.safetensors with huggingface_hub
Verified signature5c711e8 -
Upload model-00142-of-000163.safetensors with huggingface_hub
Verified signature945c8c8 -
Upload model-00141-of-000163.safetensors with huggingface_hub
Verified signature6095b35 -
Upload model-00140-of-000163.safetensors with huggingface_hub
Verified signaturef31cbea -
Upload model-00139-of-000163.safetensors with huggingface_hub
Verified signature1db830c -
Upload model-00138-of-000163.safetensors with huggingface_hub
Verified signature08c010b -
Upload model-00137-of-000163.safetensors with huggingface_hub
Verified signaturec61cd7b -
Upload model-00136-of-000163.safetensors with huggingface_hub
Verified signaturecec6497 -
Upload model-00135-of-000163.safetensors with huggingface_hub
Verified signature85227ab -
Upload model-00134-of-000163.safetensors with huggingface_hub
Verified signature93b154e -
Upload model-00133-of-000163.safetensors with huggingface_hub
Verified signature0b83dfc -
Upload model-00132-of-000163.safetensors with huggingface_hub
Verified signaturee13be1b -
Upload model-00131-of-000163.safetensors with huggingface_hub
Verified signaturefc15da8 -
Upload model-00130-of-000163.safetensors with huggingface_hub
Verified signature33098d3 -
Upload model-00129-of-000163.safetensors with huggingface_hub
Verified signaturea755bd1 -
Upload model-00128-of-000163.safetensors with huggingface_hub
Verified signature890638a -
Upload model-00127-of-000163.safetensors with huggingface_hub
Verified signature2766b61 -
Upload model-00126-of-000163.safetensors with huggingface_hub
Verified signatureefb1c4e -
Upload model-00125-of-000163.safetensors with huggingface_hub
Verified signature8482fb4 -
Upload model-00124-of-000163.safetensors with huggingface_hub
Verified signature94250bc -
Upload model-00123-of-000163.safetensors with huggingface_hub
Verified signature9d787ee -
Upload model-00122-of-000163.safetensors with huggingface_hub
Verified signature1ff8f5f -
Upload model-00121-of-000163.safetensors with huggingface_hub
Verified signature27023b4 -
Upload model-00120-of-000163.safetensors with huggingface_hub
Verified signatureb7ab8fb