Instructions to use nvidia/Nemotron-Labs-Sigma-8B-instruct with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use nvidia/Nemotron-Labs-Sigma-8B-instruct with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="nvidia/Nemotron-Labs-Sigma-8B-instruct", trust_remote_code=True) messages = [ {"role": "user", "content": "Who are you?"}, ] pipe(messages)# pip install -U transformers accelerate # Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("nvidia/Nemotron-Labs-Sigma-8B-instruct", trust_remote_code=True, device_map="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use nvidia/Nemotron-Labs-Sigma-8B-instruct with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "nvidia/Nemotron-Labs-Sigma-8B-instruct" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "nvidia/Nemotron-Labs-Sigma-8B-instruct", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/nvidia/Nemotron-Labs-Sigma-8B-instruct
- SGLang
How to use nvidia/Nemotron-Labs-Sigma-8B-instruct with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "nvidia/Nemotron-Labs-Sigma-8B-instruct" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "nvidia/Nemotron-Labs-Sigma-8B-instruct", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "nvidia/Nemotron-Labs-Sigma-8B-instruct" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "nvidia/Nemotron-Labs-Sigma-8B-instruct", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }' - Docker Model Runner
How to use nvidia/Nemotron-Labs-Sigma-8B-instruct with Docker Model Runner:
docker model run hf.co/nvidia/Nemotron-Labs-Sigma-8B-instruct
- Nemotron-Labs-Sigma-8B-instruct
- Model Overview
- Description
- Highlights
- License/Terms of Use
- Deployment Geography
- Use Case
- Release Date
- References
- Model Architecture
- Input
- Output
- Software Integration
- Model Version(s)
- Environment
- Chat with This Model
- Evaluation Results
- Training, Testing, and Evaluation Datasets
- Inference
- Ethical Considerations
- Citations
- Model Overview
Nemotron-Labs-Sigma-8B-instruct
Model Overview
Description
Nemotron-Labs-Sigma-8B-instruct is a 8-billion-parameter continuous diffusion language model. Instead of generating text one token at a time, it denoises Gaussian noise in a low-dimensional (16-D) continuous token-embedding space, where the denoiser network and the token embeddings are learned jointly by likelihood optimization. Text is decoded in parallel blocks: each block of tokens is refined over a configurable number of denoising steps (Denoising Diffusion Probabilistic Model (DDPM) or Denoising Diffusion Implicit Model (DDIM) samplers) with self-conditioning, while previously committed tokens are reused through a standard key-value (KV) cache. Fewer steps trade a little quality for large speedups, and the smooth continuous representation makes the model amenable to few-step distillation and trajectory steering.
This is the SFT (supervised fine-tuned) variant, fine-tuned for instruction following on top of the corresponding Base model. Use the bundled chat template to format conversations.
This model is ready for commercial or non-commercial use.
Highlights
- Continuous-space diffusion language modeling at 8B scale with reasoning accuracy competitive with state-of-the-art masked (discrete) diffusion LMs.
- Block-parallel decoding: whole blocks of tokens are denoised together; sampling quality holds up well down to few denoising steps per block.
- Efficient DDPM/DDIM-based samplers with self-conditioning and optional classifier-free guidance, implemented in plain PyTorch in this repository.
- Standard
transformersloading viatrust_remote_code; no custom kernels, flash-attention, or training framework required for inference.
License/Terms of Use
Use of this model is governed by the NVIDIA Nemotron Open Model License.
Deployment Geography
Global
Use Case
Developers and researchers building and studying instruction-following assistants based on diffusion parallel decoding: chat, question answering, reasoning, and code generation workloads, and research on efficient few-step text generation. The model runs offline on the user's own hardware; NVIDIA hosts no service.
Release Date
- Hugging Face 11/12/2026 via https://huggingface.co/nvidia/Nemotron-Labs-Sigma-8B-instruct
References
- Large Language Continuous Diffusion Models (arXiv:2610.02665, 2026).
Model Architecture
- Architecture Type: Transformer
- Network Architecture: Other: Ministral-style decoder-only Transformer (grouped-query attention, SwiGLU, rotary position embeddings) with a continuous-diffusion generation head (16-D continuous token-embedding space, learned jointly with the denoiser).
- Number of model parameters: 8 billion (8B)
- This model was developed based on: Nemotron-Labs-Sigma-8B
Input
- Input Type(s): Text
- Input Format(s): String
- Input Parameters: One-Dimensional (1D): Sequences
- Other Properties Related to Input: Maximum context length of 8,192 tokens (prompt plus generated tokens). Conversations should be formatted with the bundled chat template (chat_template.jinja). Supported language: English.
Output
- Output Type(s): Text
- Output Format: String
- Output Parameters: One-Dimensional (1D): Sequences
- Other Properties Related to Output: Up to 8,192 tokens total (prompt plus generation). Text is generated in parallel blocks (default 64 tokens per block) with a configurable number of denoising steps per block (Number of Function Evaluations (NFE) = steps + 1 per block); fewer steps give faster generation at some quality cost.
Our AI models are designed and/or optimized to run on NVIDIA GPU-accelerated systems. By leveraging NVIDIA's hardware (e.g. GPU cores) and software frameworks (e.g., CUDA libraries), the model achieves faster training and inference times compared to CPU-only solutions.
Software Integration
- Runtime Engine(s): PyTorch (>= 2.4) with Hugging Face Transformers (>= 4.55), loading this repository's inference code via
trust_remote_code - Supported Hardware Microarchitecture Compatibility: NVIDIA Ampere, NVIDIA Blackwell, NVIDIA Hopper, NVIDIA Lovelace
- Preferred/Supported Operating System(s): Linux
Run the model on a CUDA-capable NVIDIA GPU in bfloat16. CPU inference is not supported: the diffusion sampler is numerically degraded on CPU and produces unusable output.
The integration of foundation and fine-tuned models into AI systems requires additional testing using use-case-specific data to ensure safe and effective deployment. Following the V-model methodology, iterative testing and validation at both unit and system levels are essential to mitigate risks, meet technical and functional requirements, and ensure compliance with safety and ethical standards before deployment.
Model Version(s)
- v1.0
Environment
torch>=2.4
transformers>=4.55
A CUDA-capable NVIDIA GPU is required (bfloat16); CPU inference is not supported.
Chat with This Model
import torch
from transformers import AutoModel, AutoTokenizer
repo = "nvidia/Nemotron-Labs-Sigma-8B-instruct"
tokenizer = AutoTokenizer.from_pretrained(repo, trust_remote_code=True)
model = AutoModel.from_pretrained(repo, trust_remote_code=True)
model = model.cuda().to(torch.bfloat16).eval()
history = [
{"role": "system", "content": ""},
{"role": "user", "content": "What is 15% of 240?"},
]
prompt = tokenizer.apply_chat_template(
history, tokenize=False, add_generation_prompt=True, enable_thinking=False
)
input_ids = tokenizer(prompt, return_tensors="pt").input_ids.cuda()
out_ids, nfe = model.generate(
input_ids,
max_new_tokens=256,
sampler="ddim", # recommended (DDPM degenerates at low steps)
time_schedule="log_snr", # noise grid used by the internal evals
steps=32, # denoising steps per 32-token block
block_length=32, # tokens decoded in parallel
guidance_scale=3.0, # classifier-free guidance
score_temp=0.3, # see the table below for other step counts
)
print(tokenizer.decode(out_ids[0, input_ids.shape[1]:], skip_special_tokens=True))
print(f"[Number of Function Evaluations (NFE)={nfe}]")
Recommended sampler settings
Per-budget settings follow the paper's few-step results table and apply to both model sizes,
base and instruct. Use the DDIM sampler on the log_snr grid with 32-token blocks; lower step
counts need stronger score temperature to stay stable.
| Denoising steps | Classifier-free guidance | Score temperature |
|---|---|---|
| DDIM-32 | 3 | 0.3 |
| DDIM-16 | 3 | 0.4 |
| DDIM-8 | 3 | 0.6 |
| DDIM-4 | 2 | 0.8 |
Below roughly 16 steps the pretrained models lose quality noticeably; for few-step generation
prefer the distilled -distill-* variants, which are trained for a fixed step count and need
neither guidance nor score-temperature tuning.
Prompting convention
Use the chat template for conversational and reasoning tasks (see the example above). For code-completion benchmarks such as HumanEval and MBPP, pass the raw prompt without the chat template: those tasks hand the model a function signature to continue, and wrapping them in a chat turn makes the model answer conversationally instead of completing the code.
Evaluation Results
Benchmark results for this model family, averaged over multiple generation seeds, are reported in the technical report: Large Language Continuous Diffusion Models.
Training, Testing, and Evaluation Datasets
Training Dataset
- Data Modality: Text
- Text Training Data Size: 1 Trillion to 10 Trillion Tokens
- Data Collection Method by dataset: Hybrid: Automated, Manually-Collected
- Labeling Method by dataset: Hybrid: Automated, Manually-Labeled, Synthetic
Properties: The model was trained on an internal NVIDIA text corpus of approximately 1.3 trillion tokens in total (about 1 trillion tokens of autoregressive initialization followed by about 0.3 trillion tokens of continuous-diffusion training). The corpus consists of curated English text from webpages, articles, dialogue, code, and other written materials spanning domains such as math, science, legal, and finance. It is the same training corpus used for the released Nemotron-Labs-Diffusion model family; the training data is not distributed.
The SFT variant was additionally fine-tuned on an internal NVIDIA-curated composite supervised fine-tuning dataset (conversations, reasoning, math, and coding instruction data; English). The fine-tuning data is not distributed.
Testing Dataset
- Data Collection Method by dataset: Hybrid: Automated, Manually-Collected
- Labeling Method by dataset: Hybrid: Automated, Manually-Labeled
Properties: Held-out splits of the training corpus and public benchmark suites were used for testing during development. These comprise English text covering reasoning, math, and code tasks.
Evaluation Dataset
- Data Collection Method by dataset: Hybrid: Automated, Manually-Collected
- Labeling Method by dataset: Hybrid: Automated, Manually-Labeled
Properties: Public academic benchmarks for reasoning and code generation: GSM8K (grade-school math word problems), MBPP and MBPP+ (Python programming problems), and HumanEval (Python code synthesis). These are English-language benchmark datasets with reference answers or unit tests.
Inference
- Acceleration Engine: PyTorch
- Test Hardware: NVIDIA Hopper H100
Measured generation throughput: roughly 120-140 tokens/s for 256 generated tokens with the DDPM sampler at 32 denoising steps per 64-token block on a single NVIDIA Hopper H100 (bfloat16).
Ethical Considerations
NVIDIA believes Trustworthy AI is a shared responsibility and we have established policies and practices to enable development for a wide array of AI applications. Developers should work with their internal model team to ensure this model meets requirements for the relevant industry and use case and addresses unforeseen product misuse.
For more detailed information on ethical considerations for this model, please see the Model Card++ Bias, Explainability, Safety & Security, and Privacy Subcards.
Please report model quality, risk, security vulnerabilities or NVIDIA AI Concerns here.
Citations
@article{nemotronlabssigma2026,
title = {Large Language Continuous Diffusion Models},
author = {Zhihan Yang and Wei Guo and Jean-Marie Lemercier and Simon Welker and
Mohammad Mahdi Kamani and Yonggan Fu and Sajad Norouzi and Julius Berner and
Tomas Geffner and Karsten Kreis and Yongxin Chen and Molei Tao and
John Thickstun and Pavlo Molchanov and Ante Juki{\'c} and Arash Vahdat and
Morteza Mardani},
journal = {arXiv preprint arXiv:2610.02665},
year = {2026},
url = {https://arxiv.org/abs/2610.02665}
}
- Downloads last month
- 22