Nemotron-Labs-Sigma-8B-instruct

Model Overview

Description

Nemotron-Labs-Sigma-8B-instruct is a 8-billion-parameter continuous diffusion language model. Instead of generating text one token at a time, it denoises Gaussian noise in a low-dimensional (16-D) continuous token-embedding space, where the denoiser network and the token embeddings are learned jointly by likelihood optimization. Text is decoded in parallel blocks: each block of tokens is refined over a configurable number of denoising steps (Denoising Diffusion Probabilistic Model (DDPM) or Denoising Diffusion Implicit Model (DDIM) samplers) with self-conditioning, while previously committed tokens are reused through a standard key-value (KV) cache. Fewer steps trade a little quality for large speedups, and the smooth continuous representation makes the model amenable to few-step distillation and trajectory steering.

This is the SFT (supervised fine-tuned) variant, fine-tuned for instruction following on top of the corresponding Base model. Use the bundled chat template to format conversations.

This model is ready for commercial or non-commercial use.

Highlights

  • Continuous-space diffusion language modeling at 8B scale with reasoning accuracy competitive with state-of-the-art masked (discrete) diffusion LMs.
  • Block-parallel decoding: whole blocks of tokens are denoised together; sampling quality holds up well down to few denoising steps per block.
  • Efficient DDPM/DDIM-based samplers with self-conditioning and optional classifier-free guidance, implemented in plain PyTorch in this repository.
  • Standard transformers loading via trust_remote_code; no custom kernels, flash-attention, or training framework required for inference.

License/Terms of Use

Use of this model is governed by the NVIDIA Nemotron Open Model License.

Deployment Geography

Global

Use Case

Developers and researchers building and studying instruction-following assistants based on diffusion parallel decoding: chat, question answering, reasoning, and code generation workloads, and research on efficient few-step text generation. The model runs offline on the user's own hardware; NVIDIA hosts no service.

Release Date

References

Model Architecture

  • Architecture Type: Transformer
  • Network Architecture: Other: Ministral-style decoder-only Transformer (grouped-query attention, SwiGLU, rotary position embeddings) with a continuous-diffusion generation head (16-D continuous token-embedding space, learned jointly with the denoiser).
  • Number of model parameters: 8 billion (8B)
  • This model was developed based on: Nemotron-Labs-Sigma-8B

Input

  • Input Type(s): Text
  • Input Format(s): String
  • Input Parameters: One-Dimensional (1D): Sequences
  • Other Properties Related to Input: Maximum context length of 8,192 tokens (prompt plus generated tokens). Conversations should be formatted with the bundled chat template (chat_template.jinja). Supported language: English.

Output

  • Output Type(s): Text
  • Output Format: String
  • Output Parameters: One-Dimensional (1D): Sequences
  • Other Properties Related to Output: Up to 8,192 tokens total (prompt plus generation). Text is generated in parallel blocks (default 64 tokens per block) with a configurable number of denoising steps per block (Number of Function Evaluations (NFE) = steps + 1 per block); fewer steps give faster generation at some quality cost.

Our AI models are designed and/or optimized to run on NVIDIA GPU-accelerated systems. By leveraging NVIDIA's hardware (e.g. GPU cores) and software frameworks (e.g., CUDA libraries), the model achieves faster training and inference times compared to CPU-only solutions.

Software Integration

  • Runtime Engine(s): PyTorch (>= 2.4) with Hugging Face Transformers (>= 4.55), loading this repository's inference code via trust_remote_code
  • Supported Hardware Microarchitecture Compatibility: NVIDIA Ampere, NVIDIA Blackwell, NVIDIA Hopper, NVIDIA Lovelace
  • Preferred/Supported Operating System(s): Linux

Run the model on a CUDA-capable NVIDIA GPU in bfloat16. CPU inference is not supported: the diffusion sampler is numerically degraded on CPU and produces unusable output.

The integration of foundation and fine-tuned models into AI systems requires additional testing using use-case-specific data to ensure safe and effective deployment. Following the V-model methodology, iterative testing and validation at both unit and system levels are essential to mitigate risks, meet technical and functional requirements, and ensure compliance with safety and ethical standards before deployment.

Model Version(s)

  • v1.0

Environment

torch>=2.4
transformers>=4.55

A CUDA-capable NVIDIA GPU is required (bfloat16); CPU inference is not supported.

Chat with This Model

import torch
from transformers import AutoModel, AutoTokenizer

repo = "nvidia/Nemotron-Labs-Sigma-8B-instruct"
tokenizer = AutoTokenizer.from_pretrained(repo, trust_remote_code=True)
model = AutoModel.from_pretrained(repo, trust_remote_code=True)
model = model.cuda().to(torch.bfloat16).eval()

history = [
    {"role": "system", "content": ""},
    {"role": "user", "content": "What is 15% of 240?"},
]
prompt = tokenizer.apply_chat_template(
    history, tokenize=False, add_generation_prompt=True, enable_thinking=False
)
input_ids = tokenizer(prompt, return_tensors="pt").input_ids.cuda()

out_ids, nfe = model.generate(
    input_ids,
    max_new_tokens=256,
    sampler="ddim",           # recommended (DDPM degenerates at low steps)
    time_schedule="log_snr",  # noise grid used by the internal evals
    steps=32,                 # denoising steps per 32-token block
    block_length=32,          # tokens decoded in parallel
    guidance_scale=3.0,       # classifier-free guidance
    score_temp=0.3,           # see the table below for other step counts
)
print(tokenizer.decode(out_ids[0, input_ids.shape[1]:], skip_special_tokens=True))
print(f"[Number of Function Evaluations (NFE)={nfe}]")

Recommended sampler settings

Per-budget settings follow the paper's few-step results table and apply to both model sizes, base and instruct. Use the DDIM sampler on the log_snr grid with 32-token blocks; lower step counts need stronger score temperature to stay stable.

Denoising steps Classifier-free guidance Score temperature
DDIM-32 3 0.3
DDIM-16 3 0.4
DDIM-8 3 0.6
DDIM-4 2 0.8

Below roughly 16 steps the pretrained models lose quality noticeably; for few-step generation prefer the distilled -distill-* variants, which are trained for a fixed step count and need neither guidance nor score-temperature tuning.

Prompting convention

Use the chat template for conversational and reasoning tasks (see the example above). For code-completion benchmarks such as HumanEval and MBPP, pass the raw prompt without the chat template: those tasks hand the model a function signature to continue, and wrapping them in a chat turn makes the model answer conversationally instead of completing the code.

Evaluation Results

Benchmark results for this model family, averaged over multiple generation seeds, are reported in the technical report: Large Language Continuous Diffusion Models.

Training, Testing, and Evaluation Datasets

Training Dataset

  • Data Modality: Text
  • Text Training Data Size: 1 Trillion to 10 Trillion Tokens
  • Data Collection Method by dataset: Hybrid: Automated, Manually-Collected
  • Labeling Method by dataset: Hybrid: Automated, Manually-Labeled, Synthetic

Properties: The model was trained on an internal NVIDIA text corpus of approximately 1.3 trillion tokens in total (about 1 trillion tokens of autoregressive initialization followed by about 0.3 trillion tokens of continuous-diffusion training). The corpus consists of curated English text from webpages, articles, dialogue, code, and other written materials spanning domains such as math, science, legal, and finance. It is the same training corpus used for the released Nemotron-Labs-Diffusion model family; the training data is not distributed.

The SFT variant was additionally fine-tuned on an internal NVIDIA-curated composite supervised fine-tuning dataset (conversations, reasoning, math, and coding instruction data; English). The fine-tuning data is not distributed.

Testing Dataset

  • Data Collection Method by dataset: Hybrid: Automated, Manually-Collected
  • Labeling Method by dataset: Hybrid: Automated, Manually-Labeled

Properties: Held-out splits of the training corpus and public benchmark suites were used for testing during development. These comprise English text covering reasoning, math, and code tasks.

Evaluation Dataset

  • Data Collection Method by dataset: Hybrid: Automated, Manually-Collected
  • Labeling Method by dataset: Hybrid: Automated, Manually-Labeled

Properties: Public academic benchmarks for reasoning and code generation: GSM8K (grade-school math word problems), MBPP and MBPP+ (Python programming problems), and HumanEval (Python code synthesis). These are English-language benchmark datasets with reference answers or unit tests.

Inference

  • Acceleration Engine: PyTorch
  • Test Hardware: NVIDIA Hopper H100

Measured generation throughput: roughly 120-140 tokens/s for 256 generated tokens with the DDPM sampler at 32 denoising steps per 64-token block on a single NVIDIA Hopper H100 (bfloat16).

Ethical Considerations

NVIDIA believes Trustworthy AI is a shared responsibility and we have established policies and practices to enable development for a wide array of AI applications. Developers should work with their internal model team to ensure this model meets requirements for the relevant industry and use case and addresses unforeseen product misuse.

For more detailed information on ethical considerations for this model, please see the Model Card++ Bias, Explainability, Safety & Security, and Privacy Subcards.

Please report model quality, risk, security vulnerabilities or NVIDIA AI Concerns here.

Citations

@article{nemotronlabssigma2026,
  title       = {Large Language Continuous Diffusion Models},
  author      = {Zhihan Yang and Wei Guo and Jean-Marie Lemercier and Simon Welker and
                 Mohammad Mahdi Kamani and Yonggan Fu and Sajad Norouzi and Julius Berner and
                 Tomas Geffner and Karsten Kreis and Yongxin Chen and Molei Tao and
                 John Thickstun and Pavlo Molchanov and Ante Juki{\'c} and Arash Vahdat and
                 Morteza Mardani},
  journal     = {arXiv preprint arXiv:2610.02665},
  year        = {2026},
  url         = {https://arxiv.org/abs/2610.02665}
}
Downloads last month
22
Safetensors
Model size
8B params
Tensor type
BF16
·
F32
·
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Collection including nvidia/Nemotron-Labs-Sigma-8B-instruct

Paper for nvidia/Nemotron-Labs-Sigma-8B-instruct