DVSM Overview

Description:

DVSM is a decoder-only Transformer for novel-view synthesis. It takes user input images and reconstruct the scene as KV cache. The user then can do arbitrary novel-view synthesis within that scene.
The model was developed by NVIDIA CORPORATION as a part of DVSM.
This model is for research and development only.

Third-Party Community Consideration

This model is not owned or developed by NVIDIA. This model has been developed and built to a third-party's requirements for this application and use case; see link to Non-NVIDIA [Not Applicable (N/A) Model Card](Not Applicable (N/A)).

License/Terms of Use:

NSCLv1

Deployment Geography:

Global

Use Case:

Researchers and developers working on computer vision and graphics who need to generate novel views of 3D scenes from sparse input images. The model supports view synthesis tasks on datasets like DL3DV and Re10k, enabling applications in virtual reality, augmented reality, and 3D content creation.

Release Date:

HuggingFace: 30 June, 2026 via https://huggingface.co/nvidia/DVSM

Reference(s):

Technical report

Model Architecture:

Architecture Type: Transformer
Network Architecture: Each block consists of a intral-view attention and a cross-view attention layer, each followed by a MLP layer. We stack 12 blocks with 768 hidden dimensions.
This model was developed based on Not Applicable (N/A).
Number of model parameters: 163M

Input:

Input Type(s): Image
Input Format(s): RGB (Red, Green, Blue)
Input Parameters: Two-Dimensional (2D)
Other Properties Related to Input: We provide models working on 340p, 480p, 720p, and 960p resolutions. Cameras are normalized and embedded as plucker ray map. Linear patchfiers are applied with 8x8 patch size.

Output:

Output Type(s): Image
Output Format: Other: RGB (Red, Green, Blue)
Output Parameters: Two-Dimensional (2D)
Other Properties Related to Output: The output is in the same resolution as the input images.

Our AI models are designed and/or optimized to run on NVIDIA GPU-accelerated systems. By leveraging NVIDIA's hardware (e.g. GPU cores) and software frameworks (e.g., CUDA libraries), the model achieves faster training and inference times compared to CPU-only solutions.

Software Integration:

Runtime Engine(s): Transformers
Supported Hardware Microarchitecture Compatibility:

NVIDIA Ampere
NVIDIA Blackwell
NVIDIA Lovelace
NVIDIA Hopper

Supported Operating System(s): Linux

The integration of foundation and fine-tuned models into AI systems requires additional testing using use-case-specific data to ensure safe and effective deployment. Following the V-model methodology, iterative testing and validation at both unit and system levels are essential to mitigate risks, meet technical and functional requirements, and ensure compliance with safety and ethical standards before deployment.

Model Version(s):

DVSM

To integrate DVSM, start from the NVIDIA PyTorch container (nvcr.io/nvidia/pytorch:25.10-py3) and install the required Python packages including PyTorch, Transformers, Diffusers, Accelerate, and additional libraries such as OpenCV, Einops, XFormers, and others listed in the requirements. Then mount the dataset and checkpoint directories, and launch training or inference using the provided SLURM scripts or direct python calls.

Training, Testing, and Evaluation Datasets:

Training Dataset:

Data Modality: Image

Image Training Data Size: 1 Million to 1 Billion Images
Data Collection Method by dataset: Automatic/Sensors
Labeling Method by dataset: Automated
Properties (Quantity, Dataset Descriptions, Sensor(s)): DL3DV: 10K image sequences, camera sensor Re10K: 10K image sequences, camera sensor

Testing Dataset:

Data Collection Method by dataset: Automatic/Sensors
Labeling Method by dataset: Automated
Properties (Quantity, Dataset Descriptions, Sensor(s)): DL3DV: 10K image sequences, camera sensor Re10K: 10K image sequences, camera sensor

Evaluation Dataset:

Benchmark Score: None

Benchmark	Metric	Score
DL3DV 64 views 960p	PSNR	29.71
DL3DV 32 views 960p	PSNR	27.96
DL3DV 16 views 960p	PSNR	25.88
Re10K 2 views 512x512	PSNR	30.17
MipNerf360 64 views	PSNR	24.76
Free 64 views	PSNR	25.57
Hike 64 views	PSNR	23.10

Inference:

Acceleration Engine: PyTorch, Other: flash-attention Test Hardware: NVIDIA A100 80GB

Ethical Considerations:

NVIDIA believes Trustworthy AI is a shared responsibility and we have established policies and practices to enable development for a wide array of AI applications. When downloaded or used in accordance with our terms of service, developers should work with their internal model team to ensure this model meets requirements for the relevant industry and use case and addresses unforeseen product misuse.
Please make sure you have proper rights and permissions for all input image and video content; if image or video includes people, personal health information, or intellectual property, the image or video generated will not blur or maintain proportions of image subjects included.
For more detailed information on ethical considerations for this model, please see the Model Card++ Explainability, Bias, Safety & Security, and Privacy Subcards.

Generated by NVIDIA Model Card Generator Toolkit.