3D Dataset

The Hub previews 3D files in the browser, so you can inspect meshes, scenes, point clouds and Gaussian splats without downloading them. This guide covers:

• Previewing 3D files in an interactive viewer
• 3D files in the Dataset Viewer, with Mesh columns and a gallery of assets
• Structuring your dataset so 3D files show up next to your other columns
• SimReady datasets of OpenUSD assets for robotics simulation

Previewing 3D files

Click a 3D file in the Files and versions tab of a dataset or Storage Bucket to open it in an interactive 3D viewer. You can rotate, zoom and open the viewer in full screen.

glTF, mesh, point cloud and Gaussian splat files over 100 MB load when you click Load 3D preview. Files over 1 GB aren’t previewed.

Meshes and glTF

glTF files are rendered with their materials and textures. Use the buttons in the top right corner to switch between Solid, Wireframe and Solid + wireframe.

Point clouds

A .ply file with only vertices, such as a LiDAR or photogrammetry scan, is shown as a point cloud with its vertex colors:

Gaussian splats

Gaussian splats can be stored as .splat or .ply files:

OpenUSD

A USD asset is often split across several files: a root file that references sublayers, payloads, materials and textures. The viewer loads these referenced files from the same repository, so keep the folder structure you authored the asset with, and use relative paths for references. References to absolute paths, or to files outside the repository, can’t be loaded.

3D files in the Dataset Viewer

Mesh columns

The Dataset Viewer shows a thumbnail for each cell of a Mesh column. Thumbnails can be rendered for .glb, .ply, .stl and .obj files.

Click a thumbnail to open the file in 3D, next to the other values of its row:

3D Assets card

The dataset page also shows a 3D Assets card with up to 15 thumbnails from the first rows of the selected subset and split, like the one at the top of this page. Make sure the first rows have a mesh, otherwise the card doesn’t appear.

For example, see cfahlgren1/smithsonian-3d-glb.

Structuring your dataset

Only 3D files

If your dataset only contains .glb, .ply or .stl files, you don’t need to do anything: they get a mesh column automatically.

my_dataset_repository/
└── meshes
    ├── chair.glb
    ├── lamp.glb
    └── table.glb

Additional columns

To add captions, labels or other columns, add a metadata.jsonl (or metadata.csv) file with a file_name column next to your 3D files:

my_dataset_repository/
└── train
    ├── box.glb
    ├── sphere.glb
    └── metadata.jsonl

See Create a mesh dataset for more details, and cfahlgren1/mesh-doc-test-metadata for an example.

Parquet

If your dataset is stored as Parquet, the Dataset Viewer reads the Parquet files and doesn’t add the 3D files in the repository as a column. To show 3D files next to your other columns, keep the files in the repository and add a Mesh column that references them:

my_dataset_repository/
├── meshes
│   ├── box.glb
│   ├── cone.stl
│   └── ...
└── train.parquet

import json
import pyarrow as pa
import pyarrow.parquet as pq
from datasets import Features, Mesh, Value

table = pq.read_table("train.parquet")

cells = [
    {"bytes": None, "path": f"meshes/{row_id}.glb"}  # relative path from parquet directory to meshes
    for row_id in table["id"].to_pylist()
]
table = table.append_column("mesh", pa.array(cells, type=Mesh().pa_type))

# List all the columns of your dataset
features = Features({"id": Value("string"), "prompt": Value("string"), "mesh": Mesh()})
table = table.replace_schema_metadata(
    {"huggingface": json.dumps({"info": {"features": features.to_dict()}})}
)
pq.write_table(table, "train.parquet")

• Paths must be relative to the directory containing the Parquet file.
• Use None for rows without a 3D file.
• A folder can contain up to 10,000 files, so group larger sets of files into subfolders.

See cfahlgren1/mesh-doc-test-formats for an example.

SimReady datasets

SimReady is a specification for OpenUSD assets that are ready to use in simulation, with physics, materials and semantic labels. Datasets that contain SimReady packages show a 3D Assets card on their page, with the number of packages and up to 15 thumbnails sampled across the dataset. Clicking a thumbnail opens its USD file in the viewer.

For example, see nvidia/simready-assets or nvidia/PhysicalAI-SimReady-Warehouse-01.

These datasets also get the simready tag, so they show up when filtering datasets.

Package structure

A package is a folder that contains a com.nvidia.simready.packaging.json file. Packages can be at any depth in the repository, and folders inside a package are part of that package.

Each package needs a thumbnail of its root USD file to appear in the card. Following the SimReady requirement for thumbnails, the thumbnail of <folder>/<file>.usd is <folder>/.thumbs/256x256/<file>.usd.png (or .usda.png, .usdc.png, .usdz.png), in the package folder or one folder below it:

my_dataset_repository/
├── IV_pole_a01
│   ├── com.nvidia.simready.packaging.json
│   └── simready_usd
│       ├── .thumbs
│       │   └── 256x256
│       │       ├── sm_IV_pole_a01_01.usd.png
│       │       └── sm_IV_pole_a01_01.usd_light.png
│       ├── sm_IV_pole_a01_01.usd
│       └── payloads
│           └── ...
└── IV_pole_a02
    ├── com.nvidia.simready.packaging.json
    └── ...

• An optional <file>.usd_light.png is shown in light mode, and <file>.usd.png in dark mode.
• Thumbnails larger than 4 MB are skipped.

The card is updated after each push to the dataset.

Modality tag

Datasets with 3D files get the 3d modality, so they show up when filtering datasets by modality. You can also set it in the dataset card, see Force set a dataset modality.