Instructions to use oxide-lab/TRELLIS.2-4B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- TRELLIS.2
How to use oxide-lab/TRELLIS.2-4B with TRELLIS.2:
# No code snippets available yet for this library. # To use this model, check the repository files and the library's documentation. # Want to help? PRs adding snippets are welcome at: # https://github.com/huggingface/huggingface.js
- Notebooks
- Google Colab
- Kaggle
Lux3D TRELLIS.2-4B Canonical Weights
This repository contains the canonicalized safetensors export of the
TRELLIS.2-4B image-to-3D checkpoint used by the Lux3D runtime.
Model Details
- Model family:
trellis2 - Canonical file:
model.safetensors(16,180,079,934 bytes, about 15.07 GiB) - Tensor count:
4265 - Parameter storage: mixed BF16 / F16 / F32, preserved from the upstream
checkpoints (
BF16: 3200 tensors,F16: 612,F32: 453) - SHA-256:
aaff394292cab30af64624e816eb3ac15901620b10b9e90f89754f47d05afee7 - Raw provenance files:
19(nine safetensors shards, their configs,pipeline.json) - Primary runtime target: Lux3D runtime
- Typical output: textured GLB mesh plus a PBR vertex-attribute sidecar
(
.objand.plyare also supported) - Reference default pipeline:
1024_cascade(512 sparse-structure + shape SLAT pass, then the 1024 shape and texture passes)
TRELLIS.2-4B is Microsoft's 4-billion-parameter image-to-3D model built around the O-Voxel sparse-voxel representation. This inference slice includes:
- the DINOv3 ViT-L/16 image conditioner (
image_cond); - a 64³ sparse-structure flow DiT and its dense 3D-convolution decoder
(
ss_flow,ss_dec); - 512 and 1024 shape SLAT flow DiTs (
shape_dit_512,shape_dit_1024); - the sparse shape VAE decoder that produces the O-Voxel geometry
(
shape_dec); - 512 and 1024 texture SLAT flow DiTs (
tex_dit_512,tex_dit_1024); - the sparse texture VAE decoder with per-voxel PBR attributes (
tex_dec).
The merged package uses one <stage>. prefix per component:
| Stage prefix | Upstream source | Tensors |
|---|---|---|
image_cond |
external/dinov3/model.safetensors |
415 |
ss_flow |
ckpts/ss_flow_img_dit_1_3B_64_bf16.safetensors |
640 |
ss_dec |
external/ss_dec/ckpts/ss_dec_conv3d_16l8_fp16.safetensors |
74 |
shape_dit_512 |
ckpts/slat_flow_img2shape_dit_1_3B_512_bf16.safetensors |
640 |
shape_dit_1024 |
ckpts/slat_flow_img2shape_dit_1_3B_1024_bf16.safetensors |
640 |
shape_dec |
ckpts/shape_dec_next_dc_f16c32_fp16.safetensors |
292 |
tex_dit_512 |
ckpts/slat_flow_imgshape2tex_dit_1_3B_512_bf16.safetensors |
640 |
tex_dit_1024 |
ckpts/slat_flow_imgshape2tex_dit_1_3B_1024_bf16.safetensors |
640 |
tex_dec |
ckpts/tex_dec_next_dc_f16c32_fp16.safetensors |
284 |
| Total | 4265 |
Included Files
model.safetensors— merged canonical weights with the stage prefixes aboveresolved_config.json—pipeline.jsonarguments, every stage config, and the shape/texture SLAT normalization vectorsmanifest.json— family, provenance, source checksums, raw-file order, tensor count, and dtype histogramchecksums.json— SHA-256 and byte size for the canonical data filesREADME.md— this documentLICENSE— MIT license text and third-party notices
No tensor values are modified by canonicalization: the nine shards are merged tensor-by-tensor, keys gain their stage prefix, and original dtypes are preserved byte-exact.
Usage With Lux3D
Validate the family contract and run from an explicit canonical package:
cargo run -p lux3d-cli -- inspect --repo-root <runtime-root> trellis2
cargo run -p lux3d-cli --features cuda -- run trellis2 `
--model-path <canonical-trellis2-dir> `
--rmbg-model-path <canonical-rmbg2-dir> `
--source <input-image> `
--output <output-file.glb>
A faster geometry-only preview:
cargo run -p lux3d-cli --features cuda -- run trellis2 `
--model-path <canonical-trellis2-dir> `
--rmbg-model-path <canonical-rmbg2-dir> `
--source <input-image> `
--pipeline-type 512 --steps 12 --texture false --seed 42 `
--output <preview.obj>
--pipeline-type accepts 512 and 1024_cascade. The single-resolution
1024 path and the 1536-token cascade are not implemented by Lux3D yet.
The raw microsoft/TRELLIS.2-4B checkpoint directory (pipeline.json plus
ckpts/ and external/) is accepted as-is by the CLI, so the raw tree does
not have to be normalized before a local run.
Auto-download is reserved for the oxide-lab/TRELLIS.2-4B repository. If that
repository is not available to your account, pass an explicit --model-path.
Background removal
TRELLIS.2's reference preprocessing runs RMBG-2.0/BiRefNet when the input has
no usable alpha channel. Those weights are not part of this package.
Normalize them as the separate rmbg2 family and pass
--rmbg-model-path <canonical-rmbg2-dir>, or place an rmbg-2.0 directory
next to --model-path. A missing RMBG package is an error for RGB inputs;
Lux3D never silently treats a background as opaque. RGBA inputs with a usable
alpha channel use the alpha path and do not load RMBG.
The HTTP server accepts --trellis2-rmbg-model-path or
LUX3D_TRELLIS2_RMBG_MODEL_PATH and performs the same sibling discovery.
Quantized variants
Standalone GGUF builds are published in
oxide-lab/TRELLIS.2-4B-GGUF:
| Variant | Storage | Size | Notes |
|---|---|---|---|
f16 |
F16, all tensors | 14.50 GiB | maximum fidelity |
q8_0 |
Q8_0, F16 fallback | 8.87 GiB | recommended |
q6_k |
Q6_K, F16 fallback | 7.41 GiB | usable |
q5_k |
Q5_K, F16 fallback | 6.61 GiB | preview quality |
q4_k |
Q4_K, F16 fallback | 5.86 GiB | most aggressive |
Quantization is DiT-only: the five flow transformers use the QMatMul path;
DINOv3, the sparse VAE decoders, convolution kernels, norms, biases, and
embeddings stay in the F16 fallback. Each GGUF file embeds
resolved_config.json, so it is a complete --model-path target.
Provenance
- Upstream model:
microsoft/TRELLIS.2-4B - Upstream code:
microsoft/TRELLIS.2(MIT) - Canonicalization flow:
tools/python_baseline/normalize_weights.pyin Lux3D - Raw-file order and exact source checksums:
manifest.json
Normalization reads the nine upstream safetensors shards plus their JSON
configs, external/dinov3/config.json, and pipeline.json. It merges them
into one file without materializing the full 15 GiB payload in memory. The
resulting package is verified by reopening one tensor per stage and comparing
shape, dtype, and bytes against the source shard.
Validation
The Lux3D runtime was validated against the upstream PyTorch reference. Golden stage checks:
| Check | Result |
|---|---|
| DINOv3 conditioning relative error | 8.8e-7 |
| Sparse-VAE decoded vertices max delta | 5.96e-8 |
| Sparse-VAE decoded faces | exact |
Sparse-structure decoder coords_32 |
exact |
Shape SLAT final latent rms_rel |
1.17e-2 (upstream BF16 noise) |
Backend coverage on real weights (RTX 3060 12 GiB, 512 geometry smoke, 2 steps, synthetic RGBA input):
| Backend | Wall | Vertices / faces | Peak GPU (device) | Result |
|---|---|---|---|---|
| CPU | 567 s | 3391 / 3044 | n/a | pass |
| CUDA | 23 s | 3391 / 3044 | 6.9 GiB | pass |
| Vulkan | 209 s | 3391 / 3044 | 7.8 GiB | pass |
| WGPU | 404 s | 3391 / 3044 | 9.6 GiB | pass |
Exported meshes from all four backends are finite and agree on topology and bounding box to float noise; per-backend reruns reproduce the same counts and byte-identical Vulkan/WGPU bounding boxes.
Full reference runs:
| Run | Result |
|---|---|
| CPU, 512, 12 steps | 424,289 vertices / 759,914 faces, 2221 s |
CUDA, 1024_cascade, 12 steps, texture on |
645,960 vertices / 636,302 faces, 438 s, ~7.8 GiB peak |
The CUDA cascade run was repeated and produced identical geometry counts and byte-identical OBJ/PLY/GLB exports. Sequential stage loading keeps the full textured cascade inside a 12 GiB card with roughly 4 GiB headroom.
Quantization fidelity was measured against this package on the same CUDA backend and input (512 pipeline, 12 steps, seed 42, texture off). Drift is relative to a reference bbox diagonal of 1.494:
| Variant | Chamfer-L1 rel | F@1% | Voxel IoU | Normal consistency |
|---|---|---|---|---|
f16 |
0.0027% | 1.000 | 0.996 | 0.993 |
q8_0 |
0.187% | 0.9835 | 0.623 | 0.904 |
q6_k |
0.199% | 0.9813 | 0.609 | 0.905 |
q5_k |
0.274% | 0.9729 | 0.445 | 0.890 |
q4_k |
0.311% | 0.9625 | 0.400 | 0.885 |
The O-Voxel surface is discrete, so small latent shifts can flip individual voxel occupancies; the Q8_0/Q6_K drift is mostly topology-level while positions remain close (normals above 0.90). F16 reproduces the reference mesh almost exactly.
Intended Use
- image-to-mesh generation with Lux3D
- reproducible runtime validation against the upstream PyTorch reference
- local, offline inference from canonical safetensors weights
Limitations
- Background removal requires the separate RMBG-2.0 package. It is not included here and is licensed separately (CC-BY-NC-4.0, non-commercial).
- The single-resolution
1024pipeline and the 1536-token cascade are not implemented in Lux3D; use512or1024_cascade. - Mesh post-processing (
fill_holes, decimation) and texture-atlas UV unwrapping are out of scope for the current runtime. Textured runs export the O-Voxel dual-grid surface with per-vertex PBR colors in GLB plus a lossless<stem>.pbrvox.bin+<stem>.pbr.jsonsidecar pair. - One mesh is generated per image (
num_samples=1). - Metal inference is not tested on this Windows host. CPU, CUDA, Vulkan, and WGPU are the verified backends.
- CPU inference works but is much slower than the GPU backends.
- Canonical weights are a runtime artifact only; this package does not contain the upstream training or demo stack, and the Lux3D contract JSON ships with the runtime rather than inside the weight package.
License
TRELLIS.2 code and weights are released by Microsoft under the MIT
License; see LICENSE. The DINOv3 ViT-L/16 encoder bundled
upstream under image_cond.* is a separately licensed gated Meta model;
verify its terms before redistribution or commercial use. RMBG-2.0 is a
separate BRIA model under CC-BY-NC-4.0 and is not part of this package.
Citation
@article{xiang2025trellis2,
title={Native and Compact Structured Latents for 3D Generation},
author={Xiang, Jianfeng and Chen, Xiaoxue and Xu, Sicheng and Wang, Ruicheng and Lv, Zelong and Deng, Yu and Zhu, Hongyuan and Dong, Yue and Zhao, Hao and Yuan, Nicholas Jing and Yang, Jiaolong},
journal={Tech report},
year={2025}
}
- Downloads last month
- 34
Model tree for oxide-lab/TRELLIS.2-4B
Base model
microsoft/TRELLIS.2-4B