Lux3D TRELLIS.2-4B Canonical Weights

This repository contains the canonicalized safetensors export of the TRELLIS.2-4B image-to-3D checkpoint used by the Lux3D runtime.

Model Details

  • Model family: trellis2
  • Canonical file: model.safetensors (16,180,079,934 bytes, about 15.07 GiB)
  • Tensor count: 4265
  • Parameter storage: mixed BF16 / F16 / F32, preserved from the upstream checkpoints (BF16: 3200 tensors, F16: 612, F32: 453)
  • SHA-256: aaff394292cab30af64624e816eb3ac15901620b10b9e90f89754f47d05afee7
  • Raw provenance files: 19 (nine safetensors shards, their configs, pipeline.json)
  • Primary runtime target: Lux3D runtime
  • Typical output: textured GLB mesh plus a PBR vertex-attribute sidecar (.obj and .ply are also supported)
  • Reference default pipeline: 1024_cascade (512 sparse-structure + shape SLAT pass, then the 1024 shape and texture passes)

TRELLIS.2-4B is Microsoft's 4-billion-parameter image-to-3D model built around the O-Voxel sparse-voxel representation. This inference slice includes:

  • the DINOv3 ViT-L/16 image conditioner (image_cond);
  • a 64³ sparse-structure flow DiT and its dense 3D-convolution decoder (ss_flow, ss_dec);
  • 512 and 1024 shape SLAT flow DiTs (shape_dit_512, shape_dit_1024);
  • the sparse shape VAE decoder that produces the O-Voxel geometry (shape_dec);
  • 512 and 1024 texture SLAT flow DiTs (tex_dit_512, tex_dit_1024);
  • the sparse texture VAE decoder with per-voxel PBR attributes (tex_dec).

The merged package uses one <stage>. prefix per component:

Stage prefix Upstream source Tensors
image_cond external/dinov3/model.safetensors 415
ss_flow ckpts/ss_flow_img_dit_1_3B_64_bf16.safetensors 640
ss_dec external/ss_dec/ckpts/ss_dec_conv3d_16l8_fp16.safetensors 74
shape_dit_512 ckpts/slat_flow_img2shape_dit_1_3B_512_bf16.safetensors 640
shape_dit_1024 ckpts/slat_flow_img2shape_dit_1_3B_1024_bf16.safetensors 640
shape_dec ckpts/shape_dec_next_dc_f16c32_fp16.safetensors 292
tex_dit_512 ckpts/slat_flow_imgshape2tex_dit_1_3B_512_bf16.safetensors 640
tex_dit_1024 ckpts/slat_flow_imgshape2tex_dit_1_3B_1024_bf16.safetensors 640
tex_dec ckpts/tex_dec_next_dc_f16c32_fp16.safetensors 284
Total 4265

Included Files

  • model.safetensors — merged canonical weights with the stage prefixes above
  • resolved_config.json — pipeline.json arguments, every stage config, and the shape/texture SLAT normalization vectors
  • manifest.json — family, provenance, source checksums, raw-file order, tensor count, and dtype histogram
  • checksums.json — SHA-256 and byte size for the canonical data files
  • README.md — this document
  • LICENSE — MIT license text and third-party notices

No tensor values are modified by canonicalization: the nine shards are merged tensor-by-tensor, keys gain their stage prefix, and original dtypes are preserved byte-exact.

Usage With Lux3D

Validate the family contract and run from an explicit canonical package:

cargo run -p lux3d-cli -- inspect --repo-root <runtime-root> trellis2

cargo run -p lux3d-cli --features cuda -- run trellis2 `
  --model-path <canonical-trellis2-dir> `
  --rmbg-model-path <canonical-rmbg2-dir> `
  --source <input-image> `
  --output <output-file.glb>

A faster geometry-only preview:

cargo run -p lux3d-cli --features cuda -- run trellis2 `
  --model-path <canonical-trellis2-dir> `
  --rmbg-model-path <canonical-rmbg2-dir> `
  --source <input-image> `
  --pipeline-type 512 --steps 12 --texture false --seed 42 `
  --output <preview.obj>

--pipeline-type accepts 512 and 1024_cascade. The single-resolution 1024 path and the 1536-token cascade are not implemented by Lux3D yet.

The raw microsoft/TRELLIS.2-4B checkpoint directory (pipeline.json plus ckpts/ and external/) is accepted as-is by the CLI, so the raw tree does not have to be normalized before a local run.

Auto-download is reserved for the oxide-lab/TRELLIS.2-4B repository. If that repository is not available to your account, pass an explicit --model-path.

Background removal

TRELLIS.2's reference preprocessing runs RMBG-2.0/BiRefNet when the input has no usable alpha channel. Those weights are not part of this package. Normalize them as the separate rmbg2 family and pass --rmbg-model-path <canonical-rmbg2-dir>, or place an rmbg-2.0 directory next to --model-path. A missing RMBG package is an error for RGB inputs; Lux3D never silently treats a background as opaque. RGBA inputs with a usable alpha channel use the alpha path and do not load RMBG.

The HTTP server accepts --trellis2-rmbg-model-path or LUX3D_TRELLIS2_RMBG_MODEL_PATH and performs the same sibling discovery.

Quantized variants

Standalone GGUF builds are published in oxide-lab/TRELLIS.2-4B-GGUF:

Variant Storage Size Notes
f16 F16, all tensors 14.50 GiB maximum fidelity
q8_0 Q8_0, F16 fallback 8.87 GiB recommended
q6_k Q6_K, F16 fallback 7.41 GiB usable
q5_k Q5_K, F16 fallback 6.61 GiB preview quality
q4_k Q4_K, F16 fallback 5.86 GiB most aggressive

Quantization is DiT-only: the five flow transformers use the QMatMul path; DINOv3, the sparse VAE decoders, convolution kernels, norms, biases, and embeddings stay in the F16 fallback. Each GGUF file embeds resolved_config.json, so it is a complete --model-path target.

Provenance

  • Upstream model: microsoft/TRELLIS.2-4B
  • Upstream code: microsoft/TRELLIS.2 (MIT)
  • Canonicalization flow: tools/python_baseline/normalize_weights.py in Lux3D
  • Raw-file order and exact source checksums: manifest.json

Normalization reads the nine upstream safetensors shards plus their JSON configs, external/dinov3/config.json, and pipeline.json. It merges them into one file without materializing the full 15 GiB payload in memory. The resulting package is verified by reopening one tensor per stage and comparing shape, dtype, and bytes against the source shard.

Validation

The Lux3D runtime was validated against the upstream PyTorch reference. Golden stage checks:

Check Result
DINOv3 conditioning relative error 8.8e-7
Sparse-VAE decoded vertices max delta 5.96e-8
Sparse-VAE decoded faces exact
Sparse-structure decoder coords_32 exact
Shape SLAT final latent rms_rel 1.17e-2 (upstream BF16 noise)

Backend coverage on real weights (RTX 3060 12 GiB, 512 geometry smoke, 2 steps, synthetic RGBA input):

Backend Wall Vertices / faces Peak GPU (device) Result
CPU 567 s 3391 / 3044 n/a pass
CUDA 23 s 3391 / 3044 6.9 GiB pass
Vulkan 209 s 3391 / 3044 7.8 GiB pass
WGPU 404 s 3391 / 3044 9.6 GiB pass

Exported meshes from all four backends are finite and agree on topology and bounding box to float noise; per-backend reruns reproduce the same counts and byte-identical Vulkan/WGPU bounding boxes.

Full reference runs:

Run Result
CPU, 512, 12 steps 424,289 vertices / 759,914 faces, 2221 s
CUDA, 1024_cascade, 12 steps, texture on 645,960 vertices / 636,302 faces, 438 s, ~7.8 GiB peak

The CUDA cascade run was repeated and produced identical geometry counts and byte-identical OBJ/PLY/GLB exports. Sequential stage loading keeps the full textured cascade inside a 12 GiB card with roughly 4 GiB headroom.

Quantization fidelity was measured against this package on the same CUDA backend and input (512 pipeline, 12 steps, seed 42, texture off). Drift is relative to a reference bbox diagonal of 1.494:

Variant Chamfer-L1 rel F@1% Voxel IoU Normal consistency
f16 0.0027% 1.000 0.996 0.993
q8_0 0.187% 0.9835 0.623 0.904
q6_k 0.199% 0.9813 0.609 0.905
q5_k 0.274% 0.9729 0.445 0.890
q4_k 0.311% 0.9625 0.400 0.885

The O-Voxel surface is discrete, so small latent shifts can flip individual voxel occupancies; the Q8_0/Q6_K drift is mostly topology-level while positions remain close (normals above 0.90). F16 reproduces the reference mesh almost exactly.

Intended Use

  • image-to-mesh generation with Lux3D
  • reproducible runtime validation against the upstream PyTorch reference
  • local, offline inference from canonical safetensors weights

Limitations

  • Background removal requires the separate RMBG-2.0 package. It is not included here and is licensed separately (CC-BY-NC-4.0, non-commercial).
  • The single-resolution 1024 pipeline and the 1536-token cascade are not implemented in Lux3D; use 512 or 1024_cascade.
  • Mesh post-processing (fill_holes, decimation) and texture-atlas UV unwrapping are out of scope for the current runtime. Textured runs export the O-Voxel dual-grid surface with per-vertex PBR colors in GLB plus a lossless <stem>.pbrvox.bin + <stem>.pbr.json sidecar pair.
  • One mesh is generated per image (num_samples=1).
  • Metal inference is not tested on this Windows host. CPU, CUDA, Vulkan, and WGPU are the verified backends.
  • CPU inference works but is much slower than the GPU backends.
  • Canonical weights are a runtime artifact only; this package does not contain the upstream training or demo stack, and the Lux3D contract JSON ships with the runtime rather than inside the weight package.

License

TRELLIS.2 code and weights are released by Microsoft under the MIT License; see LICENSE. The DINOv3 ViT-L/16 encoder bundled upstream under image_cond.* is a separately licensed gated Meta model; verify its terms before redistribution or commercial use. RMBG-2.0 is a separate BRIA model under CC-BY-NC-4.0 and is not part of this package.

Citation

@article{xiang2025trellis2,
  title={Native and Compact Structured Latents for 3D Generation},
  author={Xiang, Jianfeng and Chen, Xiaoxue and Xu, Sicheng and Wang, Ruicheng and Lv, Zelong and Deng, Yu and Zhu, Hongyuan and Dong, Yue and Zhao, Hao and Yuan, Nicholas Jing and Yang, Jiaolong},
  journal={Tech report},
  year={2025}
}
Downloads last month
34
Safetensors
Model size
8B params
Tensor type
F32
·
BF16
·
F16
·
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for oxide-lab/TRELLIS.2-4B

Finetuned
(8)
this model

Collection including oxide-lab/TRELLIS.2-4B