--- license: mit pipeline_tag: image-to-3d base_model: VAST-AI/TripoSG tags: - lux3d - rust - candle - safetensors - canonical-weights - 3d - image-to-3d - mesh - triposg --- # Lux3D TripoSG Canonical Weights This repository contains the canonicalized `safetensors` export of the TripoSG image-to-3D checkpoint used by the Lux3D runtime. ## Model Details - Model family: `triposg` - Canonical file: `model.safetensors` (7,946,503,588 bytes, about 7.4 GiB) - Tensor count: `1349` - Parameter storage: F32 - Primary runtime target: Lux3D runtime - Typical output: triangle mesh export (`.obj`) - SHA-256: `68c303df01728e2b7a48c185736b6d7e1fafd7ae98618552b72e51a00da55838` TripoSG is a high-fidelity image-to-3D foundation model built on large-scale rectified flow. This inference slice includes: - a DINOv2-large image encoder (24 layers, width 1024, patch 14) producing the conditioning embedding; - a 21-block rectified-flow DiT (width 2048, 16 heads, cross-attention width 1024, 2048 latent tokens); - a signed-distance VAE decoder (16 self-attention blocks plus one cross-attention block) that evaluates the SDF on a hierarchical grid; - hierarchical marching cubes at level 0 over the canonical `±1.005` bounds (dense depth 8, refinement depth 9 by default). The source and target weights are MIT licensed. ## Included Files - `model.safetensors` — merged canonical weights with `image_encoder.*`, `transformer.*`, and `vae.*` prefixes - `resolved_config.json` — runtime configuration extracted from the upstream configs - `manifest.json` — provenance, source checksums, tensor count, and raw-file list - `checksums.json` — SHA-256 and size for every file in this package ## Usage With Lux3D ```powershell # Inspect the family contract cargo run -p lux3d-cli -- inspect --repo-root triposg # Run with an explicit canonical package directory cargo run -p lux3d-cli --features cuda -- run triposg --model-path --source --output ``` The package can also be downloaded automatically into the Lux3D cache: ```powershell cargo run -p lux3d-cli --features cuda -- run triposg --source --output ``` Default inference settings match the reference pipeline: 50 rectified-flow steps, CFG scale 7.0, seed 42, 2048 latent tokens, dense octree depth 8, hierarchical depth 9. Override them with `--steps`, `--guidance-scale`, `--seed`, `--num-tokens`, `--dense-octree-depth`, and `--hierarchical-octree-depth`. A fast preview suitable for smoke tests: ```powershell cargo run -p lux3d-cli --features cuda -- run triposg --model-path --source --steps 2 --dense-octree-depth 5 --hierarchical-octree-depth 5 --output ``` Input should be a single image. RGBA images with a valid alpha channel use the reference foreground-bbox + white-composite + 10% padding path; RGB images (or invalid alpha) are treated as opaque over white. Background removal is intentionally not part of the Lux3D runtime. ## Provenance - Upstream source model: [`VAST-AI/TripoSG`](https://huggingface.co/VAST-AI/TripoSG) - Upstream code: [`VAST-AI-Research/TripoSG`](https://github.com/VAST-AI-Research/TripoSG) (MIT) - Canonicalization flow: `tools/python_baseline/normalize_weights.py` The raw checkpoint is split across `transformer/`, `vae/`, and `image_encoder_dinov2/`; normalization merges them into one safetensors file with family prefixes and records every source checksum in `manifest.json`. No tensor values are modified. ## Validation The Lux3D runtime was validated against the upstream PyTorch reference (F32, RTX 3060) on the golden sample: | Check | Result | |---|---:| | preprocess uint8 | exact | | normalized `pixel_values` max delta | 2.384e-7 | | DINOv2 `image_embeds` max delta | 8.631e-5 | | step-0 CFG velocity max delta | 8.631e-5 | | latents at step 1 / 10 / 25 max delta | 1.67e-6 / 4.05e-6 / 9.84e-3 | | VAE SDF probe (baseline latents) max delta | 1.78e-3 | | dense depth-8 SDF mean / p99.99 | 5.27e-6 / 2.94e-3 | | golden depth-9 mesh bbox / volume delta | 1.72e-4 / 8.26e-5 | The 50-step classifier-free-guided ODE is numerically chaotic: a control run of the same reference code on a different GPU/torch build drifts at least as much as Rust at the final step (reference-to-reference final latents: mean 1.42e-2, relative L2 7.74e-2, max 3.00). Final-state parity is therefore accepted statistically (mean ≤ 2e-2, relative L2 ≤ 1.2e-1, max ≤ 4.0), while early-step, VAE, grid, and mesh parity remain tight. Reference timings on an NVIDIA GeForce RTX 3060 12 GiB (Windows/WDDM, release build): | Stage | CUDA (50 steps, depth 8→9) | Vulkan (2 steps, depth 5/5) | WGPU (2 steps, depth 5/5) | |---|---:|---:|---:| | load | 6.50 s | 6.69 s | 6.78 s | | infer (preprocess + DINO + denoise) | 118.25 s | 11.97 s | 24.25 s | | SDF grid + mesh | 178.54 s | 0.64 s | 0.79 s | | export | 0.48 s | 0.00 s | 0.00 s | | peak VRAM | 9.70 GiB | 11.96 GiB | 11.78 GiB | The default CUDA run produced 967,685 vertices and 1,932,352 faces. CPU, CUDA, Vulkan, and WGPU were exercised; Metal has no test host in this repository. ## Intended Use - image-to-mesh generation with Lux3D - reproducible runtime validation against the upstream PyTorch reference - local, offline inference from canonical safetensors weights ## Limitations - The upstream TripoSG code and weights are MIT licensed; review the upstream terms before redistribution. - The 50-step sampling trajectory is chaotic across frameworks and GPU environments. Final-state comparisons should use statistical tolerance, not bit-exact equality. - Lux3D's seeded latent initialization is deterministic but not bit-identical to the torch RNG; parity tests inject the baseline latents. - Background removal (RMBG-1.4 / BriaRMBG) is not implemented. Supply an image whose background is already white or transparent. - The flash decoder / DiffDMC path and the unused VAE encoder + quant weights are excluded; the contract records those exclusions. - Inference is F32-only in the current Lux3D runtime. - Metal inference is not tested in this package; CPU, CUDA, Vulkan, and WGPU are the verified backends. - All three submodels stay resident, so the native Vulkan/WGPU backends are very close to the 12 GiB limit; 16 GiB or more is recommended outside CUDA. - This package contains canonical runtime artifacts only, not the upstream training or demo stack. - The Lux3D contract JSON is shipped with the runtime, not inside this weight package. ## License - Upstream model page: [`VAST-AI/TripoSG`](https://huggingface.co/VAST-AI/TripoSG) - Upstream code: [`VAST-AI-Research/TripoSG`](https://github.com/VAST-AI-Research/TripoSG) - Upstream license: MIT ## Citation ```bibtex @article{li2025triposg, title={TripoSG: High-Fidelity 3D Shape Synthesis using Large-Scale Rectified Flow Models}, author={Li, Yangguang and Zou, Zi-Xin and Liu, Zexiang and Wang, Dehu and Liang, Yuan and Yu, Zhipeng and Liu, Xingchao and Guo, Yuan-Chen and Liang, Ding and Ouyang, Wanli and others}, journal={arXiv preprint arXiv:2502.06608}, year={2025} } ```