|
Download README.md from oxide-lab/TripoSG: direct link, hf CLI and curl.
- Browser
- Download file 7.18 kB
-
https://huggingface.co/oxide-lab/TripoSG/resolve/main/README.md
- Command line
-
hf download hf://oxide-lab/TripoSG/README.md
-
curl -L -o README.md https://huggingface.co/oxide-lab/TripoSG/resolve/main/README.md
7.18 kB
| license: mit | |
| pipeline_tag: image-to-3d | |
| base_model: VAST-AI/TripoSG | |
| tags: | |
| - lux3d | |
| - rust | |
| - candle | |
| - safetensors | |
| - canonical-weights | |
| - 3d | |
| - image-to-3d | |
| - mesh | |
| - triposg | |
| # Lux3D TripoSG Canonical Weights | |
| This repository contains the canonicalized `safetensors` export of the TripoSG image-to-3D checkpoint used by the Lux3D runtime. | |
| ## Model Details | |
| - Model family: `triposg` | |
| - Canonical file: `model.safetensors` (7,946,503,588 bytes, about 7.4 GiB) | |
| - Tensor count: `1349` | |
| - Parameter storage: F32 | |
| - Primary runtime target: Lux3D runtime | |
| - Typical output: triangle mesh export (`.obj`) | |
| - SHA-256: `68c303df01728e2b7a48c185736b6d7e1fafd7ae98618552b72e51a00da55838` | |
| TripoSG is a high-fidelity image-to-3D foundation model built on large-scale rectified flow. This inference slice includes: | |
| - a DINOv2-large image encoder (24 layers, width 1024, patch 14) producing the conditioning embedding; | |
| - a 21-block rectified-flow DiT (width 2048, 16 heads, cross-attention width 1024, 2048 latent tokens); | |
| - a signed-distance VAE decoder (16 self-attention blocks plus one cross-attention block) that evaluates the SDF on a hierarchical grid; | |
| - hierarchical marching cubes at level 0 over the canonical `±1.005` bounds (dense depth 8, refinement depth 9 by default). | |
| The source and target weights are MIT licensed. | |
| ## Included Files | |
| - `model.safetensors` — merged canonical weights with `image_encoder.*`, `transformer.*`, and `vae.*` prefixes | |
| - `resolved_config.json` — runtime configuration extracted from the upstream configs | |
| - `manifest.json` — provenance, source checksums, tensor count, and raw-file list | |
| - `checksums.json` — SHA-256 and size for every file in this package | |
| ## Usage With Lux3D | |
| ```powershell | |
| # Inspect the family contract | |
| cargo run -p lux3d-cli -- inspect --repo-root <runtime-root> triposg | |
| # Run with an explicit canonical package directory | |
| cargo run -p lux3d-cli --features cuda -- run triposg --model-path <canonical-triposg-dir> --source <input-image> --output <output-file.obj> | |
| ``` | |
| The package can also be downloaded automatically into the Lux3D cache: | |
| ```powershell | |
| cargo run -p lux3d-cli --features cuda -- run triposg --source <input-image> --output <output-file.obj> | |
| ``` | |
| Default inference settings match the reference pipeline: 50 rectified-flow steps, CFG scale 7.0, seed 42, 2048 latent tokens, dense octree depth 8, hierarchical depth 9. Override them with `--steps`, `--guidance-scale`, `--seed`, `--num-tokens`, `--dense-octree-depth`, and `--hierarchical-octree-depth`. | |
| A fast preview suitable for smoke tests: | |
| ```powershell | |
| cargo run -p lux3d-cli --features cuda -- run triposg --model-path <canonical-triposg-dir> --source <input-image> --steps 2 --dense-octree-depth 5 --hierarchical-octree-depth 5 --output <preview.obj> | |
| ``` | |
| Input should be a single image. RGBA images with a valid alpha channel use the reference foreground-bbox + white-composite + 10% padding path; RGB images (or invalid alpha) are treated as opaque over white. Background removal is intentionally not part of the Lux3D runtime. | |
| ## Provenance | |
| - Upstream source model: [`VAST-AI/TripoSG`](https://huggingface.co/VAST-AI/TripoSG) | |
| - Upstream code: [`VAST-AI-Research/TripoSG`](https://github.com/VAST-AI-Research/TripoSG) (MIT) | |
| - Canonicalization flow: `tools/python_baseline/normalize_weights.py` | |
| The raw checkpoint is split across `transformer/`, `vae/`, and `image_encoder_dinov2/`; normalization merges them into one safetensors file with family prefixes and records every source checksum in `manifest.json`. No tensor values are modified. | |
| ## Validation | |
| The Lux3D runtime was validated against the upstream PyTorch reference (F32, RTX 3060) on the golden sample: | |
| | Check | Result | | |
| |---|---:| | |
| | preprocess uint8 | exact | | |
| | normalized `pixel_values` max delta | 2.384e-7 | | |
| | DINOv2 `image_embeds` max delta | 8.631e-5 | | |
| | step-0 CFG velocity max delta | 8.631e-5 | | |
| | latents at step 1 / 10 / 25 max delta | 1.67e-6 / 4.05e-6 / 9.84e-3 | | |
| | VAE SDF probe (baseline latents) max delta | 1.78e-3 | | |
| | dense depth-8 SDF mean / p99.99 | 5.27e-6 / 2.94e-3 | | |
| | golden depth-9 mesh bbox / volume delta | 1.72e-4 / 8.26e-5 | | |
| The 50-step classifier-free-guided ODE is numerically chaotic: a control run of the same reference code on a different GPU/torch build drifts at least as much as Rust at the final step (reference-to-reference final latents: mean 1.42e-2, relative L2 7.74e-2, max 3.00). Final-state parity is therefore accepted statistically (mean ≤ 2e-2, relative L2 ≤ 1.2e-1, max ≤ 4.0), while early-step, VAE, grid, and mesh parity remain tight. | |
| Reference timings on an NVIDIA GeForce RTX 3060 12 GiB (Windows/WDDM, release build): | |
| | Stage | CUDA (50 steps, depth 8→9) | Vulkan (2 steps, depth 5/5) | WGPU (2 steps, depth 5/5) | | |
| |---|---:|---:|---:| | |
| | load | 6.50 s | 6.69 s | 6.78 s | | |
| | infer (preprocess + DINO + denoise) | 118.25 s | 11.97 s | 24.25 s | | |
| | SDF grid + mesh | 178.54 s | 0.64 s | 0.79 s | | |
| | export | 0.48 s | 0.00 s | 0.00 s | | |
| | peak VRAM | 9.70 GiB | 11.96 GiB | 11.78 GiB | | |
| The default CUDA run produced 967,685 vertices and 1,932,352 faces. CPU, CUDA, Vulkan, and WGPU were exercised; Metal has no test host in this repository. | |
| ## Intended Use | |
| - image-to-mesh generation with Lux3D | |
| - reproducible runtime validation against the upstream PyTorch reference | |
| - local, offline inference from canonical safetensors weights | |
| ## Limitations | |
| - The upstream TripoSG code and weights are MIT licensed; review the upstream terms before redistribution. | |
| - The 50-step sampling trajectory is chaotic across frameworks and GPU environments. Final-state comparisons should use statistical tolerance, not bit-exact equality. | |
| - Lux3D's seeded latent initialization is deterministic but not bit-identical to the torch RNG; parity tests inject the baseline latents. | |
| - Background removal (RMBG-1.4 / BriaRMBG) is not implemented. Supply an image whose background is already white or transparent. | |
| - The flash decoder / DiffDMC path and the unused VAE encoder + quant weights are excluded; the contract records those exclusions. | |
| - Inference is F32-only in the current Lux3D runtime. | |
| - Metal inference is not tested in this package; CPU, CUDA, Vulkan, and WGPU are the verified backends. | |
| - All three submodels stay resident, so the native Vulkan/WGPU backends are very close to the 12 GiB limit; 16 GiB or more is recommended outside CUDA. | |
| - This package contains canonical runtime artifacts only, not the upstream training or demo stack. | |
| - The Lux3D contract JSON is shipped with the runtime, not inside this weight package. | |
| ## License | |
| - Upstream model page: [`VAST-AI/TripoSG`](https://huggingface.co/VAST-AI/TripoSG) | |
| - Upstream code: [`VAST-AI-Research/TripoSG`](https://github.com/VAST-AI-Research/TripoSG) | |
| - Upstream license: MIT | |
| ## Citation | |
| ```bibtex | |
| @article{li2025triposg, | |
| title={TripoSG: High-Fidelity 3D Shape Synthesis using Large-Scale Rectified Flow Models}, | |
| author={Li, Yangguang and Zou, Zi-Xin and Liu, Zexiang and Wang, Dehu and Liang, Yuan and Yu, Zhipeng and Liu, Xingchao and Guo, Yuan-Chen and Liang, Ding and Ouyang, Wanli and others}, | |
| journal={arXiv preprint arXiv:2502.06608}, | |
| year={2025} | |
| } | |
| ``` | |