suvadityamuk HF Staff
Point the install instructions at github.com/suvadityamuk/diffusers-3d
843c1ac verified | license: mit | |
| base_model: VAST-AI/TripoSG | |
| pipeline_tag: image-to-3d | |
| tags: | |
| - diffusers-3d | |
| - triposg | |
| - 3d | |
| - image-to-3d | |
| # TripoSG for diffusers-3d | |
| [VAST-AI/TripoSG](https://huggingface.co/VAST-AI/TripoSG) converted into a [diffusers-3d](https://github.com/suvadityamuk/diffusers-3d) | |
| pipeline: ordinary Diffusers and Transformers component folders plus the `object3d_model_index.json` sidecar that | |
| the package's auto-loader validates before downloading anything. Nothing here requires remote code. | |
| ## Install | |
| ```bash | |
| pip install git+https://github.com/suvadityamuk/diffusers-3d.git | |
| pip install "git+https://github.com/suvadityamuk/diffusers-3d.git#subdirectory=packages/diffusers-3d" | |
| ``` | |
| `diffusers-3d` runs every network in plain PyTorch on CPU or GPU. Rendering Gaussian splats needs the optional | |
| `gsplat` backend; the PBR-textured mesh for TRELLIS.2 needs CuMesh (see the package docs). | |
| ## Usage | |
| ```python | |
| import torch | |
| from diffusers_3d import AutoPipelineForImageTo3D, ImageCondition | |
| pipeline = AutoPipelineForImageTo3D.from_pretrained("suvadityamuk/TripoSG-diffusers-3d").to("cuda") | |
| mesh = pipeline(ImageCondition(image=rgba), generator=torch.Generator("cuda").manual_seed(0)).objects[0] | |
| ``` | |
| `rgba` is a `(4, H, W)` tensor in `[0, 1]` whose alpha channel masks the object; the pipeline crops and pads it as the released code does. The result is an untextured `MeshAsset` extracted with marching cubes. | |
| ## Components | |
| | Folder | Class | | |
| |---|---| | |
| | `feature_extractor` | `transformers.BitImageProcessorPil` | | |
| | `image_encoder` | `transformers.Dinov2Model` | | |
| | `scheduler` | `diffusers.FlowMatchEulerDiscreteScheduler` | | |
| | `transformer` | `diffusers_3d.families.triposg.models.TripoSGTransformerModel` | | |
| | `vae` | `diffusers_3d.families.triposg.vae.TripoSGVAEDecoder` | | |
| ## Provenance | |
| Converted with `diffusers-3d-convert-triposg` from `diffusers-3d 0.1.0.dev0`; the package's family README lists the pinned upstream | |
| revisions and how the conversion maps each tensor. Parity tests against the pinned upstream code are part of the | |
| package test suite. | |
| ## License and attribution | |
| TripoSG weights and code: MIT License, Copyright (c) VAST-AI-Research. The DINOv2 image encoder | |
| weights are Apache-2.0, Copyright (c) Meta Platforms, Inc. TripoSG's transformer is derived from Tencent HunyuanDiT, | |
| whose Tencent Hunyuan Community License does not apply in the European Union, the United Kingdom, or South Korea; | |
| you are responsible for making sure your use complies with it. See `LICENSE-MIT` and `NOTICE`. Not affiliated with | |
| or endorsed by VAST, Tencent, or Meta. | |