File size: 3,674 Bytes
5d90070 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 | ---
license: apache-2.0
base_model: speridlabs/idol
pipeline_tag: image-to-3d
tags:
- diffusers-3d
- idol
- 3d
- image-to-3d
- gaussian-splatting
---
# IDOL for diffusers-3d
[speridlabs/idol](https://huggingface.co/speridlabs/idol) converted into a [diffusers-3d](https://github.com/suvadityamuk/diffusers-3d)
pipeline: ordinary Diffusers and Transformers component folders that the installed package loads. Nothing here
requires remote code.
## Install
```bash
pip install git+https://github.com/suvadityamuk/diffusers-3d.git
pip install "git+https://github.com/suvadityamuk/diffusers-3d.git#subdirectory=packages/diffusers-3d"
```
`diffusers-3d` runs every network in plain PyTorch on CPU or GPU. Rendering Gaussian splats needs the optional
`gsplat` backend; the PBR-textured mesh for TRELLIS.2 needs CuMesh (see the package docs).
## Usage
```python
import torch
from diffusers_3d import AutoPipelineForImageTo3D, ImageCondition
pipeline = AutoPipelineForImageTo3D.from_pretrained("suvadityamuk/idol-diffusers-3d").to("cuda")
splat = pipeline(ImageCondition(image=rgba), mode="real", generator=torch.Generator("cuda").manual_seed(42)).objects[0]
camera = splat.metadata["camera"] # predicted yaw, pitch, radius, and FOV of the input view
splat.save("object.ply")
```
**DINOv3 and the FLUX.2 VAE are not included.** Both are gated third-party weights, so
`from_pretrained` loads `image_encoder` from
[facebook/dinov3-vith16plus-pretrain-lvd1689m](https://huggingface.co/facebook/dinov3-vith16plus-pretrain-lvd1689m)
(DINOv3 License) and `vae` from the `vae/` folder of
[black-forest-labs/FLUX.2-dev](https://huggingface.co/black-forest-labs/FLUX.2-dev) (FLUX non-commercial license),
each at a pinned revision. Accept both licenses and log in (`hf auth login`) first; without access, loading raises an
error that says so. Inputs without usable alpha are cut out with BiRefNet using
[ZhengPeng7/BiRefNet](https://huggingface.co/ZhengPeng7/BiRefNet) weights (MIT), as upstream does
(`pipeline.disable_background_removal()` skips it for inputs that already carry alpha).
`mode="synthetic"` (the default, for renders) crops 1.2 times the object; `mode="real"` (photos) crops 1.7 times. Defaults match upstream: 50 steps, shift 3.0, 262144 Gaussians, `guidance_scale=4.0` (upstream's `cfg_strength=3.0` in the Diffusers convention). Without `dtype`, each component loads in upstream's precision; pass `dtype=torch.float32` on CPU.
## Components
| Folder | Class |
|---|---|
| `decoder` | `diffusers_3d.families.triposplat.decoder.TripoSplatGaussianDecoder` |
| `image_encoder` | `transformers.DINOv3ViTModel` (not included; loads from `facebook/dinov3-vith16plus-pretrain-lvd1689m`) |
| `scheduler` | `diffusers.FlowMatchEulerDiscreteScheduler` |
| `transformer` | `diffusers_3d.families.triposplat.models.TripoSplatTransformerModel` |
| `vae` | `diffusers_3d.families.triposplat.models.TripoSplatImageVAE` (not included; loads from `black-forest-labs/FLUX.2-dev`) |
## Provenance
Converted with `diffusers-3d-convert-idol` from `diffusers-3d 0.1.0.dev0`; the package's family README lists the pinned upstream
revisions and how the conversion maps each tensor. Parity tests against the pinned upstream code are part of the
package test suite.
## License and attribution
IDOL weights and code: Apache-2.0, Copyright (c) Speridlabs. IDOL keeps the architecture of
VAST-AI-Research TripoSplat (MIT, `LICENSE-MIT`). The DINOv3 and FLUX.2 VAE encoders are not redistributed; they
load from their original repositories under their own licenses. See `LICENSE-APACHE-2.0` and `NOTICE`. Not
affiliated with or endorsed by Speridlabs, VAST, Meta, or Black Forest Labs.
|