File size: 3,352 Bytes
3f28e7f | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 | ---
language:
- en
license: other
license_name: idol-with-third-party-components
license_link: https://huggingface.co/OzzyGT/idol_diffusers/blob/main/NOTICE
library_name: diffusers
pipeline_tag: image-to-3d
base_model:
- speridlabs/idol
- facebook/dinov3-vith16plus-pretrain-lvd1689m
- black-forest-labs/FLUX.2-klein-4B
tags:
- idol
- modular-diffusers
- image-to-3d
- gaussian-splatting
---
# Idol Diffusers
<video controls loop muted playsinline src="https://huggingface.co/datasets/OzzyGT/diffusers-examples/resolve/main/idol/idol_dragon_seed42_fp16.mp4"></video>
*Generated with the code below (seed `42`, 50 steps, 262,144 Gaussians).*
A Diffusers conversion of [speridlabs/idol](https://huggingface.co/speridlabs/idol) for single-image 3D Gaussian splat generation. This repository holds only weights and configs: the denoiser, the octree decoder, the Gaussian decoder and its two image encoders, DINOv3 and the FLUX.2 VAE.
For model details, see the [original model card](https://huggingface.co/speridlabs/idol) and the [Idol SDK repository](https://github.com/speridlabs/idol-sdk).
## Usage
Install PyTorch for your GPU, then:
```bash
pip install diffusers==0.42.1 transformers==5.14.1 "accelerate>=1.14" "safetensors>=0.8" numpy pillow
```
The model runs with the [Idol Modular Diffusers](https://huggingface.co/OzzyGT/idol_custom_blocks) blocks, which ship the model code and load the weights from this repository.
The example uses `trust_remote_code=True`.
> **Note:** `trust_remote_code=True` runs Python code downloaded from the Hub, here the code of the blocks repository. Only use it with repositories you trust, and review their code before passing a different one.
```python
import torch
from diffusers import ModularPipeline
from diffusers.utils import load_image
pipe = ModularPipeline.from_pretrained("OzzyGT/idol_custom_blocks", trust_remote_code=True)
pipe.load_components(dtype=torch.bfloat16)
pipe.to("cuda")
image = load_image("https://huggingface.co/datasets/OzzyGT/diffusers-examples/resolve/main/idol/nb-dragon-anime_input.png")
result = pipe(
image=image,
generator=torch.Generator("cuda").manual_seed(42),
output=["gaussians", "camera_latents"],
)
gaussians = result["gaussians"]
camera_latents = result["camera_latents"]
```
To save the result as a standard 3DGS `.ply` file that any Gaussian splat viewer opens, use the [Idol recipe](https://github.com/asomoza/diffusers-recipes/blob/main/models/idol/scripts/base_example.py) from diffusers-recipes.
The input is a prepared image (1024x1024, the object cropped with a margin and composited on black), here prepared with Idol's own background removal and crop. The pipeline returns activated Gaussians (positions, colors, scales, rotations, opacities) and the predicted input camera, not a file; the video shows them rendered on a turntable.
Built with DINOv3.
## Licenses
- Idol weights: [Apache-2.0](LICENSE), from [speridlabs/idol](https://huggingface.co/speridlabs/idol). See [NOTICE](NOTICE).
- Image encoder: [DINOv3 License](LICENSE_DINOv3), from [facebook/dinov3-vith16plus-pretrain-lvd1689m](https://huggingface.co/facebook/dinov3-vith16plus-pretrain-lvd1689m), stored in bfloat16.
- FLUX.2 VAE: [Apache-2.0](LICENSE), from [black-forest-labs/FLUX.2-klein-4B](https://huggingface.co/black-forest-labs/FLUX.2-klein-4B), unchanged.
|