suvadityamuk HF Staff
Point the install instructions at github.com/suvadityamuk/diffusers-3d
54b95e9 verified | license: mit | |
| base_model: TencentARC/Pixal3D | |
| pipeline_tag: image-to-3d | |
| tags: | |
| - diffusers-3d | |
| - pixal3d | |
| - 3d | |
| - image-to-3d | |
| # Pixal3D for diffusers-3d | |
| [TencentARC/Pixal3D](https://huggingface.co/TencentARC/Pixal3D) converted into a [diffusers-3d](https://github.com/suvadityamuk/diffusers-3d) | |
| pipeline: ordinary Diffusers and Transformers component folders plus the `object3d_model_index.json` sidecar that | |
| the package's auto-loader validates before downloading anything. Nothing here requires remote code. | |
| ## Install | |
| ```bash | |
| pip install git+https://github.com/suvadityamuk/diffusers-3d.git | |
| pip install "git+https://github.com/suvadityamuk/diffusers-3d.git#subdirectory=packages/diffusers-3d" | |
| ``` | |
| `diffusers-3d` runs every network in plain PyTorch on CPU or GPU. Rendering Gaussian splats needs the optional | |
| `gsplat` backend; the PBR-textured mesh for TRELLIS.2 needs CuMesh (see the package docs). | |
| ## Usage | |
| ```python | |
| import torch | |
| from transformers import DINOv3ViTModel | |
| from diffusers_3d import AutoPipelineForImageTo3D, ImageCondition | |
| image_encoder = DINOv3ViTModel.from_pretrained("facebook/dinov3-vitl16-pretrain-lvd1689m") | |
| pipeline = AutoPipelineForImageTo3D.from_pretrained( | |
| "suvadityamuk/Pixal3D-diffusers-3d", image_encoder=image_encoder, dtype=torch.bfloat16 | |
| ).to("cuda") | |
| output = pipeline(ImageCondition(image=rgba), camera_angle_x=0.35, formats=("o_voxel", "mesh")) | |
| ``` | |
| **The DINOv3 image encoder is not included.** Its weights are gated under Meta's DINOv3 License; accept it on [facebook/dinov3-vitl16-pretrain-lvd1689m](https://huggingface.co/facebook/dinov3-vitl16-pretrain-lvd1689m) and pass the encoder in when loading. `rgba` is a `(4, H, W)` tensor in `[0, 1]` whose alpha channel masks the object; the pipeline crops and pads it as the released code does. `camera_angle_x` is the input camera's horizontal field of view in radians; the shape is reconstructed in that camera's frame. | |
| ## Components | |
| | Folder | Class | | |
| |---|---| | |
| | `feature_upsampler` | `diffusers_3d.families.pixal3d.conditioner.Pixal3DNAFUpsampler` | | |
| | `image_encoder` | `transformers.DINOv3ViTModel` (not included, pass it to `from_pretrained`) | | |
| | `pbr_decoder` | `diffusers_3d.families.trellis2.decoders.Trellis2PBRSparseDecoder` | | |
| | `shape_slat_decoder` | `diffusers_3d.families.trellis2.decoders.Trellis2ShapeDualGridDecoder` | | |
| | `shape_slat_flow_model` | `diffusers_3d.families.pixal3d.models.Pixal3DSLatFlowModel` | | |
| | `shape_slat_flow_model_1024` | `diffusers_3d.families.pixal3d.models.Pixal3DSLatFlowModel` | | |
| | `shape_slat_scheduler` | `diffusers_3d.families.trellis2.scheduler.Trellis2FlowEulerScheduler` | | |
| | `sparse_structure_decoder` | `diffusers_3d.families.trellis2.decoders.Trellis2SparseStructureDecoder` | | |
| | `sparse_structure_flow_model` | `diffusers_3d.families.pixal3d.models.Pixal3DSparseStructureFlowModel` | | |
| | `sparse_structure_scheduler` | `diffusers_3d.families.trellis2.scheduler.Trellis2FlowEulerScheduler` | | |
| | `texture_slat_flow_model_1024` | `diffusers_3d.families.pixal3d.models.Pixal3DSLatFlowModel` | | |
| | `texture_slat_scheduler` | `diffusers_3d.families.trellis2.scheduler.Trellis2FlowEulerScheduler` | | |
| ## Provenance | |
| Converted with `diffusers-3d-convert-pixal3d` from `diffusers-3d 0.1.0.dev0`; the package's family README lists the pinned upstream | |
| revisions and how the conversion maps each tensor. Parity tests against the pinned upstream code are part of the | |
| package test suite. | |
| ## License and attribution | |
| Pixal3D weights and code: MIT License, Copyright (c) Tencent ARC Lab. The `feature_upsampler` is | |
| valeo.ai's NAF (Neighborhood Attention Filtering) upsampler, converted from the release checkpoint at | |
| https://github.com/valeoai/NAF/releases (NAF code is Apache-2.0; the checkpoint is published there without a | |
| separate license). No DINOv3 weights are redistributed. See `LICENSE-MIT` and `NOTICE`. Not affiliated with or | |
| endorsed by Tencent, valeo.ai, or Meta. | |