Add Model Card
Browse files- .gitattributes +2 -0
- README.md +132 -0
- logo.png +3 -0
- teaser.png +3 -0
.gitattributes
CHANGED
|
@@ -33,3 +33,5 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
|
| 33 |
*.zip filter=lfs diff=lfs merge=lfs -text
|
| 34 |
*.zst filter=lfs diff=lfs merge=lfs -text
|
| 35 |
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
|
|
|
|
|
|
|
|
| 33 |
*.zip filter=lfs diff=lfs merge=lfs -text
|
| 34 |
*.zst filter=lfs diff=lfs merge=lfs -text
|
| 35 |
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
| 36 |
+
logo.png filter=lfs diff=lfs merge=lfs -text
|
| 37 |
+
teaser.png filter=lfs diff=lfs merge=lfs -text
|
README.md
ADDED
|
@@ -0,0 +1,132 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: apache-2.0
|
| 3 |
+
library_name: alchemy3d
|
| 4 |
+
tags:
|
| 5 |
+
- 3d
|
| 6 |
+
- 3d-editing
|
| 7 |
+
- asset-editing
|
| 8 |
+
- image-to-3d
|
| 9 |
+
- diffusion
|
| 10 |
+
pipeline_tag: image-to-3d
|
| 11 |
+
---
|
| 12 |
+
|
| 13 |
+
<h1 align="center">
|
| 14 |
+
<img src="logo.png" alt="Alchemy3D" width="88" height="88" valign="middle" />
|
| 15 |
+
Alchemy3D
|
| 16 |
+
</h1>
|
| 17 |
+
<p align="center">
|
| 18 |
+
<b>Scaling Versatile 3D Assets Editing with a Million-Scale Dataset</b>
|
| 19 |
+
</p>
|
| 20 |
+
|
| 21 |
+
<p align="center">
|
| 22 |
+
Badi Li<sup>1,2,4</sup>,
|
| 23 |
+
<a href="https://tianxinhuang.github.io/">Tianxin Huang</a><sup>1</sup>,
|
| 24 |
+
Yu Zhou<sup>3</sup>,
|
| 25 |
+
<a href="https://isee-ai.cn/~zhwshi/">Wei-Shi Zheng</a><sup>2,4</sup>,
|
| 26 |
+
<a href="https://www.cs.hku.hk/index.php/people/academic-staff/mayi">Yi Ma</a><sup>1,2</sup>,
|
| 27 |
+
<a href="https://scholar.google.com/citations?user=fe-1v0MAAAAJ&hl=zh-CN">Shenghua Gao</a><sup>1,2†</sup>
|
| 28 |
+
</p>
|
| 29 |
+
<p align="center">
|
| 30 |
+
<sup>1</sup> The University of Hong Kong
|
| 31 |
+
<sup>2</sup> Shenzhen Loop Area Institute<br>
|
| 32 |
+
<sup>3</sup> Shanghai Innovation Institute
|
| 33 |
+
<sup>4</sup> Sun Yat-Sen University
|
| 34 |
+
</p>
|
| 35 |
+
<p align="center">
|
| 36 |
+
<sup>†</sup> Corresponding author
|
| 37 |
+
</p>
|
| 38 |
+
|
| 39 |
+
<p align="center">
|
| 40 |
+
<a href="https://libd1.github.io/Alchemy3D-Project/"><img src="https://img.shields.io/badge/Project-Page-0A7F5F.svg" alt="Project Page"></a>
|
| 41 |
+
<a href="https://arxiv.org/abs/2609.34271"><img src="https://img.shields.io/badge/arXiv-Paper-b31b1b.svg" alt="arXiv"></a>
|
| 42 |
+
<a href="https://arxiv.org/pdf/2609.34271"><img src="https://img.shields.io/badge/Paper-PDF-red.svg" alt="PDF"></a>
|
| 43 |
+
<a href="https://github.com/libd1/Alchemy3D"><img src="https://img.shields.io/badge/Code-GitHub-black.svg" alt="GitHub"></a>
|
| 44 |
+
<a href="https://huggingface.co/libadi/Alchemy3D"><img src="https://img.shields.io/badge/🤗-Model-yellow.svg" alt="Hugging Face Model"></a>
|
| 45 |
+
<a href="https://modelscope.cn/models/libd55/Alchemy3D"><img src="https://img.shields.io/badge/-Model-624AFF.svg?logo=modelscope&logoColor=white&labelColor=555" alt="ModelScope Model"></a>
|
| 46 |
+
<a href="https://huggingface.co/datasets/libadi/Alchemy3D-1M"><img src="https://img.shields.io/badge/🤗-Alchemy3D--1M-yellow.svg" alt="Dataset"></a>
|
| 47 |
+
<a href="https://huggingface.co/datasets/libadi/GEdit3D-Bench"><img src="https://img.shields.io/badge/🤗-GEdit3D--Bench-yellow.svg" alt="Benchmark"></a>
|
| 48 |
+
<a href="https://github.com/libd1/edit3dstudio"><img src="https://img.shields.io/badge/-evaluation-black.svg?logo=github&logoColor=white&labelColor=555" alt="Evaluation"></a>
|
| 49 |
+
</p>
|
| 50 |
+
|
| 51 |
+
<p align="center">
|
| 52 |
+
<img src="teaser.png" alt="Alchemy3D teaser: addition, removal, replacement, local/global appearance, and animation." width="100%" />
|
| 53 |
+
</p>
|
| 54 |
+
|
| 55 |
+
**Alchemy3D** is the primary checkpoint of an open-sourced foundation model for **editing existing 3D assets** while preserving identity and structure. Given a source mesh (e.g. `.glb`) and a target reference image, it performs versatile edits including Addition, Removal, Replacement, Local/Global Appearance, and Animation.
|
| 56 |
+
|
| 57 |
+
The model is trained on **[Alchemy3D-1M](https://huggingface.co/datasets/libadi/Alchemy3D-1M)** (1.25M unique assets, 1.38M edit pairs, 7 edit types) and builds on [TRELLIS.2](https://github.com/microsoft/TRELLIS.2) structured latents.
|
| 58 |
+
|
| 59 |
+
## Model Variants
|
| 60 |
+
|
| 61 |
+
Weights are mirrored on Hugging Face and ModelScope. If Hugging Face is unreachable, load the ModelScope IDs below — the official code falls back automatically.
|
| 62 |
+
|
| 63 |
+
| Model | Description | Hugging Face | ModelScope |
|
| 64 |
+
| :--- | :--- | :--- | :--- |
|
| 65 |
+
| **Alchemy3D** (this repo) | Primary image-conditioned editing model | [libadi/Alchemy3D](https://huggingface.co/libadi/Alchemy3D) | [libd55/Alchemy3D](https://modelscope.cn/models/libd55/Alchemy3D) |
|
| 66 |
+
| **Alchemy3D-Turbo** | Step-distilled variant for faster inference with competitive quality | [libadi/Alchemy3D-Turbo](https://huggingface.co/libadi/Alchemy3D-Turbo) | [libd55/Alchemy3D-Turbo](https://modelscope.cn/models/libd55/Alchemy3D-Turbo) |
|
| 67 |
+
| **Alchemy3D-Flux** | Replaces DINOv3 with a Flux2 encoder for stronger PBR / appearance editing | [libadi/Alchemy3D-Flux](https://huggingface.co/libadi/Alchemy3D-Flux) | [libd55/Alchemy3D-Flux](https://modelscope.cn/models/libd55/Alchemy3D-Flux) |
|
| 68 |
+
| **Alchemy3D-Instruct** | Instruction-driven editing from natural-language text instead of a target image | [libadi/Alchemy3D-Instruct](https://huggingface.co/libadi/Alchemy3D-Instruct) | [libd55/Alchemy3D-Instruct](https://modelscope.cn/models/libd55/Alchemy3D-Instruct) |
|
| 69 |
+
| **Alchemy3D-Segment** | Downstream 3D part segmentation from 1–8 multi-view 2D segmentation maps | [libadi/Alchemy3D-Segment](https://huggingface.co/libadi/Alchemy3D-Segment) | [libd55/Alchemy3D-Segment](https://modelscope.cn/models/libd55/Alchemy3D-Segment) |
|
| 70 |
+
|
| 71 |
+
- **Alchemy3D / Turbo / Flux** share the same image-conditioned editing API (`Pipeline.from_pretrained(...)`); swap the model ID to switch variants.
|
| 72 |
+
- **Alchemy3D-Instruct** uses `Alchemy3DInstructPipeline` with a text `instruction` string.
|
| 73 |
+
- **Alchemy3D-Segment** uses `Alchemy3DSegmentPipeline` and returns part meshes / labels.
|
| 74 |
+
|
| 75 |
+
## Quick Start
|
| 76 |
+
|
| 77 |
+
Install and run from the [official repository](https://github.com/libd1/Alchemy3D). Drop-in alternatives for the same image-conditioned API: `libadi/Alchemy3D-Turbo`, `libadi/Alchemy3D-Flux` (or the ModelScope IDs above).
|
| 78 |
+
|
| 79 |
+
```python
|
| 80 |
+
from alchemy3d.pipelines import Pipeline
|
| 81 |
+
import o_voxel
|
| 82 |
+
|
| 83 |
+
pipe = Pipeline.from_pretrained("libadi/Alchemy3D")
|
| 84 |
+
pipe.cuda()
|
| 85 |
+
|
| 86 |
+
output = pipe.run(
|
| 87 |
+
source="./assets/examples/edits/01/source.glb",
|
| 88 |
+
image="./assets/examples/edits/01/target_image.png",
|
| 89 |
+
comparison_video="example.mp4",
|
| 90 |
+
)[0]
|
| 91 |
+
|
| 92 |
+
glb = o_voxel.postprocess.to_glb(
|
| 93 |
+
vertices=output.vertices,
|
| 94 |
+
faces=output.faces,
|
| 95 |
+
attr_volume=output.attrs,
|
| 96 |
+
coords=output.coords,
|
| 97 |
+
attr_layout=output.layout,
|
| 98 |
+
voxel_size=output.voxel_size,
|
| 99 |
+
aabb=[[-0.5, -0.5, -0.5], [0.5, 0.5, 0.5]],
|
| 100 |
+
decimation_target=1_000_000,
|
| 101 |
+
texture_size=4096,
|
| 102 |
+
remesh=True,
|
| 103 |
+
remesh_band=1,
|
| 104 |
+
remesh_project=0,
|
| 105 |
+
verbose=False,
|
| 106 |
+
)
|
| 107 |
+
glb.export("example.glb")
|
| 108 |
+
```
|
| 109 |
+
|
| 110 |
+
Instruction editing and part segmentation use dedicated pipelines — see [`example-instruct.py`](https://github.com/libd1/Alchemy3D/blob/main/example-instruct.py) and [`example_segment.py`](https://github.com/libd1/Alchemy3D/blob/main/example_segment.py).
|
| 111 |
+
|
| 112 |
+
> **Hardware:** NVIDIA GPU recommended; ~24GB VRAM is a practical minimum. See the [GitHub README](https://github.com/libd1/Alchemy3D) for full environment setup (`setup.sh`).
|
| 113 |
+
|
| 114 |
+
## Citation
|
| 115 |
+
|
| 116 |
+
If you use Alchemy3D, please cite:
|
| 117 |
+
|
| 118 |
+
```bibtex
|
| 119 |
+
@misc{li2026scalingversatile3dassets,
|
| 120 |
+
title={Scaling Versatile 3D Assets Editing with a Million-Scale Dataset},
|
| 121 |
+
author={Badi Li and Tianxin Huang and Yu Zhou and Wei-Shi Zheng and Yi Ma and Shenghua Gao},
|
| 122 |
+
year={2026},
|
| 123 |
+
eprint={2609.34271},
|
| 124 |
+
archivePrefix={arXiv},
|
| 125 |
+
primaryClass={cs.CV},
|
| 126 |
+
url={https://arxiv.org/abs/2609.34271},
|
| 127 |
+
}
|
| 128 |
+
```
|
| 129 |
+
|
| 130 |
+
## License
|
| 131 |
+
|
| 132 |
+
Apache License 2.0. See the [GitHub repository](https://github.com/libd1/Alchemy3D) for dependency licenses (O-Voxel / TRELLIS.2, nvdiffrast, nvdiffrec, etc.).
|
logo.png
ADDED
|
Git LFS Details
|
teaser.png
ADDED
|
Git LFS Details
|