Image-to-3D
Diffusers
ONNX
Safetensors
English
3d-reconstruction
articulated-objects
robotics
simulation
Instructions to use 2bidoubi/SPARK with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Diffusers
How to use 2bidoubi/SPARK with Diffusers:
pip install -U diffusers transformers accelerate
import torch from diffusers import DiffusionPipeline # switch to "mps" for apple devices pipe = DiffusionPipeline.from_pretrained("2bidoubi/SPARK", dtype=torch.bfloat16, device_map="cuda") prompt = "Astronaut in a jungle, cold color palette, muted colors, detailed, 8k" image = pipe(prompt).images[0] - Notebooks
- Google Colab
- Kaggle
Add SPARK model card and 8-second preview video
#1
by bd469 - opened
- .gitattributes +1 -0
- README.md +88 -3
- media/spark-preview.jpg +0 -0
- media/spark-preview.mp4 +3 -0
.gitattributes
CHANGED
|
@@ -36,3 +36,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
|
| 36 |
RMBG-1.4/results.png filter=lfs diff=lfs merge=lfs -text
|
| 37 |
RMBG-1.4/t4.png filter=lfs diff=lfs merge=lfs -text
|
| 38 |
RMBG-1.4/example_input.jpg filter=lfs diff=lfs merge=lfs -text
|
|
|
|
|
|
| 36 |
RMBG-1.4/results.png filter=lfs diff=lfs merge=lfs -text
|
| 37 |
RMBG-1.4/t4.png filter=lfs diff=lfs merge=lfs -text
|
| 38 |
RMBG-1.4/example_input.jpg filter=lfs diff=lfs merge=lfs -text
|
| 39 |
+
media/spark-preview.mp4 filter=lfs diff=lfs merge=lfs -text
|
README.md
CHANGED
|
@@ -1,3 +1,88 @@
|
|
| 1 |
-
---
|
| 2 |
-
|
| 3 |
-
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
language:
|
| 3 |
+
- en
|
| 4 |
+
pipeline_tag: image-to-3d
|
| 5 |
+
tags:
|
| 6 |
+
- arxiv:2512.01629
|
| 7 |
+
- 3d-reconstruction
|
| 8 |
+
- articulated-objects
|
| 9 |
+
- robotics
|
| 10 |
+
- simulation
|
| 11 |
+
---
|
| 12 |
+
|
| 13 |
+
<div align="center">
|
| 14 |
+
|
| 15 |
+
<h1>SPARK</h1>
|
| 16 |
+
|
| 17 |
+
<p><strong>Sim-ready Part-level Articulated Reconstruction<br>with VLM Knowledge</strong></p>
|
| 18 |
+
|
| 19 |
+
<p>CVPR 2026 · <strong>Oral</strong></p>
|
| 20 |
+
|
| 21 |
+
<p>
|
| 22 |
+
<a href="https://heyumeng.com/SPARK-web/"><strong>Website</strong></a> ·
|
| 23 |
+
<a href="https://arxiv.org/abs/2512.01629"><strong>arXiv</strong></a> ·
|
| 24 |
+
<a href="https://github.com/YumengHe/SPARK"><strong>Code</strong></a>
|
| 25 |
+
</p>
|
| 26 |
+
|
| 27 |
+
</div>
|
| 28 |
+
|
| 29 |
+
SPARK is a framework for reconstructing articulated 3D objects from a single RGB image.
|
| 30 |
+
|
| 31 |
+
<video controls autoplay loop muted playsinline preload="metadata" width="100%" poster="https://huggingface.co/2bidoubi/SPARK/resolve/d9fadf7111cc965bbcfbab8a9e4ebfab61cf7725/media/spark-preview.jpg" src="https://huggingface.co/2bidoubi/SPARK/resolve/d9fadf7111cc965bbcfbab8a9e4ebfab61cf7725/media/spark-preview.mp4"></video>
|
| 32 |
+
|
| 33 |
+
*Input images and articulated reconstructions. An 8-second excerpt from the [full project video](https://heyumeng.com/SPARK-web/static/images/SPARK_video.mp4).*
|
| 34 |
+
|
| 35 |
+
## How SPARK works
|
| 36 |
+
|
| 37 |
+
The [paper](https://arxiv.org/abs/2512.01629) describes three main stages:
|
| 38 |
+
|
| 39 |
+
1. **VLM guidance** estimates the part hierarchy and motion, with image generation providing reference views.
|
| 40 |
+
2. **Joint geometry generation** combines part-level and global image guidance to reconstruct the parts together.
|
| 41 |
+
3. **Joint refinement** uses differentiable rendering to match the articulated reconstruction to an open-state reference.
|
| 42 |
+
|
| 43 |
+
## Input and output
|
| 44 |
+
|
| 45 |
+
- **Input:** one RGB image of an object.
|
| 46 |
+
- **Output:** 3D part meshes and a coarse URDF describing their hierarchy and joints, with textured meshes produced by the texturing stage.
|
| 47 |
+
|
| 48 |
+
## What's included
|
| 49 |
+
|
| 50 |
+
This repository hosts the checkpoint bundle used by the [SPARK implementation](https://github.com/YumengHe/SPARK), not a dataset or a standalone text-to-image model.
|
| 51 |
+
|
| 52 |
+
| Folder | Component |
|
| 53 |
+
| --- | --- |
|
| 54 |
+
| `PartCrafter/` | 3D part-generation checkpoint loaded by inference |
|
| 55 |
+
| `TripoSG/` | Base-model weights used by the training configuration |
|
| 56 |
+
| `RMBG-1.4/` | Optional image background removal |
|
| 57 |
+
|
| 58 |
+
The current bundle is approximately **15.6 GB**. Training details and evaluations are described in the [paper](https://arxiv.org/abs/2512.01629).
|
| 59 |
+
|
| 60 |
+
## Get started
|
| 61 |
+
|
| 62 |
+
Follow the [installation instructions](https://github.com/YumengHe/SPARK#-installation) first. From the SPARK code directory, download the checkpoints with:
|
| 63 |
+
|
| 64 |
+
```bash
|
| 65 |
+
python -m src.utils.download_weights
|
| 66 |
+
```
|
| 67 |
+
|
| 68 |
+
Weights are saved under `pretrained_weights/`. The implementation also downloads missing checkpoint folders when they are first needed.
|
| 69 |
+
|
| 70 |
+
See the [Quick Start](https://github.com/YumengHe/SPARK#-quick-start) for reconstruction and API configuration. The documented setup uses **Python 3.10, PyTorch 2.5.1, and CUDA 12.4**; the released mesh-inference scripts require an NVIDIA CUDA-capable GPU. External image-guidance and texturing services are separate from this checkpoint bundle.
|
| 71 |
+
|
| 72 |
+
In the current public release, joint refinement is provided in separate [optimizer scripts](https://github.com/YumengHe/SPARK/tree/main/URDFoptimizer); it is not run automatically by `run.py`.
|
| 73 |
+
|
| 74 |
+
## License
|
| 75 |
+
|
| 76 |
+
The [SPARK code](https://github.com/YumengHe/SPARK/blob/main/LICENSE) is MIT-licensed. Bundled third-party checkpoints retain their respective terms; SPARK's code license does not replace them. See the component cards for [PartCrafter](PartCrafter/README.md), [TripoSG](TripoSG/README.md), and [RMBG-1.4](https://huggingface.co/briaai/RMBG-1.4). BRIA describes RMBG-1.4 as available for non-commercial use, with a separate commercial agreement required.
|
| 77 |
+
|
| 78 |
+
## Citation
|
| 79 |
+
|
| 80 |
+
```bibtex
|
| 81 |
+
@misc{he2025spark,
|
| 82 |
+
title={SPARK: Sim-ready Part-level Articulated Reconstruction with VLM Knowledge},
|
| 83 |
+
author={Yumeng He and Ying Jiang and Jiayin Lu and Yin Yang and Chenfanfu Jiang},
|
| 84 |
+
year={2025},
|
| 85 |
+
eprint={2512.01629},
|
| 86 |
+
url={https://arxiv.org/abs/2512.01629}
|
| 87 |
+
}
|
| 88 |
+
```
|
media/spark-preview.jpg
ADDED
|
media/spark-preview.mp4
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:77869a7748478bc1c00c18075d484349add399b29824737fed556f53e5ac15da
|
| 3 |
+
size 396525
|