Instructions to use Akaneqwq/DualDiff with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Diffusers
How to use Akaneqwq/DualDiff with Diffusers:
pip install -U diffusers transformers accelerate
import torch from diffusers import DiffusionPipeline from diffusers.utils import load_image # switch to "mps" for apple devices pipe = DiffusionPipeline.from_pretrained("Akaneqwq/DualDiff", dtype=torch.bfloat16, device_map="cuda") prompt = "Turn this cat into a dog" input_image = load_image("https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/diffusers/cat.png") image = pipe(image=input_image, prompt=prompt).images[0] - Notebooks
- Google Colab
- Kaggle
Upload official DualDiff checkpoint
Browse files- .gitattributes +0 -34
- README.md +60 -0
- model.manifest.json +36 -0
- model.pkl +3 -0
.gitattributes
CHANGED
|
@@ -1,35 +1 @@
|
|
| 1 |
-
*.7z filter=lfs diff=lfs merge=lfs -text
|
| 2 |
-
*.arrow filter=lfs diff=lfs merge=lfs -text
|
| 3 |
-
*.bin filter=lfs diff=lfs merge=lfs -text
|
| 4 |
-
*.bz2 filter=lfs diff=lfs merge=lfs -text
|
| 5 |
-
*.ckpt filter=lfs diff=lfs merge=lfs -text
|
| 6 |
-
*.ftz filter=lfs diff=lfs merge=lfs -text
|
| 7 |
-
*.gz filter=lfs diff=lfs merge=lfs -text
|
| 8 |
-
*.h5 filter=lfs diff=lfs merge=lfs -text
|
| 9 |
-
*.joblib filter=lfs diff=lfs merge=lfs -text
|
| 10 |
-
*.lfs.* filter=lfs diff=lfs merge=lfs -text
|
| 11 |
-
*.mlmodel filter=lfs diff=lfs merge=lfs -text
|
| 12 |
-
*.model filter=lfs diff=lfs merge=lfs -text
|
| 13 |
-
*.msgpack filter=lfs diff=lfs merge=lfs -text
|
| 14 |
-
*.npy filter=lfs diff=lfs merge=lfs -text
|
| 15 |
-
*.npz filter=lfs diff=lfs merge=lfs -text
|
| 16 |
-
*.onnx filter=lfs diff=lfs merge=lfs -text
|
| 17 |
-
*.ot filter=lfs diff=lfs merge=lfs -text
|
| 18 |
-
*.parquet filter=lfs diff=lfs merge=lfs -text
|
| 19 |
-
*.pb filter=lfs diff=lfs merge=lfs -text
|
| 20 |
-
*.pickle filter=lfs diff=lfs merge=lfs -text
|
| 21 |
*.pkl filter=lfs diff=lfs merge=lfs -text
|
| 22 |
-
*.pt filter=lfs diff=lfs merge=lfs -text
|
| 23 |
-
*.pth filter=lfs diff=lfs merge=lfs -text
|
| 24 |
-
*.rar filter=lfs diff=lfs merge=lfs -text
|
| 25 |
-
*.safetensors filter=lfs diff=lfs merge=lfs -text
|
| 26 |
-
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
| 27 |
-
*.tar.* filter=lfs diff=lfs merge=lfs -text
|
| 28 |
-
*.tar filter=lfs diff=lfs merge=lfs -text
|
| 29 |
-
*.tflite filter=lfs diff=lfs merge=lfs -text
|
| 30 |
-
*.tgz filter=lfs diff=lfs merge=lfs -text
|
| 31 |
-
*.wasm filter=lfs diff=lfs merge=lfs -text
|
| 32 |
-
*.xz filter=lfs diff=lfs merge=lfs -text
|
| 33 |
-
*.zip filter=lfs diff=lfs merge=lfs -text
|
| 34 |
-
*.zst filter=lfs diff=lfs merge=lfs -text
|
| 35 |
-
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
*.pkl filter=lfs diff=lfs merge=lfs -text
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
README.md
ADDED
|
@@ -0,0 +1,60 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: other
|
| 3 |
+
library_name: diffusers
|
| 4 |
+
pipeline_tag: image-to-image
|
| 5 |
+
base_model: stabilityai/sd-turbo
|
| 6 |
+
tags:
|
| 7 |
+
- image-restoration
|
| 8 |
+
- novel-view-synthesis
|
| 9 |
+
- 3d-gaussian-splatting
|
| 10 |
+
- dualdiff3d
|
| 11 |
+
---
|
| 12 |
+
|
| 13 |
+
# DualDiff
|
| 14 |
+
|
| 15 |
+
Official DualDiff repair checkpoint for
|
| 16 |
+
[DualDiff3D](https://github.com/Akaneqwq/DualDiff3D).
|
| 17 |
+
|
| 18 |
+
DualDiff repairs a degraded novel-view rendering using its structure together
|
| 19 |
+
with the appearance of a nearby reference image.
|
| 20 |
+
|
| 21 |
+
## Usage
|
| 22 |
+
|
| 23 |
+
The DualDiff3D code downloads this checkpoint automatically:
|
| 24 |
+
|
| 25 |
+
```bash
|
| 26 |
+
python -m DualDiff.hub
|
| 27 |
+
```
|
| 28 |
+
|
| 29 |
+
Run image-pair inference:
|
| 30 |
+
|
| 31 |
+
```bash
|
| 32 |
+
python DualDiff/infer.py \
|
| 33 |
+
--input_image /path/to/degraded_render.png \
|
| 34 |
+
--ref_image /path/to/reference.png \
|
| 35 |
+
--output_dir outputs/repair
|
| 36 |
+
```
|
| 37 |
+
|
| 38 |
+
The checkpoint contains the complete structure and appearance U-Net state
|
| 39 |
+
dictionaries plus the trained VAE LoRA and skip-connection parameters. It does
|
| 40 |
+
not contain optimizer state. The SD-Turbo VAE, text encoder, tokenizer,
|
| 41 |
+
scheduler, and base configuration are loaded separately by the code.
|
| 42 |
+
|
| 43 |
+
## Citation
|
| 44 |
+
|
| 45 |
+
```bibtex
|
| 46 |
+
@inproceedings{wang2026dualdiff3d,
|
| 47 |
+
title = {DualDiff3D: Dual Structure-Appearance Diffusion
|
| 48 |
+
Priors for Reliability-Enhanced 3D Gaussian Splatting},
|
| 49 |
+
author = {Qian Wang and Yu Wang and Weiqi Li and Xinhua Cheng
|
| 50 |
+
and Xiandong Meng and Ronggang Wang and Jian Zhang},
|
| 51 |
+
booktitle = {European Conference on Computer Vision (ECCV)},
|
| 52 |
+
year = {2026}
|
| 53 |
+
}
|
| 54 |
+
```
|
| 55 |
+
|
| 56 |
+
## License
|
| 57 |
+
|
| 58 |
+
The DualDiff code is released under Apache-2.0. This checkpoint is derived from
|
| 59 |
+
[SD-Turbo](https://huggingface.co/stabilityai/sd-turbo); use of the checkpoint
|
| 60 |
+
is also subject to the applicable SD-Turbo model license.
|
model.manifest.json
ADDED
|
@@ -0,0 +1,36 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"format": "DualDiff inference checkpoint",
|
| 3 |
+
"model": "DualDiff",
|
| 4 |
+
"optimizer_removed": true,
|
| 5 |
+
"file": "model.pkl",
|
| 6 |
+
"size_bytes": 3466253977,
|
| 7 |
+
"sha256": "485ce47f2c2f18bfb54ea1ffc1908fce12e7ef1d98cc4e0f7e39afe3d01cf248",
|
| 8 |
+
"rank_vae": 4,
|
| 9 |
+
"vae_lora_target_modules": 43,
|
| 10 |
+
"state_dicts": {
|
| 11 |
+
"state_dict_unet": {
|
| 12 |
+
"entries": 686,
|
| 13 |
+
"tensors": 686,
|
| 14 |
+
"parameters": 865910724,
|
| 15 |
+
"dtypes": [
|
| 16 |
+
"torch.bfloat16"
|
| 17 |
+
]
|
| 18 |
+
},
|
| 19 |
+
"state_dict_vae": {
|
| 20 |
+
"entries": 90,
|
| 21 |
+
"tensors": 90,
|
| 22 |
+
"parameters": 1015452,
|
| 23 |
+
"dtypes": [
|
| 24 |
+
"torch.bfloat16"
|
| 25 |
+
]
|
| 26 |
+
},
|
| 27 |
+
"state_dict_ref_unet": {
|
| 28 |
+
"entries": 686,
|
| 29 |
+
"tensors": 686,
|
| 30 |
+
"parameters": 865910724,
|
| 31 |
+
"dtypes": [
|
| 32 |
+
"torch.bfloat16"
|
| 33 |
+
]
|
| 34 |
+
}
|
| 35 |
+
}
|
| 36 |
+
}
|
model.pkl
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:485ce47f2c2f18bfb54ea1ffc1908fce12e7ef1d98cc4e0f7e39afe3d01cf248
|
| 3 |
+
size 3466253977
|