|
Download README.md from ML2/CLEAR: direct link, hf CLI and curl.
- Browser
- Download file 3.8 kB
-
https://huggingface.co/ML2/CLEAR/resolve/main/README.md
- Command line
-
hf download hf://ML2/CLEAR/README.md
-
curl -L -o README.md https://huggingface.co/ML2/CLEAR/resolve/main/README.md
3.8 kB
| license: openrail++ | |
| language: en | |
| library_name: pytorch | |
| pipeline_tag: image-to-image | |
| tags: | |
| - single-image-reflection-removal | |
| - reflection-removal | |
| - image-restoration | |
| - diffusion | |
| # CLEAR: Color-Texture Latent Alignment for Single Image Reflection Removal | |
| **WACV 2027** Β· [Code](https://github.com/kc-ml2/clear) Β· [Paper / OpenReview](https://openreview.net/forum?id=QMmOrvegf1) | |
| Tu Vo, Sheir A. Zaheer, Chan Y. Park β KC Machine Learning Lab | |
| CLEAR is a deterministic, single-timestep latent framework for single image reflection removal. | |
| A physics-guided dual-branch separator predicts the transmission, reflection and residue latents | |
| in one forward pass, and a cross-latent decoder with the **Hybrid Feature Alignment Module (HFAM)** | |
| re-aligns the input skip features to the clean transmission before decoding. There is no noise | |
| schedule and no iterative sampling. | |
| ## Files | |
| ``` | |
| config.yaml Stage-1 training config of the released run | |
| checkpoint/ | |
| βββ diffusion/ | |
| β βββ unet1/ transmission branch (config.json + diffusion_pytorch_model.bin) | |
| β βββ unet2/ reflection branch (config.json + diffusion_pytorch_model.bin) | |
| βββ cld/ | |
| βββ cld_clear.pth Stage-2 HFAM cross-latent decoder | |
| ``` | |
| `cld_clear.pth` is the final Stage-2 checkpoint (212 training hours). It stores the `state_dict` | |
| plus the metadata `refiner_type: hfam`, `total_hours: 212` and `name: 0`. | |
| ## Quick start | |
| ```bash | |
| git clone https://github.com/kc-ml2/clear.git | |
| cd clear | |
| pip install -r requirements.txt | |
| python scripts/download_models.py --save_dir ./models # SDv2 + Marigold backbones | |
| CLEAR_HF_REPO_ID=ML2/CLEAR python scripts/download_models.py --weights --save_dir ./models | |
| ``` | |
| Run on a folder of images (the folder must contain a `blended/` subfolder): | |
| ```bash | |
| bash scripts/inference.sh \ | |
| models/clear/checkpoint/diffusion \ | |
| models/clear/checkpoint/cld/cld_clear.pth \ | |
| <DATA_DIR> outputs/inference | |
| ``` | |
| This writes `<name>_clear.png` (the reflection-free output), `<name>_r.png` (predicted | |
| reflection), `<name>_g1.png` (residue) and the input / target copies. | |
| ## Results | |
| Average over six real-world benchmarks (Nature, SIR2, Real20, DRR-C, DRR-S, OpenRR-1k β | |
| 1,040 images), as reported in the paper: | |
| | Method | PSNR β | SSIM β | LPIPS β | MANIQA β | QUALICLIP β | | |
| |---|---|---|---|---|---| | |
| | DAI | 25.98 | 0.869 | 0.159 | 0.460 | 0.696 | | |
| | WindowSeat | 25.48 | 0.810 | 0.208 | 0.465 | 0.674 | | |
| | **CLEAR (Ours)** | **26.59** | **0.906** | **0.106** | **0.504** | **0.731** | | |
| See the paper for the per-dataset tables and ablations. | |
| ## Notes | |
| * `config.yaml` is the original Stage-1 training config and still refers to the authors' | |
| workspace (`base_config: ../config/default.yaml`). For inference use | |
| `configs/stage1_clear.yaml` from the code repository. | |
| * Stage-1 is initialised from `stabilityai/stable-diffusion-2` and | |
| `prs-eth/marigold-depth-v1-0`; both are downloaded by `scripts/download_models.py`. | |
| * The VAE and text encoder are not duplicated here β they come from the SDv2 backbone. | |
| ## License | |
| The **code** is released under Apache-2.0. These **weights** are fine-tuned from | |
| Stable Diffusion 2 and are distributed under the | |
| [CreativeML Open RAIL++-M License](https://huggingface.co/stabilityai/stable-diffusion-2/blob/main/LICENSE-MODEL); | |
| the use restrictions of that license apply. | |
| ## Citation | |
| ```bibtex | |
| @inproceedings{vo2027clear, | |
| title = {CLEAR: Color-Texture Latent Alignment for Single Image Reflection Removal}, | |
| author = {Vo, Tu and Zaheer, Sheir A. and Park, Chan Y.}, | |
| booktitle = {The IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)}, | |
| year = {2027} | |
| } | |
| ``` | |