| --- |
| license: other |
| license_name: academic-use-only |
| license_link: https://github.com/ByungWanLim/YeTI/blob/main/LICENSE |
| datasets: |
| - BWLim/YeTI |
| tags: |
| - pytorch |
| - pytorch-lightning |
| - image-denoising |
| - noise-generation |
| - diffusion |
| - computer-vision |
| - image-to-image |
| --- |
| |
| <div align="center"> |
|
|
| # YeTI: You Only Need Two Noisy Images for Real-World sRGB Noise Generation |
|
|
| **ECCV 2026** · [Paper (arXiv:2607.09193)](https://arxiv.org/abs/2607.09193) · [Code (GitHub)](https://github.com/ByungWanLim/YeTI) · [Dataset](https://huggingface.co/datasets/BWLim/YeTI) |
|
|
| Jaekyun Ko<sup>1,2\*</sup>, Byung Wan Lim<sup>1\*</sup>, Soomin Lee<sup>1</sup>, Dongjin Kim<sup>1</sup>, Tae Hyun Kim<sup>1†</sup> |
| <br><sup>1</sup>Department of Computer Science, Hanyang University <sup>2</sup>Mobile eXperience (MX) Division, Samsung Electronics |
| <br><sup>\*</sup>equal contribution <sup>†</sup>corresponding author |
| |
| </div> |
| |
| ## Model Description |
| |
| YeTI is a real-world sRGB noise generation framework that learns to synthesize realistic, signal-dependent sensor noise from only **two noisy observations of the same scene** — no clean ground truth or camera metadata required. It uses a Reconstruction Autoencoder (RAE) to disentangle scene structure from noise characteristics, and a one-step Conditional Diffusion Transformer (C-DiT) trained with consistency objectives to model the latent noise distribution. At inference time, YeTI takes a single noisy image and generates additional realistic noisy samples of the same scene, which can be used to train downstream (self-supervised) denoisers such as AP-BSN and MM-BSN. |
| |
| Full details, training recipe, and evaluation protocol are in the [paper](https://arxiv.org/abs/2607.09193) and the [official code repository](https://github.com/ByungWanLim/YeTI). |
| |
| ## Intended Uses & Limitations |
| |
| **Intended use**: academic research on real-world sRGB noise modeling / synthesis, and as a data-augmentation source for training self-supervised image denoisers. Requires the [YeTI code repository](https://github.com/ByungWanLim/YeTI) to load and run — these checkpoints are not standalone `transformers`/`diffusers` models. |
| |
| **Limitations**: trained primarily on smartphone sRGB noise (SIDD) and MAI2021 data; noise realism may degrade on sensors, ISO ranges, or capture pipelines far outside the training distribution. Released for **academic / non-commercial research use only** — see License below. |
| |
| ## Files in this repository |
| |
| | File | Description | Size | |
| |---|---|---| |
| | `rae.ckpt` | Reconstruction AutoEncoder — disentangles structure / noise latents | ~148 MB | |
| | `c_dit.ckpt` | Conditional Diffusion Transformer — main noise generation model | ~856 MB | |
| | `apbsn.ckpt` | AP-BSN self-supervised denoiser, trained on YeTI-generated noisy data only | ~46 MB | |
| | `apbsn_mix.ckpt` | AP-BSN denoiser trained on a 50:50 mix of YeTI-generated and real noisy data | ~46 MB | |
| | `mmbsn.ckpt` | MM-BSN self-supervised denoiser, trained on YeTI-generated noisy data only | ~68 MB | |
| | `mmbsn_mix.ckpt` | MM-BSN denoiser trained on a 50:50 mix of YeTI-generated and real noisy data | ~68 MB | |
| | `config.json` | Machine-readable checkpoint index | — | |
| |
| ## Training Data |
| |
| Trained on the public **[SIDD](https://www.eecs.yorku.ca/~kamel/sidd/)** (Smartphone Image Denoising Dataset) and **MAI2021** benchmarks. Preprocessed archives matching this codebase's expected format are hosted at **[BWLim/YeTI (dataset)](https://huggingface.co/datasets/BWLim/YeTI)**. |
| |
| ## How to use |
| |
| These checkpoints are PyTorch Lightning `.ckpt` files meant to be used with the official [YeTI code repository](https://github.com/ByungWanLim/YeTI), which defines the model architectures (`yeti/archs`) and LightningModules (`yeti/models`) needed to load them. |
| |
| ```bash |
| # 1) Clone the code |
| git clone https://github.com/ByungWanLim/YeTI.git |
| cd YeTI |
| |
| # 2) Download these weights into ckpt/ |
| huggingface-cli download BWLim/YeTI --local-dir ckpt |
| |
| # 3) Run validation / generation, e.g. |
| python main.py --config configs/val/C-DiT/val_lit_c-dit.yaml --ckpt ckpt/c_dit.ckpt |
| ``` |
| |
| See the repository README for the full training / evaluation / noise-generation usage (environment setup, dataset preparation, and all `run_*.sh` scripts). |
|
|
| ## Citation |
|
|
| ```bibtex |
| @article{ko2026yeti, |
| title = {YeTI: You Only Need Two Noisy Images for Real-World sRGB Noise Generation}, |
| author = {Ko, Jaekyun and Lim, Byung Wan and Lee, Soomin and Kim, Dongjin and Kim, Tae Hyun}, |
| journal = {arXiv preprint arXiv:2607.09193}, |
| year = {2026} |
| } |
| ``` |
|
|
| ## License |
|
|
| These model weights are released for **academic / non-commercial research use only**. See [LICENSE](./LICENSE) for full terms. For commercial licensing inquiries, please contact the corresponding author (Tae Hyun Kim, Hanyang University). |
|
|
