| --- |
| license: mit |
| --- |
| <img src="https://github.com/haidog-yaqub/DiffPitcher/raw/main/img/cover.png"> |
|
|
| # Diff-Pitcher (PyTorch) |
|
|
| Official Pytorch Implementation of [Diff-Pitcher: Diffusion-based Singing Voice Pitch Correction](https://engineering.jhu.edu/lcap/data/uploads/pdfs/waspaa2023_hai.pdf) |
|
|
| -------------------- |
|
|
| Thank you all for your interest in this research project. I am currently optimizing the model's performance and computation efficiency. I plan to release a user-friendly version, either a GUI or a VST, in the first half of this year, and will update the open-source license. |
|
|
| If you are familiar with PyTorch, you can follow [Code Examples](#examples) to use Diff-Pitcher. |
|
|
| -------------------- |
|
|
| Diff-Pitcher |
|
|
| - [Demo Page](#demo) |
| - [Todo List](#todo) |
| - [Code Examples](#examples) |
| - [References](#references) |
| - [Acknowledgement](#acknowledgement) |
|
|
| ## Demo |
|
|
| 🎵 Listen to [examples](https://jhu-lcap.github.io/Diff-Pitcher/) |
|
|
| ## Todo |
| - [x] Update codes and demo |
| - [x] Support 🤗 [Diffusers](https://github.com/huggingface/diffusers) |
| - [x] Upload checkpoints |
| - [x] Pipeline tutorial |
| - [ ] Merge to [Your-Stable-Audio](https://github.com/haidog-yaqub/Your-Stable-Audio) |
| - [ ] Audio Plugin Support |
| ## Examples |
| - Download checkpoints: 🎒[ckpts](https://github.com/haidog-yaqub/DiffPitcher/tree/main/ckpts) |
| - Prepare environment: [requirements.txt](requirements.txt) |
| - Feel free to try: |
| - template-based automatic pitch correction: [template_based_apc.py](template_based_apc.py) |
| - score-based automatic pitch correction: [score_based_apc.py](score_based_apc.py) |
|
|
|
|
| ## References |
|
|
| If you find the code useful for your research, please consider citing: |
|
|
| ```bibtex |
| @inproceedings{hai2023diff, |
| title={Diff-Pitcher: Diffusion-Based Singing Voice Pitch Correction}, |
| author={Hai, Jiarui and Elhilali, Mounya}, |
| booktitle={2023 IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (WASPAA)}, |
| pages={1--5}, |
| year={2023}, |
| organization={IEEE} |
| } |
| ``` |
|
|
| This repo is inspired by: |
|
|
| ```bibtex |
| @article{popov2021diffusion, |
| title={Diffusion-based voice conversion with fast maximum likelihood sampling scheme}, |
| author={Popov, Vadim and Vovk, Ivan and Gogoryan, Vladimir and Sadekova, Tasnima and Kudinov, Mikhail and Wei, Jiansheng}, |
| journal={arXiv preprint arXiv:2109.13821}, |
| year={2021} |
| } |
| ``` |
| ```bibtex |
| @inproceedings{liu2022diffsinger, |
| title={Diffsinger: Singing voice synthesis via shallow diffusion mechanism}, |
| author={Liu, Jinglin and Li, Chengxi and Ren, Yi and Chen, Feiyang and Zhao, Zhou}, |
| booktitle={Proceedings of the AAAI conference on artificial intelligence}, |
| volume={36}, |
| number={10}, |
| pages={11020--11028}, |
| year={2022} |
| } |
| ``` |
|
|
| ## Acknowledgement |
|
|
| [Welcome to LCAP! < LCAP (jhu.edu)](https://engineering.jhu.edu/lcap/) |
|
|
| We borrow code from following repos: |
|
|
| - `Diffusion Schedulers` are based on 🤗 [Diffusers](https://github.com/huggingface/diffusers) |
| - `2D UNet` is based on [DiffVC](https://github.com/huawei-noah/Speech-Backbones/tree/main/DiffVC) |
