JusperLee commited on
Commit
c8d2f00
ยท
verified ยท
1 Parent(s): d9c63d9

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +103 -5
README.md CHANGED
@@ -1,10 +1,108 @@
1
  ---
2
  pipeline_tag: audio-to-audio
3
  tags:
4
- - model_hub_mixin
5
- - pytorch_model_hub_mixin
 
 
 
 
 
6
  ---
7
 
8
- This model has been pushed to the Hub using the [PytorchModelHubMixin](https://huggingface.co/docs/huggingface_hub/package_reference/mixins#huggingface_hub.PyTorchModelHubMixin) integration:
9
- - Library: https://github.com/JusperLee/TIGER
10
- - Docs: [More Information Needed]
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
  pipeline_tag: audio-to-audio
3
  tags:
4
+ - audio
5
+ - small
6
+ license: apache-2.0
7
+ language:
8
+ - en
9
+ base_model:
10
+ - JusperLee/TIGER-speech
11
  ---
12
 
13
+ # Tiny Version
14
+
15
+ <p align="center">
16
+ </p>
17
+ <h3 align="center">TIGER: Time-frequency Interleaved Gain Extraction and Reconstruction for Efficient Speech Separation</h3>
18
+ <p align="center">
19
+ <strong>Mohan Xu<sup>*</sup>, Kai Li<sup>*</sup>, Guo Chen, Xiaolin Hu</strong><br>
20
+ <strong>Tsinghua University, Beijing, China</strong><br>
21
+ <strong><sup>*</sup>Equal contribution</strong><br>
22
+ <a href="https://arxiv.org/abs/2410.01469">๐Ÿ“œ ICLR 2025</a> | <a href="https://cslikai.cn/TIGER/">๐ŸŽถ Demo</a> | <a href="https://huggingface.co/datasets/JusperLee/EchoSet">๐Ÿค— Dataset</a>
23
+
24
+ <p align="center">
25
+ <img src="https://visitor-badge.laobi.icu/badge?page_id=JusperLee.TIGER" alt="่ฎฟๅฎข็ปŸ่ฎก" />
26
+ <img src="https://img.shields.io/github/stars/JusperLee/TIGER?style=social" alt="GitHub stars" />
27
+ <img alt="Static Badge" src="https://img.shields.io/badge/license-Apache%202.0-blue.svg" />
28
+ </p>
29
+
30
+ <p align="center">
31
+
32
+ > TIGER is a lightweight model for speech separation which effectively extracts key acoustic features through frequency band-split, multi-scale and full-frequency-frame modeling.
33
+
34
+ ## ๐Ÿ’ฅ News
35
+
36
+ - **[2025-01-23]** We release the code and pre-trained model of TIGER! ๐Ÿš€
37
+ - **[2025-01-23]** We release the TIGER model and the EchoSet dataset! ๐Ÿš€
38
+
39
+ ## ๐Ÿ“œ Abstract
40
+
41
+ In this paper, we propose a speech separation model with significantly reduced parameter size and computational cost: Time-Frequency Interleaved Gain Extraction and Reconstruction Network (TIGER). TIGER leverages prior knowledge to divide frequency bands and applies compression on frequency information. We employ a multi-scale selective attention (MSA) module to extract contextual features, while introducing a full-frequency-frame attention (F^3A) module to capture both temporal and frequency contextual information. Additionally, to more realistically evaluate the performance of speech separation models in complex acoustic environments, we introduce a novel dataset called EchoSet. This dataset includes noise and more realistic reverberation (e.g., considering object occlusions and material properties), with speech from two speakers overlapping at random proportions. Experimental results demonstrated that TIGER significantly outperformed state-of-the-art (SOTA) model TF-GridNet on the EchoSet dataset in both inference speed and separation quality, while reducing the number of parameters by 94.3% and the MACs by 95.3%. These results indicate that by utilizing frequency band-split and interleaved modeling structures, TIGER achieves a substantial reduction in parameters and computational costs while maintaining high performance. Notably, TIGER is the first speech separation model with fewer than 1 million parameters that achieves performance close to the SOTA model.
42
+
43
+ ## TIGER
44
+
45
+ Overall pipeline of the model architecture of TIGER and its modules.
46
+
47
+ ![TIGER Model Architecture](assets/TIGER.png)
48
+
49
+ ## Results
50
+
51
+ Performance comparisons of TIGER and other existing separation models on ***Libri2Mix, LRS2-2Mix, and EchoSet***. Bold indicates optimal performance, and italics indicate suboptimal performance.
52
+
53
+ ![TIGER Model Architecture](assets/result.png)
54
+
55
+ Efficiency comparisons of TIGER and other models.
56
+
57
+ ![TIGER Model Architecture](assets/efficiency.png)
58
+
59
+ Comparison of performance and efficiency of cinematic sound separation models on DnR. '*' means the result comes from the original paper of DnR.
60
+
61
+ ![TIGER Model Architecture](assets/dnr.png)
62
+
63
+ ## ๐Ÿ“ฆ Installation
64
+
65
+ ```bash
66
+ git clone https://github.com/JusperLee/TIGER.git
67
+ cd TIGER
68
+ pip install -r requirements.txt
69
+ ```
70
+
71
+ ## ๐Ÿš€ Quick Start
72
+
73
+ ### Test with Pre-trained Model
74
+
75
+ ```bash
76
+ # Test using speech
77
+ python inference_speech.py --audio_path test/mix.wav
78
+
79
+ # Test using DnR
80
+ python inference_dnr.py --audio_path test/test_mixture_466.wav
81
+ ```
82
+
83
+ ### Train with EchoSet
84
+
85
+ ```bash
86
+ python audio_train.py --conf_dir configs/tiger.yml
87
+ ```
88
+
89
+ ### Evaluate with EchoSet
90
+
91
+ ```bash
92
+ python audio_test.py --conf_dir configs/tiger.yml
93
+ ```
94
+
95
+ ## ๐Ÿ“– Citation
96
+
97
+ ```bibtex
98
+ @article{xu2024tiger,
99
+ title={TIGER: Time-frequency Interleaved Gain Extraction and Reconstruction for Efficient Speech Separation},
100
+ author={Xu, Mohan and Li, Kai and Chen, Guo and Hu, Xiaolin},
101
+ journal={arXiv preprint arXiv:2410.01469},
102
+ year={2024}
103
+ }
104
+ ```
105
+
106
+ ## ๐Ÿ“ง Contact
107
+
108
+ If you have any questions, please feel free to contact us via `tsinghua.kaili@gmail.com`.