Card: sota package usage, GitHub link, DDD naming
Browse files
README.md
CHANGED
|
@@ -20,17 +20,19 @@ Meta's Omnilingual ASR `omniASR_CTC_300M` to **80% encoder sparsity**. It is the
|
|
| 20 |
named after DiariZen's Base-s80 / Large-s80 pruned WavLM models. The other size is
|
| 21 |
[Darayut/sota-large](https://huggingface.co/Darayut/sota-large).
|
| 22 |
|
|
|
|
|
|
|
| 23 |
## Results
|
| 24 |
|
| 25 |
Test sets; greedy CTC decoding, no language model. CER for Khmer, WER for English, in %. Lower is better.
|
| 26 |
|
| 27 |
-
| Model | Params | GMAC / s audio | DDD
|
| 28 |
|---|---|---|---|---|---|---|---|
|
| 29 |
| `omniASR_CTC_300M` (stock) | 325M | 18.3 | 9.65 | 16.26 | – | 6.53 | 14.45 |
|
| 30 |
| [omniASR-CTC-300M-km-en](https://huggingface.co/Darayut/omniASR-CTC-300M-km-en) (fine-tuned teacher) | 325M | 18.3 | 3.68 | 12.64 | 19.51 | 5.89 | 13.86 |
|
| 31 |
| **sota-base** | **73M** | **5.8** | **2.60** | **16.39** | **30.24** | **11.15** | **27.36** |
|
| 32 |
|
| 33 |
-
- **Khmer:** on DDD
|
| 34 |
- **English:** it keeps much less of its teacher's accuracy, with about 1.5–2× the teacher's WER.
|
| 35 |
- **In-domain test sets:** FLEURS-km and FLEURS-en train are in the training mix, so their test sets are in-domain.
|
| 36 |
|
|
@@ -39,7 +41,7 @@ Test sets; greedy CTC decoding, no language model. CER for Khmer, WER for Englis
|
|
| 39 |
- **Base:** a wav2vec2-style CTC encoder with Omnilingual's 9,812-entry character vocabulary, which covers Khmer and Latin script.
|
| 40 |
- **Pruned parts:** per-layer attention heads and FFN units, so every layer has its own width. `config.json` lists the exact `num_heads` and `ffn_dims`; the encoder has 63M parameters.
|
| 41 |
- **Unchanged parts:** the CNN feature extractor and the output layer.
|
| 42 |
-
- **Loading:** the layer widths are non-uniform, so fairseq2 / `omnilingual-asr` cannot load it. Use the
|
| 43 |
|
| 44 |
## How it was made
|
| 45 |
|
|
@@ -60,17 +62,27 @@ DPHuBERT (arXiv 2305.17651) as used in DiariZen.
|
|
| 60 |
## Usage
|
| 61 |
|
| 62 |
```bash
|
| 63 |
-
|
| 64 |
-
pip install torch torchaudio soundfile safetensors
|
| 65 |
-
python sota-base/inference.py audio.wav
|
| 66 |
```
|
| 67 |
|
| 68 |
```python
|
| 69 |
-
import
|
| 70 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 71 |
|
| 72 |
-
|
| 73 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
| 74 |
```
|
| 75 |
|
| 76 |
Audio of any sample rate is accepted; it is mixed to mono and resampled to 16 kHz. Output text is in the
|
|
|
|
| 20 |
named after DiariZen's Base-s80 / Large-s80 pruned WavLM models. The other size is
|
| 21 |
[Darayut/sota-large](https://huggingface.co/Darayut/sota-large).
|
| 22 |
|
| 23 |
+
**Code:** [github.com/netra-ai-lab/Sota](https://github.com/netra-ai-lab/Sota), with the `sota` inference package and the full training recipe.
|
| 24 |
+
|
| 25 |
## Results
|
| 26 |
|
| 27 |
Test sets; greedy CTC decoding, no language model. CER for Khmer, WER for English, in %. Lower is better.
|
| 28 |
|
| 29 |
+
| Model | Params | GMAC / s audio | DDD (km) | FLEURS-km | FLEURS-en | LibriSpeech test-clean | LibriSpeech test-other |
|
| 30 |
|---|---|---|---|---|---|---|---|
|
| 31 |
| `omniASR_CTC_300M` (stock) | 325M | 18.3 | 9.65 | 16.26 | – | 6.53 | 14.45 |
|
| 32 |
| [omniASR-CTC-300M-km-en](https://huggingface.co/Darayut/omniASR-CTC-300M-km-en) (fine-tuned teacher) | 325M | 18.3 | 3.68 | 12.64 | 19.51 | 5.89 | 13.86 |
|
| 33 |
| **sota-base** | **73M** | **5.8** | **2.60** | **16.39** | **30.24** | **11.15** | **27.36** |
|
| 34 |
|
| 35 |
+
- **Khmer:** on DDD it is more accurate than both its unpruned fine-tuned teacher and the stock model.
|
| 36 |
- **English:** it keeps much less of its teacher's accuracy, with about 1.5–2× the teacher's WER.
|
| 37 |
- **In-domain test sets:** FLEURS-km and FLEURS-en train are in the training mix, so their test sets are in-domain.
|
| 38 |
|
|
|
|
| 41 |
- **Base:** a wav2vec2-style CTC encoder with Omnilingual's 9,812-entry character vocabulary, which covers Khmer and Latin script.
|
| 42 |
- **Pruned parts:** per-layer attention heads and FFN units, so every layer has its own width. `config.json` lists the exact `num_heads` and `ffn_dims`; the encoder has 63M parameters.
|
| 43 |
- **Unchanged parts:** the CNN feature extractor and the output layer.
|
| 44 |
+
- **Loading:** the layer widths are non-uniform, so fairseq2 / `omnilingual-asr` cannot load it. Use the `sota` package (see Usage) or the code bundled in this repo.
|
| 45 |
|
| 46 |
## How it was made
|
| 47 |
|
|
|
|
| 62 |
## Usage
|
| 63 |
|
| 64 |
```bash
|
| 65 |
+
pip install git+https://github.com/netra-ai-lab/SOTA
|
|
|
|
|
|
|
| 66 |
```
|
| 67 |
|
| 68 |
```python
|
| 69 |
+
import sota
|
| 70 |
+
|
| 71 |
+
asr = sota.load_model("sota-base") # downloads this repo's weights once
|
| 72 |
+
print(asr.transcribe("audio.wav"))
|
| 73 |
+
print(asr.transcribe(["a.wav", "b.flac"]))
|
| 74 |
+
```
|
| 75 |
+
|
| 76 |
+
```bash
|
| 77 |
+
sota transcribe audio.wav --model sota-base
|
| 78 |
+
```
|
| 79 |
|
| 80 |
+
Without installing the package, the code bundled in this repo also works:
|
| 81 |
+
|
| 82 |
+
```bash
|
| 83 |
+
huggingface-cli download Darayut/sota-base --local-dir sota-base
|
| 84 |
+
pip install torch torchaudio soundfile safetensors
|
| 85 |
+
python sota-base/inference.py audio.wav
|
| 86 |
```
|
| 87 |
|
| 88 |
Audio of any sample rate is accepted; it is mixed to mono and resampled to 16 kHz. Output text is in the
|