Darayut commited on
Commit
fb4f5fe
·
verified ·
1 Parent(s): ba3172d

Card: sota package usage, GitHub link, DDD naming

Browse files
Files changed (1) hide show
  1. README.md +22 -10
README.md CHANGED
@@ -20,17 +20,19 @@ Meta's Omnilingual ASR `omniASR_CTC_300M` to **80% encoder sparsity**. It is the
20
  named after DiariZen's Base-s80 / Large-s80 pruned WavLM models. The other size is
21
  [Darayut/sota-large](https://huggingface.co/Darayut/sota-large).
22
 
 
 
23
  ## Results
24
 
25
  Test sets; greedy CTC decoding, no language model. CER for Khmer, WER for English, in %. Lower is better.
26
 
27
- | Model | Params | GMAC / s audio | DDD-Test (km) | FLEURS-km | FLEURS-en | LibriSpeech test-clean | LibriSpeech test-other |
28
  |---|---|---|---|---|---|---|---|
29
  | `omniASR_CTC_300M` (stock) | 325M | 18.3 | 9.65 | 16.26 | – | 6.53 | 14.45 |
30
  | [omniASR-CTC-300M-km-en](https://huggingface.co/Darayut/omniASR-CTC-300M-km-en) (fine-tuned teacher) | 325M | 18.3 | 3.68 | 12.64 | 19.51 | 5.89 | 13.86 |
31
  | **sota-base** | **73M** | **5.8** | **2.60** | **16.39** | **30.24** | **11.15** | **27.36** |
32
 
33
- - **Khmer:** on DDD-Test it is more accurate than both its unpruned fine-tuned teacher and the stock model.
34
  - **English:** it keeps much less of its teacher's accuracy, with about 1.5–2× the teacher's WER.
35
  - **In-domain test sets:** FLEURS-km and FLEURS-en train are in the training mix, so their test sets are in-domain.
36
 
@@ -39,7 +41,7 @@ Test sets; greedy CTC decoding, no language model. CER for Khmer, WER for Englis
39
  - **Base:** a wav2vec2-style CTC encoder with Omnilingual's 9,812-entry character vocabulary, which covers Khmer and Latin script.
40
  - **Pruned parts:** per-layer attention heads and FFN units, so every layer has its own width. `config.json` lists the exact `num_heads` and `ffn_dims`; the encoder has 63M parameters.
41
  - **Unchanged parts:** the CNN feature extractor and the output layer.
42
- - **Loading:** the layer widths are non-uniform, so fairseq2 / `omnilingual-asr` cannot load it. Use the bundled `omniprune` code.
43
 
44
  ## How it was made
45
 
@@ -60,17 +62,27 @@ DPHuBERT (arXiv 2305.17651) as used in DiariZen.
60
  ## Usage
61
 
62
  ```bash
63
- huggingface-cli download Darayut/sota-base --local-dir sota-base
64
- pip install torch torchaudio soundfile safetensors
65
- python sota-base/inference.py audio.wav
66
  ```
67
 
68
  ```python
69
- import sys; sys.path.insert(0, "sota-base")
70
- from inference import load, transcribe
 
 
 
 
 
 
 
 
71
 
72
- model, vocab = load("sota-base", device="cuda")
73
- print(transcribe(model, vocab, ["audio.wav"]))
 
 
 
 
74
  ```
75
 
76
  Audio of any sample rate is accepted; it is mixed to mono and resampled to 16 kHz. Output text is in the
 
20
  named after DiariZen's Base-s80 / Large-s80 pruned WavLM models. The other size is
21
  [Darayut/sota-large](https://huggingface.co/Darayut/sota-large).
22
 
23
+ **Code:** [github.com/netra-ai-lab/Sota](https://github.com/netra-ai-lab/Sota), with the `sota` inference package and the full training recipe.
24
+
25
  ## Results
26
 
27
  Test sets; greedy CTC decoding, no language model. CER for Khmer, WER for English, in %. Lower is better.
28
 
29
+ | Model | Params | GMAC / s audio | DDD (km) | FLEURS-km | FLEURS-en | LibriSpeech test-clean | LibriSpeech test-other |
30
  |---|---|---|---|---|---|---|---|
31
  | `omniASR_CTC_300M` (stock) | 325M | 18.3 | 9.65 | 16.26 | – | 6.53 | 14.45 |
32
  | [omniASR-CTC-300M-km-en](https://huggingface.co/Darayut/omniASR-CTC-300M-km-en) (fine-tuned teacher) | 325M | 18.3 | 3.68 | 12.64 | 19.51 | 5.89 | 13.86 |
33
  | **sota-base** | **73M** | **5.8** | **2.60** | **16.39** | **30.24** | **11.15** | **27.36** |
34
 
35
+ - **Khmer:** on DDD it is more accurate than both its unpruned fine-tuned teacher and the stock model.
36
  - **English:** it keeps much less of its teacher's accuracy, with about 1.5–2× the teacher's WER.
37
  - **In-domain test sets:** FLEURS-km and FLEURS-en train are in the training mix, so their test sets are in-domain.
38
 
 
41
  - **Base:** a wav2vec2-style CTC encoder with Omnilingual's 9,812-entry character vocabulary, which covers Khmer and Latin script.
42
  - **Pruned parts:** per-layer attention heads and FFN units, so every layer has its own width. `config.json` lists the exact `num_heads` and `ffn_dims`; the encoder has 63M parameters.
43
  - **Unchanged parts:** the CNN feature extractor and the output layer.
44
+ - **Loading:** the layer widths are non-uniform, so fairseq2 / `omnilingual-asr` cannot load it. Use the `sota` package (see Usage) or the code bundled in this repo.
45
 
46
  ## How it was made
47
 
 
62
  ## Usage
63
 
64
  ```bash
65
+ pip install git+https://github.com/netra-ai-lab/SOTA
 
 
66
  ```
67
 
68
  ```python
69
+ import sota
70
+
71
+ asr = sota.load_model("sota-base") # downloads this repo's weights once
72
+ print(asr.transcribe("audio.wav"))
73
+ print(asr.transcribe(["a.wav", "b.flac"]))
74
+ ```
75
+
76
+ ```bash
77
+ sota transcribe audio.wav --model sota-base
78
+ ```
79
 
80
+ Without installing the package, the code bundled in this repo also works:
81
+
82
+ ```bash
83
+ huggingface-cli download Darayut/sota-base --local-dir sota-base
84
+ pip install torch torchaudio soundfile safetensors
85
+ python sota-base/inference.py audio.wav
86
  ```
87
 
88
  Audio of any sample rate is accepted; it is mixed to mono and resampled to 16 kHz. Output text is in the