| --- |
| language: |
| - az |
| license: cc-by-nc-4.0 |
| library_name: nemo |
| pipeline_tag: automatic-speech-recognition |
| tags: |
| - asr |
| - ctc |
| - fastconformer |
| - kenlm |
| - azerbaijani |
| --- |
| |
| # speakwrite-azerbaijani |
|
|
|
|
| Azerbaijani speech recognition for CPU: FastConformer-CTC (18 layers, d512) with a 3-pass loop refiner, product-key acoustic memory |
| (700 x 700 sub-keys) and a 6-gram token KenLM. Input is 16 kHz mono; output has case and `. , ? !`, with numbers written as words. |
|
|
| ## Results |
|
|
| [LocalDoc/fleurs-azerbaijani-asr](https://huggingface.co/datasets/LocalDoc/fleurs-azerbaijani-asr), not used for training. Reference: `text_norm` |
| (numbers as words). WER and CER ignore case and punctuation. Clips with `norm_status = failed` are excluded (11 of 3977). |
|
|
| | split | clips | hours | no LM: WER / CER | KenLM: WER / CER | |
| |---|---|---|---|---| |
| | test | 914 | 3.20 | 16.34 / 4.75 | **10.98 / 3.57** | |
| | dev | 397 | 1.34 | 16.52 / 4.75 | 11.05 / 3.50 | |
| | train | 2655 | 9.27 | 15.82 / 4.80 | 11.30 / 3.78 | |
| | all | 3966 | 13.81 | 16.01 / 4.79 | 11.20 / 3.70 | |
|
|
| - Decoder weights (`alpha` 1.0, `beta` 2.0) were chosen on the FLEURS dev split from 8 pairs, so the test row is the one to quote. |
| - Sentences of the FLEURS test split were excluded from the KenLM text; dev and train sentences were not. |
| - Internal validation, no LM, on our own data mix: WER 13.48 / CER 6.01. |
|
|
| Speed, PyTorch float32 on one i7-12700K, in Docker, times faster than real time, for the acoustic model plus KenLM: |
|
|
| | threads | speed | measured on | |
| |---|---|---| |
| | 1 | 12.7x | an earlier checkpoint of this model, 100 clips | |
| | 8 | ~45x | this checkpoint, all FLEURS | |
|
|
| The machine had other load during the runs, so these are rough. KenLM takes about 1.5% of the time. ONNX runtimes were not measured. |
|
|
| ## Use |
|
|
| Needs [balbal](https://github.com/erayyap/balbal) at commit `1be9f88b77c63fbafa7713c6e46e79847f6c390e` (Python 3.11-3.12, Linux; install |
| `requirements/full.txt`, then run `inference/build_native.py`). Copy this repository's `refiner.json` over balbal's |
| `configs/balbal-v2-manas/refiner.json`: this model's memory size differs from balbal's default, and the weights will not load otherwise. |
|
|
| ```python |
| import json, sys |
| sys.path[:0] = ["balbal/inference", "balbal"] |
| from balbal.audio import read_audio |
| from runtime import Engine |
| |
| engine = Engine("speakwrite-azerbaijani", runtime="torch", settings=json.load(open("speakwrite-azerbaijani/decoder.json"))) |
| print(engine.transcribe(read_audio("speech.mp3"), itn=False)["text"]) |
| ``` |
|
|
| Keep `itn=False`: balbal's number conversion is Turkish. Long audio is cut into fixed 30 s chunks. |
|
|
| To use the model without the KenLM, take the argmax of the CTC output instead of `Engine`: without an LM, WER on FLEURS test is 16.34. The `lm/` folder is then not needed. |
|
|
| ## Training |
|
|
| About 2800 hours of Azerbaijani speech. Encoder initialized from |
| [nvidia/stt_en_fastconformer_ctc_large](https://huggingface.co/nvidia/stt_en_fastconformer_ctc_large), then modified and trained for 4.5 epochs |
| (127552 updates) with the balbal-v2-manas recipe. balbal's own weights were not used. |
|
|
| ## License |
|
|
| [CC BY-NC 4.0](https://creativecommons.org/licenses/by-nc/4.0/). Non-commercial use only; credit is required in any project using this model: |
| "speakwrite-azerbaijani by LocalDoc" with a link to this page. For commercial use, contact [LocalDoc](https://huggingface.co/LocalDoc). |
|
|
| Based on `nvidia/stt_en_fastconformer_ctc_large` (CC BY 4.0, NVIDIA) and on code from balbal (Apache 2.0). |
|
|
