|
Download README.md from scriptvox/whisper-small-ar: direct link, hf CLI and curl.
- Browser
- Download file 4.38 kB
-
https://huggingface.co/scriptvox/whisper-small-ar/resolve/main/README.md
- Command line
-
hf download hf://scriptvox/whisper-small-ar/README.md
-
curl -L -o README.md https://huggingface.co/scriptvox/whisper-small-ar/resolve/main/README.md
4.38 kB
| license: apache-2.0 | |
| language: | |
| - ar | |
| - en | |
| base_model: openai/whisper-small | |
| pipeline_tag: automatic-speech-recognition | |
| library_name: ctranslate2 | |
| tags: | |
| - whisper | |
| - arabic | |
| - gulf-arabic | |
| - emirati | |
| - ctranslate2 | |
| - faster-whisper | |
| - scriptvox | |
| datasets: | |
| - facebook/omnilingual-asr-corpus | |
| - google/fleurs | |
| - mozilla-foundation/common_voice_18_0 | |
| - openslr/librispeech_asr | |
| # Whisper small for Arabic (ScriptVox) | |
| This model is Whisper small, fine-tuned for Arabic speech, with a focus on Gulf and Emirati Arabic. It keeps the English quality of the base model. The files are in the CTranslate2 format (float16), for faster-whisper. | |
| ScriptVox uses this model for local speech to text when the language is Arabic (`--lang ar`). | |
| | Item | Value | | |
| |---|---| | |
| | Base model | `openai/whisper-small` | | |
| | Version | r12-a70 | | |
| | Format | CTranslate2, float16, 484 MB | | |
| | Languages | Arabic (Gulf, Emirati, other dialects, Modern Standard Arabic) and English | | |
| | Licence | Apache-2.0 | | |
| ## Use | |
| With ScriptVox: | |
| ```sh | |
| scriptvox setup speech-local --arabic | |
| scriptvox analyze clip.mp4 --transcript --lang ar | |
| ``` | |
| With faster-whisper: | |
| ```python | |
| from faster_whisper import WhisperModel | |
| model = WhisperModel("scriptvox/whisper-small-ar", compute_type="int8") | |
| segments, info = model.transcribe("clip.wav", language="ar", word_timestamps=True) | |
| for s in segments: | |
| print(s.start, s.end, s.text) | |
| ``` | |
| ## Results | |
| Word error rate (WER). Lower is better. The base is `openai/whisper-small` with the same decoder and the same text normalisation. The interval is a 95 percent paired bootstrap over clips. | |
| | Test set | Clips | Base | This model | Change | | |
| |---|---|---|---|---| | |
| | Gulf Arabic (speakers and prompts not in training) | 402 | 41.32 % | 27.82 % | -32.7 % (-35.3 to -30.2) | | |
| | Arabic, FLEURS test | 428 | 26.36 % | 17.69 % | -32.9 % (-36.5 to -29.3) | | |
| | Emirati Arabic (a corpus not used in training) | 1,048 | 26.52 % | 20.63 % | -22.2 % (-24.7 to -19.5) | | |
| | English, FLEURS test | 647 | 6.52 % | 6.44 % | -1.2 % (-6.1 to +4.2) | | |
| Long recordings, about 8.5 minutes each, decoded in 30 second windows: | |
| | Recording | WER | Gaps longer than 5 s with no words | | |
| |---|---|---| | |
| | Emirati, 533 s | 21.4 % | 0 | | |
| | Gulf, 513 s | 27.9 % | 0 | | |
| Word times are usable on all clips of the four test sets. The model writes punctuation on 94 to 100 percent of the clips. | |
| ## Training | |
| 1. Fine-tune `openai/whisper-small` for 14,000 steps at a learning rate of 1e-5. | |
| 2. Half of the training items are 30 second windows made from several clips, each with its own start and end times. This teaches the model the form of a long recording. | |
| 3. Mix the weights: 0.70 fine-tuned and 0.30 base. The mix keeps the English quality. The mix ratio was chosen on the dev sets. The test sets were read one time, after that choice. | |
| Training data: | |
| | Data | Licence | | |
| |---|---| | |
| | Meta Omnilingual ASR Corpus: Gulf Arabic and 13 other Arabic dialects | CC BY 4.0 | | |
| | Mozilla Common Voice 18, Arabic | CC0 | | |
| | Google FLEURS, Arabic and English | CC BY 4.0 | | |
| | LibriSpeech, English, with text from Whisper large-v3 | CC BY 4.0 | | |
| No audio from YouTube or from other web video is in the training data. | |
| ## Limits | |
| - The error rate on dialect speech is still high. Check the text before you publish it. | |
| - Some of the gain is in spelling of one letter (for example ه and ة). This can be a spelling change and not a hearing change. | |
| - The model was not tested on noisy recordings, music under speech, or speech with more than one person at a time. | |
| - Do not use this model for decisions about a person, such as legal, medical or hiring decisions. | |
| ## Credits | |
| - OpenAI Whisper: the base model (Apache-2.0 model card, MIT code). | |
| - Meta: Omnilingual ASR Corpus (CC BY 4.0). | |
| - Google: FLEURS (CC BY 4.0). | |
| - Mozilla: Common Voice (CC0). | |
| - LibriSpeech: Panayotov, Chen, Povey and Khudanpur (CC BY 4.0). | |
| - CTranslate2 and faster-whisper: the format and the decoder. | |
| ## Files | |
| | File | sha256 | | |
| |---|---| | |
| | model.bin | f09579e04bcbca6f42ab69ca2eba36ecb1a0e093a9eac2c93fc3eac93a829f8c | | |
| | config.json | b55496ac7940a7ae47d2c01eab40edfd8701feec1229d9cce3b40014383fb828 | | |
| | preprocessor_config.json | 994838f1fa6462c8b9b3c90edada831f11f3dd8b4664634e18f4694d005c9dbf | | |
| | tokenizer.json | 27fc476bfe7f17299480be2273fc0608e4d5a99aba2ab5dec5374b4482d1a566 | | |
| | vocabulary.json | 5ad28279db5e546349708b9a74736e2c018737ac5a600f1048f1f26c99b85b47 | | |