Add model card
Browse files
README.md
ADDED
|
@@ -0,0 +1,57 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
language:
|
| 3 |
+
- zh
|
| 4 |
+
- en
|
| 5 |
+
license: apache-2.0
|
| 6 |
+
pipeline_tag: automatic-speech-recognition
|
| 7 |
+
base_model: FireRedTeam/FireRedASR2-AED
|
| 8 |
+
tags:
|
| 9 |
+
- ctranslate2
|
| 10 |
+
- audio
|
| 11 |
+
- asr
|
| 12 |
+
- firered
|
| 13 |
+
---
|
| 14 |
+
|
| 15 |
+
# FireRedASR2-AED, converted to CTranslate2 (float32)
|
| 16 |
+
|
| 17 |
+
This is [FireRedTeam/FireRedASR2-AED](https://huggingface.co/FireRedTeam/FireRedASR2-AED) converted to the
|
| 18 |
+
CTranslate2 format, without quantization (float32 weights). The original model and its license
|
| 19 |
+
(Apache-2.0) belong to the FireRedTeam; see the [FireRedASR2S repository](https://github.com/FireRedTeam/FireRedASR2S).
|
| 20 |
+
Nothing about the model itself was retrained or changed.
|
| 21 |
+
|
| 22 |
+
The directory contains `model.bin`, `config.json`, `vocabulary.json`, the SentencePiece model `bpe.model` and the
|
| 23 |
+
CMVN statistics `cmvn.ark`.
|
| 24 |
+
|
| 25 |
+
## Requirements
|
| 26 |
+
|
| 27 |
+
Stock CTranslate2 and faster-whisper cannot load this model. FireRed is not a Whisper model: it has a Conformer
|
| 28 |
+
encoder with Transformer-XL relative-position attention and its own front end and tokenizer. Loading needs a
|
| 29 |
+
CTranslate2 build that includes the `FireRed` model class (the voxrun-ctranslate2 fork, with FireRed support).
|
| 30 |
+
That support is not part of a released wheel yet.
|
| 31 |
+
|
| 32 |
+
## Usage
|
| 33 |
+
|
| 34 |
+
```python
|
| 35 |
+
import ctranslate2
|
| 36 |
+
from ctranslate2.models.firered_features import FireRedFeatureExtractor
|
| 37 |
+
from ctranslate2.models.firered_tokenizer import FireRedTokenizer
|
| 38 |
+
|
| 39 |
+
path = "FireRedASR2-AED-ct2" # this repository, downloaded
|
| 40 |
+
model = ctranslate2.models.FireRed(path, device="cuda", compute_type="float16")
|
| 41 |
+
extractor = FireRedFeatureExtractor(path) # needs kaldi-native-fbank
|
| 42 |
+
tokenizer = FireRedTokenizer(path) # needs sentencepiece
|
| 43 |
+
|
| 44 |
+
samples, rate = extractor.read_wav("audio.wav") # 16 kHz mono
|
| 45 |
+
features, lengths = extractor([samples], rate)
|
| 46 |
+
result = model.transcribe(ctranslate2.StorageView.from_array(features), lengths)[0]
|
| 47 |
+
print(tokenizer.detokenize(result.sequences_ids[0]))
|
| 48 |
+
```
|
| 49 |
+
|
| 50 |
+
The weights are float32 (4.7 GB). Choose the precision at load time with `compute_type`: `float16`, `bfloat16`,
|
| 51 |
+
`int8_float16`, `int8_bfloat16` and CPU `int8` all work. Output is Simplified Chinese.
|
| 52 |
+
|
| 53 |
+
## Accuracy (FLEURS test, CTranslate2 float32, beam 3)
|
| 54 |
+
|
| 55 |
+
Character error rate after removing case, spaces and punctuation: Mandarin 10.21 %, Cantonese 8.46 % (scored
|
| 56 |
+
against Simplified Chinese). Float16 and bfloat16 stay within 0.4 points of these numbers. On the CTranslate2
|
| 57 |
+
side the token IDs of float32 decoding match the upstream PyTorch output on the clips that were compared.
|