twmht commited on
Commit
5dc75c7
·
verified ·
1 Parent(s): 9ae1866

Add model card

Browse files
Files changed (1) hide show
  1. README.md +57 -0
README.md ADDED
@@ -0,0 +1,57 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ language:
3
+ - zh
4
+ - en
5
+ license: apache-2.0
6
+ pipeline_tag: automatic-speech-recognition
7
+ base_model: FireRedTeam/FireRedASR2-AED
8
+ tags:
9
+ - ctranslate2
10
+ - audio
11
+ - asr
12
+ - firered
13
+ ---
14
+
15
+ # FireRedASR2-AED, converted to CTranslate2 (float32)
16
+
17
+ This is [FireRedTeam/FireRedASR2-AED](https://huggingface.co/FireRedTeam/FireRedASR2-AED) converted to the
18
+ CTranslate2 format, without quantization (float32 weights). The original model and its license
19
+ (Apache-2.0) belong to the FireRedTeam; see the [FireRedASR2S repository](https://github.com/FireRedTeam/FireRedASR2S).
20
+ Nothing about the model itself was retrained or changed.
21
+
22
+ The directory contains `model.bin`, `config.json`, `vocabulary.json`, the SentencePiece model `bpe.model` and the
23
+ CMVN statistics `cmvn.ark`.
24
+
25
+ ## Requirements
26
+
27
+ Stock CTranslate2 and faster-whisper cannot load this model. FireRed is not a Whisper model: it has a Conformer
28
+ encoder with Transformer-XL relative-position attention and its own front end and tokenizer. Loading needs a
29
+ CTranslate2 build that includes the `FireRed` model class (the voxrun-ctranslate2 fork, with FireRed support).
30
+ That support is not part of a released wheel yet.
31
+
32
+ ## Usage
33
+
34
+ ```python
35
+ import ctranslate2
36
+ from ctranslate2.models.firered_features import FireRedFeatureExtractor
37
+ from ctranslate2.models.firered_tokenizer import FireRedTokenizer
38
+
39
+ path = "FireRedASR2-AED-ct2" # this repository, downloaded
40
+ model = ctranslate2.models.FireRed(path, device="cuda", compute_type="float16")
41
+ extractor = FireRedFeatureExtractor(path) # needs kaldi-native-fbank
42
+ tokenizer = FireRedTokenizer(path) # needs sentencepiece
43
+
44
+ samples, rate = extractor.read_wav("audio.wav") # 16 kHz mono
45
+ features, lengths = extractor([samples], rate)
46
+ result = model.transcribe(ctranslate2.StorageView.from_array(features), lengths)[0]
47
+ print(tokenizer.detokenize(result.sequences_ids[0]))
48
+ ```
49
+
50
+ The weights are float32 (4.7 GB). Choose the precision at load time with `compute_type`: `float16`, `bfloat16`,
51
+ `int8_float16`, `int8_bfloat16` and CPU `int8` all work. Output is Simplified Chinese.
52
+
53
+ ## Accuracy (FLEURS test, CTranslate2 float32, beam 3)
54
+
55
+ Character error rate after removing case, spaces and punctuation: Mandarin 10.21 %, Cantonese 8.46 % (scored
56
+ against Simplified Chinese). Float16 and bfloat16 stay within 0.4 points of these numbers. On the CTranslate2
57
+ side the token IDs of float32 decoding match the upstream PyTorch output on the clips that were compared.