---
language:
- zh
- en
license: apache-2.0
pipeline_tag: automatic-speech-recognition
base_model: FireRedTeam/FireRedASR2-AED
tags:
- ctranslate2
- audio
- asr
- firered
---

# FireRedASR2-AED, converted to CTranslate2 (float32)

This is [FireRedTeam/FireRedASR2-AED](https://huggingface.co/FireRedTeam/FireRedASR2-AED) converted to the
CTranslate2 format, without quantization (float32 weights). The original model and its license
(Apache-2.0) belong to the FireRedTeam; see the [FireRedASR2S repository](https://github.com/FireRedTeam/FireRedASR2S).
Nothing about the model itself was retrained or changed.

The directory contains `model.bin`, `config.json`, `vocabulary.json`, the SentencePiece model `bpe.model` and the
CMVN statistics `cmvn.ark`.

## Requirements

Stock CTranslate2 and faster-whisper cannot load this model. FireRed is not a Whisper model: it has a Conformer
encoder with Transformer-XL relative-position attention and its own front end and tokenizer. Loading needs a
CTranslate2 build that includes the `FireRed` model class (the voxrun-ctranslate2 fork, with FireRed support).
That support is not part of a released wheel yet.

## Usage

```python
import ctranslate2
from ctranslate2.models.firered_features import FireRedFeatureExtractor
from ctranslate2.models.firered_tokenizer import FireRedTokenizer

path = "FireRedASR2-AED-ct2"  # this repository, downloaded
model = ctranslate2.models.FireRed(path, device="cuda", compute_type="float16")
extractor = FireRedFeatureExtractor(path)   # needs kaldi-native-fbank
tokenizer = FireRedTokenizer(path)          # needs sentencepiece

samples, rate = extractor.read_wav("audio.wav")   # 16 kHz mono
features, lengths = extractor([samples], rate)
result = model.transcribe(ctranslate2.StorageView.from_array(features), lengths)[0]
print(tokenizer.detokenize(result.sequences_ids[0]))
```

The weights are float32 (4.7 GB). Choose the precision at load time with `compute_type`: `float16`, `bfloat16`,
`int8_float16`, `int8_bfloat16` and CPU `int8` all work. Output is Simplified Chinese.

## Accuracy (FLEURS test, CTranslate2 float32, beam 3)

Character error rate after removing case, spaces and punctuation: Mandarin 10.21 %, Cantonese 8.46 % (scored
against Simplified Chinese). Float16 and bfloat16 stay within 0.4 points of these numbers. On the CTranslate2
side the token IDs of float32 decoding match the upstream PyTorch output on the clips that were compared.