---
language:
- zh
- en
license: apache-2.0
tags:
- mlx
- apple-silicon
- audio
- speech-recognition
- text-to-speech
- voice-cloning
- speech-editing
- voice-design
- multimodal
pipeline_tag: text-to-speech
---

# FireRedAudio-MLX-8bit

Apple Silicon MLX 8-bit quantized checkpoint for **FireRedAudio** (9B multimodal audio language model).

- 🐙 **GitHub Repository & WebUI Studio**: [vanch007/mlx-FireRedAudio](https://github.com/vanch007/mlx-FireRedAudio)
- ⚡ **Hardware Acceleration**: Metal GPU native unified memory, fused SDPA attention, and Gated Delta Metal kernel.
- 🚀 **Real-Time Factor (RTF)**:
  - **ASR (Speech Recognition)**: **RTF 0.1396** (~7.2x real-time speedup)
  - **TTS (Voice Cloning)**: **RTF 0.7529** (Real-time speech generation on Apple M3 Max)
- 💾 **Memory Footprint**: Only **13.54 GB** active Metal memory (runs smoothly on 16GB / 24GB+ Mac devices).

---

## Model Architecture & Quantization Details

- **Backbone LLM**: 9B Qwen3.5 language model with selective 8-bit affine quantization (group_size=64).
- **Continuous Audio Autoencoder / Vocoder**: RedAE VAE and Native ISTFT vocoder preserved in full FP32/BF16 precision for lossless audio phase fidelity.
- **Audio Understanding Encoder**: Whisper / Conv1D acoustic encoder preserved in high precision.
- **Audio Generation Head**: 11-layer Flow Matching DiT.

---

## Quick Start

### 1. Installation

```bash
git clone https://github.com/vanch007/mlx-FireRedAudio.git
cd mlx-FireRedAudio
uv venv .venv
source .venv/bin/activate
uv pip install -e .
```

### 2. Download Model

```bash
hf download vanch007/FireRedAudio-MLX-8bit --local-dir models/FireRedAudio-8bit
```

### 3. Run Inference (CLI & Python)

```python
from fireredaudio_mlx import FireRedAudioInference

engine = FireRedAudioInference(model_path="models/FireRedAudio-8bit")

# ASR Speech Transcription (RTF ~ 0.14)
res = engine.understand("assets/examples/asr_zh_fleurs.wav", task="asr")
print("Transcript:", res.answer)

# Zero-shot TTS Voice Cloning (RTF ~ 0.75)
audio_res = engine.tts(
    prompt_text="收到你的来信，我很高兴。",
    prompt_audio="assets/examples/tts_zh_prompt.wav",
    target_text="你好，欢迎体验 FireRedAudio 8-bit MLX 实时语音生成！",
)
```

### 4. Launch WebUI Studio

```bash
python run_webui.py --model models/FireRedAudio-8bit
```
Open your browser at **http://127.0.0.1:7860**.

---

## Citation & References

- FireRedAudio Official: [FireRedTeam/FireRedAudio](https://github.com/FireRedTeam/FireRedAudio)
- Apple MLX: [MLX Framework](https://github.com/ml-explore/mlx)
- MLX Port & WebUI Studio: [vanch007/mlx-FireRedAudio](https://github.com/vanch007/mlx-FireRedAudio)