Automatic Speech Recognition
Transformers
Safetensors
fun_asr_nano
speech-recognition
asr
end-to-end
multilingual
arxiv:2509.12508
Instructions to use FunAudioLLM/Fun-ASR-Nano-2512-hf with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use FunAudioLLM/Fun-ASR-Nano-2512-hf with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("automatic-speech-recognition", model="FunAudioLLM/Fun-ASR-Nano-2512-hf")# pip install -U transformers accelerate # Load model directly from transformers import AutoProcessor, AutoModelForMultimodalLM processor = AutoProcessor.from_pretrained("FunAudioLLM/Fun-ASR-Nano-2512-hf") model = AutoModelForMultimodalLM.from_pretrained("FunAudioLLM/Fun-ASR-Nano-2512-hf", device_map="auto") - Notebooks
- Google Colab
- Kaggle
zhifu gao commited on
docs: lead with released native Transformers examples
Browse filesSigned-off-by: zhifu gao <zhifu.gzf@alibaba-inc.com>
README.md
CHANGED
|
@@ -12,18 +12,9 @@ tags:
|
|
| 12 |
- asr
|
| 13 |
- end-to-end
|
| 14 |
- multilingual
|
| 15 |
-
- arxiv:
|
| 16 |
---
|
| 17 |
|
| 18 |
-
<div align="center">
|
| 19 |
-
|
| 20 |
-
### ⭐ Powered by [FunASR](https://github.com/modelscope/FunASR) — please give us a GitHub Star!
|
| 21 |
-
|
| 22 |
-
This model is part of the **FunASR** ecosystem — one industrial-grade open-source toolkit for **ASR · VAD · punctuation · speaker diarization · emotion / event · LLM-ASR**. A Star really helps the project (and keeps you updated):
|
| 23 |
-
|
| 24 |
-
[**🌟 FunASR**](https://github.com/modelscope/FunASR) · [**🌟 SenseVoice**](https://github.com/QwenAudio/SenseVoice) · [**🌟 Fun-ASR**](https://github.com/QwenAudio/Fun-ASR) · [**🌟 FunClip**](https://github.com/modelscope/FunClip)
|
| 25 |
-
|
| 26 |
-
</div>
|
| 27 |
|
| 28 |
# Fun-ASR-Nano (Hugging Face Transformers)
|
| 29 |
|
|
@@ -31,26 +22,19 @@ This is the Hugging Face Transformers-compatible version of [Fun-ASR-Nano-2512](
|
|
| 31 |
|
| 32 |
Fun-ASR-Nano is an end-to-end speech recognition model by [FunAudioLLM](https://github.com/FunAudioLLM), trained on tens of millions of hours of real speech data. This checkpoint supports Chinese, English, and Japanese; its Chinese coverage includes 7 dialect groups and 26 regional accents. For 31-language recognition, use the separate [Fun-ASR-MLT-Nano-2512](https://huggingface.co/FunAudioLLM/Fun-ASR-MLT-Nano-2512) checkpoint.
|
| 33 |
|
| 34 |
-
## Capability boundary
|
| 35 |
-
|
| 36 |
-
This Transformers checkpoint is for generation-based transcription. It does not include the native CTC branch, so this export does not provide CTC-dependent timestamps or speaker diarization. Native model support does not add an HTTP server, realtime audio transport, or known-speaker identification.
|
| 37 |
-
|
| 38 |
-
Choose the artifact and runtime together: use the [original checkpoint](https://huggingface.co/FunAudioLLM/Fun-ASR-Nano-2512) for the FunASR toolkit, or the separate [Fun-ASR-Nano-2512-vllm checkpoint](https://huggingface.co/FunAudioLLM/Fun-ASR-Nano-2512-vllm) with its [native vLLM guide](https://github.com/modelscope/FunASR/blob/main/docs/vllm_official_native_validation.md). Check the selected pipeline's timestamp and speaker capabilities separately; changing a repository suffix does not convert weights or add those capabilities.
|
| 39 |
-
|
| 40 |
## Transformers quickstart
|
| 41 |
|
| 42 |
-
|
| 43 |
|
| 44 |
-
The following
|
| 45 |
|
| 46 |
```bash
|
| 47 |
python3.12 -m venv .venv-funasr-native
|
| 48 |
. .venv-funasr-native/bin/activate
|
| 49 |
python -m pip install --index-url https://download.pytorch.org/whl/cpu \
|
| 50 |
'torch==2.10.0+cpu' 'torchaudio==2.10.0+cpu'
|
| 51 |
-
python -m pip install '
|
| 52 |
-
'
|
| 53 |
-
'https://github.com/huggingface/transformers/archive/fc501343edfccdc840eb8594a6cafa8185c2de53.zip'
|
| 54 |
python -m pip check
|
| 55 |
```
|
| 56 |
|
|
@@ -62,6 +46,7 @@ The model revision below fixes the official weights, tokenizer and chat template
|
|
| 62 |
import torch
|
| 63 |
from transformers import AutoModelForSpeechSeq2Seq, AutoProcessor
|
| 64 |
|
|
|
|
| 65 |
model_id = "FunAudioLLM/Fun-ASR-Nano-2512-hf"
|
| 66 |
revision = "d93b302ee7fd505e1b3576120fc142fc6f7820e1"
|
| 67 |
processor = AutoProcessor.from_pretrained(
|
|
@@ -116,13 +101,24 @@ inputs = processor.apply_transcription_request(
|
|
| 116 |
|
| 117 |
Hotwords are hints, not an enforced vocabulary. In the paired Chinese sample check, the candidate keyword did not force the requested spelling into the output. Evaluate names and domain terms on representative recordings instead of assuming that accepted prompt fields improve accuracy.
|
| 118 |
|
| 119 |
-
`language` accepts Chinese, English, and Japanese as ISO codes, full English names, or the checkpoint Chinese names (`中文`, `英文`, `日文`). Keep batch inputs and decoded outputs in order. Reject empty batches
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 120 |
|
| 121 |
## Verification and further reading
|
| 122 |
|
| 123 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 124 |
|
| 125 |
-
- [Native Transformers guide in English](https://www.funasr.com/en/docs/native-transformers.html) and [Chinese](https://www.funasr.com/docs/native-transformers.html), including
|
| 126 |
- [Why checkpoint format matters in an application](https://www.funasr.com/en/blog/fun-asr-nano-transformers.html).
|
| 127 |
- [Merged Transformers implementation and model documentation](https://github.com/huggingface/transformers/blob/fc501343edfccdc840eb8594a6cafa8185c2de53/docs/source/en/model_doc/fun_asr_nano.md), for the native API and further implementation details.
|
| 128 |
- [Original model card](https://huggingface.co/FunAudioLLM/Fun-ASR-Nano-2512), for the model family and toolkit-specific usage.
|
|
|
|
| 12 |
- asr
|
| 13 |
- end-to-end
|
| 14 |
- multilingual
|
| 15 |
+
- arxiv:2509.12508
|
| 16 |
---
|
| 17 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 18 |
|
| 19 |
# Fun-ASR-Nano (Hugging Face Transformers)
|
| 20 |
|
|
|
|
| 22 |
|
| 23 |
Fun-ASR-Nano is an end-to-end speech recognition model by [FunAudioLLM](https://github.com/FunAudioLLM), trained on tens of millions of hours of real speech data. This checkpoint supports Chinese, English, and Japanese; its Chinese coverage includes 7 dialect groups and 26 regional accents. For 31-language recognition, use the separate [Fun-ASR-MLT-Nano-2512](https://huggingface.co/FunAudioLLM/Fun-ASR-MLT-Nano-2512) checkpoint.
|
| 24 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 25 |
## Transformers quickstart
|
| 26 |
|
| 27 |
+
Start with the released **Transformers 5.17.0** package and this native `-hf` checkpoint. A repository clone, custom model code and `trust_remote_code=True` are not required for this path. The original toolkit checkpoint and the vLLM/GGUF artifacts are different formats, not interchangeable model names.
|
| 28 |
|
| 29 |
+
The following setup targets **Linux x86-64, Python 3.12, CPU-only**. Create a new environment rather than upgrading an existing FunASR or vLLM service in place. GPU placement, attention backends and mixed precision require their own validation; CPU is an explicit choice for this example, not a limitation of the model architecture.
|
| 30 |
|
| 31 |
```bash
|
| 32 |
python3.12 -m venv .venv-funasr-native
|
| 33 |
. .venv-funasr-native/bin/activate
|
| 34 |
python -m pip install --index-url https://download.pytorch.org/whl/cpu \
|
| 35 |
'torch==2.10.0+cpu' 'torchaudio==2.10.0+cpu'
|
| 36 |
+
python -m pip install 'transformers==5.17.0' \
|
| 37 |
+
'numpy==1.26.4' 'librosa==0.11.0' 'soundfile==0.13.1'
|
|
|
|
| 38 |
python -m pip check
|
| 39 |
```
|
| 40 |
|
|
|
|
| 46 |
import torch
|
| 47 |
from transformers import AutoModelForSpeechSeq2Seq, AutoProcessor
|
| 48 |
|
| 49 |
+
torch.set_num_threads(4)
|
| 50 |
model_id = "FunAudioLLM/Fun-ASR-Nano-2512-hf"
|
| 51 |
revision = "d93b302ee7fd505e1b3576120fc142fc6f7820e1"
|
| 52 |
processor = AutoProcessor.from_pretrained(
|
|
|
|
| 101 |
|
| 102 |
Hotwords are hints, not an enforced vocabulary. In the paired Chinese sample check, the candidate keyword did not force the requested spelling into the output. Evaluate names and domain terms on representative recordings instead of assuming that accepted prompt fields improve accuracy.
|
| 103 |
|
| 104 |
+
`language` accepts Chinese, English, and Japanese as ISO codes, full English names, or the checkpoint Chinese names (`中文`, `英文`, `日文`). Keep batch inputs and decoded outputs in order; use `processor_kwargs={"text_kwargs": {"padding": True}}` together with the other processor options for different-length batches. Reject empty batches before calling the processor. An upstream `IndexError` for `audio=[]` was observed in the historical `fc501` source environment; that observation is not a new 5.17.0 test result.
|
| 105 |
+
|
| 106 |
+
## Capability boundary
|
| 107 |
+
|
| 108 |
+
This Transformers checkpoint is for generation-based transcription. It does not include the native CTC branch, so this export does not provide CTC-dependent timestamps or speaker diarization. Native model support does not add an HTTP server, realtime audio transport, or known-speaker identification.
|
| 109 |
+
|
| 110 |
+
Choose the artifact and runtime together: use the [original checkpoint](https://huggingface.co/FunAudioLLM/Fun-ASR-Nano-2512) for the FunASR toolkit, or the separate [Fun-ASR-Nano-2512-vllm checkpoint](https://huggingface.co/FunAudioLLM/Fun-ASR-Nano-2512-vllm) with its [native vLLM guide](https://github.com/modelscope/FunASR/blob/main/docs/vllm_official_native_validation.md). Check the selected pipeline's timestamp and speaker capabilities separately; changing a repository suffix does not convert weights or add those capabilities.
|
| 111 |
|
| 112 |
## Verification and further reading
|
| 113 |
|
| 114 |
+
- [Canonical runnable Transformers example](https://github.com/QwenAudio/Fun-ASR/tree/main/examples/transformers).
|
| 115 |
+
- [Companion notebook](https://github.com/QwenAudio/Fun-ASR/blob/main/examples/colab/fun_asr_nano_transformers.ipynb).
|
| 116 |
+
|
| 117 |
+
The 5.17.0 stable-wheel checks used the pinned official snapshot, CPU float32, torch 2.10.0+cpu and four Torch threads. The official English URL, a resampled Chinese waveform, a Chinese keyword request and a padded Chinese/English batch returned non-empty text and EOS before the 128-token limit. These checks cover only two short public samples and are not a CER/WER, keyword-benefit, memory-capacity, concurrency, Space-health or GPU benchmark. Cached official weights were rechecked: SHA256 `1bbb6dcc5d8b75084a399d48c4d4b0f3aa1d3f09f2616ae40ed2c4fba03d89c9`.
|
| 118 |
+
|
| 119 |
+
Historical reproducibility: native support was merged in [huggingface/transformers#46180](https://github.com/huggingface/transformers/pull/46180) at `fc501343edfccdc840eb8594a6cafa8185c2de53`. Earlier checks used a build from that source before 5.17.0 was released. The source archive is no longer the primary installation path.
|
| 120 |
|
| 121 |
+
- [Native Transformers guide in English](https://www.funasr.com/en/docs/native-transformers.html) and [Chinese](https://www.funasr.com/docs/native-transformers.html), including local audio, batches and exact verification boundaries.
|
| 122 |
- [Why checkpoint format matters in an application](https://www.funasr.com/en/blog/fun-asr-nano-transformers.html).
|
| 123 |
- [Merged Transformers implementation and model documentation](https://github.com/huggingface/transformers/blob/fc501343edfccdc840eb8594a6cafa8185c2de53/docs/source/en/model_doc/fun_asr_nano.md), for the native API and further implementation details.
|
| 124 |
- [Original model card](https://huggingface.co/FunAudioLLM/Fun-ASR-Nano-2512), for the model family and toolkit-specific usage.
|