zhifu gao commited on
Commit
a849aa4
·
unverified ·
1 Parent(s): b4d4bb9

docs: lead with released native Transformers examples

Browse files

Signed-off-by: zhifu gao <zhifu.gzf@alibaba-inc.com>

Files changed (1) hide show
  1. README.md +20 -24
README.md CHANGED
@@ -12,18 +12,9 @@ tags:
12
  - asr
13
  - end-to-end
14
  - multilingual
15
- - arxiv:2407.04051
16
  ---
17
 
18
- <div align="center">
19
-
20
- ### ⭐ Powered by [FunASR](https://github.com/modelscope/FunASR) — please give us a GitHub Star!
21
-
22
- This model is part of the **FunASR** ecosystem — one industrial-grade open-source toolkit for **ASR · VAD · punctuation · speaker diarization · emotion / event · LLM-ASR**. A Star really helps the project (and keeps you updated):
23
-
24
- [**🌟 FunASR**](https://github.com/modelscope/FunASR) · [**🌟 SenseVoice**](https://github.com/QwenAudio/SenseVoice) · [**🌟 Fun-ASR**](https://github.com/QwenAudio/Fun-ASR) · [**🌟 FunClip**](https://github.com/modelscope/FunClip)
25
-
26
- </div>
27
 
28
  # Fun-ASR-Nano (Hugging Face Transformers)
29
 
@@ -31,26 +22,19 @@ This is the Hugging Face Transformers-compatible version of [Fun-ASR-Nano-2512](
31
 
32
  Fun-ASR-Nano is an end-to-end speech recognition model by [FunAudioLLM](https://github.com/FunAudioLLM), trained on tens of millions of hours of real speech data. This checkpoint supports Chinese, English, and Japanese; its Chinese coverage includes 7 dialect groups and 26 regional accents. For 31-language recognition, use the separate [Fun-ASR-MLT-Nano-2512](https://huggingface.co/FunAudioLLM/Fun-ASR-MLT-Nano-2512) checkpoint.
33
 
34
- ## Capability boundary
35
-
36
- This Transformers checkpoint is for generation-based transcription. It does not include the native CTC branch, so this export does not provide CTC-dependent timestamps or speaker diarization. Native model support does not add an HTTP server, realtime audio transport, or known-speaker identification.
37
-
38
- Choose the artifact and runtime together: use the [original checkpoint](https://huggingface.co/FunAudioLLM/Fun-ASR-Nano-2512) for the FunASR toolkit, or the separate [Fun-ASR-Nano-2512-vllm checkpoint](https://huggingface.co/FunAudioLLM/Fun-ASR-Nano-2512-vllm) with its [native vLLM guide](https://github.com/modelscope/FunASR/blob/main/docs/vllm_official_native_validation.md). Check the selected pipeline's timestamp and speaker capabilities separately; changing a repository suffix does not convert weights or add those capabilities.
39
-
40
  ## Transformers quickstart
41
 
42
- Fun-ASR-Nano support was merged into Transformers on **2026-09-09** in [huggingface/transformers#46180](https://github.com/huggingface/transformers/pull/46180), at commit `fc501343edfccdc840eb8594a6cafa8185c2de53`. The latest stable **5.16.1** did not contain this model when checked that day. A merged PR is not a stable-release announcement; the source build's `5.17.0.dev0` version is not a promised release number or date. Use the tested source revision below, or verify that a later stable package actually contains `fun_asr_nano`.
43
 
44
- The following reproducible baseline is **Linux x86-64, Python 3.12, CPU-only**. Create a new environment rather than upgrading an existing FunASR or vLLM service in place. GPU placement, attention backends and mixed precision require their own validation; CPU is an explicit choice for this example, not a limitation of the model architecture.
45
 
46
  ```bash
47
  python3.12 -m venv .venv-funasr-native
48
  . .venv-funasr-native/bin/activate
49
  python -m pip install --index-url https://download.pytorch.org/whl/cpu \
50
  'torch==2.10.0+cpu' 'torchaudio==2.10.0+cpu'
51
- python -m pip install 'numpy==1.26.4' 'librosa==0.11.0' 'soundfile==0.13.1' \
52
- 'huggingface-hub==1.30.0' 'tokenizers==0.23.2' \
53
- 'https://github.com/huggingface/transformers/archive/fc501343edfccdc840eb8594a6cafa8185c2de53.zip'
54
  python -m pip check
55
  ```
56
 
@@ -62,6 +46,7 @@ The model revision below fixes the official weights, tokenizer and chat template
62
  import torch
63
  from transformers import AutoModelForSpeechSeq2Seq, AutoProcessor
64
 
 
65
  model_id = "FunAudioLLM/Fun-ASR-Nano-2512-hf"
66
  revision = "d93b302ee7fd505e1b3576120fc142fc6f7820e1"
67
  processor = AutoProcessor.from_pretrained(
@@ -116,13 +101,24 @@ inputs = processor.apply_transcription_request(
116
 
117
  Hotwords are hints, not an enforced vocabulary. In the paired Chinese sample check, the candidate keyword did not force the requested spelling into the output. Evaluate names and domain terms on representative recordings instead of assuming that accepted prompt fields improve accuracy.
118
 
119
- `language` accepts Chinese, English, and Japanese as ISO codes, full English names, or the checkpoint Chinese names (`中文`, `英文`, `日文`). Keep batch inputs and decoded outputs in order. Reject empty batches: at the pinned source revision, `audio=[]` raises an upstream `IndexError` before the intended validation. Successful requests do not erase that known limitation.
 
 
 
 
 
 
120
 
121
  ## Verification and further reading
122
 
123
- The native guide checks used the pinned official snapshot, CPU float32, torch/torchaudio 2.10.0+cpu and librosa 0.11.0 on an Intel Xeon Platinum 8480+ with four Torch threads. Single Chinese/English recordings, a Chinese keyword request and a two-record batch returned non-empty text and EOS before the token limit. These checks cover only two short public samples and are not a CER/WER, keyword-benefit, memory-capacity, concurrency or GPU benchmark. Cached official weight bytes were rechecked by SHA and size.
 
 
 
 
 
124
 
125
- - [Native Transformers guide in English](https://www.funasr.com/en/docs/native-transformers.html) and [Chinese](https://www.funasr.com/docs/native-transformers.html), including a local WAV recipe and exact verification boundaries.
126
  - [Why checkpoint format matters in an application](https://www.funasr.com/en/blog/fun-asr-nano-transformers.html).
127
  - [Merged Transformers implementation and model documentation](https://github.com/huggingface/transformers/blob/fc501343edfccdc840eb8594a6cafa8185c2de53/docs/source/en/model_doc/fun_asr_nano.md), for the native API and further implementation details.
128
  - [Original model card](https://huggingface.co/FunAudioLLM/Fun-ASR-Nano-2512), for the model family and toolkit-specific usage.
 
12
  - asr
13
  - end-to-end
14
  - multilingual
15
+ - arxiv:2509.12508
16
  ---
17
 
 
 
 
 
 
 
 
 
 
18
 
19
  # Fun-ASR-Nano (Hugging Face Transformers)
20
 
 
22
 
23
  Fun-ASR-Nano is an end-to-end speech recognition model by [FunAudioLLM](https://github.com/FunAudioLLM), trained on tens of millions of hours of real speech data. This checkpoint supports Chinese, English, and Japanese; its Chinese coverage includes 7 dialect groups and 26 regional accents. For 31-language recognition, use the separate [Fun-ASR-MLT-Nano-2512](https://huggingface.co/FunAudioLLM/Fun-ASR-MLT-Nano-2512) checkpoint.
24
 
 
 
 
 
 
 
25
  ## Transformers quickstart
26
 
27
+ Start with the released **Transformers 5.17.0** package and this native `-hf` checkpoint. A repository clone, custom model code and `trust_remote_code=True` are not required for this path. The original toolkit checkpoint and the vLLM/GGUF artifacts are different formats, not interchangeable model names.
28
 
29
+ The following setup targets **Linux x86-64, Python 3.12, CPU-only**. Create a new environment rather than upgrading an existing FunASR or vLLM service in place. GPU placement, attention backends and mixed precision require their own validation; CPU is an explicit choice for this example, not a limitation of the model architecture.
30
 
31
  ```bash
32
  python3.12 -m venv .venv-funasr-native
33
  . .venv-funasr-native/bin/activate
34
  python -m pip install --index-url https://download.pytorch.org/whl/cpu \
35
  'torch==2.10.0+cpu' 'torchaudio==2.10.0+cpu'
36
+ python -m pip install 'transformers==5.17.0' \
37
+ 'numpy==1.26.4' 'librosa==0.11.0' 'soundfile==0.13.1'
 
38
  python -m pip check
39
  ```
40
 
 
46
  import torch
47
  from transformers import AutoModelForSpeechSeq2Seq, AutoProcessor
48
 
49
+ torch.set_num_threads(4)
50
  model_id = "FunAudioLLM/Fun-ASR-Nano-2512-hf"
51
  revision = "d93b302ee7fd505e1b3576120fc142fc6f7820e1"
52
  processor = AutoProcessor.from_pretrained(
 
101
 
102
  Hotwords are hints, not an enforced vocabulary. In the paired Chinese sample check, the candidate keyword did not force the requested spelling into the output. Evaluate names and domain terms on representative recordings instead of assuming that accepted prompt fields improve accuracy.
103
 
104
+ `language` accepts Chinese, English, and Japanese as ISO codes, full English names, or the checkpoint Chinese names (`中文`, `英文`, `日文`). Keep batch inputs and decoded outputs in order; use `processor_kwargs={"text_kwargs": {"padding": True}}` together with the other processor options for different-length batches. Reject empty batches before calling the processor. An upstream `IndexError` for `audio=[]` was observed in the historical `fc501` source environment; that observation is not a new 5.17.0 test result.
105
+
106
+ ## Capability boundary
107
+
108
+ This Transformers checkpoint is for generation-based transcription. It does not include the native CTC branch, so this export does not provide CTC-dependent timestamps or speaker diarization. Native model support does not add an HTTP server, realtime audio transport, or known-speaker identification.
109
+
110
+ Choose the artifact and runtime together: use the [original checkpoint](https://huggingface.co/FunAudioLLM/Fun-ASR-Nano-2512) for the FunASR toolkit, or the separate [Fun-ASR-Nano-2512-vllm checkpoint](https://huggingface.co/FunAudioLLM/Fun-ASR-Nano-2512-vllm) with its [native vLLM guide](https://github.com/modelscope/FunASR/blob/main/docs/vllm_official_native_validation.md). Check the selected pipeline's timestamp and speaker capabilities separately; changing a repository suffix does not convert weights or add those capabilities.
111
 
112
  ## Verification and further reading
113
 
114
+ - [Canonical runnable Transformers example](https://github.com/QwenAudio/Fun-ASR/tree/main/examples/transformers).
115
+ - [Companion notebook](https://github.com/QwenAudio/Fun-ASR/blob/main/examples/colab/fun_asr_nano_transformers.ipynb).
116
+
117
+ The 5.17.0 stable-wheel checks used the pinned official snapshot, CPU float32, torch 2.10.0+cpu and four Torch threads. The official English URL, a resampled Chinese waveform, a Chinese keyword request and a padded Chinese/English batch returned non-empty text and EOS before the 128-token limit. These checks cover only two short public samples and are not a CER/WER, keyword-benefit, memory-capacity, concurrency, Space-health or GPU benchmark. Cached official weights were rechecked: SHA256 `1bbb6dcc5d8b75084a399d48c4d4b0f3aa1d3f09f2616ae40ed2c4fba03d89c9`.
118
+
119
+ Historical reproducibility: native support was merged in [huggingface/transformers#46180](https://github.com/huggingface/transformers/pull/46180) at `fc501343edfccdc840eb8594a6cafa8185c2de53`. Earlier checks used a build from that source before 5.17.0 was released. The source archive is no longer the primary installation path.
120
 
121
+ - [Native Transformers guide in English](https://www.funasr.com/en/docs/native-transformers.html) and [Chinese](https://www.funasr.com/docs/native-transformers.html), including local audio, batches and exact verification boundaries.
122
  - [Why checkpoint format matters in an application](https://www.funasr.com/en/blog/fun-asr-nano-transformers.html).
123
  - [Merged Transformers implementation and model documentation](https://github.com/huggingface/transformers/blob/fc501343edfccdc840eb8594a6cafa8185c2de53/docs/source/en/model_doc/fun_asr_nano.md), for the native API and further implementation details.
124
  - [Original model card](https://huggingface.co/FunAudioLLM/Fun-ASR-Nano-2512), for the model family and toolkit-specific usage.