---
license: other
license_name: funasr-model-license
license_link: https://github.com/modelscope/FunASR/blob/main/MODEL_LICENSE
library_name: transformers
base_model:
- FunAudioLLM/SenseVoiceSmall
tags:
- duplexjev
- audio
- asr-encoder
- sensevoice
---

# SenseVoice-Small-Encoder

The audio encoder of [FunAudioLLM/SenseVoiceSmall](https://huggingface.co/FunAudioLLM/SenseVoiceSmall), packaged as a
standalone `transformers` model (`trust_remote_code=True`) so that it can feed a frozen LLM through a DuplexJev
connector. It needs no FunASR installation. The weights are those of SenseVoiceSmall, unchanged: the SANM encoder
(50 + 20 blocks, d = 512), the task-query embeddings and the CMVN statistics from `am.mvn`. The CTC head is not included.

What the model does inside `forward`:

1. SenseVoice's frontend: Kaldi fbank (80 bins, 25 ms / 10 ms, Hamming, dither 0), LFR (7 frames stacked, stride 6),
   CMVN.
2. The four task queries SenseVoice prepends (language = auto, event, emotion, text-norm = woitn).
3. The encoder. The four query frames are dropped from the output, so one output frame covers 60 ms of audio.

The feature extractor only cuts 16 kHz audio into 10 ms frames of raw samples; the encoder computes the features. This
keeps the Ultravox processor's length arithmetic unchanged (`encoder_ds_factor = 6`).

Port check: greedy CTC decoding with SenseVoiceSmall's own CTC head on top of this encoder reproduces correct
transcripts of Chinese and English test clips, in fp32 and bf16.

Requires `torchaudio` (for the Kaldi fbank): `pip install torchaudio`.

Used by the `DuplexJev-B[-Para]-SenseVoice-Small-*` connectors; `Decider.from_pretrained("adventists-ai/DuplexJev-...")`
fetches it automatically. Source of the port: [github.com/adventists-ai/duplexjev](https://github.com/adventists-ai/duplexjev).

## License and attribution

The weights are **SenseVoiceSmall by Alibaba Group (Tongyi Lab / FunAudioLLM)**, redistributed under the
[FunASR Model Open Source License v1.1](https://github.com/modelscope/FunASR/blob/main/MODEL_LICENSE) (copy in
`MODEL_LICENSE`), which requires attributing the source and author and keeping the model name. The model is provided
by its authors for reference and learning; see the licence for the full terms. The SANM layer code is copied from
[FunASR](https://github.com/modelscope/FunASR) (MIT). The packaging code (feature extractor, wrapper) is Apache-2.0.
Please cite SenseVoice / FunAudioLLM when you use it.