SenseVoiceSmall OpenVINO (SDPA)
These are OpenVINO IRs of FunAudioLLM/SenseVoiceSmall with the SANM attention exported as ScaledDotProductAttention, from huggingface/optimum-intel#2122 (pending).
| Folder | Weights | ACTIVATIONS_SCALE_FACTOR |
Exported with |
|---|---|---|---|
sensevoice-small-sdpa-scale4-fp16-ov |
fp16 | 4.0 | optimum-intel main 883580d (includes #2118, merged) + #2122 |
sensevoice-small-sdpa-scale8-fp16-ov |
fp16 | 8.0 | optimum-intel ee87618 + #2122 + an earlier version of #2118 |
sensevoice-small-sdpa-scale8-int8-ov |
int8 | 8.0 | same as above |
sensevoice-small-sdpa-scale8-int4-ov |
int4 | 8.0 | same as above |
- Use
sensevoice-small-sdpa-scale4-fp16-ovfor new tests. It matches the merged #2118 value. - The scale8 folders are kept for reference. The fp16 weights of scale4 and scale8 are identical; only the
ACTIVATIONS_SCALE_FACTORruntime option in the IR differs. ACTIVATIONS_SCALE_FACTORis used only by the GPU plugin.
Export command:
optimum-cli export openvino -m FunAudioLLM/SenseVoiceSmall <output_dir> --weight-format {fp16|int8|int4} --trust-remote-code
- scale4: transformers 4.57.6, funasr 1.4.27, OpenVINO 2026.4.1.
- scale8: transformers 4.57.6, funasr 1.4.1.
Model tree for yangsu0423/SenseVoiceSmall-OV-Optimum-SDPA
Base model
FunAudioLLM/SenseVoiceSmall