SenseVoiceSmall OpenVINO (SDPA)

These are OpenVINO IRs of FunAudioLLM/SenseVoiceSmall with the SANM attention exported as ScaledDotProductAttention, from huggingface/optimum-intel#2122 (pending).

Folder Weights ACTIVATIONS_SCALE_FACTOR Exported with
sensevoice-small-sdpa-scale4-fp16-ov fp16 4.0 optimum-intel main 883580d (includes #2118, merged) + #2122
sensevoice-small-sdpa-scale8-fp16-ov fp16 8.0 optimum-intel ee87618 + #2122 + an earlier version of #2118
sensevoice-small-sdpa-scale8-int8-ov int8 8.0 same as above
sensevoice-small-sdpa-scale8-int4-ov int4 8.0 same as above
  • Use sensevoice-small-sdpa-scale4-fp16-ov for new tests. It matches the merged #2118 value.
  • The scale8 folders are kept for reference. The fp16 weights of scale4 and scale8 are identical; only the ACTIVATIONS_SCALE_FACTOR runtime option in the IR differs.
  • ACTIVATIONS_SCALE_FACTOR is used only by the GPU plugin.

Export command:

optimum-cli export openvino -m FunAudioLLM/SenseVoiceSmall <output_dir> --weight-format {fp16|int8|int4} --trust-remote-code
  • scale4: transformers 4.57.6, funasr 1.4.27, OpenVINO 2026.4.1.
  • scale8: transformers 4.57.6, funasr 1.4.1.
Downloads last month

-

Downloads are not tracked for this model. How to track
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for yangsu0423/SenseVoiceSmall-OV-Optimum-SDPA

Quantized
(24)
this model