SenseVoiceSmall OpenVINO INT8

SenseVoiceSmall converted to OpenVINO IR with INT8 weights. The repo holds two exports for two different inference pipelines:

Folder Export Runs with
genai-pipeline/ optimum-intel PR #2038 (optimum-cli export openvino, INT8 weights) openvino_genai.ASRPipeline (OpenVINO GenAI PR #4592)
reference-pipeline/ Earlier export from sensevoice-ov-support That repo's ov_infer.py. Not loadable by ASRPipeline (no config.json).

genai-pipeline

hf download yangsu0423/SenseVoiceSmall-OV-int8 --include "genai-pipeline/*" --local-dir SenseVoiceSmall-OV-int8
import librosa
import openvino_genai

pipe = openvino_genai.ASRPipeline("SenseVoiceSmall-OV-int8/genai-pipeline", "CPU")
config = pipe.get_generation_config()
config.language = "auto"  # auto / zh / en / yue / ja / ko
config.use_itn = True      # punctuation and inverse text normalization

audio, _ = librosa.load("audio.wav", sr=16000)
result = pipe.generate(audio.tolist(), config)
print(result, result.languages)

ASRPipeline for SenseVoice is not in a released OpenVINO GenAI yet; it needs an OpenVINO GenAI build that includes PR #4592.

Downloads last month

-

Downloads are not tracked for this model. How to track
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for yangsu0423/SenseVoiceSmall-OV-int8

Finetuned
(30)
this model