SenseVoiceSmall OpenVINO INT8
SenseVoiceSmall converted to OpenVINO IR with INT8 weights. The repo holds two exports for two different inference pipelines:
| Folder | Export | Runs with |
|---|---|---|
genai-pipeline/ |
optimum-intel PR #2038 (optimum-cli export openvino, INT8 weights) |
openvino_genai.ASRPipeline (OpenVINO GenAI PR #4592) |
reference-pipeline/ |
Earlier export from sensevoice-ov-support | That repo's ov_infer.py. Not loadable by ASRPipeline (no config.json). |
genai-pipeline
hf download yangsu0423/SenseVoiceSmall-OV-int8 --include "genai-pipeline/*" --local-dir SenseVoiceSmall-OV-int8
import librosa
import openvino_genai
pipe = openvino_genai.ASRPipeline("SenseVoiceSmall-OV-int8/genai-pipeline", "CPU")
config = pipe.get_generation_config()
config.language = "auto" # auto / zh / en / yue / ja / ko
config.use_itn = True # punctuation and inverse text normalization
audio, _ = librosa.load("audio.wav", sr=16000)
result = pipe.generate(audio.tolist(), config)
print(result, result.languages)
ASRPipeline for SenseVoice is not in a released OpenVINO GenAI yet; it needs an OpenVINO GenAI build that includes PR #4592.
Model tree for yangsu0423/SenseVoiceSmall-OV-int8
Base model
FunAudioLLM/SenseVoiceSmall