SOV-Transcribe-1 (MLX, 4-bit)

Salesforce SOV-Transcribe-1 for Apple Silicon: a Whisper large-v3 audio encoder and a Gemma 4 26B-A4B mixture-of-experts decoder that transcribe English, French, German, Spanish, Italian, Portuguese and Dutch recordings of up to 30 minutes.

The decoder is quantized to 4 bits (affine, group size 64). The audio encoder and projector stay in BF16 and the expert routers in float32. Transcribing needs about 16 GB of memory.

Conversion

Converted from the official BF16 checkpoint at revision 505d8fae243924c0d8acdc622f7f0abab344eba3 with mlx-audio at commit 19c519a3ed4581e82217fef8c492deb3e3b9dead (sanitize, the model's quantization predicate, 4 bits, group size 64). The weight files are byte-identical to nativ-community/SOV-Transcribe-1-4bit at revision 8a2f46d85735e55c0b6d1c8d882c570ed20d4d5c.

Usage

Python, with mlx-audio from main (SOV-Transcribe support is not in a release yet):

from mlx_audio.stt import load

model = load("TypeWhisper/SOV-Transcribe-1-4bit")
result = model.generate(
    "audio.wav",
    prompt="Transcribe the German audio into text, ensuring all punctuation marks are included.",
)
print(result.text)

Swift, with SOVTranscribeModel from mlx-audio-swift:

let model = try await SOVTranscribeModel.fromPretrained("TypeWhisper/SOV-Transcribe-1-4bit")
let result = model.generate(
    audio: audio,  // mono, 16 kHz
    prompt: "Transcribe the German audio into text, ensuring all punctuation marks are included."
)

The model card recommends these instructions:

  • Known language: Transcribe the {language} audio into text, ensuring all punctuation marks are included.
  • Unknown language: Transcribe the audio verbatim in the language's standard writing system, using punctuation and capitalization. Output only the transcript.

This checkpoint powers the SOV-Transcribe plugin of TypeWhisper for Macs with 32 GB of memory or more.

License

The model and the Salesforce reference code are licensed under Apache 2.0.

Downloads last month
70
Safetensors
Model size
26B params
Tensor type
BF16
Β·
U32
Β·
F32
Β·
MLX
Hardware compatibility
Log In to add your hardware

4-bit

Inference Providers NEW
This model isn't deployed by any Inference Provider. πŸ™‹ Ask for provider support

Model tree for TypeWhisper/SOV-Transcribe-1-4bit