whisper-small-he-acft

Hebrew Whisper Small with ACFT (Audio-Context Fine-Tuning) for optimized short-audio performance, compatible with FUTO Keyboard and whisper.cpp.

Training

Two-stage pipeline:

1. Fine-tune: openai/whisper-small on ivrit-ai/whisper-training (~400h Hebrew) → amitkot/whisper-small-he
2. ACFT: Fine-tuned model on google/fleurs (he_il) using FUTO-aligned ACFT (partial encoder with truncated positional embeddings, 8 epochs, batch_size=1)

• Hardware: Apple M4 (MPS)
• Method: Distillation-based — teaches model to handle short audio contexts without repeating

Usage

from transformers import WhisperProcessor, WhisperForConditionalGeneration

processor = WhisperProcessor.from_pretrained("amitkot/whisper-small-he-acft")
model = WhisperForConditionalGeneration.from_pretrained("amitkot/whisper-small-he-acft")

For FUTO Keyboard / whisper.cpp, convert to ggml:

uv run python scripts/pipeline.py \
  --finetune-config configs/hebrew_small_finetune.yaml \
  --config configs/hebrew_small_acft.yaml

Training pipeline

Trained using whisper-acft-pipeline.

See also

• amitkot/whisper-small-he — Base fine-tuned model (before ACFT)
• amitkot/whisper-tiny-he-acft — Smaller/faster variant
• FUTO whisper-acft — ACFT method reference