| --- |
| language: |
| - he |
| license: mit |
| library_name: gigaam |
| pipeline_tag: automatic-speech-recognition |
| tags: |
| - automatic-speech-recognition |
| - hebrew |
| - ctc |
| - conformer |
| - gigaam |
| - speech |
| datasets: |
| - ivrit-ai/crowd-transcribe-v5 |
| - SLPRL-HUJI/HebDB |
| metrics: |
| - wer |
| - cer |
| base_model: salute-developers/GigaAM |
| --- |
| |
| # GigaAM-He — Hebrew speech recognition, 220M, CTC |
|
|
| A Hebrew recogniser built by adapting GigaAM's multilingual SSL encoder. It is |
| **not** the most accurate Hebrew model available — Whisper large-v3-turbo from |
| ivrit.ai is better on accuracy. It is roughly **a quarter of the size and four |
| to five times faster**, which is the point: it was built for real-time phone |
| calls, where a 122 ms recognition step is felt and a 27 ms one is not. |
|
|
| Trained on openly-licensed Hebrew speech. Weights are MIT, like the base model. |
|
|
| ## Accuracy against ivrit.ai Whisper |
|
|
| Both models scored through the same harness and the same text normalisation, so |
| Whisper is not penalised for emitting punctuation the task does not ask for. |
| Measured 2026-08-24/25. |
|
|
| | eval set | GigaAM-He (220M) | ivrit Whisper large-v3-turbo (809M) | |
| |---|---|---| |
| | `crowd-transcribe-v5`, 1000 clips (1.32 h) | **13.60% WER** / 5.67% CER | **9.34% WER** / 4.73% CER | |
| | `eval-whatsapp`, held out (1.18 h) | **14.71% WER** / 5.45% CER | **7.10% WER** / 3.29% CER | |
|
|
| Whisper wins on accuracy by 4.3 points in-domain and 7.6 off-domain. Reported |
| plainly because it is the number that should decide whether this model suits |
| you: if accuracy is what matters and latency is not, use Whisper. |
|
|
| **On contamination:** ivrit.ai trained Whisper on ivrit.ai data, so the |
| `crowd-transcribe-v5` test split could in principle have leaked into it. The |
| evidence argues against it — Whisper scores *worse* there (9.34%) than on the |
| definitely-held-out `eval-whatsapp` (7.10%), and leakage would push the other |
| way. Treat 9.34% as a fair bar. This model trained only on the crowd-v5 *train* |
| split, so that val set is clean for it either way. |
|
|
| **Caveat on eval-whatsapp:** 40 of its 54 clips exceed 20 s, and this model is |
| trained on segments up to 20 s. The number above uses the segmenter in |
| `longform.py`; a different chunking strategy will move it. |
|
|
| ## Speed |
|
|
| Utterance-length audio (2–6 s — what one turn of a phone call looks like), same |
| harness, same clips. Whisper measured on an RTX 5080, GigaAM-He on a *weaker* |
| 5060 Ti. |
|
|
| | | GigaAM-He (220M) | ivrit Whisper (809M) | |
| |---|---|---| |
| | p50 | **27 ms** | 122 ms | |
| | p90 | **28 ms** | 144 ms | |
| | p50 @ 4 concurrent | **103 ms** | 466 ms | |
| | throughput @ 4 | **135× realtime** | 29× realtime | |
|
|
| 4.5× lower median latency and 5× lower p90, on slower hardware. |
|
|
| ## Usage |
|
|
| ### OpenAI-compatible API |
|
|
| `server.py` exposes `/v1/audio/transcriptions` with OpenAI's request and |
| response shape, so any client that talks to Whisper's API talks to this — point |
| its base URL at the server and leave the rest alone. |
|
|
| ```bash |
| pip install fastapi uvicorn soundfile torch torchaudio numpy |
| pip install git+https://github.com/salute-developers/GigaAM.git |
| |
| huggingface-cli download asfberlin/fast-hebrew-asr --local-dir fast-hebrew-asr |
| cd fast-hebrew-asr && python server.py # listens on :6002 |
| ``` |
|
|
| ```bash |
| curl -s http://localhost:6002/v1/audio/transcriptions \ |
| -F file=@clip.wav | jq -r .text |
| ``` |
|
|
| ```python |
| from openai import OpenAI |
| |
| client = OpenAI(base_url="http://localhost:6002/v1", api_key="not-needed") |
| with open("clip.wav", "rb") as f: |
| print(client.audio.transcriptions.create(model="gigaam-he", file=f).text) |
| ``` |
|
|
| `model` and `language` are accepted and ignored — this server hosts one Hebrew |
| model, and rejecting the fields would break clients that always send them. |
|
|
| Environment: `HE_CHECKPOINT`, `HE_PORT` (6002), `HE_DEVICE`, `HE_BATCH` (8), |
| `HE_MAX_LEN` (18.0). |
|
|
| ### Directly |
|
|
| ```bash |
| python inference.py clip.wav |
| ``` |
|
|
| ```python |
| import gigaam |
| model = gigaam.load_model("gigaam-he-twostage.ckpt", device="cuda") |
| print(model.transcribe("clip.wav").transcription) |
| ``` |
|
|
| Audio at any sample rate works — the server resamples with |
| `torchaudio.functional.resample`. Do not resample with `np.interp`: linear |
| interpolation is not an anti-aliasing filter and folds everything above 8 kHz |
| back into the band. |
|
|
| ## How it was trained |
|
|
| Two stages, and the sequencing mattered: |
|
|
| 1. **937 h of HebDB** (weak labels — ASR output with alignment confidence), |
| 4 epochs, lr 1e-4. Broad exposure to spontaneous Hebrew across six sources. |
| 2. **257 h of ivrit.ai crowd-transcribe-v5** (human labels), 4 epochs, lr 3e-5. |
| The clean data gets the last word. |
|
|
| Trained on one RTX 4090. Mixing the two corpora 3:1 in a single run instead |
| scored worse (14.45% / 15.89%): machine labels outvote human ones and the model |
| inherits the weak labeller's systematic errors as if they were truth. Two-stage |
| gave 13.60% / 14.71%, and helped generalisation more than in-domain accuracy — |
| the off-domain gap narrowed from 1.44 to 1.11 points. |
|
|
| **Scaling, measured rather than hoped:** 3.6× the data bought 0.85 WER points. |
| Extrapolating honestly, another 4× (Knesset's 4,545 h) is worth perhaps 1–1.5 |
| more points — around 12–12.5%, not the 8–10% first projected. **Parity with |
| Whisper is probably not reachable by adding Hebrew data to a 220M CTC model.** |
| The levers that would close it — a 600M backbone, or an RNN-T head — both cost |
| the latency advantage that is the entire reason to prefer this one. |
|
|
| ## Limitations |
|
|
| - **Accuracy is 4–8 points behind Whisper.** Choose accordingly. |
| - **Segments up to 20 s.** Longer audio must be chunked; `longform.py` does it |
| with energy-based segmentation. |
| - **Wideband, not telephony.** Every public Hebrew corpus is podcast, |
| broadcast, lecture or parliament audio. On 8 kHz G.711 the WER rises about |
| 1.6 points. A telephony-adapted variant exists internally (band-limited |
| augmentation more than halves that penalty); it is not in this repository. |
| - **Character CTC, no punctuation or casing**, and no timestamps. |
| - **Off-domain it degrades** (13.60 → 14.71) where Whisper improves |
| (9.34 → 7.10). That is a data-diversity deficit, not an architecture limit: |
| one training domain against Whisper's thousands of hours. |
|
|
| ## Licence and attribution |
|
|
| MIT, following the base model. |
|
|
| - **Base model:** [GigaAM](https://github.com/salute-developers/GigaAM) v2 |
| `multilingual_ssl` — MIT, © 2024 GigaChat Team. The encoder is pretrained |
| HuBERT-style on 2M hours across 70+ languages, none of them Hebrew. |
| - **[SLPRL-HUJI/HebDB](https://huggingface.co/datasets/SLPRL-HUJI/HebDB)** — |
| CC-BY-4.0. Please keep this attribution in derivative work. |
| - **[ivrit-ai/crowd-transcribe-v5](https://huggingface.co/datasets/ivrit-ai/crowd-transcribe-v5)** |
| — licence permits commercial training. |
|
|
| No dataset with a non-commercial or unstated licence was used. |
|
|
