--- language: - he license: mit library_name: gigaam pipeline_tag: automatic-speech-recognition tags: - automatic-speech-recognition - hebrew - ctc - conformer - gigaam - speech datasets: - ivrit-ai/crowd-transcribe-v5 - SLPRL-HUJI/HebDB metrics: - wer - cer base_model: salute-developers/GigaAM --- # GigaAM-He — Hebrew speech recognition, 220M, CTC A Hebrew recogniser built by adapting GigaAM's multilingual SSL encoder. It is **not** the most accurate Hebrew model available — Whisper large-v3-turbo from ivrit.ai is better on accuracy. It is roughly **a quarter of the size and four to five times faster**, which is the point: it was built for real-time phone calls, where a 122 ms recognition step is felt and a 27 ms one is not. Trained on openly-licensed Hebrew speech. Weights are MIT, like the base model. ## Accuracy against ivrit.ai Whisper Both models scored through the same harness and the same text normalisation, so Whisper is not penalised for emitting punctuation the task does not ask for. Measured 2026-08-24/25. | eval set | GigaAM-He (220M) | ivrit Whisper large-v3-turbo (809M) | |---|---|---| | `crowd-transcribe-v5`, 1000 clips (1.32 h) | **13.60% WER** / 5.67% CER | **9.34% WER** / 4.73% CER | | `eval-whatsapp`, held out (1.18 h) | **14.71% WER** / 5.45% CER | **7.10% WER** / 3.29% CER | Whisper wins on accuracy by 4.3 points in-domain and 7.6 off-domain. Reported plainly because it is the number that should decide whether this model suits you: if accuracy is what matters and latency is not, use Whisper. **On contamination:** ivrit.ai trained Whisper on ivrit.ai data, so the `crowd-transcribe-v5` test split could in principle have leaked into it. The evidence argues against it — Whisper scores *worse* there (9.34%) than on the definitely-held-out `eval-whatsapp` (7.10%), and leakage would push the other way. Treat 9.34% as a fair bar. This model trained only on the crowd-v5 *train* split, so that val set is clean for it either way. **Caveat on eval-whatsapp:** 40 of its 54 clips exceed 20 s, and this model is trained on segments up to 20 s. The number above uses the segmenter in `longform.py`; a different chunking strategy will move it. ## Speed Utterance-length audio (2–6 s — what one turn of a phone call looks like), same harness, same clips. Whisper measured on an RTX 5080, GigaAM-He on a *weaker* 5060 Ti. | | GigaAM-He (220M) | ivrit Whisper (809M) | |---|---|---| | p50 | **27 ms** | 122 ms | | p90 | **28 ms** | 144 ms | | p50 @ 4 concurrent | **103 ms** | 466 ms | | throughput @ 4 | **135× realtime** | 29× realtime | 4.5× lower median latency and 5× lower p90, on slower hardware. ## Usage ### OpenAI-compatible API `server.py` exposes `/v1/audio/transcriptions` with OpenAI's request and response shape, so any client that talks to Whisper's API talks to this — point its base URL at the server and leave the rest alone. ```bash pip install fastapi uvicorn soundfile torch torchaudio numpy pip install git+https://github.com/salute-developers/GigaAM.git huggingface-cli download asfberlin/fast-hebrew-asr --local-dir fast-hebrew-asr cd fast-hebrew-asr && python server.py # listens on :6002 ``` ```bash curl -s http://localhost:6002/v1/audio/transcriptions \ -F file=@clip.wav | jq -r .text ``` ```python from openai import OpenAI client = OpenAI(base_url="http://localhost:6002/v1", api_key="not-needed") with open("clip.wav", "rb") as f: print(client.audio.transcriptions.create(model="gigaam-he", file=f).text) ``` `model` and `language` are accepted and ignored — this server hosts one Hebrew model, and rejecting the fields would break clients that always send them. Environment: `HE_CHECKPOINT`, `HE_PORT` (6002), `HE_DEVICE`, `HE_BATCH` (8), `HE_MAX_LEN` (18.0). ### Directly ```bash python inference.py clip.wav ``` ```python import gigaam model = gigaam.load_model("gigaam-he-twostage.ckpt", device="cuda") print(model.transcribe("clip.wav").transcription) ``` Audio at any sample rate works — the server resamples with `torchaudio.functional.resample`. Do not resample with `np.interp`: linear interpolation is not an anti-aliasing filter and folds everything above 8 kHz back into the band. ## How it was trained Two stages, and the sequencing mattered: 1. **937 h of HebDB** (weak labels — ASR output with alignment confidence), 4 epochs, lr 1e-4. Broad exposure to spontaneous Hebrew across six sources. 2. **257 h of ivrit.ai crowd-transcribe-v5** (human labels), 4 epochs, lr 3e-5. The clean data gets the last word. Trained on one RTX 4090. Mixing the two corpora 3:1 in a single run instead scored worse (14.45% / 15.89%): machine labels outvote human ones and the model inherits the weak labeller's systematic errors as if they were truth. Two-stage gave 13.60% / 14.71%, and helped generalisation more than in-domain accuracy — the off-domain gap narrowed from 1.44 to 1.11 points. **Scaling, measured rather than hoped:** 3.6× the data bought 0.85 WER points. Extrapolating honestly, another 4× (Knesset's 4,545 h) is worth perhaps 1–1.5 more points — around 12–12.5%, not the 8–10% first projected. **Parity with Whisper is probably not reachable by adding Hebrew data to a 220M CTC model.** The levers that would close it — a 600M backbone, or an RNN-T head — both cost the latency advantage that is the entire reason to prefer this one. ## Limitations - **Accuracy is 4–8 points behind Whisper.** Choose accordingly. - **Segments up to 20 s.** Longer audio must be chunked; `longform.py` does it with energy-based segmentation. - **Wideband, not telephony.** Every public Hebrew corpus is podcast, broadcast, lecture or parliament audio. On 8 kHz G.711 the WER rises about 1.6 points. A telephony-adapted variant exists internally (band-limited augmentation more than halves that penalty); it is not in this repository. - **Character CTC, no punctuation or casing**, and no timestamps. - **Off-domain it degrades** (13.60 → 14.71) where Whisper improves (9.34 → 7.10). That is a data-diversity deficit, not an architecture limit: one training domain against Whisper's thousands of hours. ## Licence and attribution MIT, following the base model. - **Base model:** [GigaAM](https://github.com/salute-developers/GigaAM) v2 `multilingual_ssl` — MIT, © 2024 GigaChat Team. The encoder is pretrained HuBERT-style on 2M hours across 70+ languages, none of them Hebrew. - **[SLPRL-HUJI/HebDB](https://huggingface.co/datasets/SLPRL-HUJI/HebDB)** — CC-BY-4.0. Please keep this attribution in derivative work. - **[ivrit-ai/crowd-transcribe-v5](https://huggingface.co/datasets/ivrit-ai/crowd-transcribe-v5)** — licence permits commercial training. No dataset with a non-commercial or unstated licence was used.