abedbanna/Sygma-ASR-Arabic

Sygma ASR Arabic

Paper: Sygma ASR Arabic: Improving Dialectal Speech Recognition through Length-Corrected N-Best Decoding, by Abedal-kareem Al-banna and Basel A. Al-banna (2026), preprint, DOI 10.13140/RG.2.2.33818.20164. Code and test audio: https://github.com/abedbanna/Sygma-ASR-Arabic · Demo: https://huggingface.co/spaces/abedbanna/Sygma-ASR-Arabic-demo

Sygma ASR Arabic is a decoding method for the open model CohereLabs/cohere-transcribe-arabic-07-2026. It lowers the average word error rate (WER) on the Open Universal Arabic ASR Leaderboard test sets from 24.16% to 23.04% (measured in our reproduction of the leaderboard pipeline) without retraining and without using any leaderboard training or test data for tuning.

The acoustic model and its 2.06 B parameters are Cohere's, used unchanged under Apache-2.0. Sygma contributes the decoding:

1. 8-best beam search with a word-count bonus (β = 1.0, length penalty 0). This corrects beam search's bias toward short, word-dropping hypotheses.
2. A reference-free breakdown detector that discards degenerate hypotheses (runaway word loops, symbol runs such as ____ or ****).
3. Chunked decoding for clips over 35 s, using the base model's own transcription routine.

β was tuned only on non-test Saudi audio (ahmedsamirtarjama/youtube_najdi, 600 clips, plus 8 HuggingPanda/Saudi_Podcasts_ASR clips).

Results

Same pipeline for every row: audio rebuilt from public mirrors of the six leaderboard test sets, scored with the leaderboard's own normalization (eval.py), corpus-level WER %.

Test set	Cohere greedy (base)	Sygma ASR Arabic	Audar-ASR-V1-Turbo*
SADA	36.57	33.60	28.24
Common Voice 18	5.41	5.26	8.13
MASC clean	16.33	15.95	17.20
MASC noisy	25.78	25.13	28.49
MGB-2	15.19	13.56	12.03
Casablanca	45.70	44.75	50.18
Average	24.16	23.04	24.05

*Audar-ASR-V1-Turbo (current leaderboard #1) was run in our pipeline with its model card's transformers recipe (bf16, greedy, batch 16), with the same breakdown protection applied to its 90 looping outputs. Without that protection it averages 26.37. Its official leaderboard average is 23.17. On loop-free sets our pipeline reproduces its official scores within 0.5 WER, but our Casablanca (50.18 vs 47.02) and MGB-2 (12.03 vs 11.08) remain above official, and batching costs it about 0.6 WER. These are not official leaderboard numbers.

Coverage: SADA 6,087 / 6,189 clips (98.4%; the rest could not be matched in the public mirror); all other sets 100%.

SADA breakdown (identical clips)

SADA group	Cohere greedy	Sygma	Audar-ASR-V1-Turbo*
Clean recording	30.09	28.47	23.80
Noisy recording	41.12	37.25	31.84
Background music	39.20	35.60	29.42
Several speakers	40.73	37.38	31.45
Female speakers	33.40	29.68	24.16
Male speakers	31.07	28.92	24.50

Sygma improves the base model in every group. With equal loop protection, Audar is better than both on Saudi speech in every group; Sygma's advantage over Audar comes from the other test sets (Casablanca, MASC, Common Voice).

Usage

# pip install "transformers==4.57.6" torch soundfile librosa huggingface_hub sentencepiece
from sygma_asr import SygmaASR

asr = SygmaASR(device="cuda")                 # mode="benchmark" (default); downloads the pinned Cohere revision
print(asr.transcribe(["clip.wav"]))

Modes

Mode	Word count for the bonus	All 8 hypotheses degenerate	Six-set average WER
mode="benchmark" (default)	words after the leaderboard normalization	re-rank all 8	23.04 (the paper's numbers)
mode="deployment"	words on punctuation-stripped raw text	keep the top beam hypothesis (or repair)	23.06

SygmaASR(..., mode="deployment", repair=True) adds a fallback re-decode when all 8 hypotheses are degenerate. It is not part of the benchmarked configuration.

Reproducing the paper

• Base model: CohereLabs/cohere-transcribe-arabic-07-2026 at revision c3e911b42149bf7a1e53d5cef9878aee87515a23 (pinned as BASE_REVISION); vendored model code in cohere_asr/.
• Environment: transformers 4.57.6; beam 8, length penalty 0, β = 1.0, at most 12 tokens per second + 20; bf16 autocast on GPU.
• Other pinned revisions used in the paper: audarai/Audar-ASR-V1-Turbo (remote code) 371428bea487c7aec82b27dc21f8d4324002e98e; Qwen/Qwen3-1.7B-Base ea980cb0a6c2ae4b936e82123acc929f1cec04c1.
• Two levels of reproduction: the released 8-best lists reproduce every table exactly (analysis reproduction); re-running inference on rebuilt audio can differ slightly with GPU type, batching and precision (end-to-end reproduction).
• Exact selection rules (given the released 8-best lists they reproduce all per-set WERs): reward only (the paper's primary result, 23.05) select_hypothesis(hyps, 1.0, "benchmark", use_filter=False); Sygma, reward + filter (23.04) select_hypothesis(hyps, 1.0, "benchmark", use_filter=True); Beam-8 top (24.21) select_hypothesis(hyps, 0.0, "benchmark", use_filter=False). SygmaASR(..., use_filter=False) decodes audio in the reward-only configuration.

Limitations

• On Saudi speech, Audar-ASR-V1-Turbo is clearly better in every condition we measured (SADA 28.24 vs 33.60). This method does not improve the base model's acoustic recognition.
• On very short clips (< 5 s) the word bonus can over-generate.
• When all 8 hypotheses loop, a loop can remain (16 of 6,087 SADA clips).
• Rare subtitle-style hallucinations (e.g. "اشتركوا بالقناة") inherited from the base model.
• Beam search is several times slower than greedy decoding.
• Two language-model rescorers (a 4-gram KenLM on 23 M Najdi sentences, and Qwen3-1.7B-Base) gave no gain after tuning on non-test audio, so neither is included.

License and attribution

Apache-2.0. The underlying model CohereLabs/cohere-transcribe-arabic-07-2026 is © Cohere and licensed under Apache-2.0. This repository adds decoding code only and does not imply endorsement by Cohere. See NOTICE.

Citation

@misc{albanna2026sygma,
  author    = {Al-banna, Abedal-kareem and Al-banna, Basel A.},
  title     = {{Sygma ASR Arabic}: Improving Dialectal Speech Recognition through Length-Corrected {N}-Best Decoding},
  year      = {2026},
  publisher = {ResearchGate},
  doi       = {10.13140/RG.2.2.33818.20164},
  note      = {Preprint}
}