README.md

---
base_model: CohereLabs/cohere-transcribe-arabic-07-2026
library_name: transformers
license: apache-2.0
language:
- ar
- en
pipeline_tag: automatic-speech-recognition
tags:
- automatic-speech-recognition
- arabic
- code-switching
- decoder-only-finetune
datasets:
- Ahmed1/cohere-asr-cs
---

# nsa01n/cohere-cs-decoder-full

Decoder-only fine-tune of
[`CohereLabs/cohere-transcribe-arabic-07-2026`](https://huggingface.co/CohereLabs/cohere-transcribe-arabic-07-2026)
for **Arabic/English code-switching**.

Arabic podcast speech routinely borrows English words, and a plain Arabic ASR
model transcribes them phonetically in Arabic script ("الويكند"). This model is
trained to emit them in Latin script instead ("ال weekend"), which is what
downstream consumers of the transcript actually want.

## What was trained

Variant **`full`** — every decoder layer were unfrozen. The Conformer audio
encoder is **frozen in every variant of this series**, including `full`; "full"
means the full *decoder*, not the full model.

| | |
|---|---|
| Decoder stack | `model.decoder` (8 layers) |
| Layers trained | [0, 1, 2, 3, 4, 5, 6, 7] |
| Encoder trainable params | 0 |
| Total params | 2066M |
| Trainable params | 170M (8.245%) |

## Training data

[`Ahmed1/cohere-asr-cs`](https://huggingface.co/datasets/Ahmed1/cohere-asr-cs) — VAD-segmented Arabic podcast clips
(16 kHz mono), transcribed with the base model and then rewritten so English
loanwords appear in Latin script. Splits are grouped by source episode, so no
episode appears in more than one split.

Trained on 1,900 clips, validated on
454.

## Hyperparameters

| parameter | value |
|---|---|
| gradient_accumulation_steps | 2 |
| learning_rate | 8e-05 |
| lr_scheduler_type | cosine |
| max_grad_norm | 1.0 |
| num_train_epochs | 6 |
| optim | adamw_bnb_8bit |
| per_device_train_batch_size | 16 |
| save_total_limit | 1 |
| warmup_ratio | 0.03 |
| weight_decay | 0.01 |
| effective batch size | 32 |
| precision | bf16 |
| seed | 42 |

Final metrics: `{"train_runtime": 419.5181, "train_samples_per_second": 27.174, "train_steps_per_second": 0.858, "total_flos": 5.372639220298678e+19, "train_loss": 0.10716057336992688, "epoch": 6.0}`

## Evaluation

### Code-switching (the task)

| metric | this model | base |
|---|---|---|
| clips it switched script on | 77/464 (16.6%) | 0/464 (0.0%) |
| clip-level recall | 0.520 | 0.000 |
| Latin word F1 | **0.510** | 0.000 |
| Latin word precision | 0.711 | 0.000 |
| Latin word recall | 0.398 | 0.000 |
| hallucinated Latin tokens | 35 | 0 |

Measured on 464 held-out clips, 125 of which contain English loanwords (216 Latin word tokens).

### Transcription (context, not the target)

| metric | value |
|---|---|
| WER | 0.051 |
| CER | 0.0272 |

> WER **rises** relative to the base model, and that is expected. The
> references are the base model's own transcriptions with loanwords
> rewritten into Latin script, so the base reproduces them almost exactly
> (~1% WER) while never code-switching. Every script change this model
> makes correctly still counts as edits against a base-shaped reference.


## Usage

```python
from transformers import AutoModelForSpeechSeq2Seq, AutoProcessor

processor = AutoProcessor.from_pretrained("nsa01n/cohere-cs-decoder-full")
model = AutoModelForSpeechSeq2Seq.from_pretrained("nsa01n/cohere-cs-decoder-full")
```

Feed it 16 kHz mono audio, ideally VAD-segmented to under 30 s per clip — that
is how it was trained and the base model caps at 35 s.

## Limitations

- Two podcast shows only (`sawalef-business`, `soqrat`); other domains and
  dialects are out of distribution.
- Training targets were machine-generated (base-model transcription + an LLM
  rewrite pass), not human-verified, so its ceiling is the base model's accuracy
  on this audio.
- The encoder was never trained, so acoustic robustness is unchanged from the
  base model.