model card (top)

README.md

@@ -0,0 +1,98 @@
+---
+base_model: CohereLabs/cohere-transcribe-arabic-07-2026
+library_name: transformers
+license: apache-2.0
+language:
+- ar
+- en
+pipeline_tag: automatic-speech-recognition
+tags:
+- automatic-speech-recognition
+- arabic
+- code-switching
+- decoder-only-finetune
+datasets:
+- Ahmed1/cohere-asr-cs
+---
+
+# nsa01n/cohere-cs-decoder-top
+
+Decoder-only fine-tune of
+[`CohereLabs/cohere-transcribe-arabic-07-2026`](https://huggingface.co/CohereLabs/cohere-transcribe-arabic-07-2026)
+for **Arabic/English code-switching**.
+
+Arabic podcast speech routinely borrows English words, and a plain Arabic ASR
+model transcribes them phonetically in Arabic script ("الويكند"). This model is
+trained to emit them in Latin script instead ("ال weekend"), which is what
+downstream consumers of the transcript actually want.
+
+## What was trained
+
+Variant **`top`** — the last third of the decoder layers were unfrozen. The Conformer audio
+encoder is **frozen in every variant of this series**, including `full`; "full"
+means the full *decoder*, not the full model.
+
+| | |
+|---|---|
+| Decoder stack | `model.decoder` (8 layers) |
+| Layers trained | [6, 7] |
+| Encoder trainable params | 0 |
+| Total params | 2066M |
+| Trainable params | 70M (3.366%) |
+
+## Training data
+
+[`Ahmed1/cohere-asr-cs`](https://huggingface.co/datasets/Ahmed1/cohere-asr-cs) — VAD-segmented Arabic podcast clips
+(16 kHz mono), transcribed with the base model and then rewritten so English
+loanwords appear in Latin script. Splits are grouped by source episode, so no
+episode appears in more than one split.
+
+Trained on 1,900 clips, validated on
+454.
+
+## Hyperparameters
+
+| parameter | value |
+|---|---|
+| gradient_accumulation_steps | 2 |
+| learning_rate | 8e-05 |
+| lr_scheduler_type | cosine |
+| max_grad_norm | 1.0 |
+| num_train_epochs | 6 |
+| optim | adamw_bnb_8bit |
+| per_device_train_batch_size | 16 |
+| save_total_limit | 1 |
+| warmup_ratio | 0.03 |
+| weight_decay | 0.01 |
+| effective batch size | 32 |
+| precision | bf16 |
+| seed | 42 |
+
+Final metrics: `{"train_runtime": 407.2033, "train_samples_per_second": 27.996, "train_steps_per_second": 0.884, "total_flos": 5.372639220298678e+19, "train_loss": 0.13692347241772546, "epoch": 6.0}`
+
+## Evaluation
+
+| — | Pending. |
+
+
+## Usage
+
+```python
+from transformers import AutoModelForSpeechSeq2Seq, AutoProcessor
+
+processor = AutoProcessor.from_pretrained("nsa01n/cohere-cs-decoder-top")
+model = AutoModelForSpeechSeq2Seq.from_pretrained("nsa01n/cohere-cs-decoder-top")
+```
+
+Feed it 16 kHz mono audio, ideally VAD-segmented to under 30 s per clip — that
+is how it was trained and the base model caps at 35 s.
+
+## Limitations
+
+- Two podcast shows only (`sawalef-business`, `soqrat`); other domains and
+  dialects are out of distribution.
+- Training targets were machine-generated (base-model transcription + an LLM
+  rewrite pass), not human-verified, so its ceiling is the base model's accuracy
+  on this audio.
+- The encoder was never trained, so acoustic robustness is unchanged from the
+  base model.