README.md

---
language:
- en
license: apache-2.0
library_name: transformers
pipeline_tag: automatic-speech-recognition
tags:
- automatic-speech-recognition
- speech
- audio
- whisper
- fine-tuning
- pytorch
- transformers
- huggingface
- deep-learning
- asr
---

# Whisper Fine-Tuned Automatic Speech Recognition Model

## Model Details

### Model Description

This model is a fine-tuned version of OpenAI's Whisper model for Automatic Speech Recognition (ASR). It has been trained on a custom speech dataset to improve transcription accuracy for domain-specific speech and conversational audio.

The model leverages the strong multilingual capabilities of Whisper while adapting to the characteristics of the target dataset through supervised fine-tuning.

- **Developed by:** Gad Amr , Osama Mohamed , Rojeh Wael
- **Funded by:** Self-funded Academic Graduation Project
- **Shared by:** Gad Amr
- **Model type:** Automatic Speech Recognition (ASR)
- **Base model:** OpenAI Whisper Medium 
- **Language(s):** Arabic / English
- **Framework:** Hugging Face Transformers
- **License:** Apache-2.0
- **Fine-tuned from model:** openai/whisper-Medium

---

## Direct Use

This model is designed for automatic speech recognition tasks, including:

- Speech-to-text transcription

---

## Downstream Use

The model can be integrated into:

- Meezan Graduation Project

---

## Out-of-Scope Use



# Bias, Risks, and Limitations

Like all speech recognition models, performance depends on:

- Audio quality
- Background noise
- Microphone quality
- Accent diversity
- Speaking speed
- Domain mismatch

Potential limitations include:

- Reduced accuracy on unseen accents but covered Egyptian ones .
- Difficulty with overlapping speakers.

---

## Recommendations


# How to Get Started with the Model

```python
from transformers import pipeline

pipe = pipeline(
    "automatic-speech-recognition",
    model="DBGad/Meezan-ASR"
)

result = pipe("sample.wav")

print(result["text"])
````

---



### Preprocessing

The following preprocessing steps were applied:

* Resampling audio to 16 kHz
* Removing corrupted samples
* Feature extraction using WhisperFeatureExtractor
* Text normalization
* Tokenization using WhisperTokenizer
* Dynamic padding with DataCollatorSpeechSeq2SeqWithPadding

---
### LoRA Configuration

| Parameter | Value |
|------------|-------|
| Rank (r) | 8 |
| Alpha | 16 |
| Dropout | 0.1 |

Target Modules:

- encoder_attn.q_proj
- encoder_attn.v_proj
- encoder_attn.out_proj

---

## Training Hyperparameters

| Hyperparameter | Value |
|----------------|-------|
| Base Model | openai/whisper-medium |
| Epochs | 12 |
| Batch Size (per device) | 16 |
| Gradient Accumulation | 8 |
| Effective Batch Size | 128 |
| Learning Rate | 5e-4 |
| Warmup Steps | 150 |
| Evaluation Strategy | Every 200 Steps |
| Checkpoint Saving | Every 200 Steps |
| Logging Steps | Every 50 Steps |
| Best Model Selection Metric | Word Error Rate (WER) |
| Load Best Model | Enabled |

---

## Optimization

The model was trained using:

- Mixed Precision (FP16/BF16)
- Gradient Checkpointing
- LoRA Parameter-Efficient Fine-Tuning

These techniques significantly reduced GPU memory consumption while maintaining strong model performance.

---

## Evaluation

Validation was performed every **200 training steps**.

The best checkpoint was automatically selected using the **lowest Word Error Rate (WER)**.

Evaluation Metric:

- Word Error Rate (WER)

---

## Hardware & Software

### Hardware

Training was performed on an NVIDIA GPU with CUDA support.

### Software

- Python
- PyTorch
- Hugging Face Transformers
- PEFT (LoRA)
- Datasets
- Evaluate
- Accelerate

---

## Model Objective

The objective of the model is to convert spoken English audio into accurate text transcriptions.

The model minimizes sequence-to-sequence cross-entropy loss while optimizing transcription quality measured by Word Error Rate (WER).


# Model Card Authors

* Gad Amr

---

# Model Card Contact

For questions, suggestions, or collaboration:

* GitHub: https://github.com/DBGad
* GitHub: https://github.com/Osamamoo
* GitHub: https://github.com/Rojeh-wael
* Gad Email : [gadelhaq.work@gmail.com](mailto:gadelhaq.work@gmail.com)
* Osama Email: [osama-m-ali@outlook.com](mailto:osama-m-ali@outlook.com)
* Rojeh Email: [rojehwael@yahoo.com](mailto:rojehwael@yahoo.com)

```
```