Hugging Face's logo

Spaces:
neev2211
/
arabic-native-classifier
Sleeping

App Files Community
neev2211 commited on
Commit
67c0673
·
2 Parent(s): f4a0f5425f659b

Merge branch 'main' into main

Browse files
Files changed (33) hide show
  1. .gitignore +2 -1
  2. =0.2.0 +0 -0
  3. =1.4.0 +0 -0
  4. =17.3.0 +0 -0
  5. =2.5.0 +0 -0
  6. FLOW.md +63 -0
  7. PROJECT_ANALYSIS.md +113 -0
  8. README.md +184 -102
  9. TEST_MLP_VS_CNN.md +45 -0
  10. final_predictions.csv +30 -30
  11. manifests/cnn_training_manifest.csv +545 -0
  12. manifests/embedding_dim.txt +1 -0
  13. manifests/embedding_dim_cnn.txt +1 -0
  14. manifests/final_training_manifest.csv +0 -0
  15. manifests/final_training_manifest_english.csv +0 -0
  16. manifests/processed_manifest.csv +0 -0
  17. manifests/processed_manifest_english.csv +0 -0
  18. requirements.txt +2 -0
  19. scripts/__pycache__/model.cpython-310.pyc +0 -0
  20. scripts/compare_cnn_on_test.py +133 -0
  21. scripts/compare_models_on_test.py +245 -0
  22. scripts/extract_embeddings.py +189 -38
  23. scripts/extract_embeddings_english.py +0 -70
  24. scripts/extract_embeddings_for_CNN.py +163 -0
  25. scripts/load_data.py +17 -7
  26. scripts/load_data_english.py +0 -136
  27. scripts/make_predictions.py +91 -19
  28. scripts/make_predictions_english.py +0 -130
  29. scripts/model.py +58 -1
  30. scripts/train1DCNN.py +485 -0
  31. scripts/train_classifier.py +28 -9
  32. scripts/train_classifier_compare.py +525 -0
  33. scripts/train_classifier_english.py +0 -217
.gitignore CHANGED
@@ -21,7 +21,7 @@ manifests/Non Native/
21
  embeddings/
22
  embeddings_english/
23
  models/
24
-
25
  ############################
26
  # Python / venv
27
  ############################
@@ -38,3 +38,4 @@ __pycache__/
38
  .vscode/
39
  .idea/
40
 
 
 
21
  embeddings/
22
  embeddings_english/
23
  models/
24
+ embeddings_cnn/
25
  ############################
26
  # Python / venv
27
  ############################
 
38
  .vscode/
39
  .idea/
40
 
41
+ .cache/
=0.2.0 ADDED
File without changes
=1.4.0 ADDED
File without changes
=17.3.0 ADDED
File without changes
=2.5.0 ADDED
File without changes
FLOW.md ADDED
@@ -0,0 +1,63 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Arabic Native vs Non-Native — Pipeline Flow
2
+
3
+ ## Overview
4
+
5
+ ```
6
+ Audio URLs → Chunk (10s) → XLS-R (prosody layers) → Embeddings → MLP classifier → Native/Non-Native
7
+ ```
8
+
9
+ ---
10
+
11
+ ## 1. Load data (`load_data.py`)
12
+
13
+ - **Input**: `manifests/input_data.csv` (dp_id, audio_url, nativity_status, language)
14
+ - **Action**: Download audio, resample to 16 kHz, trim silence, split into 10s chunks
15
+ - **Output**: `data/processed_audio_chunks/*.wav`, `manifests/processed_manifest.csv`
16
+
17
+ ---
18
+
19
+ ## 2. Extract embeddings (`extract_embeddings.py`)
20
+
21
+ **Prosody emphasis**: Native vs Non-Native depends on **prosodic** cues:
22
+ - **Pitch** (intonation, stress)
23
+ - **Rhythm** (speech rate, syllable timing)
24
+ - **Fluency** (pauses, disfluencies)
25
+ - **Voice quality** (articulation)
26
+
27
+ The **final layer** of wav2vec encodes lexical content (words). We use **early-mid layers (4, 6, 8, 10)** instead — these encode prosody and speaker identity better.
28
+
29
+ - **Model**: `facebook/wav2vec2-large-xlsr-53`
30
+ - **Layers**: 4, 6, 8, 10 (prosody-focused)
31
+ - **Output**: Avg of these layers over time → 1024-dim vector, L2-normalized
32
+ - **Saves**: `embeddings/*.npy`, updates `manifests/final_training_manifest.csv`
33
+
34
+ ---
35
+
36
+ ## 3. Train classifier (`train_classifier.py`)
37
+
38
+ - **Input**: Embeddings + labels from `final_training_manifest.csv`
39
+ - **Split**: 80/20 by speaker (no speaker in both train and val)
40
+ - **Model**: MLP 1024 → 384 → 256 → 128 → 2 (BatchNorm, Dropout 0.5)
41
+ - **Regularization**: Dropout 0.5, weight decay 5e-4, early stopping (patience 15)
42
+ - **Output**: `models/nativity_model.pth`
43
+
44
+ **Overfitting**: Training logs `train_F1` and `val_F1`. If `train_F1 >> val_F1`, that indicates overfitting.
45
+
46
+ ---
47
+
48
+ ## 4. Predict (`make_predictions.py`)
49
+
50
+ - **Input**: `manifests/test_data.csv` (dp_id, audio_url)
51
+ - **Flow**: Download → chunk (10s) → extract prosody embedding → classifier → confidence-weighted vote over chunks
52
+ - **Output**: `final_predictions.csv` (dp_id, prediction, confidence_score)
53
+
54
+ ---
55
+
56
+ ## Data flow summary
57
+
58
+ | Stage | Input | Output | Emphasis |
59
+ |-------|-------|--------|----------|
60
+ | Load | URLs | 10s chunks | — |
61
+ | Extract | Chunks | 1024-dim embeddings | **Prosody layers 4,6,8,10** |
62
+ | Train | Embeddings | Classifier | Anti-overfitting (dropout, L2) |
63
+ | Predict | URLs | Native/Non-Native | Same prosody extraction |
PROJECT_ANALYSIS.md ADDED
@@ -0,0 +1,113 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Hackenza — Project analysis, changes done, and what to do next
2
+
3
+ ## 1. What the project is (current state)
4
+
5
+ **Goal:** Classify Arabic speech as **Native** or **Non-Native** (binary classifier).
6
+
7
+ **Pipeline:**
8
+ 1. **Load data** — Download audio from URLs → 16 kHz, 10s chunks → `data/processed_audio_chunks/`, `processed_manifest.csv`
9
+ 2. **Extract embeddings** — Wav2Vec2 XLS-R 53 layers 4,6,8,10 → **2051-d** vector (mean + std + F0) or **2090-d** with `--mfcc`
10
+ 3. **Train** — MLP + MLP-with-attention (+ optional LightGBM), threshold sweep on validation → `models/*.pth`, `best_threshold.txt`
11
+ 4. **Evaluate** — Run on test set, compare MLP / attention / ensemble, threshold sweep on test
12
+ 5. **Predict** — New URLs → chunks → embeddings → classifier → `final_predictions.csv`
13
+
14
+ **Main scripts (what you use day-to-day):**
15
+ | Script | Purpose |
16
+ |--------|---------|
17
+ | `load_data.py` | URLs → 10s WAV chunks + `processed_manifest.csv` |
18
+ | `extract_embeddings.py` | Chunks → 2051-d (or 2090-d) .npy + `final_training_manifest.csv` + `embedding_dim.txt` |
19
+ | `train_classifier_compare.py` | Train MLP + attention + (optional) LGB, save models + best threshold |
20
+ | `compare_models_on_test.py` | Test-set metrics + ensemble threshold sweep |
21
+ | `make_predictions.py` | Test CSV → `final_predictions.csv` |
22
+
23
+ **Other scripts:**
24
+ - `train_classifier.py` — Single MLP only (no attention, no LGB).
25
+ - `extract_embeddings_for_CNN.py`, `train1DCNN.py` — CNN path (optional/experimental).
26
+
27
+ ---
28
+
29
+ ## 2. What changes were done (summary)
30
+
31
+ ### Embedding pipeline (paper-driven)
32
+ - **Before:** Single 1024-d vector = mean over time of wav2vec layers 4,6,8,10.
33
+ - **After:**
34
+ - **Mean + std:** 1024 (mean) + 1024 (std over time) so the model sees temporal variance (paper: “deltas” help).
35
+ - **Explicit pitch (F0):** `librosa.yin` → mean, std, max over voiced frames → 3-d appended → **2051-d** total.
36
+ - **Optional MFCC:** `--mfcc` adds 39-d (13 MFCC + delta + delta2) → **2090-d** (from INDICON paper).
37
+ - **Where:** `extract_embeddings.py` (default 2051-d; `embedding_dim.txt` = 2051 or 2090).
38
+ `compare_models_on_test.py` and `make_predictions.py` build the same 2051/2090-d at test time.
39
+
40
+ ### Training / evaluation
41
+ - **Input dim:** Inferred from first `.npy` (2051 or 2090) in `train_classifier_compare.py` and `train_classifier.py` — no hardcoded 1024.
42
+ - **Regularization:** DROPOUT 0.4, WEIGHT_DECAY 5e-4, attention model `final_dropout=0.65`.
43
+ - **Threshold:** Validation sweep over [0.25, 0.28, 0.30, 0.32, 0.35, 0.40]; best saved to `models/best_threshold.txt`.
44
+ - **Test threshold sweep:** `compare_models_on_test.py` reports test F1 at 0.25, 0.28, 0.30, 0.32 and “Best on test: t=…”.
45
+
46
+ ### Robustness
47
+ - **Lazy Wav2Vec load** in `extract_embeddings.py` so importing it doesn’t load the big model (avoids 2x GPU OOM in `compare_models_on_test.py`).
48
+ - **Audio loading:** Skip URLs that return HTML or non-audio; no per-URL error spam; “Skipped N URL(s)” and “N predicted, M failed to load” in predictions.
49
+
50
+ ### Docs / repo
51
+ - **README.md** — Full pipeline, setup, commands, project layout, data format, training details, example run results.
52
+ - **FLOW.md** — Short flow (still says 1024-d in one place; main source of truth is README).
53
+ - **Removed:** `manifests/final_training_manifest.csv.bak`.
54
+
55
+ ---
56
+
57
+ ## 3. What to do next and how
58
+
59
+ ### Run the full pipeline (from scratch)
60
+ ```bash
61
+ cd /mnt/nvme2/slm/tut/Hackenza
62
+ source venv/bin/activate
63
+
64
+ # 1. Chunk audio (if not done)
65
+ python scripts/load_data.py
66
+
67
+ # 2. Extract 2051-d embeddings (or add --mfcc for 2090-d)
68
+ python scripts/extract_embeddings.py
69
+
70
+ # 3. Train and save best threshold
71
+ mkdir -p models
72
+ python scripts/train_classifier_compare.py
73
+
74
+ # 4. Evaluate on test set
75
+ python scripts/compare_models_on_test.py
76
+
77
+ # 5. Write predictions CSV
78
+ python scripts/make_predictions.py
79
+ ```
80
+
81
+ ### Try to improve results
82
+ 1. **Add LightGBM to ensemble:** `pip install lightgbm` then re-run step 3 and 4.
83
+ 2. **Try MFCC (2090-d):**
84
+ `python scripts/extract_embeddings.py --mfcc`
85
+ then re-run steps 3, 4, 5 (training/inference read `embedding_dim.txt` automatically).
86
+ 3. **More data:** Add more speakers / balanced Native vs Non-Native; re-run from step 1 or 2.
87
+
88
+ ### Push to GitHub
89
+ - Create repo `Hackenza_2026` under **Utkarsharma7** if needed.
90
+ - Use **HTTPS + PAT** (not SSH) from a **normal terminal** (outside Cursor) to avoid credential socket errors:
91
+ ```bash
92
+ git remote set-url origin https://github.com/Utkarsharma7/Hackenza_2026.git
93
+ git push -u origin main
94
+ ```
95
+ When prompted: Username = `Utkarsharma7`, Password = your GitHub PAT.
96
+
97
+ ### Optional cleanup
98
+ - **FLOW.md:** Update the “Output” row that still says “1024-dim” to “2051-dim (or 2090 with MFCC)” so it matches README.
99
+ - **CNN scripts:** If you don’t use `extract_embeddings_for_CNN.py` / `train1DCNN.py`, you can leave them as experimental or add a one-line note in README.
100
+
101
+ ---
102
+
103
+ ## 4. Quick reference — key numbers
104
+
105
+ | Item | Value |
106
+ |------|--------|
107
+ | Embedding dim (default) | 2051 (mean 1024 + std 1024 + pitch 3) |
108
+ | Embedding dim (with `--mfcc`) | 2090 |
109
+ | Wav2Vec layers | 4, 6, 8, 10 |
110
+ | Chunk length | 10 s @ 16 kHz |
111
+ | Best threshold (from your runs) | 0.25 (saved in `models/best_threshold.txt`) |
112
+ | Val F1 (example) | ~0.72–0.77 |
113
+ | Test F1 (example) | ~0.71–0.76 (MLP or MLP+attention best depending on run) |
README.md CHANGED
@@ -1,120 +1,116 @@
1
- # Hackenza — Native vs Non-Native Speaker Classifier
2
 
3
- ML pipelines that classify speech as **Native** or **Non-Native** for **Arabic** and **English**. Uses Wav2Vec 2.0 (XLSR-53) for embeddings and a small neural classifier on top.
 
 
4
 
5
  ## What the project does
6
 
7
- - **Load/preprocess** audio (from URLs for Arabic; from local folders for English)
8
- - **Split** audio into 10-second chunks at 16 kHz (any input sample rate supported)
9
- - **Extract embeddings** with Facebook's Wav2Vec 2.0 large XLSR-53
10
- - **Train** a classifier (Native vs Non-Native) per language
11
- - **Predict** on new audio (via URL or local file)
12
 
13
- **Arabic**: Training from `manifests/input_data.csv` (audio_url, dp_id, nativity_status, language).
14
- **English**: Training from local folders `manifests/Native/` and `manifests/Non Native/` (e.g. british, american, malayalam, telugu).
15
 
16
  ---
17
 
18
- ## Setup (venv)
 
 
19
 
20
  ```bash
21
- # From project root
22
  cd Hackenza
23
-
24
- # Create and activate venv
25
  python3 -m venv venv
26
- source venv/bin/activate # Linux/Mac
27
- # venv\Scripts\activate # Windows
28
 
29
- # Install dependencies
30
  pip install -r requirements.txt
31
  ```
32
 
 
 
33
  ---
34
 
35
- ## Run the pipeline
36
 
37
- All commands are run from the **Hackenza** directory (project root) with the venv activated.
38
 
39
- ```bash
40
- cd Hackenza
41
- source venv/bin/activate
42
- ```
43
-
44
- ### 1. Load data (download & chunk audio)
45
 
46
- Downloads audio from `manifests/input_data.csv` and saves 10s chunks to `data/processed_audio_chunks/`.
 
47
 
48
  ```bash
49
  python scripts/load_data.py
50
  ```
51
 
52
- ### 2. Extract embeddings
53
-
54
- Reads `manifests/processed_manifest.csv`, extracts Wav2Vec embeddings, saves to `embeddings/`, and writes `manifests/final_training_manifest.csv`.
55
-
56
- ```bash
57
- python scripts/extract_embeddings.py
58
- ```
59
 
60
- ### 3. Train classifier
61
 
62
- Trains on embeddings and saves the model to `models/nativity_model.pth`.
63
 
64
- ```bash
65
- mkdir -p models
66
- python scripts/train_classifier.py
67
- ```
68
 
69
- ### 4. Make predictions (inference)
70
 
71
- Predicts on `manifests/test_data.csv`. Output: `final_predictions.csv`.
 
72
 
73
  ```bash
74
- python scripts/make_predictions.py
 
 
75
  ```
76
 
77
  ---
78
 
79
- ## English pipeline (local folders)
80
-
81
- Data must be in `manifests/Native/` and `manifests/Non Native/` (e.g. `Native/british/speaker_01/*.wav`, `Non Native/malayalam/...`). Same steps as Arabic: chunk at 16 kHz / 10 s → embeddings → train → predict. Empty files are skipped; short clips (<10 s) are zero-padded to one chunk.
82
 
83
- ### 1. Chunk English audio
84
 
85
- Reads all audio under `manifests/Native` and `manifests/Non Native`, resamples to 16 kHz, splits into 10 s chunks. Output: `data/processed_audio_chunks_english/`, `manifests/processed_manifest_english.csv`.
 
86
 
87
  ```bash
88
- python scripts/load_data_english.py
 
89
  ```
90
 
91
- ### 2. Extract English embeddings
92
 
93
- ```bash
94
- python scripts/extract_embeddings_english.py
95
- ```
96
 
97
- Writes `embeddings_english/*.npy` and `manifests/final_training_manifest_english.csv`.
98
 
99
- ### 3. Train English classifier
100
 
101
- Train/val split is **by speaker** (folder path under `Native/` or `Non Native/`), so the same person is never in both train and val. The manifest must include a `speaker_id` column (added by step 1). Saves `models/nativity_model_english.pth`.
 
102
 
103
  ```bash
104
- mkdir -p models
105
- python scripts/train_classifier_english.py
106
  ```
107
 
108
- ### 4. English predictions
 
 
 
 
109
 
110
- Put test rows in `manifests/test_data_english.csv` (columns: `dp_id`, `audio_url` or `audio_path`). Then:
 
111
 
112
  ```bash
113
- python scripts/make_predictions_english.py
114
  ```
115
 
116
- Output: `final_predictions_english.csv`.
117
-
118
  ---
119
 
120
  ## Project layout
@@ -122,60 +118,146 @@ Output: `final_predictions_english.csv`.
122
  ```
123
  Hackenza/
124
  ├── scripts/
125
- │ ├── model.py # Shared classifier
126
- │ ├── load_data.py # Arabic: download + chunk
127
- │ ├── load_data_english.py # English: local folders → chunk
128
- │ ├── extract_embeddings.py # Arabic embeddings
129
- │ ├── extract_embeddings_english.py
130
- │ ├── train_classifier.py # Arabic model
131
- │ ├── train_classifier_english.py
132
- │ ├── make_predictions.py # Arabic inference
133
- │ └── make_predictions_english.py
134
  ├── manifests/
135
- │ ├── Native/ # English native (e.g. british/, american/)
136
- │ ├── Non Native/ # English non-native (e.g. malayalam/, telugu/)
137
- │ ├── input_data.csv # Arabic training URLs
138
- │ ├── processed_manifest.csv # Arabic chunks
139
- │ ├── processed_manifest_english.csv
140
- │ ├── final_training_manifest.csv
141
- │ ├── final_training_manifest_english.csv
142
- │ ├── test_data.csv
143
- │ └── test_data_english.csv
144
- ├── data/
145
- │ ├── processed_audio_chunks/ # Arabic
146
- │ └── processed_audio_chunks_english/
147
- ├── embeddings/ # Arabic
148
- ├── embeddings_english/
149
- ├── models/
150
- │ ├── nativity_model.pth # Arabic
151
- │ └── nativity_model_english.pth
152
  ├── requirements.txt
153
- └── README.md
 
154
  ```
155
 
156
  ---
157
 
158
  ## Data format
159
 
160
- **Input CSV** (`input_data.csv`, `test_data.csv`):
 
 
 
 
 
161
 
162
- | Column | Description |
163
- |----------------|-------------------------------------|
164
- | dp_id | Sample ID |
165
- | audio_url | URL to MP3/WAV |
166
- | nativity_status| `Native` or `Non-Native` (training) |
167
- | language | e.g. Arabic_QA, Arabic_SA |
 
 
168
 
169
  ---
170
 
171
- ## Training details (improved pipeline)
 
 
172
 
173
- - **Classifier**: MLP with BatchNorm and dropout (768 → 384 → 256 → 128 → 2). See `scripts/model.py`.
174
- - **Training**: AdamW, ReduceLROnPlateau on val F1, early stopping (patience 12), best model saved by validation F1. L2-normalized embeddings, class weights for imbalance.
175
- - **Inference**: Confidence-weighted voting across 10s chunks (no plain majority vote).
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
176
 
177
  ## Notes
178
 
179
- - **Paths**: Run all commands from **project root** (`Hackenza/`). Paths in scripts are relative to project root.
180
- - **GPU**: Uses CUDA if available.
181
- - **Wav2Vec**: First run downloads `facebook/wav2vec2-base-960h` (~360 MB).
 
 
 
1
+ # Hackenza — Arabic Native vs Non-Native Speaker Classifier
2
 
3
+ ML pipeline that classifies Arabic speech as **Native** or **Non-Native**. Uses Wav2Vec2 (XLS-R 53) with **prosody-focused** features (mean + std + pitch) and optional MFCC, plus MLP / MLP+attention and an ensemble.
4
+
5
+ ---
6
 
7
  ## What the project does
8
 
9
+ - **Load/preprocess** audio from URLs and split into 10-second chunks at 16 kHz
10
+ - **Extract embeddings**: XLS-R 53 layers 4, 6, 8, 10 → **mean** + **std** over time + **F0 (pitch)** stats → 2051-d (optional +39-d MFCC → 2090-d)
11
+ - **Train** MLP baseline, MLP with channel attention, and (optionally) LightGBM; threshold sweep on validation
12
+ - **Evaluate** on test set (MLP, attention, ensemble, threshold sweep)
13
+ - **Predict** on new audio (URLs → chunks → embeddings → confidence-weighted vote → Native/Non-Native)
14
 
15
+ **Task**: Binary classification (Native vs Non-Native) for Arabic L2 speech.
16
+ **Data**: Training from manifests (e.g. `input_data.csv` → `processed_manifest.csv` → `final_training_manifest.csv`); test from `test_data.csv`.
17
 
18
  ---
19
 
20
+ ## Setup
21
+
22
+ All commands are run from the **project root** (`Hackenza/`) with a virtualenv activated.
23
 
24
  ```bash
 
25
  cd Hackenza
 
 
26
  python3 -m venv venv
27
+ source venv/bin/activate # Linux/macOS
28
+ # Windows: venv\Scripts\activate
29
 
 
30
  pip install -r requirements.txt
31
  ```
32
 
33
+ **Optional**: Install LightGBM for 3-model ensemble: `pip install lightgbm` (already in `requirements.txt`).
34
+
35
  ---
36
 
37
+ ## Pipeline (main scripts)
38
 
39
+ ### 1. Load data — `scripts/load_data.py`
40
 
41
+ Downloads audio from `manifests/input_data.csv`, resamples to 16 kHz, splits into 10s chunks.
 
 
 
 
 
42
 
43
+ - **Input**: `manifests/input_data.csv` (dp_id, audio_url, nativity_status, language)
44
+ - **Output**: `data/processed_audio_chunks/*.wav`, `manifests/processed_manifest.csv`
45
 
46
  ```bash
47
  python scripts/load_data.py
48
  ```
49
 
50
+ ---
 
 
 
 
 
 
51
 
52
+ ### 2. Extract embeddings — `scripts/extract_embeddings.py`
53
 
54
+ Builds **2051-d** prosody embeddings (or **2090-d** with `--mfcc`):
55
 
56
+ - **Wav2Vec2**: XLS-R 53, layers 4, 6, 8, 10 (prosody/speaker-rich)
57
+ - **Temporal**: mean (1024) + std (1024) over time (no flatten-to-mean only)
58
+ - **Pitch**: F0 via `librosa.yin` → mean, std, max over voiced frames (3-d)
59
+ - **Optional**: `--mfcc` appends 39-d MFCC+delta+delta2 → 2090-d
60
 
61
+ Writes `manifests/embedding_dim.txt` (2051 or 2090) so training and inference use the same dimension.
62
 
63
+ - **Input**: `manifests/processed_manifest.csv`
64
+ - **Output**: `embeddings/*.npy`, `manifests/final_training_manifest.csv`, `manifests/embedding_dim.txt`
65
 
66
  ```bash
67
+ python scripts/extract_embeddings.py
68
+ # With MFCC:
69
+ python scripts/extract_embeddings.py --mfcc
70
  ```
71
 
72
  ---
73
 
74
+ ### 3. Train — `scripts/train_classifier_compare.py`
 
 
75
 
76
+ Trains **MLP**, **MLP + channel attention**, and (if LightGBM installed) **LightGBM**; does a threshold sweep on validation and saves the best threshold.
77
 
78
+ - **Input**: `manifests/final_training_manifest.csv` (embedding paths + labels); input dim inferred from first `.npy` (2051 or 2090)
79
+ - **Output**: `models/nativity_model.pth`, `models/nativity_model_attention.pth`, `models/best_threshold.txt`, optionally `models/lgbm_nativity.txt`
80
 
81
  ```bash
82
+ mkdir -p models
83
+ python scripts/train_classifier_compare.py
84
  ```
85
 
86
+ **Single-model training** (MLP only): `python scripts/train_classifier.py`
87
 
88
+ ---
 
 
89
 
90
+ ### 4. Evaluate on test set — `scripts/compare_models_on_test.py`
91
 
92
+ Runs MLP, MLP+attention, and ensemble on `manifests/test_data.csv` (requires `nativity_status`), and prints a threshold sweep on test for the ensemble.
93
 
94
+ - **Input**: `manifests/test_data.csv`, saved models, `manifests/embedding_dim.txt`
95
+ - **Output**: Console metrics and confusion matrices; no CSV output
96
 
97
  ```bash
98
+ python scripts/compare_models_on_test.py
 
99
  ```
100
 
101
+ ---
102
+
103
+ ### 5. Predict — `scripts/make_predictions.py`
104
+
105
+ Predicts Native/Non-Native for each row in `manifests/test_data.csv` and writes `final_predictions.csv`. Uses confidence-weighted voting over 10s chunks.
106
 
107
+ - **Input**: `manifests/test_data.csv` (dp_id, audio_url)
108
+ - **Output**: `final_predictions.csv` (dp_id, prediction, confidence_score)
109
 
110
  ```bash
111
+ python scripts/make_predictions.py
112
  ```
113
 
 
 
114
  ---
115
 
116
  ## Project layout
 
118
  ```
119
  Hackenza/
120
  ├── scripts/
121
+ │ ├── model.py # NativityClassifier, NativityClassifierWithAttention
122
+ │ ├── load_data.py # Download + chunk audio
123
+ │ ├── extract_embeddings.py # Wav2Vec2 + mean/std + F0 (+ optional MFCC)
124
+ │ ├── train_classifier.py # Single MLP training
125
+ │ ├── train_classifier_compare.py # MLP + attention + LGB + threshold sweep
126
+ │ ├── compare_models_on_test.py # Test-set evaluation + threshold sweep
127
+ │ └── make_predictions.py # Inference → final_predictions.csv
 
 
128
  ├── manifests/
129
+ │ ├── input_data.csv # Training URLs + labels
130
+ │ ├── processed_manifest.csv # Chunk paths after load_data
131
+ │ ├── final_training_manifest.csv # Chunk paths + embedding_path + labels
132
+ │ ├── embedding_dim.txt # 2051 or 2090 (set by extract_embeddings)
133
+ │ └── test_data.csv # Test URLs (+ optional nativity_status)
134
+ ├── data/processed_audio_chunks/ # 10s WAVs (gitignored)
135
+ ├── embeddings/ # .npy per chunk (gitignored)
136
+ ├── models/ # .pth, best_threshold.txt, lgbm (gitignored)
 
 
 
 
 
 
 
 
 
137
  ├── requirements.txt
138
+ ├─�� README.md
139
+ └── FLOW.md # Short pipeline flow
140
  ```
141
 
142
  ---
143
 
144
  ## Data format
145
 
146
+ | Column | Description |
147
+ |-----------------|--------------------------------------|
148
+ | dp_id | Sample/speaker ID |
149
+ | audio_url | URL to audio (WAV/MP3) |
150
+ | nativity_status | `Native` or `Non-Native` (train/test labels) |
151
+ | language | e.g. Arabic_QA, Arabic_SA |
152
 
153
+ ---
154
+
155
+ ## Training details
156
+
157
+ - **Embeddings**: 2051-d = wav2vec mean (1024) + wav2vec std (1024) + pitch mean/std/max (3). Optional MFCC → 2090-d.
158
+ - **Models**: MLP (2051→384→256→128→2), BatchNorm, dropout 0.4; attention variant adds SE-style channel attention and final_dropout 0.65.
159
+ - **Training**: AdamW (lr 1e-3, weight decay 5e-4), ReduceLROnPlateau on val F1, early stopping (patience 25). SMOTE on training set for class balance. Threshold sweep on validation over [0.25, 0.28, 0.30, 0.32, 0.35, 0.40].
160
+ - **Inference**: Per-file: mean over chunk probs (MLP, attention, optional LGB) vs best threshold; confidence-weighted voting in `make_predictions.py`.
161
 
162
  ---
163
 
164
+ ## Example run results
165
+
166
+ The following outputs were produced by running the main scripts (no code changes) from project root with the existing data and embeddings.
167
 
168
+ ### Training — `train_classifier_compare.py`
169
+
170
+ ```
171
+ Embedding dimension: 2051
172
+ SMOTE: 417 -> 546 samples (Non-Native oversampled)
173
+ ...
174
+ Threshold sweep on validation (ensemble):
175
+ t=0.25 F1=0.7250
176
+ t=0.28 F1=0.7100
177
+ t=0.30 F1=0.7202
178
+ t=0.32 F1=0.7047
179
+ t=0.35 F1=0.7149
180
+ t=0.40 F1=0.7090
181
+ Best threshold (val F1): 0.25 -> F1=0.7250 (saved to models/best_threshold.txt)
182
+
183
+ ============================================================
184
+ COMPARISON (validation set)
185
+ ============================================================
186
+ Model Accuracy F1 macro Precision Recall
187
+ ------------------------------------------------------------------------
188
+ MLP (baseline) 0.7667 0.7251 0.7156 0.7964
189
+ MLP + attention (prosody) 0.7556 0.7149 0.7083 0.7893
190
+ Ensemble (2 models, t=0.25) 0.7556 0.7250 0.7275 0.8250
191
+
192
+ Confusion matrices (rows=true, cols=pred), Native=0, Non-Native=1:
193
+ MLP (baseline):
194
+ [[52 18]
195
+ [ 3 17]]
196
+ MLP + attention (prosody):
197
+ [[51 19]
198
+ [ 3 17]]
199
+ Ensemble (2 models, t=0.25):
200
+ [[49 21]
201
+ [ 1 19]]
202
+
203
+ Best by F1: MLP (baseline) (F1=0.7251)
204
+ Saved: models/nativity_model.pth, models/nativity_model_attention.pth, models/best_threshold.txt
205
+ ```
206
+
207
+ ### Test evaluation — `compare_models_on_test.py`
208
+
209
+ ```
210
+ COMPARISON ON TEST DATA
211
+ ============================================================
212
+ Test samples: 30 (Native=0, Non-Native=1)
213
+ Model Accuracy F1 macro Precision Recall
214
+ ------------------------------------------------------------------------
215
+ MLP (baseline) 0.7333 0.7129 0.7083 0.7250
216
+ MLP + attention (prosody) 0.7667 0.7600 0.7667 0.8000
217
+ Ensemble (2 models, t=0.25) 0.7333 0.7129 0.7083 0.7250
218
+
219
+ Confusion matrices (rows=true, cols=pred):
220
+ MLP (baseline):
221
+ [[15 5]
222
+ [ 3 7]]
223
+ MLP + attention (prosody):
224
+ [[14 6]
225
+ [ 1 9]]
226
+ Ensemble (2 models, t=0.25):
227
+ [[15 5]
228
+ [ 3 7]]
229
+
230
+ Best on test (F1): MLP + attention (prosody) (F1=0.7600)
231
+
232
+ Ensemble threshold sweep on test:
233
+ t Accuracy F1 macro
234
+ 0.25 0.7333 0.7129
235
+ 0.28 0.7333 0.7129
236
+ 0.30 0.7333 0.7129
237
+ 0.32 0.7333 0.7129
238
+ Best on test: t=0.25 (F1=0.7129)
239
+ ```
240
+
241
+ (Some test URLs may be skipped with a message: "Skipped N URL(s) (non-audio or load failed).")
242
+
243
+ ### Predictions — `make_predictions.py`
244
+
245
+ ```
246
+ Predicting for 33 files (device=cuda)...
247
+ Done. Saved to final_predictions.csv (30 predicted, 3 failed to load)
248
+
249
+ Test metrics (from test_data.csv, using nativity_status):
250
+ Accuracy: 0.7333
251
+ F1 macro: 0.7129
252
+ ...
253
+ ```
254
+
255
+ ---
256
 
257
  ## Notes
258
 
259
+ - **Paths**: Run all commands from project root (`Hackenza/`).
260
+ - **GPU**: Uses CUDA if available; Wav2Vec2 and classifiers run on GPU when possible.
261
+ - **First run**: `extract_embeddings.py` and test/predict scripts download `facebook/wav2vec2-large-xlsr-53` (~1.2 GB) on first use.
262
+ - **LightGBM**: Optional; if not installed, ensemble uses only MLP + attention (2 models).
263
+ - **FLOW.md**: Short pipeline flow and data stages.
TEST_MLP_VS_CNN.md ADDED
@@ -0,0 +1,45 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Test-set comparison: MLP pipeline vs CNN pipeline
2
+
3
+ Same test set: `manifests/test_data.csv` with `nativity_status` (30 valid samples after skipping failed URLs).
4
+ MLP uses threshold **0.25** (validation best for MLP/ensemble). CNN uses threshold **0.55** (validation best for CNN).
5
+
6
+ ## Metrics on test data
7
+
8
+ | Model | Accuracy | F1 macro | Precision | Recall |
9
+ |-------|----------|----------|-----------|--------|
10
+ | **1D-CNN (sequence)** | 0.6667 | 0.6411 | 0.6389 | 0.6500 |
11
+ | **MLP (baseline)** | **0.7333** | **0.7129** | **0.7083** | **0.7250** |
12
+ | **MLP + attention (prosody)** | **0.7667** | **0.7600** | **0.7667** | **0.8000** |
13
+ | **Ensemble (MLP + attention, t=0.25)** | 0.7333 | 0.7129 | 0.7083 | 0.7250 |
14
+
15
+ ## Confusion matrices (rows = true, cols = predicted; Native=0, Non-Native=1)
16
+
17
+ **1D-CNN:**
18
+ ```
19
+ [[14 6]
20
+ [ 4 6]]
21
+ ```
22
+
23
+ **MLP (baseline):**
24
+ ```
25
+ [[15 5]
26
+ [ 3 7]]
27
+ ```
28
+
29
+ **MLP + attention (best on test):**
30
+ ```
31
+ [[14 6]
32
+ [ 1 9]]
33
+ ```
34
+
35
+ **Ensemble (2 models):**
36
+ ```
37
+ [[15 5]
38
+ [ 3 7]]
39
+ ```
40
+
41
+ ## Summary
42
+
43
+ - **Best on test:** **MLP + attention** (F1 0.76, accuracy 0.7667).
44
+ - **MLP pipeline** (baseline and attention) **outperforms the CNN pipeline** on this test set (F1 ~0.71–0.76 vs 0.64).
45
+ - CNN uses 2D sequence (wav2vec2-lv-60-espeak-cv-ft, 1024-d) and 1D conv; MLP uses 2051-d prosody (mean+std+pitch from XLS-R 53). The prosody feature set and/or model choice likely explains the gap.
final_predictions.csv CHANGED
@@ -1,34 +1,34 @@
1
  dp_id,prediction,confidence_score
2
- 569,Native,0.6419
3
- 626,Native,0.8127
4
- 539,Non-Native,0.827
5
- 357,Native,0.9967
6
  532,Error,0.0
7
- 452,Native,0.841
8
- 462,Native,0.9691
9
  479,Error,0.0
10
- 502,Non-Native,0.8167
11
- 601,Native,0.9353
12
- 288,Non-Native,0.7749
13
- 404,Non-Native,0.9181
14
  407,Error,0.0
15
- 510,Non-Native,0.7393
16
- 497,Native,0.9323
17
- 568,Native,0.8986
18
- 335,Native,0.9971
19
- 625,Non-Native,0.9806
20
- 643,Native,0.9871
21
- 689,Native,0.9171
22
- 545,Native,0.9922
23
- 504,Non-Native,0.8121
24
- 648,Non-Native,0.7097
25
- 599,Native,0.9627
26
- 367,Non-Native,0.893
27
- 517,Native,0.8102
28
- 322,Native,0.9688
29
- 460,Native,0.8268
30
- 466,Native,0.9731
31
- 583,Native,0.9795
32
- 573,Native,0.899
33
- 477,Native,0.9571
34
- 635,Non-Native,0.8198
 
1
  dp_id,prediction,confidence_score
2
+ 569,Native,0.836
3
+ 626,Native,0.9008
4
+ 539,Non-Native,0.9893
5
+ 357,Native,0.9997
6
  532,Error,0.0
7
+ 452,Non-Native,0.9991
8
+ 462,Native,0.9942
9
  479,Error,0.0
10
+ 502,Non-Native,0.9995
11
+ 601,Native,0.9942
12
+ 288,Non-Native,0.8521
13
+ 404,Non-Native,0.9806
14
  407,Error,0.0
15
+ 510,Native,0.8483
16
+ 497,Native,0.8419
17
+ 568,Native,0.9958
18
+ 335,Native,0.9991
19
+ 625,Non-Native,0.7752
20
+ 643,Native,0.9306
21
+ 689,Native,0.9862
22
+ 545,Native,0.9337
23
+ 504,Native,0.9658
24
+ 648,Non-Native,0.8739
25
+ 599,Native,0.9543
26
+ 367,Native,0.9493
27
+ 517,Non-Native,0.8393
28
+ 322,Native,0.9928
29
+ 460,Non-Native,0.8499
30
+ 466,Native,0.9986
31
+ 583,Non-Native,0.7177
32
+ 573,Non-Native,0.9865
33
+ 477,Native,0.9595
34
+ 635,Non-Native,0.8775
manifests/cnn_training_manifest.csv ADDED
@@ -0,0 +1,545 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ chunk_path,original_id,label,language,embedding_path
2
+ data/processed_audio_chunks\716_chunk0.wav,716,Non-Native,Arabic_QA,embeddings_cnn/716_chunk0.npy
3
+ data/processed_audio_chunks\716_chunk1.wav,716,Non-Native,Arabic_QA,embeddings_cnn/716_chunk1.npy
4
+ data/processed_audio_chunks\716_chunk2.wav,716,Non-Native,Arabic_QA,embeddings_cnn/716_chunk2.npy
5
+ data/processed_audio_chunks\716_chunk3.wav,716,Non-Native,Arabic_QA,embeddings_cnn/716_chunk3.npy
6
+ data/processed_audio_chunks\716_chunk4.wav,716,Non-Native,Arabic_QA,embeddings_cnn/716_chunk4.npy
7
+ data/processed_audio_chunks\716_chunk5.wav,716,Non-Native,Arabic_QA,embeddings_cnn/716_chunk5.npy
8
+ data/processed_audio_chunks\716_chunk6.wav,716,Non-Native,Arabic_QA,embeddings_cnn/716_chunk6.npy
9
+ data/processed_audio_chunks\668_chunk0.wav,668,Native,Arabic_SA,embeddings_cnn/668_chunk0.npy
10
+ data/processed_audio_chunks\668_chunk1.wav,668,Native,Arabic_SA,embeddings_cnn/668_chunk1.npy
11
+ data/processed_audio_chunks\668_chunk2.wav,668,Native,Arabic_SA,embeddings_cnn/668_chunk2.npy
12
+ data/processed_audio_chunks\408_chunk0.wav,408,Non-Native,Arabic_KW,embeddings_cnn/408_chunk0.npy
13
+ data/processed_audio_chunks\408_chunk1.wav,408,Non-Native,Arabic_KW,embeddings_cnn/408_chunk1.npy
14
+ data/processed_audio_chunks\408_chunk2.wav,408,Non-Native,Arabic_KW,embeddings_cnn/408_chunk2.npy
15
+ data/processed_audio_chunks\408_chunk3.wav,408,Non-Native,Arabic_KW,embeddings_cnn/408_chunk3.npy
16
+ data/processed_audio_chunks\408_chunk4.wav,408,Non-Native,Arabic_KW,embeddings_cnn/408_chunk4.npy
17
+ data/processed_audio_chunks\408_chunk5.wav,408,Non-Native,Arabic_KW,embeddings_cnn/408_chunk5.npy
18
+ data/processed_audio_chunks\408_chunk6.wav,408,Non-Native,Arabic_KW,embeddings_cnn/408_chunk6.npy
19
+ data/processed_audio_chunks\408_chunk7.wav,408,Non-Native,Arabic_KW,embeddings_cnn/408_chunk7.npy
20
+ data/processed_audio_chunks\408_chunk8.wav,408,Non-Native,Arabic_KW,embeddings_cnn/408_chunk8.npy
21
+ data/processed_audio_chunks\408_chunk9.wav,408,Non-Native,Arabic_KW,embeddings_cnn/408_chunk9.npy
22
+ data/processed_audio_chunks\408_chunk10.wav,408,Non-Native,Arabic_KW,embeddings_cnn/408_chunk10.npy
23
+ data/processed_audio_chunks\408_chunk11.wav,408,Non-Native,Arabic_KW,embeddings_cnn/408_chunk11.npy
24
+ data/processed_audio_chunks\408_chunk12.wav,408,Non-Native,Arabic_KW,embeddings_cnn/408_chunk12.npy
25
+ data/processed_audio_chunks\408_chunk13.wav,408,Non-Native,Arabic_KW,embeddings_cnn/408_chunk13.npy
26
+ data/processed_audio_chunks\408_chunk14.wav,408,Non-Native,Arabic_KW,embeddings_cnn/408_chunk14.npy
27
+ data/processed_audio_chunks\408_chunk15.wav,408,Non-Native,Arabic_KW,embeddings_cnn/408_chunk15.npy
28
+ data/processed_audio_chunks\408_chunk16.wav,408,Non-Native,Arabic_KW,embeddings_cnn/408_chunk16.npy
29
+ data/processed_audio_chunks\408_chunk17.wav,408,Non-Native,Arabic_KW,embeddings_cnn/408_chunk17.npy
30
+ data/processed_audio_chunks\490_chunk0.wav,490,Native,Arabic_SA,embeddings_cnn/490_chunk0.npy
31
+ data/processed_audio_chunks\490_chunk1.wav,490,Native,Arabic_SA,embeddings_cnn/490_chunk1.npy
32
+ data/processed_audio_chunks\490_chunk2.wav,490,Native,Arabic_SA,embeddings_cnn/490_chunk2.npy
33
+ data/processed_audio_chunks\490_chunk3.wav,490,Native,Arabic_SA,embeddings_cnn/490_chunk3.npy
34
+ data/processed_audio_chunks\440_chunk0.wav,440,Native,Arabic_SA,embeddings_cnn/440_chunk0.npy
35
+ data/processed_audio_chunks\440_chunk1.wav,440,Native,Arabic_SA,embeddings_cnn/440_chunk1.npy
36
+ data/processed_audio_chunks\440_chunk2.wav,440,Native,Arabic_SA,embeddings_cnn/440_chunk2.npy
37
+ data/processed_audio_chunks\440_chunk3.wav,440,Native,Arabic_SA,embeddings_cnn/440_chunk3.npy
38
+ data/processed_audio_chunks\440_chunk4.wav,440,Native,Arabic_SA,embeddings_cnn/440_chunk4.npy
39
+ data/processed_audio_chunks\440_chunk5.wav,440,Native,Arabic_SA,embeddings_cnn/440_chunk5.npy
40
+ data/processed_audio_chunks\543_chunk0.wav,543,Native,Arabic_QA,embeddings_cnn/543_chunk0.npy
41
+ data/processed_audio_chunks\543_chunk1.wav,543,Native,Arabic_QA,embeddings_cnn/543_chunk1.npy
42
+ data/processed_audio_chunks\543_chunk2.wav,543,Native,Arabic_QA,embeddings_cnn/543_chunk2.npy
43
+ data/processed_audio_chunks\543_chunk3.wav,543,Native,Arabic_QA,embeddings_cnn/543_chunk3.npy
44
+ data/processed_audio_chunks\543_chunk4.wav,543,Native,Arabic_QA,embeddings_cnn/543_chunk4.npy
45
+ data/processed_audio_chunks\543_chunk5.wav,543,Native,Arabic_QA,embeddings_cnn/543_chunk5.npy
46
+ data/processed_audio_chunks\543_chunk6.wav,543,Native,Arabic_QA,embeddings_cnn/543_chunk6.npy
47
+ data/processed_audio_chunks\492_chunk0.wav,492,Native,Arabic_SA,embeddings_cnn/492_chunk0.npy
48
+ data/processed_audio_chunks\492_chunk1.wav,492,Native,Arabic_SA,embeddings_cnn/492_chunk1.npy
49
+ data/processed_audio_chunks\492_chunk2.wav,492,Native,Arabic_SA,embeddings_cnn/492_chunk2.npy
50
+ data/processed_audio_chunks\492_chunk3.wav,492,Native,Arabic_SA,embeddings_cnn/492_chunk3.npy
51
+ data/processed_audio_chunks\492_chunk4.wav,492,Native,Arabic_SA,embeddings_cnn/492_chunk4.npy
52
+ data/processed_audio_chunks\492_chunk5.wav,492,Native,Arabic_SA,embeddings_cnn/492_chunk5.npy
53
+ data/processed_audio_chunks\345_chunk0.wav,345,Non-Native,Arabic_SA,embeddings_cnn/345_chunk0.npy
54
+ data/processed_audio_chunks\345_chunk1.wav,345,Non-Native,Arabic_SA,embeddings_cnn/345_chunk1.npy
55
+ data/processed_audio_chunks\345_chunk2.wav,345,Non-Native,Arabic_SA,embeddings_cnn/345_chunk2.npy
56
+ data/processed_audio_chunks\458_chunk0.wav,458,Native,Arabic_SA,embeddings_cnn/458_chunk0.npy
57
+ data/processed_audio_chunks\458_chunk1.wav,458,Native,Arabic_SA,embeddings_cnn/458_chunk1.npy
58
+ data/processed_audio_chunks\458_chunk2.wav,458,Native,Arabic_SA,embeddings_cnn/458_chunk2.npy
59
+ data/processed_audio_chunks\389_chunk0.wav,389,Non-Native,Arabic_SA,embeddings_cnn/389_chunk0.npy
60
+ data/processed_audio_chunks\389_chunk1.wav,389,Non-Native,Arabic_SA,embeddings_cnn/389_chunk1.npy
61
+ data/processed_audio_chunks\389_chunk2.wav,389,Non-Native,Arabic_SA,embeddings_cnn/389_chunk2.npy
62
+ data/processed_audio_chunks\360_chunk0.wav,360,Native,Arabic_AE,embeddings_cnn/360_chunk0.npy
63
+ data/processed_audio_chunks\360_chunk1.wav,360,Native,Arabic_AE,embeddings_cnn/360_chunk1.npy
64
+ data/processed_audio_chunks\360_chunk2.wav,360,Native,Arabic_AE,embeddings_cnn/360_chunk2.npy
65
+ data/processed_audio_chunks\360_chunk3.wav,360,Native,Arabic_AE,embeddings_cnn/360_chunk3.npy
66
+ data/processed_audio_chunks\294_chunk0.wav,294,Native,Arabic_QA,embeddings_cnn/294_chunk0.npy
67
+ data/processed_audio_chunks\294_chunk1.wav,294,Native,Arabic_QA,embeddings_cnn/294_chunk1.npy
68
+ data/processed_audio_chunks\294_chunk2.wav,294,Native,Arabic_QA,embeddings_cnn/294_chunk2.npy
69
+ data/processed_audio_chunks\606_chunk0.wav,606,Non-Native,Arabic_SA,embeddings_cnn/606_chunk0.npy
70
+ data/processed_audio_chunks\606_chunk1.wav,606,Non-Native,Arabic_SA,embeddings_cnn/606_chunk1.npy
71
+ data/processed_audio_chunks\606_chunk2.wav,606,Non-Native,Arabic_SA,embeddings_cnn/606_chunk2.npy
72
+ data/processed_audio_chunks\590_chunk0.wav,590,Native,Arabic_SA,embeddings_cnn/590_chunk0.npy
73
+ data/processed_audio_chunks\590_chunk1.wav,590,Native,Arabic_SA,embeddings_cnn/590_chunk1.npy
74
+ data/processed_audio_chunks\590_chunk2.wav,590,Native,Arabic_SA,embeddings_cnn/590_chunk2.npy
75
+ data/processed_audio_chunks\344_chunk0.wav,344,Native,Arabic_AE,embeddings_cnn/344_chunk0.npy
76
+ data/processed_audio_chunks\344_chunk1.wav,344,Native,Arabic_AE,embeddings_cnn/344_chunk1.npy
77
+ data/processed_audio_chunks\344_chunk2.wav,344,Native,Arabic_AE,embeddings_cnn/344_chunk2.npy
78
+ data/processed_audio_chunks\328_chunk0.wav,328,Native,Arabic_AE,embeddings_cnn/328_chunk0.npy
79
+ data/processed_audio_chunks\328_chunk1.wav,328,Native,Arabic_AE,embeddings_cnn/328_chunk1.npy
80
+ data/processed_audio_chunks\328_chunk2.wav,328,Native,Arabic_AE,embeddings_cnn/328_chunk2.npy
81
+ data/processed_audio_chunks\328_chunk3.wav,328,Native,Arabic_AE,embeddings_cnn/328_chunk3.npy
82
+ data/processed_audio_chunks\328_chunk4.wav,328,Native,Arabic_AE,embeddings_cnn/328_chunk4.npy
83
+ data/processed_audio_chunks\328_chunk5.wav,328,Native,Arabic_AE,embeddings_cnn/328_chunk5.npy
84
+ data/processed_audio_chunks\663_chunk0.wav,663,Native,Arabic_SA,embeddings_cnn/663_chunk0.npy
85
+ data/processed_audio_chunks\663_chunk1.wav,663,Native,Arabic_SA,embeddings_cnn/663_chunk1.npy
86
+ data/processed_audio_chunks\657_chunk0.wav,657,Non-Native,Arabic_QA,embeddings_cnn/657_chunk0.npy
87
+ data/processed_audio_chunks\657_chunk1.wav,657,Non-Native,Arabic_QA,embeddings_cnn/657_chunk1.npy
88
+ data/processed_audio_chunks\657_chunk2.wav,657,Non-Native,Arabic_QA,embeddings_cnn/657_chunk2.npy
89
+ data/processed_audio_chunks\654_chunk0.wav,654,Non-Native,Arabic_SA,embeddings_cnn/654_chunk0.npy
90
+ data/processed_audio_chunks\654_chunk1.wav,654,Non-Native,Arabic_SA,embeddings_cnn/654_chunk1.npy
91
+ data/processed_audio_chunks\654_chunk2.wav,654,Non-Native,Arabic_SA,embeddings_cnn/654_chunk2.npy
92
+ data/processed_audio_chunks\703_chunk0.wav,703,Non-Native,Arabic_SA,embeddings_cnn/703_chunk0.npy
93
+ data/processed_audio_chunks\703_chunk1.wav,703,Non-Native,Arabic_SA,embeddings_cnn/703_chunk1.npy
94
+ data/processed_audio_chunks\402_chunk0.wav,402,Native,Arabic_QA,embeddings_cnn/402_chunk0.npy
95
+ data/processed_audio_chunks\402_chunk1.wav,402,Native,Arabic_QA,embeddings_cnn/402_chunk1.npy
96
+ data/processed_audio_chunks\402_chunk2.wav,402,Native,Arabic_QA,embeddings_cnn/402_chunk2.npy
97
+ data/processed_audio_chunks\390_chunk0.wav,390,Non-Native,Arabic_SA,embeddings_cnn/390_chunk0.npy
98
+ data/processed_audio_chunks\390_chunk1.wav,390,Non-Native,Arabic_SA,embeddings_cnn/390_chunk1.npy
99
+ data/processed_audio_chunks\390_chunk2.wav,390,Non-Native,Arabic_SA,embeddings_cnn/390_chunk2.npy
100
+ data/processed_audio_chunks\390_chunk3.wav,390,Non-Native,Arabic_SA,embeddings_cnn/390_chunk3.npy
101
+ data/processed_audio_chunks\390_chunk4.wav,390,Non-Native,Arabic_SA,embeddings_cnn/390_chunk4.npy
102
+ data/processed_audio_chunks\390_chunk5.wav,390,Non-Native,Arabic_SA,embeddings_cnn/390_chunk5.npy
103
+ data/processed_audio_chunks\390_chunk6.wav,390,Non-Native,Arabic_SA,embeddings_cnn/390_chunk6.npy
104
+ data/processed_audio_chunks\390_chunk7.wav,390,Non-Native,Arabic_SA,embeddings_cnn/390_chunk7.npy
105
+ data/processed_audio_chunks\390_chunk8.wav,390,Non-Native,Arabic_SA,embeddings_cnn/390_chunk8.npy
106
+ data/processed_audio_chunks\393_chunk0.wav,393,Native,Arabic_QA,embeddings_cnn/393_chunk0.npy
107
+ data/processed_audio_chunks\393_chunk1.wav,393,Native,Arabic_QA,embeddings_cnn/393_chunk1.npy
108
+ data/processed_audio_chunks\393_chunk2.wav,393,Native,Arabic_QA,embeddings_cnn/393_chunk2.npy
109
+ data/processed_audio_chunks\393_chunk3.wav,393,Native,Arabic_QA,embeddings_cnn/393_chunk3.npy
110
+ data/processed_audio_chunks\393_chunk4.wav,393,Native,Arabic_QA,embeddings_cnn/393_chunk4.npy
111
+ data/processed_audio_chunks\372_chunk0.wav,372,Native,Arabic_QA,embeddings_cnn/372_chunk0.npy
112
+ data/processed_audio_chunks\372_chunk1.wav,372,Native,Arabic_QA,embeddings_cnn/372_chunk1.npy
113
+ data/processed_audio_chunks\374_chunk0.wav,374,Native,Arabic_SA,embeddings_cnn/374_chunk0.npy
114
+ data/processed_audio_chunks\374_chunk1.wav,374,Native,Arabic_SA,embeddings_cnn/374_chunk1.npy
115
+ data/processed_audio_chunks\374_chunk2.wav,374,Native,Arabic_SA,embeddings_cnn/374_chunk2.npy
116
+ data/processed_audio_chunks\374_chunk3.wav,374,Native,Arabic_SA,embeddings_cnn/374_chunk3.npy
117
+ data/processed_audio_chunks\621_chunk0.wav,621,Native,Arabic_QA,embeddings_cnn/621_chunk0.npy
118
+ data/processed_audio_chunks\621_chunk1.wav,621,Native,Arabic_QA,embeddings_cnn/621_chunk1.npy
119
+ data/processed_audio_chunks\621_chunk2.wav,621,Native,Arabic_QA,embeddings_cnn/621_chunk2.npy
120
+ data/processed_audio_chunks\621_chunk3.wav,621,Native,Arabic_QA,embeddings_cnn/621_chunk3.npy
121
+ data/processed_audio_chunks\621_chunk4.wav,621,Native,Arabic_QA,embeddings_cnn/621_chunk4.npy
122
+ data/processed_audio_chunks\550_chunk0.wav,550,Non-Native,Arabic_SA,embeddings_cnn/550_chunk0.npy
123
+ data/processed_audio_chunks\550_chunk1.wav,550,Non-Native,Arabic_SA,embeddings_cnn/550_chunk1.npy
124
+ data/processed_audio_chunks\550_chunk2.wav,550,Non-Native,Arabic_SA,embeddings_cnn/550_chunk2.npy
125
+ data/processed_audio_chunks\339_chunk0.wav,339,Native,Arabic_QA,embeddings_cnn/339_chunk0.npy
126
+ data/processed_audio_chunks\339_chunk1.wav,339,Native,Arabic_QA,embeddings_cnn/339_chunk1.npy
127
+ data/processed_audio_chunks\339_chunk2.wav,339,Native,Arabic_QA,embeddings_cnn/339_chunk2.npy
128
+ data/processed_audio_chunks\339_chunk3.wav,339,Native,Arabic_QA,embeddings_cnn/339_chunk3.npy
129
+ data/processed_audio_chunks\394_chunk0.wav,394,Native,Arabic_AE,embeddings_cnn/394_chunk0.npy
130
+ data/processed_audio_chunks\394_chunk1.wav,394,Native,Arabic_AE,embeddings_cnn/394_chunk1.npy
131
+ data/processed_audio_chunks\394_chunk2.wav,394,Native,Arabic_AE,embeddings_cnn/394_chunk2.npy
132
+ data/processed_audio_chunks\342_chunk0.wav,342,Non-Native,Arabic_SA,embeddings_cnn/342_chunk0.npy
133
+ data/processed_audio_chunks\342_chunk1.wav,342,Non-Native,Arabic_SA,embeddings_cnn/342_chunk1.npy
134
+ data/processed_audio_chunks\342_chunk2.wav,342,Non-Native,Arabic_SA,embeddings_cnn/342_chunk2.npy
135
+ data/processed_audio_chunks\337_chunk0.wav,337,Native,Arabic_AE,embeddings_cnn/337_chunk0.npy
136
+ data/processed_audio_chunks\337_chunk1.wav,337,Native,Arabic_AE,embeddings_cnn/337_chunk1.npy
137
+ data/processed_audio_chunks\337_chunk2.wav,337,Native,Arabic_AE,embeddings_cnn/337_chunk2.npy
138
+ data/processed_audio_chunks\337_chunk3.wav,337,Native,Arabic_AE,embeddings_cnn/337_chunk3.npy
139
+ data/processed_audio_chunks\337_chunk4.wav,337,Native,Arabic_AE,embeddings_cnn/337_chunk4.npy
140
+ data/processed_audio_chunks\337_chunk5.wav,337,Native,Arabic_AE,embeddings_cnn/337_chunk5.npy
141
+ data/processed_audio_chunks\337_chunk6.wav,337,Native,Arabic_AE,embeddings_cnn/337_chunk6.npy
142
+ data/processed_audio_chunks\337_chunk7.wav,337,Native,Arabic_AE,embeddings_cnn/337_chunk7.npy
143
+ data/processed_audio_chunks\337_chunk8.wav,337,Native,Arabic_AE,embeddings_cnn/337_chunk8.npy
144
+ data/processed_audio_chunks\521_chunk0.wav,521,Non-Native,Arabic_SA,embeddings_cnn/521_chunk0.npy
145
+ data/processed_audio_chunks\521_chunk1.wav,521,Non-Native,Arabic_SA,embeddings_cnn/521_chunk1.npy
146
+ data/processed_audio_chunks\521_chunk2.wav,521,Non-Native,Arabic_SA,embeddings_cnn/521_chunk2.npy
147
+ data/processed_audio_chunks\644_chunk0.wav,644,Native,Arabic_SA,embeddings_cnn/644_chunk0.npy
148
+ data/processed_audio_chunks\644_chunk1.wav,644,Native,Arabic_SA,embeddings_cnn/644_chunk1.npy
149
+ data/processed_audio_chunks\632_chunk0.wav,632,Native,Arabic_SA,embeddings_cnn/632_chunk0.npy
150
+ data/processed_audio_chunks\632_chunk1.wav,632,Native,Arabic_SA,embeddings_cnn/632_chunk1.npy
151
+ data/processed_audio_chunks\632_chunk2.wav,632,Native,Arabic_SA,embeddings_cnn/632_chunk2.npy
152
+ data/processed_audio_chunks\632_chunk3.wav,632,Native,Arabic_SA,embeddings_cnn/632_chunk3.npy
153
+ data/processed_audio_chunks\632_chunk4.wav,632,Native,Arabic_SA,embeddings_cnn/632_chunk4.npy
154
+ data/processed_audio_chunks\632_chunk5.wav,632,Native,Arabic_SA,embeddings_cnn/632_chunk5.npy
155
+ data/processed_audio_chunks\501_chunk0.wav,501,Native,Arabic_QA,embeddings_cnn/501_chunk0.npy
156
+ data/processed_audio_chunks\501_chunk1.wav,501,Native,Arabic_QA,embeddings_cnn/501_chunk1.npy
157
+ data/processed_audio_chunks\501_chunk2.wav,501,Native,Arabic_QA,embeddings_cnn/501_chunk2.npy
158
+ data/processed_audio_chunks\501_chunk3.wav,501,Native,Arabic_QA,embeddings_cnn/501_chunk3.npy
159
+ data/processed_audio_chunks\382_chunk0.wav,382,Non-Native,Arabic_SA,embeddings_cnn/382_chunk0.npy
160
+ data/processed_audio_chunks\382_chunk1.wav,382,Non-Native,Arabic_SA,embeddings_cnn/382_chunk1.npy
161
+ data/processed_audio_chunks\382_chunk2.wav,382,Non-Native,Arabic_SA,embeddings_cnn/382_chunk2.npy
162
+ data/processed_audio_chunks\341_chunk0.wav,341,Native,Arabic_QA,embeddings_cnn/341_chunk0.npy
163
+ data/processed_audio_chunks\341_chunk1.wav,341,Native,Arabic_QA,embeddings_cnn/341_chunk1.npy
164
+ data/processed_audio_chunks\341_chunk2.wav,341,Native,Arabic_QA,embeddings_cnn/341_chunk2.npy
165
+ data/processed_audio_chunks\341_chunk3.wav,341,Native,Arabic_QA,embeddings_cnn/341_chunk3.npy
166
+ data/processed_audio_chunks\430_chunk0.wav,430,Native,Arabic_QA,embeddings_cnn/430_chunk0.npy
167
+ data/processed_audio_chunks\430_chunk1.wav,430,Native,Arabic_QA,embeddings_cnn/430_chunk1.npy
168
+ data/processed_audio_chunks\430_chunk2.wav,430,Native,Arabic_QA,embeddings_cnn/430_chunk2.npy
169
+ data/processed_audio_chunks\682_chunk0.wav,682,Native,Arabic_QA,embeddings_cnn/682_chunk0.npy
170
+ data/processed_audio_chunks\682_chunk1.wav,682,Native,Arabic_QA,embeddings_cnn/682_chunk1.npy
171
+ data/processed_audio_chunks\682_chunk2.wav,682,Native,Arabic_QA,embeddings_cnn/682_chunk2.npy
172
+ data/processed_audio_chunks\520_chunk0.wav,520,Native,Arabic_AE,embeddings_cnn/520_chunk0.npy
173
+ data/processed_audio_chunks\520_chunk1.wav,520,Native,Arabic_AE,embeddings_cnn/520_chunk1.npy
174
+ data/processed_audio_chunks\520_chunk2.wav,520,Native,Arabic_AE,embeddings_cnn/520_chunk2.npy
175
+ data/processed_audio_chunks\520_chunk3.wav,520,Native,Arabic_AE,embeddings_cnn/520_chunk3.npy
176
+ data/processed_audio_chunks\520_chunk4.wav,520,Native,Arabic_AE,embeddings_cnn/520_chunk4.npy
177
+ data/processed_audio_chunks\520_chunk5.wav,520,Native,Arabic_AE,embeddings_cnn/520_chunk5.npy
178
+ data/processed_audio_chunks\520_chunk6.wav,520,Native,Arabic_AE,embeddings_cnn/520_chunk6.npy
179
+ data/processed_audio_chunks\311_chunk0.wav,311,Native,Arabic_AE,embeddings_cnn/311_chunk0.npy
180
+ data/processed_audio_chunks\311_chunk1.wav,311,Native,Arabic_AE,embeddings_cnn/311_chunk1.npy
181
+ data/processed_audio_chunks\311_chunk2.wav,311,Native,Arabic_AE,embeddings_cnn/311_chunk2.npy
182
+ data/processed_audio_chunks\311_chunk3.wav,311,Native,Arabic_AE,embeddings_cnn/311_chunk3.npy
183
+ data/processed_audio_chunks\311_chunk4.wav,311,Native,Arabic_AE,embeddings_cnn/311_chunk4.npy
184
+ data/processed_audio_chunks\311_chunk5.wav,311,Native,Arabic_AE,embeddings_cnn/311_chunk5.npy
185
+ data/processed_audio_chunks\311_chunk6.wav,311,Native,Arabic_AE,embeddings_cnn/311_chunk6.npy
186
+ data/processed_audio_chunks\311_chunk7.wav,311,Native,Arabic_AE,embeddings_cnn/311_chunk7.npy
187
+ data/processed_audio_chunks\311_chunk8.wav,311,Native,Arabic_AE,embeddings_cnn/311_chunk8.npy
188
+ data/processed_audio_chunks\311_chunk9.wav,311,Native,Arabic_AE,embeddings_cnn/311_chunk9.npy
189
+ data/processed_audio_chunks\340_chunk0.wav,340,Native,Arabic_AE,embeddings_cnn/340_chunk0.npy
190
+ data/processed_audio_chunks\340_chunk1.wav,340,Native,Arabic_AE,embeddings_cnn/340_chunk1.npy
191
+ data/processed_audio_chunks\340_chunk2.wav,340,Native,Arabic_AE,embeddings_cnn/340_chunk2.npy
192
+ data/processed_audio_chunks\380_chunk0.wav,380,Native,Arabic_QA,embeddings_cnn/380_chunk0.npy
193
+ data/processed_audio_chunks\380_chunk1.wav,380,Native,Arabic_QA,embeddings_cnn/380_chunk1.npy
194
+ data/processed_audio_chunks\380_chunk2.wav,380,Native,Arabic_QA,embeddings_cnn/380_chunk2.npy
195
+ data/processed_audio_chunks\380_chunk3.wav,380,Native,Arabic_QA,embeddings_cnn/380_chunk3.npy
196
+ data/processed_audio_chunks\473_chunk0.wav,473,Native,Arabic_AE,embeddings_cnn/473_chunk0.npy
197
+ data/processed_audio_chunks\473_chunk1.wav,473,Native,Arabic_AE,embeddings_cnn/473_chunk1.npy
198
+ data/processed_audio_chunks\473_chunk2.wav,473,Native,Arabic_AE,embeddings_cnn/473_chunk2.npy
199
+ data/processed_audio_chunks\383_chunk0.wav,383,Non-Native,Arabic_SA,embeddings_cnn/383_chunk0.npy
200
+ data/processed_audio_chunks\383_chunk1.wav,383,Non-Native,Arabic_SA,embeddings_cnn/383_chunk1.npy
201
+ data/processed_audio_chunks\383_chunk2.wav,383,Non-Native,Arabic_SA,embeddings_cnn/383_chunk2.npy
202
+ data/processed_audio_chunks\383_chunk3.wav,383,Non-Native,Arabic_SA,embeddings_cnn/383_chunk3.npy
203
+ data/processed_audio_chunks\383_chunk4.wav,383,Non-Native,Arabic_SA,embeddings_cnn/383_chunk4.npy
204
+ data/processed_audio_chunks\383_chunk5.wav,383,Non-Native,Arabic_SA,embeddings_cnn/383_chunk5.npy
205
+ data/processed_audio_chunks\383_chunk6.wav,383,Non-Native,Arabic_SA,embeddings_cnn/383_chunk6.npy
206
+ data/processed_audio_chunks\383_chunk7.wav,383,Non-Native,Arabic_SA,embeddings_cnn/383_chunk7.npy
207
+ data/processed_audio_chunks\383_chunk8.wav,383,Non-Native,Arabic_SA,embeddings_cnn/383_chunk8.npy
208
+ data/processed_audio_chunks\697_chunk0.wav,697,Native,Arabic_QA,embeddings_cnn/697_chunk0.npy
209
+ data/processed_audio_chunks\697_chunk1.wav,697,Native,Arabic_QA,embeddings_cnn/697_chunk1.npy
210
+ data/processed_audio_chunks\697_chunk2.wav,697,Native,Arabic_QA,embeddings_cnn/697_chunk2.npy
211
+ data/processed_audio_chunks\544_chunk0.wav,544,Non-Native,Arabic_SA,embeddings_cnn/544_chunk0.npy
212
+ data/processed_audio_chunks\544_chunk1.wav,544,Non-Native,Arabic_SA,embeddings_cnn/544_chunk1.npy
213
+ data/processed_audio_chunks\544_chunk2.wav,544,Non-Native,Arabic_SA,embeddings_cnn/544_chunk2.npy
214
+ data/processed_audio_chunks\544_chunk3.wav,544,Non-Native,Arabic_SA,embeddings_cnn/544_chunk3.npy
215
+ data/processed_audio_chunks\544_chunk4.wav,544,Non-Native,Arabic_SA,embeddings_cnn/544_chunk4.npy
216
+ data/processed_audio_chunks\544_chunk5.wav,544,Non-Native,Arabic_SA,embeddings_cnn/544_chunk5.npy
217
+ data/processed_audio_chunks\544_chunk6.wav,544,Non-Native,Arabic_SA,embeddings_cnn/544_chunk6.npy
218
+ data/processed_audio_chunks\544_chunk7.wav,544,Non-Native,Arabic_SA,embeddings_cnn/544_chunk7.npy
219
+ data/processed_audio_chunks\544_chunk8.wav,544,Non-Native,Arabic_SA,embeddings_cnn/544_chunk8.npy
220
+ data/processed_audio_chunks\544_chunk9.wav,544,Non-Native,Arabic_SA,embeddings_cnn/544_chunk9.npy
221
+ data/processed_audio_chunks\544_chunk10.wav,544,Non-Native,Arabic_SA,embeddings_cnn/544_chunk10.npy
222
+ data/processed_audio_chunks\544_chunk11.wav,544,Non-Native,Arabic_SA,embeddings_cnn/544_chunk11.npy
223
+ data/processed_audio_chunks\544_chunk12.wav,544,Non-Native,Arabic_SA,embeddings_cnn/544_chunk12.npy
224
+ data/processed_audio_chunks\544_chunk13.wav,544,Non-Native,Arabic_SA,embeddings_cnn/544_chunk13.npy
225
+ data/processed_audio_chunks\544_chunk14.wav,544,Non-Native,Arabic_SA,embeddings_cnn/544_chunk14.npy
226
+ data/processed_audio_chunks\544_chunk15.wav,544,Non-Native,Arabic_SA,embeddings_cnn/544_chunk15.npy
227
+ data/processed_audio_chunks\544_chunk16.wav,544,Non-Native,Arabic_SA,embeddings_cnn/544_chunk16.npy
228
+ data/processed_audio_chunks\544_chunk17.wav,544,Non-Native,Arabic_SA,embeddings_cnn/544_chunk17.npy
229
+ data/processed_audio_chunks\544_chunk18.wav,544,Non-Native,Arabic_SA,embeddings_cnn/544_chunk18.npy
230
+ data/processed_audio_chunks\544_chunk19.wav,544,Non-Native,Arabic_SA,embeddings_cnn/544_chunk19.npy
231
+ data/processed_audio_chunks\544_chunk20.wav,544,Non-Native,Arabic_SA,embeddings_cnn/544_chunk20.npy
232
+ data/processed_audio_chunks\544_chunk21.wav,544,Non-Native,Arabic_SA,embeddings_cnn/544_chunk21.npy
233
+ data/processed_audio_chunks\544_chunk22.wav,544,Non-Native,Arabic_SA,embeddings_cnn/544_chunk22.npy
234
+ data/processed_audio_chunks\544_chunk23.wav,544,Non-Native,Arabic_SA,embeddings_cnn/544_chunk23.npy
235
+ data/processed_audio_chunks\544_chunk24.wav,544,Non-Native,Arabic_SA,embeddings_cnn/544_chunk24.npy
236
+ data/processed_audio_chunks\544_chunk25.wav,544,Non-Native,Arabic_SA,embeddings_cnn/544_chunk25.npy
237
+ data/processed_audio_chunks\544_chunk26.wav,544,Non-Native,Arabic_SA,embeddings_cnn/544_chunk26.npy
238
+ data/processed_audio_chunks\544_chunk27.wav,544,Non-Native,Arabic_SA,embeddings_cnn/544_chunk27.npy
239
+ data/processed_audio_chunks\544_chunk28.wav,544,Non-Native,Arabic_SA,embeddings_cnn/544_chunk28.npy
240
+ data/processed_audio_chunks\544_chunk29.wav,544,Non-Native,Arabic_SA,embeddings_cnn/544_chunk29.npy
241
+ data/processed_audio_chunks\544_chunk30.wav,544,Non-Native,Arabic_SA,embeddings_cnn/544_chunk30.npy
242
+ data/processed_audio_chunks\544_chunk31.wav,544,Non-Native,Arabic_SA,embeddings_cnn/544_chunk31.npy
243
+ data/processed_audio_chunks\544_chunk32.wav,544,Non-Native,Arabic_SA,embeddings_cnn/544_chunk32.npy
244
+ data/processed_audio_chunks\544_chunk33.wav,544,Non-Native,Arabic_SA,embeddings_cnn/544_chunk33.npy
245
+ data/processed_audio_chunks\544_chunk34.wav,544,Non-Native,Arabic_SA,embeddings_cnn/544_chunk34.npy
246
+ data/processed_audio_chunks\544_chunk35.wav,544,Non-Native,Arabic_SA,embeddings_cnn/544_chunk35.npy
247
+ data/processed_audio_chunks\544_chunk36.wav,544,Non-Native,Arabic_SA,embeddings_cnn/544_chunk36.npy
248
+ data/processed_audio_chunks\544_chunk37.wav,544,Non-Native,Arabic_SA,embeddings_cnn/544_chunk37.npy
249
+ data/processed_audio_chunks\544_chunk38.wav,544,Non-Native,Arabic_SA,embeddings_cnn/544_chunk38.npy
250
+ data/processed_audio_chunks\544_chunk39.wav,544,Non-Native,Arabic_SA,embeddings_cnn/544_chunk39.npy
251
+ data/processed_audio_chunks\387_chunk0.wav,387,Native,Arabic_SA,embeddings_cnn/387_chunk0.npy
252
+ data/processed_audio_chunks\387_chunk1.wav,387,Native,Arabic_SA,embeddings_cnn/387_chunk1.npy
253
+ data/processed_audio_chunks\387_chunk2.wav,387,Native,Arabic_SA,embeddings_cnn/387_chunk2.npy
254
+ data/processed_audio_chunks\387_chunk3.wav,387,Native,Arabic_SA,embeddings_cnn/387_chunk3.npy
255
+ data/processed_audio_chunks\387_chunk4.wav,387,Native,Arabic_SA,embeddings_cnn/387_chunk4.npy
256
+ data/processed_audio_chunks\387_chunk5.wav,387,Native,Arabic_SA,embeddings_cnn/387_chunk5.npy
257
+ data/processed_audio_chunks\485_chunk0.wav,485,Native,Arabic_QA,embeddings_cnn/485_chunk0.npy
258
+ data/processed_audio_chunks\485_chunk1.wav,485,Native,Arabic_QA,embeddings_cnn/485_chunk1.npy
259
+ data/processed_audio_chunks\485_chunk2.wav,485,Native,Arabic_QA,embeddings_cnn/485_chunk2.npy
260
+ data/processed_audio_chunks\485_chunk3.wav,485,Native,Arabic_QA,embeddings_cnn/485_chunk3.npy
261
+ data/processed_audio_chunks\485_chunk4.wav,485,Native,Arabic_QA,embeddings_cnn/485_chunk4.npy
262
+ data/processed_audio_chunks\779_chunk0.wav,779,Native,Arabic_QA,embeddings_cnn/779_chunk0.npy
263
+ data/processed_audio_chunks\779_chunk1.wav,779,Native,Arabic_QA,embeddings_cnn/779_chunk1.npy
264
+ data/processed_audio_chunks\779_chunk2.wav,779,Native,Arabic_QA,embeddings_cnn/779_chunk2.npy
265
+ data/processed_audio_chunks\779_chunk3.wav,779,Native,Arabic_QA,embeddings_cnn/779_chunk3.npy
266
+ data/processed_audio_chunks\362_chunk0.wav,362,Native,Arabic_AE,embeddings_cnn/362_chunk0.npy
267
+ data/processed_audio_chunks\362_chunk1.wav,362,Native,Arabic_AE,embeddings_cnn/362_chunk1.npy
268
+ data/processed_audio_chunks\362_chunk2.wav,362,Native,Arabic_AE,embeddings_cnn/362_chunk2.npy
269
+ data/processed_audio_chunks\362_chunk3.wav,362,Native,Arabic_AE,embeddings_cnn/362_chunk3.npy
270
+ data/processed_audio_chunks\299_chunk0.wav,299,Native,Arabic_QA,embeddings_cnn/299_chunk0.npy
271
+ data/processed_audio_chunks\299_chunk1.wav,299,Native,Arabic_QA,embeddings_cnn/299_chunk1.npy
272
+ data/processed_audio_chunks\299_chunk2.wav,299,Native,Arabic_QA,embeddings_cnn/299_chunk2.npy
273
+ data/processed_audio_chunks\299_chunk3.wav,299,Native,Arabic_QA,embeddings_cnn/299_chunk3.npy
274
+ data/processed_audio_chunks\379_chunk0.wav,379,Non-Native,Arabic_SA,embeddings_cnn/379_chunk0.npy
275
+ data/processed_audio_chunks\379_chunk1.wav,379,Non-Native,Arabic_SA,embeddings_cnn/379_chunk1.npy
276
+ data/processed_audio_chunks\379_chunk2.wav,379,Non-Native,Arabic_SA,embeddings_cnn/379_chunk2.npy
277
+ data/processed_audio_chunks\379_chunk3.wav,379,Non-Native,Arabic_SA,embeddings_cnn/379_chunk3.npy
278
+ data/processed_audio_chunks\379_chunk4.wav,379,Non-Native,Arabic_SA,embeddings_cnn/379_chunk4.npy
279
+ data/processed_audio_chunks\317_chunk0.wav,317,Native,Arabic_AE,embeddings_cnn/317_chunk0.npy
280
+ data/processed_audio_chunks\317_chunk1.wav,317,Native,Arabic_AE,embeddings_cnn/317_chunk1.npy
281
+ data/processed_audio_chunks\317_chunk2.wav,317,Native,Arabic_AE,embeddings_cnn/317_chunk2.npy
282
+ data/processed_audio_chunks\317_chunk3.wav,317,Native,Arabic_AE,embeddings_cnn/317_chunk3.npy
283
+ data/processed_audio_chunks\317_chunk4.wav,317,Native,Arabic_AE,embeddings_cnn/317_chunk4.npy
284
+ data/processed_audio_chunks\317_chunk5.wav,317,Native,Arabic_AE,embeddings_cnn/317_chunk5.npy
285
+ data/processed_audio_chunks\317_chunk6.wav,317,Native,Arabic_AE,embeddings_cnn/317_chunk6.npy
286
+ data/processed_audio_chunks\317_chunk7.wav,317,Native,Arabic_AE,embeddings_cnn/317_chunk7.npy
287
+ data/processed_audio_chunks\680_chunk0.wav,680,Native,Arabic_QA,embeddings_cnn/680_chunk0.npy
288
+ data/processed_audio_chunks\680_chunk1.wav,680,Native,Arabic_QA,embeddings_cnn/680_chunk1.npy
289
+ data/processed_audio_chunks\680_chunk2.wav,680,Native,Arabic_QA,embeddings_cnn/680_chunk2.npy
290
+ data/processed_audio_chunks\419_chunk0.wav,419,Native,Arabic_QA,embeddings_cnn/419_chunk0.npy
291
+ data/processed_audio_chunks\419_chunk1.wav,419,Native,Arabic_QA,embeddings_cnn/419_chunk1.npy
292
+ data/processed_audio_chunks\419_chunk2.wav,419,Native,Arabic_QA,embeddings_cnn/419_chunk2.npy
293
+ data/processed_audio_chunks\419_chunk3.wav,419,Native,Arabic_QA,embeddings_cnn/419_chunk3.npy
294
+ data/processed_audio_chunks\419_chunk4.wav,419,Native,Arabic_QA,embeddings_cnn/419_chunk4.npy
295
+ data/processed_audio_chunks\696_chunk0.wav,696,Native,Arabic_QA,embeddings_cnn/696_chunk0.npy
296
+ data/processed_audio_chunks\696_chunk1.wav,696,Native,Arabic_QA,embeddings_cnn/696_chunk1.npy
297
+ data/processed_audio_chunks\696_chunk2.wav,696,Native,Arabic_QA,embeddings_cnn/696_chunk2.npy
298
+ data/processed_audio_chunks\484_chunk0.wav,484,Native,Arabic_SA,embeddings_cnn/484_chunk0.npy
299
+ data/processed_audio_chunks\484_chunk1.wav,484,Native,Arabic_SA,embeddings_cnn/484_chunk1.npy
300
+ data/processed_audio_chunks\484_chunk2.wav,484,Native,Arabic_SA,embeddings_cnn/484_chunk2.npy
301
+ data/processed_audio_chunks\484_chunk3.wav,484,Native,Arabic_SA,embeddings_cnn/484_chunk3.npy
302
+ data/processed_audio_chunks\580_chunk0.wav,580,Native,Arabic_AE,embeddings_cnn/580_chunk0.npy
303
+ data/processed_audio_chunks\580_chunk1.wav,580,Native,Arabic_AE,embeddings_cnn/580_chunk1.npy
304
+ data/processed_audio_chunks\580_chunk2.wav,580,Native,Arabic_AE,embeddings_cnn/580_chunk2.npy
305
+ data/processed_audio_chunks\343_chunk0.wav,343,Native,Arabic_AE,embeddings_cnn/343_chunk0.npy
306
+ data/processed_audio_chunks\343_chunk1.wav,343,Native,Arabic_AE,embeddings_cnn/343_chunk1.npy
307
+ data/processed_audio_chunks\343_chunk2.wav,343,Native,Arabic_AE,embeddings_cnn/343_chunk2.npy
308
+ data/processed_audio_chunks\343_chunk3.wav,343,Native,Arabic_AE,embeddings_cnn/343_chunk3.npy
309
+ data/processed_audio_chunks\594_chunk0.wav,594,Native,Arabic_AE,embeddings_cnn/594_chunk0.npy
310
+ data/processed_audio_chunks\594_chunk1.wav,594,Native,Arabic_AE,embeddings_cnn/594_chunk1.npy
311
+ data/processed_audio_chunks\594_chunk2.wav,594,Native,Arabic_AE,embeddings_cnn/594_chunk2.npy
312
+ data/processed_audio_chunks\565_chunk0.wav,565,Native,Arabic_AE,embeddings_cnn/565_chunk0.npy
313
+ data/processed_audio_chunks\565_chunk1.wav,565,Native,Arabic_AE,embeddings_cnn/565_chunk1.npy
314
+ data/processed_audio_chunks\565_chunk2.wav,565,Native,Arabic_AE,embeddings_cnn/565_chunk2.npy
315
+ data/processed_audio_chunks\352_chunk0.wav,352,Native,Arabic_QA,embeddings_cnn/352_chunk0.npy
316
+ data/processed_audio_chunks\352_chunk1.wav,352,Native,Arabic_QA,embeddings_cnn/352_chunk1.npy
317
+ data/processed_audio_chunks\352_chunk2.wav,352,Native,Arabic_QA,embeddings_cnn/352_chunk2.npy
318
+ data/processed_audio_chunks\352_chunk3.wav,352,Native,Arabic_QA,embeddings_cnn/352_chunk3.npy
319
+ data/processed_audio_chunks\618_chunk0.wav,618,Non-Native,Arabic_SA,embeddings_cnn/618_chunk0.npy
320
+ data/processed_audio_chunks\618_chunk1.wav,618,Non-Native,Arabic_SA,embeddings_cnn/618_chunk1.npy
321
+ data/processed_audio_chunks\331_chunk0.wav,331,Native,Arabic_QA,embeddings_cnn/331_chunk0.npy
322
+ data/processed_audio_chunks\331_chunk1.wav,331,Native,Arabic_QA,embeddings_cnn/331_chunk1.npy
323
+ data/processed_audio_chunks\331_chunk2.wav,331,Native,Arabic_QA,embeddings_cnn/331_chunk2.npy
324
+ data/processed_audio_chunks\331_chunk3.wav,331,Native,Arabic_QA,embeddings_cnn/331_chunk3.npy
325
+ data/processed_audio_chunks\331_chunk4.wav,331,Native,Arabic_QA,embeddings_cnn/331_chunk4.npy
326
+ data/processed_audio_chunks\331_chunk5.wav,331,Native,Arabic_QA,embeddings_cnn/331_chunk5.npy
327
+ data/processed_audio_chunks\331_chunk6.wav,331,Native,Arabic_QA,embeddings_cnn/331_chunk6.npy
328
+ data/processed_audio_chunks\331_chunk7.wav,331,Native,Arabic_QA,embeddings_cnn/331_chunk7.npy
329
+ data/processed_audio_chunks\331_chunk8.wav,331,Native,Arabic_QA,embeddings_cnn/331_chunk8.npy
330
+ data/processed_audio_chunks\331_chunk9.wav,331,Native,Arabic_QA,embeddings_cnn/331_chunk9.npy
331
+ data/processed_audio_chunks\435_chunk0.wav,435,Native,Arabic_QA,embeddings_cnn/435_chunk0.npy
332
+ data/processed_audio_chunks\435_chunk1.wav,435,Native,Arabic_QA,embeddings_cnn/435_chunk1.npy
333
+ data/processed_audio_chunks\435_chunk2.wav,435,Native,Arabic_QA,embeddings_cnn/435_chunk2.npy
334
+ data/processed_audio_chunks\435_chunk3.wav,435,Native,Arabic_QA,embeddings_cnn/435_chunk3.npy
335
+ data/processed_audio_chunks\576_chunk0.wav,576,Non-Native,Arabic_SA,embeddings_cnn/576_chunk0.npy
336
+ data/processed_audio_chunks\576_chunk1.wav,576,Non-Native,Arabic_SA,embeddings_cnn/576_chunk1.npy
337
+ data/processed_audio_chunks\576_chunk2.wav,576,Non-Native,Arabic_SA,embeddings_cnn/576_chunk2.npy
338
+ data/processed_audio_chunks\576_chunk3.wav,576,Non-Native,Arabic_SA,embeddings_cnn/576_chunk3.npy
339
+ data/processed_audio_chunks\576_chunk4.wav,576,Non-Native,Arabic_SA,embeddings_cnn/576_chunk4.npy
340
+ data/processed_audio_chunks\576_chunk5.wav,576,Non-Native,Arabic_SA,embeddings_cnn/576_chunk5.npy
341
+ data/processed_audio_chunks\396_chunk0.wav,396,Non-Native,Arabic_SA,embeddings_cnn/396_chunk0.npy
342
+ data/processed_audio_chunks\396_chunk1.wav,396,Non-Native,Arabic_SA,embeddings_cnn/396_chunk1.npy
343
+ data/processed_audio_chunks\396_chunk2.wav,396,Non-Native,Arabic_SA,embeddings_cnn/396_chunk2.npy
344
+ data/processed_audio_chunks\396_chunk3.wav,396,Non-Native,Arabic_SA,embeddings_cnn/396_chunk3.npy
345
+ data/processed_audio_chunks\396_chunk4.wav,396,Non-Native,Arabic_SA,embeddings_cnn/396_chunk4.npy
346
+ data/processed_audio_chunks\396_chunk5.wav,396,Non-Native,Arabic_SA,embeddings_cnn/396_chunk5.npy
347
+ data/processed_audio_chunks\607_chunk0.wav,607,Non-Native,Arabic_SA,embeddings_cnn/607_chunk0.npy
348
+ data/processed_audio_chunks\607_chunk1.wav,607,Non-Native,Arabic_SA,embeddings_cnn/607_chunk1.npy
349
+ data/processed_audio_chunks\319_chunk0.wav,319,Non-Native,Arabic_QA,embeddings_cnn/319_chunk0.npy
350
+ data/processed_audio_chunks\319_chunk1.wav,319,Non-Native,Arabic_QA,embeddings_cnn/319_chunk1.npy
351
+ data/processed_audio_chunks\319_chunk2.wav,319,Non-Native,Arabic_QA,embeddings_cnn/319_chunk2.npy
352
+ data/processed_audio_chunks\498_chunk0.wav,498,Native,Arabic_QA,embeddings_cnn/498_chunk0.npy
353
+ data/processed_audio_chunks\498_chunk1.wav,498,Native,Arabic_QA,embeddings_cnn/498_chunk1.npy
354
+ data/processed_audio_chunks\498_chunk2.wav,498,Native,Arabic_QA,embeddings_cnn/498_chunk2.npy
355
+ data/processed_audio_chunks\463_chunk0.wav,463,Native,Arabic_AE,embeddings_cnn/463_chunk0.npy
356
+ data/processed_audio_chunks\463_chunk1.wav,463,Native,Arabic_AE,embeddings_cnn/463_chunk1.npy
357
+ data/processed_audio_chunks\463_chunk2.wav,463,Native,Arabic_AE,embeddings_cnn/463_chunk2.npy
358
+ data/processed_audio_chunks\326_chunk0.wav,326,Native,Arabic_QA,embeddings_cnn/326_chunk0.npy
359
+ data/processed_audio_chunks\326_chunk1.wav,326,Native,Arabic_QA,embeddings_cnn/326_chunk1.npy
360
+ data/processed_audio_chunks\326_chunk2.wav,326,Native,Arabic_QA,embeddings_cnn/326_chunk2.npy
361
+ data/processed_audio_chunks\326_chunk3.wav,326,Native,Arabic_QA,embeddings_cnn/326_chunk3.npy
362
+ data/processed_audio_chunks\326_chunk4.wav,326,Native,Arabic_QA,embeddings_cnn/326_chunk4.npy
363
+ data/processed_audio_chunks\326_chunk5.wav,326,Native,Arabic_QA,embeddings_cnn/326_chunk5.npy
364
+ data/processed_audio_chunks\326_chunk6.wav,326,Native,Arabic_QA,embeddings_cnn/326_chunk6.npy
365
+ data/processed_audio_chunks\326_chunk7.wav,326,Native,Arabic_QA,embeddings_cnn/326_chunk7.npy
366
+ data/processed_audio_chunks\326_chunk8.wav,326,Native,Arabic_QA,embeddings_cnn/326_chunk8.npy
367
+ data/processed_audio_chunks\326_chunk9.wav,326,Native,Arabic_QA,embeddings_cnn/326_chunk9.npy
368
+ data/processed_audio_chunks\326_chunk10.wav,326,Native,Arabic_QA,embeddings_cnn/326_chunk10.npy
369
+ data/processed_audio_chunks\719_chunk0.wav,719,Native,Arabic_QA,embeddings_cnn/719_chunk0.npy
370
+ data/processed_audio_chunks\719_chunk1.wav,719,Native,Arabic_QA,embeddings_cnn/719_chunk1.npy
371
+ data/processed_audio_chunks\719_chunk2.wav,719,Native,Arabic_QA,embeddings_cnn/719_chunk2.npy
372
+ data/processed_audio_chunks\605_chunk0.wav,605,Native,Arabic_SA,embeddings_cnn/605_chunk0.npy
373
+ data/processed_audio_chunks\605_chunk1.wav,605,Native,Arabic_SA,embeddings_cnn/605_chunk1.npy
374
+ data/processed_audio_chunks\605_chunk2.wav,605,Native,Arabic_SA,embeddings_cnn/605_chunk2.npy
375
+ data/processed_audio_chunks\605_chunk3.wav,605,Native,Arabic_SA,embeddings_cnn/605_chunk3.npy
376
+ data/processed_audio_chunks\605_chunk4.wav,605,Native,Arabic_SA,embeddings_cnn/605_chunk4.npy
377
+ data/processed_audio_chunks\500_chunk0.wav,500,Non-Native,Arabic_SA,embeddings_cnn/500_chunk0.npy
378
+ data/processed_audio_chunks\500_chunk1.wav,500,Non-Native,Arabic_SA,embeddings_cnn/500_chunk1.npy
379
+ data/processed_audio_chunks\500_chunk2.wav,500,Non-Native,Arabic_SA,embeddings_cnn/500_chunk2.npy
380
+ data/processed_audio_chunks\481_chunk0.wav,481,Non-Native,Arabic_SA,embeddings_cnn/481_chunk0.npy
381
+ data/processed_audio_chunks\481_chunk1.wav,481,Non-Native,Arabic_SA,embeddings_cnn/481_chunk1.npy
382
+ data/processed_audio_chunks\481_chunk2.wav,481,Non-Native,Arabic_SA,embeddings_cnn/481_chunk2.npy
383
+ data/processed_audio_chunks\481_chunk3.wav,481,Non-Native,Arabic_SA,embeddings_cnn/481_chunk3.npy
384
+ data/processed_audio_chunks\579_chunk0.wav,579,Native,Arabic_QA,embeddings_cnn/579_chunk0.npy
385
+ data/processed_audio_chunks\579_chunk1.wav,579,Native,Arabic_QA,embeddings_cnn/579_chunk1.npy
386
+ data/processed_audio_chunks\579_chunk2.wav,579,Native,Arabic_QA,embeddings_cnn/579_chunk2.npy
387
+ data/processed_audio_chunks\406_chunk0.wav,406,Native,Arabic_SA,embeddings_cnn/406_chunk0.npy
388
+ data/processed_audio_chunks\406_chunk1.wav,406,Native,Arabic_SA,embeddings_cnn/406_chunk1.npy
389
+ data/processed_audio_chunks\406_chunk2.wav,406,Native,Arabic_SA,embeddings_cnn/406_chunk2.npy
390
+ data/processed_audio_chunks\406_chunk3.wav,406,Native,Arabic_SA,embeddings_cnn/406_chunk3.npy
391
+ data/processed_audio_chunks\398_chunk0.wav,398,Native,Arabic_SA,embeddings_cnn/398_chunk0.npy
392
+ data/processed_audio_chunks\398_chunk1.wav,398,Native,Arabic_SA,embeddings_cnn/398_chunk1.npy
393
+ data/processed_audio_chunks\398_chunk2.wav,398,Native,Arabic_SA,embeddings_cnn/398_chunk2.npy
394
+ data/processed_audio_chunks\398_chunk3.wav,398,Native,Arabic_SA,embeddings_cnn/398_chunk3.npy
395
+ data/processed_audio_chunks\398_chunk4.wav,398,Native,Arabic_SA,embeddings_cnn/398_chunk4.npy
396
+ data/processed_audio_chunks\478_chunk0.wav,478,Native,Arabic_SA,embeddings_cnn/478_chunk0.npy
397
+ data/processed_audio_chunks\478_chunk1.wav,478,Native,Arabic_SA,embeddings_cnn/478_chunk1.npy
398
+ data/processed_audio_chunks\478_chunk2.wav,478,Native,Arabic_SA,embeddings_cnn/478_chunk2.npy
399
+ data/processed_audio_chunks\478_chunk3.wav,478,Native,Arabic_SA,embeddings_cnn/478_chunk3.npy
400
+ data/processed_audio_chunks\478_chunk4.wav,478,Native,Arabic_SA,embeddings_cnn/478_chunk4.npy
401
+ data/processed_audio_chunks\478_chunk5.wav,478,Native,Arabic_SA,embeddings_cnn/478_chunk5.npy
402
+ data/processed_audio_chunks\478_chunk6.wav,478,Native,Arabic_SA,embeddings_cnn/478_chunk6.npy
403
+ data/processed_audio_chunks\571_chunk0.wav,571,Non-Native,Arabic_SA,embeddings_cnn/571_chunk0.npy
404
+ data/processed_audio_chunks\571_chunk1.wav,571,Non-Native,Arabic_SA,embeddings_cnn/571_chunk1.npy
405
+ data/processed_audio_chunks\571_chunk2.wav,571,Non-Native,Arabic_SA,embeddings_cnn/571_chunk2.npy
406
+ data/processed_audio_chunks\571_chunk3.wav,571,Non-Native,Arabic_SA,embeddings_cnn/571_chunk3.npy
407
+ data/processed_audio_chunks\571_chunk4.wav,571,Non-Native,Arabic_SA,embeddings_cnn/571_chunk4.npy
408
+ data/processed_audio_chunks\377_chunk0.wav,377,Native,Arabic_QA,embeddings_cnn/377_chunk0.npy
409
+ data/processed_audio_chunks\377_chunk1.wav,377,Native,Arabic_QA,embeddings_cnn/377_chunk1.npy
410
+ data/processed_audio_chunks\377_chunk2.wav,377,Native,Arabic_QA,embeddings_cnn/377_chunk2.npy
411
+ data/processed_audio_chunks\377_chunk3.wav,377,Native,Arabic_QA,embeddings_cnn/377_chunk3.npy
412
+ data/processed_audio_chunks\560_chunk0.wav,560,Native,Arabic_AE,embeddings_cnn/560_chunk0.npy
413
+ data/processed_audio_chunks\560_chunk1.wav,560,Native,Arabic_AE,embeddings_cnn/560_chunk1.npy
414
+ data/processed_audio_chunks\560_chunk2.wav,560,Native,Arabic_AE,embeddings_cnn/560_chunk2.npy
415
+ data/processed_audio_chunks\560_chunk3.wav,560,Native,Arabic_AE,embeddings_cnn/560_chunk3.npy
416
+ data/processed_audio_chunks\574_chunk0.wav,574,Native,Arabic_SA,embeddings_cnn/574_chunk0.npy
417
+ data/processed_audio_chunks\574_chunk1.wav,574,Native,Arabic_SA,embeddings_cnn/574_chunk1.npy
418
+ data/processed_audio_chunks\574_chunk2.wav,574,Native,Arabic_SA,embeddings_cnn/574_chunk2.npy
419
+ data/processed_audio_chunks\655_chunk0.wav,655,Non-Native,Arabic_QA,embeddings_cnn/655_chunk0.npy
420
+ data/processed_audio_chunks\655_chunk1.wav,655,Non-Native,Arabic_QA,embeddings_cnn/655_chunk1.npy
421
+ data/processed_audio_chunks\655_chunk2.wav,655,Non-Native,Arabic_QA,embeddings_cnn/655_chunk2.npy
422
+ data/processed_audio_chunks\655_chunk3.wav,655,Non-Native,Arabic_QA,embeddings_cnn/655_chunk3.npy
423
+ data/processed_audio_chunks\363_chunk0.wav,363,Native,Arabic_AE,embeddings_cnn/363_chunk0.npy
424
+ data/processed_audio_chunks\363_chunk1.wav,363,Native,Arabic_AE,embeddings_cnn/363_chunk1.npy
425
+ data/processed_audio_chunks\363_chunk2.wav,363,Native,Arabic_AE,embeddings_cnn/363_chunk2.npy
426
+ data/processed_audio_chunks\487_chunk0.wav,487,Native,Arabic_AE,embeddings_cnn/487_chunk0.npy
427
+ data/processed_audio_chunks\487_chunk1.wav,487,Native,Arabic_AE,embeddings_cnn/487_chunk1.npy
428
+ data/processed_audio_chunks\487_chunk2.wav,487,Native,Arabic_AE,embeddings_cnn/487_chunk2.npy
429
+ data/processed_audio_chunks\617_chunk0.wav,617,Non-Native,Arabic_SA,embeddings_cnn/617_chunk0.npy
430
+ data/processed_audio_chunks\617_chunk1.wav,617,Non-Native,Arabic_SA,embeddings_cnn/617_chunk1.npy
431
+ data/processed_audio_chunks\617_chunk2.wav,617,Non-Native,Arabic_SA,embeddings_cnn/617_chunk2.npy
432
+ data/processed_audio_chunks\642_chunk0.wav,642,Native,Arabic_SA,embeddings_cnn/642_chunk0.npy
433
+ data/processed_audio_chunks\642_chunk1.wav,642,Native,Arabic_SA,embeddings_cnn/642_chunk1.npy
434
+ data/processed_audio_chunks\642_chunk2.wav,642,Native,Arabic_SA,embeddings_cnn/642_chunk2.npy
435
+ data/processed_audio_chunks\468_chunk0.wav,468,Non-Native,Arabic_SA,embeddings_cnn/468_chunk0.npy
436
+ data/processed_audio_chunks\468_chunk1.wav,468,Non-Native,Arabic_SA,embeddings_cnn/468_chunk1.npy
437
+ data/processed_audio_chunks\468_chunk2.wav,468,Non-Native,Arabic_SA,embeddings_cnn/468_chunk2.npy
438
+ data/processed_audio_chunks\468_chunk3.wav,468,Non-Native,Arabic_SA,embeddings_cnn/468_chunk3.npy
439
+ data/processed_audio_chunks\468_chunk4.wav,468,Non-Native,Arabic_SA,embeddings_cnn/468_chunk4.npy
440
+ data/processed_audio_chunks\468_chunk5.wav,468,Non-Native,Arabic_SA,embeddings_cnn/468_chunk5.npy
441
+ data/processed_audio_chunks\451_chunk0.wav,451,Non-Native,Arabic_SA,embeddings_cnn/451_chunk0.npy
442
+ data/processed_audio_chunks\451_chunk1.wav,451,Non-Native,Arabic_SA,embeddings_cnn/451_chunk1.npy
443
+ data/processed_audio_chunks\514_chunk0.wav,514,Native,Arabic_AE,embeddings_cnn/514_chunk0.npy
444
+ data/processed_audio_chunks\416_chunk0.wav,416,Non-Native,Arabic_SA,embeddings_cnn/416_chunk0.npy
445
+ data/processed_audio_chunks\416_chunk1.wav,416,Non-Native,Arabic_SA,embeddings_cnn/416_chunk1.npy
446
+ data/processed_audio_chunks\416_chunk2.wav,416,Non-Native,Arabic_SA,embeddings_cnn/416_chunk2.npy
447
+ data/processed_audio_chunks\566_chunk0.wav,566,Non-Native,Arabic_KW,embeddings_cnn/566_chunk0.npy
448
+ data/processed_audio_chunks\566_chunk1.wav,566,Non-Native,Arabic_KW,embeddings_cnn/566_chunk1.npy
449
+ data/processed_audio_chunks\566_chunk2.wav,566,Non-Native,Arabic_KW,embeddings_cnn/566_chunk2.npy
450
+ data/processed_audio_chunks\309_chunk0.wav,309,Native,Arabic_QA,embeddings_cnn/309_chunk0.npy
451
+ data/processed_audio_chunks\309_chunk1.wav,309,Native,Arabic_QA,embeddings_cnn/309_chunk1.npy
452
+ data/processed_audio_chunks\414_chunk0.wav,414,Non-Native,Arabic_SA,embeddings_cnn/414_chunk0.npy
453
+ data/processed_audio_chunks\414_chunk1.wav,414,Non-Native,Arabic_SA,embeddings_cnn/414_chunk1.npy
454
+ data/processed_audio_chunks\414_chunk2.wav,414,Non-Native,Arabic_SA,embeddings_cnn/414_chunk2.npy
455
+ data/processed_audio_chunks\464_chunk0.wav,464,Native,Arabic_QA,embeddings_cnn/464_chunk0.npy
456
+ data/processed_audio_chunks\464_chunk1.wav,464,Native,Arabic_QA,embeddings_cnn/464_chunk1.npy
457
+ data/processed_audio_chunks\464_chunk2.wav,464,Native,Arabic_QA,embeddings_cnn/464_chunk2.npy
458
+ data/processed_audio_chunks\431_chunk0.wav,431,Native,Arabic_AE,embeddings_cnn/431_chunk0.npy
459
+ data/processed_audio_chunks\431_chunk1.wav,431,Native,Arabic_AE,embeddings_cnn/431_chunk1.npy
460
+ data/processed_audio_chunks\431_chunk2.wav,431,Native,Arabic_AE,embeddings_cnn/431_chunk2.npy
461
+ data/processed_audio_chunks\431_chunk3.wav,431,Native,Arabic_AE,embeddings_cnn/431_chunk3.npy
462
+ data/processed_audio_chunks\420_chunk0.wav,420,Native,Arabic_AE,embeddings_cnn/420_chunk0.npy
463
+ data/processed_audio_chunks\420_chunk1.wav,420,Native,Arabic_AE,embeddings_cnn/420_chunk1.npy
464
+ data/processed_audio_chunks\420_chunk2.wav,420,Native,Arabic_AE,embeddings_cnn/420_chunk2.npy
465
+ data/processed_audio_chunks\420_chunk3.wav,420,Native,Arabic_AE,embeddings_cnn/420_chunk3.npy
466
+ data/processed_audio_chunks\373_chunk0.wav,373,Native,Arabic_SA,embeddings_cnn/373_chunk0.npy
467
+ data/processed_audio_chunks\373_chunk1.wav,373,Native,Arabic_SA,embeddings_cnn/373_chunk1.npy
468
+ data/processed_audio_chunks\405_chunk0.wav,405,Native,Arabic_AE,embeddings_cnn/405_chunk0.npy
469
+ data/processed_audio_chunks\405_chunk1.wav,405,Native,Arabic_AE,embeddings_cnn/405_chunk1.npy
470
+ data/processed_audio_chunks\405_chunk2.wav,405,Native,Arabic_AE,embeddings_cnn/405_chunk2.npy
471
+ data/processed_audio_chunks\405_chunk3.wav,405,Native,Arabic_AE,embeddings_cnn/405_chunk3.npy
472
+ data/processed_audio_chunks\522_chunk0.wav,522,Native,Arabic_QA,embeddings_cnn/522_chunk0.npy
473
+ data/processed_audio_chunks\522_chunk1.wav,522,Native,Arabic_QA,embeddings_cnn/522_chunk1.npy
474
+ data/processed_audio_chunks\522_chunk2.wav,522,Native,Arabic_QA,embeddings_cnn/522_chunk2.npy
475
+ data/processed_audio_chunks\522_chunk3.wav,522,Native,Arabic_QA,embeddings_cnn/522_chunk3.npy
476
+ data/processed_audio_chunks\522_chunk4.wav,522,Native,Arabic_QA,embeddings_cnn/522_chunk4.npy
477
+ data/processed_audio_chunks\522_chunk5.wav,522,Native,Arabic_QA,embeddings_cnn/522_chunk5.npy
478
+ data/processed_audio_chunks\530_chunk0.wav,530,Native,Arabic_AE,embeddings_cnn/530_chunk0.npy
479
+ data/processed_audio_chunks\530_chunk1.wav,530,Native,Arabic_AE,embeddings_cnn/530_chunk1.npy
480
+ data/processed_audio_chunks\530_chunk2.wav,530,Native,Arabic_AE,embeddings_cnn/530_chunk2.npy
481
+ data/processed_audio_chunks\530_chunk3.wav,530,Native,Arabic_AE,embeddings_cnn/530_chunk3.npy
482
+ data/processed_audio_chunks\628_chunk0.wav,628,Native,Arabic_SA,embeddings_cnn/628_chunk0.npy
483
+ data/processed_audio_chunks\628_chunk1.wav,628,Native,Arabic_SA,embeddings_cnn/628_chunk1.npy
484
+ data/processed_audio_chunks\628_chunk2.wav,628,Native,Arabic_SA,embeddings_cnn/628_chunk2.npy
485
+ data/processed_audio_chunks\628_chunk3.wav,628,Native,Arabic_SA,embeddings_cnn/628_chunk3.npy
486
+ data/processed_audio_chunks\609_chunk0.wav,609,Non-Native,Arabic_SA,embeddings_cnn/609_chunk0.npy
487
+ data/processed_audio_chunks\609_chunk1.wav,609,Non-Native,Arabic_SA,embeddings_cnn/609_chunk1.npy
488
+ data/processed_audio_chunks\609_chunk2.wav,609,Non-Native,Arabic_SA,embeddings_cnn/609_chunk2.npy
489
+ data/processed_audio_chunks\495_chunk0.wav,495,Native,Arabic_QA,embeddings_cnn/495_chunk0.npy
490
+ data/processed_audio_chunks\495_chunk1.wav,495,Native,Arabic_QA,embeddings_cnn/495_chunk1.npy
491
+ data/processed_audio_chunks\495_chunk2.wav,495,Native,Arabic_QA,embeddings_cnn/495_chunk2.npy
492
+ data/processed_audio_chunks\495_chunk3.wav,495,Native,Arabic_QA,embeddings_cnn/495_chunk3.npy
493
+ data/processed_audio_chunks\298_chunk0.wav,298,Native,Arabic_AE,embeddings_cnn/298_chunk0.npy
494
+ data/processed_audio_chunks\298_chunk1.wav,298,Native,Arabic_AE,embeddings_cnn/298_chunk1.npy
495
+ data/processed_audio_chunks\298_chunk2.wav,298,Native,Arabic_AE,embeddings_cnn/298_chunk2.npy
496
+ data/processed_audio_chunks\378_chunk0.wav,378,Native,Arabic_AE,embeddings_cnn/378_chunk0.npy
497
+ data/processed_audio_chunks\378_chunk1.wav,378,Native,Arabic_AE,embeddings_cnn/378_chunk1.npy
498
+ data/processed_audio_chunks\378_chunk2.wav,378,Native,Arabic_AE,embeddings_cnn/378_chunk2.npy
499
+ data/processed_audio_chunks\378_chunk3.wav,378,Native,Arabic_AE,embeddings_cnn/378_chunk3.npy
500
+ data/processed_audio_chunks\792_chunk0.wav,792,Native,Arabic_QA,embeddings_cnn/792_chunk0.npy
501
+ data/processed_audio_chunks\792_chunk1.wav,792,Native,Arabic_QA,embeddings_cnn/792_chunk1.npy
502
+ data/processed_audio_chunks\316_chunk0.wav,316,Native,Arabic_QA,embeddings_cnn/316_chunk0.npy
503
+ data/processed_audio_chunks\316_chunk1.wav,316,Native,Arabic_QA,embeddings_cnn/316_chunk1.npy
504
+ data/processed_audio_chunks\316_chunk2.wav,316,Native,Arabic_QA,embeddings_cnn/316_chunk2.npy
505
+ data/processed_audio_chunks\525_chunk0.wav,525,Native,Arabic_SA,embeddings_cnn/525_chunk0.npy
506
+ data/processed_audio_chunks\525_chunk1.wav,525,Native,Arabic_SA,embeddings_cnn/525_chunk1.npy
507
+ data/processed_audio_chunks\525_chunk2.wav,525,Native,Arabic_SA,embeddings_cnn/525_chunk2.npy
508
+ data/processed_audio_chunks\793_chunk0.wav,793,Native,Arabic_QA,embeddings_cnn/793_chunk0.npy
509
+ data/processed_audio_chunks\793_chunk1.wav,793,Native,Arabic_QA,embeddings_cnn/793_chunk1.npy
510
+ data/processed_audio_chunks\793_chunk2.wav,793,Native,Arabic_QA,embeddings_cnn/793_chunk2.npy
511
+ data/processed_audio_chunks\793_chunk3.wav,793,Native,Arabic_QA,embeddings_cnn/793_chunk3.npy
512
+ data/processed_audio_chunks\386_chunk0.wav,386,Native,Arabic_QA,embeddings_cnn/386_chunk0.npy
513
+ data/processed_audio_chunks\386_chunk1.wav,386,Native,Arabic_QA,embeddings_cnn/386_chunk1.npy
514
+ data/processed_audio_chunks\386_chunk2.wav,386,Native,Arabic_QA,embeddings_cnn/386_chunk2.npy
515
+ data/processed_audio_chunks\386_chunk3.wav,386,Native,Arabic_QA,embeddings_cnn/386_chunk3.npy
516
+ data/processed_audio_chunks\386_chunk4.wav,386,Native,Arabic_QA,embeddings_cnn/386_chunk4.npy
517
+ data/processed_audio_chunks\496_chunk0.wav,496,Native,Arabic_QA,embeddings_cnn/496_chunk0.npy
518
+ data/processed_audio_chunks\496_chunk1.wav,496,Native,Arabic_QA,embeddings_cnn/496_chunk1.npy
519
+ data/processed_audio_chunks\496_chunk2.wav,496,Native,Arabic_QA,embeddings_cnn/496_chunk2.npy
520
+ data/processed_audio_chunks\496_chunk3.wav,496,Native,Arabic_QA,embeddings_cnn/496_chunk3.npy
521
+ data/processed_audio_chunks\433_chunk0.wav,433,Native,Arabic_AE,embeddings_cnn/433_chunk0.npy
522
+ data/processed_audio_chunks\433_chunk1.wav,433,Native,Arabic_AE,embeddings_cnn/433_chunk1.npy
523
+ data/processed_audio_chunks\433_chunk2.wav,433,Native,Arabic_AE,embeddings_cnn/433_chunk2.npy
524
+ data/processed_audio_chunks\480_chunk0.wav,480,Native,Arabic_AE,embeddings_cnn/480_chunk0.npy
525
+ data/processed_audio_chunks\480_chunk1.wav,480,Native,Arabic_AE,embeddings_cnn/480_chunk1.npy
526
+ data/processed_audio_chunks\480_chunk2.wav,480,Native,Arabic_AE,embeddings_cnn/480_chunk2.npy
527
+ data/processed_audio_chunks\552_chunk0.wav,552,Non-Native,Arabic_SA,embeddings_cnn/552_chunk0.npy
528
+ data/processed_audio_chunks\552_chunk1.wav,552,Non-Native,Arabic_SA,embeddings_cnn/552_chunk1.npy
529
+ data/processed_audio_chunks\552_chunk2.wav,552,Non-Native,Arabic_SA,embeddings_cnn/552_chunk2.npy
530
+ data/processed_audio_chunks\552_chunk3.wav,552,Non-Native,Arabic_SA,embeddings_cnn/552_chunk3.npy
531
+ data/processed_audio_chunks\552_chunk4.wav,552,Non-Native,Arabic_SA,embeddings_cnn/552_chunk4.npy
532
+ data/processed_audio_chunks\588_chunk0.wav,588,Native,Arabic_AE,embeddings_cnn/588_chunk0.npy
533
+ data/processed_audio_chunks\588_chunk1.wav,588,Native,Arabic_AE,embeddings_cnn/588_chunk1.npy
534
+ data/processed_audio_chunks\588_chunk2.wav,588,Native,Arabic_AE,embeddings_cnn/588_chunk2.npy
535
+ data/processed_audio_chunks\588_chunk3.wav,588,Native,Arabic_AE,embeddings_cnn/588_chunk3.npy
536
+ data/processed_audio_chunks\588_chunk4.wav,588,Native,Arabic_AE,embeddings_cnn/588_chunk4.npy
537
+ data/processed_audio_chunks\441_chunk0.wav,441,Native,Arabic_SA,embeddings_cnn/441_chunk0.npy
538
+ data/processed_audio_chunks\441_chunk1.wav,441,Native,Arabic_SA,embeddings_cnn/441_chunk1.npy
539
+ data/processed_audio_chunks\441_chunk2.wav,441,Native,Arabic_SA,embeddings_cnn/441_chunk2.npy
540
+ data/processed_audio_chunks\441_chunk3.wav,441,Native,Arabic_SA,embeddings_cnn/441_chunk3.npy
541
+ data/processed_audio_chunks\441_chunk4.wav,441,Native,Arabic_SA,embeddings_cnn/441_chunk4.npy
542
+ data/processed_audio_chunks\446_chunk0.wav,446,Native,Arabic_QA,embeddings_cnn/446_chunk0.npy
543
+ data/processed_audio_chunks\446_chunk1.wav,446,Native,Arabic_QA,embeddings_cnn/446_chunk1.npy
544
+ data/processed_audio_chunks\446_chunk2.wav,446,Native,Arabic_QA,embeddings_cnn/446_chunk2.npy
545
+ data/processed_audio_chunks\446_chunk3.wav,446,Native,Arabic_QA,embeddings_cnn/446_chunk3.npy
manifests/embedding_dim.txt ADDED
@@ -0,0 +1 @@
 
 
1
+ 2051
manifests/embedding_dim_cnn.txt ADDED
@@ -0,0 +1 @@
 
 
1
+ 1024
manifests/final_training_manifest.csv CHANGED
The diff for this file is too large to render. See raw diff
 
manifests/final_training_manifest_english.csv DELETED
The diff for this file is too large to render. See raw diff
 
manifests/processed_manifest.csv CHANGED
The diff for this file is too large to render. See raw diff
 
manifests/processed_manifest_english.csv DELETED
The diff for this file is too large to render. See raw diff
 
requirements.txt CHANGED
@@ -11,3 +11,5 @@ transformers>=4.30.0
11
  requests>=2.28.0
12
  tqdm>=4.65.0
13
  speechbrain>=1.0.0
 
 
 
11
  requests>=2.28.0
12
  tqdm>=4.65.0
13
  speechbrain>=1.0.0
14
+ lightgbm>=4.0.0
15
+ imbalanced-learn>=0.11.0
scripts/__pycache__/model.cpython-310.pyc CHANGED
Binary files a/scripts/__pycache__/model.cpython-310.pyc and b/scripts/__pycache__/model.cpython-310.pyc differ
 
scripts/compare_cnn_on_test.py ADDED
@@ -0,0 +1,133 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """
2
+ Run 1D-CNN on test_data.csv; report metrics. Uses same test CSV and nativity_status as compare_models_on_test.
3
+ """
4
+ import os
5
+ import sys
6
+ import requests
7
+ import io
8
+ import numpy as np
9
+ import torch
10
+ import librosa
11
+ from tqdm import tqdm
12
+ from sklearn.metrics import accuracy_score, f1_score, precision_score, recall_score, confusion_matrix
13
+
14
+ sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
15
+ from train1DCNN import NativityCNN1D, MAX_TIME_STEPS, INPUT_DIM, DROPOUT
16
+ from extract_embeddings_for_CNN import get_embeddings_from_audio
17
+
18
+ device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
19
+ CHUNK_SIZE = 160000 # 10s at 16 kHz
20
+ BEST_THRESHOLD_PATH = "models/best_threshold.txt"
21
+ CNN_MODEL_PATH = "models/nativity_cnn1d_model.pth"
22
+
23
+
24
+ def load_audio(url):
25
+ try:
26
+ r = requests.get(url, timeout=15)
27
+ r.raise_for_status()
28
+ raw = r.content
29
+ if not raw or len(raw) < 100:
30
+ return None
31
+ ct = (r.headers.get("Content-Type") or "").lower()
32
+ if "text/html" in ct or raw.lstrip()[:50].startswith(b"<!") or raw.lstrip()[:20].startswith(b"<?"):
33
+ return None
34
+ y, _ = librosa.load(io.BytesIO(raw), sr=16000, mono=True)
35
+ return y
36
+ except Exception:
37
+ return None
38
+
39
+
40
+ def pad_or_truncate(emb, max_steps=MAX_TIME_STEPS):
41
+ """emb: (T, 1024). Return (max_steps, 1024)."""
42
+ T, F = emb.shape
43
+ if T < max_steps:
44
+ return np.pad(emb, ((0, max_steps - T), (0, 0)), mode="constant", constant_values=0).astype(np.float32)
45
+ return emb[:max_steps, :].astype(np.float32)
46
+
47
+
48
+ def main():
49
+ import pandas as pd
50
+
51
+ df = pd.read_csv("manifests/test_data.csv")
52
+ if "nativity_status" not in df.columns:
53
+ print("test_data.csv must have 'nativity_status' column")
54
+ return
55
+
56
+ model = NativityCNN1D(input_dim=INPUT_DIM, dropout=DROPOUT).to(device)
57
+ if not os.path.isfile(CNN_MODEL_PATH):
58
+ print(f"CNN model not found: {CNN_MODEL_PATH}. Run train1DCNN.py first.")
59
+ return
60
+ model.load_state_dict(torch.load(CNN_MODEL_PATH, map_location=device))
61
+ model.eval()
62
+
63
+ threshold = 0.55
64
+ if os.path.isfile(BEST_THRESHOLD_PATH):
65
+ with open(BEST_THRESHOLD_PATH) as f:
66
+ threshold = float(f.read().strip())
67
+ print(f"Using threshold: {threshold}")
68
+
69
+ preds = []
70
+ valid_idx = []
71
+ skipped = 0
72
+ for idx, row in tqdm(df.iterrows(), total=len(df), desc="CNN on test"):
73
+ y = load_audio(row["audio_url"])
74
+ if y is None or len(y) == 0:
75
+ skipped += 1
76
+ continue
77
+ chunks = [
78
+ y[i : i + CHUNK_SIZE]
79
+ for i in range(0, len(y), CHUNK_SIZE)
80
+ if len(y[i : i + CHUNK_SIZE]) == CHUNK_SIZE
81
+ ]
82
+ if not chunks:
83
+ skipped += 1
84
+ continue
85
+ seqs = []
86
+ for c in chunks:
87
+ emb = get_embeddings_from_audio(c, sr=16000)
88
+ if emb is None:
89
+ continue
90
+ seqs.append(pad_or_truncate(emb))
91
+ if not seqs:
92
+ skipped += 1
93
+ continue
94
+ seqs = np.stack(seqs, axis=0) # (N_chunks, 150, 1024)
95
+ with torch.no_grad():
96
+ x = torch.tensor(seqs, dtype=torch.float32).to(device)
97
+ logits = model(x)
98
+ probs = torch.softmax(logits, dim=1).cpu().numpy()
99
+ p1 = probs[:, 1].mean()
100
+ preds.append(1 if p1 >= threshold else 0)
101
+ valid_idx.append(idx)
102
+
103
+ if not valid_idx:
104
+ print("No valid test samples.")
105
+ return
106
+ if skipped > 0:
107
+ print(f"Skipped {skipped} URL(s).")
108
+
109
+ y_true = np.array([1 if df.loc[i, "nativity_status"] == "Non-Native" else 0 for i in valid_idx])
110
+ preds = np.array(preds)
111
+ acc = accuracy_score(y_true, preds)
112
+ f1 = f1_score(y_true, preds, average="macro", zero_division=0)
113
+ prec = precision_score(y_true, preds, average="macro", zero_division=0)
114
+ rec = recall_score(y_true, preds, average="macro", zero_division=0)
115
+ cm = confusion_matrix(y_true, preds)
116
+
117
+ print("\n" + "=" * 60)
118
+ print("1D-CNN ON TEST DATA")
119
+ print("=" * 60)
120
+ print(f"Test samples: {len(y_true)} (Native=0, Non-Native=1)")
121
+ print(f"Accuracy: {acc:.4f}")
122
+ print(f"F1 macro: {f1:.4f}")
123
+ print(f"Precision: {prec:.4f}")
124
+ print(f"Recall: {rec:.4f}")
125
+ print(f"Confusion matrix:\n{cm}")
126
+ # Write one-line summary for later parsing
127
+ with open("models/cnn_test_metrics.txt", "w") as f:
128
+ f.write(f"accuracy={acc:.4f} f1={f1:.4f} precision={prec:.4f} recall={rec:.4f}\n")
129
+ print("\nWrote models/cnn_test_metrics.txt")
130
+
131
+
132
+ if __name__ == "__main__":
133
+ main()
scripts/compare_models_on_test.py ADDED
@@ -0,0 +1,245 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """
2
+ Run MLP and MLP+attention on test_data.csv; report metrics for both + ensemble.
3
+ Requires: manifests/test_data.csv with nativity_status, both models in models/
4
+ """
5
+ import os
6
+ import sys
7
+ import pandas as pd
8
+ import numpy as np
9
+ import torch
10
+ import librosa
11
+ import requests
12
+ import io
13
+ from transformers import Wav2Vec2Model, AutoFeatureExtractor
14
+ from tqdm import tqdm
15
+ from sklearn.metrics import accuracy_score, f1_score, precision_score, recall_score, confusion_matrix
16
+
17
+ sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
18
+ from model import NativityClassifier, NativityClassifierWithAttention
19
+ from extract_embeddings import get_mfcc_from_audio, get_pitch_stats, PROSODY_EMBED_DIM, EMBED_DIM_WITH_MFCC
20
+
21
+ device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
22
+ input_csv = "manifests/test_data.csv"
23
+ HIDDEN_DIMS = (384, 256, 128)
24
+ DROPOUT = 0.35
25
+ EVAL_THRESHOLD_DEFAULT = 0.35 # fallback if best_threshold.txt missing
26
+ PROSODY_LAYERS = (4, 6, 8, 10)
27
+ BEST_THRESHOLD_PATH = "models/best_threshold.txt"
28
+ CHUNK_SIZE = 160000
29
+ EMBED_DIM_FILE = "manifests/embedding_dim.txt"
30
+ THRESHOLD_SWEEP_TEST = [0.25, 0.28, 0.30, 0.32] # report test F1 at these thresholds
31
+
32
+ # Load wav2vec once
33
+ model_name = "facebook/wav2vec2-large-xlsr-53"
34
+ feature_extractor = AutoFeatureExtractor.from_pretrained(model_name)
35
+ wav2vec = Wav2Vec2Model.from_pretrained(model_name).to(device)
36
+ wav2vec.eval()
37
+
38
+
39
+ def get_embedding(audio_np, embed_dim=PROSODY_EMBED_DIM):
40
+ """Return 2051-d (mean+std+pitch) or 2090-d (+ MFCC) to match training."""
41
+ with torch.no_grad():
42
+ input_values = feature_extractor(
43
+ audio_np, return_tensors="pt", sampling_rate=16000
44
+ ).input_values.to(device)
45
+ outputs = wav2vec(input_values, output_hidden_states=True)
46
+ h = outputs.hidden_states
47
+ num_layers = len(h) - 1
48
+ layer_idx = [min(max(l, 1), num_layers) for l in PROSODY_LAYERS]
49
+ embs = [h[i] for i in layer_idx]
50
+ time_seq = torch.stack(embs, dim=0).mean(dim=0).squeeze(0) # (T, 1024)
51
+ emb_mean = time_seq.mean(dim=0).cpu().numpy().astype(np.float32)
52
+ emb_std = time_seq.std(dim=0).cpu().numpy().astype(np.float32)
53
+ np.nan_to_num(emb_std, copy=False, nan=0.0, posinf=0.0, neginf=0.0)
54
+ pitch = get_pitch_stats(audio_np, sr=16000)
55
+ out = np.concatenate([emb_mean, emb_std, pitch]).astype(np.float32)
56
+ if embed_dim == EMBED_DIM_WITH_MFCC:
57
+ mfcc_39 = get_mfcc_from_audio(audio_np, sr=16000)
58
+ out = np.concatenate([out, mfcc_39]).astype(np.float32)
59
+ n = np.linalg.norm(out)
60
+ if n > 1e-8:
61
+ out = out / n
62
+ return out
63
+
64
+
65
+ def load_audio(url):
66
+ """Load audio from URL; return None if URL returns HTML or unreadable format."""
67
+ try:
68
+ r = requests.get(url, timeout=15)
69
+ r.raise_for_status()
70
+ raw = r.content
71
+ if not raw or len(raw) < 100:
72
+ return None
73
+ ct = (r.headers.get("Content-Type") or "").lower()
74
+ if "text/html" in ct or raw.lstrip()[:50].startswith(b"<!") or raw.lstrip()[:20].startswith(b"<?"):
75
+ return None
76
+ buf = io.BytesIO(raw)
77
+ y, _ = librosa.load(buf, sr=16000, mono=True)
78
+ return y
79
+ except Exception as e:
80
+ return None
81
+
82
+
83
+ def predict_file(embs, model_mlp, model_att, lgb_model, threshold):
84
+ """embs: (N, D) with D=2051 or 2090. Returns (pred_mlp, pred_att, pred_ens, pred_lgb). Ensemble = avg of MLP + attention + LGB (if present)."""
85
+ with torch.no_grad():
86
+ embs_t = torch.tensor(embs, dtype=torch.float32).to(device)
87
+ logits_mlp = model_mlp(embs_t)
88
+ logits_att = model_att(embs_t)
89
+ prob_mlp = torch.softmax(logits_mlp, dim=1).cpu().numpy()
90
+ prob_att = torch.softmax(logits_att, dim=1).cpu().numpy()
91
+ p1_mlp = prob_mlp[:, 1].mean()
92
+ p1_att = prob_att[:, 1].mean()
93
+ pred_mlp = 1 if p1_mlp >= threshold else 0
94
+ pred_att = 1 if p1_att >= threshold else 0
95
+
96
+ mean_emb = embs.mean(axis=0).astype(np.float32).reshape(1, -1)
97
+ p1_lgb = None
98
+ pred_lgb = None
99
+ if lgb_model is not None:
100
+ prob_lgb = lgb_model.predict(mean_emb) # (1, 2)
101
+ p1_lgb = float(prob_lgb[0, 1])
102
+ pred_lgb = 1 if p1_lgb >= threshold else 0
103
+
104
+ probs_list = [p1_mlp, p1_att]
105
+ if p1_lgb is not None:
106
+ probs_list.append(p1_lgb)
107
+ p1_ens = np.mean(probs_list)
108
+ pred_ens = 1 if p1_ens >= threshold else 0
109
+ return pred_mlp, pred_att, pred_ens, pred_lgb, p1_ens
110
+
111
+
112
+ def main():
113
+ df = pd.read_csv(input_csv)
114
+ if "nativity_status" not in df.columns:
115
+ print("test_data.csv must have 'nativity_status' column")
116
+ return
117
+
118
+ # Match training: 2051 (prosody) or 2090 (prosody+MFCC)
119
+ embed_dim = PROSODY_EMBED_DIM
120
+ if os.path.isfile(EMBED_DIM_FILE):
121
+ with open(EMBED_DIM_FILE) as f:
122
+ embed_dim = int(f.read().strip())
123
+ print(f"Embedding dimension: {embed_dim}")
124
+
125
+ model_mlp = NativityClassifier(
126
+ input_dim=embed_dim, hidden_dims=HIDDEN_DIMS, output_dim=2, dropout=DROPOUT
127
+ ).to(device)
128
+ model_mlp.load_state_dict(torch.load("models/nativity_model.pth", map_location=device))
129
+ model_mlp.eval()
130
+
131
+ model_att = NativityClassifierWithAttention(
132
+ input_dim=embed_dim, hidden_dims=HIDDEN_DIMS, output_dim=2, dropout=DROPOUT,
133
+ final_dropout=0.65, reduction_ratio=8
134
+ ).to(device)
135
+ model_att.load_state_dict(torch.load("models/nativity_model_attention.pth", map_location=device))
136
+ model_att.eval()
137
+
138
+ # Best threshold from validation (saved by train_classifier_compare.py)
139
+ threshold = EVAL_THRESHOLD_DEFAULT
140
+ if os.path.isfile(BEST_THRESHOLD_PATH):
141
+ with open(BEST_THRESHOLD_PATH) as f:
142
+ threshold = float(f.read().strip())
143
+ print(f"Using threshold from validation: {threshold}")
144
+ else:
145
+ print(f"Using default threshold: {threshold}")
146
+
147
+ lgb_model = None
148
+ if os.path.isfile("models/lgbm_nativity.txt"):
149
+ try:
150
+ import lightgbm as lgb
151
+ lgb_model = lgb.Booster(model_file="models/lgbm_nativity.txt")
152
+ except Exception:
153
+ pass
154
+
155
+ preds_mlp, preds_att, preds_ens, preds_lgb, preds_probs_ens = [], [], [], [], []
156
+ valid_idx = []
157
+ skipped = 0
158
+ print("Running on test set...")
159
+ for idx, row in tqdm(df.iterrows(), total=len(df)):
160
+ y = load_audio(row["audio_url"])
161
+ if y is None or len(y) == 0:
162
+ skipped += 1
163
+ continue
164
+ chunks = [
165
+ y[i : i + CHUNK_SIZE]
166
+ for i in range(0, len(y), CHUNK_SIZE)
167
+ if len(y[i : i + CHUNK_SIZE]) == CHUNK_SIZE
168
+ ]
169
+ if not chunks:
170
+ continue
171
+ embs = np.stack([get_embedding(c, embed_dim=embed_dim) for c in chunks])
172
+ p_mlp, p_att, p_ens, p_lgb, p1_ens = predict_file(embs, model_mlp, model_att, lgb_model, threshold)
173
+ preds_mlp.append(p_mlp)
174
+ preds_att.append(p_att)
175
+ preds_ens.append(p_ens)
176
+ preds_probs_ens.append(p1_ens)
177
+ if p_lgb is not None:
178
+ preds_lgb.append(p_lgb)
179
+ valid_idx.append(idx)
180
+
181
+ if not valid_idx:
182
+ print("No valid test samples.")
183
+ return
184
+ if skipped > 0:
185
+ print(f"Skipped {skipped} URL(s) (non-audio or load failed).")
186
+
187
+ y_true = np.array([1 if df.loc[i, "nativity_status"] == "Non-Native" else 0 for i in valid_idx])
188
+ preds_mlp = np.array(preds_mlp)
189
+ preds_att = np.array(preds_att)
190
+ preds_ens = np.array(preds_ens)
191
+ probs_ens = np.array(preds_probs_ens)
192
+ preds_lgb = np.array(preds_lgb) if len(preds_lgb) == len(y_true) else None
193
+
194
+ n_ens = 2 + (1 if preds_lgb is not None else 0)
195
+
196
+ def row(name, pred):
197
+ return {
198
+ "name": name,
199
+ "acc": accuracy_score(y_true, pred),
200
+ "f1": f1_score(y_true, pred, average="macro", zero_division=0),
201
+ "prec": precision_score(y_true, pred, average="macro", zero_division=0),
202
+ "rec": recall_score(y_true, pred, average="macro", zero_division=0),
203
+ "cm": confusion_matrix(y_true, pred),
204
+ }
205
+
206
+ results = [
207
+ row("MLP (baseline)", preds_mlp),
208
+ row("MLP + attention (prosody)", preds_att),
209
+ row(f"Ensemble ({n_ens} models, t={threshold})", preds_ens),
210
+ ]
211
+ if preds_lgb is not None:
212
+ results.append(row("LightGBM", preds_lgb))
213
+
214
+ print("\n" + "=" * 60)
215
+ print("COMPARISON ON TEST DATA")
216
+ print("=" * 60)
217
+ print(f"Test samples: {len(y_true)} (Native=0, Non-Native=1)")
218
+ print(f"{'Model':<30} {'Accuracy':>10} {'F1 macro':>10} {'Precision':>10} {'Recall':>10}")
219
+ print("-" * 72)
220
+ for r in results:
221
+ print(f"{r['name']:<30} {r['acc']:>10.4f} {r['f1']:>10.4f} {r['prec']:>10.4f} {r['rec']:>10.4f}")
222
+ print("\nConfusion matrices (rows=true, cols=pred):")
223
+ for r in results:
224
+ print(f" {r['name']}:")
225
+ print(f" {r['cm']}")
226
+ best = max(results, key=lambda x: x["f1"])
227
+ print(f"\nBest on test (F1): {best['name']} (F1={best['f1']:.4f})")
228
+
229
+ # Threshold sweep on test (ensemble): see which threshold gives best test F1
230
+ print("\nEnsemble threshold sweep on test:")
231
+ print(f" {'t':>6} {'Accuracy':>10} {'F1 macro':>10}")
232
+ best_t, best_t_f1 = threshold, 0.0
233
+ for t in THRESHOLD_SWEEP_TEST:
234
+ pred_t = (probs_ens >= t).astype(int)
235
+ acc_t = accuracy_score(y_true, pred_t)
236
+ f1_t = f1_score(y_true, pred_t, average="macro", zero_division=0)
237
+ print(f" {t:>6.2f} {acc_t:>10.4f} {f1_t:>10.4f}")
238
+ if f1_t > best_t_f1:
239
+ best_t_f1 = f1_t
240
+ best_t = t
241
+ print(f" Best on test: t={best_t} (F1={best_t_f1:.4f})")
242
+
243
+
244
+ if __name__ == "__main__":
245
+ main()
scripts/extract_embeddings.py CHANGED
@@ -1,3 +1,9 @@
 
 
 
 
 
 
1
  import pandas as pd
2
  import numpy as np
3
  import torch
@@ -5,63 +11,208 @@ import librosa
5
  import os
6
  from transformers import Wav2Vec2Model, AutoFeatureExtractor
7
  from tqdm import tqdm
 
8
 
9
- # Load Pretrained Wav2Vec 2.0 (multilingual XLSR, includes Arabic)
10
- model_name = "facebook/wav2vec2-large-xlsr-53"
11
- feature_extractor = AutoFeatureExtractor.from_pretrained(model_name)
12
- model = Wav2Vec2Model.from_pretrained(model_name)
13
- model.eval() # Set to evaluation mode
 
14
 
15
- # Move to GPU if available
 
 
16
  device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
17
- model.to(device)
18
 
19
 
20
- def get_embeddings(audio_path, l2_norm=True):
21
- """Extract wav2vec embedding; optional L2 normalize for stable training."""
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
22
  try:
23
  speech, sr = librosa.load(audio_path, sr=16000)
 
 
 
 
 
 
 
 
 
24
  input_values = feature_extractor(
25
- speech, return_tensors="pt", sampling_rate=16000
26
- ).input_values
27
- input_values = input_values.to(device)
28
  with torch.no_grad():
29
- outputs = model(input_values)
30
- emb = torch.mean(outputs.last_hidden_state, dim=1).squeeze().cpu().numpy()
 
 
 
 
 
 
 
 
 
 
 
 
 
31
  if l2_norm:
32
  n = np.linalg.norm(emb)
33
  if n > 1e-8:
34
  emb = emb / n
 
 
 
 
 
35
  return emb
 
36
  except Exception as e:
37
  print(f"Error on {audio_path}: {e}")
38
  return None
39
 
40
 
41
- # Main processing loop
42
- manifest_path = "manifests/processed_manifest.csv"
43
- output_folder = "embeddings"
44
- os.makedirs(output_folder, exist_ok=True)
45
-
46
- df = pd.read_csv(manifest_path)
47
- embedding_paths = []
48
 
49
- print(f"Extracting embeddings for {len(df)} chunks (L2-normalized)...")
50
- for idx, row in tqdm(df.iterrows(), total=len(df)):
51
- emb = get_embeddings(row["chunk_path"], l2_norm=True)
52
-
53
- if emb is not None:
54
- # Save as individual numpy files to save RAM
55
- emb_filename = os.path.basename(row["chunk_path"]).replace(".wav", ".npy")
56
- emb_path = os.path.join(output_folder, emb_filename)
57
- np.save(emb_path, emb)
58
- embedding_paths.append(emb_path)
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
59
  else:
60
- embedding_paths.append(None)
61
-
62
- # Update manifest with paths to the new embeddings
63
- df["embedding_path"] = embedding_paths
64
- df.dropna(subset=["embedding_path"], inplace=True)
65
- df.to_csv("manifests/final_training_manifest.csv", index=False)
 
 
 
 
 
66
 
67
- print("Done! Embeddings saved and manifest updated.")
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """
2
+ Extract embeddings for Arabic Native vs Non-Native classifier.
3
+ Emphasizes PROSODIC features: pitch, rhythm, timing, fluency — key for Native vs Non-Native.
4
+ Uses XLS-R 53 EARLY-TO-MID layers (4,6,8,10): prosody/speaker-rich, less lexical than final layer.
5
+ """
6
+
7
  import pandas as pd
8
  import numpy as np
9
  import torch
 
11
  import os
12
  from transformers import Wav2Vec2Model, AutoFeatureExtractor
13
  from tqdm import tqdm
14
+ from scipy.signal import butter, lfilter
15
 
16
+ # XLS-R 53 (multilingual, includes Arabic)
17
+ # model_name = "facebook/wav2vec2-large-xlsr-53"
18
+ model_name = "facebook/wav2vec2-lv-60-espeak-cv-ft"
19
+ # Prosody-focused layers: early-mid (4,6,8,10) encode pitch, rhythm, timing, speaker
20
+ # Final layers encode lexicon; we want prosodic cues for nativeness
21
+ PROSODY_LAYERS = (4, 6, 8, 10)
22
 
23
+ # Lazy-loaded so "from extract_embeddings import get_mfcc_from_audio, ..." doesn't load the big model (avoids 2x GPU OOM in compare_models_on_test)
24
+ _feature_extractor = None
25
+ _model = None
26
  device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
 
27
 
28
 
29
+ def apply_whisper_filter(audio_array, sr=16000, cutoff_hz=700):
30
+ """
31
+ Applies a High-Pass Butterworth filter to strip fundamental pitch
32
+ and isolate high-frequency phonetic articulation.
33
+ """
34
+ nyq = 0.5 * sr
35
+ normal_cutoff = cutoff_hz / nyq
36
+ b, a = butter(5, normal_cutoff, btype="high", analog=False)
37
+ whispered_audio = lfilter(b, a, audio_array)
38
+ return whispered_audio.astype(np.float32)
39
+
40
+
41
+ def _get_wav2vec():
42
+ global _feature_extractor, _model
43
+ if _model is None:
44
+ _feature_extractor = AutoFeatureExtractor.from_pretrained(model_name)
45
+ _model = Wav2Vec2Model.from_pretrained(model_name)
46
+ _model.eval()
47
+ _model.to(device)
48
+ return _feature_extractor, _model
49
+
50
+
51
+ # Embedding dims: paper uses mean + std + F0 → 1024+1024+3 = 2051; optional MFCC → 2090
52
+ WAV2VEC_DIM = 1024
53
+ PITCH_DIM = 3 # mean, std, max of F0 (voiced)
54
+ PROSODY_EMBED_DIM = WAV2VEC_DIM * 2 + PITCH_DIM # 2051 (mean + std + pitch)
55
+ MFCC_DIM = 39 # 13 + 13 + 13
56
+ EMBED_DIM_WITH_MFCC = PROSODY_EMBED_DIM + MFCC_DIM # 2090
57
+
58
+
59
+ def get_pitch_stats(speech, sr=16000, fmin=50, fmax=500):
60
+ """F0 (pitch) stats for prosody: mean, std, max over voiced frames (paper: explicit pitch helps dialect/nativity)."""
61
+ try:
62
+ f0 = librosa.yin(
63
+ speech.astype(np.float64), fmin=float(fmin), fmax=float(fmax), sr=sr
64
+ )
65
+ voiced = (f0 > 0) & np.isfinite(f0)
66
+ if np.any(voiced):
67
+ f0_v = f0[voiced]
68
+ return np.array(
69
+ [float(np.mean(f0_v)), float(np.std(f0_v)), float(np.max(f0_v))],
70
+ dtype=np.float32,
71
+ )
72
+ except Exception:
73
+ pass
74
+ return np.zeros(PITCH_DIM, dtype=np.float32)
75
+
76
+
77
+ def get_mfcc_from_audio(speech, sr=16000, n_mfcc=13):
78
+ """Classical 39-d feature: MFCC mean + delta + delta2 (paper: MFCC >> LPCC/PLP for accent)."""
79
+ mfcc = librosa.feature.mfcc(y=speech, sr=sr, n_mfcc=n_mfcc)
80
+ mfcc_mean = mfcc.mean(axis=1)
81
+ mfcc_delta = librosa.feature.delta(mfcc).mean(axis=1)
82
+ mfcc_delta2 = librosa.feature.delta(mfcc, order=2).mean(axis=1)
83
+ return np.concatenate([mfcc_mean, mfcc_delta, mfcc_delta2]).astype(np.float32)
84
+
85
+
86
+ def get_embeddings(audio_path, l2_norm=True, return_audio=False):
87
+ """Prosody embedding: wav2vec mean + std (temporal variance) + F0 stats (pitch) → 2051-d.
88
+ Paper: explicit pitch and time-derivatives (std) help dialect/nativity. Optional return_audio for MFCC."""
89
+ feature_extractor, model = _get_wav2vec()
90
  try:
91
  speech, sr = librosa.load(audio_path, sr=16000)
92
+
93
+ # 1. Calculate REAL pitch from the unfiltered audio
94
+ pitch = get_pitch_stats(speech, sr=sr)
95
+
96
+ # 2. Apply WHISPER FILTER to blind the model to speaker identity
97
+ # whispered_speech = apply_whisper_filter(speech, sr=sr)
98
+ whispered_speech = speech
99
+
100
+ # 3. Pass the WHISPERED speech to Wav2Vec2
101
  input_values = feature_extractor(
102
+ whispered_speech, return_tensors="pt", sampling_rate=16000
103
+ ).input_values.to(device)
104
+
105
  with torch.no_grad():
106
+ outputs = model(input_values, output_hidden_states=True)
107
+
108
+ h = outputs.hidden_states
109
+ num_layers = len(h) - 1
110
+ layer_idx = [min(max(l, 1), num_layers) for l in PROSODY_LAYERS]
111
+ embs = [h[i] for i in layer_idx]
112
+
113
+ # (1, T, 1024) after stack+mean over layers; keep time for mean+std
114
+ time_seq = torch.stack(embs, dim=0).mean(dim=0).squeeze(0) # (T, 1024)
115
+ emb_mean = time_seq.mean(dim=0).cpu().numpy().astype(np.float32)
116
+ emb_std = time_seq.std(dim=0).cpu().numpy().astype(np.float32)
117
+ np.nan_to_num(emb_std, copy=False, nan=0.0, posinf=0.0, neginf=0.0)
118
+
119
+ emb = np.concatenate([emb_mean, emb_std, pitch])
120
+
121
  if l2_norm:
122
  n = np.linalg.norm(emb)
123
  if n > 1e-8:
124
  emb = emb / n
125
+
126
+ if return_audio:
127
+ # Return the WHISPERED speech so the MFCC extractor also uses the filtered version
128
+ return emb, whispered_speech, sr
129
+
130
  return emb
131
+
132
  except Exception as e:
133
  print(f"Error on {audio_path}: {e}")
134
  return None
135
 
136
 
137
+ # CLI: optional (manifest_in, embedding_dir, manifest_out) for e.g. Iqra data
138
+ if __name__ == "__main__":
139
+ import argparse
 
 
 
 
140
 
141
+ ap = argparse.ArgumentParser(
142
+ description="Extract XLS-R prosody embeddings from chunk manifest."
143
+ )
144
+ ap.add_argument(
145
+ "--manifest",
146
+ default="manifests/processed_manifest.csv",
147
+ help="Input manifest CSV (chunk_path, original_id, label, language)",
148
+ )
149
+ ap.add_argument(
150
+ "--output-dir", default="embeddings", help="Folder for .npy embeddings"
151
+ )
152
+ ap.add_argument(
153
+ "--output-manifest",
154
+ default=None,
155
+ help="Output manifest CSV (default: final_training_manifest.csv or derived from --manifest)",
156
+ )
157
+ ap.add_argument(
158
+ "--mfcc",
159
+ action="store_true",
160
+ help="Append 39-d MFCC → 2090-d (default prosody is 2051-d: mean+std+pitch)",
161
+ )
162
+ args = ap.parse_args()
163
+ manifest_path = args.manifest
164
+ output_folder = args.output_dir
165
+ use_mfcc = args.mfcc
166
+ if args.output_manifest:
167
+ output_manifest_path = args.output_manifest
168
  else:
169
+ # Main pipeline: processed_manifest -> final_training_manifest; else e.g. iqra_processed_manifest -> iqra_embeddings_manifest
170
+ if manifest_path.strip() == "manifests/processed_manifest.csv":
171
+ output_manifest_path = "manifests/final_training_manifest.csv"
172
+ else:
173
+ d = os.path.dirname(manifest_path)
174
+ base = (
175
+ os.path.basename(manifest_path)
176
+ .replace(".csv", "")
177
+ .replace("_processed", "_embeddings")
178
+ )
179
+ output_manifest_path = os.path.join(d, base + ".csv")
180
 
181
+ os.makedirs(output_folder, exist_ok=True)
182
+ df = pd.read_csv(manifest_path)
183
+ embedding_paths = []
184
+ feat_desc = f"prosody mean+std+pitch ({PROSODY_EMBED_DIM}d)" + (
185
+ f" + MFCC({MFCC_DIM}d)" if use_mfcc else ""
186
+ )
187
+ print(f"Extracting embeddings for {len(df)} chunks ({feat_desc}, L2-normalized)...")
188
+ for idx, row in tqdm(df.iterrows(), total=len(df)):
189
+ if use_mfcc:
190
+ out = get_embeddings(row["chunk_path"], l2_norm=False, return_audio=True)
191
+ if out is None:
192
+ embedding_paths.append(None)
193
+ continue
194
+ emb, speech, sr = out
195
+ mfcc_39 = get_mfcc_from_audio(speech, sr=sr)
196
+ emb = np.concatenate([emb, mfcc_39]).astype(np.float32)
197
+ n = np.linalg.norm(emb)
198
+ if n > 1e-8:
199
+ emb = emb / n
200
+ else:
201
+ emb = get_embeddings(row["chunk_path"], l2_norm=True)
202
+ if emb is not None:
203
+ emb_filename = os.path.basename(row["chunk_path"]).replace(".wav", ".npy")
204
+ emb_path = os.path.join(output_folder, emb_filename)
205
+ np.save(emb_path, emb)
206
+ embedding_paths.append(emb_path)
207
+ else:
208
+ embedding_paths.append(None)
209
+ df["embedding_path"] = embedding_paths
210
+ df.dropna(subset=["embedding_path"], inplace=True)
211
+ df.to_csv(output_manifest_path, index=False)
212
+ embed_dim = EMBED_DIM_WITH_MFCC if use_mfcc else PROSODY_EMBED_DIM
213
+ dim_file = os.path.join(os.path.dirname(output_manifest_path), "embedding_dim.txt")
214
+ with open(dim_file, "w") as f:
215
+ f.write(str(embed_dim))
216
+ print(
217
+ f"Done! Embeddings in {output_folder}, manifest at {output_manifest_path}, dim={embed_dim} (saved to {dim_file})."
218
+ )
scripts/extract_embeddings_english.py DELETED
@@ -1,70 +0,0 @@
1
- """
2
- Extract Wav2Vec2 embeddings for English chunks (same model as Arabic).
3
- Reads: manifests/processed_manifest_english.csv
4
- Writes: embeddings_english/*.npy, manifests/final_training_manifest_english.csv
5
- """
6
- import os
7
- import pandas as pd
8
- import numpy as np
9
- import torch
10
- import librosa
11
- from transformers import Wav2Vec2Model, AutoFeatureExtractor
12
- from tqdm import tqdm
13
-
14
- PROJECT_ROOT = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
15
- MANIFEST_PATH = os.path.join(PROJECT_ROOT, "manifests", "processed_manifest_english.csv")
16
- OUTPUT_FOLDER = os.path.join(PROJECT_ROOT, "embeddings_english")
17
- FINAL_MANIFEST_PATH = os.path.join(PROJECT_ROOT, "manifests", "final_training_manifest_english.csv")
18
-
19
- model_name = "facebook/wav2vec2-large-xlsr-53"
20
- feature_extractor = AutoFeatureExtractor.from_pretrained(model_name)
21
- model = Wav2Vec2Model.from_pretrained(model_name)
22
- model.eval()
23
- device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
24
- model.to(device)
25
-
26
-
27
- def get_embeddings(audio_path, l2_norm=True):
28
- try:
29
- speech, sr = librosa.load(audio_path, sr=16000)
30
- input_values = feature_extractor(
31
- speech, return_tensors="pt", sampling_rate=16000
32
- ).input_values
33
- input_values = input_values.to(device)
34
- with torch.no_grad():
35
- outputs = model(input_values)
36
- emb = torch.mean(outputs.last_hidden_state, dim=1).squeeze().cpu().numpy()
37
- if l2_norm:
38
- n = np.linalg.norm(emb)
39
- if n > 1e-8:
40
- emb = emb / n
41
- return emb
42
- except Exception as e:
43
- print(f"Error on {audio_path}: {e}")
44
- return None
45
-
46
-
47
- def main():
48
- os.makedirs(OUTPUT_FOLDER, exist_ok=True)
49
- df = pd.read_csv(MANIFEST_PATH)
50
- embedding_paths = []
51
-
52
- print(f"Extracting embeddings for {len(df)} English chunks (L2-normalized)...")
53
- for idx, row in tqdm(df.iterrows(), total=len(df)):
54
- emb = get_embeddings(row["chunk_path"], l2_norm=True)
55
- if emb is not None:
56
- emb_filename = os.path.basename(row["chunk_path"]).replace(".wav", ".npy")
57
- emb_path = os.path.join(OUTPUT_FOLDER, emb_filename)
58
- np.save(emb_path, emb)
59
- embedding_paths.append(emb_path)
60
- else:
61
- embedding_paths.append(None)
62
-
63
- df["embedding_path"] = embedding_paths
64
- df.dropna(subset=["embedding_path"], inplace=True)
65
- df.to_csv(FINAL_MANIFEST_PATH, index=False)
66
- print(f"Done. Embeddings -> {OUTPUT_FOLDER}, manifest -> {FINAL_MANIFEST_PATH}")
67
-
68
-
69
- if __name__ == "__main__":
70
- main()
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
scripts/extract_embeddings_for_CNN.py ADDED
@@ -0,0 +1,163 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """
2
+ Extract SEQUENTIAL embeddings for Arabic Native vs Non-Native 1D-CNN classifier.
3
+ Outputs a 2D array (Time_Steps, 1024) instead of a flattened 1D array.
4
+ Uses XLS-R 53 EARLY-TO-MID layers (4,6,8,10).
5
+ """
6
+
7
+ import pandas as pd
8
+ import numpy as np
9
+ import torch
10
+ import librosa
11
+ import os
12
+ from transformers import Wav2Vec2Model, AutoFeatureExtractor
13
+ from tqdm import tqdm
14
+ from scipy.signal import butter, lfilter
15
+
16
+ # Prosody-focused layers
17
+ # model_name = "facebook/wav2vec2-large-xlsr-53"
18
+ model_name = "facebook/wav2vec2-lv-60-espeak-cv-ft"
19
+ PROSODY_LAYERS = (4, 6, 8, 10)
20
+
21
+ _feature_extractor = None
22
+ _model = None
23
+ device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
24
+
25
+
26
+ def _get_wav2vec():
27
+ global _feature_extractor, _model
28
+ if _model is None:
29
+ _feature_extractor = AutoFeatureExtractor.from_pretrained(model_name)
30
+ _model = Wav2Vec2Model.from_pretrained(model_name)
31
+ _model.eval()
32
+ _model.to(device)
33
+ return _feature_extractor, _model
34
+
35
+
36
+ # The exact feature size of the Wav2Vec2 layers
37
+ WAV2VEC_DIM = 1024
38
+
39
+
40
+ def get_embeddings(audio_path):
41
+ """
42
+ Extracts the chronological 2D sequence: (Time, 1024)
43
+ No averaging. No squashing.
44
+ """
45
+ feature_extractor, model = _get_wav2vec()
46
+ try:
47
+ speech, sr = librosa.load(audio_path, sr=16000)
48
+
49
+ input_values = feature_extractor(
50
+ speech, return_tensors="pt", sampling_rate=16000
51
+ ).input_values.to(device)
52
+
53
+ with torch.no_grad():
54
+ outputs = model(input_values, output_hidden_states=True)
55
+
56
+ h = outputs.hidden_states
57
+ num_layers = len(h) - 1
58
+ layer_idx = [min(max(l, 1), num_layers) for l in PROSODY_LAYERS]
59
+ embs = [h[i] for i in layer_idx]
60
+
61
+ # Stack the target layers and average them together (Layer Average, NOT Time Average)
62
+ # Shape becomes (Time, 1024)
63
+ time_seq = torch.stack(embs, dim=0).mean(dim=0).squeeze(0)
64
+
65
+ # Convert directly to numpy and return the 2D matrix
66
+ emb = time_seq.cpu().numpy().astype(np.float32)
67
+ return emb
68
+
69
+ except Exception as e:
70
+ print(f"Error on {audio_path}: {e}")
71
+ return None
72
+
73
+
74
+ def get_embeddings_from_audio(speech, sr=16000):
75
+ """Extract 2D sequence (Time, 1024) from raw audio array. For test-time use."""
76
+ feature_extractor, model = _get_wav2vec()
77
+ try:
78
+ input_values = feature_extractor(
79
+ speech, return_tensors="pt", sampling_rate=sr
80
+ ).input_values.to(device)
81
+ with torch.no_grad():
82
+ outputs = model(input_values, output_hidden_states=True)
83
+ h = outputs.hidden_states
84
+ num_layers = len(h) - 1
85
+ layer_idx = [min(max(l, 1), num_layers) for l in PROSODY_LAYERS]
86
+ embs = [h[i] for i in layer_idx]
87
+ time_seq = torch.stack(embs, dim=0).mean(dim=0).squeeze(0)
88
+ return time_seq.cpu().numpy().astype(np.float32)
89
+ except Exception:
90
+ return None
91
+
92
+
93
+ if __name__ == "__main__":
94
+ import argparse
95
+
96
+ ap = argparse.ArgumentParser(
97
+ description="Extract 2D SEQUENTIAL embeddings for 1D-CNN."
98
+ )
99
+ ap.add_argument(
100
+ "--manifest",
101
+ default="manifests/processed_manifest.csv",
102
+ help="Input manifest CSV (chunk_path, original_id, label, language)",
103
+ )
104
+ ap.add_argument(
105
+ "--output-dir", default="embeddings", help="Folder for .npy embeddings"
106
+ )
107
+ ap.add_argument(
108
+ "--output-manifest",
109
+ default=None,
110
+ help="Output manifest CSV",
111
+ )
112
+ args = ap.parse_args()
113
+
114
+ manifest_path = args.manifest
115
+ output_folder = args.output_dir
116
+
117
+ if args.output_manifest:
118
+ output_manifest_path = args.output_manifest
119
+ else:
120
+ if manifest_path.strip() == "manifests/processed_manifest.csv":
121
+ output_manifest_path = "manifests/final_training_manifest.csv"
122
+ else:
123
+ d = os.path.dirname(manifest_path)
124
+ base = (
125
+ os.path.basename(manifest_path)
126
+ .replace(".csv", "")
127
+ .replace("_processed", "_embeddings")
128
+ )
129
+ output_manifest_path = os.path.join(d, base + ".csv")
130
+
131
+ os.makedirs(output_folder, exist_ok=True)
132
+ df = pd.read_csv(manifest_path)
133
+ embedding_paths = []
134
+
135
+ print(
136
+ f"Extracting 2D Sequential Embeddings for {len(df)} chunks (Shape: Time x {WAV2VEC_DIM})..."
137
+ )
138
+
139
+ for idx, row in tqdm(df.iterrows(), total=len(df)):
140
+ chunk_path = str(row["chunk_path"]).replace("\\", "/")
141
+ emb = get_embeddings(chunk_path)
142
+
143
+ if emb is not None:
144
+ emb_filename = os.path.basename(chunk_path).replace(".wav", ".npy")
145
+ emb_path = os.path.join(output_folder, emb_filename)
146
+ np.save(emb_path, emb)
147
+ embedding_paths.append(emb_path)
148
+ else:
149
+ embedding_paths.append(None)
150
+
151
+ df["embedding_path"] = embedding_paths
152
+ df.dropna(subset=["embedding_path"], inplace=True)
153
+ df.to_csv(output_manifest_path, index=False)
154
+
155
+ # Save the dimension size so the training script knows what to expect (use separate file if custom manifest to avoid overwriting main embedding_dim.txt)
156
+ dim_name = "embedding_dim_cnn.txt" if args.output_manifest else "embedding_dim.txt"
157
+ dim_file = os.path.join(os.path.dirname(output_manifest_path), dim_name)
158
+ with open(dim_file, "w") as f:
159
+ f.write(str(WAV2VEC_DIM))
160
+
161
+ print(
162
+ f"Done! 2D Embeddings in {output_folder}, manifest at {output_manifest_path}."
163
+ )
scripts/load_data.py CHANGED
@@ -9,10 +9,12 @@ from tqdm import tqdm
9
 
10
 
11
  class AudioPreprocessor:
12
- def __init__(self, target_sr=16000, chunk_duration=10):
13
  self.target_sr = target_sr
14
  self.chunk_duration = chunk_duration
15
- self.samples_per_chunk = target_sr * chunk_duration
 
 
16
 
17
  def download_audio(self, url):
18
  """Downloads audio from URL into memory."""
@@ -25,7 +27,7 @@ class AudioPreprocessor:
25
  return None
26
 
27
  def process_and_split(self, audio_bytes, trim_silence=True, peak_norm=True):
28
- """Converts to mono, resamples, optionally trims silence, and splits into chunks."""
29
  try:
30
  audio_bytes.seek(0)
31
  y, sr = librosa.load(audio_bytes, sr=self.target_sr, mono=True)
@@ -36,15 +38,23 @@ class AudioPreprocessor:
36
 
37
  # Trim leading/trailing silence (gentle threshold to avoid cutting speech)
38
  if trim_silence and len(y) > self.samples_per_chunk:
39
- y_trim, _ = librosa.effects.trim(y, top_db=25, frame_length=2048, hop_length=512)
 
 
40
  if len(y_trim) >= self.samples_per_chunk:
41
  y = y_trim
42
 
43
  chunks = []
44
- for i in range(0, len(y), self.samples_per_chunk):
 
45
  chunk = y[i : i + self.samples_per_chunk].copy()
46
- if len(chunk) != self.samples_per_chunk:
 
 
 
 
47
  continue
 
48
  # Peak-normalize chunk so level is consistent across chunks
49
  if peak_norm:
50
  peak = np.abs(chunk).max()
@@ -94,6 +104,6 @@ class AudioPreprocessor:
94
 
95
  if __name__ == "__main__":
96
  # Usage
97
- preprocessor = AudioPreprocessor()
98
  # Ensure your csv has columns: 'audio_url', 'dp_id', 'nativity_status', 'language'
99
  preprocessor.run_pipeline("manifests/input_data.csv", "data/processed_audio_chunks")
 
9
 
10
 
11
  class AudioPreprocessor:
12
+ def __init__(self, target_sr=16000, chunk_duration=3, stride_duration=1):
13
  self.target_sr = target_sr
14
  self.chunk_duration = chunk_duration
15
+ self.stride_duration = stride_duration
16
+ self.samples_per_chunk = int(target_sr * chunk_duration)
17
+ self.samples_per_stride = int(target_sr * stride_duration)
18
 
19
  def download_audio(self, url):
20
  """Downloads audio from URL into memory."""
 
27
  return None
28
 
29
  def process_and_split(self, audio_bytes, trim_silence=True, peak_norm=True):
30
+ """Converts to mono, resamples, optionally trims silence, and splits using a rolling window."""
31
  try:
32
  audio_bytes.seek(0)
33
  y, sr = librosa.load(audio_bytes, sr=self.target_sr, mono=True)
 
38
 
39
  # Trim leading/trailing silence (gentle threshold to avoid cutting speech)
40
  if trim_silence and len(y) > self.samples_per_chunk:
41
+ y_trim, _ = librosa.effects.trim(
42
+ y, top_db=25, frame_length=2048, hop_length=512
43
+ )
44
  if len(y_trim) >= self.samples_per_chunk:
45
  y = y_trim
46
 
47
  chunks = []
48
+ # OVERLAPPING WINDOW: Step forward by samples_per_stride (1 second) instead of full chunk
49
+ for i in range(0, len(y), self.samples_per_stride):
50
  chunk = y[i : i + self.samples_per_chunk].copy()
51
+
52
+ # Keep chunks that are at least 2.5 seconds long.
53
+ # (Shorter trailing pieces are safely dropped because the overlap caught them earlier)
54
+ min_required_samples = int(self.target_sr * 2.5)
55
+ if len(chunk) < min_required_samples:
56
  continue
57
+
58
  # Peak-normalize chunk so level is consistent across chunks
59
  if peak_norm:
60
  peak = np.abs(chunk).max()
 
104
 
105
  if __name__ == "__main__":
106
  # Usage
107
+ preprocessor = AudioPreprocessor(chunk_duration=3, stride_duration=1)
108
  # Ensure your csv has columns: 'audio_url', 'dp_id', 'nativity_status', 'language'
109
  preprocessor.run_pipeline("manifests/input_data.csv", "data/processed_audio_chunks")
scripts/load_data_english.py DELETED
@@ -1,136 +0,0 @@
1
- """
2
- Load English native / non-native audio from local folders in manifests/.
3
- Same pipeline as Arabic: resample to 16 kHz, 10s chunks, trim silence, peak norm.
4
- Output: data/processed_audio_chunks_english/, manifests/processed_manifest_english.csv
5
- """
6
- import os
7
- import pandas as pd
8
- import librosa
9
- import soundfile as sf
10
- import numpy as np
11
- from tqdm import tqdm
12
-
13
- # Paths (project root = parent of scripts/)
14
- PROJECT_ROOT = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
15
- MANIFESTS_DIR = os.path.join(PROJECT_ROOT, "manifests")
16
- NATIVE_DIR = os.path.join(MANIFESTS_DIR, "Native")
17
- NON_NATIVE_DIR = os.path.join(MANIFESTS_DIR, "Non Native")
18
- OUTPUT_CHUNK_DIR = os.path.join(PROJECT_ROOT, "data", "processed_audio_chunks_english")
19
- PROCESSED_MANIFEST_PATH = os.path.join(MANIFESTS_DIR, "processed_manifest_english.csv")
20
-
21
- AUDIO_EXTENSIONS = {".wav", ".mp3", ".flac", ".m4a", ".ogg"}
22
- TARGET_SR = 16000
23
- CHUNK_DURATION = 10
24
- SAMPLES_PER_CHUNK = TARGET_SR * CHUNK_DURATION
25
-
26
-
27
- def find_audio_files(root_dir, label, language_tag):
28
- """Recursively find audio files; return list of (abspath, label, language_tag)."""
29
- rows = []
30
- for dirpath, _dirnames, filenames in os.walk(root_dir):
31
- for f in filenames:
32
- ext = os.path.splitext(f)[1].lower()
33
- if ext in AUDIO_EXTENSIONS:
34
- abspath = os.path.join(dirpath, f)
35
- rows.append((abspath, label, language_tag))
36
- return rows
37
-
38
-
39
- def process_and_split(y, sr, target_sr=TARGET_SR, samples_per_chunk=SAMPLES_PER_CHUNK, trim_silence=True, peak_norm=True):
40
- """Resample to target_sr if needed, trim silence, split into fixed-length chunks. Short clips get one zero-padded chunk."""
41
- if sr != target_sr:
42
- y = librosa.resample(y, orig_sr=sr, target_sr=target_sr)
43
- if len(y) == 0:
44
- return []
45
- if trim_silence and len(y) > samples_per_chunk:
46
- y_trim, _ = librosa.effects.trim(y, top_db=25, frame_length=2048, hop_length=512)
47
- if len(y_trim) >= samples_per_chunk:
48
- y = y_trim
49
- chunks = []
50
- for i in range(0, len(y), samples_per_chunk):
51
- chunk = y[i : i + samples_per_chunk].copy()
52
- if len(chunk) != samples_per_chunk:
53
- if len(chunk) >= 16000:
54
- # Pad last partial chunk to full length (min ~1s of content)
55
- chunk = np.pad(chunk, (0, samples_per_chunk - len(chunk)), mode="constant", constant_values=0)
56
- else:
57
- continue
58
- if peak_norm:
59
- peak = np.abs(chunk).max()
60
- if peak > 1e-8:
61
- chunk = chunk / peak
62
- chunks.append(chunk)
63
- # If no full chunks but we have at least ~1s of audio, yield one padded chunk
64
- if not chunks and len(y) >= 16000:
65
- chunk = np.pad(y, (0, max(0, samples_per_chunk - len(y))), mode="constant", constant_values=0)[:samples_per_chunk]
66
- if peak_norm:
67
- peak = np.abs(chunk).max()
68
- if peak > 1e-8:
69
- chunk = chunk / peak
70
- chunks.append(chunk)
71
- return chunks
72
-
73
-
74
- def main():
75
- os.makedirs(OUTPUT_CHUNK_DIR, exist_ok=True)
76
-
77
- file_list = []
78
- file_list.extend(find_audio_files(NATIVE_DIR, "Native", "English_Native"))
79
- file_list.extend(find_audio_files(NON_NATIVE_DIR, "Non-Native", "English_NonNative"))
80
-
81
- if not file_list:
82
- print("No audio files found under Native and Non Native.")
83
- return
84
-
85
- print(f"Found {len(file_list)} audio files. Processing (16 kHz, {CHUNK_DURATION}s chunks)...")
86
- processed_data = []
87
- file_id = 0
88
-
89
- skipped_empty = 0
90
- skipped_error = 0
91
- for abspath, label, language_tag in tqdm(file_list, desc="Files"):
92
- if os.path.getsize(abspath) == 0:
93
- skipped_empty += 1
94
- continue
95
- try:
96
- y, sr = librosa.load(abspath, sr=None, mono=True)
97
- except Exception as e:
98
- skipped_error += 1
99
- if skipped_error <= 5:
100
- print(f"Skip (load error): {os.path.basename(abspath)}: {e}")
101
- continue
102
-
103
- chunks = process_and_split(y, sr)
104
- base_id = f"en_{file_id}"
105
- # Speaker ID = folder path under Native/ or Non Native/ (so we can split by human speaker later)
106
- parent_dir = os.path.dirname(abspath)
107
- if parent_dir.startswith(NATIVE_DIR):
108
- rel = os.path.relpath(parent_dir, NATIVE_DIR)
109
- speaker_id = "nat_" + rel.replace(os.sep, "_")
110
- else:
111
- rel = os.path.relpath(parent_dir, NON_NATIVE_DIR)
112
- speaker_id = "nonnat_" + rel.replace(os.sep, "_")
113
-
114
- for idx, chunk in enumerate(chunks):
115
- chunk_filename = f"{base_id}_chunk{idx}.wav"
116
- chunk_path = os.path.join(OUTPUT_CHUNK_DIR, chunk_filename)
117
- sf.write(chunk_path, chunk, TARGET_SR)
118
- processed_data.append({
119
- "chunk_path": chunk_path,
120
- "original_id": base_id,
121
- "label": label,
122
- "language": language_tag,
123
- "speaker_id": speaker_id,
124
- })
125
- file_id += 1
126
-
127
- manifest_df = pd.DataFrame(processed_data)
128
- manifest_df.to_csv(PROCESSED_MANIFEST_PATH, index=False)
129
- print(f"Done. {len(manifest_df)} chunks -> {OUTPUT_CHUNK_DIR}")
130
- print(f"Manifest -> {PROCESSED_MANIFEST_PATH}")
131
- if skipped_empty or skipped_error:
132
- print(f"Skipped: {skipped_empty} empty files, {skipped_error} load errors.")
133
-
134
-
135
- if __name__ == "__main__":
136
- main()
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
scripts/make_predictions.py CHANGED
@@ -1,6 +1,6 @@
1
  """
2
- Run inference on test CSV: download audio, chunk, embed with Wav2Vec, classify.
3
- Uses confidence-weighted voting across chunks and shared model from model.py.
4
  """
5
  import os
6
  import sys
@@ -12,10 +12,18 @@ import requests
12
  import io
13
  from transformers import Wav2Vec2Model, AutoFeatureExtractor
14
  from tqdm import tqdm
 
 
 
 
 
 
 
15
 
16
  # Import shared classifier (same architecture as training)
17
  sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
18
  from model import NativityClassifier
 
19
 
20
  # -----------------------------------------------------------------------------
21
  # Config
@@ -24,10 +32,27 @@ device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
24
  model_path = "models/nativity_model.pth"
25
  input_csv = "manifests/test_data.csv"
26
  output_csv = "final_predictions.csv"
 
27
 
28
- # Model arch must match train_classifier.py / model.py
29
  HIDDEN_DIMS = (384, 256, 128)
30
  DROPOUT = 0.35
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
31
 
32
  # -----------------------------------------------------------------------------
33
  # Load models
@@ -38,7 +63,7 @@ wav2vec = Wav2Vec2Model.from_pretrained(model_name).to(device)
38
  wav2vec.eval()
39
 
40
  classifier = NativityClassifier(
41
- input_dim=1024,
42
  hidden_dims=HIDDEN_DIMS,
43
  output_dim=2,
44
  dropout=DROPOUT,
@@ -47,17 +72,33 @@ classifier.load_state_dict(torch.load(model_path, map_location=device))
47
  classifier.eval()
48
 
49
 
50
- def get_embedding(audio_tensor):
51
- """Extract wav2vec embedding; apply L2 norm to match training."""
 
 
52
  with torch.no_grad():
53
  input_values = feature_extractor(
54
- audio_tensor, return_tensors="pt", sampling_rate=16000
55
  ).input_values.to(device)
56
- outputs = wav2vec(input_values)
57
- emb = torch.mean(outputs.last_hidden_state, dim=1)
58
- # L2 normalize to match training dataset
59
- emb = emb / (emb.norm(dim=1, keepdim=True) + 1e-8)
60
- return emb
 
 
 
 
 
 
 
 
 
 
 
 
 
 
61
 
62
 
63
  def process_file_and_predict(url):
@@ -68,10 +109,15 @@ def process_file_and_predict(url):
68
  try:
69
  response = requests.get(url, timeout=15)
70
  response.raise_for_status()
71
- audio_bytes = io.BytesIO(response.content)
 
 
 
 
 
 
72
  y, sr = librosa.load(audio_bytes, sr=16000, mono=True)
73
- except Exception as e:
74
- print(f"Error loading {url[:60]}...: {e}")
75
  return "Error", 0.0
76
 
77
  chunk_size = 160000 # 10 sec at 16 kHz
@@ -88,11 +134,11 @@ def process_file_and_predict(url):
88
 
89
  with torch.no_grad():
90
  for chunk in chunks:
91
- emb = get_embedding(chunk)
92
  logits = classifier(emb)
93
  probs = torch.softmax(logits, dim=1)
94
  p1 = probs[0, 1].item()
95
- pred = 1 if p1 > 0.5 else 0
96
  conf = p1 if pred == 1 else (1.0 - p1)
97
  chunk_preds.append(pred)
98
  chunk_confs.append(conf)
@@ -101,7 +147,7 @@ def process_file_and_predict(url):
101
  weights = np.array(chunk_confs, dtype=np.float64)
102
  weights = weights / (weights.sum() + 1e-10)
103
  weighted_score = np.sum(np.array(chunk_preds) * weights)
104
- final_label = "Non-Native" if weighted_score > 0.5 else "Native"
105
  # Final confidence = weighted average of confidences (or max)
106
  final_conf = float(np.average(chunk_confs, weights=weights))
107
 
@@ -121,7 +167,33 @@ def main():
121
  })
122
  out_df = pd.DataFrame(results)
123
  out_df.to_csv(output_csv, index=False)
124
- print(f"Done. Saved to {output_csv}")
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
125
 
126
 
127
  if __name__ == "__main__":
 
1
  """
2
+ Run inference on test CSV: download audio, chunk, embed with Wav2Vec (middle layers), classify.
3
+ Arabic Native vs Non-Native. Uses language-agnostic middle-layer embeddings.
4
  """
5
  import os
6
  import sys
 
12
  import io
13
  from transformers import Wav2Vec2Model, AutoFeatureExtractor
14
  from tqdm import tqdm
15
+ from sklearn.metrics import (
16
+ accuracy_score,
17
+ f1_score,
18
+ precision_score,
19
+ recall_score,
20
+ confusion_matrix,
21
+ )
22
 
23
  # Import shared classifier (same architecture as training)
24
  sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
25
  from model import NativityClassifier
26
+ from extract_embeddings import get_mfcc_from_audio, get_pitch_stats, PROSODY_EMBED_DIM, EMBED_DIM_WITH_MFCC
27
 
28
  # -----------------------------------------------------------------------------
29
  # Config
 
32
  model_path = "models/nativity_model.pth"
33
  input_csv = "manifests/test_data.csv"
34
  output_csv = "final_predictions.csv"
35
+ EMBED_DIM_FILE = "manifests/embedding_dim.txt"
36
 
37
+ # Model arch must match train_classifier_compare / model.py
38
  HIDDEN_DIMS = (384, 256, 128)
39
  DROPOUT = 0.35
40
+ EVAL_THRESHOLD_DEFAULT = 0.35
41
+ BEST_THRESHOLD_PATH = "models/best_threshold.txt"
42
+ PROSODY_LAYERS = (4, 6, 8, 10) # must match extract_embeddings.py (prosody-focused)
43
+
44
+ # Embedding dim: 2051 (prosody) or 2090 (prosody+MFCC)
45
+ embedding_dim = PROSODY_EMBED_DIM
46
+ if os.path.isfile(EMBED_DIM_FILE):
47
+ with open(EMBED_DIM_FILE) as f:
48
+ embedding_dim = int(f.read().strip())
49
+
50
+ # Use validation-picked threshold if available (from train_classifier_compare.py)
51
+ if os.path.isfile(BEST_THRESHOLD_PATH):
52
+ with open(BEST_THRESHOLD_PATH) as f:
53
+ EVAL_THRESHOLD = float(f.read().strip())
54
+ else:
55
+ EVAL_THRESHOLD = EVAL_THRESHOLD_DEFAULT
56
 
57
  # -----------------------------------------------------------------------------
58
  # Load models
 
63
  wav2vec.eval()
64
 
65
  classifier = NativityClassifier(
66
+ input_dim=embedding_dim,
67
  hidden_dims=HIDDEN_DIMS,
68
  output_dim=2,
69
  dropout=DROPOUT,
 
72
  classifier.eval()
73
 
74
 
75
+ def get_embedding(audio_np, embed_dim=None):
76
+ """Prosody embedding: mean+std+pitch (2051-d) or +MFCC (2090-d); L2 norm to match training."""
77
+ if embed_dim is None:
78
+ embed_dim = embedding_dim
79
  with torch.no_grad():
80
  input_values = feature_extractor(
81
+ audio_np, return_tensors="pt", sampling_rate=16000
82
  ).input_values.to(device)
83
+ outputs = wav2vec(input_values, output_hidden_states=True)
84
+ h = outputs.hidden_states
85
+ num_layers = len(h) - 1
86
+ layer_idx = [min(max(l, 1), num_layers) for l in PROSODY_LAYERS]
87
+ embs = [h[i] for i in layer_idx]
88
+ time_seq = torch.stack(embs, dim=0).mean(dim=0).squeeze(0) # (T, 1024)
89
+ emb_mean = time_seq.mean(dim=0).cpu().numpy().astype(np.float32)
90
+ emb_std = time_seq.std(dim=0).cpu().numpy().astype(np.float32)
91
+ np.nan_to_num(emb_std, copy=False, nan=0.0, posinf=0.0, neginf=0.0)
92
+ pitch = get_pitch_stats(audio_np, sr=16000)
93
+ out = np.concatenate([emb_mean, emb_std, pitch]).astype(np.float32)
94
+ if embed_dim == EMBED_DIM_WITH_MFCC:
95
+ mfcc_39 = get_mfcc_from_audio(audio_np, sr=16000)
96
+ out = np.concatenate([out, mfcc_39]).astype(np.float32)
97
+ n = np.linalg.norm(out)
98
+ if n > 1e-8:
99
+ out = out / n
100
+ out = out.reshape(1, -1)
101
+ return torch.tensor(out, dtype=torch.float32, device=device)
102
 
103
 
104
  def process_file_and_predict(url):
 
109
  try:
110
  response = requests.get(url, timeout=15)
111
  response.raise_for_status()
112
+ raw = response.content
113
+ if not raw or len(raw) < 100:
114
+ return "Error", 0.0
115
+ ct = (response.headers.get("Content-Type") or "").lower()
116
+ if "text/html" in ct or raw.lstrip()[:50].startswith(b"<!") or raw.lstrip()[:20].startswith(b"<?"):
117
+ return "Error", 0.0
118
+ audio_bytes = io.BytesIO(raw)
119
  y, sr = librosa.load(audio_bytes, sr=16000, mono=True)
120
+ except Exception:
 
121
  return "Error", 0.0
122
 
123
  chunk_size = 160000 # 10 sec at 16 kHz
 
134
 
135
  with torch.no_grad():
136
  for chunk in chunks:
137
+ emb = get_embedding(chunk, embed_dim=embedding_dim)
138
  logits = classifier(emb)
139
  probs = torch.softmax(logits, dim=1)
140
  p1 = probs[0, 1].item()
141
+ pred = 1 if p1 >= EVAL_THRESHOLD else 0
142
  conf = p1 if pred == 1 else (1.0 - p1)
143
  chunk_preds.append(pred)
144
  chunk_confs.append(conf)
 
147
  weights = np.array(chunk_confs, dtype=np.float64)
148
  weights = weights / (weights.sum() + 1e-10)
149
  weighted_score = np.sum(np.array(chunk_preds) * weights)
150
+ final_label = "Non-Native" if weighted_score >= EVAL_THRESHOLD else "Native"
151
  # Final confidence = weighted average of confidences (or max)
152
  final_conf = float(np.average(chunk_confs, weights=weights))
153
 
 
167
  })
168
  out_df = pd.DataFrame(results)
169
  out_df.to_csv(output_csv, index=False)
170
+ n_ok = (out_df["prediction"] != "Error").sum()
171
+ n_fail = len(out_df) - n_ok
172
+ print(f"Done. Saved to {output_csv} ({n_ok} predicted, {n_fail} failed to load)")
173
+
174
+ # If ground-truth labels exist (nativity_status), print test metrics like validation
175
+ if "nativity_status" in df.columns:
176
+ mask = out_df["prediction"] != "Error"
177
+ y_true_str = df.loc[mask, "nativity_status"].values
178
+ y_pred_str = out_df.loc[mask, "prediction"].values
179
+ map_fn = lambda s: 1 if s == "Non-Native" else 0
180
+ y_true = np.array([map_fn(s) for s in y_true_str])
181
+ y_pred = np.array([map_fn(s) for s in y_pred_str])
182
+
183
+ acc = accuracy_score(y_true, y_pred)
184
+ f1 = f1_score(y_true, y_pred, average="macro", zero_division=0)
185
+ prec = precision_score(y_true, y_pred, average="macro", zero_division=0)
186
+ rec = recall_score(y_true, y_pred, average="macro", zero_division=0)
187
+ cm = confusion_matrix(y_true, y_pred)
188
+
189
+ print("\nTest metrics (from test_data.csv, using nativity_status):")
190
+ print(f" Accuracy: {acc:.4f}")
191
+ print(f" F1 macro: {f1:.4f}")
192
+ print(f" Precision: {prec:.4f}")
193
+ print(f" Recall: {rec:.4f}")
194
+ print(" Confusion matrix (rows=true, cols=pred):")
195
+ print(f" {cm}")
196
+ print(" (Native=0, Non-Native=1)")
197
 
198
 
199
  if __name__ == "__main__":
scripts/make_predictions_english.py DELETED
@@ -1,130 +0,0 @@
1
- """
2
- Inference for English Native vs Non-Native classifier.
3
- Uses models/nativity_model_english.pth; expects test CSV with audio_url or local paths.
4
- Output: final_predictions_english.csv
5
- """
6
- import os
7
- import sys
8
- import pandas as pd
9
- import numpy as np
10
- import torch
11
- import librosa
12
- import requests
13
- import io
14
- from transformers import Wav2Vec2Model, AutoFeatureExtractor
15
- from tqdm import tqdm
16
-
17
- sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
18
- from model import NativityClassifier
19
-
20
- PROJECT_ROOT = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
21
- MODEL_PATH = os.path.join(PROJECT_ROOT, "models", "nativity_model_english.pth")
22
- INPUT_CSV = os.path.join(PROJECT_ROOT, "manifests", "test_data_english.csv")
23
- OUTPUT_CSV = os.path.join(PROJECT_ROOT, "final_predictions_english.csv")
24
-
25
- device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
26
- HIDDEN_DIMS = (384, 256, 128)
27
- DROPOUT = 0.35
28
- CHUNK_SIZE = 160000 # 10 sec at 16 kHz
29
-
30
- model_name = "facebook/wav2vec2-large-xlsr-53"
31
- feature_extractor = AutoFeatureExtractor.from_pretrained(model_name)
32
- wav2vec = Wav2Vec2Model.from_pretrained(model_name).to(device)
33
- wav2vec.eval()
34
- classifier = NativityClassifier(
35
- input_dim=1024,
36
- hidden_dims=HIDDEN_DIMS,
37
- output_dim=2,
38
- dropout=DROPOUT,
39
- ).to(device)
40
- classifier.load_state_dict(torch.load(MODEL_PATH, map_location=device))
41
- classifier.eval()
42
-
43
-
44
- def load_audio(row):
45
- """Load audio from URL or local path."""
46
- url_or_path = row.get("audio_url") or row.get("audio_path") or row.get("chunk_path")
47
- if not url_or_path:
48
- return None, None
49
- if os.path.isfile(url_or_path):
50
- try:
51
- y, sr = librosa.load(url_or_path, sr=16000, mono=True)
52
- return y, sr
53
- except Exception as e:
54
- print(f"Error loading file {url_or_path}: {e}")
55
- return None, None
56
- try:
57
- response = requests.get(url_or_path, timeout=15)
58
- response.raise_for_status()
59
- audio_bytes = io.BytesIO(response.content)
60
- y, sr = librosa.load(audio_bytes, sr=16000, mono=True)
61
- return y, sr
62
- except Exception as e:
63
- print(f"Error loading URL: {e}")
64
- return None, None
65
-
66
-
67
- def get_embedding(audio_tensor):
68
- with torch.no_grad():
69
- input_values = feature_extractor(
70
- audio_tensor, return_tensors="pt", sampling_rate=16000
71
- ).input_values.to(device)
72
- outputs = wav2vec(input_values)
73
- emb = torch.mean(outputs.last_hidden_state, dim=1)
74
- emb = emb / (emb.norm(dim=1, keepdim=True) + 1e-8)
75
- return emb
76
-
77
-
78
- def process_file_and_predict(row):
79
- y, sr = load_audio(row)
80
- if y is None or len(y) == 0:
81
- return "Error", 0.0
82
- chunks = [
83
- y[i : i + CHUNK_SIZE]
84
- for i in range(0, len(y), CHUNK_SIZE)
85
- if len(y[i : i + CHUNK_SIZE]) == CHUNK_SIZE
86
- ]
87
- if not chunks:
88
- return "Error", 0.0
89
- chunk_preds = []
90
- chunk_confs = []
91
- with torch.no_grad():
92
- for chunk in chunks:
93
- emb = get_embedding(chunk)
94
- logits = classifier(emb)
95
- probs = torch.softmax(logits, dim=1)
96
- p1 = probs[0, 1].item()
97
- pred = 1 if p1 > 0.5 else 0
98
- conf = p1 if pred == 1 else (1.0 - p1)
99
- chunk_preds.append(pred)
100
- chunk_confs.append(conf)
101
- weights = np.array(chunk_confs, dtype=np.float64)
102
- weights = weights / (weights.sum() + 1e-10)
103
- weighted_score = np.sum(np.array(chunk_preds) * weights)
104
- final_label = "Non-Native" if weighted_score > 0.5 else "Native"
105
- final_conf = float(np.average(chunk_confs, weights=weights))
106
- return final_label, final_conf
107
-
108
-
109
- def main():
110
- if not os.path.isfile(INPUT_CSV):
111
- print(f"Create {INPUT_CSV} with columns: dp_id, audio_url (or audio_path).")
112
- return
113
- df = pd.read_csv(INPUT_CSV)
114
- id_col = "dp_id" if "dp_id" in df.columns else df.columns[0]
115
- results = []
116
- print(f"Predicting for {len(df)} files (device={device})...")
117
- for _, row in tqdm(df.iterrows(), total=len(df)):
118
- label, conf = process_file_and_predict(row)
119
- results.append({
120
- "dp_id": row[id_col],
121
- "prediction": label,
122
- "confidence_score": round(conf, 4),
123
- })
124
- out_df = pd.DataFrame(results)
125
- out_df.to_csv(OUTPUT_CSV, index=False)
126
- print(f"Done. Saved to {OUTPUT_CSV}")
127
-
128
-
129
- if __name__ == "__main__":
130
- main()
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
scripts/model.py CHANGED
@@ -1,11 +1,30 @@
1
  """
2
  Shared NativityClassifier for training and inference.
3
- Single source of truth so architecture always matches.
4
  """
5
  import torch
6
  import torch.nn as nn
7
 
8
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
9
  class NativityClassifier(nn.Module):
10
  """
11
  MLP classifier on wav2vec embeddings -> Native vs Non-Native.
@@ -38,3 +57,41 @@ class NativityClassifier(nn.Module):
38
 
39
  def forward(self, x):
40
  return self.net(x)
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  """
2
  Shared NativityClassifier for training and inference.
3
+ Variants: MLP (baseline), MLP + channel attention (Squeeze-and-Excitation style).
4
  """
5
  import torch
6
  import torch.nn as nn
7
 
8
 
9
+ class ImprovedAttentionBlock(nn.Module):
10
+ """
11
+ Squeeze-and-Excitation style attention: squeeze 1024 dims to a bottleneck,
12
+ then excite back to 1024 channel weights. Learns which channels matter for nativity.
13
+ """
14
+ def __init__(self, input_dim=1024, reduction_ratio=8):
15
+ super().__init__()
16
+ reduced = max(input_dim // reduction_ratio, 32)
17
+ self.fc1 = nn.Linear(input_dim, reduced)
18
+ self.relu = nn.ReLU()
19
+ self.fc2 = nn.Linear(reduced, input_dim)
20
+ self.sigmoid = nn.Sigmoid()
21
+
22
+ def forward(self, x):
23
+ # x shape: (batch_size, 1024)
24
+ attention_weights = self.sigmoid(self.fc2(self.relu(self.fc1(x))))
25
+ return x * attention_weights
26
+
27
+
28
  class NativityClassifier(nn.Module):
29
  """
30
  MLP classifier on wav2vec embeddings -> Native vs Non-Native.
 
57
 
58
  def forward(self, x):
59
  return self.net(x)
60
+
61
+
62
+ class NativityClassifierWithAttention(nn.Module):
63
+ """
64
+ Squeeze-and-Excitation attention over 1024-d embedding then MLP.
65
+ SE block learns which channels matter for non-native prosody; higher final_dropout
66
+ reduces overfitting to majority class.
67
+ """
68
+
69
+ def __init__(
70
+ self,
71
+ input_dim=1024,
72
+ hidden_dims=(384, 256, 128),
73
+ output_dim=2,
74
+ dropout=0.35,
75
+ final_dropout=0.6,
76
+ reduction_ratio=8,
77
+ ):
78
+ super(NativityClassifierWithAttention, self).__init__()
79
+ self.input_dim = input_dim
80
+ self.output_dim = output_dim
81
+ self.attention = ImprovedAttentionBlock(input_dim=input_dim, reduction_ratio=reduction_ratio)
82
+ layers = []
83
+ prev = input_dim
84
+ for i, h in enumerate(hidden_dims):
85
+ layers.extend([
86
+ nn.Linear(prev, h),
87
+ nn.BatchNorm1d(h),
88
+ nn.ReLU(inplace=True),
89
+ nn.Dropout(final_dropout if i == len(hidden_dims) - 1 else dropout),
90
+ ])
91
+ prev = h
92
+ layers.append(nn.Linear(prev, output_dim))
93
+ self.net = nn.Sequential(*layers)
94
+
95
+ def forward(self, x):
96
+ x = self.attention(x)
97
+ return self.net(x)
scripts/train1DCNN.py ADDED
@@ -0,0 +1,485 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """
2
+ Train and compare: 1D-CNN (Chronological Sequence) and LightGBM Ensemble.
3
+ Reports metrics for each so you can see which gives higher accuracy/F1.
4
+ """
5
+
6
+ import os
7
+ import sys
8
+ import random
9
+
10
+ sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
11
+ import pandas as pd
12
+ import numpy as np
13
+ import torch
14
+ import torch.nn as nn
15
+ import torch.nn.functional as F
16
+ import torch.optim as optim
17
+ from torch.utils.data import DataLoader, TensorDataset
18
+ from sklearn.model_selection import train_test_split
19
+ from sklearn.preprocessing import LabelEncoder
20
+ from sklearn.utils.class_weight import compute_class_weight
21
+ from sklearn.metrics import (
22
+ accuracy_score,
23
+ f1_score,
24
+ precision_score,
25
+ recall_score,
26
+ confusion_matrix,
27
+ )
28
+
29
+ import matplotlib
30
+
31
+ matplotlib.use("Agg")
32
+ import matplotlib.pyplot as plt
33
+
34
+ SEED = 42
35
+ MANIFEST_PATH = "manifests/final_training_manifest.csv"
36
+ MODEL_SAVE_DIR = "models"
37
+ EPOCHS = 100
38
+ BATCH_SIZE = 32
39
+
40
+ # --- HYPERPARAMETERS for 1D-CNN ---
41
+ LR = 1e-3
42
+ WEIGHT_DECAY = 1e-4
43
+ PATIENCE = 15
44
+ MIN_EPOCHS = 40
45
+ DROPOUT = 0.4
46
+ INPUT_DIM = 1024
47
+ MAX_TIME_STEPS = 150 # Standardizes varying chunk lengths to exactly 3 seconds
48
+ # ----------------------------------
49
+
50
+ OUTPUT_DIM = 2
51
+ EVAL_THRESHOLD = 0.50
52
+ BEST_THRESHOLD_PATH = os.path.join(MODEL_SAVE_DIR, "best_threshold.txt")
53
+ MIN_LR = 1e-6
54
+ THRESHOLD_CANDIDATES = [0.25, 0.30, 0.35, 0.40, 0.45, 0.50, 0.55]
55
+
56
+
57
+ def set_seed(seed):
58
+ random.seed(seed)
59
+ np.random.seed(seed)
60
+ torch.manual_seed(seed)
61
+ if torch.cuda.is_available():
62
+ torch.cuda.manual_seed_all(seed)
63
+ torch.backends.cudnn.deterministic = True
64
+ torch.backends.cudnn.benchmark = False
65
+ else:
66
+ torch.backends.cudnn.benchmark = True
67
+
68
+
69
+ # =========================================================================
70
+ # --- NEW: 1D Convolutional Neural Network ---
71
+ # =========================================================================
72
+ class NativityCNN1D(nn.Module):
73
+ def __init__(self, input_dim=1024, num_classes=2, dropout=0.5):
74
+ super(NativityCNN1D, self).__init__()
75
+
76
+ # Scans chronologically for phonetic mispronunciations
77
+ self.conv1 = nn.Conv1d(
78
+ in_channels=input_dim, out_channels=256, kernel_size=3, padding=1
79
+ )
80
+ self.bn1 = nn.BatchNorm1d(256)
81
+
82
+ self.conv2 = nn.Conv1d(
83
+ in_channels=256, out_channels=128, kernel_size=3, padding=1
84
+ )
85
+ self.bn2 = nn.BatchNorm1d(128)
86
+
87
+ self.conv3 = nn.Conv1d(
88
+ in_channels=128, out_channels=64, kernel_size=3, padding=1
89
+ )
90
+ self.bn3 = nn.BatchNorm1d(64)
91
+
92
+ # Squeezes the sequence down to a final decision at the end
93
+ self.global_pool = nn.AdaptiveAvgPool1d(1)
94
+ self.dropout = nn.Dropout(dropout)
95
+ self.fc = nn.Linear(64, num_classes)
96
+
97
+ def forward(self, x):
98
+ # Incoming shape: (Batch, Time, Features) -> Convert to (Batch, Features, Time) for PyTorch Conv1d
99
+ x = x.permute(0, 2, 1)
100
+
101
+ x = F.relu(self.bn1(self.conv1(x)))
102
+ x = F.max_pool1d(x, kernel_size=2)
103
+
104
+ x = F.relu(self.bn2(self.conv2(x)))
105
+ x = F.max_pool1d(x, kernel_size=2)
106
+
107
+ x = F.relu(self.bn3(self.conv3(x)))
108
+
109
+ x = self.global_pool(x).squeeze(-1)
110
+ x = self.dropout(x)
111
+ logits = self.fc(x)
112
+
113
+ return logits
114
+
115
+
116
+ # =========================================================================
117
+
118
+
119
+ class EmbeddingDataset(torch.utils.data.Dataset):
120
+ def __init__(self, df, embedding_path_col="embedding_path", label_col="label_idx"):
121
+ self.paths = df[embedding_path_col].tolist()
122
+ self.labels = df[label_col].values
123
+
124
+ def __len__(self):
125
+ return len(self.labels)
126
+
127
+ def __getitem__(self, idx):
128
+ emb = np.load(self.paths[idx]).astype(np.float32)
129
+
130
+ # Handle 1D arrays if older data is mixed in
131
+ if len(emb.shape) == 1:
132
+ emb = np.expand_dims(emb, axis=0)
133
+
134
+ time_len, feat_dim = emb.shape
135
+
136
+ # --- NEW: Sequence Padding/Truncation ---
137
+ # Ensures all batch items have the exact same time dimension
138
+ if time_len < MAX_TIME_STEPS:
139
+ pad_width = MAX_TIME_STEPS - time_len
140
+ emb = np.pad(emb, ((0, pad_width), (0, 0)), mode="constant")
141
+ elif time_len > MAX_TIME_STEPS:
142
+ emb = emb[:MAX_TIME_STEPS, :]
143
+
144
+ return torch.tensor(emb, dtype=torch.float32), torch.tensor(
145
+ self.labels[idx], dtype=torch.long
146
+ )
147
+
148
+
149
+ def train_model(
150
+ model,
151
+ train_loader,
152
+ val_loader,
153
+ criterion,
154
+ device,
155
+ save_path,
156
+ epochs=EPOCHS,
157
+ patience=PATIENCE,
158
+ min_epochs=MIN_EPOCHS,
159
+ ):
160
+ optimizer = optim.AdamW(model.parameters(), lr=LR, weight_decay=WEIGHT_DECAY)
161
+ scheduler = optim.lr_scheduler.ReduceLROnPlateau(
162
+ optimizer, mode="max", factor=0.5, patience=5, min_lr=MIN_LR
163
+ )
164
+ best_f1 = 0.0
165
+ patience_counter = 0
166
+
167
+ history = {
168
+ "train_loss": [],
169
+ "val_loss": [],
170
+ "train_acc": [],
171
+ "val_acc": [],
172
+ "val_f1": [],
173
+ }
174
+
175
+ for epoch in range(epochs):
176
+ model.train()
177
+ train_loss_sum = 0.0
178
+ train_correct = 0
179
+ train_total = 0
180
+ for batch_x, batch_y in train_loader:
181
+ batch_x, batch_y = batch_x.to(device), batch_y.to(device)
182
+
183
+ optimizer.zero_grad()
184
+ logits = model(batch_x)
185
+ loss = criterion(logits, batch_y)
186
+ loss.backward()
187
+ torch.nn.utils.clip_grad_norm_(model.parameters(), max_norm=1.0)
188
+ optimizer.step()
189
+
190
+ train_loss_sum += loss.item() * batch_x.size(0)
191
+ preds = logits.argmax(dim=1)
192
+ train_correct += (preds == batch_y).sum().item()
193
+ train_total += batch_x.size(0)
194
+
195
+ train_loss = train_loss_sum / max(1, train_total)
196
+ train_acc = train_correct / max(1, train_total)
197
+
198
+ model.eval()
199
+ val_loss_sum = 0.0
200
+ val_correct = 0
201
+ val_total = 0
202
+ preds_list = []
203
+ labels_list = []
204
+ with torch.no_grad():
205
+ for batch_x, batch_y in val_loader:
206
+ batch_x, batch_y = batch_x.to(device), batch_y.to(device)
207
+ logits = model(batch_x)
208
+ loss = criterion(logits, batch_y)
209
+ val_loss_sum += loss.item() * batch_x.size(0)
210
+ preds = logits.argmax(dim=1)
211
+ val_correct += (preds == batch_y).sum().item()
212
+ val_total += batch_x.size(0)
213
+ preds_list.extend(preds.cpu().numpy())
214
+ labels_list.extend(batch_y.cpu().numpy())
215
+
216
+ val_loss = val_loss_sum / max(1, val_total)
217
+ val_acc = val_correct / max(1, val_total)
218
+ val_f1 = f1_score(
219
+ np.array(labels_list),
220
+ np.array(preds_list),
221
+ average="macro",
222
+ zero_division=0,
223
+ )
224
+
225
+ history["train_loss"].append(train_loss)
226
+ history["val_loss"].append(val_loss)
227
+ history["train_acc"].append(train_acc)
228
+ history["val_acc"].append(val_acc)
229
+ history["val_f1"].append(val_f1)
230
+
231
+ scheduler.step(val_f1)
232
+
233
+ if val_f1 > best_f1:
234
+ best_f1 = val_f1
235
+ patience_counter = 0
236
+ torch.save(model.state_dict(), save_path)
237
+ else:
238
+ patience_counter += 1
239
+
240
+ if patience_counter >= patience and (epoch + 1) >= min_epochs:
241
+ print(f"Early stopping triggered at epoch {epoch + 1}")
242
+ break
243
+
244
+ return best_f1, history
245
+
246
+
247
+ def eval_model(model, val_loader, device, threshold=0.50):
248
+ model.eval()
249
+ preds, probs_list = [], []
250
+ with torch.no_grad():
251
+ for batch_x, batch_y in val_loader:
252
+ batch_x = batch_x.to(device)
253
+ logits = model(batch_x)
254
+ probs = torch.softmax(logits, dim=1).cpu().numpy()
255
+ probs_list.append(probs)
256
+ batch_preds = (probs[:, 1] >= threshold).astype(int)
257
+ preds.extend(batch_preds)
258
+ probs = np.vstack(probs_list)
259
+ return np.array(preds), probs
260
+
261
+
262
+ def plot_history(history, model_name, out_dir=MODEL_SAVE_DIR):
263
+ os.makedirs(out_dir, exist_ok=True)
264
+ epochs = list(range(1, len(history["train_loss"]) + 1))
265
+
266
+ plt.figure(figsize=(8, 5))
267
+ plt.plot(epochs, history["train_loss"], label="train loss", marker="o")
268
+ plt.plot(epochs, history["val_loss"], label="val loss", marker="o")
269
+ plt.xlabel("Epoch")
270
+ plt.ylabel("Loss")
271
+ plt.title(f"{model_name} - Loss per epoch")
272
+ plt.legend()
273
+ plt.grid(True)
274
+ out_path = os.path.join(out_dir, f"{model_name}_loss.png")
275
+ plt.tight_layout()
276
+ plt.savefig(out_path)
277
+ plt.close()
278
+
279
+ plt.figure(figsize=(8, 5))
280
+ plt.plot(epochs, history["train_acc"], label="train acc", marker="o")
281
+ plt.plot(epochs, history["val_acc"], label="val acc", marker="o")
282
+ plt.xlabel("Epoch")
283
+ plt.ylabel("Accuracy")
284
+ plt.title(f"{model_name} - Accuracy per epoch")
285
+ plt.legend()
286
+ plt.grid(True)
287
+ out_path2 = os.path.join(out_dir, f"{model_name}_acc.png")
288
+ plt.tight_layout()
289
+ plt.savefig(out_path2)
290
+ plt.close()
291
+
292
+
293
+ def main():
294
+ import argparse
295
+ ap = argparse.ArgumentParser(description="Train 1D-CNN on sequential embeddings.")
296
+ ap.add_argument("--manifest", default=MANIFEST_PATH, help="Manifest CSV with embedding_path (default: final_training_manifest.csv)")
297
+ args = ap.parse_args()
298
+ manifest_path = args.manifest
299
+
300
+ set_seed(SEED)
301
+ device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
302
+ os.makedirs(MODEL_SAVE_DIR, exist_ok=True)
303
+
304
+ df = pd.read_csv(manifest_path)
305
+
306
+ le = LabelEncoder()
307
+ df["label_idx"] = le.fit_transform(df["label"])
308
+ speaker_df = df.groupby("original_id")["label_idx"].first().reset_index()
309
+ train_speakers, val_speakers = train_test_split(
310
+ speaker_df["original_id"],
311
+ test_size=0.2,
312
+ stratify=speaker_df["label_idx"],
313
+ random_state=SEED,
314
+ )
315
+ train_df = df[df["original_id"].isin(train_speakers)].copy()
316
+ val_df = df[df["original_id"].isin(val_speakers)].copy()
317
+
318
+ train_set = EmbeddingDataset(train_df)
319
+ val_set = EmbeddingDataset(val_df)
320
+
321
+ train_loader = DataLoader(
322
+ train_set, batch_size=BATCH_SIZE, shuffle=True, drop_last=True, num_workers=0
323
+ )
324
+ val_loader = DataLoader(
325
+ val_set, batch_size=BATCH_SIZE, shuffle=False, num_workers=0
326
+ )
327
+
328
+ weights = compute_class_weight(
329
+ "balanced", classes=np.unique(df["label_idx"]), y=df["label_idx"]
330
+ )
331
+ class_weights = torch.tensor(weights, dtype=torch.float32).to(device)
332
+
333
+ # Removed class weights to prevent 97% class-1 guessing
334
+ criterion = nn.CrossEntropyLoss()
335
+
336
+ y_true = np.concatenate(
337
+ [val_set[i][1].numpy().reshape(1) for i in range(len(val_set))]
338
+ )
339
+
340
+ # ---- 1. Train 1D-CNN Sequence Classifier ----
341
+ print("\n" + "=" * 60)
342
+ print("1. Training 1D-CNN (Chronological Sequence)")
343
+ print("=" * 60)
344
+ model_cnn = NativityCNN1D(
345
+ input_dim=INPUT_DIM,
346
+ dropout=DROPOUT,
347
+ ).to(device)
348
+
349
+ best_f1_cnn, hist_cnn = train_model(
350
+ model_cnn,
351
+ train_loader,
352
+ val_loader,
353
+ criterion,
354
+ device,
355
+ os.path.join(MODEL_SAVE_DIR, "nativity_cnn1d_model.pth"),
356
+ )
357
+ try:
358
+ plot_history(hist_cnn, "cnn1d_model", MODEL_SAVE_DIR)
359
+ print(f"Saved training plots for cnn1d_model in {MODEL_SAVE_DIR}")
360
+ except Exception as e:
361
+ print(f"Plotting cnn1d_model failed: {e}")
362
+
363
+ # ---- 2. LightGBM on flattened embeddings ----
364
+ print("\n2. Training LightGBM on mean-pooled vectors")
365
+ print("=" * 60)
366
+
367
+ # LightGBM cannot read 2D Sequences, so we mathematically average the sequence just for the tree
368
+ X_train_seq = np.stack([train_set[i][0].numpy() for i in range(len(train_set))])
369
+ y_train_lgb = np.array([train_set[i][1].item() for i in range(len(train_set))])
370
+ X_val_seq = np.stack([val_set[i][0].numpy() for i in range(len(val_set))])
371
+
372
+ X_train_lgb = X_train_seq.mean(
373
+ axis=1
374
+ ) # Flatten (Batch, 150, 1024) -> (Batch, 1024)
375
+ X_val_lgb = X_val_seq.mean(axis=1)
376
+
377
+ try:
378
+ import lightgbm as lgb
379
+
380
+ lgb_train = lgb.Dataset(X_train_lgb, y_train_lgb)
381
+ lgb_val = lgb.Dataset(X_val_lgb, reference=lgb_train)
382
+ params = {
383
+ "objective": "binary",
384
+ "metric": "binary_logloss",
385
+ "is_unbalance": True,
386
+ "num_leaves": 31,
387
+ "max_depth": 5,
388
+ "learning_rate": 0.01,
389
+ "feature_fraction": 0.6,
390
+ "bagging_fraction": 0.7,
391
+ "bagging_freq": 1,
392
+ "min_data_in_leaf": 20,
393
+ "verbose": -1,
394
+ "seed": SEED,
395
+ }
396
+ callbacks = [lgb.early_stopping(stopping_rounds=25, verbose=False)]
397
+ lgb_model = lgb.train(
398
+ params,
399
+ lgb_train,
400
+ num_boost_round=500,
401
+ valid_sets=[lgb_val],
402
+ callbacks=callbacks,
403
+ )
404
+ lgb_model.save_model(os.path.join(MODEL_SAVE_DIR, "lgbm_nativity.txt"))
405
+ prob_lgb = lgb_model.predict(X_val_lgb)
406
+ except Exception as e:
407
+ print(f"LightGBM skipped: {e}")
408
+ prob_lgb = None
409
+
410
+ # ---- 3. Evaluate and Ensemble ----
411
+ model_cnn.load_state_dict(
412
+ torch.load(
413
+ os.path.join(MODEL_SAVE_DIR, "nativity_cnn1d_model.pth"),
414
+ map_location=device,
415
+ )
416
+ )
417
+
418
+ pred_cnn, prob_cnn = eval_model(
419
+ model_cnn, val_loader, device, threshold=EVAL_THRESHOLD
420
+ )
421
+
422
+ # Convert binary LightGBM probabilities to a pseudo-multiclass output so the ensemble stack works
423
+ if prob_lgb is not None:
424
+ prob_lgb_multi = np.vstack([1 - prob_lgb, prob_lgb]).T
425
+ prob_list = [prob_cnn, prob_lgb_multi]
426
+ else:
427
+ prob_list = [prob_cnn]
428
+
429
+ prob_ens = np.stack(prob_list, axis=0).mean(axis=0)
430
+
431
+ # Threshold sweep on validation
432
+ best_threshold = EVAL_THRESHOLD
433
+ best_f1_ens = 0.0
434
+ print("\nThreshold sweep on validation (ensemble):")
435
+ for t in THRESHOLD_CANDIDATES:
436
+ pred_t = (prob_ens[:, 1] >= t).astype(int)
437
+ f1_t = f1_score(y_true, pred_t, average="macro", zero_division=0)
438
+ print(f" t={t:.2f} F1={f1_t:.4f}")
439
+ if f1_t > best_f1_ens:
440
+ best_f1_ens = f1_t
441
+ best_threshold = t
442
+ with open(BEST_THRESHOLD_PATH, "w") as f:
443
+ f.write(str(best_threshold))
444
+ print(
445
+ f"Best threshold (val F1): {best_threshold} -> F1={best_f1_ens:.4f} (saved to {BEST_THRESHOLD_PATH})"
446
+ )
447
+
448
+ pred_ens = (prob_ens[:, 1] >= best_threshold).astype(int)
449
+
450
+ def metrics_row(name, preds):
451
+ acc = accuracy_score(y_true, preds)
452
+ f1 = f1_score(y_true, preds, average="macro", zero_division=0)
453
+ prec = precision_score(y_true, preds, average="macro", zero_division=0)
454
+ rec = recall_score(y_true, preds, average="macro", zero_division=0)
455
+ cm = confusion_matrix(y_true, preds)
456
+ return {"name": name, "acc": acc, "f1": f1, "prec": prec, "rec": rec, "cm": cm}
457
+
458
+ pred_lgb_final = (prob_lgb >= 0.50).astype(int) if prob_lgb is not None else None
459
+
460
+ results = [
461
+ metrics_row("1D-CNN (Sequence)", pred_cnn),
462
+ metrics_row(f"Ensemble (CNN + LGB, t={best_threshold})", pred_ens),
463
+ ]
464
+ if pred_lgb_final is not None:
465
+ results.append(metrics_row("LightGBM", pred_lgb_final))
466
+
467
+ print("\n" + "=" * 60)
468
+ print("COMPARISON (validation set)")
469
+ print("=" * 60)
470
+ print(
471
+ f"{'Model':<30} {'Accuracy':>10} {'F1 macro':>10} {'Precision':>10} {'Recall':>10}"
472
+ )
473
+ print("-" * 72)
474
+ for r in results:
475
+ print(
476
+ f"{r['name']:<30} {r['acc']:>10.4f} {r['f1']:>10.4f} {r['prec']:>10.4f} {r['rec']:>10.4f}"
477
+ )
478
+ print("\nConfusion matrices (rows=true, cols=pred), Native=0, Non-Native=1:")
479
+ for r in results:
480
+ print(f" {r['name']}:")
481
+ print(f" {r['cm']}")
482
+
483
+
484
+ if __name__ == "__main__":
485
+ main()
scripts/train_classifier.py CHANGED
@@ -39,14 +39,14 @@ BEST_MODEL_PATH = os.path.join(MODEL_SAVE_DIR, "nativity_model.pth")
39
  EPOCHS = 200 # max epochs; early stopping will usually stop earlier
40
  BATCH_SIZE = 32
41
  LR = 1e-3
42
- WEIGHT_DECAY = 1e-4
43
- PATIENCE = 30 # epochs without F1 improvement before stopping
44
  MIN_LR = 1e-6
45
 
46
  # Model architecture (must match model.py and embeddings dim)
47
  HIDDEN_DIMS = (384, 256, 128)
48
- DROPOUT = 0.35
49
- INPUT_DIM = 1024 # wav2vec2-large-xlsr-53 hidden size
50
  OUTPUT_DIM = 2
51
 
52
  # -----------------------------------------------------------------------------
@@ -98,6 +98,9 @@ def main():
98
  os.makedirs(MODEL_SAVE_DIR, exist_ok=True)
99
 
100
  df = pd.read_csv(MANIFEST_PATH)
 
 
 
101
  le = LabelEncoder()
102
  df["label_idx"] = le.fit_transform(df["label"])
103
  print(f"Classes: {le.classes_} (Native=0, Non-Native=1)")
@@ -124,12 +127,17 @@ def main():
124
  val_set = EmbeddingDataset(val_df, l2_norm=True)
125
 
126
  train_loader = DataLoader(
127
- train_set, batch_size=BATCH_SIZE, shuffle=True, num_workers=0, pin_memory=(device.type == "cuda")
 
 
 
 
 
128
  )
129
  val_loader = DataLoader(val_set, batch_size=BATCH_SIZE, shuffle=False, num_workers=0)
130
 
131
  model = NativityClassifier(
132
- input_dim=INPUT_DIM,
133
  hidden_dims=HIDDEN_DIMS,
134
  output_dim=OUTPUT_DIM,
135
  dropout=DROPOUT,
@@ -160,8 +168,18 @@ def main():
160
 
161
  train_loss = total_loss / len(train_loader)
162
 
163
- # Validation
164
  model.eval()
 
 
 
 
 
 
 
 
 
 
165
  all_preds = []
166
  all_labels = []
167
  val_loss = 0.0
@@ -195,8 +213,9 @@ def main():
195
  patience_counter += 1
196
 
197
  print(
198
- f"Epoch {epoch + 1:3d} | loss {train_loss:.4f} | val_loss {val_loss:.4f} | "
199
- f"val_acc {acc:.4f} | val_F1 {f1:.4f} | val_P {prec:.4f} | val_R {rec:.4f}"
 
200
  )
201
 
202
  if patience_counter >= PATIENCE:
 
39
  EPOCHS = 200 # max epochs; early stopping will usually stop earlier
40
  BATCH_SIZE = 32
41
  LR = 1e-3
42
+ WEIGHT_DECAY = 5e-4 # stronger L2 to reduce overfitting
43
+ PATIENCE = 15 # stricter early stopping (was 30) — stop sooner
44
  MIN_LR = 1e-6
45
 
46
  # Model architecture (must match model.py and embeddings dim)
47
  HIDDEN_DIMS = (384, 256, 128)
48
+ DROPOUT = 0.5 # increased from 0.35 to reduce overfitting
49
+ INPUT_DIM = 1024 # overridden in main() from first .npy (1063 when using --mfcc)
50
  OUTPUT_DIM = 2
51
 
52
  # -----------------------------------------------------------------------------
 
98
  os.makedirs(MODEL_SAVE_DIR, exist_ok=True)
99
 
100
  df = pd.read_csv(MANIFEST_PATH)
101
+ input_dim = int(np.load(df["embedding_path"].iloc[0]).shape[0])
102
+ print(f"Embedding dimension: {input_dim}")
103
+
104
  le = LabelEncoder()
105
  df["label_idx"] = le.fit_transform(df["label"])
106
  print(f"Classes: {le.classes_} (Native=0, Non-Native=1)")
 
127
  val_set = EmbeddingDataset(val_df, l2_norm=True)
128
 
129
  train_loader = DataLoader(
130
+ train_set,
131
+ batch_size=BATCH_SIZE,
132
+ shuffle=True,
133
+ num_workers=0,
134
+ pin_memory=(device.type == "cuda"),
135
+ drop_last=True, # avoid batch size 1 with BatchNorm in training
136
  )
137
  val_loader = DataLoader(val_set, batch_size=BATCH_SIZE, shuffle=False, num_workers=0)
138
 
139
  model = NativityClassifier(
140
+ input_dim=input_dim,
141
  hidden_dims=HIDDEN_DIMS,
142
  output_dim=OUTPUT_DIM,
143
  dropout=DROPOUT,
 
168
 
169
  train_loss = total_loss / len(train_loader)
170
 
171
+ # Train metrics (to monitor overfitting: train_F1 >> val_F1 = overfitting)
172
  model.eval()
173
+ train_preds, train_labels = [], []
174
+ with torch.no_grad():
175
+ for batch_x, batch_y in train_loader:
176
+ batch_x = batch_x.to(device)
177
+ preds = model(batch_x).argmax(dim=1).cpu().numpy()
178
+ train_preds.extend(preds)
179
+ train_labels.extend(batch_y.numpy())
180
+ train_f1 = f1_score(train_labels, train_preds, average="macro", zero_division=0)
181
+
182
+ # Validation
183
  all_preds = []
184
  all_labels = []
185
  val_loss = 0.0
 
213
  patience_counter += 1
214
 
215
  print(
216
+ f"Epoch {epoch + 1:3d} | loss {train_loss:.4f} | "
217
+ f"train_F1 {train_f1:.4f} | val_F1 {f1:.4f} | val_acc {acc:.4f} | "
218
+ f"(gap={(train_f1-f1):.3f})"
219
  )
220
 
221
  if patience_counter >= PATIENCE:
scripts/train_classifier_compare.py ADDED
@@ -0,0 +1,525 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """
2
+ Train and compare: MLP (baseline), MLP+channel attention (prosody emphasis), and ensemble.
3
+ Reports metrics for each so you can see which gives higher accuracy/F1.
4
+ """
5
+
6
+ import os
7
+ import sys
8
+ import random
9
+
10
+ sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
11
+ import pandas as pd
12
+ import numpy as np
13
+ import torch
14
+ import torch.nn as nn
15
+ import torch.optim as optim
16
+ from torch.utils.data import DataLoader, TensorDataset
17
+ from sklearn.model_selection import train_test_split
18
+ from sklearn.preprocessing import LabelEncoder
19
+ from sklearn.utils.class_weight import compute_class_weight
20
+ from sklearn.decomposition import PCA # --- NEW: Import PCA ---
21
+ from sklearn.metrics import (
22
+ accuracy_score,
23
+ f1_score,
24
+ precision_score,
25
+ recall_score,
26
+ confusion_matrix,
27
+ )
28
+
29
+ import matplotlib
30
+
31
+ matplotlib.use("Agg")
32
+ import matplotlib.pyplot as plt
33
+
34
+ from model import NativityClassifier, NativityClassifierWithAttention
35
+
36
+ SEED = 42
37
+ MANIFEST_PATH = "manifests/final_training_manifest.csv"
38
+ MODEL_SAVE_DIR = "models"
39
+ EPOCHS = 100
40
+ BATCH_SIZE = 32
41
+
42
+ # --- ANTI-OVERFITTING: Hyperparameter adjustments ---
43
+ LR = 1e-3 # Slowed down from 1e-3
44
+ WEIGHT_DECAY = 1e-4 # Punish extreme weights (up from 5e-4)
45
+ PATIENCE = 15 # Stop earlier if it starts memorizing
46
+ MIN_EPOCHS = 40
47
+ PCA_COMPONENTS = 256 # Compress 1000+ dims down to top 64
48
+ HIDDEN_DIMS = (256, 128) # Shrunk from (384, 256, 128)
49
+ DROPOUT = 0.4
50
+ # ----------------------------------------------------
51
+
52
+ OUTPUT_DIM = 2
53
+ # Lower threshold for Class 1 (Non-Native) to boost recall on imbalanced data
54
+ EVAL_THRESHOLD = 0.50
55
+ # SMOTE oversamples minority (Non-Native) in embedding space; already using weighted CE
56
+ USE_SMOTE = False
57
+ SMOTE_K_NEIGHBORS = 5
58
+ # Threshold sweep on val to pick best (don't tune on test)
59
+ THRESHOLD_CANDIDATES = [0.25, 0.28, 0.30, 0.32, 0.35, 0.40]
60
+ BEST_THRESHOLD_PATH = os.path.join(MODEL_SAVE_DIR, "best_threshold.txt")
61
+
62
+ MIN_LR = 1e-6
63
+
64
+
65
+ def set_seed(seed):
66
+ random.seed(seed)
67
+ np.random.seed(seed)
68
+ torch.manual_seed(seed)
69
+ if torch.cuda.is_available():
70
+ torch.cuda.manual_seed_all(seed)
71
+ torch.backends.cudnn.deterministic = True
72
+ torch.backends.cudnn.benchmark = False
73
+ else:
74
+ torch.backends.cudnn.benchmark = True
75
+
76
+
77
+ class EmbeddingDataset(torch.utils.data.Dataset):
78
+ def __init__(
79
+ self,
80
+ df,
81
+ embedding_path_col="embedding_path",
82
+ label_col="label_idx",
83
+ l2_norm=True,
84
+ ):
85
+ self.paths = df[embedding_path_col].tolist()
86
+ self.labels = df[label_col].values
87
+ self.l2_norm = l2_norm
88
+
89
+ def __len__(self):
90
+ return len(self.labels)
91
+
92
+ def __getitem__(self, idx):
93
+ emb = np.load(self.paths[idx]).astype(np.float32)
94
+ if self.l2_norm:
95
+ n = np.linalg.norm(emb)
96
+ if n > 1e-8:
97
+ emb = emb / n
98
+ return torch.tensor(emb, dtype=torch.float32), torch.tensor(
99
+ self.labels[idx], dtype=torch.long
100
+ )
101
+
102
+
103
+ def train_model(
104
+ model,
105
+ train_loader,
106
+ val_loader,
107
+ criterion,
108
+ device,
109
+ save_path,
110
+ epochs=EPOCHS,
111
+ patience=PATIENCE,
112
+ min_epochs=MIN_EPOCHS,
113
+ ):
114
+ optimizer = optim.AdamW(model.parameters(), lr=LR, weight_decay=WEIGHT_DECAY)
115
+ scheduler = optim.lr_scheduler.ReduceLROnPlateau(
116
+ optimizer, mode="max", factor=0.5, patience=5, min_lr=MIN_LR
117
+ )
118
+ best_f1 = 0.0
119
+ patience_counter = 0
120
+
121
+ history = {
122
+ "train_loss": [],
123
+ "val_loss": [],
124
+ "train_acc": [],
125
+ "val_acc": [],
126
+ "val_f1": [],
127
+ }
128
+
129
+ for epoch in range(epochs):
130
+ model.train()
131
+ train_loss_sum = 0.0
132
+ train_correct = 0
133
+ train_total = 0
134
+ for batch_x, batch_y in train_loader:
135
+ batch_x, batch_y = batch_x.to(device), batch_y.to(device)
136
+
137
+ # --- NEW: Inject Gaussian Noise during training to prevent memorization ---
138
+ if model.training:
139
+ noise = torch.randn_like(batch_x) * 0.02
140
+ batch_x = batch_x + noise
141
+ # ------------------------------------------------------------------------
142
+
143
+ optimizer.zero_grad()
144
+ logits = model(batch_x)
145
+ loss = criterion(logits, batch_y)
146
+ loss.backward()
147
+ torch.nn.utils.clip_grad_norm_(model.parameters(), max_norm=1.0)
148
+ optimizer.step()
149
+
150
+ train_loss_sum += loss.item() * batch_x.size(0)
151
+ preds = logits.argmax(dim=1)
152
+ train_correct += (preds == batch_y).sum().item()
153
+ train_total += batch_x.size(0)
154
+
155
+ train_loss = train_loss_sum / max(1, train_total)
156
+ train_acc = train_correct / max(1, train_total)
157
+
158
+ # Validation
159
+ model.eval()
160
+ val_loss_sum = 0.0
161
+ val_correct = 0
162
+ val_total = 0
163
+ preds_list = []
164
+ labels_list = []
165
+ with torch.no_grad():
166
+ for batch_x, batch_y in val_loader:
167
+ batch_x, batch_y = batch_x.to(device), batch_y.to(device)
168
+ logits = model(batch_x)
169
+ loss = criterion(logits, batch_y)
170
+ val_loss_sum += loss.item() * batch_x.size(0)
171
+ preds = logits.argmax(dim=1)
172
+ val_correct += (preds == batch_y).sum().item()
173
+ val_total += batch_x.size(0)
174
+ preds_list.extend(preds.cpu().numpy())
175
+ labels_list.extend(batch_y.cpu().numpy())
176
+
177
+ val_loss = val_loss_sum / max(1, val_total)
178
+ val_acc = val_correct / max(1, val_total)
179
+ val_f1 = f1_score(
180
+ np.array(labels_list),
181
+ np.array(preds_list),
182
+ average="macro",
183
+ zero_division=0,
184
+ )
185
+
186
+ history["train_loss"].append(train_loss)
187
+ history["val_loss"].append(val_loss)
188
+ history["train_acc"].append(train_acc)
189
+ history["val_acc"].append(val_acc)
190
+ history["val_f1"].append(val_f1)
191
+
192
+ scheduler.step(val_f1)
193
+
194
+ if val_f1 > best_f1:
195
+ best_f1 = val_f1
196
+ patience_counter = 0
197
+ torch.save(model.state_dict(), save_path)
198
+ else:
199
+ patience_counter += 1
200
+
201
+ if patience_counter >= patience and (epoch + 1) >= min_epochs:
202
+ print(f"Early stopping triggered at epoch {epoch + 1}")
203
+ break
204
+
205
+ return best_f1, history
206
+
207
+
208
+ def eval_model(model, val_loader, device, threshold=0.35):
209
+ model.eval()
210
+ preds, probs_list = [], []
211
+ with torch.no_grad():
212
+ for batch_x, batch_y in val_loader:
213
+ batch_x = batch_x.to(device)
214
+ logits = model(batch_x)
215
+ probs = torch.softmax(logits, dim=1).cpu().numpy()
216
+ probs_list.append(probs)
217
+ batch_preds = (probs[:, 1] >= threshold).astype(int)
218
+ preds.extend(batch_preds)
219
+ probs = np.vstack(probs_list)
220
+ return np.array(preds), probs
221
+
222
+
223
+ def plot_history(history, model_name, out_dir=MODEL_SAVE_DIR):
224
+ os.makedirs(out_dir, exist_ok=True)
225
+ epochs = list(range(1, len(history["train_loss"]) + 1))
226
+
227
+ plt.figure(figsize=(8, 5))
228
+ plt.plot(epochs, history["train_loss"], label="train loss", marker="o")
229
+ plt.plot(epochs, history["val_loss"], label="val loss", marker="o")
230
+ plt.xlabel("Epoch")
231
+ plt.ylabel("Loss")
232
+ plt.title(f"{model_name} - Loss per epoch")
233
+ plt.legend()
234
+ plt.grid(True)
235
+ out_path = os.path.join(out_dir, f"{model_name}_loss.png")
236
+ plt.tight_layout()
237
+ plt.savefig(out_path)
238
+ plt.close()
239
+
240
+ plt.figure(figsize=(8, 5))
241
+ plt.plot(epochs, history["train_acc"], label="train acc", marker="o")
242
+ plt.plot(epochs, history["val_acc"], label="val acc", marker="o")
243
+ plt.xlabel("Epoch")
244
+ plt.ylabel("Accuracy")
245
+ plt.title(f"{model_name} - Accuracy per epoch")
246
+ plt.legend()
247
+ plt.grid(True)
248
+ out_path2 = os.path.join(out_dir, f"{model_name}_acc.png")
249
+ plt.tight_layout()
250
+ plt.savefig(out_path2)
251
+ plt.close()
252
+
253
+
254
+ def main():
255
+ set_seed(SEED)
256
+ device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
257
+ os.makedirs(MODEL_SAVE_DIR, exist_ok=True)
258
+
259
+ df = pd.read_csv(MANIFEST_PATH)
260
+
261
+ le = LabelEncoder()
262
+ df["label_idx"] = le.fit_transform(df["label"])
263
+ speaker_df = df.groupby("original_id")["label_idx"].first().reset_index()
264
+ train_speakers, val_speakers = train_test_split(
265
+ speaker_df["original_id"],
266
+ test_size=0.2,
267
+ stratify=speaker_df["label_idx"],
268
+ random_state=SEED,
269
+ )
270
+ train_df = df[df["original_id"].isin(train_speakers)].copy()
271
+ val_df = df[df["original_id"].isin(val_speakers)].copy()
272
+
273
+ train_set = EmbeddingDataset(train_df, l2_norm=True)
274
+ val_set = EmbeddingDataset(val_df, l2_norm=True)
275
+
276
+ # =========================================================================
277
+ # --- NEW: PCA Dimensionality Reduction (Fitted ONLY on training data) ---
278
+ # =========================================================================
279
+ print(f"\nApplying PCA to crush dimensions down to {PCA_COMPONENTS}...")
280
+
281
+ # 1. Extract raw numpy arrays
282
+ X_train_raw = np.stack([train_set[i][0].numpy() for i in range(len(train_set))])
283
+ y_train_raw = np.array([train_set[i][1].item() for i in range(len(train_set))])
284
+
285
+ X_val_raw = np.stack([val_set[i][0].numpy() for i in range(len(val_set))])
286
+ y_val_raw = np.array([val_set[i][1].item() for i in range(len(val_set))])
287
+
288
+ # 2. Fit PCA strictly on training data
289
+ pca = PCA(n_components=PCA_COMPONENTS, random_state=SEED)
290
+ X_train_pca = pca.fit_transform(X_train_raw)
291
+ X_val_pca = pca.transform(X_val_raw)
292
+
293
+ # 3. Overwrite Datasets with compressed tensors
294
+ train_set = TensorDataset(
295
+ torch.tensor(X_train_pca, dtype=torch.float32),
296
+ torch.tensor(y_train_raw, dtype=torch.long),
297
+ )
298
+ val_set = TensorDataset(
299
+ torch.tensor(X_val_pca, dtype=torch.float32),
300
+ torch.tensor(y_val_raw, dtype=torch.long),
301
+ )
302
+
303
+ # 4. Override input_dim for the PyTorch models
304
+ input_dim = PCA_COMPONENTS
305
+ print(f"New Model Input Dimension: {input_dim}")
306
+ # =========================================================================
307
+
308
+ # Optional SMOTE (Now operating safely on the 64-dimensional PCA space)
309
+ if USE_SMOTE:
310
+ try:
311
+ from imblearn.over_sampling import SMOTE
312
+
313
+ X_train = np.stack([train_set[i][0].numpy() for i in range(len(train_set))])
314
+ y_train = np.array([train_set[i][1].item() for i in range(len(train_set))])
315
+ n_minority = int((y_train == 1).sum())
316
+ k = min(SMOTE_K_NEIGHBORS, n_minority - 1) if n_minority > 1 else 1
317
+ if k >= 1 and n_minority < (y_train == 0).sum():
318
+ smote = SMOTE(k_neighbors=k, random_state=SEED)
319
+ X_res, y_res = smote.fit_resample(X_train, y_train)
320
+ train_set = TensorDataset(
321
+ torch.tensor(X_res, dtype=torch.float32),
322
+ torch.tensor(y_res, dtype=torch.long),
323
+ )
324
+ print(
325
+ f"SMOTE: {len(y_train)} -> {len(y_res)} samples (Non-Native oversampled)"
326
+ )
327
+ else:
328
+ print("SMOTE: skipped (minority too small or already balanced)")
329
+ except Exception as e:
330
+ print(f"SMOTE skipped: {e}")
331
+
332
+ train_loader = DataLoader(
333
+ train_set, batch_size=BATCH_SIZE, shuffle=True, drop_last=True, num_workers=0
334
+ )
335
+ val_loader = DataLoader(
336
+ val_set, batch_size=BATCH_SIZE, shuffle=False, num_workers=0
337
+ )
338
+
339
+ weights = compute_class_weight(
340
+ "balanced", classes=np.unique(df["label_idx"]), y=df["label_idx"]
341
+ )
342
+ class_weights = torch.tensor(weights, dtype=torch.float32).to(device)
343
+
344
+ # --- Label Smoothing included in the loss function ---
345
+ criterion = nn.CrossEntropyLoss(weight=class_weights, label_smoothing=0.15)
346
+
347
+ y_true = np.concatenate(
348
+ [val_set[i][1].numpy().reshape(1) for i in range(len(val_set))]
349
+ )
350
+
351
+ # ---- 1. Train MLP (baseline) ----
352
+ print("\n" + "=" * 60)
353
+ print("1. Training MLP (baseline)")
354
+ print("=" * 60)
355
+ model_mlp = NativityClassifier(
356
+ input_dim=input_dim,
357
+ hidden_dims=HIDDEN_DIMS,
358
+ output_dim=OUTPUT_DIM,
359
+ dropout=DROPOUT,
360
+ ).to(device)
361
+ best_f1_mlp, hist_mlp = train_model(
362
+ model_mlp,
363
+ train_loader,
364
+ val_loader,
365
+ criterion,
366
+ device,
367
+ os.path.join(MODEL_SAVE_DIR, "nativity_model.pth"),
368
+ )
369
+ try:
370
+ plot_history(hist_mlp, "mlp_baseline", MODEL_SAVE_DIR)
371
+ print(f"Saved training plots for mlp_baseline in {MODEL_SAVE_DIR}")
372
+ except Exception as e:
373
+ print(f"Plotting mlp_baseline failed: {e}")
374
+
375
+ # ---- 2. Train MLP + channel attention (prosody emphasis) ----
376
+ print("\n2. Training MLP + channel attention (prosody emphasis)")
377
+ print("=" * 60)
378
+ model_att = NativityClassifierWithAttention(
379
+ input_dim=input_dim,
380
+ hidden_dims=HIDDEN_DIMS,
381
+ output_dim=OUTPUT_DIM,
382
+ dropout=DROPOUT,
383
+ final_dropout=0.75, # <--- Increased to 0.75 for safety
384
+ reduction_ratio=8,
385
+ ).to(device)
386
+ best_f1_att, hist_att = train_model(
387
+ model_att,
388
+ train_loader,
389
+ val_loader,
390
+ criterion,
391
+ device,
392
+ os.path.join(MODEL_SAVE_DIR, "nativity_model_attention.pth"),
393
+ )
394
+ try:
395
+ plot_history(hist_att, "mlp_attention", MODEL_SAVE_DIR)
396
+ print(f"Saved training plots for mlp_attention in {MODEL_SAVE_DIR}")
397
+ except Exception as e:
398
+ print(f"Plotting mlp_attention failed: {e}")
399
+
400
+ # ---- 3. LightGBM on same train/val embeddings ----
401
+ print("\n3. Training LightGBM on embedding vectors")
402
+ print("=" * 60)
403
+ X_train = np.stack([train_set[i][0].numpy() for i in range(len(train_set))])
404
+ y_train = np.array([train_set[i][1].item() for i in range(len(train_set))])
405
+ X_val = np.stack([val_set[i][0].numpy() for i in range(len(val_set))])
406
+ try:
407
+ import lightgbm as lgb
408
+
409
+ lgb_train = lgb.Dataset(X_train, y_train)
410
+ lgb_val = lgb.Dataset(X_val, reference=lgb_train)
411
+ params = {
412
+ "objective": "multiclass",
413
+ "num_class": 2,
414
+ "metric": "multi_logloss",
415
+ "num_leaves": 31, # Reduced leaves slightly since we only have 64 dimensions now
416
+ "learning_rate": 0.05,
417
+ "feature_fraction": 0.8,
418
+ "bagging_fraction": 0.8,
419
+ "bagging_freq": 5,
420
+ "verbose": -1,
421
+ "seed": SEED,
422
+ }
423
+ callbacks = [lgb.early_stopping(stopping_rounds=25, verbose=False)]
424
+ lgb_model = lgb.train(
425
+ params,
426
+ lgb_train,
427
+ num_boost_round=500,
428
+ valid_sets=[lgb_val],
429
+ callbacks=callbacks,
430
+ )
431
+ lgb_model.save_model(os.path.join(MODEL_SAVE_DIR, "lgbm_nativity.txt"))
432
+ prob_lgb = lgb_model.predict(X_val)
433
+ except Exception as e:
434
+ print(f"LightGBM skipped: {e}")
435
+ prob_lgb = None
436
+
437
+ # ---- 4. Load best and evaluate all + ensemble ----
438
+ model_mlp.load_state_dict(
439
+ torch.load(
440
+ os.path.join(MODEL_SAVE_DIR, "nativity_model.pth"), map_location=device
441
+ )
442
+ )
443
+ model_att.load_state_dict(
444
+ torch.load(
445
+ os.path.join(MODEL_SAVE_DIR, "nativity_model_attention.pth"),
446
+ map_location=device,
447
+ )
448
+ )
449
+
450
+ pred_mlp, prob_mlp = eval_model(
451
+ model_mlp, val_loader, device, threshold=EVAL_THRESHOLD
452
+ )
453
+ pred_att, prob_att = eval_model(
454
+ model_att, val_loader, device, threshold=EVAL_THRESHOLD
455
+ )
456
+
457
+ # Ensemble: average probs from MLP, attention, and LightGBM (if present)
458
+ prob_list = [prob_mlp, prob_att]
459
+ if prob_lgb is not None:
460
+ prob_list.append(prob_lgb)
461
+ prob_ens = np.stack(prob_list, axis=0).mean(axis=0)
462
+
463
+ # Threshold sweep on validation
464
+ best_threshold = EVAL_THRESHOLD
465
+ best_f1_ens = 0.0
466
+ print("\nThreshold sweep on validation (ensemble):")
467
+ for t in THRESHOLD_CANDIDATES:
468
+ pred_t = (prob_ens[:, 1] >= t).astype(int)
469
+ f1_t = f1_score(y_true, pred_t, average="macro", zero_division=0)
470
+ print(f" t={t:.2f} F1={f1_t:.4f}")
471
+ if f1_t > best_f1_ens:
472
+ best_f1_ens = f1_t
473
+ best_threshold = t
474
+ with open(BEST_THRESHOLD_PATH, "w") as f:
475
+ f.write(str(best_threshold))
476
+ print(
477
+ f"Best threshold (val F1): {best_threshold} -> F1={best_f1_ens:.4f} (saved to {BEST_THRESHOLD_PATH})"
478
+ )
479
+
480
+ pred_ens = (prob_ens[:, 1] >= best_threshold).astype(int)
481
+
482
+ def metrics_row(name, preds):
483
+ acc = accuracy_score(y_true, preds)
484
+ f1 = f1_score(y_true, preds, average="macro", zero_division=0)
485
+ prec = precision_score(y_true, preds, average="macro", zero_division=0)
486
+ rec = recall_score(y_true, preds, average="macro", zero_division=0)
487
+ cm = confusion_matrix(y_true, preds)
488
+ return {"name": name, "acc": acc, "f1": f1, "prec": prec, "rec": rec, "cm": cm}
489
+
490
+ pred_lgb = np.argmax(prob_lgb, axis=1) if prob_lgb is not None else None
491
+ n_ens = len(prob_list)
492
+ results = [
493
+ metrics_row("MLP (baseline)", pred_mlp),
494
+ metrics_row("MLP + attention (prosody)", pred_att),
495
+ metrics_row(f"Ensemble ({n_ens} models, t={best_threshold})", pred_ens),
496
+ ]
497
+ if pred_lgb is not None:
498
+ results.append(metrics_row("LightGBM", pred_lgb))
499
+
500
+ print("\n" + "=" * 60)
501
+ print("COMPARISON (validation set)")
502
+ print("=" * 60)
503
+ print(
504
+ f"{'Model':<30} {'Accuracy':>10} {'F1 macro':>10} {'Precision':>10} {'Recall':>10}"
505
+ )
506
+ print("-" * 72)
507
+ for r in results:
508
+ print(
509
+ f"{r['name']:<30} {r['acc']:>10.4f} {r['f1']:>10.4f} {r['prec']:>10.4f} {r['rec']:>10.4f}"
510
+ )
511
+ print("\nConfusion matrices (rows=true, cols=pred), Native=0, Non-Native=1:")
512
+ for r in results:
513
+ print(f" {r['name']}:")
514
+ print(f" {r['cm']}")
515
+
516
+ best = max(results, key=lambda x: x["f1"])
517
+ print(f"\nBest by F1: {best['name']} (F1={best['f1']:.4f})")
518
+ print(
519
+ "\nSaved: models/nativity_model.pth, models/nativity_model_attention.pth, models/best_threshold.txt"
520
+ )
521
+ print(" models/lgbm_nativity.txt (if LightGBM)")
522
+
523
+
524
+ if __name__ == "__main__":
525
+ main()
scripts/train_classifier_english.py DELETED
@@ -1,217 +0,0 @@
1
- """
2
- Train Native vs Non-Native classifier for English (same architecture as Arabic).
3
- Uses manifests/final_training_manifest_english.csv, saves models/nativity_model_english.pth.
4
- """
5
- import os
6
- import sys
7
- import random
8
-
9
- sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
10
- import pandas as pd
11
- import numpy as np
12
- import torch
13
- import torch.nn as nn
14
- import torch.optim as optim
15
- from torch.utils.data import DataLoader
16
- from sklearn.model_selection import train_test_split
17
- from sklearn.preprocessing import LabelEncoder
18
- from sklearn.utils.class_weight import compute_class_weight
19
- from sklearn.metrics import (
20
- accuracy_score,
21
- f1_score,
22
- precision_score,
23
- recall_score,
24
- confusion_matrix,
25
- )
26
-
27
- from model import NativityClassifier
28
-
29
- PROJECT_ROOT = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
30
- MANIFEST_PATH = os.path.join(PROJECT_ROOT, "manifests", "final_training_manifest_english.csv")
31
- MODEL_SAVE_DIR = os.path.join(PROJECT_ROOT, "models")
32
- BEST_MODEL_PATH = os.path.join(MODEL_SAVE_DIR, "nativity_model_english.pth")
33
-
34
- SEED = 42
35
- EPOCHS = 200
36
- BATCH_SIZE = 32
37
- LR = 1e-3
38
- WEIGHT_DECAY = 1e-4
39
- PATIENCE = 30
40
- MIN_LR = 1e-6
41
- HIDDEN_DIMS = (384, 256, 128)
42
- DROPOUT = 0.35
43
- INPUT_DIM = 1024
44
- OUTPUT_DIM = 2
45
-
46
-
47
- def set_seed(seed):
48
- random.seed(seed)
49
- np.random.seed(seed)
50
- torch.manual_seed(seed)
51
- if torch.cuda.is_available():
52
- torch.cuda.manual_seed_all(seed)
53
- torch.backends.cudnn.deterministic = True
54
- torch.backends.cudnn.benchmark = False
55
- else:
56
- torch.backends.cudnn.benchmark = True
57
-
58
-
59
- class EmbeddingDataset(torch.utils.data.Dataset):
60
- def __init__(self, df, embedding_path_col="embedding_path", label_col="label_idx", l2_norm=True):
61
- self.paths = df[embedding_path_col].tolist()
62
- self.labels = df[label_col].values
63
- self.l2_norm = l2_norm
64
-
65
- def __len__(self):
66
- return len(self.labels)
67
-
68
- def __getitem__(self, idx):
69
- emb = np.load(self.paths[idx]).astype(np.float32)
70
- if self.l2_norm:
71
- n = np.linalg.norm(emb)
72
- if n > 1e-8:
73
- emb = emb / n
74
- return torch.tensor(emb, dtype=torch.float32), torch.tensor(
75
- self.labels[idx], dtype=torch.long
76
- )
77
-
78
-
79
- def main():
80
- set_seed(SEED)
81
- device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
82
- print(f"Device: {device}")
83
-
84
- os.makedirs(MODEL_SAVE_DIR, exist_ok=True)
85
- df = pd.read_csv(MANIFEST_PATH)
86
- le = LabelEncoder()
87
- df["label_idx"] = le.fit_transform(df["label"])
88
- print(f"Classes: {le.classes_} (Native=0, Non-Native=1)")
89
- print(f"Total samples: {len(df)}")
90
-
91
- weights = compute_class_weight(
92
- "balanced", classes=np.unique(df["label_idx"]), y=df["label_idx"]
93
- )
94
- class_weights = torch.tensor(weights, dtype=torch.float32).to(device)
95
-
96
- # Split by HUMAN speaker (speaker_id from folder path), not by file (original_id).
97
- # Otherwise the same person can have files in both train and val -> inflated 99% accuracy.
98
- if "speaker_id" not in df.columns:
99
- raise RuntimeError(
100
- "Manifest missing 'speaker_id'. Re-run load_data_english.py and extract_embeddings_english.py "
101
- "to generate a manifest with speaker_id (folder path under Native/ or Non Native/)."
102
- )
103
- speaker_df = df.groupby("speaker_id")["label_idx"].first().reset_index()
104
- train_speaker_ids, val_speaker_ids = train_test_split(
105
- speaker_df["speaker_id"],
106
- test_size=0.2,
107
- stratify=speaker_df["label_idx"],
108
- random_state=SEED,
109
- )
110
- train_df = df[df["speaker_id"].isin(train_speaker_ids)].copy()
111
- val_df = df[df["speaker_id"].isin(val_speaker_ids)].copy()
112
- print(f"Train: {len(train_df)} chunks ({len(train_speaker_ids)} speakers), Val: {len(val_df)} chunks ({len(val_speaker_ids)} speakers)")
113
-
114
- train_set = EmbeddingDataset(train_df, l2_norm=True)
115
- val_set = EmbeddingDataset(val_df, l2_norm=True)
116
- train_loader = DataLoader(
117
- train_set,
118
- batch_size=BATCH_SIZE,
119
- shuffle=True,
120
- num_workers=0,
121
- pin_memory=(device.type == "cuda"),
122
- drop_last=True, # avoid batch size 1 -> BatchNorm needs >1 sample in train mode
123
- )
124
- val_loader = DataLoader(val_set, batch_size=BATCH_SIZE, shuffle=False, num_workers=0)
125
-
126
- model = NativityClassifier(
127
- input_dim=INPUT_DIM,
128
- hidden_dims=HIDDEN_DIMS,
129
- output_dim=OUTPUT_DIM,
130
- dropout=DROPOUT,
131
- ).to(device)
132
- criterion = nn.CrossEntropyLoss(weight=class_weights)
133
- optimizer = optim.AdamW(model.parameters(), lr=LR, weight_decay=WEIGHT_DECAY)
134
- scheduler = optim.lr_scheduler.ReduceLROnPlateau(
135
- optimizer, mode="max", factor=0.5, patience=5, min_lr=MIN_LR
136
- )
137
-
138
- best_val_f1 = 0.0
139
- best_epoch = 0
140
- patience_counter = 0
141
-
142
- for epoch in range(EPOCHS):
143
- model.train()
144
- total_loss = 0.0
145
- for batch_x, batch_y in train_loader:
146
- batch_x, batch_y = batch_x.to(device), batch_y.to(device)
147
- optimizer.zero_grad()
148
- logits = model(batch_x)
149
- loss = criterion(logits, batch_y)
150
- loss.backward()
151
- torch.nn.utils.clip_grad_norm_(model.parameters(), max_norm=1.0)
152
- optimizer.step()
153
- total_loss += loss.item()
154
- train_loss = total_loss / len(train_loader)
155
-
156
- model.eval()
157
- all_preds, all_labels = [], []
158
- val_loss = 0.0
159
- with torch.no_grad():
160
- for batch_x, batch_y in val_loader:
161
- batch_x, batch_y = batch_x.to(device), batch_y.to(device)
162
- logits = model(batch_x)
163
- val_loss += criterion(logits, batch_y).item()
164
- preds = logits.argmax(dim=1)
165
- all_preds.extend(preds.cpu().numpy())
166
- all_labels.extend(batch_y.cpu().numpy())
167
- all_preds = np.array(all_preds)
168
- all_labels = np.array(all_labels)
169
- val_loss /= len(val_loader)
170
- acc = accuracy_score(all_labels, all_preds)
171
- f1 = f1_score(all_labels, all_preds, average="macro", zero_division=0)
172
- prec = precision_score(all_labels, all_preds, average="macro", zero_division=0)
173
- rec = recall_score(all_labels, all_preds, average="macro", zero_division=0)
174
- scheduler.step(f1)
175
-
176
- if f1 > best_val_f1:
177
- best_val_f1 = f1
178
- best_epoch = epoch + 1
179
- patience_counter = 0
180
- torch.save(model.state_dict(), BEST_MODEL_PATH)
181
- print(f" -> New best model saved (val F1={f1:.4f})")
182
- else:
183
- patience_counter += 1
184
-
185
- print(
186
- f"Epoch {epoch + 1:3d} | loss {train_loss:.4f} | val_loss {val_loss:.4f} | "
187
- f"val_acc {acc:.4f} | val_F1 {f1:.4f} | val_P {prec:.4f} | val_R {rec:.4f}"
188
- )
189
- if patience_counter >= PATIENCE:
190
- print(f"Early stopping at epoch {epoch + 1}.")
191
- break
192
-
193
- print(f"\nBest val F1: {best_val_f1:.4f} (epoch {best_epoch})")
194
- model.load_state_dict(torch.load(BEST_MODEL_PATH, map_location=device))
195
- model.eval()
196
- all_preds, all_labels = [], []
197
- with torch.no_grad():
198
- for batch_x, batch_y in val_loader:
199
- batch_x = batch_x.to(device)
200
- logits = model(batch_x)
201
- all_preds.extend(logits.argmax(dim=1).cpu().numpy())
202
- all_labels.extend(batch_y.numpy())
203
- all_preds = np.array(all_preds)
204
- all_labels = np.array(all_labels)
205
- print("\nFinal validation (best model):")
206
- print(f" Accuracy: {accuracy_score(all_labels, all_preds):.4f}")
207
- print(f" F1 macro: {f1_score(all_labels, all_preds, average='macro', zero_division=0):.4f}")
208
- print(f" Precision: {precision_score(all_labels, all_preds, average='macro', zero_division=0):.4f}")
209
- print(f" Recall: {recall_score(all_labels, all_preds, average='macro', zero_division=0):.4f}")
210
- print(" Confusion matrix (rows=true, cols=pred):")
211
- print(f" {confusion_matrix(all_labels, all_preds)}")
212
- print(f" (Native=0, Non-Native=1)")
213
- print(f"\nModel saved to {BEST_MODEL_PATH}")
214
-
215
-
216
- if __name__ == "__main__":
217
- main()