Instructions to use espnet/geolid_combined_shared_trainable with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- ESPnet
How to use espnet/geolid_combined_shared_trainable with ESPnet:
unknown model type (must be text-to-speech or automatic-speech-recognition)
- Notebooks
- Google Colab
- Kaggle
Report complete VoxLingua107 dev rerun with unified ESPnetLIDModel
Browse files- README.md +35 -20
- evaluation.json +166 -12
- evaluation.log +425 -0
- evaluation_prediction_differences.json +1 -0
- evaluation_predictions.jsonl +0 -0
- evaluation_previous.json +38 -0
- evaluation_report.md +61 -0
- validation.json +27 -0
README.md
CHANGED
|
@@ -189,25 +189,39 @@ converted checkpoint is reported separately from historical paper results below.
|
|
| 189 |
| --- | ---: | ---: | ---: |
|
| 190 |
| VoxLingua107 dev | 94.4 | **93.29** | 1501 / 1609 |
|
| 191 |
|
| 192 |
-
The converted checkpoint was evaluated on all 1,609 VoxLingua107 dev
|
| 193 |
-
|
| 194 |
-
|
| 195 |
-
|
| 196 |
-
|
| 197 |
-
|
| 198 |
-
|
|
|
|
|
|
|
| 199 |
|
| 200 |
-
[Evaluation
|
| 201 |
-
|
| 202 |
-
|
| 203 |
-
|
| 204 |
-
|
| 205 |
-
|
| 206 |
-
|
| 207 |
-
Historical paper scores for other datasets are not new measurements of this
|
| 208 |
-
converted checkpoint.
|
| 209 |
|
| 210 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 211 |
|
| 212 |
## Installation
|
| 213 |
|
|
@@ -257,9 +271,10 @@ from the original model card and the original sorted recipe label construction;
|
|
| 257 |
it is not a copy of the missing original training utterance manifest.
|
| 258 |
|
| 259 |
`conversion.json` records the source checkpoint revision and label provenance.
|
| 260 |
-
`validation.json`
|
| 261 |
-
|
| 262 |
-
|
|
|
|
| 263 |
uses speech and predicts its own conditioning vectors.
|
| 264 |
The old training logs and figures retained under `exp_*` describe the original
|
| 265 |
training run. Earlier checkpoint layouts remain available in repository history.
|
|
|
|
| 189 |
| --- | ---: | ---: | ---: |
|
| 190 |
| VoxLingua107 dev | 94.4 | **93.29** | 1501 / 1609 |
|
| 191 |
|
| 192 |
+
The converted checkpoint was evaluated again on all **1,609 VoxLingua107 dev
|
| 193 |
+
utterances** after integrating lang2vec into the standard `ESPnetLIDModel`.
|
| 194 |
+
This full evaluation used the unmodified `espnet2.bin.lid_inference` CLI at
|
| 195 |
+
ESPnet revision `301372e4da311d2a3d9574a5a43b3037ae6e67c1`: GPU float32,
|
| 196 |
+
batch size 1, 8 data-loading workers, full-length waveforms,
|
| 197 |
+
`--use_preprocessor false` and `--fix_duration false`. The input `wav.scp`
|
| 198 |
+
was sorted by utterance ID. The environment used PyTorch 2.11.0+cu128,
|
| 199 |
+
Transformers 5.14.1 and espnet-s3prl 2026.5.14.
|
| 200 |
+
Accuracy is the fraction of correctly classified utterances.
|
| 201 |
|
| 202 |
+
[Evaluation report and reproduction command](evaluation_report.md),
|
| 203 |
+
[all 1,609 predictions with reference labels](evaluation_predictions.jsonl),
|
| 204 |
+
[inference log](evaluation.log), and [evaluation metadata](evaluation.json)
|
| 205 |
+
provide the results, runtime settings, input revision and checkpoint SHA-256.
|
| 206 |
+
All 1,609 predicted labels match the earlier full-dev evaluation at `ca777d59eb49519e3827e40cc34997f2123d6682`.
|
| 207 |
+
The previous evaluation metadata is preserved in
|
| 208 |
+
[evaluation_previous.json](evaluation_previous.json).
|
|
|
|
|
|
|
| 209 |
|
| 210 |
+
These are batch-size-1 results. `utterance_mvn` and final `chn_attn_stat`
|
| 211 |
+
pooling do mask padding. The remaining batch dependence comes earlier:
|
| 212 |
+
S3PRL aligns features to `ceil(samples / 320)`, so the extra frames of a
|
| 213 |
+
shorter utterance can differ from its single-utterance run, and ECAPA's
|
| 214 |
+
temporal convolutions and unmasked SE global means depend on padded sequence
|
| 215 |
+
length. Final masked pooling cannot undo those feature changes. Historical
|
| 216 |
+
paper scores for other datasets are not new measurements of this checkpoint.
|
| 217 |
+
|
| 218 |
+
The configuration selects the standard `ESPnetLIDModel`; lang2vec prediction
|
| 219 |
+
and conditioning are optional features of that model. The refactor preserved
|
| 220 |
+
the converted parameter names and weight file. The earlier strict-loading
|
| 221 |
+
and six-utterance checks remain recorded at their original revisions in
|
| 222 |
+
[validation.json](validation.json). Its separate `full_dev_model_integration`
|
| 223 |
+
record describes this new complete evaluation with the stock CLI loader;
|
| 224 |
+
the full evaluation did not instrument that loader to force strict loading.
|
| 225 |
|
| 226 |
## Installation
|
| 227 |
|
|
|
|
| 271 |
it is not a copy of the missing original training utterance manifest.
|
| 272 |
|
| 273 |
`conversion.json` records the source checkpoint revision and label provenance.
|
| 274 |
+
`validation.json` preserves the historical strict-loading and layout-migration
|
| 275 |
+
checks, and separately records the complete evaluation after model integration.
|
| 276 |
+
`evaluation.json` records the latest measured accuracy, with per-utterance
|
| 277 |
+
results in `evaluation_predictions.jsonl`. The metadata targets are used only for training; inference
|
| 278 |
uses speech and predicts its own conditioning vectors.
|
| 279 |
The old training logs and figures retained under `exp_*` describe the original
|
| 280 |
training run. Earlier checkpoint layouts remain available in repository history.
|
evaluation.json
CHANGED
|
@@ -1,11 +1,20 @@
|
|
| 1 |
{
|
| 2 |
"dataset": "VoxLingua107",
|
| 3 |
"split": "dev",
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 4 |
"utterances": 1609,
|
| 5 |
"languages": 33,
|
| 6 |
"metric": "utterance-level accuracy (%)",
|
|
|
|
|
|
|
|
|
|
| 7 |
"entrypoint": "espnet2.bin.lid_inference",
|
| 8 |
-
"implementation_revision": "ca777d59eb49519e3827e40cc34997f2123d6682",
|
| 9 |
"device": "cuda",
|
| 10 |
"dtype": "float32",
|
| 11 |
"batch_size": 1,
|
|
@@ -13,26 +22,171 @@
|
|
| 13 |
"use_preprocessor": false,
|
| 14 |
"fix_duration": false,
|
| 15 |
"input_order": "wav.scp sorted by utterance ID; standard ESPnet streaming iterator",
|
| 16 |
-
"manifest_sha256": "6ad3c8fc6c74fab2720fe186525fc3bd20c4ab7b1be35d227c5c7f5456ba6c54",
|
| 17 |
"environment": {
|
| 18 |
-
"python": "3.12.3
|
| 19 |
"torch": "2.11.0+cu128",
|
| 20 |
-
"torch_cuda_build": "12.8",
|
| 21 |
"torchaudio": "2.11.0+cu128",
|
| 22 |
"transformers": "5.14.1",
|
| 23 |
-
"
|
| 24 |
"soundfile": "0.14.0"
|
| 25 |
},
|
| 26 |
-
"
|
| 27 |
-
"
|
| 28 |
-
"
|
| 29 |
-
"
|
| 30 |
-
"
|
| 31 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 32 |
"batch_dependence": {
|
| 33 |
"s3prl_frame_alignment": "S3PRL reports ceil(samples/320) frames and repeats the final frame only at the batch boundary. Shorter items can instead expose padded-query frames from Wav2Vec2.",
|
| 34 |
"ecapa_tdnn": "Temporal convolutions and unmasked SE global means depend on the padded sequence length before the masked final pooling.",
|
| 35 |
"mvn_and_final_pooling_mask_padding": true,
|
| 36 |
"reported_measurement_batch_size": 1
|
| 37 |
-
}
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 38 |
}
|
|
|
|
| 1 |
{
|
| 2 |
"dataset": "VoxLingua107",
|
| 3 |
"split": "dev",
|
| 4 |
+
"repo_id": "espnet/geolid_combined_shared_trainable",
|
| 5 |
+
"source_hf_revision": "613d18645d1be3bd585a2cbae0485350a2a4c027",
|
| 6 |
+
"implementation_revision": "301372e4da311d2a3d9574a5a43b3037ae6e67c1",
|
| 7 |
+
"model": "ESPnetLIDModel",
|
| 8 |
+
"model_sha256": "8a8f9dd767c141dcae0cb34a7780d7f593bc67f560267cd4d2d2fadd54f6bd16",
|
| 9 |
+
"config_sha256": "b06ed46b21a44a48ee0860407d1fa772344fedae302380f74ae2b02df7757abb",
|
| 10 |
+
"manifest_sha256": "6ad3c8fc6c74fab2720fe186525fc3bd20c4ab7b1be35d227c5c7f5456ba6c54",
|
| 11 |
"utterances": 1609,
|
| 12 |
"languages": 33,
|
| 13 |
"metric": "utterance-level accuracy (%)",
|
| 14 |
+
"correct": 1501,
|
| 15 |
+
"accuracy_percent": 93.2877563704164,
|
| 16 |
+
"paper_accuracy_percent": 94.4,
|
| 17 |
"entrypoint": "espnet2.bin.lid_inference",
|
|
|
|
| 18 |
"device": "cuda",
|
| 19 |
"dtype": "float32",
|
| 20 |
"batch_size": 1,
|
|
|
|
| 22 |
"use_preprocessor": false,
|
| 23 |
"fix_duration": false,
|
| 24 |
"input_order": "wav.scp sorted by utterance ID; standard ESPnet streaming iterator",
|
|
|
|
| 25 |
"environment": {
|
| 26 |
+
"python": "3.12.3",
|
| 27 |
"torch": "2.11.0+cu128",
|
|
|
|
| 28 |
"torchaudio": "2.11.0+cu128",
|
| 29 |
"transformers": "5.14.1",
|
| 30 |
+
"espnet-s3prl": "2026.5.14",
|
| 31 |
"soundfile": "0.14.0"
|
| 32 |
},
|
| 33 |
+
"finished_at": "2026-10-23T00:55:52.840208+00:00",
|
| 34 |
+
"seconds": 161.17425368726254,
|
| 35 |
+
"predictions_file": "evaluation_predictions.jsonl",
|
| 36 |
+
"predictions_sha256": "40264d2766f60dfce717ebc638ef76bc85380709693ca3e32178829e5df5453e",
|
| 37 |
+
"comparison": {
|
| 38 |
+
"baseline_implementation_revision": "ca777d59eb49519e3827e40cc34997f2123d6682",
|
| 39 |
+
"baseline_correct": 1501,
|
| 40 |
+
"matched_predictions": 1609,
|
| 41 |
+
"changed_predictions": 0,
|
| 42 |
+
"baseline_predictions_sha256": "40264d2766f60dfce717ebc638ef76bc85380709693ca3e32178829e5df5453e"
|
| 43 |
+
},
|
| 44 |
+
"per_language": {
|
| 45 |
+
"ara": {
|
| 46 |
+
"total": 100,
|
| 47 |
+
"correct": 95
|
| 48 |
+
},
|
| 49 |
+
"aze": {
|
| 50 |
+
"total": 34,
|
| 51 |
+
"correct": 31
|
| 52 |
+
},
|
| 53 |
+
"cmn": {
|
| 54 |
+
"total": 23,
|
| 55 |
+
"correct": 22
|
| 56 |
+
},
|
| 57 |
+
"dan": {
|
| 58 |
+
"total": 99,
|
| 59 |
+
"correct": 93
|
| 60 |
+
},
|
| 61 |
+
"deu": {
|
| 62 |
+
"total": 82,
|
| 63 |
+
"correct": 77
|
| 64 |
+
},
|
| 65 |
+
"ell": {
|
| 66 |
+
"total": 5,
|
| 67 |
+
"correct": 4
|
| 68 |
+
},
|
| 69 |
+
"eng": {
|
| 70 |
+
"total": 80,
|
| 71 |
+
"correct": 73
|
| 72 |
+
},
|
| 73 |
+
"est": {
|
| 74 |
+
"total": 85,
|
| 75 |
+
"correct": 80
|
| 76 |
+
},
|
| 77 |
+
"fas": {
|
| 78 |
+
"total": 100,
|
| 79 |
+
"correct": 89
|
| 80 |
+
},
|
| 81 |
+
"fin": {
|
| 82 |
+
"total": 93,
|
| 83 |
+
"correct": 91
|
| 84 |
+
},
|
| 85 |
+
"fra": {
|
| 86 |
+
"total": 100,
|
| 87 |
+
"correct": 96
|
| 88 |
+
},
|
| 89 |
+
"hrv": {
|
| 90 |
+
"total": 4,
|
| 91 |
+
"correct": 3
|
| 92 |
+
},
|
| 93 |
+
"hun": {
|
| 94 |
+
"total": 4,
|
| 95 |
+
"correct": 4
|
| 96 |
+
},
|
| 97 |
+
"hye": {
|
| 98 |
+
"total": 100,
|
| 99 |
+
"correct": 96
|
| 100 |
+
},
|
| 101 |
+
"isl": {
|
| 102 |
+
"total": 26,
|
| 103 |
+
"correct": 26
|
| 104 |
+
},
|
| 105 |
+
"ita": {
|
| 106 |
+
"total": 49,
|
| 107 |
+
"correct": 49
|
| 108 |
+
},
|
| 109 |
+
"jpn": {
|
| 110 |
+
"total": 42,
|
| 111 |
+
"correct": 41
|
| 112 |
+
},
|
| 113 |
+
"lav": {
|
| 114 |
+
"total": 95,
|
| 115 |
+
"correct": 94
|
| 116 |
+
},
|
| 117 |
+
"lit": {
|
| 118 |
+
"total": 13,
|
| 119 |
+
"correct": 12
|
| 120 |
+
},
|
| 121 |
+
"mkd": {
|
| 122 |
+
"total": 4,
|
| 123 |
+
"correct": 4
|
| 124 |
+
},
|
| 125 |
+
"nld": {
|
| 126 |
+
"total": 100,
|
| 127 |
+
"correct": 95
|
| 128 |
+
},
|
| 129 |
+
"nno": {
|
| 130 |
+
"total": 3,
|
| 131 |
+
"correct": 3
|
| 132 |
+
},
|
| 133 |
+
"nor": {
|
| 134 |
+
"total": 24,
|
| 135 |
+
"correct": 13
|
| 136 |
+
},
|
| 137 |
+
"pol": {
|
| 138 |
+
"total": 2,
|
| 139 |
+
"correct": 2
|
| 140 |
+
},
|
| 141 |
+
"por": {
|
| 142 |
+
"total": 1,
|
| 143 |
+
"correct": 1
|
| 144 |
+
},
|
| 145 |
+
"rus": {
|
| 146 |
+
"total": 29,
|
| 147 |
+
"correct": 29
|
| 148 |
+
},
|
| 149 |
+
"slv": {
|
| 150 |
+
"total": 9,
|
| 151 |
+
"correct": 9
|
| 152 |
+
},
|
| 153 |
+
"spa": {
|
| 154 |
+
"total": 55,
|
| 155 |
+
"correct": 47
|
| 156 |
+
},
|
| 157 |
+
"srp": {
|
| 158 |
+
"total": 7,
|
| 159 |
+
"correct": 5
|
| 160 |
+
},
|
| 161 |
+
"swe": {
|
| 162 |
+
"total": 100,
|
| 163 |
+
"correct": 94
|
| 164 |
+
},
|
| 165 |
+
"tur": {
|
| 166 |
+
"total": 72,
|
| 167 |
+
"correct": 71
|
| 168 |
+
},
|
| 169 |
+
"ukr": {
|
| 170 |
+
"total": 17,
|
| 171 |
+
"correct": 17
|
| 172 |
+
},
|
| 173 |
+
"urd": {
|
| 174 |
+
"total": 52,
|
| 175 |
+
"correct": 35
|
| 176 |
+
}
|
| 177 |
+
},
|
| 178 |
"batch_dependence": {
|
| 179 |
"s3prl_frame_alignment": "S3PRL reports ceil(samples/320) frames and repeats the final frame only at the batch boundary. Shorter items can instead expose padded-query frames from Wav2Vec2.",
|
| 180 |
"ecapa_tdnn": "Temporal convolutions and unmasked SE global means depend on the padded sequence length before the masked final pooling.",
|
| 181 |
"mvn_and_final_pooling_mask_padding": true,
|
| 182 |
"reported_measurement_batch_size": 1
|
| 183 |
+
},
|
| 184 |
+
"resolved_config_sha256": "fe2e8acd705871a4dea823f80c7afabfcb89b2cb40cf51161178b6f3209b72f5",
|
| 185 |
+
"languages_sha256": "285a350acd7dc5b109e950e54560b8bab70d282bae12b25328327d28a2b5226d",
|
| 186 |
+
"log_file": "evaluation.log",
|
| 187 |
+
"log_sha256": "d6831bc2d1e154c487052b31b10adc9072db4c8d045e4a1c6897125aa7a4c6c1",
|
| 188 |
+
"prediction_differences_file": "evaluation_prediction_differences.json",
|
| 189 |
+
"previous_evaluation_file": "evaluation_previous.json",
|
| 190 |
+
"source_config_url": "https://huggingface.co/espnet/geolid_combined_shared_trainable/blob/613d18645d1be3bd585a2cbae0485350a2a4c027/config.yaml",
|
| 191 |
+
"loader": "Unmodified standard ESPnet CLI loader; strict loading was not forced"
|
| 192 |
}
|
evaluation.log
ADDED
|
@@ -0,0 +1,425 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
Failed to import Flash Attention, using ESPnet default: No module named 'flash_attn'
|
| 2 |
+
/home/qingzhengw/espnet/.geolid-reproduction/venv/bin/python /home/qingzhengw/espnet/.geolid-reproduction/review-code/espnet2/bin/lid_inference.py --output_dir /home/qingzhengw/espnet/.geolid-reproduction/full-dev-unified-301372e4/models/combined_shared_trainable --ngpu 1 --dtype float32 --data_path_and_name_and_type /home/qingzhengw/espnet/.geolid-reproduction/full-dev-unified-301372e4/data/voxlingua107-dev/wav.scp,speech,sound --valid_batch_size 1 --lid_train_config /home/qingzhengw/espnet/.geolid-reproduction/full-dev-unified-301372e4/models/combined_shared_trainable/config.resolved.yaml --lid_model_file /home/qingzhengw/espnet/.geolid-reproduction/round2-native/combined_shared_trainable/model.pth --use_preprocessor false --fix_duration false --num_workers 8 --extract_embd false --save_embd_per_utt false --save_embd_avg_lang false --save_tsne_plot false --checkpoint_interval 200
|
| 3 |
+
[host-10-0-93-146] 2026-10-23 00:53:32,012 (abs_task:2514) INFO: config file: /home/qingzhengw/espnet/.geolid-reproduction/full-dev-unified-301372e4/models/combined_shared_trainable/config.resolved.yaml
|
| 4 |
+
|
| 5 |
+
[transformers] [1mWav2Vec2Model LOAD REPORT[0m from: facebook/mms-1b
|
| 6 |
+
Key | Status | |
|
| 7 |
+
-----------------------------+------------+--+-
|
| 8 |
+
project_q.weight | UNEXPECTED | |
|
| 9 |
+
quantizer.codevectors | UNEXPECTED | |
|
| 10 |
+
project_hid.weight | UNEXPECTED | |
|
| 11 |
+
quantizer.weight_proj.weight | UNEXPECTED | |
|
| 12 |
+
project_q.bias | UNEXPECTED | |
|
| 13 |
+
project_hid.bias | UNEXPECTED | |
|
| 14 |
+
quantizer.weight_proj.bias | UNEXPECTED | |
|
| 15 |
+
|
| 16 |
+
Notes:
|
| 17 |
+
- UNEXPECTED: can be ignored when loading from different task/architecture; not ok if you expect identical arch.
|
| 18 |
+
[host-10-0-93-146] 2026-10-23 00:53:43,296 (lid_inference:111) INFO: Model structure:
|
| 19 |
+
ESPnetLIDModel(
|
| 20 |
+
(frontend): S3prlFrontend(
|
| 21 |
+
(upstream): S3PRLUpstream(
|
| 22 |
+
(upstream): UpstreamExpert(
|
| 23 |
+
(model): Wav2Vec2Model(
|
| 24 |
+
(feature_extractor): Wav2Vec2FeatureEncoder(
|
| 25 |
+
(conv_layers): ModuleList(
|
| 26 |
+
(0): Wav2Vec2LayerNormConvLayer(
|
| 27 |
+
(conv): Conv1d(1, 512, kernel_size=(10,), stride=(5,))
|
| 28 |
+
(layer_norm): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
|
| 29 |
+
(activation): GELUActivation()
|
| 30 |
+
)
|
| 31 |
+
(1-4): 4 x Wav2Vec2LayerNormConvLayer(
|
| 32 |
+
(conv): Conv1d(512, 512, kernel_size=(3,), stride=(2,))
|
| 33 |
+
(layer_norm): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
|
| 34 |
+
(activation): GELUActivation()
|
| 35 |
+
)
|
| 36 |
+
(5-6): 2 x Wav2Vec2LayerNormConvLayer(
|
| 37 |
+
(conv): Conv1d(512, 512, kernel_size=(2,), stride=(2,))
|
| 38 |
+
(layer_norm): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
|
| 39 |
+
(activation): GELUActivation()
|
| 40 |
+
)
|
| 41 |
+
)
|
| 42 |
+
)
|
| 43 |
+
(feature_projection): Wav2Vec2FeatureProjection(
|
| 44 |
+
(layer_norm): LayerNorm((512,), eps=1e-05, elementwise_affine=True)
|
| 45 |
+
(projection): Linear(in_features=512, out_features=1280, bias=True)
|
| 46 |
+
(dropout): Dropout(p=0.1, inplace=False)
|
| 47 |
+
)
|
| 48 |
+
(encoder): Wav2Vec2EncoderStableLayerNorm(
|
| 49 |
+
(pos_conv_embed): Wav2Vec2PositionalConvEmbedding(
|
| 50 |
+
(conv): ParametrizedConv1d(
|
| 51 |
+
1280, 1280, kernel_size=(128,), stride=(1,), padding=(64,), groups=16
|
| 52 |
+
(parametrizations): ModuleDict(
|
| 53 |
+
(weight): ParametrizationList(
|
| 54 |
+
(0): _WeightNorm()
|
| 55 |
+
)
|
| 56 |
+
)
|
| 57 |
+
)
|
| 58 |
+
(padding): Wav2Vec2SamePadLayer()
|
| 59 |
+
(activation): GELUActivation()
|
| 60 |
+
)
|
| 61 |
+
(layer_norm): LayerNorm((1280,), eps=1e-05, elementwise_affine=True)
|
| 62 |
+
(dropout): Dropout(p=0.1, inplace=False)
|
| 63 |
+
(layers): ModuleList(
|
| 64 |
+
(0-31): 32 x Wav2Vec2EncoderLayerStableLayerNorm(
|
| 65 |
+
(attention): Wav2Vec2Attention(
|
| 66 |
+
(k_proj): Linear(in_features=1280, out_features=1280, bias=True)
|
| 67 |
+
(v_proj): Linear(in_features=1280, out_features=1280, bias=True)
|
| 68 |
+
(q_proj): Linear(in_features=1280, out_features=1280, bias=True)
|
| 69 |
+
(out_proj): Linear(in_features=1280, out_features=1280, bias=True)
|
| 70 |
+
)
|
| 71 |
+
(dropout): Dropout(p=0.1, inplace=False)
|
| 72 |
+
(layer_norm): LayerNorm((1280,), eps=1e-05, elementwise_affine=True)
|
| 73 |
+
(feed_forward): Wav2Vec2FeedForward(
|
| 74 |
+
(intermediate_dropout): Dropout(p=0.0, inplace=False)
|
| 75 |
+
(intermediate_dense): Linear(in_features=1280, out_features=5120, bias=True)
|
| 76 |
+
(intermediate_act_fn): GELUActivation()
|
| 77 |
+
(output_dense): Linear(in_features=5120, out_features=1280, bias=True)
|
| 78 |
+
(output_dropout): Dropout(p=0.1, inplace=False)
|
| 79 |
+
)
|
| 80 |
+
(final_layer_norm): LayerNorm((1280,), eps=1e-05, elementwise_affine=True)
|
| 81 |
+
)
|
| 82 |
+
(32): Lang2VecConditionedLayer(
|
| 83 |
+
(layer): Wav2Vec2EncoderLayerStableLayerNorm(
|
| 84 |
+
(attention): Wav2Vec2Attention(
|
| 85 |
+
(k_proj): Linear(in_features=1280, out_features=1280, bias=True)
|
| 86 |
+
(v_proj): Linear(in_features=1280, out_features=1280, bias=True)
|
| 87 |
+
(q_proj): Linear(in_features=1280, out_features=1280, bias=True)
|
| 88 |
+
(out_proj): Linear(in_features=1280, out_features=1280, bias=True)
|
| 89 |
+
)
|
| 90 |
+
(dropout): Dropout(p=0.1, inplace=False)
|
| 91 |
+
(layer_norm): LayerNorm((1280,), eps=1e-05, elementwise_affine=True)
|
| 92 |
+
(feed_forward): Wav2Vec2FeedForward(
|
| 93 |
+
(intermediate_dropout): Dropout(p=0.0, inplace=False)
|
| 94 |
+
(intermediate_dense): Linear(in_features=1280, out_features=5120, bias=True)
|
| 95 |
+
(intermediate_act_fn): GELUActivation()
|
| 96 |
+
(output_dense): Linear(in_features=5120, out_features=1280, bias=True)
|
| 97 |
+
(output_dropout): Dropout(p=0.1, inplace=False)
|
| 98 |
+
)
|
| 99 |
+
(final_layer_norm): LayerNorm((1280,), eps=1e-05, elementwise_affine=True)
|
| 100 |
+
)
|
| 101 |
+
(encoder): IdentityEncoder()
|
| 102 |
+
(pooling): ChnAttnStatPooling(
|
| 103 |
+
(attention): Sequential(
|
| 104 |
+
(0): Conv1d(3840, 128, kernel_size=(1,), stride=(1,))
|
| 105 |
+
(1): ReLU()
|
| 106 |
+
(2): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
|
| 107 |
+
(3): Conv1d(128, 1280, kernel_size=(1,), stride=(1,))
|
| 108 |
+
)
|
| 109 |
+
(softmax): Softmax(dim=2)
|
| 110 |
+
)
|
| 111 |
+
(projector): RawNet3Projector(
|
| 112 |
+
(bn): BatchNorm1d(2560, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
|
| 113 |
+
(fc): Linear(in_features=2560, out_features=192, bias=True)
|
| 114 |
+
)
|
| 115 |
+
(prediction): Linear(in_features=192, out_features=299, bias=True)
|
| 116 |
+
(conditioning_layer): Linear(in_features=299, out_features=1280, bias=True)
|
| 117 |
+
)
|
| 118 |
+
(33-35): 3 x Wav2Vec2EncoderLayerStableLayerNorm(
|
| 119 |
+
(attention): Wav2Vec2Attention(
|
| 120 |
+
(k_proj): Linear(in_features=1280, out_features=1280, bias=True)
|
| 121 |
+
(v_proj): Linear(in_features=1280, out_features=1280, bias=True)
|
| 122 |
+
(q_proj): Linear(in_features=1280, out_features=1280, bias=True)
|
| 123 |
+
(out_proj): Linear(in_features=1280, out_features=1280, bias=True)
|
| 124 |
+
)
|
| 125 |
+
(dropout): Dropout(p=0.1, inplace=False)
|
| 126 |
+
(layer_norm): LayerNorm((1280,), eps=1e-05, elementwise_affine=True)
|
| 127 |
+
(feed_forward): Wav2Vec2FeedForward(
|
| 128 |
+
(intermediate_dropout): Dropout(p=0.0, inplace=False)
|
| 129 |
+
(intermediate_dense): Linear(in_features=1280, out_features=5120, bias=True)
|
| 130 |
+
(intermediate_act_fn): GELUActivation()
|
| 131 |
+
(output_dense): Linear(in_features=5120, out_features=1280, bias=True)
|
| 132 |
+
(output_dropout): Dropout(p=0.1, inplace=False)
|
| 133 |
+
)
|
| 134 |
+
(final_layer_norm): LayerNorm((1280,), eps=1e-05, elementwise_affine=True)
|
| 135 |
+
)
|
| 136 |
+
(36): Lang2VecConditionedLayer(
|
| 137 |
+
(layer): Wav2Vec2EncoderLayerStableLayerNorm(
|
| 138 |
+
(attention): Wav2Vec2Attention(
|
| 139 |
+
(k_proj): Linear(in_features=1280, out_features=1280, bias=True)
|
| 140 |
+
(v_proj): Linear(in_features=1280, out_features=1280, bias=True)
|
| 141 |
+
(q_proj): Linear(in_features=1280, out_features=1280, bias=True)
|
| 142 |
+
(out_proj): Linear(in_features=1280, out_features=1280, bias=True)
|
| 143 |
+
)
|
| 144 |
+
(dropout): Dropout(p=0.1, inplace=False)
|
| 145 |
+
(layer_norm): LayerNorm((1280,), eps=1e-05, elementwise_affine=True)
|
| 146 |
+
(feed_forward): Wav2Vec2FeedForward(
|
| 147 |
+
(intermediate_dropout): Dropout(p=0.0, inplace=False)
|
| 148 |
+
(intermediate_dense): Linear(in_features=1280, out_features=5120, bias=True)
|
| 149 |
+
(intermediate_act_fn): GELUActivation()
|
| 150 |
+
(output_dense): Linear(in_features=5120, out_features=1280, bias=True)
|
| 151 |
+
(output_dropout): Dropout(p=0.1, inplace=False)
|
| 152 |
+
)
|
| 153 |
+
(final_layer_norm): LayerNorm((1280,), eps=1e-05, elementwise_affine=True)
|
| 154 |
+
)
|
| 155 |
+
(encoder): IdentityEncoder()
|
| 156 |
+
(pooling): ChnAttnStatPooling(
|
| 157 |
+
(attention): Sequential(
|
| 158 |
+
(0): Conv1d(3840, 128, kernel_size=(1,), stride=(1,))
|
| 159 |
+
(1): ReLU()
|
| 160 |
+
(2): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
|
| 161 |
+
(3): Conv1d(128, 1280, kernel_size=(1,), stride=(1,))
|
| 162 |
+
)
|
| 163 |
+
(softmax): Softmax(dim=2)
|
| 164 |
+
)
|
| 165 |
+
(projector): RawNet3Projector(
|
| 166 |
+
(bn): BatchNorm1d(2560, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
|
| 167 |
+
(fc): Linear(in_features=2560, out_features=192, bias=True)
|
| 168 |
+
)
|
| 169 |
+
(prediction): Linear(in_features=192, out_features=299, bias=True)
|
| 170 |
+
(conditioning_layer): Linear(in_features=299, out_features=1280, bias=True)
|
| 171 |
+
)
|
| 172 |
+
(37-39): 3 x Wav2Vec2EncoderLayerStableLayerNorm(
|
| 173 |
+
(attention): Wav2Vec2Attention(
|
| 174 |
+
(k_proj): Linear(in_features=1280, out_features=1280, bias=True)
|
| 175 |
+
(v_proj): Linear(in_features=1280, out_features=1280, bias=True)
|
| 176 |
+
(q_proj): Linear(in_features=1280, out_features=1280, bias=True)
|
| 177 |
+
(out_proj): Linear(in_features=1280, out_features=1280, bias=True)
|
| 178 |
+
)
|
| 179 |
+
(dropout): Dropout(p=0.1, inplace=False)
|
| 180 |
+
(layer_norm): LayerNorm((1280,), eps=1e-05, elementwise_affine=True)
|
| 181 |
+
(feed_forward): Wav2Vec2FeedForward(
|
| 182 |
+
(intermediate_dropout): Dropout(p=0.0, inplace=False)
|
| 183 |
+
(intermediate_dense): Linear(in_features=1280, out_features=5120, bias=True)
|
| 184 |
+
(intermediate_act_fn): GELUActivation()
|
| 185 |
+
(output_dense): Linear(in_features=5120, out_features=1280, bias=True)
|
| 186 |
+
(output_dropout): Dropout(p=0.1, inplace=False)
|
| 187 |
+
)
|
| 188 |
+
(final_layer_norm): LayerNorm((1280,), eps=1e-05, elementwise_affine=True)
|
| 189 |
+
)
|
| 190 |
+
(40): Lang2VecConditionedLayer(
|
| 191 |
+
(layer): Wav2Vec2EncoderLayerStableLayerNorm(
|
| 192 |
+
(attention): Wav2Vec2Attention(
|
| 193 |
+
(k_proj): Linear(in_features=1280, out_features=1280, bias=True)
|
| 194 |
+
(v_proj): Linear(in_features=1280, out_features=1280, bias=True)
|
| 195 |
+
(q_proj): Linear(in_features=1280, out_features=1280, bias=True)
|
| 196 |
+
(out_proj): Linear(in_features=1280, out_features=1280, bias=True)
|
| 197 |
+
)
|
| 198 |
+
(dropout): Dropout(p=0.1, inplace=False)
|
| 199 |
+
(layer_norm): LayerNorm((1280,), eps=1e-05, elementwise_affine=True)
|
| 200 |
+
(feed_forward): Wav2Vec2FeedForward(
|
| 201 |
+
(intermediate_dropout): Dropout(p=0.0, inplace=False)
|
| 202 |
+
(intermediate_dense): Linear(in_features=1280, out_features=5120, bias=True)
|
| 203 |
+
(intermediate_act_fn): GELUActivation()
|
| 204 |
+
(output_dense): Linear(in_features=5120, out_features=1280, bias=True)
|
| 205 |
+
(output_dropout): Dropout(p=0.1, inplace=False)
|
| 206 |
+
)
|
| 207 |
+
(final_layer_norm): LayerNorm((1280,), eps=1e-05, elementwise_affine=True)
|
| 208 |
+
)
|
| 209 |
+
(encoder): IdentityEncoder()
|
| 210 |
+
(pooling): ChnAttnStatPooling(
|
| 211 |
+
(attention): Sequential(
|
| 212 |
+
(0): Conv1d(3840, 128, kernel_size=(1,), stride=(1,))
|
| 213 |
+
(1): ReLU()
|
| 214 |
+
(2): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
|
| 215 |
+
(3): Conv1d(128, 1280, kernel_size=(1,), stride=(1,))
|
| 216 |
+
)
|
| 217 |
+
(softmax): Softmax(dim=2)
|
| 218 |
+
)
|
| 219 |
+
(projector): RawNet3Projector(
|
| 220 |
+
(bn): BatchNorm1d(2560, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
|
| 221 |
+
(fc): Linear(in_features=2560, out_features=192, bias=True)
|
| 222 |
+
)
|
| 223 |
+
(prediction): Linear(in_features=192, out_features=299, bias=True)
|
| 224 |
+
(conditioning_layer): Linear(in_features=299, out_features=1280, bias=True)
|
| 225 |
+
)
|
| 226 |
+
(41-43): 3 x Wav2Vec2EncoderLayerStableLayerNorm(
|
| 227 |
+
(attention): Wav2Vec2Attention(
|
| 228 |
+
(k_proj): Linear(in_features=1280, out_features=1280, bias=True)
|
| 229 |
+
(v_proj): Linear(in_features=1280, out_features=1280, bias=True)
|
| 230 |
+
(q_proj): Linear(in_features=1280, out_features=1280, bias=True)
|
| 231 |
+
(out_proj): Linear(in_features=1280, out_features=1280, bias=True)
|
| 232 |
+
)
|
| 233 |
+
(dropout): Dropout(p=0.1, inplace=False)
|
| 234 |
+
(layer_norm): LayerNorm((1280,), eps=1e-05, elementwise_affine=True)
|
| 235 |
+
(feed_forward): Wav2Vec2FeedForward(
|
| 236 |
+
(intermediate_dropout): Dropout(p=0.0, inplace=False)
|
| 237 |
+
(intermediate_dense): Linear(in_features=1280, out_features=5120, bias=True)
|
| 238 |
+
(intermediate_act_fn): GELUActivation()
|
| 239 |
+
(output_dense): Linear(in_features=5120, out_features=1280, bias=True)
|
| 240 |
+
(output_dropout): Dropout(p=0.1, inplace=False)
|
| 241 |
+
)
|
| 242 |
+
(final_layer_norm): LayerNorm((1280,), eps=1e-05, elementwise_affine=True)
|
| 243 |
+
)
|
| 244 |
+
(44): Lang2VecConditionedLayer(
|
| 245 |
+
(layer): Wav2Vec2EncoderLayerStableLayerNorm(
|
| 246 |
+
(attention): Wav2Vec2Attention(
|
| 247 |
+
(k_proj): Linear(in_features=1280, out_features=1280, bias=True)
|
| 248 |
+
(v_proj): Linear(in_features=1280, out_features=1280, bias=True)
|
| 249 |
+
(q_proj): Linear(in_features=1280, out_features=1280, bias=True)
|
| 250 |
+
(out_proj): Linear(in_features=1280, out_features=1280, bias=True)
|
| 251 |
+
)
|
| 252 |
+
(dropout): Dropout(p=0.1, inplace=False)
|
| 253 |
+
(layer_norm): LayerNorm((1280,), eps=1e-05, elementwise_affine=True)
|
| 254 |
+
(feed_forward): Wav2Vec2FeedForward(
|
| 255 |
+
(intermediate_dropout): Dropout(p=0.0, inplace=False)
|
| 256 |
+
(intermediate_dense): Linear(in_features=1280, out_features=5120, bias=True)
|
| 257 |
+
(intermediate_act_fn): GELUActivation()
|
| 258 |
+
(output_dense): Linear(in_features=5120, out_features=1280, bias=True)
|
| 259 |
+
(output_dropout): Dropout(p=0.1, inplace=False)
|
| 260 |
+
)
|
| 261 |
+
(final_layer_norm): LayerNorm((1280,), eps=1e-05, elementwise_affine=True)
|
| 262 |
+
)
|
| 263 |
+
(encoder): IdentityEncoder()
|
| 264 |
+
(pooling): ChnAttnStatPooling(
|
| 265 |
+
(attention): Sequential(
|
| 266 |
+
(0): Conv1d(3840, 128, kernel_size=(1,), stride=(1,))
|
| 267 |
+
(1): ReLU()
|
| 268 |
+
(2): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
|
| 269 |
+
(3): Conv1d(128, 1280, kernel_size=(1,), stride=(1,))
|
| 270 |
+
)
|
| 271 |
+
(softmax): Softmax(dim=2)
|
| 272 |
+
)
|
| 273 |
+
(projector): RawNet3Projector(
|
| 274 |
+
(bn): BatchNorm1d(2560, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
|
| 275 |
+
(fc): Linear(in_features=2560, out_features=192, bias=True)
|
| 276 |
+
)
|
| 277 |
+
(prediction): Linear(in_features=192, out_features=299, bias=True)
|
| 278 |
+
(conditioning_layer): Linear(in_features=299, out_features=1280, bias=True)
|
| 279 |
+
)
|
| 280 |
+
(45-47): 3 x Wav2Vec2EncoderLayerStableLayerNorm(
|
| 281 |
+
(attention): Wav2Vec2Attention(
|
| 282 |
+
(k_proj): Linear(in_features=1280, out_features=1280, bias=True)
|
| 283 |
+
(v_proj): Linear(in_features=1280, out_features=1280, bias=True)
|
| 284 |
+
(q_proj): Linear(in_features=1280, out_features=1280, bias=True)
|
| 285 |
+
(out_proj): Linear(in_features=1280, out_features=1280, bias=True)
|
| 286 |
+
)
|
| 287 |
+
(dropout): Dropout(p=0.1, inplace=False)
|
| 288 |
+
(layer_norm): LayerNorm((1280,), eps=1e-05, elementwise_affine=True)
|
| 289 |
+
(feed_forward): Wav2Vec2FeedForward(
|
| 290 |
+
(intermediate_dropout): Dropout(p=0.0, inplace=False)
|
| 291 |
+
(intermediate_dense): Linear(in_features=1280, out_features=5120, bias=True)
|
| 292 |
+
(intermediate_act_fn): GELUActivation()
|
| 293 |
+
(output_dense): Linear(in_features=5120, out_features=1280, bias=True)
|
| 294 |
+
(output_dropout): Dropout(p=0.1, inplace=False)
|
| 295 |
+
)
|
| 296 |
+
(final_layer_norm): LayerNorm((1280,), eps=1e-05, elementwise_affine=True)
|
| 297 |
+
)
|
| 298 |
+
)
|
| 299 |
+
)
|
| 300 |
+
)
|
| 301 |
+
)
|
| 302 |
+
)
|
| 303 |
+
(featurizer): Featurizer()
|
| 304 |
+
)
|
| 305 |
+
(normalize): UtteranceMVN(norm_means=True, norm_vars=False)
|
| 306 |
+
(encoder): EcapaTdnnEncoder(
|
| 307 |
+
(conv): Conv1d(1280, 512, kernel_size=(5,), stride=(1,), padding=(2,))
|
| 308 |
+
(relu): ReLU()
|
| 309 |
+
(bn): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
|
| 310 |
+
(layer1): EcapaBlock(
|
| 311 |
+
(conv1): Conv1d(512, 512, kernel_size=(1,), stride=(1,))
|
| 312 |
+
(bn1): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
|
| 313 |
+
(convs): ModuleList(
|
| 314 |
+
(0-6): 7 x Conv1d(64, 64, kernel_size=(3,), stride=(1,), padding=(2,), dilation=(2,))
|
| 315 |
+
)
|
| 316 |
+
(bns): ModuleList(
|
| 317 |
+
(0-6): 7 x BatchNorm1d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
|
| 318 |
+
)
|
| 319 |
+
(conv3): Conv1d(512, 512, kernel_size=(1,), stride=(1,))
|
| 320 |
+
(bn3): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
|
| 321 |
+
(relu): ReLU()
|
| 322 |
+
(se): SEModule(
|
| 323 |
+
(se): Sequential(
|
| 324 |
+
(0): AdaptiveAvgPool1d(output_size=1)
|
| 325 |
+
(1): Conv1d(512, 128, kernel_size=(1,), stride=(1,))
|
| 326 |
+
(2): ReLU()
|
| 327 |
+
(3): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
|
| 328 |
+
(4): Conv1d(128, 512, kernel_size=(1,), stride=(1,))
|
| 329 |
+
(5): Sigmoid()
|
| 330 |
+
)
|
| 331 |
+
)
|
| 332 |
+
)
|
| 333 |
+
(layer2): EcapaBlock(
|
| 334 |
+
(conv1): Conv1d(512, 512, kernel_size=(1,), stride=(1,))
|
| 335 |
+
(bn1): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
|
| 336 |
+
(convs): ModuleList(
|
| 337 |
+
(0-6): 7 x Conv1d(64, 64, kernel_size=(3,), stride=(1,), padding=(3,), dilation=(3,))
|
| 338 |
+
)
|
| 339 |
+
(bns): ModuleList(
|
| 340 |
+
(0-6): 7 x BatchNorm1d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
|
| 341 |
+
)
|
| 342 |
+
(conv3): Conv1d(512, 512, kernel_size=(1,), stride=(1,))
|
| 343 |
+
(bn3): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
|
| 344 |
+
(relu): ReLU()
|
| 345 |
+
(se): SEModule(
|
| 346 |
+
(se): Sequential(
|
| 347 |
+
(0): AdaptiveAvgPool1d(output_size=1)
|
| 348 |
+
(1): Conv1d(512, 128, kernel_size=(1,), stride=(1,))
|
| 349 |
+
(2): ReLU()
|
| 350 |
+
(3): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
|
| 351 |
+
(4): Conv1d(128, 512, kernel_size=(1,), stride=(1,))
|
| 352 |
+
(5): Sigmoid()
|
| 353 |
+
)
|
| 354 |
+
)
|
| 355 |
+
)
|
| 356 |
+
(layer3): EcapaBlock(
|
| 357 |
+
(conv1): Conv1d(512, 512, kernel_size=(1,), stride=(1,))
|
| 358 |
+
(bn1): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
|
| 359 |
+
(convs): ModuleList(
|
| 360 |
+
(0-6): 7 x Conv1d(64, 64, kernel_size=(3,), stride=(1,), padding=(4,), dilation=(4,))
|
| 361 |
+
)
|
| 362 |
+
(bns): ModuleList(
|
| 363 |
+
(0-6): 7 x BatchNorm1d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
|
| 364 |
+
)
|
| 365 |
+
(conv3): Conv1d(512, 512, kernel_size=(1,), stride=(1,))
|
| 366 |
+
(bn3): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
|
| 367 |
+
(relu): ReLU()
|
| 368 |
+
(se): SEModule(
|
| 369 |
+
(se): Sequential(
|
| 370 |
+
(0): AdaptiveAvgPool1d(output_size=1)
|
| 371 |
+
(1): Conv1d(512, 128, kernel_size=(1,), stride=(1,))
|
| 372 |
+
(2): ReLU()
|
| 373 |
+
(3): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
|
| 374 |
+
(4): Conv1d(128, 512, kernel_size=(1,), stride=(1,))
|
| 375 |
+
(5): Sigmoid()
|
| 376 |
+
)
|
| 377 |
+
)
|
| 378 |
+
)
|
| 379 |
+
(extra_layers): ModuleList()
|
| 380 |
+
(layer4): Conv1d(1536, 1536, kernel_size=(1,), stride=(1,))
|
| 381 |
+
(mp3): MaxPool1d(kernel_size=3, stride=3, padding=0, dilation=1, ceil_mode=False)
|
| 382 |
+
)
|
| 383 |
+
(pooling): ChnAttnStatPooling(
|
| 384 |
+
(attention): Sequential(
|
| 385 |
+
(0): Conv1d(4608, 128, kernel_size=(1,), stride=(1,))
|
| 386 |
+
(1): ReLU()
|
| 387 |
+
(2): BatchNorm1d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
|
| 388 |
+
(3): Conv1d(128, 1536, kernel_size=(1,), stride=(1,))
|
| 389 |
+
)
|
| 390 |
+
(softmax): Softmax(dim=2)
|
| 391 |
+
)
|
| 392 |
+
(projector): RawNet3Projector(
|
| 393 |
+
(bn): BatchNorm1d(3072, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
|
| 394 |
+
(fc): Linear(in_features=3072, out_features=192, bias=True)
|
| 395 |
+
)
|
| 396 |
+
(loss): ArcMarginProduct_intertopk_subcenter(
|
| 397 |
+
(ce): CrossEntropyLoss()
|
| 398 |
+
)
|
| 399 |
+
(lang2vec_head): Linear(in_features=192, out_features=299, bias=True)
|
| 400 |
+
(lang2vec_loss): MSELoss()
|
| 401 |
+
)
|
| 402 |
+
|
| 403 |
+
Model summary:
|
| 404 |
+
Class Name: ESPnetLIDModel
|
| 405 |
+
Total Number of model parameters: 977.14 M
|
| 406 |
+
Number of trainable parameters: 977.14 M (100.0%)
|
| 407 |
+
Size: 3.91 GB
|
| 408 |
+
Type: torch.float32
|
| 409 |
+
/home/qingzhengw/espnet/.geolid-reproduction/venv/lib/python3.12/site-packages/torch/utils/data/dataloader.py:424: UserWarning: This DataLoader will create 8 worker processes in total. Our suggested max number of worker in current system is 2, which is smaller than what this DataLoader is going to create. Please be aware that excessive worker creation might get DataLoader running slow or even freeze, lower the worker number to avoid potential slowness/freeze if necessary.
|
| 410 |
+
self.check_worker_number_rationality()
|
| 411 |
+
/home/qingzhengw/espnet/.geolid-reproduction/review-code/espnet2/train/reporter.py:320: UserWarning: The stats of the previous epoch=-1doesn't exist.
|
| 412 |
+
warnings.warn(
|
| 413 |
+
[host-10-0-93-146] 2026-10-23 00:53:43,304 (lid_trainer:121) INFO: [Rank 0] Resume: 0 utterances found in /home/qingzhengw/espnet/.geolid-reproduction/full-dev-unified-301372e4/models/combined_shared_trainable/lids0
|
| 414 |
+
/home/qingzhengw/espnet/.geolid-reproduction/venv/lib/python3.12/site-packages/torch/utils/data/dataloader.py:432: UserWarning: This DataLoader will create 8 worker processes in total. Our suggested max number of worker in current system is 2, which is smaller than what this DataLoader is going to create. Please be aware that excessive worker creation might get DataLoader running slow or even freeze, lower the worker number to avoid potential slowness/freeze if necessary.
|
| 415 |
+
self.check_worker_number_rationality()
|
| 416 |
+
[host-10-0-93-146] 2026-10-23 00:54:01,839 (lid_trainer:250) INFO: [Rank 0] Saved 200 utts at step 0
|
| 417 |
+
[host-10-0-93-146] 2026-10-23 00:54:18,771 (lid_trainer:250) INFO: [Rank 0] Saved 200 utts at step 1
|
| 418 |
+
[host-10-0-93-146] 2026-10-23 00:54:34,599 (lid_trainer:250) INFO: [Rank 0] Saved 200 utts at step 2
|
| 419 |
+
[host-10-0-93-146] 2026-10-23 00:54:50,150 (lid_trainer:250) INFO: [Rank 0] Saved 200 utts at step 3
|
| 420 |
+
[host-10-0-93-146] 2026-10-23 00:55:04,729 (lid_trainer:250) INFO: [Rank 0] Saved 200 utts at step 4
|
| 421 |
+
[host-10-0-93-146] 2026-10-23 00:55:19,633 (lid_trainer:250) INFO: [Rank 0] Saved 200 utts at step 5
|
| 422 |
+
[host-10-0-93-146] 2026-10-23 00:55:34,813 (lid_trainer:250) INFO: [Rank 0] Saved 200 utts at step 6
|
| 423 |
+
[host-10-0-93-146] 2026-10-23 00:55:49,924 (lid_trainer:250) INFO: [Rank 0] Saved 200 utts at step 7
|
| 424 |
+
[host-10-0-93-146] 2026-10-23 00:55:50,841 (lid_inference:274) INFO: args.save_embd_per_utt: False
|
| 425 |
+
[host-10-0-93-146] 2026-10-23 00:55:50,842 (lid_inference:305) INFO: args.save_tsne_plot: False
|
evaluation_prediction_differences.json
ADDED
|
@@ -0,0 +1 @@
|
|
|
|
|
|
|
| 1 |
+
[]
|
evaluation_predictions.jsonl
ADDED
|
The diff for this file is too large to render.
See raw diff
|
|
|
evaluation_previous.json
ADDED
|
@@ -0,0 +1,38 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"dataset": "VoxLingua107",
|
| 3 |
+
"split": "dev",
|
| 4 |
+
"utterances": 1609,
|
| 5 |
+
"languages": 33,
|
| 6 |
+
"metric": "utterance-level accuracy (%)",
|
| 7 |
+
"entrypoint": "espnet2.bin.lid_inference",
|
| 8 |
+
"implementation_revision": "ca777d59eb49519e3827e40cc34997f2123d6682",
|
| 9 |
+
"device": "cuda",
|
| 10 |
+
"dtype": "float32",
|
| 11 |
+
"batch_size": 1,
|
| 12 |
+
"num_workers": 8,
|
| 13 |
+
"use_preprocessor": false,
|
| 14 |
+
"fix_duration": false,
|
| 15 |
+
"input_order": "wav.scp sorted by utterance ID; standard ESPnet streaming iterator",
|
| 16 |
+
"manifest_sha256": "6ad3c8fc6c74fab2720fe186525fc3bd20c4ab7b1be35d227c5c7f5456ba6c54",
|
| 17 |
+
"environment": {
|
| 18 |
+
"python": "3.12.3 (main, Jun 19 2026, 12:46:00) [GCC 13.3.0]",
|
| 19 |
+
"torch": "2.11.0+cu128",
|
| 20 |
+
"torch_cuda_build": "12.8",
|
| 21 |
+
"torchaudio": "2.11.0+cu128",
|
| 22 |
+
"transformers": "5.14.1",
|
| 23 |
+
"espnet_s3prl": "2026.5.14",
|
| 24 |
+
"soundfile": "0.14.0"
|
| 25 |
+
},
|
| 26 |
+
"repo_id": "espnet/geolid_combined_shared_trainable",
|
| 27 |
+
"model_sha256": "8a8f9dd767c141dcae0cb34a7780d7f593bc67f560267cd4d2d2fadd54f6bd16",
|
| 28 |
+
"correct": 1501,
|
| 29 |
+
"accuracy_percent": 93.2877563704164,
|
| 30 |
+
"paper_accuracy_percent": 94.4,
|
| 31 |
+
"source_hf_revision_before_layout_migration": "0c7ee08dc9ab7d9ac651c7858fd6ce40849475b6",
|
| 32 |
+
"batch_dependence": {
|
| 33 |
+
"s3prl_frame_alignment": "S3PRL reports ceil(samples/320) frames and repeats the final frame only at the batch boundary. Shorter items can instead expose padded-query frames from Wav2Vec2.",
|
| 34 |
+
"ecapa_tdnn": "Temporal convolutions and unmasked SE global means depend on the padded sequence length before the masked final pooling.",
|
| 35 |
+
"mvn_and_final_pooling_mask_padding": true,
|
| 36 |
+
"reported_measurement_batch_size": 1
|
| 37 |
+
}
|
| 38 |
+
}
|
evaluation_report.md
ADDED
|
@@ -0,0 +1,61 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Full VoxLingua107 dev evaluation after model integration
|
| 2 |
+
|
| 3 |
+
`espnet/geolid_combined_shared_trainable` classified **1501 / 1,609 utterances
|
| 4 |
+
correctly (93.29%)** across 33 languages.
|
| 5 |
+
This is a complete new inference run using `ESPnetLIDModel` at
|
| 6 |
+
`301372e4da311d2a3d9574a5a43b3037ae6e67c1`, completed at `2026-10-23T00:55:52.840208+00:00`.
|
| 7 |
+
|
| 8 |
+
All 1,609 predicted labels match the earlier full-dev evaluation at `ca777d59eb49519e3827e40cc34997f2123d6682`.
|
| 9 |
+
|
| 10 |
+
The measured checkpoint SHA-256 is `8a8f9dd767c141dcae0cb34a7780d7f593bc67f560267cd4d2d2fadd54f6bd16`.
|
| 11 |
+
The model inputs came from this immutable
|
| 12 |
+
[HF revision](https://huggingface.co/espnet/geolid_combined_shared_trainable/tree/613d18645d1be3bd585a2cbae0485350a2a4c027).
|
| 13 |
+
The checkpoint and published model configuration have not been modified by
|
| 14 |
+
this evaluation. Only the local path to `languages.txt` was resolved for the
|
| 15 |
+
run; its published `./languages.txt` value works from the model directory.
|
| 16 |
+
|
| 17 |
+
## Reproduce inference
|
| 18 |
+
|
| 19 |
+
Install the ESPnet revision documented in the model card, and download this
|
| 20 |
+
model's `config.yaml`, `model.pth` and `languages.txt`. Run the following command
|
| 21 |
+
from that model directory. Prepare `data/voxlingua107-dev/wav.scp` as sorted
|
| 22 |
+
`<utterance-id> <absolute-waveform-path>` lines for all 1,609 dev utterances;
|
| 23 |
+
use the original full-length, mono, 16 kHz waveforms. Choose a fresh output
|
| 24 |
+
directory because the CLI can skip inference when a completion marker exists.
|
| 25 |
+
|
| 26 |
+
```bash
|
| 27 |
+
OMP_NUM_THREADS=4 MKL_NUM_THREADS=4 python -m espnet2.bin.lid_inference \
|
| 28 |
+
--output_dir evaluation_output \
|
| 29 |
+
--ngpu 1 --dtype float32 \
|
| 30 |
+
--data_path_and_name_and_type data/voxlingua107-dev/wav.scp,speech,sound \
|
| 31 |
+
--valid_batch_size 1 --num_workers 8 \
|
| 32 |
+
--lid_train_config config.yaml --lid_model_file model.pth \
|
| 33 |
+
--use_preprocessor false --fix_duration false \
|
| 34 |
+
--extract_embd false --save_embd_per_utt false \
|
| 35 |
+
--save_embd_avg_lang false --save_tsne_plot false \
|
| 36 |
+
--checkpoint_interval 200
|
| 37 |
+
```
|
| 38 |
+
|
| 39 |
+
The output `evaluation_output/voxlingua107-dev_lids` contains the predicted
|
| 40 |
+
labels. Compare each utterance ID with the reference language in
|
| 41 |
+
[evaluation_predictions.jsonl](evaluation_predictions.jsonl). The reported
|
| 42 |
+
accuracy is `100 * correct / 1609`; no duration weighting or macro averaging
|
| 43 |
+
is used. The JSONL also records the predictions from this run and a correctness
|
| 44 |
+
flag for every utterance, so the score can be independently recomputed.
|
| 45 |
+
|
| 46 |
+
## Evidence and scope
|
| 47 |
+
|
| 48 |
+
- [Evaluation metadata](evaluation.json): counts, settings, per-language counts,
|
| 49 |
+
environment, source revision and hashes.
|
| 50 |
+
- [All predictions and reference labels](evaluation_predictions.jsonl).
|
| 51 |
+
- [Complete captured inference log](evaluation.log): paths refer to the actual
|
| 52 |
+
evaluation workspace; its output is retained unchanged.
|
| 53 |
+
- [Prediction differences from the previous run](evaluation_prediction_differences.json).
|
| 54 |
+
- [Previous evaluation metadata](evaluation_previous.json), preserved unchanged.
|
| 55 |
+
- [Validation history](validation.json): earlier strict-loading checks plus a
|
| 56 |
+
separate record of this full evaluation through the unmodified CLI.
|
| 57 |
+
|
| 58 |
+
This run evaluates VoxLingua107 dev only. Historical paper results for other
|
| 59 |
+
test sets have not been remeasured here. The new full run uses the standard
|
| 60 |
+
loader without forcing strict loading; the earlier explicit strict-loading
|
| 61 |
+
check is a separate historical validation.
|
validation.json
CHANGED
|
@@ -35,5 +35,32 @@
|
|
| 35 |
"ell_2KtniiXOx90__U__S216---2054.480-2059.730.wav": "ell"
|
| 36 |
},
|
| 37 |
"loading_check": "Strict loading required in the CLI checkpoint loader; inference computation unchanged"
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 38 |
}
|
| 39 |
}
|
|
|
|
| 35 |
"ell_2KtniiXOx90__U__S216---2054.480-2059.730.wav": "ell"
|
| 36 |
},
|
| 37 |
"loading_check": "Strict loading required in the CLI checkpoint loader; inference computation unchanged"
|
| 38 |
+
},
|
| 39 |
+
"full_dev_model_integration": {
|
| 40 |
+
"implementation_revision": "301372e4da311d2a3d9574a5a43b3037ae6e67c1",
|
| 41 |
+
"model": "ESPnetLIDModel",
|
| 42 |
+
"model_sha256": "8a8f9dd767c141dcae0cb34a7780d7f593bc67f560267cd4d2d2fadd54f6bd16",
|
| 43 |
+
"source_hf_revision": "613d18645d1be3bd585a2cbae0485350a2a4c027",
|
| 44 |
+
"entrypoint": "espnet2.bin.lid_inference",
|
| 45 |
+
"loader": "Unmodified standard ESPnet CLI loader",
|
| 46 |
+
"strict_loading_forced": false,
|
| 47 |
+
"weight_file_changed": false,
|
| 48 |
+
"full_dev_evaluation": true,
|
| 49 |
+
"utterances": 1609,
|
| 50 |
+
"correct": 1501,
|
| 51 |
+
"accuracy_percent": 93.2877563704164,
|
| 52 |
+
"batch_size": 1,
|
| 53 |
+
"num_workers": 8,
|
| 54 |
+
"dtype": "float32",
|
| 55 |
+
"finished_at": "2026-10-23T00:55:52.840208+00:00",
|
| 56 |
+
"predictions_equal_previous": true,
|
| 57 |
+
"matched_predictions": 1609,
|
| 58 |
+
"changed_predictions": 0,
|
| 59 |
+
"baseline_implementation_revision": "ca777d59eb49519e3827e40cc34997f2123d6682",
|
| 60 |
+
"evaluation_file": "evaluation.json",
|
| 61 |
+
"predictions_file": "evaluation_predictions.jsonl",
|
| 62 |
+
"predictions_sha256": "40264d2766f60dfce717ebc638ef76bc85380709693ca3e32178829e5df5453e",
|
| 63 |
+
"log_file": "evaluation.log",
|
| 64 |
+
"log_sha256": "d6831bc2d1e154c487052b31b10adc9072db4c8d045e4a1c6897125aa7a4c6c1"
|
| 65 |
}
|
| 66 |
}
|