Ship the scored epoch-60 ASR checkpoint and the character LM

The previous upload held valid.loss.ave_10best.pth, which was never scored; every reported number came from the epoch-60 checkpoint (latest.pth), now shipped as 60epoch.pth. Adds the character LSTM LM used for the lm_weight 0.3 results, sets the usage example to the reported decoding settings, and replaces the dataset citation with the corpus paper (Abraham et al., LREC 2020).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

README.md

@@ -15,7 +15,14 @@ license: cc-by-4.0
 import librosa
 from espnet2.bin.asr_inference import Speech2Text
 
-speech2text = Speech2Text.from_pretrained(model_tag="espnet/marathi_lrec2020")
+# from_pretrained also loads the bundled character LM; these are the settings
+# behind the reported 27.0 % WER (lm_weight=0.0 gives the 29.4 % no-LM result)
+speech2text = Speech2Text.from_pretrained(
+    model_tag="espnet/marathi_lrec2020",
+    beam_size=20,
+    ctc_weight=0.5,
+    lm_weight=0.3,
+)
 # librosa resamples and mixes to one channel, so any file works; 16000 is
 # what nearly every espnet recogniser is trained on - check this model's
 # config if its audio is not 16 kHz
@@ -58,14 +65,22 @@ cd egs2/marathi_lrec2020/asr1
 
 ## Marathi ASR — `marathi_lrec2020`
 
-Recipe for **Marathi** ASR on the
-[**IndicCorpora Marathi subset**](https://www.cse.iitb.ac.in/~pjyothi/indiccorpora/#marathi).
+Recipe for **Marathi** ASR on the 109-hour crowdsourced
+[**Marathi speech corpus**](https://www.cse.iitb.ac.in/~pjyothi/indiccorpora/#marathi) (Abraham et al., LREC 2020).
 
-Training uses **`conf/train_asr_transformer.yaml`** (character Conformer: 3 blocks, 256-dim encoder, `batch_bins: 16000000`, `accum_grad: 4`, Adam `lr: 0.0005`, warmup 20k, SpecAugment, hybrid CTC/attention `ctc_weight: 0.3`).
+**ASR model:** `exp/asr_marathi_conf_old_try_3conf_16Mbin_4grad/60epoch.pth`, the epoch-60 checkpoint (`latest.pth` in the recipe) that produced every number below.
+Training uses **`conf/train_asr_transformer.yaml`** (character Conformer: 3 blocks, 256-dim encoder, 3-block Transformer decoder, `batch_bins: 16000000`, `accum_grad: 4`, Adam `lr: 0.0005`, warmup 20k, SpecAugment, hybrid CTC/attention `ctc_weight: 0.3`) on speed-perturbed (0.9/1.0/1.1) training data.
+
+**Language model:** `exp/lm_marathi_char_lm/6epoch.pth`, a character-level LSTM LM (4 layers, 650 units, tied embeddings, dropout 0.3, 13.6 M parameters).
+It was trained with Adam `lr: 0.0005` on the transcripts of the speed-perturbed training set (220,939 lines, 2,307 unique sentences).
+Epoch 6 had the lowest dev loss (1.061 nats per character token, perplexity 2.89), and training stopped early at epoch 13.
 
 Decoding without LM: **`conf/decode_asr.yaml`** (`lm_weight: 0.0`).  
 Decoding with LM (match reported fusion): **`conf/decode_asr_lm.yaml`** (beam 20, `ctc_weight: 0.5`, `lm_weight: 0.3`).
 
+Note: the LM's token list orders `ौ` and `-` (indices 63 and 64) the other way round from the ASR model's.
+ESPnet fuses the two models by token index, so the LM results below include this mismatch; the two characters are 0.1 % of test-set characters.
+
 ---
 
 ### Test-set decoding (`marathi_test`)
@@ -86,12 +101,22 @@ Beam **20**, **CTC weight 0.5** unless noted.
 | **CER** | 89.0 | 6.6 | 4.4 | 1.7 | 12.6 | 74.3 |
 | **WER** | 76.0 | 21.6 | 2.4 | 3.0 | 27.0 | 75.0 |
 
+These are sclite's per-speaker Mean rows; the recipe's speaker ID is the prompt ID, so each is a mean over the 300 test prompts.
+Corpus-level (Sum/Avg) error rates are WER 27.3 (no LM) and 24.9 (LM), and CER 12.4 and 12.0.
+
 ---
 
 ### Dataset reference
 
-> P. Jyothi et al., *“IndicCorpora: A Large Multilingual Corpus for Indic Languages.”*  
-> [IIT Bombay IndicCorpora — Marathi](https://www.cse.iitb.ac.in/~pjyothi/indiccorpora/#marathi)
+```bibtex
+@inproceedings{abraham-etal-2020-crowdsourcing,
+  title     = {Crowdsourcing Speech Data for Low-Resource Languages from Low-Income Workers},
+  author    = {Abraham, Basil and Goel, Danish and Siddarth, Divya and Bali, Kalika and Chopra, Manu and Choudhury, Monojit and Joshi, Pratik and Jyothi, Preethi and Sitaram, Sunayana and Seshadri, Vivek},
+  booktitle = {Proceedings of the 12th Conference on Language Resources and Evaluation (LREC)},
+  pages     = {2819--2826},
+  year      = {2020}
+}
+```
 
 ## ASR config
 
@@ -422,6 +447,256 @@ distributed: false
 
 </details>
 
+## LM config
+
+<details><summary>expand</summary>
+
+```
+config: conf/train_lm_char.yaml
+print_config: false
+log_level: INFO
+drop_last_iter: false
+dry_run: false
+iterator_type: sequence
+valid_iterator_type: null
+output_dir: exp/lm_marathi_char_lm
+ngpu: 1
+seed: 0
+num_workers: 1
+num_att_plot: 3
+dist_backend: nccl
+dist_init_method: env://
+dist_world_size: 2
+dist_rank: 0
+local_rank: 0
+dist_master_addr: localhost
+dist_master_port: 58689
+dist_launcher: null
+multiprocessing_distributed: true
+unused_parameters: false
+sharded_ddp: false
+use_deepspeed: false
+deepspeed_config: null
+gradient_as_bucket_view: true
+ddp_comm_hook: null
+cudnn_enabled: true
+cudnn_benchmark: false
+cudnn_deterministic: true
+use_tf32: false
+collect_stats: false
+write_collected_feats: false
+max_epoch: 30
+patience: 5
+val_scheduler_criterion:
+- valid
+- loss
+early_stopping_criterion:
+- valid
+- loss
+- min
+best_model_criterion:
+-   - valid
+    - loss
+    - min
+keep_nbest_models: 5
+nbest_averaging_interval: 0
+grad_clip: 5.0
+grad_clip_type: 2.0
+grad_noise: false
+accum_grad: 1
+no_forward_run: false
+resume: true
+train_dtype: float32
+use_amp: false
+log_interval: null
+use_matplotlib: true
+use_tensorboard: true
+create_graph_in_tensorboard: false
+use_wandb: false
+wandb_project: null
+wandb_id: null
+wandb_entity: null
+wandb_name: null
+wandb_model_log_interval: -1
+detect_anomaly: false
+use_adapter: false
+adapter: lora
+save_strategy: all
+adapter_conf: {}
+pretrain_path: null
+init_param: []
+ignore_init_mismatch: false
+freeze_param: []
+num_iters_per_epoch: null
+batch_size: 256
+valid_batch_size: null
+batch_bins: 1000000
+valid_batch_bins: null
+category_sample_size: 10
+upsampling_factor: 0.5
+category_upsampling_factor: 0.5
+dataset_upsampling_factor: 0.5
+dataset_scaling_factor: 1.2
+max_batch_size: null
+min_batch_size: 1
+train_shape_file:
+- exp/lm_stats_char/train/text_shape.char
+valid_shape_file:
+- exp/lm_stats_char/valid/text_shape.char
+batch_type: folded
+valid_batch_type: null
+fold_length:
+- 150
+sort_in_batch: descending
+shuffle_within_batch: false
+sort_batch: descending
+multiple_iterator: false
+chunk_length: 500
+chunk_shift_ratio: 0.5
+num_cache_chunks: 1024
+chunk_excluded_key_prefixes: []
+chunk_default_fs: null
+chunk_max_abs_length: null
+chunk_discard_short_samples: true
+train_data_path_and_name_and_type:
+-   - dump/raw/lm_train.txt
+    - text
+    - text
+valid_data_path_and_name_and_type:
+-   - dump/raw/org/marathi_dev_lm_ver/text
+    - text
+    - text
+multi_task_dataset: false
+allow_variable_data_keys: false
+max_cache_size: 0.0
+max_cache_fd: 32
+allow_multi_rates: false
+valid_max_cache_size: null
+exclude_weight_decay: false
+exclude_weight_decay_conf: {}
+optim: adam
+optim_conf:
+    lr: 0.0005
+    weight_decay: 1.0e-06
+scheduler: null
+scheduler_conf: {}
+token_list:
+- <blank>
+- <unk>
+- <space>
+- ा
+- े
+- र
+- ्
+- क
+- स
+- ल
+- म
+- ि
+- ं
+- ी
+- य
+- त
+- न
+- व
+- ग
+- ह
+- ट
+- च
+- प
+- ड
+- आ
+- .
+- ज
+- श
+- ो
+- द
+- ब
+- अ
+- ू
+- ु
+- '?'
+- ण
+- इ
+- ध
+- ए
+- फ
+- ख
+- ॉ
+- ॅ
+- ळ
+- ँ
+- भ
+- थ
+- ठ
+- ई
+- झ
+- ष
+- उ
+- ऑ
+- ऊ
+- घ
+- ढ
+- ै
+- ओ
+- '2'
+- (
+- )
+- '0'
+- ृ
+- '-'
+- ौ
+- '3'
+- '1'
+- ऐ
+- ‍
+- '5'
+- '8'
+- छ
+- '4'
+- '"'
+- ','
+- '9'
+- औ
+- '!'
+- ़
+- ञ
+- '7'
+- '6'
+- ९
+- ऋ
+- e
+- g
+- ३
+- X
+- १
+- ०
+- <sos/eos>
+init: null
+use_preprocessor: true
+token_type: char
+bpemodel: null
+non_linguistic_symbols: null
+cleaner: null
+g2p: null
+lm: seq_rnn
+lm_conf:
+    rnn_type: lstm
+    nlayers: 4
+    unit: 650
+    tie_weights: true
+    dropout_rate: 0.3
+model: lm
+model_conf: {}
+required:
+- output_dir
+- token_list
+version: '202509'
+distributed: true
+```
+
+</details>
+
 
 
 ### Citing ESPnet

exp/asr_marathi_conf_old_try_3conf_16Mbin_4grad/{valid.loss.ave_10best.pth → 60epoch.pth}

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:af3a004e0637c01688276834e8a57ec410810f6dacadef3e3101fd767d8dbf1f
-size 33154802
+oid sha256:2f84f338d92332d8eca50675faf18f078a98bc6acaad417bc6b39e17212d5bc5
+size 33154214

exp/asr_marathi_conf_old_try_3conf_16Mbin_4grad/RESULTS.md

@@ -13,14 +13,22 @@
 
 ## Marathi ASR — `marathi_lrec2020`
 
-Recipe for **Marathi** ASR on the
-[**IndicCorpora Marathi subset**](https://www.cse.iitb.ac.in/~pjyothi/indiccorpora/#marathi).
+Recipe for **Marathi** ASR on the 109-hour crowdsourced
+[**Marathi speech corpus**](https://www.cse.iitb.ac.in/~pjyothi/indiccorpora/#marathi) (Abraham et al., LREC 2020).
 
-Training uses **`conf/train_asr_transformer.yaml`** (character Conformer: 3 blocks, 256-dim encoder, `batch_bins: 16000000`, `accum_grad: 4`, Adam `lr: 0.0005`, warmup 20k, SpecAugment, hybrid CTC/attention `ctc_weight: 0.3`).
+**ASR model:** `exp/asr_marathi_conf_old_try_3conf_16Mbin_4grad/60epoch.pth`, the epoch-60 checkpoint (`latest.pth` in the recipe) that produced every number below.
+Training uses **`conf/train_asr_transformer.yaml`** (character Conformer: 3 blocks, 256-dim encoder, 3-block Transformer decoder, `batch_bins: 16000000`, `accum_grad: 4`, Adam `lr: 0.0005`, warmup 20k, SpecAugment, hybrid CTC/attention `ctc_weight: 0.3`) on speed-perturbed (0.9/1.0/1.1) training data.
+
+**Language model:** `exp/lm_marathi_char_lm/6epoch.pth`, a character-level LSTM LM (4 layers, 650 units, tied embeddings, dropout 0.3, 13.6 M parameters).
+It was trained with Adam `lr: 0.0005` on the transcripts of the speed-perturbed training set (220,939 lines, 2,307 unique sentences).
+Epoch 6 had the lowest dev loss (1.061 nats per character token, perplexity 2.89), and training stopped early at epoch 13.
 
 Decoding without LM: **`conf/decode_asr.yaml`** (`lm_weight: 0.0`).  
 Decoding with LM (match reported fusion): **`conf/decode_asr_lm.yaml`** (beam 20, `ctc_weight: 0.5`, `lm_weight: 0.3`).
 
+Note: the LM's token list orders `ौ` and `-` (indices 63 and 64) the other way round from the ASR model's.
+ESPnet fuses the two models by token index, so the LM results below include this mismatch; the two characters are 0.1 % of test-set characters.
+
 ---
 
 ### Test-set decoding (`marathi_test`)
@@ -41,9 +49,19 @@ Beam **20**, **CTC weight 0.5** unless noted.
 | **CER** | 89.0 | 6.6 | 4.4 | 1.7 | 12.6 | 74.3 |
 | **WER** | 76.0 | 21.6 | 2.4 | 3.0 | 27.0 | 75.0 |
 
+These are sclite's per-speaker Mean rows; the recipe's speaker ID is the prompt ID, so each is a mean over the 300 test prompts.
+Corpus-level (Sum/Avg) error rates are WER 27.3 (no LM) and 24.9 (LM), and CER 12.4 and 12.0.
+
 ---
 
 ### Dataset reference
 
-> P. Jyothi et al., *“IndicCorpora: A Large Multilingual Corpus for Indic Languages.”*  
-> [IIT Bombay IndicCorpora — Marathi](https://www.cse.iitb.ac.in/~pjyothi/indiccorpora/#marathi)
+```bibtex
+@inproceedings{abraham-etal-2020-crowdsourcing,
+  title     = {Crowdsourcing Speech Data for Low-Resource Languages from Low-Income Workers},
+  author    = {Abraham, Basil and Goel, Danish and Siddarth, Divya and Bali, Kalika and Chopra, Manu and Choudhury, Monojit and Joshi, Pratik and Jyothi, Preethi and Sitaram, Sunayana and Seshadri, Vivek},
+  booktitle = {Proceedings of the 12th Conference on Language Resources and Evaluation (LREC)},
+  pages     = {2819--2826},
+  year      = {2020}
+}
+```

exp/lm_marathi_char_lm/6epoch.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e606283953611611215b1e93759636c5764d10e01e63a02386c32cad3c78c60e
+size 54403341

exp/lm_marathi_char_lm/config.yaml

@@ -0,0 +1,241 @@
+config: conf/train_lm_char.yaml
+print_config: false
+log_level: INFO
+drop_last_iter: false
+dry_run: false
+iterator_type: sequence
+valid_iterator_type: null
+output_dir: exp/lm_marathi_char_lm
+ngpu: 1
+seed: 0
+num_workers: 1
+num_att_plot: 3
+dist_backend: nccl
+dist_init_method: env://
+dist_world_size: 2
+dist_rank: 0
+local_rank: 0
+dist_master_addr: localhost
+dist_master_port: 58689
+dist_launcher: null
+multiprocessing_distributed: true
+unused_parameters: false
+sharded_ddp: false
+use_deepspeed: false
+deepspeed_config: null
+gradient_as_bucket_view: true
+ddp_comm_hook: null
+cudnn_enabled: true
+cudnn_benchmark: false
+cudnn_deterministic: true
+use_tf32: false
+collect_stats: false
+write_collected_feats: false
+max_epoch: 30
+patience: 5
+val_scheduler_criterion:
+- valid
+- loss
+early_stopping_criterion:
+- valid
+- loss
+- min
+best_model_criterion:
+-   - valid
+    - loss
+    - min
+keep_nbest_models: 5
+nbest_averaging_interval: 0
+grad_clip: 5.0
+grad_clip_type: 2.0
+grad_noise: false
+accum_grad: 1
+no_forward_run: false
+resume: true
+train_dtype: float32
+use_amp: false
+log_interval: null
+use_matplotlib: true
+use_tensorboard: true
+create_graph_in_tensorboard: false
+use_wandb: false
+wandb_project: null
+wandb_id: null
+wandb_entity: null
+wandb_name: null
+wandb_model_log_interval: -1
+detect_anomaly: false
+use_adapter: false
+adapter: lora
+save_strategy: all
+adapter_conf: {}
+pretrain_path: null
+init_param: []
+ignore_init_mismatch: false
+freeze_param: []
+num_iters_per_epoch: null
+batch_size: 256
+valid_batch_size: null
+batch_bins: 1000000
+valid_batch_bins: null
+category_sample_size: 10
+upsampling_factor: 0.5
+category_upsampling_factor: 0.5
+dataset_upsampling_factor: 0.5
+dataset_scaling_factor: 1.2
+max_batch_size: null
+min_batch_size: 1
+train_shape_file:
+- exp/lm_stats_char/train/text_shape.char
+valid_shape_file:
+- exp/lm_stats_char/valid/text_shape.char
+batch_type: folded
+valid_batch_type: null
+fold_length:
+- 150
+sort_in_batch: descending
+shuffle_within_batch: false
+sort_batch: descending
+multiple_iterator: false
+chunk_length: 500
+chunk_shift_ratio: 0.5
+num_cache_chunks: 1024
+chunk_excluded_key_prefixes: []
+chunk_default_fs: null
+chunk_max_abs_length: null
+chunk_discard_short_samples: true
+train_data_path_and_name_and_type:
+-   - dump/raw/lm_train.txt
+    - text
+    - text
+valid_data_path_and_name_and_type:
+-   - dump/raw/org/marathi_dev_lm_ver/text
+    - text
+    - text
+multi_task_dataset: false
+allow_variable_data_keys: false
+max_cache_size: 0.0
+max_cache_fd: 32
+allow_multi_rates: false
+valid_max_cache_size: null
+exclude_weight_decay: false
+exclude_weight_decay_conf: {}
+optim: adam
+optim_conf:
+    lr: 0.0005
+    weight_decay: 1.0e-06
+scheduler: null
+scheduler_conf: {}
+token_list:
+- <blank>
+- <unk>
+- <space>
+- ा
+- े
+- र
+- ्
+- क
+- स
+- ल
+- म
+- ि
+- ं
+- ी
+- य
+- त
+- न
+- व
+- ग
+- ह
+- ट
+- च
+- प
+- ड
+- आ
+- .
+- ज
+- श
+- ो
+- द
+- ब
+- अ
+- ू
+- ु
+- '?'
+- ण
+- इ
+- ध
+- ए
+- फ
+- ख
+- ॉ
+- ॅ
+- ळ
+- ँ
+- भ
+- थ
+- ठ
+- ई
+- झ
+- ष
+- उ
+- ऑ
+- ऊ
+- घ
+- ढ
+- ै
+- ओ
+- '2'
+- (
+- )
+- '0'
+- ृ
+- '-'
+- ौ
+- '3'
+- '1'
+- ऐ
+- ‍
+- '5'
+- '8'
+- छ
+- '4'
+- '"'
+- ','
+- '9'
+- औ
+- '!'
+- ़
+- ञ
+- '7'
+- '6'
+- ९
+- ऋ
+- e
+- g
+- ३
+- X
+- १
+- ०
+- <sos/eos>
+init: null
+use_preprocessor: true
+token_type: char
+bpemodel: null
+non_linguistic_symbols: null
+cleaner: null
+g2p: null
+lm: seq_rnn
+lm_conf:
+    rnn_type: lstm
+    nlayers: 4
+    unit: 650
+    tie_weights: true
+    dropout_rate: 0.3
+model: lm
+model_conf: {}
+required:
+- output_dir
+- token_list
+version: '202509'
+distributed: true

meta.yaml

@@ -1,8 +1,10 @@
-espnet: '202511'
+espnet: '202604'
 files:
-  asr_model_file: exp/asr_marathi_conf_old_try_3conf_16Mbin_4grad/valid.loss.ave_10best.pth
+  asr_model_file: exp/asr_marathi_conf_old_try_3conf_16Mbin_4grad/60epoch.pth
+  lm_file: exp/lm_marathi_char_lm/6epoch.pth
 python: "3.9.23 | packaged by conda-forge | (main, Jun  4 2025, 17:57:12) \n[GCC 13.3.0]"
-timestamp: 1776596828.579581
+timestamp: 1792538684.854284
 torch: 2.3.0+cu121
 yaml_files:
   asr_train_config: exp/asr_marathi_conf_old_try_3conf_16Mbin_4grad/config.yaml
+  lm_train_config: exp/lm_marathi_char_lm/config.yaml