Update README.md

README.md

@@ -84,204 +84,78 @@ Finally, the auditory-lingual decoder adopts point-wise dynamic rescaled Gaussia
 <!-- Provide the basic links for the model. -->
 
 - **Repository:** https://github.com/Kimang18/KrorngAI/tree/main/tror-yong-asr
-- **Blog Post:** https://kimang18.github.io/krorngai-blog/TrorYongASR/
+- **Blog Post:** [https://kimang18.github.io/krorngai-blog/TrorYongASR2/](https://kimang18.github.io/krorngai-blog/TrorYongASR2/)
 - **Demo:** https://krorngai-troryongasr-demo.hf.space
 
 
-### Model Architecture
-
-#### Length-Aware Rotary Position Embedding
-
-Length-Aware Rotary Position Embedding (LARoPE), a modification to RoPE, calculates relative distances between query and key vectors using length-normalized indices instead of absolute positional indices [@kim2026length].
-This extension induces diagonal bias in the attention score maps to better align the query vector from one source to the key vector from another source.
-LARoPE helps improve text-speech alignment in Text-To-Speech models [@kim2026length].
-
-With permuted sequence modeling design, speech-text alignment in cross-attention layer is crucial to TrorYongASR.
-Leveraging LARoPE, TrorYongASR2 achieves significant performance improvement (see @fig-loss).
-In addition, to enhance the expressivity of LARoPE as well as RoPE, the lowest frequency portion of the rotation are replaced by zero frequency (_i.e.,_ no rotation).
-This is a hybrid architecture of NoPE and RoPE proposed by [@barbero2024round].
-It enables channels without rotation to better carry semantic information.
-
-LARoPE requires the knowledge of audio length and text sequence length.
-As the text sequence length is unknown during inference, a length predictor module is designed to predict the sequence length given the audio.
-
-
-##### Length Predictor
-
-To make TrorYongASR2 as lightweight as possible, length predictor infers the length of text sequence conditioned directly on the audio encoding.
-TrorYongASR2 limits the output text to 448 tokens.
-Therefore, length predictor's head is a linear layer with a softcap as the following:
-
-$$
-\mathrm{logits}\leftarrow \mathrm{\beta} \times tanh(\mathrm{logits} / \mathrm{\beta})
-$$
-where \\( \beta=224 \\).
-So, the output of length predictor stays between $-224$ and $224$.
-Since the target output space is $\{0, \dots, 448\}$, the length prediction is shifted by $+224$ and mean-squared-error loss can be used to train the length predictor.
-Note that no gradient from this length loss should flow to the audio encoder of TrorYongASR2.
-The only role of audio encoder is to capture meaningful features of input audio for the auditory-lingual decoder.
-
-#### Gated Attention Layers
-
-Auditory-lingual decoder of TrorYongASR is just a single Transformer.
-So, the expressivity of the attention mechanisms in this block plays a vital role in improving the model performance.
-TrorYongASR2 augments the attention layers of this block with an input-dependent head-specific output gate.
-This gate allows the input to modulate the scaled dot-product attention and thus mitigate the 'attention sink' [@qiu2026gated].
-
-Concretely, for hidden state $\bh$, attention layers have additional parameter $W^g$ such that the gate score for head $i$ is given by $\mathrm{Sigmoid}(W^g_i\bh)$.
-Let $\mathrm{head}_i=\mathrm{Attention}(\bh, W^q_i, W^k_i, W^v_i)$ be the scaled dot-product attention of head $i$.
-The gated output of multi-head attention is given by
-$$
-\bo=W^o\mathrm{Concat}\bigg[\mathrm{Sigmoid}(W^g_1\bh)\mathrm{head}_1, \dots, \mathrm{Sigmoid}(W^g_H\bh)\mathrm{head}_H\bigg]
-$$
-where $H$ is the number of heads in the attention layer.
-Hence, the hidden state $\bh$ can shut down or streamlining the scaled dot-product attention from specific head into the residual stream.
-This makes the attention layers more expressive.
-
-With the custom design of TrorYongASR, the head-specific score in self-attention layer is derived from context vectors, not the target token position embeddings (see @fig-archit).
-For cross-attention layer, the gate score is generated from the hidden state that derives the query vector for speech-text alignment.
-
-
-#### Normalization-free Transformer
-
-While normalization layers are crucial to the Transformer, they require a significant amount of compute compared to point-wise activation function.
-This induces longer training time and slower inference speed.
-[@chen2026stronger] illustrates that Dynamic rescaled Gaussian cumulative distribution function (abbreviated to Derf) can be used to replace normalization layers without hurting the performance.
-In fact, their analysis suggests performance gains compared to LayerNorm and RMSNorm across a wide range of domains, including visual recognition and generation, speech representation, and DNA sequence modeling.
-TrorYongASR2, thus, embraces this activation function in its auditory-lingual decoder and witnesses the gains in decoding performance and inference speed.
-However, TrorYongASR2 still maintains normalization layers over input representation (context vector, and audio encoding) and over pre-logit hidden state (see @fig-archit), as training otherwise became unstable.
-
-
-#### Additional Model's Capabilities
-
-
-##### Audio with arbitrary length
-
-TrorYongASR2 is trained with audio of arbitrary length.
-This aims at minimizing compute due to pad frames.
-Concretely, rather than padding each audio to 30-second length, audios are padded batch by batch to the longest length in the batch (the maximum length is still limited to 30-second long) and the attention mask is derived accordingly.
-
-In the audio encoder, the number of frames of log-mel spectrogram is reduced by half by 1D convolutional layer.
-So, the attention mask is also down sampled by half using 1D max pooling layer with the same configuration of the 1D convolutional layer (_i.e.,_ the same kernel size, stride, etc.).
-This updated attention mask is used all the way from encoder transformers in audio encoder to the cross-attention layer of auditory-lingual decoder, as well as the length predictor module.
-
-With this training pipeline, TrorYongASR2 works directly with audios of arbitrary length during inference.
-This means that the inference speed is also faster than the previous version.
-
-##### Voice-Activity Detection
-
-Another feature of speech-to-text model is to detect whether there is speech in the audio or not.
-A well-performed model in such detection task reduces its hallucination in both transcription and translation tasks.
-For this feature, `nonspeech7k` is used in training TrorYongASR2.
-
-TrorYongASR2 follows the multi-task training format of Whisper but without timestamps:
-For any audio, the token sequence starts with _begin-of-sequence_ token and ends with _end-of-sequence_ token.
-If the audio contains no speech, the first token is followed by _no-speech_.
-Otherwise, the second and third tokens define the spoken language and the task respectively.
-Follow the third token are text tokens (see @fig-multi).
-
-```mermaid
-flowchart LR
-  classDef alert fill:#cb4b4f,stroke:#000,color:#fff;
-  classDef success fill:#009245,stroke:#000,color:#fff;
-  classDef warn fill:#FFEB3B,stroke:#000,color:#fff;
-
-  A[Begin of sequence]:::success --> B[Language]
-  A[Begin of sequence]:::success --> C[No speech]
-  A ~~~ C
-  B --> D[Transcribe]
-  B --> E[Translate]
-  D --> F[Text]
-  E --> F[Text]
-  C --> G[End of sequence]:::alert
-  F --> G[End of sequence]:::alert
-```
-
-Multi-task training format of TrorYongASR2.
-The token sequence of any audio starts with `[B]`, begin-of-sequence token.
-For audio without speech, `[B]` is followed by `nospeech` token and `[E]`, end-of-sequence token.
-For other audios, the second token, `[L]`, represents the language of speech (either `km` for Khmer or `en` for English).
-The third token is the task token `[T]` that conditions the distribution of the text tokens that follow: for English audio, `transcribe` token favors English text tokens, and `translate` token favors Khmer text tokens.
-For Khmer audio, it is the reverse
-
-
-##### Translation
-
-TrorYongASR2 is trained with larger translation dataset.
-Concretely, there are around `142` hours of Khmer audio with English text and around `67` hours of English audio with Khmer text.
-This allows TrorYongASR2 to perform better for translation task.
-
-
 ## Training
 
 TrorYongASR2 is trained on 5 datasets:
 
-- `234` hours of `Digital-Divide-Data/khmer-speech-dataset` (the whole dataset contains 727.94 hours) for Khmer audio and Khmer text (transcription)
+- 234 hours of `Digital-Divide-Data/khmer-speech-dataset` (the whole dataset contains 727.94 hours) for Khmer audio and Khmer text (transcription)
 - `train.360` split of `openslr/librispeech_asr` for English audio and English text (transcription)
 - `KrorngAI/DDD-Cambodia-khmer-speech-dataset-parquet-000-159-en-translate` for Khmer audio and English text (translation)
 - `KrorngAI/openslr-librispeech_asr-clean100-00000-19999-km-translate` for English audio and Khmer text (translation)
 - `nospeech7k` for no speech audio (voice activity detection)
 
-So, there are around `804` hours of audio in the training dataset.
+So, there are around 804 hours of audio in the training dataset.
+
+TrorYongASR2 has 2 configurations as the following
+
+|  **Model Size**   |       Tiny        |        Small        |
+|:-----------------:|:-----------------:|:-------------------:|
+| **Audio Encoder** | 4 layers, 6 heads | 12 layers, 12 heads |
+| **Audio Context** |       1500        |        1500         |
+| **Auditory-lingual Decoder**  | 1 layer, 12 heads |  1 layer, 24 heads  |
+| **Text Context**  |        448        |         448         |
+| **Embedding Dimension** |        384        |         768         |
+|  **Parameters**   |        29M        |        136M         |
+
+The audio array is processed to log-mel spectrogram with 80 mel bins (the same as Whisper models of the same size).
+These configurations are the same as the ones of TrorYongASR1.
 
 To underline the gain from architectural improvement, TrorYongASR1 is retrained on the same 5 datasets above using the same set of hyperparameters used by TrorYongASR2.
-@fig-loss shows the training history of `tiny` configuration (for more information about the configuration, see section blow).
+@fig-loss shows the training history of `tiny` configuration.
 TrorYongASR2 has smaller training and validation losses, and higher validation token accuracy.
 This shows the effectiveness of the new architectural design in TrorYongASR2.
 The refinement also manifests in the evaluation below.
 
-![Training history of TrorYongASR1 represented by the blue color and TrorYongASR2 represented by the red one. The first row shows the training loss and the second one describe validation performance. The top right plot shows the moving average of training loss. All plots suggest that TrorYongASR2 has better performance compared to TrorYongASR1.](./figures/train_val.png){#fig-loss}
+<figure>
+  <div style="text-align: center;"><a name='fig-loss' ><img src="https://kimang18.github.io/krorngai-blog/TrorYongASR2/figures/train_val.png" width="100%" /></a></div>
+  <figcaption>
+    Training history of TrorYongASR1 represented by the blue color and TrorYongASR2 represented by the red one.
+    The first row shows the training loss and the second one describes validation performance.
+    The top right plot shows the moving average of training loss.
+    All plots suggest that TrorYongASR2 has better performance compared to TrorYongASR1.
+  </figcaption>
+</figure>
 
-<!--
 ### Training Hyperparameters
 
 - **Training regime:** 16-mixed precision training using `LightningAI` package
+- **Number of epochs:** roughly 2 epochs
 - **Optimizer:** MuonAdamW (custom implementation)
-- **Learning rate:** Linear Warmup (38 optimizer steps) + CosineAnnealing (3774 optimizer steps)
-- **Weight decay:** 0.1
+- **Number of optimizer steps:** 10149
+- **Learning rate:** Linear Warmup (153 optimizer steps) + CosineAnnealing (9996 optimizer steps)
 - **Effective Batch size:** 64
-- **Number of optimizer steps:** 3812
-- **Number of epochs:** roughly 2 epochs
 - **Gradient Clip Value:** 0.5 (only for parameters trained by AdamW)
 
 
-### Speeds, Sizes, Times
-
-The training was conducted over 3812 optimizer steps.
-
-- For tiny variant, the training took around 6 hours on 1 Tesla T4 GPU.
-- For small variant, the training took around 7 hours on 2 Tesla T4 GPU (using DDP strategy).
--->
-
-# Evaluation
+## Evaluation
 
-The evaluation assesses two capabilities — 1) language identification and 2) transcription — on two datasets ([`google/fleurs`](https://huggingface.co/datasets/Kimang18/google-fleurs-km-kh) for Khmer and [`openslr/librispeech_asr`](https://huggingface.co/datasets/openslr/librispeech_asr) for English).
+### Dataset
+The evaluation assesses transcription capability on two datasets: 1) [`google/fleurs`](https://huggingface.co/datasets/Kimang18/google-fleurs-km-kh) for Khmer and 2) [`openslr/librispeech_asr`](https://huggingface.co/datasets/openslr/librispeech_asr) for English.
 All results are from the **test split** of each dataset.
 
 | Dataset               | Language   | Testing samples  | Description                                       |
 |-----------------------|------------|------------------|---------------------------------------------------|
 | **google/fleurs**     | Khmer      | 765              | Multi-lingual dataset with Khmer language samples |
-| **librispeech.clean** | English    | 2611             | Clean speech dataset for English transcription    |
+| **librispeech_asr.clean** | English    | 2611             | Clean speech dataset for English transcription    |
 
-Audios longer than `30 seconds` are excluded from the evaluation (that is why `google/fleurs` has 765 samples instead of 771 and `librispeech.clean` has 2611 instead of 2620).
+Audio longer than 30 seconds are excluded from the evaluation (that is why `google/fleurs` has 765 samples instead of 771 and `librispeech_asr.clean` has 2611 instead of 2620).
 
-**TrorYongASR2** has 2 configurations as the following
 
-|  **Model Size**   |       Tiny        |        Small        |
-|:-----------------:|:-----------------:|:-------------------:|
-| **Audio Encoder** | 4 layers, 6 heads | 12 layers, 12 heads |
-| **Audio Context** |       1500        |        1500         |
-| **Text Decoder**  | 1 layer, 12 heads |  1 layer, 24 heads  |
-| **Text Context**  |        448        |         448         |
-| **Embedding Dim** |        384        |         768         |
-|  **Parameters**   |        29M        |        136M         |
-
-The audio array is processed to log-mel spectrogram with `80` mel bins (the same as Whisper models of the same size)
-
-These configurations are the same as the ones of TrorYongASR1.
-
-
-### Transcription
+### Metrics
 
 For transcription task, 3 metrics below are used
 
@@ -289,11 +163,11 @@ For transcription task, 3 metrics below are used
 - **Character Error Rate (CER)**: Proportion of characters that are incorrect
 - **Word Error Rate (WER)**: Proportion of words that are incorrect
 
-Token Error Rate measures model's capability in predicting the next token given the audio input and the current sequence of tokens.
-This metric is weaker than Word Error Rate (WER) and Character Error Rate (CER) because it doesn't account for insertions, deletions, substitutions, and autoregression as comprehensively.
+TER measures model's capability in predicting the next token given the audio input and the current sequence of tokens.
+This metric is weaker than WER and CER because it doesn't account for insertions, deletions, substitutions, and autoregression as comprehensively.
 
 
-**Results:**
+### Results
 
 | Model     | Metric | Khmer (`fleurs`) | English (`librispeech.clean`) |
 |-----------|--------|------------------|-------------------------------|
@@ -304,7 +178,6 @@ This metric is weaker than Word Error Rate (WER) and Character Error Rate (CER)
 |           | CER    | 29.26%           | 6.05%                         |
 |           | TER    | 38%              | 5%                            |
 
-: {.striped .hover}
 
 Both configurations performance well on English language.
 However, Khmer language remains a challenge.
@@ -318,166 +191,27 @@ It is due to no-boundary Khmer words that Token Error Rate is utilized as one of
 
 | Size     | Model        | Parameters | Khmer (`fleurs`)             | English (`librispeech.clean`) |
 | -------- | ------------ | ---------- | ---------------------------  | ----------------------------- |
-| **Tiny** | TrorYongASR2 | 29M        | 57.07%                      | 20.35%                         |
+| **Tiny** | TrorYongASR2 | 29M        | **57.07%**                      | 20.35%                         |
 |          | TrorYongASR1 | 29M        | 75.88%                      | 54.33%                         |
-|          | Whisper      | 39M        | 100.6%                      | 7.6%                           |
-| **Small**| TrorYongASR2 | 136M       | 36.70%                      | 10.12%                         |
+|          | Whisper      | 39M        | 100.6%                      | **7.6%**                           |
+|          |              |            |                              |                               |
+| **Small**| TrorYongASR2 | 136M       | **36.70%**                      | 10.12%                         |
 |          | TrorYongASR1 | 135M       | 50.46%                      | 21.75%                         |
-|          | Whisper      | 244M       | 104.4%                      | 3.4%                           |
+|          | Whisper      | 244M       | 104.4%                      | **3.4%**                           |
 
-: {.striped .hover}
 
 **Key Observations:**
 
 - As mentioned above, TrorYongASR2 clearly outperforms TrorYongASR1 and requires just a small increase in the number of parameters (the increase comes from additional length predictor and gate projection layers)
-- Whisper models have more parameters for comparable configuration (39M vs 29M for Tiny, 244M vs 136M for Small)
-- Whisper shows significantly lower word error rates on English (7.6% vs 20.35% for Tiny, 3.4% vs 10.12% for Small)
-- Whisper performs worse on Khmer (100.6% vs 57.07% for Tiny, 36.70% vs 50.46% for Small)
-
-**Note:** `WER` data of Whisper is taken from their [paper](https://arxiv.org/abs/2212.04356).
-
-**Note on Translation Task:** According to [KrorngAI](https://huggingface.co/KrorngAI), the 2 translation datasets above is built using API of [TranslateKH](https://translatekh.mptc.gov.kh/) which, in turn, uses an AI model to translate Khmer to English and vice versa.
-Without the means to check the correctness of the translation over `209` hours of audio, TrorYongASR2 is not evaluated on any translation datasets.
-It should be noted that the translation performance of TrorYongASR2 is upper bounded by the AI model used in the backend of [TranslateKH](https://translatekh.mptc.gov.kh/).
-
-
-## Downstream Tasks and Limitations
-
-TrorYongASR2 can be integrated into:
+- Whisper has most parameters for comparable configuration (39M vs 29M for `tiny`, and 244M vs 136M for `small`)
+- Whisper shows significantly lower word error rates on English (7.6% for `tiny`, 3.4% for `small`)
+- TrorYongASR2 performs best on Khmer dataset (57.07% for `tiny`, and 36.70% for `small`)
 
-- IoT devices: Smart speakers, voice assistants
-- Mobile application: Android/iOS apps with speech recognition
-- Larger ASR systems: As a component in multi-language ASR pipelines
+**Note:** `WER` data of Whisper is taken directly from their [paper](https://arxiv.org/abs/2212.04356).
 
-However, TrorYongASR2 does not have built-in support for timestamps prediction.
-It might be possible to develop this feature around built-in voice-activity detection.
-TrorYongASR2 is an Encoder-Decoder model for Automatic Speech Recognition (ASR) task.
-It is inspired by [PARSeq](https://github.com/baudm/parseq/tree/main) and [Whisper](https://github.com/openai/whisper/tree/main): the auditory-lingual decoder has only one transformer block.
-
-
-### Old section
-
-<div align="center">
-  <picture>
-      <img src="figures/architecture.png" width="100%" alt="TrorYongASR">
-  </picture>
-</div>
-
-
-## Evaluation
-
-The evaluation assesses two capabilities — language detection and transcription — on two datasets ([`google/fleurs`](https://huggingface.co/datasets/Kimang18/google-fleurs-km-kh) for Khmer and [`openslr/librispeech_asr`](https://huggingface.co/datasets/openslr/librispeech_asr) for English). All results are from the **test split** of each dataset, representing the model's generalization ability to unseen data.
-
-### Testing Data
-
-<!-- This should link to a Dataset Card if possible. -->
-
-<div align="center">
-
-| Dataset               | Language   | Testing examples | Description                                       |
-| -------------         | ---------- | -------------    | -                                                 |
-| **google/fleurs**     | Khmer      | 765              | Multi-lingual dataset with Khmer language samples |
-| **librispeech.clean** | English    | 2620             | Clean speech dataset for English transcription    |
-</div>
-
-**Note:** Audios longer than `30 seconds` are excluded from the evaluation (that is why `google/fleurs` has 765 examples instead of 771).
-
-### Metrics and Results
-
-<!-- These are the evaluation metrics being used, ideally with a description of why. -->
-
-#### Language Detection
-
-Language detection measures model’s capability to recognize the spoken language from audio input. Since TrorYongASR currently supports 2 languages, this task becomes binary classification task. Classic metrics are used:
-
-- **Precision**: Proportion of predicted languages that are correct
-- **Recall** : Proportion of actual language samples correctly identified
-- **F1-score** : Harmonic mean of precision and recall
-
-**Results:**
-
-<div align="center">
-
-| Model | Metrics   | Khmer (`fleurs`) | English (`librispeech.clean`) |
-|-------|-----------|------------------|-------------------------------|
-| Tiny  | Precision | 100%             | 100%                          |
-|       | Recall    | 100%             | 100%                          |
-|       | F1-score  | 100%             | 100%                          |
-| Small | Precision | 100%             | 99%                          |
-|       | Recall    | 96%              | 100%                          |
-|       | F1-score  | 98%              | 99%                          |
-</div>
-
-Tiny size achieved perfect language detection performance on both datasets, indicating excellent binary classification capability for distinguishing between Khmer and English audio. Small size performs slightly worst by tending to predict English language.
-
-The 100% language detection scores may appear unusually high. This is expected because during pre-training, the model performs permutations on word tokens starting from position 3, while the first three positions (start token, language token, and task token) remain fixed. Since language detection relies on the language token at position 1, and this token is never permuted during pre-training, the model can achieve perfect accuracy on language detection tasks.
-
-
-#### Transcription
-
-For transcription task, 3 metrics below are used
-
-- **Token Error Rate (TER)** : Proportion of incorrectly transcribed tokens
-- **Character Error Rate (CER)** : Proportion of characters that are incorrect
-- **Word Error Rate (WER)** : Proportion of words that are incorrect
-
-Token Error Rate (TER) measures model's capability in predicting the next token given the audio input and the current sequence of tokens. This metric is weaker than Word Error Rate (WER) and Character Error Rate (CER) because it doesn't account for insertions, deletions, substitutions, and autoregression as comprehensively. Token Error Rate is used here because Khmer text lacks word boundaries, making WER and CER calculations challenging without additional preprocessing.
-
-
-**Transcription Results:**
-
-<div align="center">
-
-| Model     | Metric                     | Khmer (`fleurs`) | English (`librispeech.clean`) | Mixed (Khmer + English) |
-|-----------|----------------------------|------------------|-------------------------------|-------------------------|
-| **Tiny**  | WER | 75.81%           | 54.33%                        | 60.36%                  |
-|           | CER | 54.99%           | 42.41%                        | 46.18%                  |
-|           | TER | 54%              | 17%                           | 27%                     |
-| **Small** | WER | 50.46%           | 21.75%                        | 29.78%                  |
-|           | CER | 35.89%           | 16.58%                        | 22.37%                  |
-|           | TER | 43%              | 8%                            | 18%                     |
-</div>
-
-**Key Observations:**
-
-- The tiny model shows strong performance on English (54.33% WER, 42.41% CER, 17% TER)
-- Performance drops significantly for Khmer (75.88% WER, 54.99% CER, 54% TER)
-- The small model shows strong performance on English (21.75% WER, 16.58% CER, 8% TER)
-- Performance for Khmer is moderate (50.46% WER, 35.89% CER, 43% TER)
-- The larger model benefits from increased embedding dimension (768 vs 384) and more layers for audio encoder (12 vs 4)
-
-**Note:** To compute `CER` and `WER`, whitespaces are added between words in Khmer text (Khmer text does not have word boundaries like English text). To do so, `khmercut` PyPI package is used to tokenize Khmer text into words, and then the words are joined back together with whitespaces.
-
-
-**WER Comparison with Whisper:**
-
-| Tiny        | Parameters | Khmer (`fleurs`)            | English (`librispeech.clean`) |
-| -------     | --------   | --------------------------- | ---                           |
-| TrorYongASR | 29M        | 75.88%                      | 54.33%                        |
-| Whisper     | 39M        | 100.6%                      | 7.6%                          |
-
-| Small       | Parameters | Khmer (`fleurs`)            | English (`librispeech.clean`) |
-| -------     | --------   | --------------------------- | ---                           |
-| TrorYongASR | 135M       | 50.46%                      | 21.75%                        |
-| Whisper     | 244M       | 104.4%                      | 3.4%                          |
-
-**Key Observations:**
-
-- Whisper models have more parameters for comparable sizes (39M vs 29M for Tiny, 244M vs 135M for Small)
-- Whisper shows significantly lower word error rates on English (7.6% vs 54.33% for Tiny, 3.4% vs 12.95% for Small)
-- Whisper performs worse on Khmer (100.6% vs 75.88% for Tiny, 104.4% vs 50.46% for Small)
-- Error rates > 100% for Whisper on Khmer indicate the model is overfitting to the training data
-
-**Note:** `WER` data of Whisper is taken from their [paper](https://arxiv.org/abs/2212.04356).
-
-
-### Result Summary
-
-**Language Detection:** Both model sizes achieved great performance across all metrics (Precision, Recall, F1-score) on both datasets, indicating excellent binary classification capability for distinguishing between Khmer and English audio. This high score is expected because during pre-training, the model performs permutations on word tokens starting from position 3, while the first three positions (start token, language token, and task token) remain fixed. Since language detection relies on the language token at position 1, and this token is never permuted during pre-training, the model can achieve perfect accuracy on language detection tasks.
-
-**Transcription:** The Small model shows strong performance on English (21.75% WER, 16.58% CER, 8% TER) and moderate performance for Khmer (50.46% WER, 35.89% CER, 43% TER). The Tiny model shows strong performance on English (54.33% WER, 42.41% CER, 17% TER) but significantly lower performance for Khmer (75.88% WER, 54.99% CER, 54% TER). This shows that TrorYongASR can be scaled to get higher performance.
-
-**Note on Translation Task:** The models are also trained for `translation` task, but evaluation is deferred to future work due to scarce data (there are only 2000 examples from Khmer audio to English text, and 1000 examples from English audio to Khmer text in the pre-training).
+**Note on Translation Task:** According to [KrorngAI](https://huggingface.co/KrorngAI), the 2 translation datasets above are built using the API of [TranslateKH](https://translatekh.mptc.gov.kh/) which, in turn, uses an AI model for the translation (Khmer to English and vice-versa).
+Without any means to check the correctness of the translation over 209 hours of audio, TrorYongASR2 is not evaluated on any translation datasets.
+It can be assumed that the translation performance of TrorYongASR2 is upper bounded by the AI model used in the backend of [TranslateKH](https://translatekh.mptc.gov.kh/).
 
 
 ## How to Get Started with the Model
@@ -494,7 +228,7 @@ from transformers import AutoProcessor
 from tror_yong_asr import TrorYongASRModel, transcribe, translate, detect_language
 
 
-model_id = "KrorngAI/TrorYongASR-small"
+model_id = "KrorngAI/TrorYongASR2-small"
 processor = AutoProcessor.from_pretrained(model_id, trust_remote_code=True)
 model = TrorYongASRModel.from_pretrained(model_id, trust_remote_code=True)
 
@@ -518,15 +252,15 @@ Notebook (TBA)
 
 ### Direct Use
 
-The Tiny model can be used directly for:
+The model can be used directly for:
 - **Speech-to-text transcription**: transcribe Khmer and English audio
 - **Speech-to-text translation**: translate Khmer audio to English text and English audio to Khmer text
-- **Language detection**: Identify whether audio is in Khmer or English (100% accuracy)
-- **Edge computing**: Deploy on mobile devices, IoT devices, and embedded systems due to its small size (29M parameters)
+- **Language identification**: identify if the spoken language in an audio is Khmer or English (100% accuracy)
+- **Edge computing**: Deploy on mobile devices, IoT devices, and embedded systems due to its small size (29M parameters for tiny or 136M for small)
 - **Real-time applications**: Low latency inference suitable for real-time speech interfaces
 
 
-### Downstream Use [optional]
+### Downstream Use
 
 The model can be integrated into:
 - **Mobile applications**: Android/iOS apps with speech recognition
@@ -538,10 +272,8 @@ The model can be integrated into:
 ## Bias, Risks, and Limitations
 
 **Technical Limitations:**
-- **No speech detection**: The model was not trained for this task. User needs to fine-tune the model for this task (TrorYongASRTokenizer has `<|nospeech|>` token.)
-- **Translate task**: The training data for translation task is scarce. User needs to fine-tune the model for better translation performance
-- **Noise robustness**: Performance may degrade in noisy environments
 - **No timestamp output**: The model does not support timestamp output
+- **Noise robustness**: Performance may degrade in noisy environments
 
 **Sociotechnical Limitations:**
 - **Accent variability**: May not perform well on diverse Khmer accents
@@ -549,15 +281,6 @@ The model can be integrated into:
 - **Speaker variability**: May struggle with different speaking styles and rates
 
 
-### Recommendations
-
-<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
-
-Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
-
-
-
-
 ## Citation
 
 <!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
@@ -567,10 +290,11 @@ Users (both direct and downstream) should be made aware of the risks, biases and
 ```bibtex
 @online{khun2026,
   author = {Khun, Kimang},
-  title = {TrorYongASR: {Permuted} {AutoRegressive} {Sequence}
-    {Modeling} for {Automatic} {Speech} {Recognition}},
-  date = {2026-05-07},
-  url = {https://kimang18.github.io/krorngai-blog/TrorYongASR/},
+  title = {TrorYongASR2: {Permuted} {Language} {Modeling} with
+    {Monotonic} {Alignment,} {Gated} {Attention,} and
+    {Normalization-free} {Design}},
+  date = {2026-10-19},
+  url = {https://kimang18.github.io/krorngai-blog/TrorYongASR2/},
   langid = {en}
 }
 ```
@@ -583,13 +307,9 @@ Users (both direct and downstream) should be made aware of the risks, biases and
 
 ## Acknowledgement
 
-[`LightningAI`](https://lightning.ai), Kaggle, and `Google Colab` did not specifically sponsor this project.
-But, both models are trained thanks to their free credits.
-So, huge thanks to [`LightningAI`](https://lightning.ai), Kaggle and `Google Colab`.
-
-Thanks to the authors of [`PARSeq`](https://github.com/baudm/parseq/tree/main) and [`Whisper`](https://github.com/openai/whisper/tree/main) for their publicly available sourcecode.
+Both models are trained thanks to the free credits from Kaggle, and `Google Colab`.
 
-Thanks to [`openslr`](https://openslr.org), [Mozilla Data Collective](https://mozilladatacollective.com/datasets/cml9h5vgc01bxmn075sjeftek) and Google for their publicly available dataset.
+Thanks to [`KrorngAI`](https://huggingface.co/KrorngAI), [`openslr`](https://openslr.org), and [Mozilla Data Collective](https://mozilladatacollective.com/datasets/cml9h5vgc01bxmn075sjeftek) for their publicly available dataset.
 
 ## Model Card Contact