Add model architecture section and diagram
Browse files- .gitattributes +1 -0
- README.md +6 -0
- images/lambda_audio_architecture.png +3 -0
.gitattributes
CHANGED
|
@@ -36,3 +36,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
|
| 36 |
images/reazonspeech_test.png filter=lfs diff=lfs merge=lfs -text
|
| 37 |
images/common_voice_25_ja_test.png filter=lfs diff=lfs merge=lfs -text
|
| 38 |
images/fleurs_ja_test.png filter=lfs diff=lfs merge=lfs -text
|
|
|
|
|
|
| 36 |
images/reazonspeech_test.png filter=lfs diff=lfs merge=lfs -text
|
| 37 |
images/common_voice_25_ja_test.png filter=lfs diff=lfs merge=lfs -text
|
| 38 |
images/fleurs_ja_test.png filter=lfs diff=lfs merge=lfs -text
|
| 39 |
+
images/lambda_audio_architecture.png filter=lfs diff=lfs merge=lfs -text
|
README.md
CHANGED
|
@@ -55,6 +55,12 @@ On Japanese subset of Google FLEURS ASR benchmark, it achieves a character error
|
|
| 55 |
| Output | Japanese transcription text |
|
| 56 |
| Audio feature interval | 20 ms after the encoder; 40 ms after the adapter |
|
| 57 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 58 |
## Training data
|
| 59 |
|
| 60 |
| Stage | Dataset | Sources and labels |
|
|
|
|
| 55 |
| Output | Japanese transcription text |
|
| 56 |
| Audio feature interval | 20 ms after the encoder; 40 ms after the adapter |
|
| 57 |
|
| 58 |
+
## Architecture
|
| 59 |
+
|
| 60 |
+
The Zipformer encoder extracts features from 16 kHz audio. An audio adapter reduces the sequence length and projects these features into the decoder's hidden dimension. Gated cross-attention in the LFM2 full-attention blocks lets the decoder use audio features to predict the next text token.
|
| 61 |
+
|
| 62 |
+

|
| 63 |
+
|
| 64 |
## Training data
|
| 65 |
|
| 66 |
| Stage | Dataset | Sources and labels |
|
images/lambda_audio_architecture.png
ADDED
|
Git LFS Details
|