KeisukeMiyamoto commited on
Commit
8ede679
·
verified ·
1 Parent(s): 4746ea3

Add model architecture section and diagram

Browse files
.gitattributes CHANGED
@@ -36,3 +36,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
36
  images/reazonspeech_test.png filter=lfs diff=lfs merge=lfs -text
37
  images/common_voice_25_ja_test.png filter=lfs diff=lfs merge=lfs -text
38
  images/fleurs_ja_test.png filter=lfs diff=lfs merge=lfs -text
 
 
36
  images/reazonspeech_test.png filter=lfs diff=lfs merge=lfs -text
37
  images/common_voice_25_ja_test.png filter=lfs diff=lfs merge=lfs -text
38
  images/fleurs_ja_test.png filter=lfs diff=lfs merge=lfs -text
39
+ images/lambda_audio_architecture.png filter=lfs diff=lfs merge=lfs -text
README.md CHANGED
@@ -55,6 +55,12 @@ On Japanese subset of Google FLEURS ASR benchmark, it achieves a character error
55
  | Output | Japanese transcription text |
56
  | Audio feature interval | 20 ms after the encoder; 40 ms after the adapter |
57
 
 
 
 
 
 
 
58
  ## Training data
59
 
60
  | Stage | Dataset | Sources and labels |
 
55
  | Output | Japanese transcription text |
56
  | Audio feature interval | 20 ms after the encoder; 40 ms after the adapter |
57
 
58
+ ## Architecture
59
+
60
+ The Zipformer encoder extracts features from 16 kHz audio. An audio adapter reduces the sequence length and projects these features into the decoder's hidden dimension. Gated cross-attention in the LFM2 full-attention blocks lets the decoder use audio features to predict the next text token.
61
+
62
+ ![LambdaAudio architecture: Zipformer encoder, audio adapter, and LFM2 decoder with gated cross-attention](images/lambda_audio_architecture.png)
63
+
64
  ## Training data
65
 
66
  | Stage | Dataset | Sources and labels |
images/lambda_audio_architecture.png ADDED

Git LFS Details

  • SHA256: 54d9efd83cd4af054164124d772db90113f9248472784b8aecfc3da4520b6b5a
  • Pointer size: 131 Bytes
  • Size of remote file: 470 kB