Split the architecture figure from the pruning figure
Browse filesThe model is now its own figure (assets/architecture.png), with one transformer layer opened up; assets/pruning.png shows only the before-and-after pruning comparison.
- .gitattributes +1 -0
- README.md +5 -1
- assets/architecture.png +3 -0
- assets/pruning.png +2 -2
.gitattributes
CHANGED
|
@@ -35,3 +35,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
|
| 35 |
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
| 36 |
assets/tradeoff.png filter=lfs diff=lfs merge=lfs -text
|
| 37 |
assets/pruning.png filter=lfs diff=lfs merge=lfs -text
|
|
|
|
|
|
| 35 |
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
| 36 |
assets/tradeoff.png filter=lfs diff=lfs merge=lfs -text
|
| 37 |
assets/pruning.png filter=lfs diff=lfs merge=lfs -text
|
| 38 |
+
assets/architecture.png filter=lfs diff=lfs merge=lfs -text
|
README.md
CHANGED
|
@@ -42,7 +42,11 @@ Test sets; greedy CTC decoding, no language model. CER for Khmer, WER for Englis
|
|
| 42 |
|
| 43 |
## Architecture
|
| 44 |
|
| 45 |
-
<p align="center"><img src="https://huggingface.co/Darayut/sota-base/resolve/main/assets/
|
|
|
|
|
|
|
|
|
|
|
|
|
| 46 |
|
| 47 |
*What is pruned*, shown for sota-large: whole attention heads (MHA) and FFN units are removed from every transformer layer, and the CNN front end is kept. Each cell is a quarter of a layer's heads or units.
|
| 48 |
|
|
|
|
| 42 |
|
| 43 |
## Architecture
|
| 44 |
|
| 45 |
+
<p align="center"><img src="https://huggingface.co/Darayut/sota-base/resolve/main/assets/architecture.png" width="70%" alt="Sota's architecture, reading upwards from 16 kHz audio to the CTC head, with one transformer layer opened up"></p>
|
| 46 |
+
|
| 47 |
+
*The model*, reading upwards, with one transformer layer opened up: 16 kHz audio, a feature extractor of 7 convolutional layers giving one frame every 20 ms, a convolutional position encoder added back to the frames, 48 pre-LayerNorm transformer layers of width 1,280, a final LayerNorm and a CTC head over 9,812 characters. Every attention head and every FFN unit has its own Hard Concrete gate z. The cells in the opened layer are layer 12 of sota-large: 8 of 16 heads and 3,355 of 5,120 FFN units kept; which ones are kept is schematic.
|
| 48 |
+
|
| 49 |
+
<p align="center"><img src="https://huggingface.co/Darayut/sota-base/resolve/main/assets/pruning.png" width="100%" alt="The fine-tuned omniASR CTC-1B before structured pruning and sota-large after: CNN and CTC head kept, attention heads and FFN units removed"></p>
|
| 50 |
|
| 51 |
*What is pruned*, shown for sota-large: whole attention heads (MHA) and FFN units are removed from every transformer layer, and the CNN front end is kept. Each cell is a quarter of a layer's heads or units.
|
| 52 |
|
assets/architecture.png
ADDED
|
Git LFS Details
|
assets/pruning.png
CHANGED
|
Git LFS Details
|
|
Git LFS Details
|