|
Download README.md from kennynakamura/embedding-lite73: direct link, hf CLI and curl.
- Browser
- Download file 742 Bytes
-
https://huggingface.co/kennynakamura/embedding-lite73/resolve/main/README.md
- Command line
-
hf download hf://kennynakamura/embedding-lite73/README.md
-
curl -L -o README.md https://huggingface.co/kennynakamura/embedding-lite73/resolve/main/README.md
742 Bytes
| license: mit | |
| tags: | |
| - approx-gelu | |
| - constant-warmup | |
| - generation | |
| - grouped-query | |
| - huge | |
| - hybrid | |
| - lamb | |
| - scalenorm | |
| - trunc-normal | |
| - tucker | |
| # train.py | |
| ## Model Overview | |
| A **huge**-scale implementation of the **hybrid** architecture, built for **generation** tasks. | |
| ## Architecture | |
| - **Architecture**: hybrid | |
| - **Scale**: huge | |
| - **Attention**: grouped query | |
| - **Fusion strategy**: tucker | |
| - **Task head**: generation | |
| - **Activation**: approx gelu | |
| - **Normalization**: scalenorm | |
| - **Initialization**: trunc normal | |
| ## Training | |
| - **Optimizer**: lamb | |
| - **LR scheduler**: constant warmup | |
| ## Files | |
| - `train.py` — main artifact of this repository | |
| ## License | |
| See the license field above. | |