| license: apache-2.0 | |
| language: | |
| - en | |
| base_model: | |
| - meta-llama/Llama-3.1-8B-Instruct | |
| pipeline_tag: text-generation | |
| We simply applied the methodology from the paper titled "TransMLA: MLA Is All You Need" to convert the attention mechanism within the Llama-3 model from Grouped Query Attention (GQA) to Multi-Head Latent Attention (MLA), Kinda. |