robinzixuan/Phi4_attn_residual
Text Generation • 4B • Updated • 808 • 1
OASIS
-
-
robinzixuan/Phi4_attn_OASIS
Text Generation • 4B • Updated • 550 -
robinzixuan/Phi4_attn_AoS
Text Generation • 4B • Updated • 536 -
Attention Sinks and Outliers in Attention Residuals
Paper • 2605.17887 • Published • 1 -
robinzixuan/qwen3-0.6b-oasis
Text Generation • 0.6B • Updated -
robinzixuan/qwen3-0.6b-vanilla
Text Generation • 0.6B • Updated -
robinzixuan/llama-3.2-1b-vanilla
Text Generation • 1B • Updated -
robinzixuan/llama-3.2-1b-oasis
Text Generation • 1B • Updated -
robinzixuan/oasis_llama3.2_1b_6k
Text Generation • 1B • Updated • 275 -
robinzixuan/vanilla_llama3.2_1b_6k
Text Generation • 1B • Updated • 302 -
robinzixuan/oasis_qwen3_0.6b_6k
Text Generation • 0.6B • Updated • 281 -
robinzixuan/vanilla_qwen3_0.6b_6k
Text Generation • 0.6B • Updated • 307 -
robinzixuan/sink_logit_llama3.2_1b
Text Generation • 1B • Updated • 295 -
robinzixuan/sink_token_llama3.2_1b
Text Generation • 1B • Updated • 305 -
robinzixuan/s0_softmax1_noroute_llama3.2_1b
Text Generation • 1B • Updated • 293