Teaching Pretrained Language Models to Think Deeper with Retrofitted Recurrence
Paper • 2511.07384 • Published • 21
Retrofitting Recurrence
-
-
smcleish/Recurrent-Llama-3.2-train-recurrence-32
Text Generation • 1B • Updated • 1.72k • 1 -
smcleish/Recurrent-Llama-3.2-train-recurrence-16
Text Generation • 1B • Updated • 139 -
smcleish/Recurrent-Llama-3.2-train-recurrence-8
Text Generation • 1B • Updated • 150 -
smcleish/Recurrent-Llama-3.2-train-recurrence-4
Text Generation • 1B • Updated • 457 -
smcleish/Recurrent-TinyLlama-3T-train-recurrence-32
Text Generation • 0.8B • Updated • 699 • 2 -
smcleish/Recurrent-TinyLlama-3T-train-recurrence-16
Text Generation • 0.8B • Updated • 46 • 1 -
smcleish/Recurrent-TinyLlama-3T-train-recurrence-8
Text Generation • 0.8B • Updated • 109 -
smcleish/Recurrent-TinyLlama-3T-train-recurrence-4
Text Generation • 0.8B • Updated • 186 -
smcleish/Recurrent-OLMo-2-0425-train-recurrence-32
Text Generation • 1B • Updated • 706 • 3 -
smcleish/Recurrent-OLMo-2-0425-train-recurrence-16
Text Generation • 1B • Updated • 327 -
smcleish/Recurrent-OLMo-2-0425-train-recurrence-8
Text Generation • 1B • Updated • 71 -
smcleish/Recurrent-OLMo-2-0425-train-recurrence-4
Text Generation • 1B • Updated • 103 • 1 -
smcleish/Recurrent-TinyLlama-3T-train-recurrence-4-single-phase
Text Generation • 0.8B • Updated • 20 -
smcleish/Recurrent-TinyLlama-3T-train-recurrence-4-two-phase
Text Generation • 0.8B • Updated • 36 -
smcleish/Recurrent-Llama-3.2-untrained
Text Generation • 1B • Updated • 76 -
smcleish/Recurrent-TinyLlama-3T-untrained
Text Generation • 0.8B • Updated • 37 -
smcleish/Recurrent-OLMo-2-0425-untrained
Text Generation • 1B • Updated • 29 -
smcleish/Recurrent-Llama-3.2-2-4-2-untrained
Text Generation • 1B • Updated • 32 • 1 -
smcleish/retrofitting-llama-fineweb-edu-tokenized
Viewer • Updated • 332M • 99 • 1 -
smcleish/Llama-3.2-non-recurrent-posttrained
Text Generation • 1B • Updated • 77