gpjt/8xa100m40-lore-1-baseline-disabled
Text Generation • 0.2B • Updated • 132
LoRE
Models using low-rank bottlenecks for the embeddings or the output head. See https://www.gilesthomas.com/2026/10/low-rank-vocab-matrices
-
-
gpjt/8xa100m40-lore-10-both-corrected-smart-init
Text Generation • 0.1B • Updated • 133 -
gpjt/8xa100m40-lore-11-output-only-corrected-smart-init
Text Generation • 0.1B • Updated • 139 -
gpjt/8xa100m40-lore-2-both-ordinary-init
Text Generation • 0.1B • Updated • 152 -
gpjt/8xa100m40-lore-3-both-smart-init
Text Generation • 0.1B • Updated • 164 -
gpjt/8xa100m40-lore-4-input-only
Text Generation • 0.1B • Updated • 158 -
gpjt/8xa100m40-lore-5-output-only
Text Generation • 0.1B • Updated • 156 -
gpjt/8xa100m40-lore-6-new-seed-baseline-disabled
Text Generation • 0.2B • Updated • 152 -
gpjt/8xa100m40-lore-7-new-seed-input-only
Text Generation • 0.1B • Updated • 160 -
gpjt/8xa100m40-lore-8-new-seed-both-smart-init
Text Generation • 0.1B • Updated • 166 -
gpjt/8xa100m40-lore-9-new-seed-output-only
Text Generation • 0.1B • Updated • 173