Running 2 Sparsely gated tiny linear experts ๐ฅ 2 A compute-efficient and interpretable transformer FFN layer
Running 2 Sparsely gated tiny linear experts ๐ฅ 2 A compute-efficient and interpretable transformer FFN layer
Running 2 Sparsely gated tiny linear experts ๐ฅ 2 A compute-efficient and interpretable transformer FFN layer
Discovering modular solutions that generalize compositionally Paper โข 2312.15001 โข Published Dec 22, 2023