Drop-Upcycling
updated
9B • Updated • 96
9B • Updated • 121
19B • Updated • 131
9B • Updated • 137
9B • Updated • 131
0.4B • Updated • 140
0.4B • Updated • 122
0.4B • Updated • 143
0.4B • Updated • 144
9B • Updated • 130
19B • Updated • 145
0.4B • Updated • 135
9B • Updated • 133
0.4B • Updated • 135
• 1
2B • Updated • 92
0.2B • Updated • 96
4B • Updated • 90
14B • Updated • 118
llm-jp/Dense-btx-code-expert-152M
0.2B • Updated • 87
• 1
llm-jp/Dense-btx-english-expert-1.5B
2B • Updated • 85
llm-jp/Dense-btx-code-expert-1.5B
2B • Updated • 82
• 1
llm-jp/Dense-btx-japanese-expert-1.5B
2B • Updated • 153
• 1
llm-jp/Dense-btx-english-expert-152M
0.2B • Updated • 80
llm-jp/Dense-btx-japanese-expert-152M
0.2B • Updated • 131
Drop-Upcycling: Training Sparse Mixture of Experts with Partial
Re-initialization
Paper
• 2502.19261
• Published • 5
Text Generation
• 73B • Updated • 139
llm-jp/llm-jp-3-8x13b-instruct3
Text Generation
• 73B • Updated • 277
• 8
llm-jp/llm-jp-3-8x1.8b-instruct3
Text Generation
• 9B • Updated • 199
• 4
Text Generation
• 9B • Updated • 189
llm-jp/llm-jp-3-8x13b-instruct2
Text Generation
• 73B • Updated • 242
llm-jp/llm-jp-3-8x1.8b-instruct2
Text Generation
• 9B • Updated • 160
llm-jp/llm-jp-3.1-8x13b-instruct4
Text Generation
• 73B • Updated • 186
• 4
Text Generation
• 73B • Updated • 152