Article
Scaling Pedagogical Pre-training: From Optimal Mixing to 10 Billion Tokens
codelion
• • 8
Uchiha Rephilo
Rephilo
·
AI & ML interests
None yet
Recent Activity
upvoted an article 4 days ago
Scaling Pedagogical Pre-training: From Optimal Mixing to 10 Billion Tokens upvoted an article 4 days ago
The 1 Billion Token Challenge: Finding the Perfect Pre-training Mix upvoted an article 4 days ago
The Optimal Architecture for Small Language ModelsOrganizations
upvoted 3
articles 4 days ago
Article
The 1 Billion Token Challenge: Finding the Perfect Pre-training Mix
codelion
• • 67 Article
The Optimal Architecture for Small Language Models
codelion
• • 123