Kimi Linear: An Expressive, Efficient Attention Architecture
Paper • 2510.26692 • Published • 137
sdsd
sdd5125
·
AI & ML interests
None yet
Recent Activity
reacted to Banaxi-Tech's post with 🔥 2 days ago
We're announcing BananaMind 2 Micro, our smallest model in the BananaMind 2 model family.
This model is not released yet, training has not started yet.
It uses only 2.9M parameters, while being overtrained on 75B tokens to get the maximum intelligence per parameter.
The key changes are:
No more AdamW, the model will use the Muon optimizer offering up to 2x faster convergence and higher lr.
LR goes to 2.2e-2.
We're adding the XSA refresh gate from the TX4 architecture into our own.
Training will start on August 3, release date is estimated to be August 4-6.
On August 3 we will also release our Public Preview of BananaMind 2 Pro.
Follow us to know when our models release
https://huggingface.co/BananaMind
@Banaxi-Tech
liked a model 2 days ago
deepseek-ai/DeepSeek-V4-Flash-0731 upvoted a paper 5 days ago
Kimi Linear: An Expressive, Efficient Attention ArchitectureOrganizations
None yet
upvoted a paper 5 days ago
upvoted a paper 13 days ago
upvoted an article 10 months ago
Article
Simplifying Alignment: From RLHF to Direct Preference Optimization (DPO)
ariG23498
• • 53