DeltonLabs

community

AI & ML interests

None defined yet.

Recent Activity

Harley-ml updated a model 10 days ago

DeltonLabs/TT-A-Micro

Harley-ml published a model 10 days ago

DeltonLabs/TT-A-Micro

Harley-ml updated a dataset 13 days ago

DeltonLabs/DCLM-Filtered-HQ

View all activity

updated a model 10 days ago

DeltonLabs/TT-A-Micro

3.44M • Updated 10 days ago • 113

published a model 10 days ago

DeltonLabs/TT-A-Micro

3.44M • Updated 10 days ago • 113

updated a dataset 13 days ago

DeltonLabs/DCLM-Filtered-HQ

Preview • Updated 12 days ago • 3.77k • 3

updated a dataset 15 days ago

DeltonLabs/HQ-EN-Sentences

Viewer • Updated 15 days ago • 29.5M • 731 • 3

published a dataset 15 days ago

DeltonLabs/HQ-EN-Sentences

Viewer • Updated 15 days ago • 29.5M • 731 • 3

updated a dataset 19 days ago

DeltonLabs/CoreWiki

Viewer • Updated 19 days ago • 184k • 125

published 2 datasets 21 days ago

DeltonLabs/CoreWiki

Viewer • Updated 19 days ago • 184k • 125

DeltonLabs/DCLM-Filtered-HQ

Preview • Updated 12 days ago • 3.77k • 3

posted an update about 2 months ago

Post

141

Zero-v1.0-144M is currently training. So far, it has completed 26% of its training run, with about 98 billion tokens to go.

Current Val PPL: 13.705

Zero-v1.0 uses a custom architecture consisting of RMSNorm, RoPe, SwiGLU, Engram Conditional Memory, mHC, and XSA GQA.