LakoreAI/sev
Text Classification • 0.4B • Updated
Minh Le Duc PRO
minhleduc
AI & ML interests
Edge Device-oriented Models (language, audio, ...)
Recent Activity
updated a model about 14 hours ago
LakoreAI/sev published a model about 14 hours ago
LakoreAI/sev updated a model about 14 hours ago
minhleduc/laya-typed-decisions-ce-1024Organizations
updated a
model about 14 hours ago
published a
model about 14 hours ago
updated a
model about 14 hours ago
published a
model about 14 hours ago
updated a
model about 18 hours ago
published a
model about 19 hours ago
updated 2
datasets 8 days ago
published 2
datasets 12 days ago
updated a
dataset 25 days ago
published a
dataset 25 days ago
reacted to Felladrin's post with 👍 about 1 month ago
Post
4211
I've open-sourced the trainer I've been using to build tiny language models from scratch, together with the 95M base model I trained with it.
The trainer runs on Deno (https://deno.com, cross-platform), trains on WebGPU, and it writes GGUF directly. No Python/PyTorch. The weights live in a GGUF file from the first step to the last, so every checkpoint is already something llama.cpp can load.
The model is Felladrin/Minueza-3-95M-Base: 94.7M parameters, 1.95B tokens seen, 8192 context.
And here’s the repository on GitHub: https://github.com/felladrin/gguf-trainer
Here on Hugging Face, I published the optimizer state next to the weights, so you can continue the pretraining instead of starting over.
Or start your own from nothing:
And the docs are written for coding agents, so you can point your agent of choice at the GitHub repo and have it drive the whole pipeline.
The trainer runs on Deno (https://deno.com, cross-platform), trains on WebGPU, and it writes GGUF directly. No Python/PyTorch. The weights live in a GGUF file from the first step to the last, so every checkpoint is already something llama.cpp can load.
The model is Felladrin/Minueza-3-95M-Base: 94.7M parameters, 1.95B tokens seen, 8192 context.
And here’s the repository on GitHub: https://github.com/felladrin/gguf-trainer
Here on Hugging Face, I published the optimizer state next to the weights, so you can continue the pretraining instead of starting over.
Or start your own from nothing:
deno run -A cli.ts demo trains a tiny one end to end in under a minute.And the docs are written for coding agents, so you can point your agent of choice at the GitHub repo and have it drive the whole pipeline.
- 36 replies
upvoted 2
articles about 1 month ago
Article
🕳️ Attention Sinks in LLMs for endless fluency
tomaarsen
• • 38 Article
What We Learned by Reproducing 2,200 papers from ICML
abidlabs
• • 118 upvoted a collection about 2 months ago
liked a
model 3 months ago
updated a
model 3 months ago
published a
model 3 months ago
updated a
Space 3 months ago