Post
87
i love reinforcement learning
AI & ML interests
Models and API early access previews for KoFi donators
Recent Activity
Post
1299
A Small Model is All You Need. Meet
After 3 years of experiments, we are finally releasing our flagship tiny model: **palmer-006**.
If you are building for edge hardware, SBCs (Raspberry Pi, etc.), or low-power devices, this is for you. Inspired by Andrej Karpathy's idea of a self-contained "cognitive core," we wanted to see how much power we could pack into a sub-100M parameter footprint.
🧠 **How we "Palmerized" it:**
We believe in starting our experiments with the absolute strongest baseline possible.
1. Light fine-tuning on highly curated data
2. Model merging
3. Another light fine-tuning round
4. Adjusted Mamba for maximum token speed ⚡️
⚠️ *Note: This is a foundational language model. It has not been instruction-tuned yet!*
Also, since this needs instruction tuning next to become a chat assistant—**what dataset would you recommend we use for the instruct tune?**
---
🔗 **Quick Links & Info:**
* **License:** Open for research, education, hobby, and modification! (For commercial use/hosted APIs, shoot an email to nosoyhackercodigo@gmail.com. *PS: Donators can claim a free commercial license!*)
* **Attribution:** Built using AI tech from the Technology Innovation Institute (TII).
Can't wait to see what you build at the edge. Let me know your prompt completions below! 👇
appvoid/palmer-006
palmer-006 (90M)After 3 years of experiments, we are finally releasing our flagship tiny model: **palmer-006**.
If you are building for edge hardware, SBCs (Raspberry Pi, etc.), or low-power devices, this is for you. Inspired by Andrej Karpathy's idea of a self-contained "cognitive core," we wanted to see how much power we could pack into a sub-100M parameter footprint.
🧠 **How we "Palmerized" it:**
We believe in starting our experiments with the absolute strongest baseline possible.
1. Light fine-tuning on highly curated data
2. Model merging
3. Another light fine-tuning round
4. Adjusted Mamba for maximum token speed ⚡️
⚠️ *Note: This is a foundational language model. It has not been instruction-tuned yet!*
Also, since this needs instruction tuning next to become a chat assistant—**what dataset would you recommend we use for the instruct tune?**
---
🔗 **Quick Links & Info:**
* **License:** Open for research, education, hobby, and modification! (For commercial use/hosted APIs, shoot an email to nosoyhackercodigo@gmail.com. *PS: Donators can claim a free commercial license!*)
* **Attribution:** Built using AI tech from the Technology Innovation Institute (TII).
Can't wait to see what you build at the edge. Let me know your prompt completions below! 👇
appvoid/palmer-006
- 10 replies
Post
3323
Two big projects are open sourced soon. Get ready...
- 11 replies
Post
217
If you make cool smol 🤖🤏 models (below 0.5b parameters), leave a reply and I will follow you! I'm serious, you don't need to follow me at all just share something through the replies and I (and potentially more people) will follow you (if your models are decent ofc).
- 11 replies
Post
4647
if you are a tinkerer of small language models and want to stay ahead of what small models can do, follow me!!! seriously, start following people that actually still makes small models
i've made one recently btw
also, i'm keeping an eye on AxiomicLabs leaderboard, looks like the only current alternative to check where the things are going to
though, between us, i think they should add agentic/tool use benchmarks there
anyways,
enjoy!
appvoid/a-cool-model
i've made one recently btw
also, i'm keeping an eye on AxiomicLabs leaderboard, looks like the only current alternative to check where the things are going to
though, between us, i think they should add agentic/tool use benchmarks there
anyways,
enjoy!
appvoid/a-cool-model
- 19 replies
Post
104
A huge amount of large synthetic datasets on huggingface looks surprisingly like templates, that might be one of the main reasons open models might not be as good as other models, we need more people to create smaller, human-curated datasets instead of lazily sending millions of requests to large models for us to fulfill.
- 4 replies
Post
258
small reasoning models are overrated, these little ones just doom loop a lot by default. good data will always be the moat when training or finetuning small models and latest sota models like fable 5 and gpt 5.6 are increasingly making this a lot easier to do.
- 3 replies
Post
191
yikes! i missed the small model hackathon i guess i'll have to make sota for people to notice
Post
112
model | epoch | step | val_loss |
c-gpt-1| 5 | 41365 | 1.0175 |
If you don't understand what you see, this is just one the coolest proof of concepts I've ever made. I just trained a char-level, super small model (~80 million parameters) on millions of high-quality examples on a curated dataset and is slowly getting davinci-003 vibes
c-gpt-1| 5 | 41365 | 1.0175 |
If you don't understand what you see, this is just one the coolest proof of concepts I've ever made. I just trained a char-level, super small model (~80 million parameters) on millions of high-quality examples on a curated dataset and is slowly getting davinci-003 vibes
Post
183
As an advocate for small language models I just want to say. It might not actually be the end for small models. We are just getting started! Now that we have super good models we can find creative ways to replicate the behavior at small scale!
I'll show you in a few weeks what a small model is capable of, you will surprised.
I'll show you in a few weeks what a small model is capable of, you will surprised.
- 1 reply
Post
203
Cheers for a year of sota AI on cpus 🥂 people actually liked my last model, here's another sota for you. This one should feel way different in terms of quality.
appvoid/palmer-005-core
appvoid/palmer-005-core
Post
143
new model tomorrow, less than 100m params, 1b level on text editing tasks
🤖 palmer is coming back...
🤖 palmer is coming back...
- 1 reply
Post
163
Yesterday someone faked an anthropic account: https://huggingface.co/Anthropic-ai/claude
Be careful... all I'm saying.
Be careful... all I'm saying.
- 1 reply
Post
2529
Let's keep the momentum for small models. I just published dot. It's the first pretrained causal model that is trained on math/symbols rather than english. The goal is to get an agnostic fewshot meta learner that learns from reality itself instead of language.
It's already decent at some tasks, with next version coming in a few weeks.
appvoid/dot
It's already decent at some tasks, with next version coming in a few weeks.
appvoid/dot
- 5 replies
Post
246
Are you ready for some ●s? Tomorrow will be a good day.
- 4 replies
Post
963
granite-4.0-350m, rwkv7-g1d-0.4b and LFM2-350M are currently the best sub 0.5b models currently for fewshot, simple language tasks
no one is saying this:
if you need the absolute speed + small size + quality, granite 350m is the current king
no one is saying this:
if you need the absolute speed + small size + quality, granite 350m is the current king
- 3 replies