| --- |
| license: mit |
| datasets: |
| - SupraLabs/chat-titles-filtered-115K |
| - ogrnz/chat-titles |
| - Michionlion/chat-titles-english |
| language: |
| - en |
| --- |
| # TinyTitle |
|
|
| A tiny model (~1.8M params) that turns a chat message into a short title (2-10 words). English only. The whole thing (model + tokenizer + runtime) runs in under 5 MiB of ram, in a few tens of ms, on one small C binary. |
|
|
| It's a small GRU with a copy trick: for each title word it either makes up a word from its 8k vocabulary or copies a whole word straight from your message (so it keeps your spelling and casing). |
|
|
| Honest note: an LLM wrote almost all of this while I nodded along. It works tho. It's a toy, not a real llm. |
|
|
| ## quickstart |
|
|
| This is a custom format (`.ttm1`) with a custom C runtime, so it does not work with transformers or llama.cpp. You need the runtime source from the [github repo](https://github.com/azomDev/TinyTitle). |
|
|
| ```bash |
| # build the runtime (needs cc, nothing else) |
| cc -std=c11 -O3 -DNDEBUG -o title-v1 runtime/main.c -lm |
| |
| # run it |
| ./title-v1 model.ttm1 "Why does my wifi keep dropping?" |
| ``` |
|
|
| Files in this repo: |
|
|
| - `model.ttm1` - the int8 model (1.98 MB) |
| - `tokenizer-8k.json` + `tok-8k.ttok` - the tokenizer |
| - `runtime/` - main.c, ttm.h, Makefile (the whole runtime is just those files) |
|
|
| ## numbers |
|
|
| | metric | value | |
| |---|---| |
| | params | ~1.8M (int8) | |
| | model file | 1.98 MB | |
| | peak rss | 4.89 mib (all pages touched) | |
| | typical cpu | ~25 ms | |
| | vocab | 8k unigram | |
|
|
| ## train |
|
|
| Training needs python + torch. Full instructions in the [github repo](https://github.com/azomDev/TinyTitle). It trains in under 0.5 GB of vram. |
|
|
| ## license |
|
|
| MIT. Training data comes from three public huggingface datasets ([SupraLabs/chat-titles-filtered-115K](https://huggingface.co/datasets/SupraLabs/chat-titles-filtered-115K) cc-by-4.0, [ogrnz/chat-titles](https://huggingface.co/datasets/ogrnz/chat-titles) MIT, [Michionlion/chat-titles-english](https://huggingface.co/datasets/Michionlion/chat-titles-english) cc-by-4.0), assembled by the github repo's `build_dataset.py`. |
