| --- |
| license: mit |
| language: [en] |
| pipeline_tag: text-generation |
| tags: [udgamlabs, small-language-model, character-level, on-device-training, tiny-shakespeare] |
| --- |
| |
|  |
|
|
| **UDGAM LABS** · Language model · 1.21M parameters |
|
|
| # PhoneGPT |
|
|
| A 1.21M-parameter character-level language model, trained on the CPU of a 2021 Samsung Galaxy F62, live on YouTube and X, on 5 November 2026. It started from random weights and learned from about 1 MB of Shakespeare, in flight mode, with no cloud. |
|
|
| We publish it as a record of that run, not as a model to use. It writes Shakespeare-shaped nonsense. PhoneGPT is only our project name, with no link to the offline-AI iPhone app of the same name. |
|
|
| **Read and watch:** [the write-up and tutorial](https://udgamlabs.com/blog/phonegpt) · [code on GitHub](https://github.com/udgamlabs/phonegpt) · [the stream recording](https://www.youtube.com/watch?v=gXQFqCBB91U) |
|
|
| ## Quick start |
|
|
| These checkpoints are in the trainer's own format, so they need the small C programs from GitHub, not `transformers`. You need [Zig](https://ziglang.org/download/). This writes text on your computer: |
|
|
| ```bash |
| git clone https://github.com/udgamlabs/phonegpt |
| cd phonegpt |
| curl -LO https://huggingface.co/udgamlabs/PhoneGPT/resolve/main/checkpoints/live_step018000.bin |
| zig cc -O3 -ffast-math -o phone_sample phone_sample.c -lpthread -lm |
| ./phone_sample live_step018000.bin 500 |
| ``` |
|
|
| To train your own on a phone, follow [the tutorial](https://udgamlabs.com/blog/phonegpt#train-your-own). |
|
|
| ## Results |
|
|
| The target was 1.4697, the best validation loss in the README of [nanoGPT](https://github.com/karpathy/nanoGPT), on one A100 GPU. Lower is better. Our rule, set before the start: at or below 1.4697 at two checkpoints in a row. |
|
|
| **It missed by 0.0047.** The best score was 1.4663 at one checkpoint, and the closest two in a row were 1.4744 and 1.4663. |
|
|
| | Step | 16,000 | 17,000 | 18,000 | 19,000 | 20,000 | 24,000 | |
| |---|---:|---:|---:|---:|---:|---:| |
| | Hours in | 7.2 | 7.6 | 8.1 | 8.5 | 9.0 | 10.75 | |
| | Score | 1.4782 | 1.4744 | **1.4663** | 1.4748 | 1.4785 | 1.4817 | |
|
|
| Each score is on every window of the held-out 10% of [Tiny Shakespeare](https://github.com/karpathy/char-rnn), computed on a Mac from the phone's checkpoint. All 24 scores are in `results/scores.csv`. |
|
|
| This isn't a like-for-like comparison: |
|
|
| | | nanoGPT's README | PhoneGPT | |
| |---|---|---:| |
| | Hardware | One A100 GPU | Galaxy F62, 2 CPU cores | |
| | Time | About 3 minutes | 10 h 45 min | |
| | Parameters | 10.65M | 1.21M | |
| | Context | 256 characters | 128 characters | |
| | Score | Best of its sampled checks | Every window of the same held-out split | |
|
|
| Output at step 24,000, real and unedited, cut for length: |
|
|
| ```text |
| DION: |
| Say, where I may be made what I add |
| And to my lord, like a mercy than to my father. |
| |
| LORD WILLOUGHBY: |
| As I am, in king, and so follow my uncle? |
| ``` |
|
|
| ## Where it falls short |
|
|
| - It missed the target we set. One checkpoint under the line doesn't meet the rule. |
| - It has no practical use. It reads and writes single characters, knows only one small text, and its words and sentences often make no sense. |
| - 1.4697 comes from a small teaching run, not a measure of a good language model, and a GPU reaches it in minutes. |
| - 1.4663 is the best of 24 scores on the same text, so it flatters the model a little. From step 18,000 to 24,000 the score rose by 0.0154, which looks like overfitting; we didn't test that with another run. |
| - This is one run on one phone. |
| - The numbers in `results/live_log.txt` and in the headers of `results/live_samples.txt` are the trainer's quick check on a small sample, and they read lower than the scores above. Use `results/scores.csv` to compare. |
|
|
| ## How it was trained |
|
|
| - **Model:** a nanoGPT-style GPT with 6 layers, 4 attention heads, width 128 and context 128: 1,214,976 parameters. It isn't nanoGPT. |
| - **Data:** the first 90% of Tiny Shakespeare, about 1 MB in all. No other data, and no other model's weights. |
| - **Trainer:** `phone_gpt.c`, adapted from [llm.c](https://github.com/karpathy/llm.c) by Andrej Karpathy, who isn't involved in this. Seed 20261105, 24,000 steps. The exact command is in the [GitHub README](https://github.com/udgamlabs/phonegpt). |
| - **Compute:** training ran on 2 big CPU cores of the phone's Exynos 9825, about 1.5 seconds a step, plugged in over USB. |
|
|
| ## Files |
|
|
| | Path | What it is | |
| |---|---| |
| | `checkpoints/live_stepNNNNNN.bin` | All 25 checkpoints, from random weights at step 0 to step 24,000; about 5 MB each. Step 18,000 scored best | |
| | `hashes.csv` | SHA-256 of each checkpoint, made when it came off the phone | |
| | `results/scores.csv` | Step and whole-split score | |
| | `results/live_log.txt` | The trainer's log from the run | |
| | `results/live_samples.txt` | The text it wrote every 250 steps | |
|
|
| ## Intended use and limitations |
|
|
| A record of the run, and a small object for learning: look at how a character model improves between checkpoints, or check our scores. Not for any real writing task. |
|
|
| ## Licence |
|
|
| MIT. The trainer it came from is also MIT. |
|
|
| ## Citation |
|
|
| ```bibtex |
| @misc{phonegpt2026, |
| title = {PhoneGPT: a small language model trained live on a 5-year-old phone}, |
| author = {Udgam Labs}, |
| year = {2026}, |
| url = {https://huggingface.co/udgamlabs/PhoneGPT} |
| } |
| ``` |
|
|
| --- |
|
|
| **About Udgam Labs.** Udgam Labs is an independent AI research lab. We work from first principles: before we build, we understand why things work, whether we are training a model, fine-tuning one or applying AI to a new problem. Udgam (उद्गम) is Sanskrit for origin. [udgamlabs.com](https://udgamlabs.com) |
|
|
