---
license: mit
language: [en]
pipeline_tag: text-generation
tags: [udgamlabs, small-language-model, character-level, on-device-training, tiny-shakespeare]
---

![PhoneGPT · Udgam Labs](./banner.png)

**UDGAM LABS** · Language model · 1.21M parameters

# PhoneGPT

A 1.21M-parameter character-level language model, trained on the CPU of a 2021 Samsung Galaxy F62, live on YouTube and X, on 5 November 2026. It started from random weights and learned from about 1 MB of Shakespeare, in flight mode, with no cloud.

We publish it as a record of that run, not as a model to use. It writes Shakespeare-shaped nonsense. PhoneGPT is only our project name, with no link to the offline-AI iPhone app of the same name.

**Read and watch:** [the write-up and tutorial](https://udgamlabs.com/blog/phonegpt) · [code on GitHub](https://github.com/udgamlabs/phonegpt) · [the stream recording](https://www.youtube.com/watch?v=gXQFqCBB91U)

## Quick start

These checkpoints are in the trainer's own format, so they need the small C programs from GitHub, not `transformers`. You need [Zig](https://ziglang.org/download/). This writes text on your computer:

```bash
git clone https://github.com/udgamlabs/phonegpt
cd phonegpt
curl -LO https://huggingface.co/udgamlabs/PhoneGPT/resolve/main/checkpoints/live_step018000.bin
zig cc -O3 -ffast-math -o phone_sample phone_sample.c -lpthread -lm
./phone_sample live_step018000.bin 500
```

To train your own on a phone, follow [the tutorial](https://udgamlabs.com/blog/phonegpt#train-your-own).

## Results

The target was 1.4697, the best validation loss in the README of [nanoGPT](https://github.com/karpathy/nanoGPT), on one A100 GPU. Lower is better. Our rule, set before the start: at or below 1.4697 at two checkpoints in a row.

**It missed by 0.0047.** The best score was 1.4663 at one checkpoint, and the closest two in a row were 1.4744 and 1.4663.

| Step | 16,000 | 17,000 | 18,000 | 19,000 | 20,000 | 24,000 |
|---|---:|---:|---:|---:|---:|---:|
| Hours in | 7.2 | 7.6 | 8.1 | 8.5 | 9.0 | 10.75 |
| Score | 1.4782 | 1.4744 | **1.4663** | 1.4748 | 1.4785 | 1.4817 |

Each score is on every window of the held-out 10% of [Tiny Shakespeare](https://github.com/karpathy/char-rnn), computed on a Mac from the phone's checkpoint. All 24 scores are in `results/scores.csv`.

This isn't a like-for-like comparison:

| | nanoGPT's README | PhoneGPT |
|---|---|---:|
| Hardware | One A100 GPU | Galaxy F62, 2 CPU cores |
| Time | About 3 minutes | 10 h 45 min |
| Parameters | 10.65M | 1.21M |
| Context | 256 characters | 128 characters |
| Score | Best of its sampled checks | Every window of the same held-out split |

Output at step 24,000, real and unedited, cut for length:

```text
DION:
Say, where I may be made what I add
And to my lord, like a mercy than to my father.

LORD WILLOUGHBY:
As I am, in king, and so follow my uncle?
```

## Where it falls short

- It missed the target we set. One checkpoint under the line doesn't meet the rule.
- It has no practical use. It reads and writes single characters, knows only one small text, and its words and sentences often make no sense.
- 1.4697 comes from a small teaching run, not a measure of a good language model, and a GPU reaches it in minutes.
- 1.4663 is the best of 24 scores on the same text, so it flatters the model a little. From step 18,000 to 24,000 the score rose by 0.0154, which looks like overfitting; we didn't test that with another run.
- This is one run on one phone.
- The numbers in `results/live_log.txt` and in the headers of `results/live_samples.txt` are the trainer's quick check on a small sample, and they read lower than the scores above. Use `results/scores.csv` to compare.

## How it was trained

- **Model:** a nanoGPT-style GPT with 6 layers, 4 attention heads, width 128 and context 128: 1,214,976 parameters. It isn't nanoGPT.
- **Data:** the first 90% of Tiny Shakespeare, about 1 MB in all. No other data, and no other model's weights.
- **Trainer:** `phone_gpt.c`, adapted from [llm.c](https://github.com/karpathy/llm.c) by Andrej Karpathy, who isn't involved in this. Seed 20261105, 24,000 steps. The exact command is in the [GitHub README](https://github.com/udgamlabs/phonegpt).
- **Compute:** training ran on 2 big CPU cores of the phone's Exynos 9825, about 1.5 seconds a step, plugged in over USB.

## Files

| Path | What it is |
|---|---|
| `checkpoints/live_stepNNNNNN.bin` | All 25 checkpoints, from random weights at step 0 to step 24,000; about 5 MB each. Step 18,000 scored best |
| `hashes.csv` | SHA-256 of each checkpoint, made when it came off the phone |
| `results/scores.csv` | Step and whole-split score |
| `results/live_log.txt` | The trainer's log from the run |
| `results/live_samples.txt` | The text it wrote every 250 steps |

## Intended use and limitations

A record of the run, and a small object for learning: look at how a character model improves between checkpoints, or check our scores. Not for any real writing task.

## Licence

MIT. The trainer it came from is also MIT.

## Citation

```bibtex
@misc{phonegpt2026,
  title  = {PhoneGPT: a small language model trained live on a 5-year-old phone},
  author = {Udgam Labs},
  year   = {2026},
  url    = {https://huggingface.co/udgamlabs/PhoneGPT}
}
```

---

**About Udgam Labs.** Udgam Labs is an independent AI research lab. We work from first principles: before we build, we understand why things work, whether we are training a model, fine-tuning one or applying AI to a new problem. Udgam (उद्गम) is Sanskrit for origin. [udgamlabs.com](https://udgamlabs.com)