{% extends "base.html" %} {% block title %}NanoDex — train a language model from scratch{% endblock %} {% block body %}
{{ n_devices }} {{ 'GPU' if on_gpu else 'CPU' }} worker{{ 's' if n_devices != 1 }} · {{ stats.done }} models born here

Train your first
decoder-only model
from scratch.

Not a fine-tune. Not an adapter. Random weights, a real corpus, and a real loss curve you watch fall in real time — from a browser tab, on our GPUs.

Start a run  → Browse the models

Signing in opens NanoDex full screen — Hugging Face's login page can't run inside an embedded frame.

1

Pick a size

Four architectures, from 492k to 8.1M parameters. That's the real total — embeddings included, no asterisk.

2

Pick a token budget

Anywhere from {{ min_tokens // 10**6 }}M to 1.5 billion tokens of fineweb-edu. More tokens, sharper model, longer wait.

3

Watch it learn

Your run joins the queue. Live loss curve, throughput, logs — then the weights land in your own Hugging Face account.

The four sizes

Small enough to finish. Big enough to learn something.

{% for t in tiers %}
{{ t.label }}
{{ "{:,}".format(t.params) }} params
{{ t.layers }} layers · {{ t.hidden }} hidden
{{ t.heads }} heads ({{ t.kv_heads }} KV) · FFN {{ t.ffn }}
{% endfor %}
What you actually get

A model that is genuinely yours

Every finished run is pushed to your Hugging Face namespace: config.json, model.safetensors, the tokenizer, a training_run.json with the full recipe, and a generated model card. Load it with transformers like anything else on the Hub.

Honest expectations

It will not be an assistant

At a few hundred thousand parameters, a model learns word shapes, common collocations and a little syntax. It will produce English-looking text with no facts in it. That's the correct outcome — and watching cross-entropy fall from 7.6 (uniform noise over 2,048 tokens) to somewhere near 4 is the whole point.

{% if recent %}
Trained here

The newest models on the shelf

Explore all →
{% for m in recent %}

{{ m.name }}

completed
{{ m.tier_label }} {{ "{:,}".format(m.n_params) }} params {{ (m.tokens_seen / 1000000)|round|int }}M tokens {% if m.best_loss %}loss {{ "%.4f"|format(m.best_loss) }}{% endif %} @{{ m.username }}
{% endfor %}
{% endif %}

Ready when you are.

Sign in with Hugging Face — no separate account, no password.

Start a run  →
{% endblock %}