---
license: other
license_name: openai-gpt-2-modified-mit
license_link: LICENSE
language:
- en
tags:
- gpt-2
- tensorflow
pipeline_tag: text-generation
---

# GPT-2 TensorFlow checkpoints

This repository mirrors OpenAI's original GPT-2 checkpoints for the weight-loading example in [Chapter 5 of Build a Large Language Model (From Scratch)](https://github.com/rasbt/LLMs-from-scratch/tree/main/ch05/01_main-chapter-code).

The four folders, `124M`, `355M`, `774M`, and `1558M`, follow OpenAI's directory layout. Each contains the TensorFlow checkpoint, its hyperparameters (`hparams.json`), and the GPT-2 tokenizer files (`encoder.json` and `vocab.bpe`).

## Loading a checkpoint

The following example uses the Chapter 5 downloader with the `source` argument to load the 124M checkpoint directly from Hugging Face.

```python
from gpt_download import download_and_load_gpt2

settings, params = download_and_load_gpt2("124M", models_dir="gpt2", source="hf")
```

The function saves the seven files in `gpt2/124M/` and returns the settings and NumPy arrays used in the chapter. Change `"124M"` to load one of the larger models. You can also download the seven files directly from a [model folder on the Hub](https://huggingface.co/rasbt/gpt2-tensorflow/tree/main/124M).

OpenAI released the weights under its [modified MIT license](https://github.com/openai/gpt-2/blob/master/LICENSE). For the same weights converted to PyTorch state dictionaries and safetensors, see [rasbt/gpt2-from-scratch-pytorch](https://huggingface.co/rasbt/gpt2-from-scratch-pytorch).