README.md

metadata

pipeline_tag: text-generation
license: mit
base_model: openai-community/gpt2-medium
library_name: zeromodels
language:
  - en
tags:
  - keras
  - zeromodels
  - gpt2
  - gpt-2
  - text-generation
  - pytorch
  - jax
  - tf

Run GPT-2 with Keras 3: JAX, PyTorch, or TensorFlow

zeromodels/gpt2_medium

Paper: Language Models are Unsupervised Multitask Learners (Radford et al., 2019)

GPT-2 is OpenAI's decoder-only transformer language model trained on WebText: learned absolute position embeddings, pre-LayerNorm blocks, gelu_new activations, a tied output head, and a byte-level BPE tokenizer. This is the 355M variant, a base completion model (no chat template).

For more details, see the upstream model card.

Pure-Keras 3 conversion of openai-community/gpt2-medium for zeromodels. One implementation runs unmodified on TensorFlow / Torch / JAX.

Quick start

import os
os.environ["KERAS_BACKEND"] = "torch"  # or "jax" / "tensorflow"

from zeromodels.models.gpt2 import GPT2TextGenerate, GPT2Tokenizer

model = GPT2TextGenerate.from_weights("zeromodels/gpt2_medium")
tokenizer = GPT2Tokenizer.from_weights("zeromodels/gpt2_medium")

inputs = tokenizer("The meaning of life is")
outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0]))

All GPT-2 sizes load the same way with from_weights("zeromodels/<variant>"):

Variant	Hub	Params
gpt2	zeromodels/gpt2	124M
gpt2_medium	zeromodels/gpt2_medium	355M
gpt2_large	zeromodels/gpt2_large	774M
gpt2_xl	zeromodels/gpt2_xl	1.5B

Tips

• Set KERAS_BACKEND before importing Keras / zeromodels.
• This is a base completion model: it continues a prompt and is not instruction-tuned.
• See the GPT-2 docs and Loading Weights.
• Upstream safetensors still work via the hf: prefix, e.g. GPT2TextGenerate.from_weights("hf:openai-community/gpt2-medium").

Special Thanks

A huge thank you to the OpenAI GPT-2 authors for creating and releasing these models.

License: MIT, inherited from the upstream OpenAI GPT-2 release.