Migrate to zeromodels (rename kf_*.json -> zm_*.json, fix refs in config + README, ensure tag + badge)
b5e34cf verified | pipeline_tag: text-generation | |
| license: mit | |
| base_model: openai-community/openai-gpt | |
| library_name: zeromodels | |
| language: | |
| - en | |
| tags: | |
| - keras | |
| - zeromodels | |
| - gpt | |
| - openai-gpt | |
| - text-generation | |
| - pytorch | |
| - jax | |
| - tf | |
| # Run GPT with Keras 3: JAX, PyTorch, or TensorFlow | |
| [](https://github.com/IMvision12/ZeroModels) [](https://imvision12.github.io/ZeroModels/gpt/) | |
| # zeromodels/gpt | |
| Paper: [Improving Language Understanding by Generative Pre-Training (Radford et al., 2018)](https://cdn.openai.com/research-covers/language-unsupervised/language_understanding_paper.pdf) | |
| GPT (the original GPT-1) is OpenAI's first generative pre-trained transformer: a | |
| 12-layer decoder-only model with learned position embeddings (512-token context), | |
| `gelu_new` activations, and a byte-pair-encoding tokenizer, trained on BookCorpus. | |
| This is the **117M** base completion model (no chat template). | |
| For more details, see the upstream [model card](https://huggingface.co/openai-community/openai-gpt). | |
| Pure-**Keras 3** conversion of [`openai-community/openai-gpt`](https://huggingface.co/openai-community/openai-gpt) for | |
| [zeromodels](https://github.com/IMvision12/ZeroModels). One implementation runs unmodified on | |
| **TensorFlow / Torch / JAX**. | |
| ## Quick start | |
| ```python | |
| import os | |
| os.environ["KERAS_BACKEND"] = "torch" # or "jax" / "tensorflow" | |
| from zeromodels.models.gpt import GptTextGenerate, GptTokenizer | |
| model = GptTextGenerate.from_weights("zeromodels/gpt") | |
| tokenizer = GptTokenizer.from_weights("zeromodels/gpt") | |
| inputs = tokenizer("the meaning of life is") | |
| outputs = model.generate(**inputs, max_new_tokens=40) | |
| print(tokenizer.decode(outputs[0])) | |
| ``` | |
| ## Tips | |
| - Set `KERAS_BACKEND` **before** importing Keras / zeromodels. | |
| - Context length is 512 tokens; this is a base completion model, not | |
| instruction-tuned. | |
| - See the [GPT docs](https://imvision12.github.io/ZeroModels/gpt/) and [Loading Weights](https://imvision12.github.io/ZeroModels/loading_weights/). | |
| - Upstream safetensors still work via the `hf:` prefix, e.g. | |
| `GptTextGenerate.from_weights("hf:openai-community/openai-gpt")`. | |
| ## Special Thanks | |
| A huge thank you to the OpenAI GPT authors for creating and releasing this model. | |
| License: MIT, inherited from the upstream OpenAI GPT release. | |