| library_name: aneforge | |
| license: mit | |
| base_model: openai-community/gpt2 | |
| pipeline_tag: text-generation | |
| tags: | |
| - aneforge | |
| - apple-neural-engine | |
| # GPT-2 (ANEForge) | |
| An unmodified duplicate of [`openai-community/gpt2`](https://huggingface.co/openai-community/gpt2), tagged for use with | |
| [**ANEForge**](https://github.com/sbryngelson/ANEForge) so the weights load and run directly on the | |
| Apple Neural Engine (no CoreML). Weights are byte-identical to the source; see the original repo for | |
| the model details. Licensed `mit` from the source. | |
| ## Use with ANEForge | |
| ```python | |
| # pip install "aneforge[models]" | |
| import aneforge as af | |
| from transformers import AutoTokenizer | |
| tok = AutoTokenizer.from_pretrained("aneforge/gpt2") | |
| model = af.load_llm("aneforge/gpt2") # prefill + resident-KV-cache decode on the ANE | |
| ids = tok.encode("The Neural Engine is") | |
| print(tok.decode(model.generate(ids, max_new_tokens=20))) | |
| ``` | |
| ANEForge compiles the model's graph into a single ANE program and streams the weights from this repo | |
| via `huggingface_hub`. See the [docs](https://aneforge.readthedocs.io) and the | |
| [paper](https://arxiv.org/abs/2606.17090). | |