How to use aneforge/gpt2 with ANEForge:
# Run this model on the Apple Neural Engine, without CoreML. import aneforge as af from transformers import AutoTokenizer tok = AutoTokenizer.from_pretrained("aneforge/gpt2") model = af.load_llm("aneforge/gpt2") # prefill + resident-KV-cache decode on the ANE ids = tok.encode("The Neural Engine is") print(tok.decode(model.generate(ids, max_new_tokens=20)))
https://huggingface.co/aneforge/gpt2/resolve/main/tokenizer.json
hf download hf://aneforge/gpt2/tokenizer.json
curl -L -o tokenizer.json https://huggingface.co/aneforge/gpt2/resolve/main/tokenizer.json
