How to use xwjzds/pretrain with Transformers:
# pip install -U transformers accelerate # Load model directly from transformers import AutoTokenizer, Net tokenizer = AutoTokenizer.from_pretrained("xwjzds/pretrain") model = Net.from_pretrained("xwjzds/pretrain", device_map="auto")
https://huggingface.co/xwjzds/pretrain/resolve/main/tokenizer.json
hf download hf://xwjzds/pretrain/tokenizer.json
curl -L -o tokenizer.json https://huggingface.co/xwjzds/pretrain/resolve/main/tokenizer.json
