Instructions to use nornor02/CustomTokenizer with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use nornor02/CustomTokenizer with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("fill-mask", model="nornor02/CustomTokenizer")# pip install -U transformers accelerate # Load model directly from transformers import AutoTokenizer, AutoModelForMaskedLM tokenizer = AutoTokenizer.from_pretrained("nornor02/CustomTokenizer") model = AutoModelForMaskedLM.from_pretrained("nornor02/CustomTokenizer", device_map="auto") - Notebooks
- Google Colab
- Kaggle
Download training_args.bin from nornor02/CustomTokenizer: direct link, hf CLI and curl.
- Browser
- Download file 4.28 kB
-
https://huggingface.co/nornor02/CustomTokenizer/resolve/main/training_args.bin
- Command line
-
hf download hf://nornor02/CustomTokenizer/training_args.bin
-
curl -L -o training_args.bin https://huggingface.co/nornor02/CustomTokenizer/resolve/main/training_args.bin
4.28 kB
- Xet hash:
- 10367552fa179acae1b996ca446901e2b8a3da0d0fcb45259d1e9215ea4ee6dd
- Size of remote file:
- 4.28 kB
- SHA256:
- a3e54757f0a3baa80d1f61d2de060fb50a8609929fe9f38759f5b486814f40f4
·
Xet efficiently stores Large Files inside Git, intelligently splitting files into unique chunks and accelerating uploads and downloads. More info.