Instructions to use nornor02/CustomTokenizer with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use nornor02/CustomTokenizer with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("fill-mask", model="nornor02/CustomTokenizer")# pip install -U transformers accelerate # Load model directly from transformers import AutoTokenizer, AutoModelForMaskedLM tokenizer = AutoTokenizer.from_pretrained("nornor02/CustomTokenizer") model = AutoModelForMaskedLM.from_pretrained("nornor02/CustomTokenizer", device_map="auto") - Notebooks
- Google Colab
- Kaggle
Model save
Browse files
model.safetensors
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
size 328693404
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:19b70ab5a45d6f4fb861c78c43d1cd27ba586b861dbc3ed9e93c5fdf27e0aefc
|
| 3 |
size 328693404
|
runs/Jan08_16-33-07_c156b750c2e0/events.out.tfevents.1704731588.c156b750c2e0.108.0
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
-
size
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:efb9d153a4ca78c2160754002fec6cf796cf353afab59695fe59a29f94444f30
|
| 3 |
+
size 5557
|