Instructions to use nornor02/CustomTokenizer with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use nornor02/CustomTokenizer with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("fill-mask", model="nornor02/CustomTokenizer")# pip install -U transformers accelerate # Load model directly from transformers import AutoTokenizer, AutoModelForMaskedLM tokenizer = AutoTokenizer.from_pretrained("nornor02/CustomTokenizer") model = AutoModelForMaskedLM.from_pretrained("nornor02/CustomTokenizer", device_map="auto") - Notebooks
- Google Colab
- Kaggle
Model save
Browse files
model.safetensors
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
size 328693404
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:436a7593ebe4f2df307f5ee8d6f54c12af4747623e0689a1399ea659cf64de57
|
| 3 |
size 328693404
|
runs/Jan10_01-30-15_e28f27bc0ed8/events.out.tfevents.1704850216.e28f27bc0ed8.6087.0
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
-
size
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:484a6c468c6e38f3394432da73311ec398afa1e3c481f8231797bf6bc584b78c
|
| 3 |
+
size 6342
|