Instructions to use AgentBull/CJK-Tokenizer with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use AgentBull/CJK-Tokenizer with Transformers:
# Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("AgentBull/CJK-Tokenizer", device_map="auto") - Notebooks
- Google Colab
- Kaggle
File size: 131 Bytes
f040328 | 1 2 3 4 | version https://git-lfs.github.com/spec/v1
oid sha256:72f9a32ae6620d306d7278863fae281f21dddaaa223f87087c3b218e18dd3c7a
size 690403
|