Instructions to use AgentBull/CJK-Tokenizer with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use AgentBull/CJK-Tokenizer with Transformers:
# Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("AgentBull/CJK-Tokenizer", device_map="auto") - Notebooks
- Google Colab
- Kaggle
- Xet hash:
- 2f9d3fc0e73230e6b0b237f44381ea8698ae22d735320b4a4c4c76c03af9c9ed
- Size of remote file:
- 690 kB
- SHA256:
- 72f9a32ae6620d306d7278863fae281f21dddaaa223f87087c3b218e18dd3c7a
·
Xet efficiently stores Large Files inside Git, intelligently splitting files into unique chunks and accelerating uploads and downloads. More info.