How to use sujalgawas/code-bert-python with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("feature-extraction", model="sujalgawas/code-bert-python")
# pip install -U transformers accelerate # Load model directly from transformers import AutoTokenizer, AutoModel tokenizer = AutoTokenizer.from_pretrained("sujalgawas/code-bert-python") model = AutoModel.from_pretrained("sujalgawas/code-bert-python", device_map="auto")
https://huggingface.co/sujalgawas/code-bert-python/resolve/1af6f96adb621513df5fe15e2e7c96289a7a9c3e/tokenizer.json
hf download hf://sujalgawas/code-bert-python@1af6f96adb621513df5fe15e2e7c96289a7a9c3e/tokenizer.json
curl -L -o tokenizer.json https://huggingface.co/sujalgawas/code-bert-python/resolve/1af6f96adb621513df5fe15e2e7c96289a7a9c3e/tokenizer.json
