How to use novelcore/gem-roberta-bilingual with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("fill-mask", model="novelcore/gem-roberta-bilingual")
# pip install -U transformers accelerate # Load model directly from transformers import AutoTokenizer, AutoModelForMaskedLM tokenizer = AutoTokenizer.from_pretrained("novelcore/gem-roberta-bilingual") model = AutoModelForMaskedLM.from_pretrained("novelcore/gem-roberta-bilingual", device_map="auto")
https://huggingface.co/novelcore/gem-roberta-bilingual/resolve/main/tokenizer.json
hf download hf://novelcore/gem-roberta-bilingual/tokenizer.json
curl -L -o tokenizer.json https://huggingface.co/novelcore/gem-roberta-bilingual/resolve/main/tokenizer.json
