Instructions to use facebook/rag-token-base with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use facebook/rag-token-base with Transformers:
# pip install -U transformers accelerate # Load model directly from transformers import AutoTokenizer, RagTokenForGeneration tokenizer = AutoTokenizer.from_pretrained("facebook/rag-token-base") model = RagTokenForGeneration.from_pretrained("facebook/rag-token-base", device_map="auto") - Notebooks
- Google Colab
- Kaggle
Commit History
-
Adding `safetensors` variant of this model
Verified signatureThis is an automated PR created with https://huggingface.co/spaces/safetensors/convert This new file is equivalent to `pytorch_model.bin` but safe in the sense that no arbitrary code can be put into it. These files also happen to load much faster than their pytorch counterpart: https://colab.research.google.com/github/huggingface/notebooks/blob/main/safetensors_doc/en/speed.ipynb The widgets on your model page will run using this model even if this is not merged making sure the file actually works. If you find any issues: please report here: https://huggingface.co/spaces/safetensors/convert/discussions Feel free to ignore this PR.5da0fa9
-
Migrate model card from transformers-repo
Read announcement at https://discuss.huggingface.co/t/announcement-all-model-cards-will-be-migrated-to-hf-co-model-repos/2755 Original file history: https://github.com/huggingface/transformers/commits/master/model_cards/facebook/rag-token-base/README.mdbdeb4ef
-
use full wiki_dpr instead of dummy
Quentin Lhoest798dbab
-
Update config.json
c448093 -
Update generator_tokenizer/merges.txt
90547c2 -
Update generator_tokenizer/special_tokens_map.json
4008b1c -
Update generator_tokenizer/tokenizer_config.json
9bcacda -
Update generator_tokenizer/vocab.json
b5e9cc9 -
Update pytorch_model.bin
e839965 -
Update question_encoder_tokenizer/special_tokens_map.json
50db4db -
Update question_encoder_tokenizer/tokenizer_config.json
ba6b80c -
Update question_encoder_tokenizer/vocab.txt
61f248f
-
Update config.json
1e04b21 -
Update generator_tokenizer/merges.txt
81599e7 -
Update generator_tokenizer/special_tokens_map.json
dc15bf1 -
Update generator_tokenizer/tokenizer_config.json
cb141f5 -
Update generator_tokenizer/vocab.json
dce4e29 -
Update question_encoder_tokenizer/special_tokens_map.json
ca75801 -
Update question_encoder_tokenizer/tokenizer_config.json
c520bb7 -
Update question_encoder_tokenizer/vocab.txt
df7cf2f -
Update config.json
0c8892c -
initial commit
71d837b