This is a Norwegian sentence embedding model trained using the SimCSE methodology (Gao et.al, SimCSE: Simple Contrastive Learning of Sentence Embeddings, EMNLP 2021). It is trained from the NB-BERT-large model (https://huggingface.co/NbAiLab/nb-bert-large/). It is trained supervised on the Norwegian automatically translated mnli dataset (https://huggingface.co/datasets/NbAiLab/mnli-norwegian).
The training and performance of this model is described in the paper "Training and Evaluating Norwegian Sentece Embedding Models", published at NoDaLiDa 2023 (https://openreview.net/forum?id=tcxy7vRVKlg). In that paper we describe training several different Norwegian sentence embedding models. This is the best performing model on STS data of those described in the paper.
- Downloads last month
- 6
# pip install -U transformers accelerate # Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("FFI/SimCSE-NB-BERT-large", device_map="auto")