Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks
Paper • 1908.10084 • Published • 19
How to use SepKeyPro/bge-base-en-trivia-anchor-positive with sentence-transformers:
from sentence_transformers import SentenceTransformer
model = SentenceTransformer("SepKeyPro/bge-base-en-trivia-anchor-positive")
sentences = [
"Globe, Omaha Fiesole and Chianti are all varieties of which vegetable?",
"What is the Rugby Union Six Nations tournament? - CBBC Newsround What is the Rugby Union Six Nations tournament? 11 February 2015 Image copyright Getty Images Check out this guide to find out all you need to know about the Six Nations. Rugby Union Six Nations tournament The Six Nations is a rugby union tournament played every year between the top countries in Europe. The six countries who take part are England, Scotland, Wales, Ireland, France and Italy. For the first 90 years of the championships there were only five countries involved, but Italy were invited to take part in 2000. Rugby Union Rugby Union is played by teams of 15 players, with each team made up of eight forwards and seven backs. Even at the top level it used to be amateur; that is, played for fun by people who had other jobs too, but now the top players are all professionals. The biggest competition in Union is the World Cup, played every four years, but the most famous one in this country is the Six Nations championship. The Grand Slam If a team wins all five of its matches it is called a Grand Slam, but to win a Grand Slam is very hard. England won a Grand Slam in 2003, but only after losing their final match in the three seasons before. The Triple Crown The Triple Crown is a special prize that only the four home unions are able to win. Image copyright PA Image caption A team can only win the triple crown if they beat all three of the other home unions Rugby facts The sport gets its name from the place where it was invented, Rugby School in Warwickshire, England. In 1816 a pupil called William Webb Ellis got a bit bored during a match of football and decided that picking up the ball would make things more interesting. Although the game has come a long way since, even splitting into two codes; Rugby Union and Rugby League, that's where it started. One of the most important rules of the sport is that the ball can only be passed backwards.",
"Baked Artichoke Recipe Baked Artichoke Recipe written by Heather Restrepo So there are a variety of types when it comes to artichokes (Red-Babyanzio, Big-heart, Siena, Mercury, Omaha, Fiesole, Chianti, etc.) But perhaps the most common is the Classic Green Globe artichoke. This is probably the type of artichoke you will find at your local grocer. Having moved to Hawaii as a teen, that is when I was first introduced to this intimidating looking veggie. I would never have predicted that I could ever encompass the foodie-love-affair that I now have with them! The Classic Green Globe artichoke is by far my favorite because of its’ buttery-tasting heart and bottom. There is also a good amount of meat within the petals – SCORE if you ask me! Many people enjoy perfectly grilled, baked, or steamed artichokes at fancy restaurants, which are usually served with a garlic aioli type of sauce (find my aioli recipe here ). Nonetheless, making them at home is not as scary as you might think! I find that steaming artichokes gives them a bitter taste compared to baking, plus popping them in the oven is so much easier than messing with a steamer! So, here is my go-to recipe for BAKED artichokes: INGREDIENTS: 2 tbsp. Grass-fed Butter (un-salted) 1 tsp. Garlic Powder *NOTE: You will need a baking dish with a lid/cover. To start – Preheat the oven to 425 degrees. Give the artichoke a good rinse in cold water, and use a kitchen brush to lightly scrub the outside. Then cut about a half-inch from the stem and discard the bottom piece. I also cut about a half-inch from the very top, just to open it up and allow the seasoning to get inside for cooking. *OPTIONAL: Some people like to trim the tops of the leaves to get rid of the thorns, but I find that they become soft during cooking so I don’t bother. Then, carefully cut the artichoke vertically in half. Line a baking dish with parchment paper and place the halves on top. Then lightly coat both pieces with a halved lemon. Take the other half of the lemon and squeeze along the inside of the artichoke, as well as between the leaves. Then sprinkle the sea salt and garlic powder all-over both sides and a bit between the leaves. Next, lightly drizzle your oil all over the artichoke pieces (in between the leaves also). Then place a tbsp. of grass-fed butter in each of the heart pockets. Cover the baking dish with lid, and place in to the oven. Allow to cook until sizzling., usually about 1 hour. (Ovens will vary, so check often after 30-40 minutes of cooking) After cooking, allow to cool, remove the choke with a spoon (the hairy inside part), and then enjoy the meat on petals, and heart/bottom! x. Heather",
"Lady Lever Art Gallery - Gallery in Port Sunlight, Port Sunlight - Visit Liverpool You are here: Things To Do > Lady Lever Art Gallery Lady Lever Art Gallery Note: Prices are a guide only and may change on a daily basis. About The Lady Lever Art Gallery is regarded as one of the finest art galleries in Europe. It's located in model village, Port Sunlight in Wirral, a place rich in architectural charm. The gallery was founded by William Hesketh Lever (1851-1925) and is dedicated to his wife Elizabeth, Lady Lever. Lever wanted to share his collections with the public and the works on display at the gallery have been personally selected. Inside the gallery, visitors will find the best of Lever's personal art collection and the finest collection of Wedgewood jasperware anywhere in the world. The Pre-Raphaelite painting collection is internationally renowned and features works by Millais, Rossetti, Burne-Jones and Holman Hunt. For younger visitors, the activity rooms are an interactive space where they can get hands-on and have fun with crafts, dressing up and story telling. Lady Lever often houses temporary exhibitions and popular free events, be sure to check the website before visiting. Before leaving, browse the gift shop or enjoy a bite to eat in the Gallery Cafe. All Areas Accessible to Disabled Visitors Cafe/Restaurant Guided Tours Available for Groups Large Parties Map & Directions Road Directions From Liverpool: Go through the Birkenhead (Queensway) Tunnel (£1.40 toll for cars, £4.20 for coaches). Once you leave the tunnel follow signs for Port Sunlight, driving along A41(New Chester Road) - the gallery is sign posted all the way from the tunnel and situated opposite Port Sunlight Museum. From elsewhere: Leave the M53 at junction 4, follow the B5137 and take the second left onto the B5136 towards Port Sunlight. Follow the brown and white road signs for Port Sunlight Village. Once you are in the village follow the signs for Lady Lever Art Gallery. Public Transport Directions By Train: The nearest station is Bebington although Port Sunlight is also within walking distance. They are both on the Chester and Ellesmere Port Merseyrail lines. Leave the station and come out onto Old Chester Road (use the ramp if you require level access). Turn left, cross at the traffic lights, then turn left again down Bebington Road, passing under the railway bridge. Turn next right down Greendale Road. Continue along the pavement on the same side as the cottages for approximately 400 yards until you see the Leverhulme memorial and the Lady Lever Art Gallery on your left. Take the pathway on your left leading into Windy Bank and towards the memorial and the gallery. The entrance to the gallery is to the right side of the building opposite the fountain. By bus: Take number 464 to Bebington Road bus stop (starts at Sir Thomas Street in Liverpool city centre) or number 38 to Bebington rail station bus stop (runs between Clatterbridge Hospital and West Kirby station). Once you get off the bus refer to the above directions from Bebington railway station. TripAdvisor"
]
embeddings = model.encode(sentences)
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [4, 4]This is a sentence-transformers model finetuned from BAAI/bge-base-en-v1.5. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.
SentenceTransformer(
(0): Transformer({'max_seq_length': 512, 'do_lower_case': True}) with Transformer model: BertModel
(1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
(2): Normalize()
)
First install the Sentence Transformers library:
pip install -U sentence-transformers
Then you can load this model and run inference.
from sentence_transformers import SentenceTransformer
# Download from the 🤗 Hub
model = SentenceTransformer("SepKeyPro/bge-base-en-trivia-anchor-positive")
# Run inference
sentences = [
'The Azores island group is administered by which country?',
'The Azores Islands - Portugal | Portugal.com Porto and the North Azores Consisting of nine islands, the Azores \xa0are divided into three groups: the eastern ( Sao Miguel and Santa\xa0Maria islands), the central ( Terceira , Graciosa , Sao Jorge ,\xa0 Pico and Faial islands), and the western ( Flores and Corvo \xa0islands). Apart from international airports of Santa Maria,\xa0Ponta Delgada and Angra do Heroismo, there are flights to the islands\xa0(operated by the regional airline TAP Air Portugal) and ferry boats\xa0between the islands. Even the blase visitor will be touched by the sapphire blue and emerald green lakes, fertile prairies, volcanic cones and craters, colorful hydrangeas and azaleas, 15th century churches, and majestic manor houses. This legendary land, consisting of nine poetically-named islands, enjoys year-round mild temperatures (between 14°C and 22°C–57°F and 71°F) and is a peaceful shelter with a population of 250000 inhabitants, for whom the words “stress” and “pollution” are unheard. There are many stories to tell of the archipelago’s beauty, of fishermen or shepherds, but among them there is one which was told by a holidaymaker. As a foreign couple was silently looking at the Caldeira das Sete Cidades when they were interrupted by their six-year-old son, who asked them: “Is this God’s home?” Sao Miguel Island The largest of all. In Ponta Delgada, the capital, the famous 18th century portals open up to a number of monuments that are worth visiting, most of them built between the 16th and the 18th century: Carlos Machado Museum and churches of Sao Sebastiao, Sao Pedro, Sao Jose, Colegio and Nossa Senhora da Conceicao; convent and chapel of Nossa Senhora da Esperanca and Santa Ana Chapel. Palaces: Fonte Bela and Santa Ana; Conceicao and Santa Catarina; Casa de Carlos Bicudo and the Pacos do Concelho. Other places to visit: Caldeira das Sete Cidades (green and blue lakes); Lagoa do Fogo; Ribeira Grande; Vale das Furnas (spas and hot mineral pools) and Vila Franca do Campo.\xa0 Terceira Island The historic centre of its capital, Angra do Heroismo, has been classified in UNESCO’s International Heritage list. Special reference to the forts of Sao Sebastiao and Sao Joao Baptista (16th-17th-centuries); the palaces of the Bettencourts (Baroque) and of the Capitaes-Generais; the Cathedral, with its silver altar front and treasure; the churches of Colegio dos Jesuitas, Sao Goncalo and Nossa Senhora da Conceicao (17th-century); the churches of Misericordia and Nossa Senhora da Guia (18th-century, the latter encloses the Angra Museum). Other points of interest: Praia da Vitoria, Santa Barbara, Sao Sebastiao and Vila Nova. Graciosa Island In Santa Cruz da Graciosa you will find ancient streets and manor-houses, a beautiful mother-church (16th-18th centuries), Santo Cristo Church (16th century), Cruz da Barra (Manueline) and Ethnographic House. In the Furna do Enxofre, dazzling sights and a vaulted cave over an underground lake (between 11am and 2pm the sunlight filters in). You must also visit Guadalupe and its Baroque church, Luz and Praia (typical windmills). Faial Island In Horta, a famous yacht harbor, look at the beautiful tiles and gilded carvings in the 17th and 18th century churches of Sao Salvador, Nossa Senhora do Carmo and Sao Francisco. To visit: Sacred Art Museum, Nossa Senhora das Angústias Church, Nossa Senhora do Pilar Chapel, Imperio dos Nobres and Porto Pim fortifications, Caldeira Natural Reserve, Capelinhos, grottoes and caves in Costa da Feteira and Monte da Guia belvedere. Pico Island Owes its name to the 7713 ft high volcanic cone. Special reference to Sao Roque do Pico, with its 18th century churches of Sao Roque and Sao Pedro de Alcântara; Lajes do Pico, with its Whale Museum; Madalena, with its Wine Museum and 17th-century church, and Areia Larga, with beautiful winery manor houses. Other places: Calheta de Nesquim, Candelaria, Criacao Velha, Piedade (forest preserve), Prainha do Norte, Santa Luzia, Santo Amaro, Sao Caetano, Sao Joao and Sao Mateus. Sao Jorge Island Velas, with its fishing port, is the main to',
'Football - Summer Olympic Sport Football Singapore 2010 adopts new sport formats 12 Aug 2010 Football has its roots in ancient China, while the modern version of the game began on the streets of medieval England before evolving into the most popular sport in the world. Medieval origins Modern football has its origins in the streets of medieval England. Neighbouring towns would play each other in games where a heaving mass of players would struggle to drag a pig’s bladder by any means possible to markers at either end of town. A royal ban Football became so violent in England it was banned by the king for more than 300 years. English public schools are credited with subsequently establishing the modern football codes, thus turning the mob riot into a sport in the 16th century. Olympic history Football first appeared on the programme of the Games of the II Olympiad, Paris 1900. It has been on the programme of each edition of the Games ever since, with the exception of Los Angeles 1932. Europe dominated the competition until after 1992 in Barcelona, where Spain became the last European team to win a gold medal. Since the 1996 Olympic Games in Atlanta, African and South American teams have won all the gold medals. Also in 1996, women’s football was introduced into the Olympic programme. Three times, the USA has been on the highest step of the podium - in 1996, in 2004 in Athens and in 2008 in Beijing. But this team was beaten by the Norwegians in the final of the 2000 Games in Sydney.',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]
# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]
trivia-anchor-positive-devInformationRetrievalEvaluator| Metric | Value |
|---|---|
| cosine_accuracy@1 | 0.672 |
| cosine_accuracy@3 | 0.842 |
| cosine_accuracy@5 | 0.877 |
| cosine_accuracy@10 | 0.914 |
| cosine_precision@1 | 0.672 |
| cosine_precision@3 | 0.2807 |
| cosine_precision@5 | 0.1754 |
| cosine_precision@10 | 0.0914 |
| cosine_recall@1 | 0.672 |
| cosine_recall@3 | 0.842 |
| cosine_recall@5 | 0.877 |
| cosine_recall@10 | 0.914 |
| cosine_ndcg@10 | 0.8005 |
| cosine_mrr@10 | 0.7634 |
| cosine_map@100 | 0.7662 |
| dot_accuracy@1 | 0.672 |
| dot_accuracy@3 | 0.842 |
| dot_accuracy@5 | 0.877 |
| dot_accuracy@10 | 0.914 |
| dot_precision@1 | 0.672 |
| dot_precision@3 | 0.2807 |
| dot_precision@5 | 0.1754 |
| dot_precision@10 | 0.0914 |
| dot_recall@1 | 0.672 |
| dot_recall@3 | 0.842 |
| dot_recall@5 | 0.877 |
| dot_recall@10 | 0.914 |
| dot_ndcg@10 | 0.8005 |
| dot_mrr@10 | 0.7634 |
| dot_map@100 | 0.7662 |
eval_strategy: epochper_device_train_batch_size: 16per_device_eval_batch_size: 16gradient_accumulation_steps: 4learning_rate: 2e-05num_train_epochs: 1lr_scheduler_type: cosinewarmup_ratio: 0.1bf16: Truetf32: Trueoptim: adamw_torch_fusedbatch_sampler: no_duplicatesoverwrite_output_dir: Falsedo_predict: Falseeval_strategy: epochprediction_loss_only: Trueper_device_train_batch_size: 16per_device_eval_batch_size: 16per_gpu_train_batch_size: Noneper_gpu_eval_batch_size: Nonegradient_accumulation_steps: 4eval_accumulation_steps: Nonelearning_rate: 2e-05weight_decay: 0.0adam_beta1: 0.9adam_beta2: 0.999adam_epsilon: 1e-08max_grad_norm: 1.0num_train_epochs: 1max_steps: -1lr_scheduler_type: cosinelr_scheduler_kwargs: {}warmup_ratio: 0.1warmup_steps: 0log_level: passivelog_level_replica: warninglog_on_each_node: Truelogging_nan_inf_filter: Truesave_safetensors: Truesave_on_each_node: Falsesave_only_model: Falserestore_callback_states_from_checkpoint: Falseno_cuda: Falseuse_cpu: Falseuse_mps_device: Falseseed: 42data_seed: Nonejit_mode_eval: Falseuse_ipex: Falsebf16: Truefp16: Falsefp16_opt_level: O1half_precision_backend: autobf16_full_eval: Falsefp16_full_eval: Falsetf32: Truelocal_rank: 0ddp_backend: Nonetpu_num_cores: Nonetpu_metrics_debug: Falsedebug: []dataloader_drop_last: Falsedataloader_num_workers: 0dataloader_prefetch_factor: Nonepast_index: -1disable_tqdm: Falseremove_unused_columns: Truelabel_names: Noneload_best_model_at_end: Falseignore_data_skip: Falsefsdp: []fsdp_min_num_params: 0fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}fsdp_transformer_layer_cls_to_wrap: Noneaccelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}deepspeed: Nonelabel_smoothing_factor: 0.0optim: adamw_torch_fusedoptim_args: Noneadafactor: Falsegroup_by_length: Falselength_column_name: lengthddp_find_unused_parameters: Noneddp_bucket_cap_mb: Noneddp_broadcast_buffers: Falsedataloader_pin_memory: Truedataloader_persistent_workers: Falseskip_memory_metrics: Trueuse_legacy_prediction_loop: Falsepush_to_hub: Falseresume_from_checkpoint: Nonehub_model_id: Nonehub_strategy: every_savehub_private_repo: Falsehub_always_push: Falsegradient_checkpointing: Falsegradient_checkpointing_kwargs: Noneinclude_inputs_for_metrics: Falseeval_do_concat_batches: Truefp16_backend: autopush_to_hub_model_id: Nonepush_to_hub_organization: Nonemp_parameters: auto_find_batch_size: Falsefull_determinism: Falsetorchdynamo: Noneray_scope: lastddp_timeout: 1800torch_compile: Falsetorch_compile_backend: Nonetorch_compile_mode: Nonedispatch_batches: Nonesplit_batches: Noneinclude_tokens_per_second: Falseinclude_num_input_tokens_seen: Falseneftune_noise_alpha: Noneoptim_target_modules: Nonebatch_eval_metrics: Falseeval_on_start: Falsebatch_sampler: no_duplicatesmulti_dataset_batch_sampler: proportional| Epoch | Step | Training Loss | loss | trivia-anchor-positive-dev_cosine_map@100 |
|---|---|---|---|---|
| 0 | 0 | - | - | 0.7809 |
| 0.0710 | 10 | 0.1474 | - | - |
| 0.1421 | 20 | 0.1112 | - | - |
| 0.2131 | 30 | 0.0828 | - | - |
| 0.2842 | 40 | 0.0767 | - | - |
| 0.3552 | 50 | 0.0575 | - | - |
| 0.4263 | 60 | 0.0614 | - | - |
| 0.4973 | 70 | 0.0542 | - | - |
| 0.5684 | 80 | 0.0566 | - | - |
| 0.6394 | 90 | 0.068 | - | - |
| 0.7105 | 100 | 0.072 | - | - |
| 0.7815 | 110 | 0.0872 | - | - |
| 0.8526 | 120 | 0.0654 | - | - |
| 0.9236 | 130 | 0.0793 | - | - |
| 0.9947 | 140 | 0.0563 | - | - |
| 0.0710 | 10 | 0.0222 | - | - |
| 0.1421 | 20 | 0.0096 | - | - |
| 0.2131 | 30 | 0.0093 | - | - |
| 0.2842 | 40 | 0.0106 | - | - |
| 0.3552 | 50 | 0.0078 | - | - |
| 0.4263 | 60 | 0.0099 | - | - |
| 0.4973 | 70 | 0.01 | - | - |
| 0.5684 | 80 | 0.0166 | - | - |
| 0.6394 | 90 | 0.0272 | - | - |
| 0.7105 | 100 | 0.041 | - | - |
| 0.7815 | 110 | 0.0677 | - | - |
| 0.8526 | 120 | 0.0539 | - | - |
| 0.9236 | 130 | 0.074 | - | - |
| 0.9947 | 140 | 0.0484 | - | 0.7792 |
| 0.0710 | 10 | 0.0028 | - | - |
| 0.1421 | 20 | 0.0026 | - | - |
| 0.2131 | 30 | 0.0021 | - | - |
| 0.2842 | 40 | 0.0075 | - | - |
| 0.3552 | 50 | 0.0021 | - | - |
| 0.4263 | 60 | 0.0026 | - | - |
| 0.4973 | 70 | 0.0028 | - | - |
| 0.5684 | 80 | 0.004 | - | - |
| 0.6394 | 90 | 0.006 | - | - |
| 0.7105 | 100 | 0.0137 | - | - |
| 0.7815 | 110 | 0.0449 | - | - |
| 0.8526 | 120 | 0.0433 | - | - |
| 0.9236 | 130 | 0.0693 | - | - |
| 0.9947 | 140 | 0.0451 | 0.0405 | 0.7751 |
| 0.0710 | 10 | 0.0009 | - | - |
| 0.1421 | 20 | 0.0022 | - | - |
| 0.2131 | 30 | 0.0007 | - | - |
| 0.2842 | 40 | 0.001 | - | - |
| 0.3552 | 50 | 0.0009 | - | - |
| 0.4263 | 60 | 0.0009 | - | - |
| 0.4973 | 70 | 0.0011 | - | - |
| 0.5684 | 80 | 0.0015 | - | - |
| 0.6394 | 90 | 0.0019 | - | - |
| 0.7105 | 100 | 0.0037 | - | - |
| 0.7815 | 110 | 0.0229 | - | - |
| 0.8526 | 120 | 0.0318 | - | - |
| 0.9236 | 130 | 0.0661 | - | - |
| 0.9947 | 140 | 0.0451 | - | 0.7662 |
@inproceedings{reimers-2019-sentence-bert,
title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
author = "Reimers, Nils and Gurevych, Iryna",
booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
month = "11",
year = "2019",
publisher = "Association for Computational Linguistics",
url = "https://arxiv.org/abs/1908.10084",
}
@misc{henderson2017efficient,
title={Efficient Natural Language Response Suggestion for Smart Reply},
author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
year={2017},
eprint={1705.00652},
archivePrefix={arXiv},
primaryClass={cs.CL}
}
Base model
BAAI/bge-base-en-v1.5