Instructions to use nebhailema/afrixir-multivector-mini with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- sentence-transformers
How to use nebhailema/afrixir-multivector-mini with sentence-transformers:
from sentence_transformers import MultiVectorEncoder model = MultiVectorEncoder("nebhailema/afrixir-multivector-mini") queries = ["Which planet is known as the Red Planet?"] documents = [ "Venus is often called Earth's twin because of its similar size and proximity.", "Mars, known for its reddish appearance, is often referred to as the Red Planet.", "Jupiter, the largest planet in our solar system, has a prominent red spot.", ] query_embeddings = model.encode_query(queries) document_embeddings = model.encode_document(documents) similarities = model.similarity(query_embeddings, document_embeddings) print(similarities) - Notebooks
- Google Colab
- Kaggle
AfriXIR multi-vector retriever
This is a Multi-Vector Encoder model finetuned from Davlan/afro-xlmr-mini on the AfriXIR dataset using the sentence-transformers library. It maps inputs to sequences of 128-dimensional token-level vectors and scores them with late interaction (MaxSim), useful for passage retrieval.
Model Details
Model Description
- Model Type: Multi-Vector Encoder
- Base model: Davlan/afro-xlmr-mini
- Maximum Sequence Length: 180 tokens
- Maximum Query Length: 32 tokens
- Maximum Document Length: 180 tokens
- Output Dimensionality: 128 dimensions
- Similarity Function: MaxSim
- Supported Modality: Text
- Training Dataset:
- Languages: am, ha, om, sn, so, sw, yo, zu
- License: apache-2.0
Model Sources
- Documentation: Sentence Transformers Documentation
- Documentation: Multi-Vector Encoder Documentation
- Repository: Sentence Transformers on GitHub
- Hugging Face: Multi-Vector Encoders on Hugging Face
Full Model Architecture
MultiVectorEncoder(
(0): Transformer({'transformer_task': 'feature-extraction', 'modality_config': {'text': {'method': 'forward', 'method_output_name': 'last_hidden_state'}}, 'module_output_name': 'token_embeddings', 'query_length': 32, 'document_length': 180, 'architecture': 'XLMRobertaModel'})
(1): Dense({'in_features': 384, 'out_features': 128, 'bias': False, 'activation_function': 'torch.nn.modules.linear.Identity', 'module_input_name': 'token_embeddings', 'module_output_name': 'token_embeddings'})
(2): MultiVectorMask({'skiplist_words': ['!', '"', '#', '$', '%', '&', "'", '(', ')', '*', '+', ',', '-', '.', '/', ':', ';', '<', '=', '>', '?', '@', '[', '\\', ']', '^', '_', '`', '{', '|', '}', '~'], 'skiplist_tasks': ['document'], 'keep_only_token_ids': None})
(3): Normalize({'module_input_name': 'token_embeddings', 'module_output_name': 'token_embeddings'})
)
Usage
Direct Usage (Sentence Transformers)
First install the Sentence Transformers library:
pip install -U sentence-transformers
Then you can load this model and run inference.
from sentence_transformers import MultiVectorEncoder
# Download from the 🤗 Hub
model = MultiVectorEncoder("nebhailema/afrixir-multivector-mini")
# Run inference: each input becomes a sequence of per-token vectors (variable length).
queries = [
'የኪርጊዝ ሪፐብሊክ እግር ኳስ ፌዴሬሽን',
]
documents = [
'የኪርጊዝ ሪፐብሊክ እግር ኳስ ፌዴሬሽን የኪርጊዝስታን እግር ኳስ አስተዳዳሪ አካል ነው። ፌዴሬሽኑ የኪርጊዝስታን ብሔራዊ እግር ኳስ ቡድንንና የኪርጊዝስታን ብሔራዊ ፉትሳል ቡድንን ይቆጣጠራል። በተጨማሪም የተለያዩ ብሔራዊ ውድድሮችን ያዘጋጃል።\n\nየውጭ መያያዣዎች\n ይፋዊ ድረ ገጽ\n የፊፋ ድረ ገጽ ስለ ኪርጊዝስታን \n\nኪ\nኪርጊዝስታን',
'የኮንጎ ዲሞክራስያዊ ሪፑብሊክ ሞሴ ካቱምቢ (Moise Katumbi) የተባሉ ተቃዋሚ ፖለቲከኛ ባዕዳን ቅጥረኞችን ተጠቅመዋል በሚለው ወሬ ላይ ምርመራ እንደጀመረች የመንግስት ባለስልጣኖች ትላንት ረቡዕ ተናግረዋል።የፍትህ ሚኒስትር አሌክስ ታምብዌ ምዋምባ በርካታ የቀድሞ አሜሪካውያን ወታደሮች በካታንጋ ክፍለ-ሀገር የካቱምቢ ጠባቂዎች ሆነው እንየሰሩ መሆናቸውን የሚያሳይ “የሰነድ መረጃ” አለን ብለዋል።ካታንጋ የተባለችው የነሃስ ማዕዳን ያላት ክፍለ-ሀገር አስተዳዳሪ ሆነው ሰርተው የነበሩት ካቱምቢ ባለፈው መስከረም ወር ከፕረዚዳንት ጆሴፍ ካቢላ መንግስት ወጥተው ከተቃዋሚዎች ጋር ተቀላቅለዋል።አንድ ተቃዋሚ ቡድን ካቱምቢ በያዝነው አመት በሚካሄደው ፕረዚዳንታዊ ምርጫ እንዲወዳደሩ የሚጠይቅ የስም ዝርዝር ባለፈው እሁድ አቅርቦ ካቱምቢ ተቀብለውታል።ተቃዋሚው ፖለቲከኛ ባዕዳን ቅጥረኞችን እየመለመሉ ነው የሚለውን ክስ አስተባብለዋል። እኔን በፖለቲካ ለመጉዳት የታቀደ ክስ ነው ብለዋል።',
'ሱክሬ በይፋ የቦሊቪያ ዋና ከተማ ነው።\n\nበ1890 ዓ.ም. የቦሊቪያ መንግሥት መቀመጫ በተግባር ከሱክሬ ወደ ላፓዝ ተዛወረ። ይሁንና በሕጋዊ ረገድ (በስም ብቻ) ሱክሬ እስከ ዛሬ ድረስ ዋና ከተማነቱን ይዟል። ላይኛ ችሎቱ ግን እስካሁን በሱክሬ ይገኛል። \n\nቦሊቪያ\nዋና ከተሞች',
]
query_embeddings = model.encode_query(queries)
document_embeddings = model.encode_document(documents)
print(query_embeddings[0].shape, document_embeddings[0].shape)
# (16, 128) (106, 128)
# Get the MaxSim similarity scores
similarities = model.similarity(query_embeddings, document_embeddings)
print(similarities)
# tensor([[14.9542, 4.3677, 4.6184]])
Evaluation
Full-corpus passage retrieval on AfriXIR for Amharic, Hausa, Oromo, Shona, Somali, Swahili, Tigrinya, and Yoruba. avg is the query-weighted average across those languages. The best value in that column is bold.
Reranker rows rerank the top 50 passages from a frozen dense retriever.
Multivector
Multivector retrievers compared with each other on recall@5, recall@10, nDCG@10, and MRR@10.
recall@5
| model | amharic | hausa | oromo | shona | somali | swahili | tigrinya | yoruba | avg |
|---|---|---|---|---|---|---|---|---|---|
| multivector mini | 0.8710 | 0.8177 | 0.8126 | 0.8162 | 0.8358 | 0.8311 | 0.9388 | 0.7841 | 0.8471 |
| multivector e5-small | 0.8868 | 0.8204 | 0.7832 | 0.8108 | 0.8515 | 0.8662 | 0.8980 | 0.7691 | 0.8578 |
| multivector e5-base | 0.9034 | 0.8718 | 0.8246 | 0.8432 | 0.8695 | 0.8719 | 0.9388 | 0.7922 | 0.8789 |
| M3 multi | 0.8485 | 0.7960 | 0.7919 | 0.8649 | 0.8211 | 0.8260 | 0.9184 | 0.7108 | 0.8246 |
recall@10
| model | amharic | hausa | oromo | shona | somali | swahili | tigrinya | yoruba | avg |
|---|---|---|---|---|---|---|---|---|---|
| multivector mini | 0.9008 | 0.8809 | 0.8725 | 0.8541 | 0.8898 | 0.8802 | 0.9388 | 0.8323 | 0.8877 |
| multivector e5-small | 0.9183 | 0.8755 | 0.8573 | 0.8757 | 0.9033 | 0.9146 | 0.9592 | 0.8102 | 0.8995 |
| multivector e5-base | 0.9300 | 0.9206 | 0.8758 | 0.8649 | 0.9123 | 0.9152 | 0.9388 | 0.8394 | 0.9140 |
| M3 multi | 0.8896 | 0.8484 | 0.8529 | 0.8865 | 0.8763 | 0.8878 | 0.9388 | 0.7751 | 0.8732 |
ndcg@10
| model | amharic | hausa | oromo | shona | somali | swahili | tigrinya | yoruba | avg |
|---|---|---|---|---|---|---|---|---|---|
| multivector mini | 0.8028 | 0.7163 | 0.7364 | 0.7599 | 0.7551 | 0.7352 | 0.8785 | 0.6876 | 0.7690 |
| multivector e5-small | 0.8272 | 0.7280 | 0.7167 | 0.7688 | 0.7617 | 0.7801 | 0.8876 | 0.6858 | 0.7880 |
| multivector e5-base | 0.8428 | 0.7853 | 0.7429 | 0.7801 | 0.7867 | 0.7989 | 0.8569 | 0.7137 | 0.8098 |
| M3 multi | 0.7690 | 0.6967 | 0.7113 | 0.7824 | 0.7315 | 0.7365 | 0.8879 | 0.6265 | 0.7410 |
mrr@10
| model | amharic | hausa | oromo | shona | somali | swahili | tigrinya | yoruba | avg |
|---|---|---|---|---|---|---|---|---|---|
| multivector mini | 0.7709 | 0.6631 | 0.6930 | 0.7291 | 0.7118 | 0.6884 | 0.8571 | 0.6407 | 0.7305 |
| multivector e5-small | 0.7977 | 0.6803 | 0.6723 | 0.7352 | 0.7160 | 0.7366 | 0.8652 | 0.6452 | 0.7520 |
| multivector e5-base | 0.8145 | 0.7413 | 0.7000 | 0.7524 | 0.7459 | 0.7612 | 0.8296 | 0.6731 | 0.7760 |
| M3 multi | 0.7296 | 0.6476 | 0.6660 | 0.7481 | 0.6850 | 0.6880 | 0.8716 | 0.5789 | 0.6982 |
All models
Dense, sparse, multivector, and reranker models compared with each other on recall@5, recall@10, nDCG@10, and MRR@10.
recall@5
| model | amharic | hausa | oromo | shona | somali | swahili | tigrinya | yoruba | avg |
|---|---|---|---|---|---|---|---|---|---|
| dense mini | 0.7805 | 0.6940 | 0.5643 | 0.6270 | 0.6637 | 0.7081 | 0.7755 | 0.6024 | 0.7233 |
| e5-small | 0.8741 | 0.8014 | 0.7418 | 0.8162 | 0.8088 | 0.8451 | 0.8571 | 0.7359 | 0.8379 |
| e5-base | 0.8886 | 0.8691 | 0.7810 | 0.8000 | 0.8526 | 0.8738 | 0.8571 | 0.7590 | 0.8629 |
| e5-large | 0.8948 | 0.8809 | 0.7974 | 0.8324 | 0.8841 | 0.8993 | 0.7755 | 0.7550 | 0.8738 |
| M3 dense | 0.7067 | 0.6264 | 0.4630 | 0.5892 | 0.5894 | 0.6488 | 0.7551 | 0.5231 | 0.6501 |
| SPLADE mini | 0.7759 | 0.7446 | 0.7603 | 0.7405 | 0.7705 | 0.7546 | 0.8163 | 0.6657 | 0.7599 |
| M3 sparse | 0.7826 | 0.7356 | 0.7593 | 0.7189 | 0.7627 | 0.7463 | 0.8367 | 0.6536 | 0.7598 |
| multivector mini | 0.8710 | 0.8177 | 0.8126 | 0.8162 | 0.8358 | 0.8311 | 0.9388 | 0.7841 | 0.8471 |
| multivector e5-small | 0.8868 | 0.8204 | 0.7832 | 0.8108 | 0.8515 | 0.8662 | 0.8980 | 0.7691 | 0.8578 |
| multivector e5-base | 0.9034 | 0.8718 | 0.8246 | 0.8432 | 0.8695 | 0.8719 | 0.9388 | 0.7922 | 0.8789 |
| M3 multi | 0.8485 | 0.7960 | 0.7919 | 0.8649 | 0.8211 | 0.8260 | 0.9184 | 0.7108 | 0.8246 |
| reranker mini | 0.8647 | 0.8159 | 0.7462 | 0.7946 | 0.8245 | 0.8305 | 0.9184 | 0.7359 | 0.8336 |
| reranker e5-small | 0.8907 | 0.8430 | 0.8279 | 0.8541 | 0.8673 | 0.8693 | 0.9592 | 0.7871 | 0.8690 |
| reranker e5-base | 0.9082 | 0.8727 | 0.8497 | 0.8486 | 0.8943 | 0.8725 | 0.8980 | 0.7932 | 0.8853 |
recall@10
| model | amharic | hausa | oromo | shona | somali | swahili | tigrinya | yoruba | avg |
|---|---|---|---|---|---|---|---|---|---|
| dense mini | 0.8285 | 0.7644 | 0.6362 | 0.6973 | 0.7334 | 0.7827 | 0.8163 | 0.6737 | 0.7820 |
| e5-small | 0.9072 | 0.8718 | 0.8007 | 0.8811 | 0.8774 | 0.9012 | 0.8980 | 0.7982 | 0.8844 |
| e5-base | 0.9209 | 0.9070 | 0.8410 | 0.8649 | 0.9224 | 0.9133 | 0.8571 | 0.8233 | 0.9042 |
| e5-large | 0.9273 | 0.9242 | 0.8551 | 0.8811 | 0.9359 | 0.9344 | 0.8163 | 0.8143 | 0.9131 |
| M3 dense | 0.7584 | 0.6931 | 0.5381 | 0.6595 | 0.6704 | 0.7208 | 0.7755 | 0.5944 | 0.7112 |
| SPLADE mini | 0.8293 | 0.7924 | 0.8115 | 0.8162 | 0.8425 | 0.8317 | 0.8776 | 0.7440 | 0.8192 |
| M3 sparse | 0.8279 | 0.8087 | 0.8137 | 0.8000 | 0.8256 | 0.8107 | 0.8776 | 0.7169 | 0.8138 |
| multivector mini | 0.9008 | 0.8809 | 0.8725 | 0.8541 | 0.8898 | 0.8802 | 0.9388 | 0.8323 | 0.8877 |
| multivector e5-small | 0.9183 | 0.8755 | 0.8573 | 0.8757 | 0.9033 | 0.9146 | 0.9592 | 0.8102 | 0.8995 |
| multivector e5-base | 0.9300 | 0.9206 | 0.8758 | 0.8649 | 0.9123 | 0.9152 | 0.9388 | 0.8394 | 0.9140 |
| M3 multi | 0.8896 | 0.8484 | 0.8529 | 0.8865 | 0.8763 | 0.8878 | 0.9388 | 0.7751 | 0.8732 |
| reranker mini | 0.8872 | 0.8547 | 0.7800 | 0.8324 | 0.8605 | 0.8662 | 0.9592 | 0.7620 | 0.8615 |
| reranker e5-small | 0.9206 | 0.8917 | 0.8704 | 0.8811 | 0.9168 | 0.9191 | 0.9592 | 0.8414 | 0.9072 |
| reranker e5-base | 0.9345 | 0.9224 | 0.8987 | 0.8919 | 0.9314 | 0.9344 | 0.9592 | 0.8454 | 0.9229 |
ndcg@10
| model | amharic | hausa | oromo | shona | somali | swahili | tigrinya | yoruba | avg |
|---|---|---|---|---|---|---|---|---|---|
| dense mini | 0.7129 | 0.5933 | 0.4881 | 0.5533 | 0.5709 | 0.6234 | 0.6532 | 0.5160 | 0.6464 |
| e5-small | 0.8086 | 0.7077 | 0.6464 | 0.7233 | 0.7270 | 0.7653 | 0.7810 | 0.6609 | 0.7635 |
| e5-base | 0.8264 | 0.7661 | 0.6851 | 0.7362 | 0.7691 | 0.7929 | 0.7583 | 0.6855 | 0.7897 |
| e5-large | 0.8294 | 0.7962 | 0.7089 | 0.7499 | 0.8061 | 0.8147 | 0.7093 | 0.6677 | 0.7997 |
| M3 dense | 0.6340 | 0.5278 | 0.4034 | 0.5053 | 0.5045 | 0.5699 | 0.6650 | 0.4406 | 0.5734 |
| SPLADE mini | 0.6816 | 0.6213 | 0.6629 | 0.6763 | 0.6768 | 0.6579 | 0.7696 | 0.5729 | 0.6633 |
| M3 sparse | 0.6987 | 0.6469 | 0.6776 | 0.6599 | 0.6835 | 0.6593 | 0.7754 | 0.5562 | 0.6750 |
| multivector mini | 0.8028 | 0.7163 | 0.7364 | 0.7599 | 0.7551 | 0.7352 | 0.8785 | 0.6876 | 0.7690 |
| multivector e5-small | 0.8272 | 0.7280 | 0.7167 | 0.7688 | 0.7617 | 0.7801 | 0.8876 | 0.6858 | 0.7880 |
| multivector e5-base | 0.8428 | 0.7853 | 0.7429 | 0.7801 | 0.7867 | 0.7989 | 0.8569 | 0.7137 | 0.8098 |
| M3 multi | 0.7690 | 0.6967 | 0.7113 | 0.7824 | 0.7315 | 0.7365 | 0.8879 | 0.6265 | 0.7410 |
| reranker mini | 0.8082 | 0.7483 | 0.6906 | 0.7416 | 0.7613 | 0.7523 | 0.8987 | 0.6671 | 0.7722 |
| reranker e5-small | 0.8284 | 0.7397 | 0.7435 | 0.7852 | 0.7814 | 0.7815 | 0.9037 | 0.6999 | 0.7946 |
| reranker e5-base | 0.8449 | 0.7681 | 0.7668 | 0.7903 | 0.7968 | 0.7947 | 0.8769 | 0.7018 | 0.8107 |
mrr@10
| model | amharic | hausa | oromo | shona | somali | swahili | tigrinya | yoruba | avg |
|---|---|---|---|---|---|---|---|---|---|
| dense mini | 0.6759 | 0.5387 | 0.4412 | 0.5077 | 0.5192 | 0.5729 | 0.6004 | 0.4658 | 0.6032 |
| e5-small | 0.7766 | 0.6548 | 0.5970 | 0.6736 | 0.6790 | 0.7214 | 0.7437 | 0.6169 | 0.7246 |
| e5-base | 0.7957 | 0.7201 | 0.6350 | 0.6957 | 0.7202 | 0.7534 | 0.7245 | 0.6418 | 0.7527 |
| e5-large | 0.7976 | 0.7544 | 0.6620 | 0.7077 | 0.7643 | 0.7756 | 0.6741 | 0.6208 | 0.7629 |
| M3 dense | 0.5941 | 0.4746 | 0.3609 | 0.4563 | 0.4518 | 0.5221 | 0.6291 | 0.3917 | 0.5295 |
| SPLADE mini | 0.6338 | 0.5656 | 0.6148 | 0.6319 | 0.6237 | 0.6027 | 0.7358 | 0.5185 | 0.6130 |
| M3 sparse | 0.6568 | 0.5950 | 0.6337 | 0.6157 | 0.6376 | 0.6108 | 0.7417 | 0.5047 | 0.6302 |
| multivector mini | 0.7709 | 0.6631 | 0.6930 | 0.7291 | 0.7118 | 0.6884 | 0.8571 | 0.6407 | 0.7305 |
| multivector e5-small | 0.7977 | 0.6803 | 0.6723 | 0.7352 | 0.7160 | 0.7366 | 0.8652 | 0.6452 | 0.7520 |
| multivector e5-base | 0.8145 | 0.7413 | 0.7000 | 0.7524 | 0.7459 | 0.7612 | 0.8296 | 0.6731 | 0.7760 |
| M3 multi | 0.7296 | 0.6476 | 0.6660 | 0.7481 | 0.6850 | 0.6880 | 0.8716 | 0.5789 | 0.6982 |
| reranker mini | 0.7824 | 0.7139 | 0.6618 | 0.7122 | 0.7296 | 0.7152 | 0.8793 | 0.6361 | 0.7431 |
| reranker e5-small | 0.7984 | 0.6900 | 0.7023 | 0.7539 | 0.7379 | 0.7369 | 0.8850 | 0.6541 | 0.7580 |
| reranker e5-base | 0.8156 | 0.7178 | 0.7240 | 0.7572 | 0.7527 | 0.7498 | 0.8509 | 0.6553 | 0.7741 |
Training Details
Training Dataset
AfriXIR
- Dataset: AfriXIR
- Size: 112,864 training samples
- Columns:
anchor,positive,negative_0,negative_1,negative_2, andnegative_3
Evaluation Dataset
AfriXIR
- Dataset: AfriXIR
- Size: 12,546 evaluation samples
- Columns:
anchor,positive,negative_0,negative_1,negative_2, andnegative_3
Citation
BibTeX
Sentence Transformers
@inproceedings{reimers-2019-sentence-bert,
title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
author = "Reimers, Nils and Gurevych, Iryna",
booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
month = "11",
year = "2019",
publisher = "Association for Computational Linguistics",
url = "https://arxiv.org/abs/1908.10084",
}
CachedMultiVectorMultipleNegativesRankingLoss
@misc{gao2021scaling,
title={Scaling Deep Contrastive Learning Batch Size under Memory Limited Setup},
author={Luyu Gao and Yunyi Zhang and Jiawei Han and Jamie Callan},
year={2021},
eprint={2101.06983},
archivePrefix={arXiv},
primaryClass={cs.LG}
}
- Downloads last month
- 1
Model tree for nebhailema/afrixir-multivector-mini
Base model
Davlan/afro-xlmr-mini