|
Download README.md from Badawi-a/context-reranker: direct link, hf CLI and curl.
- Browser
- Download file 2.78 kB
-
https://huggingface.co/Badawi-a/context-reranker/resolve/main/README.md
- Command line
-
hf download hf://Badawi-a/context-reranker/README.md
-
curl -L -o README.md https://huggingface.co/Badawi-a/context-reranker/resolve/main/README.md
2.78 kB
| license: apache-2.0 | |
| language: | |
| - en | |
| base_model: cross-encoder/ms-marco-MiniLM-L-6-v2 | |
| pipeline_tag: text-ranking | |
| library_name: fastembed | |
| tags: | |
| - cross-encoder | |
| - reranker | |
| - onnx | |
| - int8 | |
| - distillation | |
| # Context reranker (MiniLM-L6, int8 ONNX) | |
| A small cross-encoder that scores how well a passage answers a question. In a | |
| retrieval-augmented system it orders the parts of the retrieved documents the | |
| answer model reads, so the passage with the answer gets into a limited context | |
| budget first. | |
| - **Base model:** [cross-encoder/ms-marco-MiniLM-L-6-v2](https://huggingface.co/cross-encoder/ms-marco-MiniLM-L-6-v2) (22M parameters) | |
| - **Teacher:** [BAAI/bge-reranker-v2-m3](https://huggingface.co/BAAI/bge-reranker-v2-m3) — strong, but seconds per question on CPU | |
| - **Training:** listwise distillation of the teacher's scores (KL divergence), 2 epochs, on about 9,000 | |
| English questions that a locally hosted LLM wrote from 3,000 passages of a public-finance document | |
| corpus. For each question, the teacher scored the passages of the top-10 sections the retrieval | |
| system returned for it. No evaluation questions were used for training. | |
| - **Format:** ONNX (exported with classic attention, opset 14), dynamic int8 quantization, 23 MB. | |
| About 65 ms to score 16 passages on a desktop CPU. | |
| ## Results | |
| | Test | Base MiniLM | This model | Teacher | | |
| |---|---|---|---| | |
| | In-domain: answer passage inside an 18,000-character context (475 questions; 396 with no reranker; this model scored by copies that never saw the question's document) | 398 | **409** | 415 | | |
| | SciFact, 100 questions, BM25 top-30 reranked (nDCG@10) | 0.723 | **0.757** | 0.793 | | |
| | 20 everyday questions, answer ranked first of 4 | 17/20 | **17/20** | 18/20 | | |
| ## Usage | |
| With [fastembed](https://github.com/qdrant/fastembed): | |
| ```python | |
| from huggingface_hub import snapshot_download | |
| from fastembed.rerank.cross_encoder import TextCrossEncoder | |
| from fastembed.common.model_description import ModelSource | |
| path = snapshot_download("Badawi-a/context-reranker", revision="v1") | |
| TextCrossEncoder.add_custom_model(model="local/context-reranker", sources=ModelSource(hf="local/context-reranker"), | |
| model_file="onnx/model.onnx") | |
| reranker = TextCrossEncoder("local/context-reranker", specific_model_path=path) | |
| scores = list(reranker.rerank("What is the VAT rate?", ["The VAT rate is 14 percent.", "It rained in Cairo."], batch_size=8)) | |
| ``` | |
| Higher scores mean more relevant. | |
| ## Limitations | |
| - English only. Questions in other languages should be translated first. | |
| - Trained for one domain (public finance); it generalizes (see SciFact), but less well than the teacher. | |
| - Like other small rerankers, it can prefer a passage that repeats the question's words over one that answers it. | |