s-m-quadri/ltr-bert-sir

LTR-BERT SIR

Pool-restricted bi-encoder for MS MARCO document ranking

Fine-tuned LTR-BERT-style bi-encoder and scoring code for pool-restricted hybrid document ranking on MS MARCO.

Authors: Syed Minnatullah Quadri, Vrishali A. Chakkarwar
Affiliation: Department of Computer Science and Engineering, Government Engineering College, Aurangabad, Maharashtra 431001, India
Contact: mt24f05f001@geca.ac.in, hi@s-m-quadri.me, vachakkarwar@geca.ac.in
Hub: s-m-quadri/ltr-bert-sir

Related resources

1. Bi-encoder weights, metrics, and offline LTRBertSIR inference (this repository) at s-m-quadri/ltr-bert-sir on Hugging Face
2. Interactive ELSIE demo and REST API at s-m-quadri/sir-elsie on Hugging Face Spaces
3. JavaScript and TypeScript client at @s-m-quadri/ltr-bert-sir-client on npm
4. Python client at ltr-bert-sir-client on PyPI
5. TypeScript source at ltr-bert-sir-client-js on GitHub
6. Python source at ltr-bert-sir-client-py on GitHub
7. Self-hosted ELSIE API at smquadri/ltr-bert-sir-elsie on Docker Hub
8. Next.js frontend at sir-elsie on GitHub

Try the demo

You do not need a local install to run the full hybrid pipeline on sample corpora.

1. Open sir-elsie.
2. Wait for boot indexing of Alice in Wonderland (default seed) or choose another title from the library catalog.
3. Enter a query, select Hybrid (blend) or Precise (ce_cascade), and inspect ranked passages, timing, and optional evaluation when qrels are loaded.

The Space downloads this checkpoint on first use (~1.3 GB). CPU hardware works; GPU tiers reduce encode and cross-encoder latency.

HTTP clients

Programmatic access to the same API as the Space UI:

Package	Install
JavaScript / TypeScript	npm install @s-m-quadri/ltr-bert-sir-client
Python	pip install ltr-bert-sir-client

Default API origin: https://huggingface.co/spaces/s-m-quadri/sir-elsie. Use LTRBertSIR in this repository for offline bi-encoder scoring without the Space.

Limitations

• Dense scores are pool-restricted: the bi-encoder ranks within a BM25 (or fused) candidate pool, not the full corpus.
• This repository ships inference for the LTR-BERT stage; full MS MARCO reproduction requires external BM25 indices, qrels, and the scripts referenced in the associated publications.
• Cross-encoder reranking in ELSIE uses cross-encoder/ms-marco-MiniLM-L-6-v2 on a short shortlist; it is not bundled as weights in this model repo.

Quick start (notebook)

pip install torch transformers huggingface_hub

import sys
from huggingface_hub import snapshot_download

root = snapshot_download("s-m-quadri/ltr-bert-sir")
sys.path.insert(0, root)

from inference import LTRBertSIR

ranker = LTRBertSIR.from_pretrained(root)

query = "bm25 lexical ranking information retrieval"
docs = [
    "BM25 is a bag-of-words ranking function using term frequency and inverse document frequency.",
    "The weather tomorrow will be sunny with light winds.",
]

print(ranker.score(query, docs[0]))
# 0.8685

print(ranker.rank(query, docs))
# [
#   ("BM25 is a bag-of-words ranking function ...", 0.8685),
#   ("The weather tomorrow will be sunny ...", 0.7953),
# ]

Input	query string + passage text (or a list for .rank)
Output	float, or [(text, score), ...] sorted high to low
Load	LTRBertSIR.from_pretrained(...) (Hub id or snapshot_download path)
Long docs	ranker.score(query, text, document=True)

Scores are cosine-style (about [-1, 1]). Rank by relative order inside a candidate list. Paper pipeline uses BM25 first. First run downloads ~1.3 GB (cached). CPU works.

Associated publications

Publication details and bibliographies are listed here on the Hub model card only.

Syed Minnatullah Quadri and Vrishali A. Chakkarwar.
Empirical Analysis of Score Fusion Strategies under Pool-Restricted Dense Encoding for Ad Hoc Document Retrieval.
MSW Management Journal, article 4038.

Syed Minnatullah Quadri and Vrishali A. Chakkarwar.
Lexical First Stage, Pool-Restricted Dense Scores, and Cross-Encoder Reranking on MS MARCO Document Development.
Extended manuscript (in preparation): RRF3 prior, MiniLM CE cascade, out-of-fold fusion; source of the full metrics table below.

System overview

MS MARCO document development (5,193 topics, official qrels):

1. First stage: PySerini BM25@100 (k1=4.46, b=0.82)
2. Bi-encoder: this checkpoint; dense scores only inside each BM25@100 pool
3. Fusion: min-max linear blend with alpha = 0.85 (BM25 weight); RRF / stratified / OOF in the extended manuscript
4. Late stage: MiniLM (cross-encoder/ms-marco-MiniLM-L-6-v2) on top-32 of the RRF3 prior

This Hub repo ships the fine-tuned bi-encoder and scoring code.

Model

Item	Value
Backbone	bert-base-uncased
Compression	Linear 768 to 24
Special tokens	[Q], [D]
Query length	50 (expanded + MASK pad)
Passage segment length	150 content tokens
Document score	max over segments (presentation matching)
Training	MS MARCO passage triples; BCE; AdamW 2e-5; 3 epochs
Checkpoint	checkpoints/ltr_bert_final.pt (~1.3 GB)

Architecture follows Wang et al. (LTR-BERT), with pool-restricted encoding as in the associated papers.

Results (MS MARCO document development)

Macro-averages over 5,193 topics on the BM25@100 pool. Extended manuscript table:

Method	nDCG@10	nDCG@100	RR@10	AP@100	P@10
BM25	0.3155	0.3773	0.2565	0.2690	0.0506
LTR alpha-blend (alpha=0.85)	0.3197	0.3793	0.2587	0.2711	0.0515
First-stage RRF3	0.3026	0.3629	0.2391	0.2507	0.0509
Learned fusion (OOF)	0.3859	0.4275	0.3200	0.3280	0.0596
CE cascade (MiniLM, top-32 on RRF3 prior)	0.4110	0.4514	0.3544	0.3610	0.0590

CE vs BM25 on nDCG@10: mean paired difference 0.09541; 95% bootstrap CI [0.08625, 0.10470] (5,193 topics, 4,000 resamples).

Corpus: MS MARCO document ranking v1 (3,213,835 documents). Training: passage triples.train.small. Microsoft MS MARCO. Corpus files are not in this repo.

Intended use

• Bi-encoder stage for the associated publications
• Pool-restricted hybrid reranking: BM25 (or other first stage) then this model, optional fusion / CE
• Query-passage scoring via LTRBertSIR

Citation

@article{quadri2025poolrestricted,
  title   = {Empirical Analysis of Score Fusion Strategies under Pool-Restricted Dense Encoding for Ad Hoc Document Retrieval},
  author  = {Quadri, Syed Minnatullah and Chakkarwar, Vrishali A.},
  journal = {MSW Management Journal},
  url     = {https://mswmanagementj.com/index.php/home/article/view/4038}
}

@article{wang2024ltrbert,
  author  = {Wang, Junmei and Huang, Jimmy X. and Sheng, Jinhua},
  title   = {An Efficient Long-Text Semantic Retrieval Approach via Utilizing Presentation Learning on Short-Text},
  journal = {Complex \& Intelligent Systems},
  volume  = {10},
  pages   = {963--979},
  year    = {2024},
  doi     = {10.1007/s40747-023-01192-3}
}

Cite Bajaj et al. (MS MARCO) when using the dataset.

License

Apache-2.0 for weights and code here. MS MARCO remains under Microsoft research terms.