Mice-P-61m

This model is a MICE (Minimal Interaction Cross-Encoder) model based on jhu-clsp/ettin-encoder-68m (ModernBERT backbone).

MICE models separate query and document processing in the bottom layers, enabling independent document contextualization and faster re-ranking with cross-attention only in the top interaction layers.

Contents

• Model Description
• Usage
• Evaluations

Model Description

• Architecture: MICE v2 (Minimal Interaction Cross-Encoder)
• Base Model: jhu-clsp/ettin-encoder-68m
• Total Parameters: 61,328,385 (61.33M)
• Contextualization Layers: 10
• Interaction Layers: 4
• Training Strategy: MSE on mxbai-rerank-large-v2 (following the Ettin reranker Family recipe)
• Language: English

Usage

1. With sentence-transformers

from sentence_transformers import CrossEncoder

model = CrossEncoder("xpmir/Mice-P-61m", trust_remote_code=True)
scores = model.predict([
    ("What is the capital of France?", "Paris is the capital and most populous city of France.")
])
print(scores)

2. With Hugging Face transformers

import torch
from transformers import AutoModelForSequenceClassification, AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained("xpmir/Mice-P-61m")
model = AutoModelForSequenceClassification.from_pretrained("xpmir/Mice-P-61m", trust_remote_code=True)
model.eval()

inputs = tokenizer(["What is the capital of France?"], ["Paris is the capital of France."], return_tensors="pt")
with torch.no_grad():
    logits = model(**inputs).logits
print(logits)

3. With xpmir

import torch
from xpmir.neural.sentence_transformers import STCrossEncoder
from xpmir.letor.records import PointwiseItems

scorer = STCrossEncoder.C(model_id="xpmir/Mice-P-61m").instance()
scorer.initialize()
scorer.eval()

queries = ["What is the capital of France?"]
documents = ["Paris is the capital and most populous city of France."]
input_records = PointwiseItems.from_texts(topics=queries, documents=documents)
with torch.no_grad():
    output = scorer(input_records)
print(output)

Evaluations

We provide evaluations of this cross-encoder re-ranking top 100 of naver/splade-v3-distilbert:

dataset	Success@5	RR@10	nDCG@10
arguana_decontaminated	70.65	37.88	49.83
climate_fever_decontaminated	69.56	57.82	43.98
dbpedia_decontaminated	55.3	44.52	33.63
fever_decontaminated	97.88	94.45	92.18
fiqa_decontaminated	73.77	62.15	52.45
hotpotqa_decontaminated	82.89	79.67	79.15
nfcorpus_decontaminated	52.66	42.85	24.65
nq_decontaminated	100	92.44	94.3
quora_decontaminated	97.78	90.69	91.18
scidocs_decontaminated	13.93	8.92	9.04
scifact_decontaminated	90.24	85.61	87.45
touche_decontaminated	83.33	63.26	36.32
trec_covid_decontaminated	100	91.67	77.2
BEIR13 Decontaminated	76	65.53	59.34
mean	76	65.53	59.34