Mice-P-130m

This model is a MICE (Minimal Interaction Cross-Encoder) model based on cross-encoder/ettin-reranker-150m-v1 (ModernBERT backbone).

MICE models separate query and document processing in the bottom layers, enabling independent document contextualization and faster re-ranking with cross-attention only in the top interaction layers.

Contents

• Model Description
• Usage
• Evaluations

Model Description

• Architecture: MICE v2 (Minimal Interaction Cross-Encoder)
• Base Model: cross-encoder/ettin-reranker-150m-v1
• Total Parameters: 128,564,737 (128.56M)
• Contextualization Layers: 12
• Interaction Layers: 4
• Training Strategy: MSE on mxbai-rerank-large-v2 (following the Ettin reranker Family recipe)
• Language: English

Usage

1. With sentence-transformers

from sentence_transformers import CrossEncoder

model = CrossEncoder("xpmir/Mice-P-130m", trust_remote_code=True)
scores = model.predict([
    ("What is the capital of France?", "Paris is the capital and most populous city of France.")
])
print(scores)

2. With Hugging Face transformers

import torch
from transformers import AutoModelForSequenceClassification, AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained("xpmir/Mice-P-130m")
model = AutoModelForSequenceClassification.from_pretrained("xpmir/Mice-P-130m", trust_remote_code=True)
model.eval()

inputs = tokenizer(["What is the capital of France?"], ["Paris is the capital of France."], return_tensors="pt")
with torch.no_grad():
    logits = model(**inputs).logits
print(logits)

3. With xpmir

import torch
from xpmir.neural.sentence_transformers import STCrossEncoder
from xpmir.letor.records import PointwiseItems

scorer = STCrossEncoder.C(model_id="xpmir/Mice-P-130m").instance()
scorer.initialize()
scorer.eval()

queries = ["What is the capital of France?"]
documents = ["Paris is the capital and most populous city of France."]
input_records = PointwiseItems.from_texts(topics=queries, documents=documents)
with torch.no_grad():
    output = scorer(input_records)
print(output)

Evaluations

We provide evaluations of this cross-encoder re-ranking top 100 of naver/splade-v3-distilbert:

dataset	Success@5	RR@10	nDCG@10
arguana_decontaminated	72.61	42.64	53.56
climate_fever_decontaminated	70.38	58.24	43.99
dbpedia_decontaminated	55.3	46.31	34.43
fever_decontaminated	97.98	95.17	92.69
fiqa_decontaminated	73.77	63.75	54.13
hotpotqa_decontaminated	83.03	80.19	79.59
nfcorpus_decontaminated	52.66	42.43	24.88
nq_decontaminated	100	96.92	97.64
quora_decontaminated	98.03	91.13	91.63
scidocs_decontaminated	12.94	8.81	9.44
scifact_decontaminated	90.24	84.88	86.51
touche_decontaminated	76.67	61.75	35.13
trec_covid_decontaminated	100	94.67	78.96
BEIR13 Decontaminated	75.66	66.68	60.2
mean	75.66	66.68	60.2