Q-P1NG/multilingual-reranker-small-LiteRT

multilingual-reranker-small v1.2 — LiteRT (int8 dynamic)

A LiteRT model of Horizon-Labs/multilingual-reranker-small (v1.2, commit 23139fec6f1f448cdf89a5ac2d2c6bf5e8c7b398): a multilingual cross-encoder (141M, mmBERT-small) that scores how well a passage answers a question. Q-P1NG uses it on the phone's CPU to order the passages its archive finds, in place of the GGUF on llama.cpp.

File	Size	SHA-256
multilingual-reranker-small-v1.2-L128-L512-int8dyn.tflite (two signatures, recommended)	152,184,768 bytes	e062475fcf329e910ab360ceadb872dc5c069b29eb8fbbc9647b73ce3b0a0716
multilingual-reranker-small-v1.2-L512-int8dyn.tflite (512 tokens only)	146,999,712 bytes	1c2e5b70d7c3aa75b87e0e692919bc2f83b5ad569c891748aafa2f2f5a063ebb
multilingual-reranker-small-v1.2-vocab.gguf	14,866,112 bytes	b6682a2bddab4974b0a6d52acf6f6f976f89225b4fa125ef1ab76ccdd19ebd90

Use

LiteRT 2.3 (CompiledModel, CPU). Inputs: input_ids and attention_mask, int32; output: the pair's score, a logit (higher: more relevant; sigmoid gives 0-1). The pair is [BOS] question [EOS] passage [EOS], the passage cut to fit, padded with id 0 (masked out).

The two-signature model has a signature per length, L128 ([1, 128]) and L512 ([1, 512]), over one copy of the weights: a pair that fits in 128 tokens (a sentence) runs in L128, with the same tokens and so the same score, at a fraction of the work of L512 (a page). The other model has the 512-token shape only.

The vocabulary file is a GGUF without tensors (the metadata of the Q8_0 GGUF): llama.cpp loads it with vocab_only and tokenizes the pairs exactly as the original tokenizer does, so no Python or Rust tokenizer is needed on the device.

Measured

38 Italian and English questions about two PDFs, 10 candidate passages each (the passage holding the answer and the 9 most similar to the question), against the original model in FP32 (LiteRT 2.3, laptop CPU):

	FP32	this model, L512	this model, L128 (pairs cut at 128)
Same best passage as FP32 (same length)	—	37 / 38	37 / 38
Passage with the answer first	31	32	18 (FP32 at 128: 18)
Passage with the answer in the first three	34	34	—

llama.cpp's tokens from the vocabulary file: identical to transformers' on all 380 pairs, at 512 and at 128 tokens. In Q-P1NG's archive search (68 questions), the passage holding the answer reaches the model as often with the two signatures as with the 512-token one alone.

Conversion

See multilingual-reranker-small-CONVERSION.txt; the tools are in the Q-P1NG repository (tools/litert).

Licence

Apache-2.0, as the original model: see multilingual-reranker-small-LICENSE.txt and multilingual-reranker-small-NOTICE.txt (mmBERT-small, MIT). The original was trained on openly licensed web text (FineWeb, FineWeb-2: ODC-BY), MIRACL (Apache-2.0) and WebFAQ (CC-BY-4.0), with labels from Qwen3-Reranker-4B (Apache-2.0). The changes made here are listed in multilingual-reranker-small-CONVERSION.txt.