wiiiktor/chopper_mmbert

Chopper: mmBERT

Reads the snippets your retriever found (from a database or a long conversation) and cuts out chops: exact quotes that answer the question, typically 3–5% of the text, never rewritten. Plain text in, plain text out, so the chops work with any answering model. Code: github.com/wiiiktor/chopper_demo.

folder	model	trained on
mmbert-base/	mmBERT-base chopper, 0.31B (recommended)	102,654 rows in one mixed run: multi-hop (MuSiQue, HotpotQA, 2WikiMultiHopQA, MuSiQue unanswerable), 8-language rerank pools, scientific papers (QASPER), conversations (WildChat, Multi-Session Chat)
mmbert-small/	mmBERT-small chopper, 0.14B	the same 102,654-row mix, same recipe

Results

Multi-hop, one retrieval per question. 400 MuSiQue dev questions, 100 each at 1–4 hops, each with its own 20 snippets; Qwen3-14B (4-bit, no fine-tuning) answers from the given text only, gpt-5.6-luna judges. Correct answers; in brackets, only questions whose chops kept every hop and the answer:

Qwen3-14B answering from	tokens	1-hop	2-hop	3-hop	4-hop
all 20 snippets	2,620	84	28	22	12
chops from mmBERT-base	228	83 (82)	48 (44)	45 (35)	29 (21)
chops from mmBERT-small	217	72 (67)	49 (41)	32 (22)	23 (11)
chops from Claude Opus 5.5	144	90 (90)	64 (60)	55 (40)	43 (30)

Validation (decoded at p > 0.5, gaps ≤ 5 tokens merged, runs < 4 dropped; character F1 of kept text / share of questions whose answer is in the kept text):

chopper	MuSiQue	HotpotQA	TriviaQA	QASPER	Multi-Session Chat	rerank holdout, 8 languages
base	.56 / .55	.63 / .85	.42 / .69	.30 / .32	.68 / .59	.52
small	.54 / .49	.60 / .80	.32 / .64	.26 / .28	.63 / .59	.47

On pools whose answer sentences were deleted (8 languages), it chooses SEARCH 67% of the time (mmBERT-small: 33%).

Input and output

Input: one string, up to 8,192 tokens: Question: <question> [SEP] <passage 1> [SEP] … [SEP] <passage N>.

Output:

• per token, p(KEEP) from keep.* in heads.safetensors. Keep tokens with p > 0.5 (the threshold is in chopper_config.json), merge runs separated by ≤ 5 tokens, drop runs shorter than 4 tokens.
• one ANSWER / SEARCH decision from decision.* on the first token (0 = SEARCH, 1 = ANSWER).

Give the answering model only the chops:

Notes:
- <fragment 1>
- <fragment 2>

Question: <question>
Answer:

Minimal use

import json, torch
from safetensors.torch import load_file
from transformers import AutoModel, AutoTokenizer
from huggingface_hub import snapshot_download

d = snapshot_download("wiiiktor/chopper_mmbert", allow_patterns=["mmbert-base/*"]) + "/mmbert-base"
tok, enc = AutoTokenizer.from_pretrained(f"{d}/encoder"), AutoModel.from_pretrained(f"{d}/encoder").eval()
H, thr = load_file(f"{d}/heads.safetensors"), json.load(open(f"{d}/chopper_config.json"))["decode"]["pred_thr"]
text = "Question: " + question + "".join(f" {tok.sep_token} {p}" for p in passages)
x = tok(text, return_tensors="pt", return_offsets_mapping=True, truncation=True, max_length=8192)
off = x.pop("offset_mapping")[0]
with torch.no_grad():
    h = enc(**x).last_hidden_state[0].float()
p_keep = torch.sigmoid(h @ H["keep.weight"].float().T + H["keep.bias"].float())[:, 0]
# kept characters of `text`: off[i] for every token i with p_keep[i] > thr (then merge gaps ≤ 5, drop runs < 4)

Labels from commercial APIs. Claude Opus 5.5 (Anthropic) wrote the MuSiQue span labels used in training. They were used for an allowed purpose: training a non-competing model, a small information-extraction and summarization tool (the chopper), not a general-purpose chat or text-generation model. gpt-5.6-luna and GPT-6 Luna (OpenAI) were used only as YES/NO relevance filters for the rerank rows and as answer judges in evaluations; the resulting model is likewise an extraction tool that does not compete with OpenAI's products.

A complete retrieve → chop → answer loop over your own documents: operator-for-llm-with-mmbert.py in the GitHub repo.