| --- |
| license: mit |
| library_name: transformers |
| pipeline_tag: text-classification |
| base_model: microsoft/deberta-v3-large |
| language: |
| - en |
| tags: |
| - cross-encoder |
| - reward-model |
| - forecasting |
| --- |
| |
| # REM matcher |
|
|
| A DeBERTa-v3-large cross-encoder that decides whether a predicted move by a named actor is the same |
| response-equivalent move (REM) as a realized one. It is the learned matcher used as the reinforcement |
| learning reward in *Toward Simulating Real-World Actors: Training Small Language Models to Predict |
| Their Next Moves*. |
|
|
| ## Input |
|
|
| The model reads a text pair. Each side is the actor's name followed by a move: |
|
|
| ``` |
| Actor context: {actor} |
| Action: {move} |
| ``` |
|
|
| During training, actors with known aliases were written as `{actor}; aliases: {alias1}; {alias2}`. |
| The predicted move goes first and the realized (target) move second. Pairs are truncated to 512 tokens. |
| Labels are `no` (0) and `yes` (1). |
|
|
| ## Calibrated probability |
|
|
| Raw softmax scores are not calibrated. Use the Platt scaling in `calibration.json` on the logit margin: |
|
|
| ``` |
| p_match = sigmoid((logit_yes - logit_no) / temperature + bias) |
| ``` |
|
|
| ```python |
| import json, torch |
| from huggingface_hub import hf_hub_download |
| from transformers import AutoModelForSequenceClassification, AutoTokenizer |
| |
| repo = "HiddenSystemsAI/rem-matcher" |
| tokenizer = AutoTokenizer.from_pretrained(repo) |
| model = AutoModelForSequenceClassification.from_pretrained(repo).eval() |
| cal = json.load(open(hf_hub_download(repo, "calibration.json"))) |
| |
| def side(actor, move): |
| return f"Actor context: {actor}\nAction: {move}" |
| |
| actor = "Bitget" |
| pairs = [ |
| (side(actor, "Bitget resumes withdrawals"), side(actor, "Bitget reopens withdrawals for users")), |
| (side(actor, "Bitget offers a bounty for the stolen funds"), side(actor, "Bitget reopens withdrawals for users")), |
| ] |
| batch = tokenizer([a for a, _ in pairs], [b for _, b in pairs], padding=True, |
| truncation=True, max_length=512, return_tensors="pt") |
| with torch.inference_mode(): |
| logits = model(**batch).logits.float() |
| p = torch.sigmoid((logits[:, 1] - logits[:, 0]) / cal["temperature"] + cal["bias"]) |
| print(p.tolist()) |
| ``` |
|
|
| In the paper's reward a predicted move credits a target when `p_match >= 0.75`, and two predicted moves |
| count as duplicates when `p_match >= 0.8`. |
|
|
| ## Training |
|
|
| The model was initialized from an earlier ordinal matcher of the same architecture, with a new two-class |
| head trained by cross-entropy for one epoch (learning rate 1e-5, effective batch 32). All pair labels |
| come from Claude models (Sonnet labels, a blind second pass on borderline pairs, Opus adjudication). |
| Splits are disjoint by scenario; the test split holds only real pairs of model-predicted and realized |
| moves from held-out stories. |
|
|
| | Split | Pairs | Positives | Scenarios | |
| |---|---:|---:|---:| |
| | Train | 81,439 | 43,904 | 397 | |
| | Development | 7,148 | 3,709 | 37 | |
| | Calibration | 7,937 | 4,367 | 45 | |
| | Test (real pairs) | 4,048 | 667 | 54 | |
|
|
| ## Evaluation |
|
|
| On the 4,048 held-out real test pairs, with calibrated probabilities at a 0.5 threshold: |
| accuracy 94.4%, precision 83.4%, recall 82.6%, expected calibration error 0.017. |
| The paper reports precision and recall at the 0.75 training gate and compares the matcher's credits |
| with a three-judge panel on the benchmark lists. |
|
|
| ## Limitations |
|
|
| The matcher was trained and evaluated on news-derived moves of governments, companies and other public |
| actors, written at the REM level. It undercounts matches that human-style judges accept (it misses |
| about a fifth to a third of judged matches on the paper's benchmark), and optimizing a policy against |
| it can raise its score faster than judged recall. Use it as a training signal or for grouping moves, |
| not as an evaluation metric on its own. |
|
|
