Release REM matcher (v10b checkpoint used as the RL reward)

README.md

@@ -0,0 +1,99 @@
+---
+license: mit
+library_name: transformers
+pipeline_tag: text-classification
+base_model: microsoft/deberta-v3-large
+language:
+- en
+tags:
+- cross-encoder
+- reward-model
+- forecasting
+---
+
+# REM matcher
+
+A DeBERTa-v3-large cross-encoder that decides whether a predicted move by a named actor is the same
+response-equivalent move (REM) as a realized one. It is the learned matcher used as the reinforcement
+learning reward in *Toward Simulating Real-World Actors: Training Small Language Models to Predict
+Their Next Moves*.
+
+## Input
+
+The model reads a text pair. Each side is the actor's name followed by a move:
+
+```
+Actor context: {actor}
+Action: {move}
+```
+
+During training, actors with known aliases were written as `{actor}; aliases: {alias1}; {alias2}`.
+The predicted move goes first and the realized (target) move second. Pairs are truncated to 512 tokens.
+Labels are `no` (0) and `yes` (1).
+
+## Calibrated probability
+
+Raw softmax scores are not calibrated. Use the Platt scaling in `calibration.json` on the logit margin:
+
+```
+p_match = sigmoid((logit_yes - logit_no) / temperature + bias)
+```
+
+```python
+import json, torch
+from huggingface_hub import hf_hub_download
+from transformers import AutoModelForSequenceClassification, AutoTokenizer
+
+repo = "HiddenSystemsAI/rem-matcher"
+tokenizer = AutoTokenizer.from_pretrained(repo)
+model = AutoModelForSequenceClassification.from_pretrained(repo).eval()
+cal = json.load(open(hf_hub_download(repo, "calibration.json")))
+
+def side(actor, move):
+    return f"Actor context: {actor}\nAction: {move}"
+
+actor = "Bitget"
+pairs = [
+    (side(actor, "Bitget resumes withdrawals"), side(actor, "Bitget reopens withdrawals for users")),
+    (side(actor, "Bitget offers a bounty for the stolen funds"), side(actor, "Bitget reopens withdrawals for users")),
+]
+batch = tokenizer([a for a, _ in pairs], [b for _, b in pairs], padding=True,
+                  truncation=True, max_length=512, return_tensors="pt")
+with torch.inference_mode():
+    logits = model(**batch).logits.float()
+p = torch.sigmoid((logits[:, 1] - logits[:, 0]) / cal["temperature"] + cal["bias"])
+print(p.tolist())
+```
+
+In the paper's reward a predicted move credits a target when `p_match >= 0.75`, and two predicted moves
+count as duplicates when `p_match >= 0.8`.
+
+## Training
+
+The model was initialized from an earlier ordinal matcher of the same architecture, with a new two-class
+head trained by cross-entropy for one epoch (learning rate 1e-5, effective batch 32). All pair labels
+come from Claude models (Sonnet labels, a blind second pass on borderline pairs, Opus adjudication).
+Splits are disjoint by scenario; the test split holds only real pairs of model-predicted and realized
+moves from held-out stories.
+
+| Split | Pairs | Positives | Scenarios |
+|---|---:|---:|---:|
+| Train | 81,439 | 43,904 | 397 |
+| Development | 7,148 | 3,709 | 37 |
+| Calibration | 7,937 | 4,367 | 45 |
+| Test (real pairs) | 4,048 | 667 | 54 |
+
+## Evaluation
+
+On the 4,048 held-out real test pairs, with calibrated probabilities at a 0.5 threshold:
+accuracy 94.4%, precision 83.4%, recall 82.6%, expected calibration error 0.017.
+The paper reports precision and recall at the 0.75 training gate and compares the matcher's credits
+with a three-judge panel on the benchmark lists.
+
+## Limitations
+
+The matcher was trained and evaluated on news-derived moves of governments, companies and other public
+actors, written at the REM level. It undercounts matches that human-style judges accept (it misses
+about a fifth to a third of judged matches on the paper's benchmark), and optimizing a policy against
+it can raise its score faster than judged recall. Use it as a training signal or for grouping moves,
+not as an evaluation metric on its own.

calibration.json

@@ -0,0 +1,10 @@
+{
+  "temperature": 1.7073526474706913,
+  "bias": -0.4470148298452159,
+  "decision_threshold": 0.5,
+  "confidence_threshold": 0.8,
+  "method": "platt_on_logit_margin",
+  "positive_label": "yes",
+  "probability_scope": "L2 matching on the held-out pair distribution",
+  "max_length": 512
+}

config.json

@@ -0,0 +1,47 @@
+{
+  "architectures": [
+    "DebertaV2ForSequenceClassification"
+  ],
+  "attention_probs_dropout_prob": 0.1,
+  "bos_token_id": 1,
+  "dtype": "float32",
+  "eos_token_id": 2,
+  "hidden_act": "gelu",
+  "hidden_dropout_prob": 0.1,
+  "hidden_size": 1024,
+  "id2label": {
+    "0": "no",
+    "1": "yes"
+  },
+  "initializer_range": 0.02,
+  "intermediate_size": 4096,
+  "label2id": {
+    "no": 0,
+    "yes": 1
+  },
+  "layer_norm_eps": 1e-07,
+  "legacy": true,
+  "max_position_embeddings": 512,
+  "max_relative_positions": -1,
+  "model_type": "deberta-v2",
+  "norm_rel_ebd": "layer_norm",
+  "num_attention_heads": 16,
+  "num_hidden_layers": 24,
+  "pad_token_id": 0,
+  "pooler_dropout": 0,
+  "pooler_hidden_act": "gelu",
+  "pooler_hidden_size": 1024,
+  "pos_att_type": [
+    "p2c",
+    "c2p"
+  ],
+  "position_biased_input": false,
+  "position_buckets": 256,
+  "relative_attention": true,
+  "share_att_key": true,
+  "tie_word_embeddings": true,
+  "transformers_version": "5.13.1",
+  "type_vocab_size": 0,
+  "use_cache": false,
+  "vocab_size": 128100
+}

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:89a34874abc5ac6842ff3227925cc5e6886f94841a2dc8aa17007cd607610aa0
+size 1740304440

tokenizer_config.json

@@ -0,0 +1,24 @@
+{
+  "add_prefix_space": true,
+  "backend": "tokenizers",
+  "bos_token": "[CLS]",
+  "cls_token": "[CLS]",
+  "do_lower_case": false,
+  "eos_token": "[SEP]",
+  "is_local": false,
+  "local_files_only": false,
+  "mask_token": "[MASK]",
+  "max_length": 512,
+  "model_max_length": 512,
+  "pad_token": "[PAD]",
+  "sep_token": "[SEP]",
+  "sp_model_kwargs": {},
+  "split_by_punct": false,
+  "stride": 0,
+  "tokenizer_class": "DebertaV2Tokenizer",
+  "truncation_side": "right",
+  "truncation_strategy": "longest_first",
+  "unk_id": 3,
+  "unk_token": "[UNK]",
+  "vocab_type": "spm"
+}