Update model card

README.md

@@ -23,21 +23,64 @@ tags:
 > simulated users on a public benchmark. It has not been tested with real
 > customers, real data or real business systems.
 
-Paste this file into the Hugging Face repository's `README.md` when the
-checkpoint is uploaded ([docs/release.md](https://github.com/JaspinXu/KnowWhenToHandOff/blob/main/docs/release.md)).
+**Code** [github.com/JaspinXu/KnowWhenToHandOff](https://github.com/JaspinXu/KnowWhenToHandOff) · **Data** [KnowWhenToHandOff-data](https://huggingface.co/datasets/JaspinXu/KnowWhenToHandOff-data) · **Family** [SFT](https://huggingface.co/JaspinXu/KnowWhenToHandOff-Qwen3-4B-SFT) · [R1](https://huggingface.co/JaspinXu/KnowWhenToHandOff-Qwen3-4B-R1) · [R2](https://huggingface.co/JaspinXu/KnowWhenToHandOff-Qwen3-4B-R2)
+
+## Quick start
+
+The model is a standard Qwen3 chat model with tool calling. It was trained with
+the τ²-bench domain policy as the system prompt and the domain's tools,
+including `transfer_to_human_agents`; for faithful behaviour run it inside the
+evaluation harness of the [code repository](https://github.com/JaspinXu/KnowWhenToHandOff).
+
+```python
+from transformers import AutoModelForCausalLM, AutoTokenizer
+
+repo = "JaspinXu/KnowWhenToHandOff-Qwen3-4B-R2"
+tokenizer = AutoTokenizer.from_pretrained(repo)
+model = AutoModelForCausalLM.from_pretrained(repo, torch_dtype="auto", device_map="auto")
+
+tools = [{"type": "function", "function": {
+    "name": "transfer_to_human_agents",
+    "description": "Transfer the user to a human agent with a summary of the issue.",
+    "parameters": {"type": "object", "properties": {"summary": {"type": "string"}}, "required": ["summary"]},
+}}]
+messages = [
+    {"role": "system", "content": "<the airline or retail policy from tau2-bench>"},
+    {"role": "user", "content": "I need to book the unaccompanied-minor escort service for my child."},
+]
+inputs = tokenizer.apply_chat_template(messages, tools=tools, add_generation_prompt=True, return_tensors="pt").to(model.device)
+output = model.generate(inputs, max_new_tokens=512)
+print(tokenizer.decode(output[0][inputs.shape[-1]:], skip_special_tokens=True))
+```
+
+Serving with vLLM (OpenAI-compatible, tool calls parsed):
+
+```bash
+vllm serve JaspinXu/KnowWhenToHandOff-Qwen3-4B-R2 --enable-auto-tool-choice --tool-call-parser hermes
+```
 
 ## Model
 
 - Base model: Qwen/Qwen3-4B-Instruct-2507 (revision
   `cdbee75f17c01a7cc42f958dc650907174af0554`), Apache-2.0.
-- Training: GRPO from the SFT checkpoint B2 with the composite reward (`configs/reward/r2.yaml`: task reward − 0.25 × weighted violations − 0.5 × unneeded transfer − 1.0 × missed transfer − 0.1 × format errors); LoRA rank 64, alpha 128, lr 1e-5; 8 tasks × 8 rollouts per step, 60 steps, no KL term.
+- Training: GRPO from the SFT checkpoint B2 with the composite reward ([`configs/reward/r2.yaml`](https://github.com/JaspinXu/KnowWhenToHandOff/blob/main/configs/reward/r2.yaml): task reward − 0.25 × weighted violations − 0.5 × unneeded transfer − 1.0 × missed transfer − 0.1 × format errors); LoRA rank 64, alpha 128, lr 1e-5; 8 tasks × 8 rollouts per step, 60 steps, no KL term.
 - Training run `R2-s1-20261006` (seed 1); exported step-60 checkpoint
-  `sha256:be6198a0c51773f52ba4f20c9605a5931f23b0206e92a9fe0521a93761348c96` (`runs/R2-s1-20261006/export/global_step_60`, merged bf16 Hugging Face
-  format).
+  `sha256:be6198a0c51773f52ba4f20c9605a5931f23b0206e92a9fe0521a93761348c96` (merged bf16, Hugging Face format).
 - Evaluation code at git `462b5d17b4e9add637a2ee7802974c720b625a5b`; τ²-bench
   `fc0055dc4e0a316c3f83133267fbd6faaa770992`; all sources in
   [reports/main/sources.json](https://github.com/JaspinXu/KnowWhenToHandOff/blob/main/reports/main/sources.json).
 
+## Model family
+
+| Model | Training | Hand-off F1 | Over-escalation | Use it for |
+| --- | --- | --- | --- | --- |
+| [SFT (B2)](https://huggingface.co/JaspinXu/KnowWhenToHandOff-Qwen3-4B-SFT) | LoRA SFT on 790 filtered teacher dialogues | 0.768 | 0.016 | starting point; precise but misses hand-offs |
+| [R1](https://huggingface.co/JaspinXu/KnowWhenToHandOff-Qwen3-4B-R1) | GRPO from B2, task reward only | 0.802 | 0.462 | studying the outcome-reward loophole |
+| [R2](https://huggingface.co/JaspinXu/KnowWhenToHandOff-Qwen3-4B-R2) | GRPO from B2, task reward + violation and hand-off penalties | **0.812** | 0.103 | the best-calibrated hand-off model |
+
+Task success on the official test tasks is the same for all three within
+noise (0.33–0.35).
+
 ## Intended use
 
 Studying when a tool-using customer-service agent should hand off to a human,
@@ -54,17 +97,17 @@ never used for training or selection.
 
 ## Evaluation
 
-Test split, protocol frozen in ADR-005; 95 % bootstrap intervals over tasks.
+Test split, protocol frozen in [ADR-005](https://github.com/JaspinXu/KnowWhenToHandOff/blob/main/docs/adr/ADR-005-evaluation-protocol.md); 95 % bootstrap intervals over tasks.
 
 | Metric | This model | B2 (SFT start) | Source |
 | --- | --- | --- | --- |
-| Success (pass^1), test_main | 0.342 [0.245, 0.449] | 0.327 [0.224, 0.434] | `reports/main/main_table.md` |
-| Paired Δ success vs B2 | +0.015 [−0.056, 0.082] | — | `reports/main/paired_differences.md` |
-| pass^4 | 0.122 [0.041, 0.224] | 0.102 [0.041, 0.204] | `reports/main/main_table.md` |
+| Success (pass^1), test_main | 0.342 [0.245, 0.449] | 0.327 [0.224, 0.434] | [main table](https://github.com/JaspinXu/KnowWhenToHandOff/blob/main/reports/main/main_table.md) |
+| Paired Δ success vs B2 | +0.015 [−0.056, 0.082] | — | [paired differences](https://github.com/JaspinXu/KnowWhenToHandOff/blob/main/reports/main/paired_differences.md) |
+| pass^4 | 0.122 [0.041, 0.224] | 0.102 [0.041, 0.204] | [main table](https://github.com/JaspinXu/KnowWhenToHandOff/blob/main/reports/main/main_table.md) |
 | Violation rate (incl. off-reference writes) | 0.444 | 0.469 | same |
-| Hand-off F1 / precision / recall, test_handoff | **0.812** / 0.886 / 0.750 | 0.768 / 0.976 / 0.633 | `R2-final-test_handoff-s0-20261007` metrics |
+| Hand-off F1 / precision / recall, test_handoff | **0.812** / 0.886 / 0.750 | 0.768 / 0.976 / 0.633 | [`R2-final-test_handoff-s0-20261007`](https://huggingface.co/datasets/JaspinXu/KnowWhenToHandOff-data/blob/main/runs/R2-final-test_handoff-s0-20261007/metrics.json) |
 | Over-escalation, test_handoff | 0.103 | 0.016 | same |
-| Worst-style success / style gap | 0.316 / 0.112 | 0.327 / 0.112 | `reports/main/styles.md` |
+| Worst-style success / style gap | 0.316 / 0.112 | 0.327 / 0.112 | [styles](https://github.com/JaspinXu/KnowWhenToHandOff/blob/main/reports/main/styles.md) |
 
 **Known behaviour.** R2 removes R1's over-escalation but sometimes makes the opposite error on out-of-scope requests: it refuses and tells the user to contact the company directly without calling the transfer tool, or substitutes an unrelated write (under-escalation 0.25). Once it justified a hand-off with a policy that does not exist.
 
@@ -91,3 +134,15 @@ its Apache-2.0 licence. τ²-bench code and data (pinned commit
 its `LICENSE` file;
 released dialogue samples are τ²-bench-derived synthetic conversations with
 no real user data.
+
+## Citation
+
+```bibtex
+@misc{knowwhentohandoff2026,
+  title  = {Know When to Hand Off: Multi-turn Reinforcement Learning for
+            Trustworthy Customer-Service Agents},
+  author = {{KnowWhenToHandOff contributors}},
+  year   = {2026},
+  howpublished = {\url{https://github.com/JaspinXu/KnowWhenToHandOff}}
+}
+```