---
license: apache-2.0
base_model: SUSTech/Qwen3-8B-CalibSFT
base_model_relation: finetune
library_name: transformers
pipeline_tag: text-generation
language:
- en
datasets:
- SUSTech/CalibSFT-DeepScaleR
tags:
- reasoning
- math
- confidence-calibration
- verbalized-confidence
---
# Qwen3-8B-CalibSFT-RLCR
## Introduction
Qwen3-8B-CalibSFT-RLCR is [Qwen3-8B-CalibSFT](https://huggingface.co/SUSTech/Qwen3-8B-CalibSFT) further trained with RLCR, a confidence-aware reinforcement learning method. It is released with our paper [On the Pitfalls of Verbalized Confidence Priors for Calibrating Large Reasoning Models](https://arxiv.org/abs/2609.32470). Compared with RLCR trained from Qwen3-8B, it improves accuracy, discrimination, and calibration on both in-distribution and out-of-distribution benchmarks. For training details, please refer to our [GitHub repository](https://github.com/ml-stat-Sustech/verbalized-confidence-training).
## Usage
Use the system prompt and user format from training:
```python
from transformers import AutoModelForCausalLM, AutoTokenizer
model_id = "SUSTech/Qwen3-8B-CalibSFT-RLCR"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, dtype="auto", device_map="auto")
SYSTEM_PROMPT = (
"A conversation between User and Assistant. The user asks a question, and the Assistant solves it. "
"The assistant first thinks about the reasoning process in the mind and analyzes its confidence about "
"the solution and then provides the user with the final answer as well as its confidence level. "
"The confidence level indicates how certain the Assistant is about its answer, expressed as a decimal "
"between 0 and 1 with exactly two decimal places, enclosed within tags. "
"The response must strictly follow this format: reasoning process here "
" final short answer only 0.xx . "
"The tag must contain only the final answer string needed for exact-match evaluation, "
"not a full sentence, explanation, or reasoning."
)
question = "What is the smallest positive integer n such that n^2 + n is divisible by 12?"
messages = [
{"role": "system", "content": SYSTEM_PROMPT},
{"role": "user", "content": f"\n\nPROBLEM: {question}\n\n"},
]
text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(text, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=16384, do_sample=True, temperature=0.6, top_p=0.95, top_k=20)
print(tokenizer.decode(outputs[0][inputs.input_ids.shape[1]:], skip_special_tokens=True))
# ... ... 0.xx
```
## Evaluation
Results are averaged over 8 in-distribution math benchmarks (DeepScaleR-Eval, MATH-500, MinervaMath, OlympiadBench, GSM8K, AIME 2024–2026) and 8 out-of-distribution benchmarks (HotpotQA, TriviaQA, DROP, MuSiQue, LiveBenchReasoning, NQOpen, PopQA, WebQuestions), sampling at temperature 0.6 with 4 responses per question (32 on AIME). DeepScaleR-Eval consists of 2,000 DeepScaleR questions held out from training.
| Model |
In-distribution |
Out-of-distribution |
| Pass@1 ↑ | AUROC ↑ | Brier ↓ | ECE ↓ |
Pass@1 ↑ | AUROC ↑ | Brier ↓ | ECE ↓ |
| Qwen3-8B | 59.32 | 71.90 | 33.20 | 34.36 | 45.36 | 65.71 | 45.15 | 46.94 |
| Qwen3-8B-RLCR | 65.33 | 82.36 | 14.43 | 14.46 | 43.96 | 69.38 | 31.14 | 31.17 |
| Qwen3-8B-CalibSFT-RLCR | 67.20 | 86.32 | 12.77 | 8.59 | 45.71 | 71.31 | 23.24 | 18.20 |
## Citation
```bibtex
@article{wang2026pitfalls,
title={On the Pitfalls of Verbalized Confidence Priors for Calibrating Large Reasoning Models},
author={Wang, Shuoyuan and Luo, Beier and Zeng, Hao and Yu, Chengyao and Zhang, Songxin and Xie, Zejian and Jing, Bingyi and Wei, Hongxin},
journal={arXiv preprint arXiv:2609.32470},
year={2026}
}
```