Instructions to use SUSTech/Qwen3-8B-CalibSFT-RLCR with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use SUSTech/Qwen3-8B-CalibSFT-RLCR with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="SUSTech/Qwen3-8B-CalibSFT-RLCR")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("SUSTech/Qwen3-8B-CalibSFT-RLCR")
model = AutoModelForCausalLM.from_pretrained("SUSTech/Qwen3-8B-CalibSFT-RLCR", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use SUSTech/Qwen3-8B-CalibSFT-RLCR with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "SUSTech/Qwen3-8B-CalibSFT-RLCR"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "SUSTech/Qwen3-8B-CalibSFT-RLCR",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/SUSTech/Qwen3-8B-CalibSFT-RLCR

SGLang

How to use SUSTech/Qwen3-8B-CalibSFT-RLCR with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "SUSTech/Qwen3-8B-CalibSFT-RLCR" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "SUSTech/Qwen3-8B-CalibSFT-RLCR",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "SUSTech/Qwen3-8B-CalibSFT-RLCR" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "SUSTech/Qwen3-8B-CalibSFT-RLCR",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use SUSTech/Qwen3-8B-CalibSFT-RLCR with Docker Model Runner:
```
docker model run hf.co/SUSTech/Qwen3-8B-CalibSFT-RLCR
```

claytonwang commited on 22 days ago

Commit

d0f7033

verified ·

1 Parent(s): 19ea6f0

Update model card

Browse files

Files changed (1) hide show

README.md +12 -26

README.md CHANGED Viewed

@@ -17,21 +17,26 @@ tags:
 # Qwen3-8B-CalibSFT-RLCR
-Qwen3-8B trained with **CalibSFT followed by RLCR**, from the paper [On the Pitfalls of Verbalized Confidence Priors for Calibrating Large Reasoning Models](https://arxiv.org/abs/2609.32470). The model solves a problem, then states its answer and a verbalized confidence between 0 and 1. It is initialized from [SUSTech/Qwen3-8B-CalibSFT](https://huggingface.co/SUSTech/Qwen3-8B-CalibSFT).
-- **Code:** https://github.com/ml-stat-Sustech/verbalized-confidence-training
-- **Training data:** [SUSTech/CalibSFT-DeepScaleR](https://huggingface.co/datasets/SUSTech/CalibSFT-DeepScaleR)
 ## Usage
-The model expects the system prompt and user format used during training:
 ```python
 from transformers import AutoModelForCausalLM, AutoTokenizer
 model_id = "SUSTech/Qwen3-8B-CalibSFT-RLCR"
 tokenizer = AutoTokenizer.from_pretrained(model_id)
-model = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype="auto", device_map="auto")
 SYSTEM_PROMPT = (
     "A conversation between User and Assistant. The user asks a question, and the Assistant solves it. "
@@ -54,38 +59,19 @@ text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_pr
 inputs = tokenizer(text, return_tensors="pt").to(model.device)
 outputs = model.generate(**inputs, max_new_tokens=16384, do_sample=True, temperature=0.6, top_p=0.95, top_k=20)
 print(tokenizer.decode(outputs[0][inputs.input_ids.shape[1]:], skip_special_tokens=True))
-# Output format: <think> ... </think> <answer> ... </answer> <confidence> 0.xx </confidence>
 ```
-## Training
-The model starts from [SUSTech/Qwen3-8B-CalibSFT](https://huggingface.co/SUSTech/Qwen3-8B-CalibSFT) and is trained with RLCR on DeepScaleR. RLCR rewards answer correctness and format together with a Brier calibration reward on the stated confidence. Training follows the DAPO recipe with dynamic sampling, and questions that the base model already solves in every sampled response are removed beforehand.
-| Hyperparameter | Value |
-| --- | --- |
-| Responses per question | 8 |
-| Questions per batch | 64 |
-| Mini-batch size | 32 |
-| Steps | 160 |
-| Learning rate | 2e-6, cosine decay to 1e-6 after 10 warm-up steps |
-| Clip ratio (low / high) | 0.2 / 0.28 |
-| KL coefficient | 0 |
-| Max response length | 8,192 tokens |
-| Seed | 45 |
 ## Evaluation
-Temperature 0.6, 4 responses per question (32 on AIME). Metrics are percentages averaged over 8 in-distribution math benchmarks (DeepScaleR-Eval, MATH-500, MinervaMath, OlympiadBench, GSM8K, AIME 2024–2026) and 8 out-of-distribution benchmarks (HotpotQA, TriviaQA, DROP, MuSiQue, LiveBenchReasoning, NQOpen, PopQA, WebQuestions).
 | Model | ID Pass@1 ↑ | ID AUROC ↑ | ID Brier ↓ | ID ECE ↓ | OOD Pass@1 ↑ | OOD AUROC ↑ | OOD Brier ↓ | OOD ECE ↓ |
 | --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
 | Qwen3-8B | 59.32 | 71.90 | 33.20 | 34.36 | 45.36 | 65.71 | 45.15 | 46.94 |
 | Qwen3-8B-RLCR | 65.33 | 82.36 | 14.43 | 14.46 | 43.96 | 69.38 | 31.14 | 31.17 |
-| Qwen3-8B-CalibSFT | 58.51 | 79.93 | 17.30 | 14.23 | 46.37 | 61.20 | 29.15 | 25.49 |
 | **Qwen3-8B-CalibSFT-RLCR** | 67.20 | 86.32 | 12.77 | 8.59 | 45.71 | 71.31 | 23.24 | 18.20 |
-Qwen3-8B and CalibSFT were evaluated with up to 6,144 generated tokens and the RLCR models with up to 16,384.
 ## Citation
 ```bibtex

 # Qwen3-8B-CalibSFT-RLCR
+<p align="center">
+  <a href="https://arxiv.org/abs/2609.32470"><img src="https://img.shields.io/badge/Paper-arXiv-b31b1b.svg?logo=arxiv" alt="Paper"></a>
+  <a href="https://github.com/ml-stat-Sustech/verbalized-confidence-training"><img src="https://img.shields.io/badge/Code-GitHub-181717.svg?logo=github" alt="Code"></a>
+  <a href="https://huggingface.co/datasets/SUSTech/CalibSFT-DeepScaleR"><img src="https://img.shields.io/badge/Dataset-Hugging%20Face-ffd21e.svg?logo=huggingface" alt="Dataset"></a>
+</p>
+## Introduction
+Qwen3-8B-CalibSFT-RLCR is [Qwen3-8B-CalibSFT](https://huggingface.co/SUSTech/Qwen3-8B-CalibSFT) further trained with RLCR, a confidence-aware reinforcement learning method. It is released with our paper [On the Pitfalls of Verbalized Confidence Priors for Calibrating Large Reasoning Models](https://arxiv.org/abs/2609.32470). Compared with RLCR trained from Qwen3-8B, it improves accuracy, discrimination, and calibration on both in-distribution and out-of-distribution benchmarks. For training details, please refer to our [GitHub repository](https://github.com/ml-stat-Sustech/verbalized-confidence-training).
 ## Usage
+Use the system prompt and user format from training:
 ```python
 from transformers import AutoModelForCausalLM, AutoTokenizer
 model_id = "SUSTech/Qwen3-8B-CalibSFT-RLCR"
 tokenizer = AutoTokenizer.from_pretrained(model_id)
+model = AutoModelForCausalLM.from_pretrained(model_id, dtype="auto", device_map="auto")
 SYSTEM_PROMPT = (
     "A conversation between User and Assistant. The user asks a question, and the Assistant solves it. "
 inputs = tokenizer(text, return_tensors="pt").to(model.device)
 outputs = model.generate(**inputs, max_new_tokens=16384, do_sample=True, temperature=0.6, top_p=0.95, top_k=20)
 print(tokenizer.decode(outputs[0][inputs.input_ids.shape[1]:], skip_special_tokens=True))
+# <think> ... </think> <answer> ... </answer> <confidence> 0.xx </confidence>
 ```
 ## Evaluation
+Results are averaged over 8 in-distribution math benchmarks (DeepScaleR-Eval, MATH-500, MinervaMath, OlympiadBench, GSM8K, AIME 2024–2026) and 8 out-of-distribution benchmarks (HotpotQA, TriviaQA, DROP, MuSiQue, LiveBenchReasoning, NQOpen, PopQA, WebQuestions), sampling at temperature 0.6 with 4 responses per question (32 on AIME). DeepScaleR-Eval consists of 2,000 DeepScaleR questions held out from training.
 | Model | ID Pass@1 ↑ | ID AUROC ↑ | ID Brier ↓ | ID ECE ↓ | OOD Pass@1 ↑ | OOD AUROC ↑ | OOD Brier ↓ | OOD ECE ↓ |
 | --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
 | Qwen3-8B | 59.32 | 71.90 | 33.20 | 34.36 | 45.36 | 65.71 | 45.15 | 46.94 |
 | Qwen3-8B-RLCR | 65.33 | 82.36 | 14.43 | 14.46 | 43.96 | 69.38 | 31.14 | 31.17 |
 | **Qwen3-8B-CalibSFT-RLCR** | 67.20 | 86.32 | 12.77 | 8.59 | 45.71 | 71.31 | 23.24 | 18.20 |
 ## Citation
 ```bibtex