Text Generation
Transformers
Safetensors
qwen2
care
selective-prediction
confidence-estimation
grpo
conversational
text-generation-inference
Instructions to use hector-gr/CARE-Qwen2.5-7B-Math with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use hector-gr/CARE-Qwen2.5-7B-Math with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="hector-gr/CARE-Qwen2.5-7B-Math") messages = [ {"role": "user", "content": "Who are you?"}, ] pipe(messages)# pip install -U transformers accelerate # Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("hector-gr/CARE-Qwen2.5-7B-Math") model = AutoModelForCausalLM.from_pretrained("hector-gr/CARE-Qwen2.5-7B-Math", device_map="auto") messages = [ {"role": "user", "content": "Who are you?"}, ] inputs = tokenizer.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=256) print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use hector-gr/CARE-Qwen2.5-7B-Math with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "hector-gr/CARE-Qwen2.5-7B-Math" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "hector-gr/CARE-Qwen2.5-7B-Math", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/hector-gr/CARE-Qwen2.5-7B-Math
- SGLang
How to use hector-gr/CARE-Qwen2.5-7B-Math with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "hector-gr/CARE-Qwen2.5-7B-Math" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "hector-gr/CARE-Qwen2.5-7B-Math", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "hector-gr/CARE-Qwen2.5-7B-Math" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "hector-gr/CARE-Qwen2.5-7B-Math", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }' - Docker Model Runner
How to use hector-gr/CARE-Qwen2.5-7B-Math with Docker Model Runner:
docker model run hf.co/hector-gr/CARE-Qwen2.5-7B-Math
Update model card
Browse files
README.md
CHANGED
|
@@ -15,15 +15,22 @@ tags:
|
|
| 15 |
|
| 16 |
# CARE Qwen2.5-7B — Math
|
| 17 |
|
| 18 |
-
|
| 19 |
|
| 20 |
-
|
| 21 |
|
| 22 |
-
##
|
| 23 |
|
| 24 |
-
|
| 25 |
|
| 26 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 27 |
|
| 28 |
## Use
|
| 29 |
|
|
@@ -31,10 +38,12 @@ Load with `AutoTokenizer.from_pretrained("hector-gr/CARE-Qwen2.5-7B-Math")` and
|
|
| 31 |
|
| 32 |
Outputs contain `<answer>...</answer>` and `<confidence>...</confidence>`. Rank answers by confidence and choose a threshold using suitable held-out data. A risk measured on one dataset need not transfer to a different distribution.
|
| 33 |
|
| 34 |
-
|
|
|
|
|
|
|
| 35 |
|
| 36 |
-
|
| 37 |
|
| 38 |
## Attribution
|
| 39 |
|
| 40 |
-
|
|
|
|
| 15 |
|
| 16 |
# CARE Qwen2.5-7B — Math
|
| 17 |
|
| 18 |
+
Qwen2.5-7B trained on Big-Math-Digits (`mehuldamani/big-math-digits`) to verbalize the confidence in its own answer with CARE.
|
| 19 |
|
| 20 |
+
Paper: *Cross-Question Reliable Reinforcement Learning* — Hector G. Rodriguez and Marcus Rohrbach.
|
| 21 |
|
| 22 |
+
## Abstract
|
| 23 |
|
| 24 |
+
AI systems operating near the limits of their capabilities can maximize utility while minimizing the risk of error by abstaining or deferring questions they are not confident about to a human or a stronger model. Recently, large language models (LLMs) have been post-trained to estimate confidence in their own answers. To achieve this, methods reinforce confidence scores based on the correctness signal to each question-answer pair *individually*, with rewards that encourage assigning confidence 1 to correct answers and 0 to incorrect answers. However, estimating confidence scores is useful in that it enables selecting only the most confident answers, to maximize the share of solved questions while accommodating the user's risk tolerance. Assigning confidence 1 to correct answers and 0 to incorrect ones is theoretically optimal, but unattainable in practice and detrimental for selective prediction with a non-zero risk tolerance: confidences pushed to 0 and 1 carry no information about how answers compare to one another. We therefore introduce CARE (Cross-question Abstention Reward Estimation), which computes rewards over a set of question-answer pairs: At each of several risk tolerances, the confidences across the set induce an abstention threshold, and each confidence is rewarded for lying on the correct side of it. Since the threshold itself comes from the set, every verbalized confidence is reinforced relative to other questions' confidence, not against its own binary correctness label. On math, CARE increases the coverage at 5% risk (C@5) across MATH-500, GSM8K, and Big-Math-Digits from 0.8% to 42.4%, compared to existing pointwise correctness-matching rewards. When trained on multi-hop Wikipedia QA from HotpotQA, CARE improves C@5 in out-of-distribution general-knowledge, reasoning, and math benchmarks (including CommonsenseQA, GPQA, SimpleQA, and TriviaQA) from 12.3% to 23.6%.
|
| 25 |
|
| 26 |
+
## Results
|
| 27 |
+
|
| 28 |
+
Selective prediction (all values in %; C@r is coverage at risk r; AURC lower is better). ID: Big-Math-Digits, MATH-500, GSM8K. OOD: CommonsenseQA, GPQA, SimpleQA, TriviaQA, HotpotQA.
|
| 29 |
+
|
| 30 |
+
| Model | ID C@0 | ID C@1 | ID C@5 | ID AURC ↓ | ID Acc | OOD C@0 | OOD C@1 | OOD C@5 | OOD AURC ↓ | OOD Acc |
|
| 31 |
+
|---|---:|---:|---:|---:|---:|---:|---:|---:|---:|---:|
|
| 32 |
+
| RLCR (Cold) | 0.0 | 0.0 | 0.8 | 19.3 | 73.1 | 0.0 | 0.0 | 4.9 | 38.7 | 56.6 |
|
| 33 |
+
| **CARE (this model)** | **5.4** | **18.2** | **42.4** | **12.5** | 73.1 | **2.9** | **5.4** | **15.9** | 37.7 | 53.6 |
|
| 34 |
|
| 35 |
## Use
|
| 36 |
|
|
|
|
| 38 |
|
| 39 |
Outputs contain `<answer>...</answer>` and `<confidence>...</confidence>`. Rank answers by confidence and choose a threshold using suitable held-out data. A risk measured on one dataset need not transfer to a different distribution.
|
| 40 |
|
| 41 |
+
The training configuration is included in `training_config.yaml` (seed 43).
|
| 42 |
+
|
| 43 |
+
## Limitations
|
| 44 |
|
| 45 |
+
The model can still give incorrect answers with high confidence. It has not been validated for safety-critical decisions. Model weights are released under Apache 2.0, with the base-model license included.
|
| 46 |
|
| 47 |
## Attribution
|
| 48 |
|
| 49 |
+
Built on RLCR, Beyond Binary Rewards: Training LMs to Reason About Their Uncertainty (Damani et al., 2025), and TRL. Base model: Qwen2.5-7B, Qwen Team.
|