Instructions to use hector-gr/CARE-Qwen2.5-7B-Math with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use hector-gr/CARE-Qwen2.5-7B-Math with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="hector-gr/CARE-Qwen2.5-7B-Math") messages = [ {"role": "user", "content": "Who are you?"}, ] pipe(messages)# pip install -U transformers accelerate # Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("hector-gr/CARE-Qwen2.5-7B-Math") model = AutoModelForCausalLM.from_pretrained("hector-gr/CARE-Qwen2.5-7B-Math", device_map="auto") messages = [ {"role": "user", "content": "Who are you?"}, ] inputs = tokenizer.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=256) print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use hector-gr/CARE-Qwen2.5-7B-Math with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "hector-gr/CARE-Qwen2.5-7B-Math" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "hector-gr/CARE-Qwen2.5-7B-Math", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/hector-gr/CARE-Qwen2.5-7B-Math
- SGLang
How to use hector-gr/CARE-Qwen2.5-7B-Math with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "hector-gr/CARE-Qwen2.5-7B-Math" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "hector-gr/CARE-Qwen2.5-7B-Math", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "hector-gr/CARE-Qwen2.5-7B-Math" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "hector-gr/CARE-Qwen2.5-7B-Math", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }' - Docker Model Runner
How to use hector-gr/CARE-Qwen2.5-7B-Math with Docker Model Runner:
docker model run hf.co/hector-gr/CARE-Qwen2.5-7B-Math
CARE Qwen2.5-7B — Math
Qwen2.5-7B trained on Big-Math-Digits (mehuldamani/big-math-digits) to verbalize the confidence in its own answer with CARE.
Paper: Cross-Question Reliable Reinforcement Learning — Hector G. Rodriguez and Marcus Rohrbach.
Abstract
AI systems operating near the limits of their capabilities can maximize utility while minimizing the risk of error by abstaining or deferring questions they are not confident about to a human or a stronger model. Recently, large language models (LLMs) have been post-trained to estimate confidence in their own answers. To achieve this, methods reinforce confidence scores based on the correctness signal to each question-answer pair individually, with rewards that encourage assigning confidence 1 to correct answers and 0 to incorrect answers. However, estimating confidence scores is useful in that it enables selecting only the most confident answers, to maximize the share of solved questions while accommodating the user's risk tolerance. Assigning confidence 1 to correct answers and 0 to incorrect ones is theoretically optimal, but unattainable in practice and detrimental for selective prediction with a non-zero risk tolerance: confidences pushed to 0 and 1 carry no information about how answers compare to one another. We therefore introduce CARE (Cross-question Abstention Reward Estimation), which computes rewards over a set of question-answer pairs: At each of several risk tolerances, the confidences across the set induce an abstention threshold, and each confidence is rewarded for lying on the correct side of it. Since the threshold itself comes from the set, every verbalized confidence is reinforced relative to other questions' confidence, not against its own binary correctness label. On math, CARE increases the coverage at 5% risk (C@5) across MATH-500, GSM8K, and Big-Math-Digits from 0.8% to 42.4%, compared to existing pointwise correctness-matching rewards. When trained on multi-hop Wikipedia QA from HotpotQA, CARE improves C@5 in out-of-distribution general-knowledge, reasoning, and math benchmarks (including CommonsenseQA, GPQA, SimpleQA, and TriviaQA) from 12.3% to 23.6%.
Results
Selective prediction (all values in %; C@r is coverage at risk r; AURC lower is better). ID: Big-Math-Digits, MATH-500, GSM8K. OOD: CommonsenseQA, GPQA, SimpleQA, TriviaQA, HotpotQA.
| Model | ID C@0 | ID C@1 | ID C@5 | ID AURC ↓ | ID Acc | OOD C@0 | OOD C@1 | OOD C@5 | OOD AURC ↓ | OOD Acc |
|---|---|---|---|---|---|---|---|---|---|---|
| RLCR (Cold) | 0.0 | 0.0 | 0.8 | 19.3 | 73.1 | 0.0 | 0.0 | 4.9 | 38.7 | 56.6 |
| CARE (this model) | 5.4 | 18.2 | 42.4 | 12.5 | 73.1 | 2.9 | 5.4 | 15.9 | 37.7 | 53.6 |
Use
Load with AutoTokenizer.from_pretrained("hector-gr/CARE-Qwen2.5-7B-Math") and AutoModelForCausalLM.from_pretrained("hector-gr/CARE-Qwen2.5-7B-Math"). Use the exact system prompt in system_prompt.txt and a user message of the form PROBLEM: <question>. The paper evaluates with temperature 0, one completion and a 4096-token generation limit.
Outputs contain <answer>...</answer> and <confidence>...</confidence>. Rank answers by confidence and choose a threshold using suitable held-out data. A risk measured on one dataset need not transfer to a different distribution.
The training configuration is included in training_config.yaml (seed 43).
Limitations
The model can still give incorrect answers with high confidence. It has not been validated for safety-critical decisions. Model weights are released under Apache 2.0, with the base-model license included.
Attribution
Built on RLCR, Beyond Binary Rewards: Training LMs to Reason About Their Uncertainty (Damani et al., 2025), and TRL. Base model: Qwen2.5-7B, Qwen Team.
- Downloads last month
- 19
Model tree for hector-gr/CARE-Qwen2.5-7B-Math
Base model
Qwen/Qwen2.5-7B