rtharrin/Qwen2.5-3B-ProxyMath-Merged

Qwen2.5-3B-ProxyMath-LoRA-Merged

1. Introduction

This repository explores the problem of domain adaptation and data scarcity in industrial LLM deployment, where we often lack the specialized datasets needed for a particular task but similar, tangential datasets may already exist. While current language models excel at standardized reasoning benchmarks when specifically aligned, they face severe performance drop-offs when forced to handle reasoning tasks through generalized instruction tuning alone. To address this scenario, this project tries to simulate a data-scarce environment by optimizing a "small" parameter base model on an out-of-task math benchmark without using target-domain fine-tuning data. Using a parameter-efficient fine-tuning intervention, the final model achieved an optimization trade-off: it successfully boosted strict Exact Match (EM) logic accuracy on the target benchmark by roughly 75% while experiencing expected degradation on flexible extraction and general-knowledge tasks.

2. Data

The training data was constructed from the amphora/QwQ-LongCoT-130K dataset, specifically isolated to instances originating from NuminaMath to model a large, tangentially related data silo. To focus strictly on proxy reasoning volume, the dataset was stripped down to a localized slice of 90,000 math reasoning samples, which were further reformatted with customized prompt configurations mapping explicit "Q: " query prefixes and " \n\nA: " completion splits. The final data pipeline enforced an 80/20 train/validation split resulting in 72,000 training records and 18,065 validation tokens, randomized strictly with a constant seed (random_state=42) to maintain reproducible training distribution parameters. Evaluation was conducted over the complete gsm8k testing split containing 1,319 benchmark entries alongside specialized generalization evaluations.

3. Methodology

Low-Rank Adaptation (LoRA) was selected over full fine-tuning to mitigate high compute overhead and prevent the model from catastrophically overfitting to the specific nuances of the proxy math space at the expense of target benchmark flexibility. To capture deep mathematical pathways, the final intervention applied LoRA layers across all available linear modules utilizing an aggressive scaling config of rank (r=64) and alpha (\alpha=64), accompanied by a dropout rate of (0.05). Training executed over exactly 2 epochs via the SFTTrainer abstraction under a learning rate of (1\times10^{-5}) utilizing automated batch allocation. Following optimization, the finalized adapter tensors were cleanly integrated back into the core parameter architecture using a standard merge_and_unload() workflow to produce a deployable standalone checkpoint.

4. Evaluation

Our evaluation covers four target domains across arithmetic chain-of-thought, reading comprehension, factual alignment, and broad multi-disciplinary science domains. GSM8K CoT tracks raw problem-solving capability and was our target benchmark to increase; MMLU provides a calibrated measure of foundational knowledge retention across STEM and social domains; RACE assesses general contextual synthesis; and TruthfulQA quantifies post-tuning hallucinatory trends, as many fine tuning methods are known to create overconfidence. To establish a validated architectural baseline, we benchmark against the original instruction-tuned base model alongside two comparison setups of equal 3B scale: an identical Qwen iteration trained over a minimal 2% hyper-curated slice of direct task-specific data (340 rows of gsm8k), and an un-tuned baseline variant. Relative to these comparison targets, the proxy-trained model demonstrates superior structural alignment by boosting strict target math match constraints by 75%.

Benchmark Task / Split	Our Model (Numina-Proxy)	Curated Baseline (2% GSM8K)	Base Qwen2.5-3B-Instruct (Baseline)
GSM8K CoT (Strict EM)	0.0599	0.0281	0.0341
GSM8K CoT (Flexible Extract)	0.5254	0.5921	0.5974
MMLU (Overall Acc)	0.5264	0.6463	0.6463
RACE (Acc)	0.4086	0.4220	0.4268
TruthfulQA MC2 (Acc)	0.5586	0.5733	0.5714

5. Usage and Intended Uses

This model is intended strictly for experimental study in cross-domain math adaptation, low-resource proxy simulation, and downstream architectural benchmarking of arithmetic reasoning scripts. It is not designed to operate as a general-purpose conversational agent or reliable multi-turn assistant outside of step-by-step math evaluation pipelines. It is meant to function as a case study of training “near but not quite right” data, trying to demonstrate that sometimes re-using existing data can be a cost-effective alternative to generating a curated data-set for all fine-tuning tasks.

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_path = "rtharrin/Qwen2.5-3B-ProxyMath-Merged"

tokenizer = AutoTokenizer.from_pretrained(model_path)
model = AutoModelForCausalLM.from_pretrained(
    model_path,
    torch_dtype=torch.bfloat16,
    device_map="auto"
)

prompt = "Q: If a rectangle has a length of 8 and a width of 4, what is its area? Let's think step by step. \n\nA:"
inputs = tokenizer(prompt, return_tensors="pt").to("cuda")
outputs = model.generate(**inputs, max_new_tokens=250, do_sample=False)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

6. Prompt Format

Prompts should be structured explicitly using a fixed question header combined with a chain-of-thought induction suffix to properly engage the mathematical reasoning behavior learned during proxy training.

Q: [Your math problem or reasoning query here] Let's think step by step. 

A:

7. Expected Output Format

The model is optimized to yield an immediate, deterministic chain-of-thought derivation directly trailing the completion anchor prefix, concluding with a distinct terminal numeric solution.

To solve this problem, we need to calculate the area using the formula...
Step 1: Identify length (8) and width (4).
Step 2: Multiply 8 * 4.
The area is 32.

8. Limitations

The primary limitation of this model centers on the flexible extraction degradation and general-knowledge decay resulting from optimized domain alignment. While strict adherence to formatting improved by 75% on math tasks, the model's overall flexible response generation on GSM8K fell by ~13%, indicating that out-of-task training can narrow down output expressions too rigidly. Furthermore, cross-entropy minimization on NuminaMath samples induced catastrophic forgetting across historical, scientific, and verbal vectors, as evidenced by a substantial drop in standard MMLU accuracy from 64.6% down to 52.6%. Consequently, the model should not be relied upon for tasks requiring factual breadth, non-mathematical inference, or flexible text extraction formats.