mabera/math-solution-verification-classifier

Math Solution Verification Classifier

Author: Hussein Adeiza (mabera) Role: Licensed Environmental Health Officer, Abuja Nigeria Base Model: Mixtral 8x7B Fine-tuned with: AutoScientist by Adaption Labs

Model Description

A LoRA adapter fine-tuned to verify whether a proposed answer to a real competition math problem is correct, classifying it as Correct or Incorrect. This is a genuinely global-scope submission (not Nigeria- specific), addressing a universal AI capability question: can a model reliably grade mathematical correctness?

Training Data

• Source: MATH dataset (Hendrycks et al., NeurIPS 2021), accessed via a properly-cited GitHub derivative (rasbt/math_full_minus_math500), downloaded directly, 12,000 real competition problems
• Dataset: 20 rows (10 real problems x 2 answer variants each: one correct, one deliberately perturbed incorrect answer, disclosed as a constructed perturbation, not a real student error)
• Kaggle: https://www.kaggle.com/datasets/yunusahusseinadeiza/math-solution-verification-classifier

Important Note: Column Selection Correction

During training setup, the platform defaulted to training on "Enhanced completion" text, which had drifted away from the closed- label Correct/Incorrect structure into generic step-by-step tutoring language, losing the classification task entirely. This was caught and manually corrected by selecting "Original completion" instead before training. Worth flagging for other builders working on closed-label tasks: check which completion column is actually selected before training, since the platform default may not be the one you expect.

Training Metrics

• Win rate (on dataset): 76% adapted vs 24% base model
• Base model: mistralai/Mixtral-8x7B-Instruct-v0.1
• Method: LoRA (confirmed via training config), no recipe modifications
• Dataset quality: 7.0 → 9.0 (+28.6% relative improvement, Grade A)
• Percentile: 33.0
• Domain classification: Math (100%), a clean, accurate match

Verification

All 20 rows independently, programmatically verified before training: every Correct/Incorrect classification checked against the real ground-truth answer looked up directly in the raw MATH dataset source file. 20/20 pass rate, demonstrated live in the accompanying Kaggle notebook.

Why This Result Matters

This is the highest quality grade (A) and highest quality score improvement (+28.6%) in this author's 18-submission AutoScientist portfolio, achieved using the same disciplined closed-label methodology (no recipe modifications, deterministic ground truth, independent verification) established across the AMR and Loan Classifier submissions, applied here to a genuinely global rather than Nigeria-specific problem.

Credits

Powered by Adaptive Data — Adaption Labs AutoScientist Challenge 2026, Part 2 — Math & Code Category