MATH-247 GPT-2 Fine-Tuning Pipeline

Fine-tune GPT-2 on McGill MATH 247 (Honours Applied Linear Algebra) course notes, then push the trained model to 13ilguun/MATH-247 on Hugging Face.

Setup

pip install -r requirements.txt

Get your HF write token from https://huggingface.co/settings/tokens

---

Step 1 — Prepare & push dataset

python 1_prepare_dataset.py \
  --pdf     path/to/MATH247_2026.pdf \
  --repo    13ilguun/MATH-247-dataset \
  --token   hf_YOUR_TOKEN \
  --save-local              # optional: also saves math247_dataset.jsonl locally

This will:

• Extract text from the PDF
• Chunk it at Definition/Theorem/Lemma boundaries (~400 tokens each)
• Push a train/val split to 13ilguun/MATH-247-dataset on HF

Note: The dataset repo needs to be created first on huggingface.co (New → Dataset, name it MATH-247-dataset).

---

Step 2 — Fine-tune GPT-2

python 2_finetune.py \
  --dataset_repo  13ilguun/MATH-247-dataset \
  --model_repo    13ilguun/MATH-247 \
  --token         hf_YOUR_TOKEN

Training config (adjust in script for your VRAM):

VRAM	per_device_batch	grad_accum	Effective batch
8 GB	4	4	16
12 GB	8	2	16
24 GB	16	1	16

Training takes ~5–15 min on a modern GPU for 5 epochs over ~500 samples. Expect final perplexity in the 20–50 range (reasonable for domain-specific math text).

---

Step 3 — Run inference / study

# Interactive study session
python 3_inference.py --model_repo 13ilguun/MATH-247

# Or run demo prompts
python 3_inference.py --model_repo 13ilguun/MATH-247 --demo

Example prompts to try:

• Definition (Eigenvalue). Let f be an endomorphism of V. A scalar λ is called
• Theorem (Rank-Nullity). Let f: V → W be a linear map. Then
• Proof. Let B = {b1, ..., bn} be a basis. We want to show

---

Model card

After training, add a model card on huggingface.co/13ilguun/MATH-247 describing:

• Base model: gpt2
• Fine-tuned on: MATH 247 lecture notes (McGill, Winter 2026)
• Task: Causal language modeling / math text completion