AbdDagi/qwen3-4b-first-error-lora

First-error localisation and classification in step-by-step school-math solutions (LoRA, Qwen3-4B)

Three LoRA adapters (r=16, all linear projections) fine-tuned on 293 labelled solutions (train.csv of the task) with seeds 42, 43 and 44 on an A100. The final predictor is a deterministic ensemble of the three: the category is the argmax of the mean probability over the five category tokens, the error step is a majority vote (see src/ensemble.py in the solution archive). Prompt = system message with the five categories, 3 retrieved train examples (TF-IDF over the solution steps), the solution, and the note of a deterministic arithmetic checker. Decoding is greedy; one solution per call.

folder	seed	role
adapter_full	42	member 1
adapter_full_s43	43	member 2
adapter_full_s44	44	member 3

Reproduction (inference only, from the solution archive, Colab A100 is the reference environment):

python lora/run_lora.py infer --split test \
  --adapter lora/adapter_full,lora/adapter_full_s43,lora/adapter_full_s44 --out lora/test_pred_ens.csv
python lora/make_submission.py lora/test_pred_ens.csv submission_lora_ens.csv

Out-of-fold (3-fold by problem template): macro-F1 0.787, error-step accuracy 0.867, combined 0.811. Training: 293 examples × 3 epochs per adapter, ~10 min each on A100; test inference (1145 rows × 3 adapters) ~1 h. No paid API, no external labelled data; the test set is never used for training or tuning.