data-formatting
Collection
Models and data for Same Content, Different Format: QA Augmentation and Faded Worked Examples Under a Fixed Training Budget. โข 4 items โข Updated
The trained checkpoints of the worked-example experiment of Same Content, Different Format: QA Augmentation and Faded Worked Examples Under a Fixed Training Budget. Each subfolder is one arm, continually pretrained from the public DataDecide dolma1_7 no-FLAN 300M checkpoint (an OLMo 1 architecture, 371M parameters), converted to the Hugging Face Llama format.
| Subfolder | Arm | Passes over its corpus |
|---|---|---|
bare |
problem and final answer only | 298 |
complete |
problem, full step-by-step solution and final answer | 97 |
fade_ordered |
faded solutions, support removed in decreasing order | 19 |
fade_shuffled |
the same faded documents in random order | 19 |
bare_pm |
bare, trained for the fade arms' pass count | 19 |
complete_pm |
complete, trained for the fade arms' pass count | 19 |
from transformers import AutoModelForCausalLM, AutoTokenizer
tok = AutoTokenizer.from_pretrained("nzhao721/worked-examples-models", subfolder="complete")
model = AutoModelForCausalLM.from_pretrained("nzhao721/worked-examples-models", subfolder="complete")
The training data and the evaluation holdout are in nzhao721/worked-examples-data. Every arm was trained once with one seed, so results should be read as directional.
Base model
allenai/DataDecide-dolma1_7-no-flan-300M