--- license: apache-2.0 language: - en base_model: allenai/DataDecide-dolma1_7-no-flan-300M tags: - math - worked-examples - continual-pretraining --- # Same Content, Different Format: worked-example models The trained checkpoints of the worked-example experiment of *Same Content, Different Format: QA Augmentation and Faded Worked Examples Under a Fixed Training Budget*. Each subfolder is one arm, continually pretrained from the public DataDecide dolma1_7 no-FLAN 300M checkpoint (an OLMo 1 architecture, 371M parameters), converted to the Hugging Face Llama format. | Subfolder | Arm | Passes over its corpus | |---|---|---| | `bare` | problem and final answer only | 298 | | `complete` | problem, full step-by-step solution and final answer | 97 | | `fade_ordered` | faded solutions, support removed in decreasing order | 19 | | `fade_shuffled` | the same faded documents in random order | 19 | | `bare_pm` | bare, trained for the fade arms' pass count | 19 | | `complete_pm` | complete, trained for the fade arms' pass count | 19 | ```python from transformers import AutoModelForCausalLM, AutoTokenizer tok = AutoTokenizer.from_pretrained("nzhao721/worked-examples-models", subfolder="complete") model = AutoModelForCausalLM.from_pretrained("nzhao721/worked-examples-models", subfolder="complete") ``` The training data and the evaluation holdout are in [nzhao721/worked-examples-data](https://huggingface.co/datasets/nzhao721/worked-examples-data). Every arm was trained once with one seed, so results should be read as directional.