File size: 1,546 Bytes
4682051 5a31845 4682051 5a31845 4682051 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 | ---
license: apache-2.0
language:
- en
base_model: allenai/DataDecide-dolma1_7-no-flan-300M
tags:
- math
- worked-examples
- continual-pretraining
---
# Same Content, Different Format: worked-example models
The trained checkpoints of the worked-example experiment of *Same Content, Different Format: QA Augmentation and
Faded Worked Examples Under a Fixed Training Budget*. Each subfolder is one arm, continually pretrained from the
public DataDecide dolma1_7 no-FLAN 300M checkpoint (an OLMo 1 architecture, 371M parameters), converted to the
Hugging Face Llama format.
| Subfolder | Arm | Passes over its corpus |
|---|---|---|
| `bare` | problem and final answer only | 298 |
| `complete` | problem, full step-by-step solution and final answer | 97 |
| `fade_ordered` | faded solutions, support removed in decreasing order | 19 |
| `fade_shuffled` | the same faded documents in random order | 19 |
| `bare_pm` | bare, trained for the fade arms' pass count | 19 |
| `complete_pm` | complete, trained for the fade arms' pass count | 19 |
```python
from transformers import AutoModelForCausalLM, AutoTokenizer
tok = AutoTokenizer.from_pretrained("nzhao721/worked-examples-models", subfolder="complete")
model = AutoModelForCausalLM.from_pretrained("nzhao721/worked-examples-models", subfolder="complete")
```
The training data and the evaluation holdout are in
[nzhao721/worked-examples-data](https://huggingface.co/datasets/nzhao721/worked-examples-data). Every arm was
trained once with one seed, so results should be read as directional.
|