nzhao721 commited on
Commit
4682051
·
verified ·
1 Parent(s): 5cedd3b

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +36 -0
README.md ADDED
@@ -0,0 +1,36 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: apache-2.0
3
+ language:
4
+ - en
5
+ base_model: allenai/DataDecide-dolma1_7-no-flan-300M
6
+ tags:
7
+ - math
8
+ - worked-examples
9
+ - continual-pretraining
10
+ ---
11
+
12
+ # Same Content, Different Format: worked-example models
13
+
14
+ The trained checkpoints of the worked-example experiment of *Same Content, Different Format: QA Augmentation and
15
+ Faded Worked Examples Under a Fixed Training Budget*. Each subfolder is one arm, continually pretrained from the
16
+ public DataDecide dolma1_7 no-FLAN 300M checkpoint (an OLMo 1 architecture, 371M parameters), converted to the
17
+ Hugging Face Llama format.
18
+
19
+ | Subfolder | Arm | Passes over its corpus |
20
+ |---|---|---|
21
+ | `bare` | problem and final answer only | 298 |
22
+ | `complete` | problem, full step-by-step solution and final answer | 97 |
23
+ | `fade_ordered` | faded solutions, support removed in decreasing order | 19 |
24
+ | `fade_shuffled` | the same faded documents in random order | 19 |
25
+ | `bare_pm` | bare, trained for the fade arms' pass count | 19 |
26
+ | `complete_pm` | complete, trained for the fade arms' pass count | 19 |
27
+
28
+ ```python
29
+ from transformers import AutoModelForCausalLM, AutoTokenizer
30
+ tok = AutoTokenizer.from_pretrained("nzhao721/p1-worked-examples", subfolder="complete")
31
+ model = AutoModelForCausalLM.from_pretrained("nzhao721/p1-worked-examples", subfolder="complete")
32
+ ```
33
+
34
+ The training data and the evaluation holdout are in
35
+ [nzhao721/p1-worked-examples-data](https://huggingface.co/datasets/nzhao721/p1-worked-examples-data). Every arm was
36
+ trained once with one seed, so results should be read as directional.