|
Download README.md from nzhao721/worked-examples-models: direct link, hf CLI and curl.
- Browser
- Download file 1.55 kB
-
https://huggingface.co/nzhao721/worked-examples-models/resolve/main/README.md
- Command line
-
hf download hf://nzhao721/worked-examples-models/README.md
-
curl -L -o README.md https://huggingface.co/nzhao721/worked-examples-models/resolve/main/README.md
1.55 kB
| license: apache-2.0 | |
| language: | |
| - en | |
| base_model: allenai/DataDecide-dolma1_7-no-flan-300M | |
| tags: | |
| - math | |
| - worked-examples | |
| - continual-pretraining | |
| # Same Content, Different Format: worked-example models | |
| The trained checkpoints of the worked-example experiment of *Same Content, Different Format: QA Augmentation and | |
| Faded Worked Examples Under a Fixed Training Budget*. Each subfolder is one arm, continually pretrained from the | |
| public DataDecide dolma1_7 no-FLAN 300M checkpoint (an OLMo 1 architecture, 371M parameters), converted to the | |
| Hugging Face Llama format. | |
| | Subfolder | Arm | Passes over its corpus | | |
| |---|---|---| | |
| | `bare` | problem and final answer only | 298 | | |
| | `complete` | problem, full step-by-step solution and final answer | 97 | | |
| | `fade_ordered` | faded solutions, support removed in decreasing order | 19 | | |
| | `fade_shuffled` | the same faded documents in random order | 19 | | |
| | `bare_pm` | bare, trained for the fade arms' pass count | 19 | | |
| | `complete_pm` | complete, trained for the fade arms' pass count | 19 | | |
| ```python | |
| from transformers import AutoModelForCausalLM, AutoTokenizer | |
| tok = AutoTokenizer.from_pretrained("nzhao721/worked-examples-models", subfolder="complete") | |
| model = AutoModelForCausalLM.from_pretrained("nzhao721/worked-examples-models", subfolder="complete") | |
| ``` | |
| The training data and the evaluation holdout are in | |
| [nzhao721/worked-examples-data](https://huggingface.co/datasets/nzhao721/worked-examples-data). Every arm was | |
| trained once with one seed, so results should be read as directional. | |