File size: 1,531 Bytes
e957ffe
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
---
license: cc-by-nc-sa-4.0
language:
- en
tags:
- question-answering
- knowledge-retrieval
---

# Same Content, Different Format: QA-formatting models

Checkpoints of the QA-formatting experiment of *Same Content, Different Format: QA Augmentation and Faded Worked
Examples Under a Fixed Training Budget*. Each subfolder is a GPT-2-geometry decoder (162.5M parameters, 512-token
context, Dolma2 tokenizer) trained from scratch only on its arm's renderings of 5,886 OpenStax claims, at the final
continuation checkpoint (891.5 passes).

| Subfolder | Arm |
|---|---|
| `openstax_prose` | diverse prose rewrites of each claim |
| `openstax_qa` | diverse question-answer rewrites of each claim |

```python
from transformers import AutoModelForCausalLM, AutoTokenizer
tok = AutoTokenizer.from_pretrained("nzhao721/qa-formatting-models", subfolder="openstax_qa")
model = AutoModelForCausalLM.from_pretrained("nzhao721/qa-formatting-models", subfolder="openstax_qa")
```

The paper scores both with the prompt `Question: {question}\n\nAnswer:` and grades the first `Answer:` field. The
base checkpoints, the normal-fact arm and the synthetic-data preflight checkpoints are not included. The training
data and question sets are in
[nzhao721/qa-formatting-data](https://huggingface.co/datasets/nzhao721/qa-formatting-data).

These are research checkpoints trained on OpenStax-derived material, released for non-commercial research use under
CC BY-NC-SA 4.0; they are not educational QA models. Every arm was trained once with one seed.