|
Download README.md from nzhao721/qa-formatting-models: direct link, hf CLI and curl.
- Browser
- Download file 1.53 kB
-
https://huggingface.co/nzhao721/qa-formatting-models/resolve/main/README.md
- Command line
-
hf download hf://nzhao721/qa-formatting-models/README.md
-
curl -L -o README.md https://huggingface.co/nzhao721/qa-formatting-models/resolve/main/README.md
1.53 kB
| license: cc-by-nc-sa-4.0 | |
| language: | |
| - en | |
| tags: | |
| - question-answering | |
| - knowledge-retrieval | |
| # Same Content, Different Format: QA-formatting models | |
| Checkpoints of the QA-formatting experiment of *Same Content, Different Format: QA Augmentation and Faded Worked | |
| Examples Under a Fixed Training Budget*. Each subfolder is a GPT-2-geometry decoder (162.5M parameters, 512-token | |
| context, Dolma2 tokenizer) trained from scratch only on its arm's renderings of 5,886 OpenStax claims, at the final | |
| continuation checkpoint (891.5 passes). | |
| | Subfolder | Arm | | |
| |---|---| | |
| | `openstax_prose` | diverse prose rewrites of each claim | | |
| | `openstax_qa` | diverse question-answer rewrites of each claim | | |
| ```python | |
| from transformers import AutoModelForCausalLM, AutoTokenizer | |
| tok = AutoTokenizer.from_pretrained("nzhao721/qa-formatting-models", subfolder="openstax_qa") | |
| model = AutoModelForCausalLM.from_pretrained("nzhao721/qa-formatting-models", subfolder="openstax_qa") | |
| ``` | |
| The paper scores both with the prompt `Question: {question}\n\nAnswer:` and grades the first `Answer:` field. The | |
| base checkpoints, the normal-fact arm and the synthetic-data preflight checkpoints are not included. The training | |
| data and question sets are in | |
| [nzhao721/qa-formatting-data](https://huggingface.co/datasets/nzhao721/qa-formatting-data). | |
| These are research checkpoints trained on OpenStax-derived material, released for non-commercial research use under | |
| CC BY-NC-SA 4.0; they are not educational QA models. Every arm was trained once with one seed. | |