Add model card

README.md

@@ -0,0 +1,43 @@
+---
+license: mit
+language:
+- en
+library_name: transformers
+pipeline_tag: text-generation
+tags:
+- gpt2
+datasets:
+- craa/exceptions
+---
+
+# exceptions
+
+GPT-2 language models trained from scratch for the exceptions project
+(Leong and Linzen, [Language Models Can Learn Exceptions to Syntactic Rules](https://doi.org/10.7275/h25z-0y75), and follow-up experiments).
+Code: [github.com/craaaa/exceptions](https://github.com/craaaa/exceptions).
+
+All models are GPT-2 small (12 layers, 12 heads, 768 hidden). Each folder holds the final model of one training run
+(best checkpoint by validation loss), with its tokenizer and Trainer logs (`trainer_state.json`, `all_results.json`).
+
+## Contents
+
+| Folder | Runs | Naming | Seeds | Vocab |
+|---|---|---|---|---|
+| `exp1b/` | 5 | `100M_<seed>` | 1001, 2128, 3591, 5039, 40817 | 50,257 (GPT-2) |
+| `exp2a/` | 60 | `<cond>_to_<verb>_frequency_<seed>` — cond ∈ {cost, last, resemble}, verb ∈ {carry, drop, hit, push} | 1001, 2128, 3591, 5039, 40817 | 50,257 (GPT-2) |
+| `exp3/` | 170 | `{high,low}_<n>_<seed>` — n ∈ {0, 1, 2, 4, 8, 16, 32, 64, 10, 100, 128, 256, 500, 512, 1000, 1500, 2000} | 495, 634, 1208, 6910, 8397 | 52,000 (custom) |
+
+Training corpora are in the dataset repo [`craa/exceptions`](https://huggingface.co/datasets/craa/exceptions);
+the `exp2a` folder names correspond to its `frequency_<cond>_to_<verb>` configs.
+
+## Usage
+
+```python
+from transformers import AutoModelForCausalLM, AutoTokenizer
+
+repo, run = "craa/exceptions", "exp3/high_0_1208"
+model = AutoModelForCausalLM.from_pretrained(repo, subfolder=run)
+tokenizer = AutoTokenizer.from_pretrained(repo, subfolder=run)
+```
+
+Intermediate checkpoints are not published here.