jedld commited on
Commit
331e539
·
verified ·
1 Parent(s): 4d3a4db

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +3 -204
README.md CHANGED
@@ -1,204 +1,3 @@
1
- ---
2
- language:
3
- - en
4
- license: other
5
- library_name: pytorch
6
- tags:
7
- - mtg
8
- - magic-the-gathering
9
- - reinforcement-learning
10
- - system-one
11
- - choice-model
12
- - distillation
13
- ---
14
-
15
- # Model card — System One Choice (`SystemOneChoice`)
16
-
17
- **Hub:** [jedld/system-one-choice](https://huggingface.co/jedld/system-one-choice)
18
- (override with `MTGCARDS_HF_MODEL`). Download:
19
-
20
- ```bash
21
- python -m mtgcards.fetch_assets --ckpt
22
- # or evaluate without a local tree:
23
- python -m mtgcards.rl.evaluate --ckpt hf:jedld/system-one-choice --vs greedy --games 10
24
- ```
25
-
26
- Local **JEV-shaped text Choice** student for Magic: The Gathering play
27
- inside this engine. Not an autoregressive chatbot: one forward pass
28
- scores every legal action and returns `{choice, probabilities, confidence}`.
29
-
30
- Canonical code: [`mtgcards/rl/model.py`](https://github.com/jedld/mtgcards/blob/master/mtgcards/rl/model.py)
31
- (`arch=choice_text_v1`). Training walkthrough:
32
- [`docs/SYSTEM_ONE.md`](https://github.com/jedld/mtgcards/blob/master/docs/SYSTEM_ONE.md). Self-improvement loop:
33
- [`mtgcards/rl/SELF_IMPROVE.md`](https://github.com/jedld/mtgcards/blob/master/mtgcards/rl/SELF_IMPROVE.md).
34
-
35
- ## Model details
36
-
37
- | | |
38
- | --- | --- |
39
- | **Developed by** | mtgcards / System One stack |
40
- | **Model type** | Bidirectional Transformer encoder + parallel criteria scorer + value head |
41
- | **Language(s)** | English game text (engine `render_state` / `action_options`) |
42
- | **License** | Code follows the repo MIT license. Weights are **not** MIT game content: FineWeb sample is ODC-By-1.0; Comprehensive Rules © Wizards of the Coast; card oracle text subject to Wizards terms. See repo [`NOTICE`](https://github.com/jedld/mtgcards/blob/master/NOTICE). |
43
- | **Finetuned from** | Random init → causal LM warm-start on `data/systemone_corpus` → BC → PPO |
44
-
45
- ### Architecture (`choice_text_v1`)
46
-
47
- 1. **Byte-level `TextVocab`** — 5 specials + 256 UTF-8 bytes (size 261). Checkpoint `text_vocab.json` is a kind/size marker; the mapping is fixed in code.
48
- 2. Shared **TransformerEncoder** trunk (`norm_first`, GELU FFN = 2×`d_model`).
49
- 3. Encode **state** once → mean-pool → `state_proj`.
50
- 4. Encode each criterion `"key: label"` with the same trunk → pool → `crit_proj`.
51
- 5. Score `(state, crit, state⊙crit)` with a 2-layer MLP → logits over K options.
52
- 6. Softmax → probabilities; **TypeSafe confidence** `(p_max − 1/n) / (1 − 1/n)`.
53
- 7. **Value head** on the state pool (PPO baseline).
54
-
55
- Legacy closed-vocab CAND policy remains loadable as
56
- `LegacySystemOnePolicy` (`--arch legacy`).
57
-
58
- ### Size presets
59
-
60
- | Preset | `d_model` | layers | heads | ~params | state bytes |
61
- | --- | ---: | ---: | ---: | ---: | ---: |
62
- | Default CLI | 192 | 4 | 6 | ~1.4M | 8192 (CLI default) |
63
- | **choice_fresh (this run)** | 384 | 6 | 6 | ~11.1M | 8192 |
64
- | Smoke notebook | 64 | 2 | 4 | small | 384 |
65
-
66
- Criteria are capped at `--max-crit` (default 64 bytes each).
67
-
68
- ## Intended use
69
-
70
- - **In-scope:** Seat-balanced play inside this rules engine (TUI, Arena,
71
- Gym, evaluate harness); BC from greedy/JEV; PPO vs a league; soft-label
72
- distillation from captured `jev_request` / `probs`.
73
- - **Out-of-scope:** Real Magic Online / Arena clients; claiming parity with
74
- hosted TypeSafe JEV; Noul/Score/image inputs; legal advice or tournament
75
- rulings outside the engine’s known simplifications.
76
-
77
- ## Training data
78
-
79
- ### Text LM warm-start (`data/systemone_corpus`)
80
-
81
- Built by `python -m mtgcards.rl.corpus build`. Manifest after the
82
- 2026-11-20 rebuild (`--fineweb-docs 8000`): **13 953 docs / ~58M chars**.
83
-
84
- | Source | Docs (approx.) | Notes |
85
- | --- | ---: | --- |
86
- | FineWeb `sample-10BT` | 8000 | ODC-By-1.0; bounded stream sample, not the full FineWeb |
87
- | Card catalog + oracle JSON | ~5800 | Engine cards + Scryfall stubs under `data/cards` |
88
- | Game logs | ~48 | `data/game_logs/**/*.jsonl` when present |
89
- | Comprehensive Rules | 1 + chunks | Wizards plain-text CR |
90
-
91
- Causal LM uses a **capped** window (`min(max_state, 2048)`) even when
92
- Choice state budget is larger (attention is O(T²)).
93
-
94
- ### Decision training
95
-
96
- - **BC teacher (default):** `GreedyAgent` rollouts recording
97
- `state_text` / `options` (JEV surface; distillation parity omits YAML
98
- strategy prose).
99
- - **Optional teachers:**
100
- - JEV Choice API (`--bc-source jev`) — soft `probs` + `jev_request`.
101
- - **Hybrid Strata+JEV** (`--bc-source hybrid` or
102
- `trajectory collect --teacher hybrid`) — Strata writes a main-phase
103
- plan; JEV executes with that plan (and optional combat advice) in
104
- the Choice state. Soft labels are JEV’s distribution; the stored
105
- state **includes** the strategist plan (strategist dataset).
106
- - **PPO:** self-play vs league (past checkpoints + scripted baselines);
107
- reward = terminal seat outcome ±1.
108
- - **Paraphrase augmentation:** `python -m mtgcards.rl.paraphrase` restates
109
- state/option text via Strata while freezing option keys + soft labels
110
- (`data/trajectories/*_para.pkl`).
111
-
112
- **Hidden information:** the deciding seat never sees opponent hand
113
- identities, library order, or uncast cards
114
- ([`tests/test_hidden_info.py`](https://github.com/jedld/mtgcards/blob/master/tests/test_hidden_info.py)).
115
-
116
- ## Training procedure (choice_fresh recipe)
117
-
118
- ```bash
119
- python -m mtgcards.rl.corpus build --fineweb-docs 8000
120
-
121
- python -m mtgcards.rl.train \
122
- --arch choice --device cuda \
123
- --out checkpoints/choice_fresh \
124
- --bank data/trajectories/choice_fresh.pkl \
125
- --d-model 384 --layers 6 --heads 6 \
126
- --max-tokens 8192 --max-crit 64 \
127
- --batch-size 2 --lr 1.5e-4 \
128
- --pretrain-corpus data/systemone_corpus/corpus.jsonl \
129
- --pretrain-epochs 2 \
130
- --bc-games 16 --bc-epochs 6 \
131
- --iters 20 --games 12 --epochs 3 \
132
- --log-games 2 --log-dir data/game_logs/choice_fresh
133
-
134
- # later: soft-label distill from hybrid bank onto a PPO ckpt
135
- python -m mtgcards.rl.train --resume checkpoints/choice_fresh/policy_NNNN \
136
- --bc-source bank:data/trajectories/hybrid_strategist_plus_para.pkl \
137
- --bc-epochs 8 --iters 10 --games 12 --device cuda ...
138
- ```
139
-
140
- See [`data/README.md`](https://github.com/jedld/mtgcards/blob/master/data/README.md) for the full dataset tree.
141
-
142
- Stages: corpus LM → greedy BC → PPO league → hybrid soft distill
143
- (+ optional PPO). Checkpoints: `checkpoints/choice_fresh/policy_NNNN/`
144
- with `model.pt`, `text_vocab.json`, `meta.json`.
145
-
146
- **Hub package (this card):** weights from **`policy_0023`** — hybrid
147
- strategist soft BC on top of the choice_fresh stack (~11.1M params,
148
- `d_model=384`, 6 layers). League `best.json` may point at a later PPO
149
- iter with a tiny N=12 win-rate spike; do not treat that as ladder
150
- evidence.
151
-
152
- ## Evaluation
153
-
154
- Primary metric: **seat-balanced win rate** vs a fixed opponent.
155
-
156
- ```bash
157
- python -m mtgcards.rl.evaluate \
158
- --ckpt checkpoints/choice_fresh/policy_best \
159
- --vs greedy --games 40 --device cuda
160
- ```
161
-
162
- Sparse-reward honesty: do not treat small-budget win rates as ladder
163
- evidence; always report game count. JEV soft-label agreement and
164
- judge-LLM audits (`--log-games`) are secondary signals.
165
-
166
- | Checkpoint | vs greedy (N games) | Soft agree (JEV-labelled) | Notes |
167
- | --- | --- | --- | --- |
168
- | `policy_0019` | 0.42 (12, PPO sample) | — | early PPO plateau; noisy |
169
- | `policy_0022` | 0.36 (11 scored / 12) | train 0.63 / held-out 0.51 | first hybrid+para distill |
170
- | `policy_0023` | *(pending ≥20-game eval)* | train 0.65 / held-out 0.53 | **Hub candidate**; larger hybrid bank BC |
171
- | `policy_0024` | 1.00 (12, PPO sample) | — | league “best” after distill; **N=12 fluke** — not published |
172
-
173
- ## Limitations and risks
174
-
175
- - Engine fidelity gaps (see README “Known simplifications”) transfer to
176
- the policy.
177
- - Long states are **byte-truncated** (tail-preferring); very long games
178
- still lose early history even at 8192.
179
- - Greedy BC alone clones a weak teacher; strength needs JEV distill
180
- and/or enough PPO.
181
- - Hosted JEV is stronger zero-shot; this student is the cheap local
182
- policy, not a drop-in API replacement.
183
- - Do not deploy as an authority on real-world Magic rules disputes.
184
-
185
- ## Environmental impact
186
-
187
- Training is local GPU (e.g. single RTX 3090). Wall-clock for the
188
- choice_fresh recipe is on the order of hours (LM + BC dominate before
189
- PPO). Exact kWh not measured.
190
-
191
- ## Citation / links
192
-
193
- If you use System One weights or this training stack in research, cite the
194
- software via the repo [`CITATION.cff`](https://github.com/jedld/mtgcards/blob/master/CITATION.cff) /
195
- [`CITATION.bib`](https://github.com/jedld/mtgcards/blob/master/CITATION.bib) (and note training-data lineage in
196
- [`NOTICE`](https://github.com/jedld/mtgcards/blob/master/NOTICE)).
197
-
198
- - Hugging Face model: https://huggingface.co/jedld/system-one-choice
199
- - TypeSafe System One concepts: https://docs.typesafe.ai/concepts/system-one
200
- - FineWeb: https://huggingface.co/datasets/HuggingFaceFW/fineweb
201
- - Repo docs: [`SYSTEM_ONE.md`](https://github.com/jedld/mtgcards/blob/master/docs/SYSTEM_ONE.md), [`SELF_IMPROVE.md`](https://github.com/jedld/mtgcards/blob/master/mtgcards/rl/SELF_IMPROVE.md)
202
- - Staging (maintainers): `checkpoints/hf_staging/jedld-system-one-choice/`
203
- (weights = `policy_0023` + this card as `README.md`). Upload:
204
- `huggingface-cli upload jedld/system-one-choice checkpoints/hf_staging/jedld-system-one-choice . --repo-type model`
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:099b2b944e6602a8128581153b2788e3e49ced3799b8bfd9dfac6d5d71f3678f
3
+ size 9468