Upload README.md with huggingface_hub
Browse files
README.md
CHANGED
|
@@ -1,204 +1,3 @@
|
|
| 1 |
-
-
|
| 2 |
-
|
| 3 |
-
|
| 4 |
-
license: other
|
| 5 |
-
library_name: pytorch
|
| 6 |
-
tags:
|
| 7 |
-
- mtg
|
| 8 |
-
- magic-the-gathering
|
| 9 |
-
- reinforcement-learning
|
| 10 |
-
- system-one
|
| 11 |
-
- choice-model
|
| 12 |
-
- distillation
|
| 13 |
-
---
|
| 14 |
-
|
| 15 |
-
# Model card — System One Choice (`SystemOneChoice`)
|
| 16 |
-
|
| 17 |
-
**Hub:** [jedld/system-one-choice](https://huggingface.co/jedld/system-one-choice)
|
| 18 |
-
(override with `MTGCARDS_HF_MODEL`). Download:
|
| 19 |
-
|
| 20 |
-
```bash
|
| 21 |
-
python -m mtgcards.fetch_assets --ckpt
|
| 22 |
-
# or evaluate without a local tree:
|
| 23 |
-
python -m mtgcards.rl.evaluate --ckpt hf:jedld/system-one-choice --vs greedy --games 10
|
| 24 |
-
```
|
| 25 |
-
|
| 26 |
-
Local **JEV-shaped text Choice** student for Magic: The Gathering play
|
| 27 |
-
inside this engine. Not an autoregressive chatbot: one forward pass
|
| 28 |
-
scores every legal action and returns `{choice, probabilities, confidence}`.
|
| 29 |
-
|
| 30 |
-
Canonical code: [`mtgcards/rl/model.py`](https://github.com/jedld/mtgcards/blob/master/mtgcards/rl/model.py)
|
| 31 |
-
(`arch=choice_text_v1`). Training walkthrough:
|
| 32 |
-
[`docs/SYSTEM_ONE.md`](https://github.com/jedld/mtgcards/blob/master/docs/SYSTEM_ONE.md). Self-improvement loop:
|
| 33 |
-
[`mtgcards/rl/SELF_IMPROVE.md`](https://github.com/jedld/mtgcards/blob/master/mtgcards/rl/SELF_IMPROVE.md).
|
| 34 |
-
|
| 35 |
-
## Model details
|
| 36 |
-
|
| 37 |
-
| | |
|
| 38 |
-
| --- | --- |
|
| 39 |
-
| **Developed by** | mtgcards / System One stack |
|
| 40 |
-
| **Model type** | Bidirectional Transformer encoder + parallel criteria scorer + value head |
|
| 41 |
-
| **Language(s)** | English game text (engine `render_state` / `action_options`) |
|
| 42 |
-
| **License** | Code follows the repo MIT license. Weights are **not** MIT game content: FineWeb sample is ODC-By-1.0; Comprehensive Rules © Wizards of the Coast; card oracle text subject to Wizards terms. See repo [`NOTICE`](https://github.com/jedld/mtgcards/blob/master/NOTICE). |
|
| 43 |
-
| **Finetuned from** | Random init → causal LM warm-start on `data/systemone_corpus` → BC → PPO |
|
| 44 |
-
|
| 45 |
-
### Architecture (`choice_text_v1`)
|
| 46 |
-
|
| 47 |
-
1. **Byte-level `TextVocab`** — 5 specials + 256 UTF-8 bytes (size 261). Checkpoint `text_vocab.json` is a kind/size marker; the mapping is fixed in code.
|
| 48 |
-
2. Shared **TransformerEncoder** trunk (`norm_first`, GELU FFN = 2×`d_model`).
|
| 49 |
-
3. Encode **state** once → mean-pool → `state_proj`.
|
| 50 |
-
4. Encode each criterion `"key: label"` with the same trunk → pool → `crit_proj`.
|
| 51 |
-
5. Score `(state, crit, state⊙crit)` with a 2-layer MLP → logits over K options.
|
| 52 |
-
6. Softmax → probabilities; **TypeSafe confidence** `(p_max − 1/n) / (1 − 1/n)`.
|
| 53 |
-
7. **Value head** on the state pool (PPO baseline).
|
| 54 |
-
|
| 55 |
-
Legacy closed-vocab CAND policy remains loadable as
|
| 56 |
-
`LegacySystemOnePolicy` (`--arch legacy`).
|
| 57 |
-
|
| 58 |
-
### Size presets
|
| 59 |
-
|
| 60 |
-
| Preset | `d_model` | layers | heads | ~params | state bytes |
|
| 61 |
-
| --- | ---: | ---: | ---: | ---: | ---: |
|
| 62 |
-
| Default CLI | 192 | 4 | 6 | ~1.4M | 8192 (CLI default) |
|
| 63 |
-
| **choice_fresh (this run)** | 384 | 6 | 6 | ~11.1M | 8192 |
|
| 64 |
-
| Smoke notebook | 64 | 2 | 4 | small | 384 |
|
| 65 |
-
|
| 66 |
-
Criteria are capped at `--max-crit` (default 64 bytes each).
|
| 67 |
-
|
| 68 |
-
## Intended use
|
| 69 |
-
|
| 70 |
-
- **In-scope:** Seat-balanced play inside this rules engine (TUI, Arena,
|
| 71 |
-
Gym, evaluate harness); BC from greedy/JEV; PPO vs a league; soft-label
|
| 72 |
-
distillation from captured `jev_request` / `probs`.
|
| 73 |
-
- **Out-of-scope:** Real Magic Online / Arena clients; claiming parity with
|
| 74 |
-
hosted TypeSafe JEV; Noul/Score/image inputs; legal advice or tournament
|
| 75 |
-
rulings outside the engine’s known simplifications.
|
| 76 |
-
|
| 77 |
-
## Training data
|
| 78 |
-
|
| 79 |
-
### Text LM warm-start (`data/systemone_corpus`)
|
| 80 |
-
|
| 81 |
-
Built by `python -m mtgcards.rl.corpus build`. Manifest after the
|
| 82 |
-
2026-11-20 rebuild (`--fineweb-docs 8000`): **13 953 docs / ~58M chars**.
|
| 83 |
-
|
| 84 |
-
| Source | Docs (approx.) | Notes |
|
| 85 |
-
| --- | ---: | --- |
|
| 86 |
-
| FineWeb `sample-10BT` | 8000 | ODC-By-1.0; bounded stream sample, not the full FineWeb |
|
| 87 |
-
| Card catalog + oracle JSON | ~5800 | Engine cards + Scryfall stubs under `data/cards` |
|
| 88 |
-
| Game logs | ~48 | `data/game_logs/**/*.jsonl` when present |
|
| 89 |
-
| Comprehensive Rules | 1 + chunks | Wizards plain-text CR |
|
| 90 |
-
|
| 91 |
-
Causal LM uses a **capped** window (`min(max_state, 2048)`) even when
|
| 92 |
-
Choice state budget is larger (attention is O(T²)).
|
| 93 |
-
|
| 94 |
-
### Decision training
|
| 95 |
-
|
| 96 |
-
- **BC teacher (default):** `GreedyAgent` rollouts recording
|
| 97 |
-
`state_text` / `options` (JEV surface; distillation parity omits YAML
|
| 98 |
-
strategy prose).
|
| 99 |
-
- **Optional teachers:**
|
| 100 |
-
- JEV Choice API (`--bc-source jev`) — soft `probs` + `jev_request`.
|
| 101 |
-
- **Hybrid Strata+JEV** (`--bc-source hybrid` or
|
| 102 |
-
`trajectory collect --teacher hybrid`) — Strata writes a main-phase
|
| 103 |
-
plan; JEV executes with that plan (and optional combat advice) in
|
| 104 |
-
the Choice state. Soft labels are JEV’s distribution; the stored
|
| 105 |
-
state **includes** the strategist plan (strategist dataset).
|
| 106 |
-
- **PPO:** self-play vs league (past checkpoints + scripted baselines);
|
| 107 |
-
reward = terminal seat outcome ±1.
|
| 108 |
-
- **Paraphrase augmentation:** `python -m mtgcards.rl.paraphrase` restates
|
| 109 |
-
state/option text via Strata while freezing option keys + soft labels
|
| 110 |
-
(`data/trajectories/*_para.pkl`).
|
| 111 |
-
|
| 112 |
-
**Hidden information:** the deciding seat never sees opponent hand
|
| 113 |
-
identities, library order, or uncast cards
|
| 114 |
-
([`tests/test_hidden_info.py`](https://github.com/jedld/mtgcards/blob/master/tests/test_hidden_info.py)).
|
| 115 |
-
|
| 116 |
-
## Training procedure (choice_fresh recipe)
|
| 117 |
-
|
| 118 |
-
```bash
|
| 119 |
-
python -m mtgcards.rl.corpus build --fineweb-docs 8000
|
| 120 |
-
|
| 121 |
-
python -m mtgcards.rl.train \
|
| 122 |
-
--arch choice --device cuda \
|
| 123 |
-
--out checkpoints/choice_fresh \
|
| 124 |
-
--bank data/trajectories/choice_fresh.pkl \
|
| 125 |
-
--d-model 384 --layers 6 --heads 6 \
|
| 126 |
-
--max-tokens 8192 --max-crit 64 \
|
| 127 |
-
--batch-size 2 --lr 1.5e-4 \
|
| 128 |
-
--pretrain-corpus data/systemone_corpus/corpus.jsonl \
|
| 129 |
-
--pretrain-epochs 2 \
|
| 130 |
-
--bc-games 16 --bc-epochs 6 \
|
| 131 |
-
--iters 20 --games 12 --epochs 3 \
|
| 132 |
-
--log-games 2 --log-dir data/game_logs/choice_fresh
|
| 133 |
-
|
| 134 |
-
# later: soft-label distill from hybrid bank onto a PPO ckpt
|
| 135 |
-
python -m mtgcards.rl.train --resume checkpoints/choice_fresh/policy_NNNN \
|
| 136 |
-
--bc-source bank:data/trajectories/hybrid_strategist_plus_para.pkl \
|
| 137 |
-
--bc-epochs 8 --iters 10 --games 12 --device cuda ...
|
| 138 |
-
```
|
| 139 |
-
|
| 140 |
-
See [`data/README.md`](https://github.com/jedld/mtgcards/blob/master/data/README.md) for the full dataset tree.
|
| 141 |
-
|
| 142 |
-
Stages: corpus LM → greedy BC → PPO league → hybrid soft distill
|
| 143 |
-
(+ optional PPO). Checkpoints: `checkpoints/choice_fresh/policy_NNNN/`
|
| 144 |
-
with `model.pt`, `text_vocab.json`, `meta.json`.
|
| 145 |
-
|
| 146 |
-
**Hub package (this card):** weights from **`policy_0023`** — hybrid
|
| 147 |
-
strategist soft BC on top of the choice_fresh stack (~11.1M params,
|
| 148 |
-
`d_model=384`, 6 layers). League `best.json` may point at a later PPO
|
| 149 |
-
iter with a tiny N=12 win-rate spike; do not treat that as ladder
|
| 150 |
-
evidence.
|
| 151 |
-
|
| 152 |
-
## Evaluation
|
| 153 |
-
|
| 154 |
-
Primary metric: **seat-balanced win rate** vs a fixed opponent.
|
| 155 |
-
|
| 156 |
-
```bash
|
| 157 |
-
python -m mtgcards.rl.evaluate \
|
| 158 |
-
--ckpt checkpoints/choice_fresh/policy_best \
|
| 159 |
-
--vs greedy --games 40 --device cuda
|
| 160 |
-
```
|
| 161 |
-
|
| 162 |
-
Sparse-reward honesty: do not treat small-budget win rates as ladder
|
| 163 |
-
evidence; always report game count. JEV soft-label agreement and
|
| 164 |
-
judge-LLM audits (`--log-games`) are secondary signals.
|
| 165 |
-
|
| 166 |
-
| Checkpoint | vs greedy (N games) | Soft agree (JEV-labelled) | Notes |
|
| 167 |
-
| --- | --- | --- | --- |
|
| 168 |
-
| `policy_0019` | 0.42 (12, PPO sample) | — | early PPO plateau; noisy |
|
| 169 |
-
| `policy_0022` | 0.36 (11 scored / 12) | train 0.63 / held-out 0.51 | first hybrid+para distill |
|
| 170 |
-
| `policy_0023` | *(pending ≥20-game eval)* | train 0.65 / held-out 0.53 | **Hub candidate**; larger hybrid bank BC |
|
| 171 |
-
| `policy_0024` | 1.00 (12, PPO sample) | — | league “best” after distill; **N=12 fluke** — not published |
|
| 172 |
-
|
| 173 |
-
## Limitations and risks
|
| 174 |
-
|
| 175 |
-
- Engine fidelity gaps (see README “Known simplifications”) transfer to
|
| 176 |
-
the policy.
|
| 177 |
-
- Long states are **byte-truncated** (tail-preferring); very long games
|
| 178 |
-
still lose early history even at 8192.
|
| 179 |
-
- Greedy BC alone clones a weak teacher; strength needs JEV distill
|
| 180 |
-
and/or enough PPO.
|
| 181 |
-
- Hosted JEV is stronger zero-shot; this student is the cheap local
|
| 182 |
-
policy, not a drop-in API replacement.
|
| 183 |
-
- Do not deploy as an authority on real-world Magic rules disputes.
|
| 184 |
-
|
| 185 |
-
## Environmental impact
|
| 186 |
-
|
| 187 |
-
Training is local GPU (e.g. single RTX 3090). Wall-clock for the
|
| 188 |
-
choice_fresh recipe is on the order of hours (LM + BC dominate before
|
| 189 |
-
PPO). Exact kWh not measured.
|
| 190 |
-
|
| 191 |
-
## Citation / links
|
| 192 |
-
|
| 193 |
-
If you use System One weights or this training stack in research, cite the
|
| 194 |
-
software via the repo [`CITATION.cff`](https://github.com/jedld/mtgcards/blob/master/CITATION.cff) /
|
| 195 |
-
[`CITATION.bib`](https://github.com/jedld/mtgcards/blob/master/CITATION.bib) (and note training-data lineage in
|
| 196 |
-
[`NOTICE`](https://github.com/jedld/mtgcards/blob/master/NOTICE)).
|
| 197 |
-
|
| 198 |
-
- Hugging Face model: https://huggingface.co/jedld/system-one-choice
|
| 199 |
-
- TypeSafe System One concepts: https://docs.typesafe.ai/concepts/system-one
|
| 200 |
-
- FineWeb: https://huggingface.co/datasets/HuggingFaceFW/fineweb
|
| 201 |
-
- Repo docs: [`SYSTEM_ONE.md`](https://github.com/jedld/mtgcards/blob/master/docs/SYSTEM_ONE.md), [`SELF_IMPROVE.md`](https://github.com/jedld/mtgcards/blob/master/mtgcards/rl/SELF_IMPROVE.md)
|
| 202 |
-
- Staging (maintainers): `checkpoints/hf_staging/jedld-system-one-choice/`
|
| 203 |
-
(weights = `policy_0023` + this card as `README.md`). Upload:
|
| 204 |
-
`huggingface-cli upload jedld/system-one-choice checkpoints/hf_staging/jedld-system-one-choice . --repo-type model`
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:099b2b944e6602a8128581153b2788e3e49ced3799b8bfd9dfac6d5d71f3678f
|
| 3 |
+
size 9468
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|