Blame for README.md

---
license: mit
library_name: pytorch
tags:
- steam
- game-reviews
- representation-learning
- contrastive-learning
- set-encoder
---

# I-CE checkpoints

The trained set encoders ("towers") of the paper *Invariant Game Representation Learning from
Steam Reviews* (AICCC 2026). A tower is a small attention-pooling network of about 0.36 M
parameters: a few latent queries cross-attend over the cached sentence embeddings of a game's
reviews (Qwen3-Embedding-0.6B, 1,024-d) and an MLP maps the pooled result to one L2-normalized
128-d vector per game.

- Code, appendices and supplement:
  [`Nice9Tian/I2CE-GameRepresention-by-SteamReviews`](https://github.com/Nice9Tian/I2CE-GameRepresention-by-SteamReviews)
- The tensors the towers read:
  [`chironYu/latenarray-i2ce-data`](https://huggingface.co/datasets/chironYu/latenarray-i2ce-data)

## Contents

275 checkpoints, about 593 MiB, and four JSON files that list every one of them:

| Listed in | Checkpoints | What they are |
|---|---|---|
| `rank_ckpt_plan.json` | 166 | the five-fold runs behind the paper's numbers, each at the epoch the validation rank rule of Section 5 selects. One is the DINO-style control of Table A1, trained on fold 0 only; five, CE capped at 2,048 sentences, are the partners of I-CE runs the paper prints and are not printed themselves |
| `i2bce_sel_pool.json` | 5 | I + SimCLR on the five folds (Figure 8, Appendix L), selected by the same rule |
| `fixed_split_ckpt_plan.json` | 69 | the fixed-split runs: 15 runs at their selected epochs, two of them with an earlier checkpoint kept beside the selected one, and 52 more checkpoints of two runs that the paper reads across their epochs (field `trajectory`) |
| `rank_ckpt_plan.json`, field `init_ckpt` | 5 | the BYOL checkpoints the five two-stage runs started from |
| `superseded_ckpt_plan.json` | 30 | kept for the record; no number of the paper reads them |

An entry of the three plan files gives the arm, the fold, the anchor pack, the selected epoch,
the file name and the file's SHA-256; `i2bce_sel_pool.json` is the selection as the evaluation
script wrote it, with the rank score of every checkpoint of the run. The same four files are in
`contrast_experiment/w9/eval/` of the code repository, where the evaluation script reads them.

Every tower that a number or a figure of the paper reads is here, at the checkpoint it is read
at. A run saved a checkpoint every 50 epochs and the rank rule chose among them; the ones it did
not choose are not here, except for the two runs listed under `trajectory`. The other passages
that follow runs over training quote what the workers logged at each checkpoint: the learnable
temperature and the view-count and view-size sweeps in Appendix C, and the decline of the
views-first anchors after epoch 50 in Supplement A.

## File names

    ckpt_w9cv_wcle_<arm>_fold<k><pack><suffix>_fp_ep<N>.pt     five folds, fold k is the test fold
    ckpt_w9_wcle_<arm><pack><suffix>_fp_ep<N>.pt               the fixed split

- `<arm>` is the objective, listed in the tables below.
- `<pack>` is the anchor pack the run trained with, counted in sentences with the store-page
  prefix included: nothing for 512, `_g1024`, `_g2048`, or `_gpool` for the prefix plus the game's
  whole 2,048-sentence review pool (a median of 2,068 sentences). The paper calls `_g2048` and
  `_gpool` alike its 2,048-sentence budget; the main five-fold towers are the `_gpool` runs.
- `<suffix>`: `_wllm` for LLM-rewritten documents as the document view, `_bw` for a run
  initialized from a BYOL tower, `_w<N>` for review views drawn until they hold at least N
  sentences (the recipe stops at 16), `_d512` for a run on the current 512-sentence packs (see
  below).
- `_fp`: the strong views were drawn from the full review pool. `ep<N>` is the epoch.

## The five-fold runs

Table and figure numbers are those of the paper, its appendices (Tables A1 to A19) and its
supplement (Tables S1 to S4). The last column comes from the provenance table of the code
repository, `contrast_experiment/w9/Pod/w9_profiles.py`, which cites by cell: a table named for a
row reads at least one of the row's runs. That file also names the notebook that trains each cell.

| Runs | What they are | Pack | Folds | Read by |
|---|---|---|---|---|
| `ai2ce_icetf_d512` | CE + anchor-in-I | 512 | 5 | Table 2, Section 6.2, Table A15 |
| `ce_cetf_d512`, `i2ce_icetf_d512` | CE; I-CE, the recipe | 512 | 5 | Table 2, Table A2, Table A3, Table A4, Table A5, Table A6, Table A7, Table A11, Table A15, Table A19, Table S1 |
| `d1r2_i2ce_icetf_d512`, `d1r4_i2ce_icetf_d512`, `d1r5_i2ce_icetf_d512`, `d1r5_i3ce_icetf_d512`, `d1r6_i2ce_icetf_d512` | I-CE, V = 3; I-CE, V = 5; I-CE, V = 6; I-CE, V = 6 with λ = 3; I-CE, V = 7 | 512 | 0 to 2 | Table A5 |
| `i2ce_icetf_w8_d512`, `i2ce_icetf_w32_d512`, `i2ce_icetf_w64_d512` | I-CE, review views of at least 8 sentences; of at least 32; of at least 64 | 512 | 0 to 2 | Table A6 |
| `i2ce_icetf_wllm_d512`, `nodoc_i2ce_icetf_d512` | I-CE, LLM-rewritten documents; I-CE without the document view | 512 | 5 | Table A19 |
| `i2cet05_icetf_d512`, `i2cet10_icetf_d512` | I-CE, τ = 0.05; I-CE, τ = 0.10 | 512 | 5 | Table A3 |
| `i3ce_icetf_d512`, `i4ce_icetf_d512`, `ice_icetf_d512` | I-CE, λ = 3; I-CE, λ = 4; I-CE, λ = 1 | 512 | 0 to 2 | Table A4 |
| `slot16i2cemean_icetf_d512`, `slot1i2cemean_icetf_d512`, `slot2i2cemean_icetf_d512`, `slot8i2cemean_icetf_d512` | I-CE, 16 latent queries; I-CE, 1 latent query; I-CE, 2 latent queries; I-CE, 8 latent queries | 512 | 0 to 2 | Table A7 |
| `vfai2ce_icetf_d512` | I-CE, views-first anchors | 512 | 5 | Table S1 |
| `ce_cetf_g1024`, `i2ce_icetf_g1024` | CE; I-CE, the recipe | 1,024 | 5 | Table A2, Table A11, Table A15 |
| `ce_cetf_g2048`, `i2ce_icetf_g2048` | CE; I-CE, the recipe | 2,048 | 5 | Table A3, Table A11 |
| `i2ce_icetl_g2048`, `i2cet05_icetf_g2048`, `i2cet10_icetf_g2048` | I-CE, learnable τ; I-CE, τ = 0.05; I-CE, τ = 0.10 | 2,048 | 5 | Table A3 |
| `bce_cetf_gpool`, `byol_bytf_gpool`, `epd_v20i10c20_cetf_gpool`, `mq3072i2ce_icetf_gpool` | SimCLR-style; BYOL; VICReg (batch 192); I-CE with an EMA teacher and a memory bank | pool | 5 | Figure 7, Figure 8, Table 2, Section 3.2, Table A1, Table A13, Table A16, Table A17, Table A18, Figure A2, Table S2, Table S3, Table S4 |
| `ce_cetf_gpool`, `i2ce_icetf_gpool` | CE; I-CE, the recipe | pool | 5 | Figure 7, Figure 8, Table 2, Section 6.1, Table A1, Table A2, Table A7, Table A13, Table A14, Table A15, Table A16, Table A17, Table A18, Figure A2, Table S2, Table S3, Table S4 |
| `slot8i2cemean_icetf_gpool` | I-CE, 8 latent queries | pool | 5 | Table A7 |
| `swin168step84loop2i2ce_icetf_gpool`, `swin168step84loop2i2ce_icetf_gpool_bw` | I-CE with the windowed teacher; I-CE, two-stage: BYOL warm start, then the windowed teacher | pool | 5 | Table A13, Table S2, Table S3, Table S4 |
| `dino_dntf_gpool` | DINO at its published settings, the control of Section 4.3 | pool | 0 only | Table A1 |

I + SimCLR (`i2bce_icetf`, files without a pack or a suffix, five folds) backs Figure 8 and the
five-fold readings of Appendix L. The
fold-0 run of `i2ce_icetf_g2048` also gives the game vectors of Figure 10. The DINO-style control
has one fold, so Table A1 gives its spread over the ten anchor draws; its file is 169 MiB because
it carries the two prototype heads (see Format).

## The fixed-split runs

| Runs | What they are | Pack | Folds | Read by |
|---|---|---|---|---|
| `ai25auni2_icetf_d512`, `ai2auni25_icetf_d512`, `ai4auni2_icetf_d512`, `ai6auni2_icetf_d512`, `i4uni2_icetf` | alignment + uniformity aimed at the anchor gallery, the four settings of Section 6.2 (`ai25auni2` is Table 3's a+u @ gallery row); alignment + uniformity aimed at the batch views (Table 3's a+u @ views row) | 512 | fixed split | Table 3, Section 6.2, Appendix L |
| `ai2ce_icetf_d512`, `ce_cetf_d512`, `i2ce_icetf_d512` | CE + anchor-in-I; CE; I-CE, the recipe | 512 | fixed split | Table 3, Section 6.2, Table A8 |
| `ce_cetf_g2048`, `i2ce_icetf_g2048` | CE; I-CE, the recipe | 2,048 | fixed split | Figure 9, Figure A1, Figure A3, Appendix L |
| `bce_cetf`, `i2bce_icetf`, `epd_v20i10c20_cetf` | SimCLR-style; I + SimCLR; VICReg, v/i/c = 20/10/20 (the recipe weights) | 512 | fixed split | Figure 9, Figure A3, Appendix L, Table A8 |
| `epd_v25i25c1_cetf`, `epd_v20i10c15_cetf` | VICReg, v/i/c = 25/25/1; VICReg, v/i/c = 20/10/15 | 512 | fixed split | Table A8 |

Figure 9, Figure A3 and the neighborhood statistics of Appendix L read six encoders on the fixed
split: the frozen embedder, which has no checkpoint, and five towers, namely `i2ce_icetf_g2048`
(epoch 1,750), `ce_cetf_g2048` (1,150), `bce_cetf` (1,350), `i2bce_icetf` (1,350) and
`epd_v20i10c20_cetf` (100). Two runs have more than their selected checkpoint here, listed under
`trajectory` in the plan file: all 40 checkpoints of `i2ce_icetf_g2048`, whose four latent queries
Figure A1 draws at every one, and the `bce_cetf` checkpoints from epoch 1,350 to 2,000, the span
of the test curve Appendix L describes.

Runs that read no anchor in training (`bce_cetf`, `i2bce_icetf`, `i4uni2_icetf` and the three
VICReg weightings; the 512 in their rows is only the pack their worker encoded at each
checkpoint) were selected after training: each of the run's 40 checkpoints encodes the
4,096-sentence anchor pack of the fixed split and the rank rule chooses on that.

Ten of the fifteen runs stopped at 1,000 epochs under their first schedule and were trained on to
2,000 from the weights of epoch 1,000 with a fresh optimizer: those six, CE + anchor-in-I, and
three of the gallery-aimed settings (`ai6auni2`, `ai4auni2`, `ai2auni25`). The plan entries say so
under `extended`. Two selections moved with the longer runs, CE + anchor-in-I to epoch 1,650 and
`ai4auni2` to epoch 1,250; the checkpoints selected before (epochs 1,000 and 900) are kept and
named under `previous`.

## Format

Every file holds tensors only, so `torch.load(path, weights_only=True)` reads it. Most are the
state dict of the tower itself, `SetPoolN(Q)` of `contrast_experiment/w9/Pod/w9_cv_worker.py`
(keys `q0`, `attn.*`, `head.*`; `q0` has shape `[1, Q, 128]`, Q = 4 except in the `slot<Q>` runs).
Four families wrap it: BYOL files hold `{"model", "pred", "target"}`, VICReg files
`{"model", "expander"}`, the memory-bank files `{"model", "shadow"}`, and the DINO-style file
`{"model", "student", "t_head", "s_head", "center"}`; the tower is `model`. In the DINO-style file
`model` is the teacher's tower, the one DINO evaluates and the paper reads, `student` the
student's, and the two heads hold the 65,536 prototypes.

```python
import sys, torch
sys.path[:0] = ["contrast_experiment/w9", "contrast_experiment/w9/Pod"]   # from the code repository's root
from Pod.w9_cv_worker import SetPoolN

sd = torch.load("checkpoints/ckpt_w9cv_wcle_i2ce_icetf_fold0_gpool_fp_ep2000.pt", weights_only=True)
sd = sd.get("model", sd)
sd = {k: v for k, v in sd.items() if k == "q0" or k.startswith(("attn.", "head."))}
tower = SetPoolN(sd["q0"].shape[1]); tower.load_state_dict(sd); tower.eval()

# S: [games, sentences, 1024] sentence vectors, each row standardized over its 1,024 dimensions
# pad: [games, sentences] bool, True where a row is padding
z = tower(S, pad)        # [games, 128], L2-normalized
```

## Scoring them as the paper does

The paper scores a five-fold tower against ten independent 4,096-sentence anchor packs per game
and reports the mean. From the root of the code repository, with the data folder in place:

```bash
hf download chironYu/latenarray-i2ce-checkpoints --local-dir checkpoints
export W9_DATA_DIR=/path/to/latenarray-i2ce-data
python contrast_experiment/w9/eval/k10_eval.py cells  cells.json --ckpt-dir checkpoints
python contrast_experiment/w9/eval/k10_eval.py encode cells.json draws.npz
python contrast_experiment/w9/eval/k10_eval.py score  cells.json draws.npz scores.json
python contrast_experiment/w9/eval/k10_eval.py table  scores.json
```

`cells` takes the 171 five-fold checkpoints of `rank_ckpt_plan.json` and `i2bce_sel_pool.json`;
`--only <regex>` keeps a subset, for example `--only "_d512$"` for the 512-sentence runs. In its
table the five families of the main comparison go by the paper's names (`I-CE`, `CE`, `SimCLR`,
`VICReg`, `BYOL`) and every other family by its plan key without the fold; a family with one
fold, the DINO-style control, is printed with its spread over the ten draws.

The fixed split has one command of its own:

```bash
python contrast_experiment/w9/eval/fixed_split_eval.py --ckpt-dir checkpoints
```

It reads every run of `fixed_split_ckpt_plan.json` at its selected checkpoint and prints each
reading beside the number the paper prints: Table 3, the gallery-aimed settings of Section 6.2
and Table A8 on the one 4,096-sentence anchor draw of the fixed split, the SimCLR-style test
curve of Appendix L, the latent queries of Figure A1, and, through `k10_eval.py`, the towers of
Figure 9 against the ten draws. A near tie can move a cell by one query of 204 (0.005), and the
collapsed a+u @ gallery row by about 0.01.

The figures built on these towers are drawn from the checkpoints as well:
`python figures/neighborhoods/run_all.py --ckpt-dir checkpoints --data-dir <data folder>` makes
Figures 8, 9 and A3 and the neighborhood statistics of Appendix L.

## The 512-sentence packs and the `_d512` runs

Until 2026-11-07 the 512-sentence anchor packs (`wscan_gal_rev.npz` of the dataset) carried the
store-page prefix rows without the per-row standardization every other input receives (row std
about 2.9 against 1.0), and the tower has no input normalization. Only the 512 budget was
affected; larger packs were standardized. The dataset now carries the corrected packs, and every
run whose loss reads the 512-sentence anchors was trained again on them under the paper's recipe.
Those runs are named `_d512`, and they are the ones the paper reads. The workers of the code
repository give a run on the current packs that name themselves.

Runs whose loss never reads the anchors (the SimCLR-style tower, I + SimCLR, BYOL, VICReg and
alignment plus uniformity aimed at the batch views) train the same weights on either pack, so
their checkpoints were not replaced; only readings that used the 512-sentence packs were redone.
Their files therefore carry no `_d512`: the fixed-split ones, and those of the five-fold
I + SimCLR.

Stripped hit@1 on the ten 4,096-sentence draws, five-fold mean ± std, before and after:

| Tower | on the packs as first built | on the corrected packs (`_d512`) |
|---|---|---|
| I-CE (`i2ce_icetf`) | 0.669 ± 0.031 | 0.715 ± 0.010 |
| CE (`ce_cetf`) | 0.638 ± 0.016 | 0.676 ± 0.013 |
| CE + anchor-in-I (`ai2ce_icetf`) | 0.663 ± 0.014 | 0.725 ± 0.015 |
| I-CE without the document view (`nodoc_i2ce_icetf`) | 0.650 ± 0.022 | 0.683 ± 0.025 |
| I-CE, LLM-rewritten documents (`i2ce_icetf`, `_wllm`) | 0.680 ± 0.034 | 0.711 ± 0.009 |
| I-CE, τ = 0.05 (`i2cet05_icetf`) | 0.601 ± 0.029 | 0.642 ± 0.026 |
| I-CE, τ = 0.10 (`i2cet10_icetf`) | 0.510 ± 0.022 (four folds) | 0.520 ± 0.015 |

The checkpoints behind the left column are the superseded files of the next section, except
those of CE + anchor-in-I, whose first runs were not uploaded. The other `_d512` families (the
sweeps on folds 0 to 2 and the views-first anchors) have no earlier counterpart here.

## Superseded checkpoints

`superseded_ckpt_plan.json` lists 30 files that stay here although the paper reads none of them:
the 29 five-fold runs at 512 sentences made on the packs as first built, whose names have no
`_d512` (I-CE, CE, the two document-view runs and the two temperatures), each with a note naming
the run that replaced it, and one fold of a VICReg arm that used the whole training set as its
batch, which the paper dropped. The file has
the format of a selection file, so `k10_eval.py cells` scores them when it is given the file. The
dataset keeps the first packs as `wscan_gal_rev_rawprefix.npz`; the workers read them with
`--raw-prefix` and then name the run without the tag.

## Changes

- 2026-11-12, later that day. Nine checkpoints were added, the view-size sweep of the appendix's
  new Table A6: I-CE with each review view drawn until it holds at least 8, 32 or 64 sentences
  (`i2ce_icetf_fold<k>_w<N>_d512`, folds 0 to 2). `rank_ckpt_plan.json` lists them. The new table
  moved every later appendix table one place, so the old Tables A6 to A18 are Tables A7 to A19
  in the tables above; no earlier entry changed its checkpoint.
- 2026-11-12. The code repository gained `figures/neighborhoods/`, the scripts that drew
  Figures 8, 9 and A3.
- 2026-11-11, at night. The code repository gained `fixed_split_eval.py`, which reads the
  fixed-split checkpoints as the paper does.
- 2026-11-11, later that day. 62 checkpoints were added, so that every tower the paper reads is
  here: the DINO-style control (fold 0), eight fixed-split runs at their selected epochs (CE and
  I-CE trained at 2,048 sentences, the SimCLR-style tower, I + SimCLR, a+u @ views and the three
  VICReg weightings), the 52 `trajectory` checkpoints, and epoch 1,250 of the 4:1 gallery-aimed
  run of Section 6.2 (`ai4auni2_icetf_d512`). That run's entry in `fixed_split_ckpt_plan.json`
  named epoch 900, its selection before it was trained on to 2,000 epochs; the paper reads epoch
  1,250. No other earlier entry changed its checkpoint.
- 2026-11-11. The four JSON files above replace the earlier per-campaign lists
  (`docfix512_ckpt_plan.json`, `c2_docfix512_ckpt_plan.json`, `c3_docfix512_ckpt_plan.json`,
  `q8_gpool_ckpt_plan.json`); no selection changed. The five BYOL checkpoints the two-stage runs
  started from were added. The `_gpool` files were renamed from `_g4096`, a cap the review pool
  never reached; the files are the same.
- 2026-11-07 to 2026-11-10. The `_d512` runs were added.

## Citation

Zhilong Yu et al. *Invariant Game Representation Learning from Steam Reviews.* 2026 9th Artificial
Intelligence and Cloud Computing Conference (AICCC 2026), December 18 to 21, 2026, Tokyo, Japan.

```bibtex
@inproceedings{yu2026invariant,
  author    = {Yu, Zhilong and others},
  title     = {Invariant Game Representation Learning from Steam Reviews},
  booktitle = {2026 9th Artificial Intelligence and Cloud Computing Conference (AICCC 2026)},
  address   = {Tokyo, Japan},
  year      = {2026}
}
```

Code and checkpoints are under the MIT license.