README.md

metadata

license: mit
library_name: pytorch
tags:
  - steam
  - game-reviews
  - representation-learning
  - contrastive-learning
  - set-encoder

I-CE checkpoints

The trained set encoders ("towers") of the paper Invariant Game Representation Learning from Steam Reviews (AICCC 2026). A tower is a small attention-pooling network of about 0.36 M parameters: a few latent queries cross-attend over the cached sentence embeddings of a game's reviews (Qwen3-Embedding-0.6B, 1,024-d) and an MLP maps the pooled result to one L2-normalized 128-d vector per game.

• Code, appendices and supplement: Nice9Tian/I2CE-GameRepresention-by-SteamReviews
• The tensors the towers read: chironYu/latenarray-i2ce-data

Contents

275 checkpoints, about 593 MiB, and four JSON files that list every one of them:

Listed in	Checkpoints	What they are
rank_ckpt_plan.json	166	the five-fold runs behind the paper's numbers, each at the epoch the validation rank rule of Section 5 selects. One is the DINO-style control of Table A1, trained on fold 0 only; five, CE capped at 2,048 sentences, are the partners of I-CE runs the paper prints and are not printed themselves
i2bce_sel_pool.json	5	I + SimCLR on the five folds (Figure 8, Appendix L), selected by the same rule
fixed_split_ckpt_plan.json	69	the fixed-split runs: 15 runs at their selected epochs, two of them with an earlier checkpoint kept beside the selected one, and 52 more checkpoints of two runs that the paper reads across their epochs (field trajectory)
rank_ckpt_plan.json, field init_ckpt	5	the BYOL checkpoints the five two-stage runs started from
superseded_ckpt_plan.json	30	kept for the record; no number of the paper reads them

An entry of the three plan files gives the arm, the fold, the anchor pack, the selected epoch, the file name and the file's SHA-256; i2bce_sel_pool.json is the selection as the evaluation script wrote it, with the rank score of every checkpoint of the run. The same four files are in contrast_experiment/w9/eval/ of the code repository, where the evaluation script reads them.

Every tower that a number or a figure of the paper reads is here, at the checkpoint it is read at. A run saved a checkpoint every 50 epochs and the rank rule chose among them; the ones it did not choose are not here, except for the two runs listed under trajectory. The other passages that follow runs over training quote what the workers logged at each checkpoint: the learnable temperature and the view-count and view-size sweeps in Appendix C, and the decline of the views-first anchors after epoch 50 in Supplement A.

File names

ckpt_w9cv_wcle_<arm>_fold<k><pack><suffix>_fp_ep<N>.pt     five folds, fold k is the test fold
ckpt_w9_wcle_<arm><pack><suffix>_fp_ep<N>.pt               the fixed split

• <arm> is the objective, listed in the tables below.
• <pack> is the anchor pack the run trained with, counted in sentences with the store-page prefix included: nothing for 512, _g1024, _g2048, or _gpool for the prefix plus the game's whole 2,048-sentence review pool (a median of 2,068 sentences). The paper calls _g2048 and _gpool alike its 2,048-sentence budget; the main five-fold towers are the _gpool runs.
• <suffix>: _wllm for LLM-rewritten documents as the document view, _bw for a run initialized from a BYOL tower, _w<N> for review views drawn until they hold at least N sentences (the recipe stops at 16), _d512 for a run on the current 512-sentence packs (see below).
• _fp: the strong views were drawn from the full review pool. ep<N> is the epoch.

The five-fold runs

Table and figure numbers are those of the paper, its appendices (Tables A1 to A19) and its supplement (Tables S1 to S4). The last column comes from the provenance table of the code repository, contrast_experiment/w9/Pod/w9_profiles.py, which cites by cell: a table named for a row reads at least one of the row's runs. That file also names the notebook that trains each cell.

Runs	What they are	Pack	Folds	Read by
ai2ce_icetf_d512	CE + anchor-in-I	512	5	Table 2, Section 6.2, Table A15
ce_cetf_d512, i2ce_icetf_d512	CE; I-CE, the recipe	512	5	Table 2, Table A2, Table A3, Table A4, Table A5, Table A6, Table A7, Table A11, Table A15, Table A19, Table S1
d1r2_i2ce_icetf_d512, d1r4_i2ce_icetf_d512, d1r5_i2ce_icetf_d512, d1r5_i3ce_icetf_d512, d1r6_i2ce_icetf_d512	I-CE, V = 3; I-CE, V = 5; I-CE, V = 6; I-CE, V = 6 with λ = 3; I-CE, V = 7	512	0 to 2	Table A5
i2ce_icetf_w8_d512, i2ce_icetf_w32_d512, i2ce_icetf_w64_d512	I-CE, review views of at least 8 sentences; of at least 32; of at least 64	512	0 to 2	Table A6
i2ce_icetf_wllm_d512, nodoc_i2ce_icetf_d512	I-CE, LLM-rewritten documents; I-CE without the document view	512	5	Table A19
i2cet05_icetf_d512, i2cet10_icetf_d512	I-CE, τ = 0.05; I-CE, τ = 0.10	512	5	Table A3
i3ce_icetf_d512, i4ce_icetf_d512, ice_icetf_d512	I-CE, λ = 3; I-CE, λ = 4; I-CE, λ = 1	512	0 to 2	Table A4
slot16i2cemean_icetf_d512, slot1i2cemean_icetf_d512, slot2i2cemean_icetf_d512, slot8i2cemean_icetf_d512	I-CE, 16 latent queries; I-CE, 1 latent query; I-CE, 2 latent queries; I-CE, 8 latent queries	512	0 to 2	Table A7
vfai2ce_icetf_d512	I-CE, views-first anchors	512	5	Table S1
ce_cetf_g1024, i2ce_icetf_g1024	CE; I-CE, the recipe	1,024	5	Table A2, Table A11, Table A15
ce_cetf_g2048, i2ce_icetf_g2048	CE; I-CE, the recipe	2,048	5	Table A3, Table A11
i2ce_icetl_g2048, i2cet05_icetf_g2048, i2cet10_icetf_g2048	I-CE, learnable τ; I-CE, τ = 0.05; I-CE, τ = 0.10	2,048	5	Table A3
bce_cetf_gpool, byol_bytf_gpool, epd_v20i10c20_cetf_gpool, mq3072i2ce_icetf_gpool	SimCLR-style; BYOL; VICReg (batch 192); I-CE with an EMA teacher and a memory bank	pool	5	Figure 7, Figure 8, Table 2, Section 3.2, Table A1, Table A13, Table A16, Table A17, Table A18, Figure A2, Table S2, Table S3, Table S4
ce_cetf_gpool, i2ce_icetf_gpool	CE; I-CE, the recipe	pool	5	Figure 7, Figure 8, Table 2, Section 6.1, Table A1, Table A2, Table A7, Table A13, Table A14, Table A15, Table A16, Table A17, Table A18, Figure A2, Table S2, Table S3, Table S4
slot8i2cemean_icetf_gpool	I-CE, 8 latent queries	pool	5	Table A7
swin168step84loop2i2ce_icetf_gpool, swin168step84loop2i2ce_icetf_gpool_bw	I-CE with the windowed teacher; I-CE, two-stage: BYOL warm start, then the windowed teacher	pool	5	Table A13, Table S2, Table S3, Table S4
dino_dntf_gpool	DINO at its published settings, the control of Section 4.3	pool	0 only	Table A1

I + SimCLR (i2bce_icetf, files without a pack or a suffix, five folds) backs Figure 8 and the five-fold readings of Appendix L. The fold-0 run of i2ce_icetf_g2048 also gives the game vectors of Figure 10. The DINO-style control has one fold, so Table A1 gives its spread over the ten anchor draws; its file is 169 MiB because it carries the two prototype heads (see Format).

The fixed-split runs

Runs	What they are	Pack	Folds	Read by
ai25auni2_icetf_d512, ai2auni25_icetf_d512, ai4auni2_icetf_d512, ai6auni2_icetf_d512, i4uni2_icetf	alignment + uniformity aimed at the anchor gallery, the four settings of Section 6.2 (ai25auni2 is Table 3's a+u @ gallery row); alignment + uniformity aimed at the batch views (Table 3's a+u @ views row)	512	fixed split	Table 3, Section 6.2, Appendix L
ai2ce_icetf_d512, ce_cetf_d512, i2ce_icetf_d512	CE + anchor-in-I; CE; I-CE, the recipe	512	fixed split	Table 3, Section 6.2, Table A8
ce_cetf_g2048, i2ce_icetf_g2048	CE; I-CE, the recipe	2,048	fixed split	Figure 9, Figure A1, Figure A3, Appendix L
bce_cetf, i2bce_icetf, epd_v20i10c20_cetf	SimCLR-style; I + SimCLR; VICReg, v/i/c = 20/10/20 (the recipe weights)	512	fixed split	Figure 9, Figure A3, Appendix L, Table A8
epd_v25i25c1_cetf, epd_v20i10c15_cetf	VICReg, v/i/c = 25/25/1; VICReg, v/i/c = 20/10/15	512	fixed split	Table A8

Figure 9, Figure A3 and the neighborhood statistics of Appendix L read six encoders on the fixed split: the frozen embedder, which has no checkpoint, and five towers, namely i2ce_icetf_g2048 (epoch 1,750), ce_cetf_g2048 (1,150), bce_cetf (1,350), i2bce_icetf (1,350) and epd_v20i10c20_cetf (100). Two runs have more than their selected checkpoint here, listed under trajectory in the plan file: all 40 checkpoints of i2ce_icetf_g2048, whose four latent queries Figure A1 draws at every one, and the bce_cetf checkpoints from epoch 1,350 to 2,000, the span of the test curve Appendix L describes.

Runs that read no anchor in training (bce_cetf, i2bce_icetf, i4uni2_icetf and the three VICReg weightings; the 512 in their rows is only the pack their worker encoded at each checkpoint) were selected after training: each of the run's 40 checkpoints encodes the 4,096-sentence anchor pack of the fixed split and the rank rule chooses on that.

Ten of the fifteen runs stopped at 1,000 epochs under their first schedule and were trained on to 2,000 from the weights of epoch 1,000 with a fresh optimizer: those six, CE + anchor-in-I, and three of the gallery-aimed settings (ai6auni2, ai4auni2, ai2auni25). The plan entries say so under extended. Two selections moved with the longer runs, CE + anchor-in-I to epoch 1,650 and ai4auni2 to epoch 1,250; the checkpoints selected before (epochs 1,000 and 900) are kept and named under previous.

Format

Every file holds tensors only, so torch.load(path, weights_only=True) reads it. Most are the state dict of the tower itself, SetPoolN(Q) of contrast_experiment/w9/Pod/w9_cv_worker.py (keys q0, attn.*, head.*; q0 has shape [1, Q, 128], Q = 4 except in the slot<Q> runs). Four families wrap it: BYOL files hold {"model", "pred", "target"}, VICReg files {"model", "expander"}, the memory-bank files {"model", "shadow"}, and the DINO-style file {"model", "student", "t_head", "s_head", "center"}; the tower is model. In the DINO-style file model is the teacher's tower, the one DINO evaluates and the paper reads, student the student's, and the two heads hold the 65,536 prototypes.

import sys, torch
sys.path[:0] = ["contrast_experiment/w9", "contrast_experiment/w9/Pod"]   # from the code repository's root
from Pod.w9_cv_worker import SetPoolN

sd = torch.load("checkpoints/ckpt_w9cv_wcle_i2ce_icetf_fold0_gpool_fp_ep2000.pt", weights_only=True)
sd = sd.get("model", sd)
sd = {k: v for k, v in sd.items() if k == "q0" or k.startswith(("attn.", "head."))}
tower = SetPoolN(sd["q0"].shape[1]); tower.load_state_dict(sd); tower.eval()

# S: [games, sentences, 1024] sentence vectors, each row standardized over its 1,024 dimensions
# pad: [games, sentences] bool, True where a row is padding
z = tower(S, pad)        # [games, 128], L2-normalized

Scoring them as the paper does

The paper scores a five-fold tower against ten independent 4,096-sentence anchor packs per game and reports the mean. From the root of the code repository, with the data folder in place:

hf download chironYu/latenarray-i2ce-checkpoints --local-dir checkpoints
export W9_DATA_DIR=/path/to/latenarray-i2ce-data
python contrast_experiment/w9/eval/k10_eval.py cells  cells.json --ckpt-dir checkpoints
python contrast_experiment/w9/eval/k10_eval.py encode cells.json draws.npz
python contrast_experiment/w9/eval/k10_eval.py score  cells.json draws.npz scores.json
python contrast_experiment/w9/eval/k10_eval.py table  scores.json

cells takes the 171 five-fold checkpoints of rank_ckpt_plan.json and i2bce_sel_pool.json; --only <regex> keeps a subset, for example --only "_d512$" for the 512-sentence runs. In its table the five families of the main comparison go by the paper's names (I-CE, CE, SimCLR, VICReg, BYOL) and every other family by its plan key without the fold; a family with one fold, the DINO-style control, is printed with its spread over the ten draws.

The fixed split has one command of its own:

python contrast_experiment/w9/eval/fixed_split_eval.py --ckpt-dir checkpoints

It reads every run of fixed_split_ckpt_plan.json at its selected checkpoint and prints each reading beside the number the paper prints: Table 3, the gallery-aimed settings of Section 6.2 and Table A8 on the one 4,096-sentence anchor draw of the fixed split, the SimCLR-style test curve of Appendix L, the latent queries of Figure A1, and, through k10_eval.py, the towers of Figure 9 against the ten draws. A near tie can move a cell by one query of 204 (0.005), and the collapsed a+u @ gallery row by about 0.01.

The figures built on these towers are drawn from the checkpoints as well: python figures/neighborhoods/run_all.py --ckpt-dir checkpoints --data-dir <data folder> makes Figures 8, 9 and A3 and the neighborhood statistics of Appendix L.

The 512-sentence packs and the _d512 runs

Until 2026-11-07 the 512-sentence anchor packs (wscan_gal_rev.npz of the dataset) carried the store-page prefix rows without the per-row standardization every other input receives (row std about 2.9 against 1.0), and the tower has no input normalization. Only the 512 budget was affected; larger packs were standardized. The dataset now carries the corrected packs, and every run whose loss reads the 512-sentence anchors was trained again on them under the paper's recipe. Those runs are named _d512, and they are the ones the paper reads. The workers of the code repository give a run on the current packs that name themselves.

Runs whose loss never reads the anchors (the SimCLR-style tower, I + SimCLR, BYOL, VICReg and alignment plus uniformity aimed at the batch views) train the same weights on either pack, so their checkpoints were not replaced; only readings that used the 512-sentence packs were redone. Their files therefore carry no _d512: the fixed-split ones, and those of the five-fold I + SimCLR.

Stripped hit@1 on the ten 4,096-sentence draws, five-fold mean ± std, before and after:

Tower	on the packs as first built	on the corrected packs (_d512)
I-CE (i2ce_icetf)	0.669 ± 0.031	0.715 ± 0.010
CE (ce_cetf)	0.638 ± 0.016	0.676 ± 0.013
CE + anchor-in-I (ai2ce_icetf)	0.663 ± 0.014	0.725 ± 0.015
I-CE without the document view (nodoc_i2ce_icetf)	0.650 ± 0.022	0.683 ± 0.025
I-CE, LLM-rewritten documents (i2ce_icetf, _wllm)	0.680 ± 0.034	0.711 ± 0.009
I-CE, τ = 0.05 (i2cet05_icetf)	0.601 ± 0.029	0.642 ± 0.026
I-CE, τ = 0.10 (i2cet10_icetf)	0.510 ± 0.022 (four folds)	0.520 ± 0.015

The checkpoints behind the left column are the superseded files of the next section, except those of CE + anchor-in-I, whose first runs were not uploaded. The other _d512 families (the sweeps on folds 0 to 2 and the views-first anchors) have no earlier counterpart here.

Superseded checkpoints

superseded_ckpt_plan.json lists 30 files that stay here although the paper reads none of them: the 29 five-fold runs at 512 sentences made on the packs as first built, whose names have no _d512 (I-CE, CE, the two document-view runs and the two temperatures), each with a note naming the run that replaced it, and one fold of a VICReg arm that used the whole training set as its batch, which the paper dropped. The file has the format of a selection file, so k10_eval.py cells scores them when it is given the file. The dataset keeps the first packs as wscan_gal_rev_rawprefix.npz; the workers read them with --raw-prefix and then name the run without the tag.

Changes

• 2026-11-12, later that day. Nine checkpoints were added, the view-size sweep of the appendix's new Table A6: I-CE with each review view drawn until it holds at least 8, 32 or 64 sentences (i2ce_icetf_fold<k>_w<N>_d512, folds 0 to 2). rank_ckpt_plan.json lists them. The new table moved every later appendix table one place, so the old Tables A6 to A18 are Tables A7 to A19 in the tables above; no earlier entry changed its checkpoint.
• 2026-11-12. The code repository gained figures/neighborhoods/, the scripts that drew Figures 8, 9 and A3.
• 2026-11-11, at night. The code repository gained fixed_split_eval.py, which reads the fixed-split checkpoints as the paper does.
• 2026-11-11, later that day. 62 checkpoints were added, so that every tower the paper reads is here: the DINO-style control (fold 0), eight fixed-split runs at their selected epochs (CE and I-CE trained at 2,048 sentences, the SimCLR-style tower, I + SimCLR, a+u @ views and the three VICReg weightings), the 52 trajectory checkpoints, and epoch 1,250 of the 4:1 gallery-aimed run of Section 6.2 (ai4auni2_icetf_d512). That run's entry in fixed_split_ckpt_plan.json named epoch 900, its selection before it was trained on to 2,000 epochs; the paper reads epoch 1,250. No other earlier entry changed its checkpoint.
• 2026-11-11. The four JSON files above replace the earlier per-campaign lists (docfix512_ckpt_plan.json, c2_docfix512_ckpt_plan.json, c3_docfix512_ckpt_plan.json, q8_gpool_ckpt_plan.json); no selection changed. The five BYOL checkpoints the two-stage runs started from were added. The _gpool files were renamed from _g4096, a cap the review pool never reached; the files are the same.
• 2026-11-07 to 2026-11-10. The _d512 runs were added.

Citation

Zhilong Yu et al. Invariant Game Representation Learning from Steam Reviews. 2026 9th Artificial Intelligence and Cloud Computing Conference (AICCC 2026), December 18 to 21, 2026, Tokyo, Japan.

@inproceedings{yu2026invariant,
  author    = {Yu, Zhilong and others},
  title     = {Invariant Game Representation Learning from Steam Reviews},
  booktitle = {2026 9th Artificial Intelligence and Cloud Computing Conference (AICCC 2026)},
  address   = {Tokyo, Japan},
  year      = {2026}
}

Code and checkpoints are under the MIT license.