Publish gamespace.2: Steam game embeddings trained with co-like pairs

1_Pooling/config.json

@@ -0,0 +1,5 @@
+{
+    "embedding_dimension": 384,
+    "pooling_mode": "mean",
+    "include_prompt": true
+}

2_Dense/config.json

@@ -0,0 +1,8 @@
+{
+    "in_features": 384,
+    "out_features": 128,
+    "bias": true,
+    "activation_function": "torch.nn.modules.linear.Identity",
+    "module_input_name": "sentence_embedding",
+    "module_output_name": "sentence_embedding"
+}

2_Dense/model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0864454833b8281dce65b28ad0519e95e89bdecfe105d5bdbfce69b28d10a429
+size 197280

3_Normalize/config.json

@@ -0,0 +1,4 @@
+{
+    "module_input_name": "sentence_embedding",
+    "module_output_name": "sentence_embedding"
+}

README.md

@@ -0,0 +1,124 @@
+---
+license: apache-2.0
+language:
+- en
+library_name: sentence-transformers
+pipeline_tag: sentence-similarity
+base_model: sentence-transformers/all-MiniLM-L6-v2
+datasets:
+- FronkonGames/steam-games-dataset
+tags:
+- transformers.js
+- onnx
+- sentence-transformers
+- sentence-similarity
+- feature-extraction
+- recommendation
+- video-games
+- steam
+---
+
+# gamespace.2
+
+gamespace.2 embeds video games into a 128-dimensional space where games that the same people enjoy
+sit close together. It powers the [GameMerge](https://huggingface.co/spaces/GulkoA/gamemerge)
+recommender. Output vectors are L2-normalised, so a dot product is the cosine similarity.
+
+It is [all-MiniLM-L6-v2](https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2) fine-tuned on
+Steam store data, like [gamespace.1](https://huggingface.co/GulkoA/gamespace.1). It also learns from
+player behaviour: pairs of games that the same Steam reviewers engaged with are positive examples.
+The model still reads only a game's store text, so it works for any game, including new ones with no
+players yet.
+
+## Steam next-game benchmark
+
+The benchmark follows the protocol of SASRec (Kang & McAuley, ICDM 2018) on their Steam review crawl.
+Each user's most recent reviewed game is held out and ranked against 100 randomly sampled games, and
+also against the whole catalog. Text models score candidates by cosine similarity to the mean of the
+user's earlier games. Games no longer on the Steam store have no text to embed, so they were removed
+before the five-review filter. That leaves 250,955 users, 10,318 games and 3.05M interactions. The
+paper had 334,730 users and 13,047 games. Popularity and SASRec were re-run on this exact split.
+
+| Model | Learns from user histories | Hit@10, 100 negatives | NDCG@10, 100 negatives | Hit@10, all games | NDCG@10, all games |
+|---|---|---|---|---|---|
+| SASRec, reproduced | yes | 0.836 | 0.581 | 0.115 | 0.060 |
+| Popularity | yes | 0.688 | 0.420 | 0.050 | 0.025 |
+| **gamespace.2** | **pairs from training histories** | **0.511** | **0.301** | **0.034** | **0.018** |
+| gamespace.2 + CLIP header art | pairs from training histories | 0.520 | 0.309 | 0.037 | 0.019 |
+| gamespace.1 | no | 0.438 | 0.251 | 0.027 | 0.014 |
+| e5-large-v2 | no | 0.404 | 0.234 | 0.030 | 0.016 |
+| Qwen3-Embedding-0.6B | no | 0.336 | 0.185 | 0.018 | 0.010 |
+| bge-large-en-v1.5 | no | 0.334 | 0.191 | 0.026 | 0.014 |
+| all-MiniLM-L6-v2 | no | 0.306 | 0.162 | 0.017 | 0.010 |
+| CLIP header art only | no | 0.288 | 0.158 | 0.014 | 0.007 |
+
+For reference, the SASRec paper reports 0.873 and 0.631 on its full 13,047-game set.
+
+gamespace.2 is the best of the text embedding models, ahead of much larger general-purpose embedders.
+Sequential recommenders trained on the interaction data remain far stronger at predicting the next
+game, because they also learn popularity and order. gamespace.2 trades that for working on any game
+from its description alone.
+
+## Training
+
+- **Data.** The [Steam Games Dataset](https://huggingface.co/datasets/FronkonGames/steam-games-dataset)
+  has 147,352 apps. Soundtracks, demos, playtests, servers, SDKs, and empty records were removed,
+  leaving 138,920 games.
+- **Objective.** Contrastive training uses `CachedMultipleNegativesRankingLoss` with 128 in-batch
+  negatives. There are three kinds of positive pairs:
+  - 70,000 pairs are two randomly degraded views of the same game.
+  - 70,000 pairs are games whose vote-weighted user-tag profiles have a cosine of at least 0.6.
+  - 66,904 pairs are co-liked games. There are 36,371 distinct pairs, each shown in both orders.
+    They are mined from the benchmark's training histories only, so no validation or test game is
+    used. A pair must be shared by at least 5 reviewers, it is scored by normalised PMI, and each
+    game keeps its 10 strongest partners.
+- **Settings.** The projection is `Dense(384→128)`, initialised from PCA. Training runs one epoch at
+  learning rate 3e-5, with 128-token documents.
+
+A looser pair set, with at least 3 shared reviewers and 20 partners per game, scored lower on
+validation, with 0.477 against 0.503 Hit@10. Pair quality matters more than quantity.
+
+## Usage
+
+The model reads one short document per game. Build it with `docs.py` from this repo, using the same
+format as gamespace.1:
+
+```
+Hollow Knight. Genres: Action, Adventure, Indie. Tags: Metroidvania, Souls-like, ... Modes: Single-player. Developer: Team Cherry. Forge your own path...
+```
+
+```python
+from sentence_transformers import SentenceTransformer
+model = SentenceTransformer("GulkoA/gamespace.2")
+vec = model.encode("Hollow Knight. Genres: Action, Adventure, Indie. Tags: Metroidvania, Souls-like.")
+```
+
+### In the browser
+
+`onnx/model_quantized.onnx` is 22 MB, per-channel int8, with cosine 0.995 to the PyTorch model. It
+runs with [transformers.js](https://huggingface.co/docs/transformers.js). Apply the 128x384 projection
+in `dense.bin`, which holds float32 weights in row-major order followed by the 128 biases, to the
+mean-pooled output:
+
+```js
+import { pipeline } from "@huggingface/transformers";
+
+const extractor = await pipeline("feature-extraction", "GulkoA/gamespace.2", { dtype: "q8" });
+const w = new Float32Array(await (await fetch("https://huggingface.co/GulkoA/gamespace.2/resolve/main/dense.bin")).arrayBuffer());
+const { data: x } = await extractor("cozy farming game with fishing", { pooling: "mean", normalize: false });
+const y = Array.from({ length: 128 }, (_, i) => w[128 * 384 + i] + x.reduce((s, xj, j) => s + w[i * 384 + j] * xj, 0));
+const norm = Math.hypot(...y);
+const vec = y.map((v) => v / norm);
+```
+
+## Limitations
+
+- The behaviour pairs come from reviews written between 2010 and January 2018. Newer games benefit
+  only through what the model generalises from text.
+- Steam display names identify reviewers, and different people can share a name.
+- Similarity ignores popularity and quality, so obscure games rank alongside well-known ones.
+
+## License
+
+The model is licensed under Apache 2.0, the same as the base model. The store data is MIT-licensed.
+The review crawl is from the UCSD Steam datasets (Kang & McAuley, 2018).

config.json

@@ -0,0 +1,30 @@
+{
+  "add_cross_attention": false,
+  "architectures": [
+    "BertModel"
+  ],
+  "attention_probs_dropout_prob": 0.1,
+  "bos_token_id": null,
+  "classifier_dropout": null,
+  "dtype": "float32",
+  "eos_token_id": null,
+  "gradient_checkpointing": false,
+  "hidden_act": "gelu",
+  "hidden_dropout_prob": 0.1,
+  "hidden_size": 384,
+  "initializer_range": 0.02,
+  "intermediate_size": 1536,
+  "is_decoder": false,
+  "layer_norm_eps": 1e-12,
+  "max_position_embeddings": 512,
+  "model_type": "bert",
+  "num_attention_heads": 12,
+  "num_hidden_layers": 6,
+  "pad_token_id": 0,
+  "position_embedding_type": "absolute",
+  "tie_word_embeddings": true,
+  "transformers_version": "5.22.2",
+  "type_vocab_size": 2,
+  "use_cache": false,
+  "vocab_size": 30522
+}

config_sentence_transformers.json

@@ -0,0 +1,14 @@
+{
+  "__version__": {
+    "pytorch": "2.14.1+cu126",
+    "sentence_transformers": "6.2.0",
+    "transformers": "5.22.2"
+  },
+  "default_prompt_name": null,
+  "model_type": "SentenceTransformer",
+  "prompts": {
+    "document": "",
+    "query": ""
+  },
+  "similarity_fn_name": "cosine"
+}

dense.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0256c1aee86727d21a96829fd9a858603adea66b6012176a9bad6a96f82bb602
+size 197120

docs.py

@@ -0,0 +1,41 @@
+"""Turn a catalog row into the text document the embedding model reads."""
+import numpy as np
+
+KEEP_CATEGORIES = {
+    "Single-player", "Multi-player", "Co-op", "Online Co-op", "LAN Co-op", "Shared/Split Screen Co-op",
+    "PvP", "Online PvP", "Shared/Split Screen PvP", "MMO", "VR Support", "VR Only", "Cross-Platform Multiplayer",
+}
+
+
+def build_doc(row, rng: np.random.Generator | None = None, drop_tags: bool = False, max_tags: int = 15) -> str:
+    """Deterministic when rng is None. With rng, returns a randomly degraded view for contrastive training."""
+    aug = rng is not None
+    tags = list(row["tags"])[:max_tags]
+    if drop_tags or (aug and rng.random() < 0.5):
+        tags = []
+    elif aug and tags:
+        keep = rng.random(len(tags)) >= 0.3
+        tags = [t for t, k in zip(tags, keep) if k]
+    genres = list(row["genres"])
+    cats = [c for c in row["categories"] if c in KEEP_CATEGORIES][:5]
+    desc = (row["short"] + " " + row["about"]).strip()
+    if aug and desc:
+        if rng.random() < 0.3 and (tags or genres):
+            desc = ""
+        else:
+            words = desc.split()
+            cut = int(len(words) * rng.uniform(0.3, 1.0)) if len(words) > 20 else len(words)
+            desc = " ".join(words[:cut])
+
+    parts = [row["name"] + "."]
+    if genres:
+        parts.append("Genres: " + ", ".join(genres) + ".")
+    if tags:
+        parts.append("Tags: " + ", ".join(tags) + ".")
+    if cats:
+        parts.append("Modes: " + ", ".join(cats) + ".")
+    if len(row["developers"]):
+        parts.append("Developer: " + ", ".join(list(row["developers"])[:2]) + ".")
+    if desc:
+        parts.append(desc)
+    return " ".join(parts)

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:447eefdc22a444719100b151bb331ed6f71381427e27c030ed293d52c32f77b3
+size 90864192

modules.json

@@ -0,0 +1,26 @@
+[
+  {
+    "idx": 0,
+    "name": "0",
+    "path": "",
+    "type": "sentence_transformers.base.modules.transformer.Transformer"
+  },
+  {
+    "idx": 1,
+    "name": "1",
+    "path": "1_Pooling",
+    "type": "sentence_transformers.sentence_transformer.modules.pooling.Pooling"
+  },
+  {
+    "idx": 2,
+    "name": "2",
+    "path": "2_Dense",
+    "type": "sentence_transformers.base.modules.dense.Dense"
+  },
+  {
+    "idx": 3,
+    "name": "3",
+    "path": "3_Normalize",
+    "type": "sentence_transformers.base.modules.normalize.Normalize"
+  }
+]

onnx/model.onnx

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:403bf5ef3313e841d3bd384b7ac4c68dc78752078984f6aacb5d0aec5412d4c8
+size 90373210

onnx/model_quantized.onnx

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:db7047b3e693ca3ab70b7f6480dfc8198f860d77ac65c0adf611a100b211f0c2
+size 22965180

sentence_bert_config.json

@@ -0,0 +1,10 @@
+{
+    "transformer_task": "feature-extraction",
+    "modality_config": {
+        "text": {
+            "method": "forward",
+            "method_output_name": "last_hidden_state"
+        }
+    },
+    "module_output_name": "token_embeddings"
+}

tokenizer_config.json

@@ -0,0 +1,24 @@
+{
+  "backend": "tokenizers",
+  "cls_token": "[CLS]",
+  "do_basic_tokenize": true,
+  "do_lower_case": true,
+  "is_local": false,
+  "local_files_only": false,
+  "mask_token": "[MASK]",
+  "max_length": 128,
+  "model_max_length": 128,
+  "never_split": null,
+  "pad_to_multiple_of": null,
+  "pad_token": "[PAD]",
+  "pad_token_type_id": 0,
+  "padding_side": "right",
+  "sep_token": "[SEP]",
+  "stride": 0,
+  "strip_accents": null,
+  "tokenize_chinese_chars": true,
+  "tokenizer_class": "BertTokenizer",
+  "truncation_side": "right",
+  "truncation_strategy": "longest_first",
+  "unk_token": "[UNK]"
+}