holdoutlabs/gamedecider
Text Classification • Updated • 4
AI & ML interests
None defined yet.
Recent Activity
P1ayer-1
updated a
model 4 days ago
Post
103
GameDecider: a 400M model that answers your game's "what happens next?" questions, open weights (Apache-2.0
GameDecider is an encoder (Ettin-400M base) fine-tuned to take *a scene + a question + options* and return a
probability for each option, or say "not sure". It runs locally on CPU or GPU (ONNX: about 0.1 s per check on an RTX 3060 Ti, about 1 s on a 6-core desktop CPU; up to 5 questions about the same scene
cost the same as one), and it ships with compiled adapters for common NPC questions.
What's in the repo:
- ONNX weights (fp16/fp32) + tokenizer, calibrated thresholds:
(abstains more; for setups that hand "not sure" to an LLM)
- 39 compiled adapters for common NPC questions (threat, apology, persuasion, quest accept, ...), plus SkyrimNet-style memory matching and NPC mood, and a character-reaction adapter (14 reaction animations)
- Model card with evals, data sources and licences, a minor-safety section, and known limits; 200 sample rows
Honest numbers (private held-out sets, CIs in the card): 0.600 on task families it never saw (heldout_v3), 0.707 on real
game transcripts. Planning: 0.939 on our procedural "what should I do next to reach X?" eval, but the training data has
rows from the same generator, so treat that as in-family, not general planning; use an LLM or a planner for real plans.
Training data: ~334k decision rows from permissive datasets (MIT / Apache / CC-BY / ODC-By, all credited) plus
procedurally generated game scenarios written by open models. Removed before release: anything without a clear licence
and anything generated by models whose terms forbid training.
Try it: P1ayer-1/gamedecider-playground · Weights: holdoutlabs/gamedecider
Feedback very welcome — especially question types it gets wrong in your game.
GameDecider is an encoder (Ettin-400M base) fine-tuned to take *a scene + a question + options* and return a
probability for each option, or say "not sure". It runs locally on CPU or GPU (ONNX: about 0.1 s per check on an RTX 3060 Ti, about 1 s on a 6-core desktop CPU; up to 5 questions about the same scene
cost the same as one), and it ships with compiled adapters for common NPC questions.
What's in the repo:
- ONNX weights (fp16/fp32) + tokenizer, calibrated thresholds:
default (answers ~97% of the time) and strict(abstains more; for setups that hand "not sure" to an LLM)
- 39 compiled adapters for common NPC questions (threat, apology, persuasion, quest accept, ...), plus SkyrimNet-style memory matching and NPC mood, and a character-reaction adapter (14 reaction animations)
- Model card with evals, data sources and licences, a minor-safety section, and known limits; 200 sample rows
Honest numbers (private held-out sets, CIs in the card): 0.600 on task families it never saw (heldout_v3), 0.707 on real
game transcripts. Planning: 0.939 on our procedural "what should I do next to reach X?" eval, but the training data has
rows from the same generator, so treat that as in-family, not general planning; use an LLM or a planner for real plans.
Training data: ~334k decision rows from permissive datasets (MIT / Apache / CC-BY / ODC-By, all credited) plus
procedurally generated game scenarios written by open models. Removed before release: anything without a clear licence
and anything generated by models whose terms forbid training.
Try it: P1ayer-1/gamedecider-playground · Weights: holdoutlabs/gamedecider
Feedback very welcome — especially question types it gets wrong in your game.
- 5 replies
P1ayer-1
published a
model 5 days ago
Post
51
Fine-tuning a 1.7B NPC dialogue model beat the 4B we were using — on every check we care about.
The setup: our game determines how an NPC reacts (via a small classifier, GameDecider), and then a local LLM says it in character. The untuned Qwen3-4B kept repeating itself. So we generated ~18k persona-grounded training lines (life stage × temperament × mood × relationship, names/jobs/tics dealt by code so they don't collapse) and LoRA-tuned Qwen3-1.7B. 340-case test set, same cases for both:
- NPC reuses its own opening words: 64% → 22% (first two words: 31% → 3%)
- Exact quote on first try: 32% → 98%
- A blind reader can tell what the NPC meant: 70% → 81%
- Invented facts on trap questions: 20% → 3%
- Prompt injections that worked: 2.3% → 0%
- Speed: 0.41 s → 0.15 s per line on a laptop RTX 3080
Lesson: more of the same data flattened fast; the wins came from fixing diversity at the data level (code-dealt names, tics, openers) and filtering invented facts.
Caveats: automatic checks + a blind model reader, no human ratings yet.
Play with the decision side: https://p1ayer-1-gamedecider-playground.hf.space/mean?ref=hfpost2
Building a game? Get the model: https://p1ayer-1-gamedecider-playground.hf.space/alpha?ref=hfpost2
The setup: our game determines how an NPC reacts (via a small classifier, GameDecider), and then a local LLM says it in character. The untuned Qwen3-4B kept repeating itself. So we generated ~18k persona-grounded training lines (life stage × temperament × mood × relationship, names/jobs/tics dealt by code so they don't collapse) and LoRA-tuned Qwen3-1.7B. 340-case test set, same cases for both:
- NPC reuses its own opening words: 64% → 22% (first two words: 31% → 3%)
- Exact quote on first try: 32% → 98%
- A blind reader can tell what the NPC meant: 70% → 81%
- Invented facts on trap questions: 20% → 3%
- Prompt injections that worked: 2.3% → 0%
- Speed: 0.41 s → 0.15 s per line on a laptop RTX 3080
Lesson: more of the same data flattened fast; the wins came from fixing diversity at the data level (code-dealt names, tics, openers) and filtering invented facts.
Caveats: automatic checks + a blind model reader, no human ratings yet.
Play with the decision side: https://p1ayer-1-gamedecider-playground.hf.space/mean?ref=hfpost2
Building a game? Get the model: https://p1ayer-1-gamedecider-playground.hf.space/alpha?ref=hfpost2
- 5 replies
Post
73
GameDecider will be fully open source. The alpha is currently private. Email me at noah@holdoutlabs.ai or leave your Discord in the replies for access.
This game is a tech demo to gather validation data.
https://p1ayer-1-gamedecider-playground.hf.space/mean?ref=hfpost
Can you tell when someone doesn't mean it? Mean It? is a 3-minute daily game: 5 short lines (apologies, threats, alibis, sarcasm), you call sincere or faking. Then you get a secret brief and write a line to fool tomorrow's players.
So far players got 100% of the hidden-intent cards but only 50% of the apologies. Fake sorry is hard to spot.
This game is a tech demo to gather validation data.
https://p1ayer-1-gamedecider-playground.hf.space/mean?ref=hfpost
Can you tell when someone doesn't mean it? Mean It? is a 3-minute daily game: 5 short lines (apologies, threats, alibis, sarcasm), you call sincere or faking. Then you get a secret brief and write a line to fool tomorrow's players.
So far players got 100% of the hidden-intent cards but only 50% of the apologies. Fake sorry is hard to spot.
- 3 replies
Post
114
Can you fool an AI judge? Six NPCs, one line each: threaten, bribe or lie so smoothly that the judge says "Fooled it!" instead of "Caught". Post the best line that fooled it
P1ayer-1/gamedecider-playground
Attempts are saved (no personal info) to improve the judge.
P1ayer-1/gamedecider-playground
Attempts are saved (no personal info) to improve the judge.
- 5 replies