Hangman SlimBERT Multitask Model
This is a small BERT-style character model trained from scratch for English movie-title Hangman.
The model predicts the next useful letter from a partial board, such as:
GENUINE EXPERIENCE -> _EN__NE E__E__EN_E
It uses two prediction heads:
- MLM head: predicts hidden characters at each blank position.
- Letter head: predicts which alphabet letters are still present anywhere in the title.
In gameplay, the MLM head performed best, so the default inference setting is:
letter_weight = 0.0
mlm_weight = 1.0
Model Details
| Item | Value |
|---|---|
| Architecture | Custom SlimBERT multitask model |
| Input | Character IDs, attention mask, missed-letter vector |
| Output | Per-position MLM logits and 26-way letter logits |
| Task | Hangman next-letter prediction |
| Training data | English movie titles |
| Dataset | anilsathyan7/hangman-movie-titles |
This checkpoint uses custom model classes and gameplay code from the project repository. It is not directly loadable with AutoModel alone.
Training
The model was trained with Hugging Face Trainer on dynamically generated Hangman game states.
Main setup:
- Fixed train/validation/test split
- Dynamic board masking
- Missed-letter input
- 30% late-game sampling
- Validation loss used for best checkpoint selection
- Weights & Biases used for logging
Evaluation
Gameplay evaluation used the held-out test split from the Hangman movie-title dataset.
| Index Fallback | Win Rate | Avg Fails | Avg Guesses | Avg Score |
|---|---|---|---|---|
| No | 0.9689 | 0.9503 | 5.2058 | 0.8784 |
| Yes | 0.9899 | 0.7029 | 4.9653 | 0.9117 |
Best validation checkpoint metrics:
| Metric | Value |
|---|---|
| Eval loss | 1.0504 |
| MLM masked accuracy | 0.7056 |
| Letter top-1 accuracy | 0.9156 |
Usage
This model needs the project code to run inference or evaluation.
Clone the project repository, place this checkpoint as the model directory, and use the project scripts described in the project README.
Github: https://github.com/anilsathyan7/hangman-ai
Limitations
- Supports English alphabetic movie titles only.
- Numbers and special characters are removed from the training data.
- Very noisy or invented title words are difficult to predict reliably.
- Late-game states can still be ambiguous when several letters fit the same pattern.