README.md

---
license: apache-2.0
pipeline_tag: reinforcement-learning
tags:
  - vizdoom
  - doom
  - reinforcement-learning
  - ppo
  - behavior-cloning
  - modernbert
datasets:
  - anakin87/doom-defend-the-center-100k-oracle
model-index:
  - name: tiny-doom-defender
    results:
      - task:
          type: reinforcement-learning
          name: reinforcement-learning
        dataset:
          name: doom_defend_the_center
          type: doom_defend_the_center
        metrics:
          - type: mean_kills
            value: 23.12 +/- 2.81
            name: mean_kills
            verified: false
---

# tiny-doom-defender

A **1.1M-parameter** ModernBERT encoder that plays VizDoom's *Defend the Center* in real time on a CPU from pixels alone.

📝 [Read the full story/see the model play live](https://huggingface.co/spaces/anakin87/tiny-doom-defender) · 💻 [GitHub repo](https://github.com/anakin87/tiny-doom-defender)

![cover](./cover.png)

Inspired by [VAGOsolutions/SauerkrautLM-Doom-MultiVec-1.3M](https://huggingface.co/VAGOsolutions/SauerkrautLM-Doom-MultiVec-1.3M).

The model underwent SFT from a programmatic oracle and has been refined with PPO.

# Checkpoints in this repo

| Checkpoint | Path | Training | Mean kills | σ | Size |
| --- | --- | --- | ---: | ---: | ---: |
| **tiny-doom-defender** (default) | *root* | SFT + PPO | 23.12 | 2.81 | 4.4 MB |
| tiny-doom-defender-sft | `sft/` | SFT only | 22.19 | 3.76 | 4.4 MB |
| tiny-doom-defender-int8 | `int8/` | SFT + PPO, int8 | 23.13 | 2.78 | **1.1 MB** |

SauerkrautLM-Doom-MultiVec-1.3M (the inspiration) kills 20.38 enemies, with a standard deviation of 5.35.

Evaluation on 1000 episodes, seeds 10000–10999.

## Usage

Install the package, download the repo, and let it play:

```bash
pip install git+https://github.com/anakin87/tiny-doom-defender
hf download anakin87/tiny-doom-defender --local-dir tiny-doom-defender

# watch it play in a live DOOM window
play-doom --ckpt tiny-doom-defender

# score it on the held-out test seeds
eval-model --ckpt tiny-doom-defender --episodes 100
```

Point `--ckpt` at a subfolder to use the other checkpoints:

```bash
eval-model --ckpt tiny-doom-defender/sft  --episodes 100
eval-model --ckpt tiny-doom-defender/int8 --episodes 100
```

## Architecture

| | |
| --- | --- |
| Parameters | 1,096,134 |
| Observation | 3 stacked 160×100 RGB frames (t, t−1, t−2) + the 2 previous actions |
| Stem | two stride-2 3×3 convs, 32 channels → a 40×25 = 1000-token grid |
| Encoder | 4-layer ModernBERT, hidden size 128, 4 heads, fed via `inputs_embeds` |
| Head | learned attention pooling → turn (3) / shoot (2) / value heads |
| Action space | `MultiDiscrete([3, 2])` — turn left/none/right × shoot/no-shoot |

![Tiny Doom Defender architecture](architecture.png)