Initial upload

README.md

@@ -0,0 +1,85 @@
+---
+license: apache-2.0
+pipeline_tag: reinforcement-learning
+tags:
+  - vizdoom
+  - doom
+  - reinforcement-learning
+  - ppo
+  - behavior-cloning
+  - modernbert
+datasets:
+  - anakin87/doom-defend-the-center-100k-oracle
+model-index:
+  - name: tiny-doom-defender
+    results:
+      - task:
+          type: reinforcement-learning
+          name: reinforcement-learning
+        dataset:
+          name: doom_defend_the_center
+          type: doom_defend_the_center
+        metrics:
+          - type: mean_kills
+            value: 23.12 +/- 2.81
+            name: mean_kills
+            verified: false
+---
+
+# tiny-doom-defender
+
+![cover](https://github.com/anakin87/tiny-doom-defender/blob/main/assets/cover.png?raw=true)
+
+A **1.1M-parameter** ModernBERT encoder that plays VizDoom's *Defend the Center* in real time on a CPU from pixels alone.
+
+Inspired from [VAGOsolutions/SauerkrautLM-Doom-MultiVec-1.3M](https://huggingface.co/VAGOsolutions/SauerkrautLM-Doom-MultiVec-1.3M).
+
+The model underwent SFT from a programmatic oracle and has been refined with PPO.
+
+📝 Read the full story: [blog post](ADD_LINK) · 💻 [GitHub repo](https://github.com/anakin87/tiny-doom-defender)
+
+
+## Checkpoints in this repo
+
+| Checkpoint | Path | Training | Mean kills | σ | Size |
+| --- | --- | --- | ---: | ---: | ---: |
+| **tiny-doom-defender** (default) | *root* | SFT + PPO | 23.12 | 2.81 | 4.4 MB |
+| tiny-doom-defender-sft | `sft/` | SFT only | 22.19 | 3.76 | 4.4 MB |
+| tiny-doom-defender-int8 | `int8/` | SFT + PPO, int8 | 23.13 | 2.78 | **1.1 MB** |
+
+Evaluation on 1000 episodes, seeds 10000–10999.
+
+## Usage
+
+Install the package, download the repo, and let it play:
+
+```bash
+pip install git+https://github.com/anakin87/tiny-doom-defender
+hf download anakin87/tiny-doom-defender --local-dir tiny-doom-defender
+
+# watch it play in a live DOOM window
+play-doom --ckpt tiny-doom-defender
+
+# score it on the held-out test seeds
+eval-model --ckpt tiny-doom-defender --episodes 100
+```
+
+Point `--ckpt` at a subfolder to use the other checkpoints:
+
+```bash
+eval-model --ckpt tiny-doom-defender/sft  --episodes 100
+eval-model --ckpt tiny-doom-defender/int8 --episodes 100
+```
+
+## Architecture
+
+| | |
+| --- | --- |
+| Parameters | 1,096,134 |
+| Observation | 3 stacked 160×100 RGB frames (t, t−1, t−2) + the 2 previous actions |
+| Stem | two stride-2 3×3 convs, 32 channels → a 40×25 = 1000-token grid |
+| Encoder | 4-layer ModernBERT, hidden size 128, 4 heads, fed via `inputs_embeds` |
+| Head | learned attention pooling → turn (3) / shoot (2) / value heads |
+| Action space | `MultiDiscrete([3, 2])` — turn left/none/right × shoot/no-shoot |
+
+![Tiny Doom Defender architecture](architecture.png)

config.json

@@ -0,0 +1,70 @@
+{
+  "architectures": [
+    "DoomConvStemPolicy"
+  ],
+  "dtype": "float32",
+  "encoder_config": {
+    "architectures": [
+      "ModernBertModel"
+    ],
+    "attention_bias": false,
+    "attention_dropout": 0.0,
+    "bos_token_id": 0,
+    "classifier_activation": "gelu",
+    "classifier_bias": false,
+    "classifier_dropout": 0.0,
+    "classifier_pooling": "cls",
+    "cls_token_id": 0,
+    "decoder_bias": true,
+    "deterministic_flash_attn": false,
+    "dtype": "float32",
+    "embedding_dropout": 0.0,
+    "eos_token_id": 0,
+    "global_attn_every_n_layers": 3,
+    "hidden_activation": "gelu",
+    "hidden_size": 128,
+    "initializer_cutoff_factor": 2.0,
+    "initializer_range": 0.02,
+    "intermediate_size": 512,
+    "layer_norm_eps": 1e-05,
+    "layer_types": [
+      "full_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "full_attention"
+    ],
+    "local_attention": 128,
+    "max_position_embeddings": 1536,
+    "mlp_bias": false,
+    "mlp_dropout": 0.0,
+    "model_type": "modernbert",
+    "norm_bias": false,
+    "norm_eps": 1e-05,
+    "num_attention_heads": 4,
+    "num_hidden_layers": 4,
+    "pad_token_id": 0,
+    "rope_parameters": {
+      "full_attention": {
+        "rope_theta": 160000.0,
+        "rope_type": "default"
+      },
+      "sliding_attention": {
+        "rope_theta": 10000.0,
+        "rope_type": "default"
+      }
+    },
+    "sep_token_id": 0,
+    "sparse_pred_ignore_index": -100,
+    "sparse_prediction": false,
+    "tie_word_embeddings": false,
+    "vocab_size": 32
+  },
+  "model_type": "doom_conv_stem",
+  "n_action_states": 7,
+  "n_frames": 3,
+  "n_prev_actions": 2,
+  "res_h": 100,
+  "res_w": 160,
+  "stem_channels": 32,
+  "transformers_version": "5.13.1"
+}

int8/config.json

@@ -0,0 +1,71 @@
+{
+  "architectures": [
+    "DoomConvStemPolicy"
+  ],
+  "dtype": "float32",
+  "encoder_config": {
+    "architectures": [
+      "ModernBertModel"
+    ],
+    "attention_bias": false,
+    "attention_dropout": 0.0,
+    "bos_token_id": 0,
+    "classifier_activation": "gelu",
+    "classifier_bias": false,
+    "classifier_dropout": 0.0,
+    "classifier_pooling": "cls",
+    "cls_token_id": 0,
+    "decoder_bias": true,
+    "deterministic_flash_attn": false,
+    "dtype": "float32",
+    "embedding_dropout": 0.0,
+    "eos_token_id": 0,
+    "global_attn_every_n_layers": 3,
+    "hidden_activation": "gelu",
+    "hidden_size": 128,
+    "initializer_cutoff_factor": 2.0,
+    "initializer_range": 0.02,
+    "intermediate_size": 512,
+    "layer_norm_eps": 1e-05,
+    "layer_types": [
+      "full_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "full_attention"
+    ],
+    "local_attention": 128,
+    "max_position_embeddings": 1536,
+    "mlp_bias": false,
+    "mlp_dropout": 0.0,
+    "model_type": "modernbert",
+    "norm_bias": false,
+    "norm_eps": 1e-05,
+    "num_attention_heads": 4,
+    "num_hidden_layers": 4,
+    "pad_token_id": 0,
+    "rope_parameters": {
+      "full_attention": {
+        "rope_theta": 160000.0,
+        "rope_type": "default"
+      },
+      "sliding_attention": {
+        "rope_theta": 10000.0,
+        "rope_type": "default"
+      }
+    },
+    "sep_token_id": 0,
+    "sparse_pred_ignore_index": -100,
+    "sparse_prediction": false,
+    "tie_word_embeddings": false,
+    "vocab_size": 32
+  },
+  "model_type": "doom_conv_stem",
+  "n_action_states": 7,
+  "n_frames": 3,
+  "n_prev_actions": 2,
+  "quantization": "int8-symmetric",
+  "res_h": 100,
+  "res_w": 160,
+  "stem_channels": 32,
+  "transformers_version": "5.13.1"
+}

int8/model_int8.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3028d8ea28b778759134c599c7b8178baa61732fbd72791adcfc5e71844b74b8
+size 1130802

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:50c3852e747cabaa6f981b50e730611723a069a9ac32c9d920f8e54a26ed2f61
+size 4388152

sft/config.json

@@ -0,0 +1,70 @@
+{
+  "architectures": [
+    "DoomConvStemForActionClassification"
+  ],
+  "dtype": "float32",
+  "encoder_config": {
+    "architectures": [
+      "ModernBertModel"
+    ],
+    "attention_bias": false,
+    "attention_dropout": 0.0,
+    "bos_token_id": 0,
+    "classifier_activation": "gelu",
+    "classifier_bias": false,
+    "classifier_dropout": 0.0,
+    "classifier_pooling": "cls",
+    "cls_token_id": 0,
+    "decoder_bias": true,
+    "deterministic_flash_attn": false,
+    "dtype": "float32",
+    "embedding_dropout": 0.0,
+    "eos_token_id": 0,
+    "global_attn_every_n_layers": 3,
+    "hidden_activation": "gelu",
+    "hidden_size": 128,
+    "initializer_cutoff_factor": 2.0,
+    "initializer_range": 0.02,
+    "intermediate_size": 512,
+    "layer_norm_eps": 1e-05,
+    "layer_types": [
+      "full_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "full_attention"
+    ],
+    "local_attention": 128,
+    "max_position_embeddings": 1536,
+    "mlp_bias": false,
+    "mlp_dropout": 0.0,
+    "model_type": "modernbert",
+    "norm_bias": false,
+    "norm_eps": 1e-05,
+    "num_attention_heads": 4,
+    "num_hidden_layers": 4,
+    "pad_token_id": 0,
+    "rope_parameters": {
+      "full_attention": {
+        "rope_theta": 160000.0,
+        "rope_type": "default"
+      },
+      "sliding_attention": {
+        "rope_theta": 10000.0,
+        "rope_type": "default"
+      }
+    },
+    "sep_token_id": 0,
+    "sparse_pred_ignore_index": -100,
+    "sparse_prediction": false,
+    "tie_word_embeddings": false,
+    "vocab_size": 32
+  },
+  "model_type": "doom_conv_stem",
+  "n_action_states": 7,
+  "n_frames": 3,
+  "n_prev_actions": 2,
+  "res_h": 100,
+  "res_w": 160,
+  "stem_channels": 32,
+  "transformers_version": "5.13.1"
+}

sft/model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ab7296cc1d9271b4f9dedc4732dbd0803cd7bb991155793a2b3f520041b7be3b
+size 4387476