|
Download README.md from DJLougen/pixelgen: direct link, hf CLI and curl.
- Browser
- Download file 9.22 kB
-
https://huggingface.co/DJLougen/pixelgen/resolve/main/README.md
- Command line
-
hf download hf://DJLougen/pixelgen/README.md
-
curl -L -o README.md https://huggingface.co/DJLougen/pixelgen/resolve/main/README.md
9.22 kB
| license: mit | |
| library_name: pytorch | |
| pipeline_tag: text-to-image | |
| tags: | |
| - pixel-art | |
| - weight-space-programming | |
| - hand-written-weights | |
| - no-training | |
| - tiny-model | |
| - pico-8 | |
| - toy | |
| language: | |
| - en | |
| <div align="center"> | |
| # πΉοΈ PixelGen | |
| **A text-to-pixel-art model with 1,257 parameters. Zero of them were trained.** | |
| Every weight was typed in by hand. It still works. | |
| <img src="showcase.png" alt="11 sprite classes Γ 9 colour ramps, all generated by PixelGen" width="100%"/> | |
| <sub>Every sprite above: `"<colour> <class>"` β PixelGen β 16Γ16 PICO-8 sprite. No gradients were harmed.</sub> | |
| <br/> | |
|  | |
|  | |
|  | |
|  | |
|  | |
| </div> | |
| --- | |
| ## What this is | |
| A tiny PyTorch `nn.Module` that turns a text prompt like `"red potion"` or `"blue invader"` into a 16Γ16 pixel-art sprite in the PICO-8 palette, upscaled to 128Γ128. | |
| It's an experiment in **weight-space programming**, inspired by [this post](https://rikiyat.github.io/blog/weight-space-programming/): instead of training a network, you write a program and *compile it into weights*. Linear layers become lookup tables, argmax becomes a decision, a 1Γ1 conv becomes a palette. The model is a real `nn.Module` β you can call `.forward()`, save a `state_dict`, load it back β but no optimizer ever touched it. | |
| The fun part is that an image generator with the parameter count of a single small MLP layer produces consistently cute, correctly-coloured, correctly-shaped sprites, and you can read the entire "model" as source code. | |
| ## Numbers (measured, not estimated) | |
| | | | | |
| |---|---| | |
| | Registered parameters | **1,257** | | |
| | Non-trainable buffers (shape/shade/ramp banks) | 3,611 | | |
| | Total values in the checkpoint | 4,868 (β19.5 KB fp32) | | |
| | `pixelgen.pt` on disk | 23.5 KB | | |
| | Training steps | **0** | | |
| | Optimizers, backward passes, `requires_grad=True` in the repo | 0 (grep-verified) | | |
| | Sprite classes | 11 | | |
| | Colour ramps | 9 | | |
| | Output | 16Γ16 β 128Γ128 RGB, every pixel β PICO-8 (16 colours) | | |
| | Build reproducibility | `build.py` twice β identical SHA-256 | | |
| | `pixelgen.pt` SHA-256 | `175cc5249fba8c756ea019f418f4aa0a9633017ffad7f238b863865758c49b80` | | |
| <details> | |
| <summary>Parameter breakdown</summary> | |
| | Tensor | Shape | Count | What it is | | |
| |---|---|---|---| | |
| | `word_to_class.weight` | 11 Γ 59 | 649 | vocab β class scores (hand-set synonym table) | | |
| | `word_to_class.bias` | 11 | 11 | default class bias (potion) | | |
| | `word_to_ramp.weight` | 9 Γ 59 | 531 | vocab β colour-ramp scores | | |
| | `word_to_ramp.bias` | 9 | 9 | default ramp bias (green) | | |
| | `outline.weight` | 1Γ1Γ3Γ3 | 9 | 3Γ3 dilation kernel | | |
| | `palette_conv.weight` | 3Γ16Γ1Γ1 | 48 | one-hot colour β RGB (exact PICO-8 values Γ· 255) | | |
| | **Parameters** | | **1,257** | | | |
| | `on_bank` / `free_bank` | 11Γ16Γ8 Γ2 | 2,816 | half-sprite templates (`#` solid, `?` noise-controlled) | | |
| | `shade_bank` | 3Γ16Γ16 | 768 | light / mid / dark row masks | | |
| | `ramp_table` | 9Γ3 | 27 | palette indices per ramp | | |
| | **Buffers** | | **3,611** | | | |
| </details> | |
| ## How it works | |
| The forward pass is nine steps. Each one is a normal tensor op; together they're a program. | |
| ``` | |
| "red potion" | |
| β | |
| βΌ | |
| 1. bag of words prompt β count vector over a 59-word vocab | |
| 2. class WTA Linear(59β11) β argmax β one-hot class [potion] | |
| 3. shape bank one-hot selects a 16Γ8 half-template; | |
| '?' cells flip on when seed-noise β₯ 0.5 | |
| 4. mirror half + horizontal flip β 16Γ16 body (always symmetric) | |
| 5. outline 3Γ3 dilate(body) β body β outline pixels (palette index 1) | |
| 6. shading rows 0β6 light, 7β10 mid, 11β15 dark | |
| 7. ramp WTA Linear(59β9) β argmax β colour ramp [red] | |
| 8. one-hot palette every pixel β one-hot over 16 PICO-8 colours | |
| 9. palette conv Conv2d(16β3, 1Γ1) β RGB | |
| β | |
| βΌ | |
| 16Γ16 sprite β nearest-neighbour Γ8 β 128Γ128 PNG | |
| ``` | |
| The "generative" part is step 3: templates have `?` cells that are gated by per-seed noise, so the same prompt gives small variations across seeds: | |
| <div align="center"> | |
| <img src="seeds.png" alt="blue invader, seeds 0β7" width="80%"/> | |
| <sub><code>"blue invader"</code>, seeds 0 through 7</sub> | |
| </div> | |
| ## How to use | |
| ```bash | |
| git clone https://huggingface.co/DJLougen/pixelgen && cd pixelgen | |
| pip install torch numpy pillow # that's the whole dependency list | |
| python generate.py "red potion" --seed 0 --out out/red_potion.png | |
| python generate.py "golden key" --seed 3 --scale 16 | |
| ``` | |
| Or from Python: | |
| ```python | |
| from pixelgen.model import PixelGen | |
| model = PixelGen() # weights are built in __init__, no file needed | |
| img = model.sprite("purple slime", seed=4, scale=8) # PIL.Image, 128Γ128 | |
| img.save("slime.png") | |
| ``` | |
| The `pixelgen.pt` checkpoint is included for completeness (`pixelgen.weights.load(path)`), but it's redundant β `PixelGen()` reconstructs identical weights from source every time. | |
| **Prompt vocabulary.** Class words (and synonyms): | |
| | class | words | | |
| |---|---| | |
| | potion | potion, bottle, flask, elixir, vial | | |
| | sword | sword, blade, dagger | | |
| | shield | shield, buckler | | |
| | ship | ship, rocket, spaceship, starship | | |
| | invader | invader, alien, monster | | |
| | slime | slime, blob, jelly | | |
| | skull | skull, bone, skeleton | | |
| | tree | tree, pine, forest | | |
| | key | key | | |
| | heart | heart, love, life | | |
| | star | star, sparkle | | |
| Colour words: `red blue green yellow purple orange pink brown white`, plus synonyms (`crimson ruby β red`, `azure sapphire β blue`, `emerald lime β green`, `gold golden β yellow`, `violet amethyst β purple`, `amber β orange`, `rose β pink`, `wood wooden β brown`, `grey gray silver bone β white`). Anything else is ignored. Fun side effect: `"bone"` is both a skull synonym and a white synonym, so `"bone"` alone gives a white skull. | |
| ## Limitations | |
| Being upfront, because this is the part that matters: | |
| - **It only knows 11 things.** Prompt for a dragon and you get a potion (the default class). There is no generalisation β zero β because there is no learned representation. The "model" is a lookup table with a tokenizer in front of it. | |
| - **Every sprite is horizontally symmetric.** The mirror step is what makes 16Γ8 templates cheap, and it's also why the key had to be drawn as a top-down key rather than a side-view one. | |
| - **Variation is cosmetic.** Seeds only toggle the hand-marked `?` cells (a few pixels per sprite). Two seeds of the same prompt are the same sprite with slightly different speckles. | |
| - **Shading is positional, not structural.** Light/mid/dark bands are fixed rows, so a tall sprite and a short sprite get lit differently for no good reason. | |
| - **Multi-word prompts resolve by word count, not meaning.** `"sword shield"` is a tie broken by index order. `"not red"` is red. | |
| - **Colour choice is global.** One ramp per sprite; no two-tone sprites. | |
| - **It's PyTorch for the bit, not for the need.** You could rewrite this in 100 lines of NumPy. Using `nn.Linear` and `nn.Conv2d` is the point of the exercise, not a requirement. | |
| - **Art quality is "programmer art".** The templates were hand-drawn in a text editor by someone who is not a pixel artist. `invader` and `key` went through a few iterations; the rest are first drafts. | |
| ## Why bother? | |
| Because it's a nice demonstration that a lot of what we call "a model" is just *structure*, and structure can be written directly. A linear layer with the right weights *is* a dictionary. A conv with the right kernel *is* an edge detector. Winner-take-all *is* an `if`. The blog post argues this is a useful lens for understanding what trained networks are doing; this repo is the version where you skip the training and just write the weights. | |
| Also it's cool that it worked. | |
| ## Repo contents | |
| ``` | |
| pixelgen/ | |
| model.py PixelGen nn.Module β all weights set in __init__ | |
| templates.py 11 half-sprite templates ('#' solid, '?' noise, '.' empty) | |
| palette.py PICO-8 palette + 9 colour ramps | |
| tokenizer.py vocab + synonyms | |
| weights.py save/load helpers | |
| build.py writes weights/pixelgen.pt (deterministic) | |
| generate.py CLI | |
| tests/ 10 property tests (shape, palette membership, determinism, symmetry, β¦) | |
| weights/pixelgen.pt | |
| showcase.png the grid at the top | |
| seeds.png seed variation strip | |
| ``` | |
| Tests: `python -m pytest tests -q` β 10 passed. | |
| ## Citation | |
| ```bibtex | |
| @misc{pixelgen2025, | |
| title = {PixelGen: a hand-weighted pixel-art generator}, | |
| author = {Lougen, D. J.}, | |
| year = {2025}, | |
| url = {https://huggingface.co/DJLougen/pixelgen}, | |
| note = {Weight-space programming experiment; no parameters were trained.} | |
| } | |
| ``` | |
| Inspired by *[Weight-Space Programming](https://rikiyat.github.io/blog/weight-space-programming/)*. | |