pixie-16 / README.md
EmilioSchwaiger's picture
Release pixie-16
8c84f54 verified
|
Raw History Blame Contribute Delete
4.61 kB
metadata
license: apache-2.0
library_name: pytorch
pipeline_tag: text-to-image
tags:
  - pixel-art
  - sprite-generation
  - text-to-image
datasets:
  - Angais/pixel-art-atlas-1.4m

Pixie v3 (pixie-16)

A 66M-parameter autoregressive model that paints 16×16 palette-indexed pixel-art sprites from a short text prompt. Code, training pipeline and sampler: github.com/QuarkOS/pixie. Sibling model: EmilioSchwaiger/pixie-32.

Files: model.safetensors (weights), config.json (architecture), palette.npy (4097×3 uint8 palette; index 0 is transparent).

Raw samples (no ranking): red apple, wooden chair, blue teapot, green backpack, iron hammer, yellow raincoat (6 each, seed 0, CFG 2.0)

Model

Field Value
Name Pixie v3 / P50 (pixie-16)
Version Checkpoint served by the Pixie chat app: run v3-full-s7-condboost40M, file full_064M. 64M image views in total: a 36M-view constant-LR run decayed to 40M, then a 24M-view decay fine-tune (step 187,500 of that fine-tune) with heavier palette and style conditioning.
Architecture v3 / P50: autoregressive transformer over palette indices. Width 512, 16 layers, 8 heads, 2×2 patch tokens (64 patches of four palette slots), slot-aware causal head, three palette slots and one style slot. See the README.
Parameters 66,309,632 (counted from the released model.safetensors; 224 float32 tensors)
Checkpoint SHA256 fd6507350aedf58fa839fd94b95f84f889b72c7fc1e4c6f4bf66e2f6dd6178e8 (model.safetensors)
Output One 16×16 RGBA sprite per sample; index 0 is transparent
Text encoder External, not shipped: the first 11 layers of the openai/clip-vit-large-patch14 text model (penultimate features, 18 positions)
Intended use Generate 16×16 palette-indexed pixel-art game icons and sprites from a short English prompt.
Out of scope Photoreal images, text rendering, anything larger than 16×16, and subjects far outside game-sprite content.

Training data

Field Value
Training set About 1.36M training rows built from Angais/pixel-art-atlas-1.4m (CC0-1.0; synthetic 16×16 sprites generated with Krea 2 Turbo and a pixel-art LoRA), with typo-augmented caption copies.
License of the data CC0-1.0. The atlas images are Krea 2 Outputs; see NOTICE.
Holdouts excluded The Blind-50 and Eval-50 phrase lists were dropped from training (1,635 Blind-50 phrase/subject rows and 118 Eval-50 exact captions). The lists are not published.
Personal data None in the weights files. Captions come from the dataset.

Weights license

Field Value
License Apache-2.0, the same as the code. See LICENSE and NOTICE.
Inherited restrictions None claimed. No Krea or CLIP weights are part of the release.

Evaluation

Field Value
IID validation NLL 1.0231 (with palette/style conditioning: 0.9698; held-out-subject probe: 1.0896), from the run summary at 64M views. Not comparable with the P32 number: different resolution and data.
Blind-50 / Eval-50 Not reported as a number for this release.

Known limitations

  • 16×16 only; small details (faces, text, thin tools) are often lost.
  • Raw samples vary. The chat app draws 64 candidates per prompt and ranks them with a reward model fitted on preference votes; that ranker is not released, so expect to pick from several samples yourself.
  • Sprites tend to fill the whole 16×16 cell.
  • Needs the external CLIP ViT-L/14 text encoder, which is linked, not shipped.

How to sample

git clone https://github.com/QuarkOS/pixie && cd pixie
python3 -m venv .venv && . .venv/bin/activate
pip install torch numpy pillow tokenizers safetensors huggingface_hub
hf download EmilioSchwaiger/pixie-16 --local-dir pixie-16
hf download openai/clip-vit-large-patch14 model.safetensors tokenizer.json --local-dir clip-vit-large-patch14
python -m train.sample_pixie --model-dir pixie-16 \
  --clip clip-vit-large-patch14/model.safetensors \
  --tokenizer clip-vit-large-patch14/tokenizer.json \
  --prompt "red apple" --prompt "blue teapot" --n 4 --out sprites.png

Each row of sprites.png is one prompt (16×16 sprites, upscaled with nearest-neighbour).

License

Apache-2.0 for the weights. See LICENSE and NOTICE (Krea 2 attribution and third-party credits).