SpriteX32

A small text-to-image model that generates 32×32 pixel-art sprites from English prompts. It is a conditional U-Net trained with flow matching, conditioned on CLIP text embeddings, and sampled with classifier-free guidance. It has 11.4M parameters and runs comfortably on a CPU or a small GPU.

Seed 0, 50 steps, guidance 3.0, upscaled 4× with nearest-neighbour. Prompts, one per row, top to bottom:

1. a red sword
2. a green slime monster
3. a blue wizard
4. Once it takes aim at its foe, it makes a headlong charge. It is famous for its violent nature.

Quick start

pip install torch transformers safetensors huggingface_hub pillow

import sys
from huggingface_hub import snapshot_download

path = snapshot_download("viknesh100/SpriteX32")
sys.path.insert(0, path)
from pipeline import SpriteFlowPipeline, make_grid

pipe = SpriteFlowPipeline.from_pretrained(path)          # uses GPU if available
images = pipe("a green slime monster", num_images=8, seed=0, output_size=128)
make_grid(images, cols=4).save("sprites.png")

Command line (from a clone of this repo):

python inference.py --prompt "a blue wizard" --num_images 8 --seed 0 --out wizard.png

Parameters

Argument	Default	Notes
num_images	4	Images per prompt. Multiple prompts can be passed as a list.
num_steps	50	Euler steps. 20–100 is a reasonable range.
guidance_scale	3.0	Higher follows the prompt more strongly but reduces variety. Try 1.5–6.
seed	None	Set for reproducible output.
output_size	None	Native size is 32. Set e.g. 128 for a nearest-neighbour upscale.

Prompting tip: the model only saw captions in the style of its training data, so prompts that resemble short, descriptive sprite captions work best.

Model details

Output	32×32 RGB image (transparency is not generated)
Architecture	U-Net with residual blocks, self-attention at the lowest resolution, time and text conditioning
Text encoder	openai/clip-vit-base-patch32 (frozen, pooled embedding, loaded separately at inference)
Objective	Flow matching: linear path from noise to image, velocity prediction
Sampler	Euler, with classifier-free guidance
Weights	model.safetensors (EMA weights)

Training

• Data: tkarr/sprite_caption_dataset, 12,830 image–caption pairs.
• Preprocessing: images converted to RGBA, composited on a white background, resized to 32×32, scaled to [-1, 1].
• Optimisation: AdamW, learning rate 2e-4 with cosine decay, batch size 128, 15,000 steps, gradient clipping at 1.0.
• Guidance: the text condition was dropped for 15% of samples during training.
• EMA: decay 0.999, and the EMA weights are the ones released.
• Hardware: a single Google Colab GPU.

Limitations

• Resolution: fixed at 32×32. The model cannot produce larger images natively.
• Prompt understanding: the text is reduced to a single pooled CLIP vector, so colors and broad object types work much better than detailed or compositional prompts.
• Training data: a small dataset, so the model covers a limited range of subjects and styles. Caption quality was not audited.
• Background: outputs are RGB on a light background, with no alpha channel.
• Evaluation: only qualitative. No quantitative metrics (such as FID) were computed.

Intended use

Hobby, research, and educational use: prototyping sprite ideas, studying small text-to-image models, or as a base for further fine-tuning. Review the training-data and license notes below before any commercial use.

License and attribution

The weights in this repository are released under: other. Training data came from tkarr/sprite_caption_dataset; check that dataset's page for the terms that apply to its contents. The CLIP text encoder is a separate model from OpenAI, released under the MIT license and downloaded at inference time.