license: other
library_name: pytorch
pipeline_tag: text-to-image
language:
- en
tags:
- text-to-image
- pixel-art
- sprites
- flow-matching
- classifier-free-guidance
- safetensors
datasets:
- tkarr/sprite_caption_dataset
SpriteX32
A small text-to-image model that generates 32×32 pixel-art sprites from English prompts. It is a conditional U-Net trained with flow matching, conditioned on CLIP text embeddings, and sampled with classifier-free guidance. It has 11.4M parameters and runs comfortably on a CPU or a small GPU.
Seed 0, 50 steps, guidance 3.0, upscaled 4× with nearest-neighbour. Prompts, one per row, top to bottom:
- a red sword
- a green slime monster
- a blue wizard
- Once it takes aim at its foe, it makes a headlong charge. It is famous for its violent nature.
Quick start
pip install torch transformers safetensors huggingface_hub pillow
import sys
from huggingface_hub import snapshot_download
path = snapshot_download("viknesh100/SpriteX32")
sys.path.insert(0, path)
from pipeline import SpriteFlowPipeline, make_grid
pipe = SpriteFlowPipeline.from_pretrained(path) # uses GPU if available
images = pipe("a green slime monster", num_images=8, seed=0, output_size=128)
make_grid(images, cols=4).save("sprites.png")
Command line (from a clone of this repo):
python inference.py --prompt "a blue wizard" --num_images 8 --seed 0 --out wizard.png
Parameters
| Argument | Default | Notes |
|---|---|---|
num_images |
4 | Images per prompt. Multiple prompts can be passed as a list. |
num_steps |
50 | Euler steps. 20–100 is a reasonable range. |
guidance_scale |
3.0 | Higher follows the prompt more strongly but reduces variety. Try 1.5–6. |
seed |
None | Set for reproducible output. |
output_size |
None | Native size is 32. Set e.g. 128 for a nearest-neighbour upscale. |
Prompting tip: the model only saw captions in the style of its training data, so prompts that resemble short, descriptive sprite captions work best.
Model details
| Output | 32×32 RGB image (transparency is not generated) |
| Architecture | U-Net with residual blocks, self-attention at the lowest resolution, time and text conditioning |
| Text encoder | openai/clip-vit-base-patch32 (frozen, pooled embedding, loaded separately at inference) |
| Objective | Flow matching: linear path from noise to image, velocity prediction |
| Sampler | Euler, with classifier-free guidance |
| Weights | model.safetensors (EMA weights) |
Training
- Data:
tkarr/sprite_caption_dataset, 12,830 image–caption pairs. - Preprocessing: images converted to RGBA, composited on a white background, resized to 32×32, scaled to [-1, 1].
- Optimisation: AdamW, learning rate 2e-4 with cosine decay, batch size 128, 15,000 steps, gradient clipping at 1.0.
- Guidance: the text condition was dropped for 15% of samples during training.
- EMA: decay 0.999, and the EMA weights are the ones released.
- Hardware: a single Google Colab GPU.
Limitations
- Resolution: fixed at 32×32. The model cannot produce larger images natively.
- Prompt understanding: the text is reduced to a single pooled CLIP vector, so colors and broad object types work much better than detailed or compositional prompts.
- Training data: a small dataset, so the model covers a limited range of subjects and styles. Caption quality was not audited.
- Background: outputs are RGB on a light background, with no alpha channel.
- Evaluation: only qualitative. No quantitative metrics (such as FID) were computed.
Intended use
Hobby, research, and educational use: prototyping sprite ideas, studying small text-to-image models, or as a base for further fine-tuning. Review the training-data and license notes below before any commercial use.
License and attribution
The weights in this repository are released under: other.
Training data came from tkarr/sprite_caption_dataset; check that dataset's page for the
terms that apply to its contents. The CLIP text encoder is a separate model from OpenAI,
released under the MIT license and downloaded at inference time.