---
license: other
library_name: pytorch
pipeline_tag: text-to-image
language:
- en
tags:
- text-to-image
- pixel-art
- sprites
- flow-matching
- classifier-free-guidance
- safetensors
datasets:
- tkarr/sprite_caption_dataset
---

# SpriteX32

A small text-to-image model that generates **32×32 pixel-art sprites** from English prompts.
It is a conditional U-Net trained with flow matching, conditioned on CLIP text embeddings,
and sampled with classifier-free guidance. It has 11.4M parameters and runs
comfortably on a CPU or a small GPU.

![Samples](assets/samples.png)

*Seed 0, 50 steps, guidance 3.0, upscaled 4× with nearest-neighbour. Prompts, one per row, top to bottom:*

1. a red sword
2. a green slime monster
3. a blue wizard
4. Once it takes aim at its foe, it makes a headlong charge. It is famous for its violent nature.

## Quick start

```bash
pip install torch transformers safetensors huggingface_hub pillow
```

```python
import sys
from huggingface_hub import snapshot_download

path = snapshot_download("viknesh100/SpriteX32")
sys.path.insert(0, path)
from pipeline import SpriteFlowPipeline, make_grid

pipe = SpriteFlowPipeline.from_pretrained(path)          # uses GPU if available
images = pipe("a green slime monster", num_images=8, seed=0, output_size=128)
make_grid(images, cols=4).save("sprites.png")
```

Command line (from a clone of this repo):

```bash
python inference.py --prompt "a blue wizard" --num_images 8 --seed 0 --out wizard.png
```

### Parameters

| Argument | Default | Notes |
|---|---|---|
| `num_images` | 4 | Images per prompt. Multiple prompts can be passed as a list. |
| `num_steps` | 50 | Euler steps. 20–100 is a reasonable range. |
| `guidance_scale` | 3.0 | Higher follows the prompt more strongly but reduces variety. Try 1.5–6. |
| `seed` | None | Set for reproducible output. |
| `output_size` | None | Native size is 32. Set e.g. 128 for a nearest-neighbour upscale. |

**Prompting tip:** the model only saw captions in the style of its training data, so prompts
that resemble short, descriptive sprite captions work best.

## Model details

| | |
|---|---|
| Output | 32×32 RGB image (transparency is not generated) |
| Architecture | U-Net with residual blocks, self-attention at the lowest resolution, time and text conditioning |
| Text encoder | `openai/clip-vit-base-patch32` (frozen, pooled embedding, loaded separately at inference) |
| Objective | Flow matching: linear path from noise to image, velocity prediction |
| Sampler | Euler, with classifier-free guidance |
| Weights | `model.safetensors` (EMA weights) |

## Training

- **Data:** [`tkarr/sprite_caption_dataset`](https://huggingface.co/datasets/tkarr/sprite_caption_dataset), 12,830 image–caption pairs.
- **Preprocessing:** images converted to RGBA, composited on a white background, resized to 32×32, scaled to [-1, 1].
- **Optimisation:** AdamW, learning rate 2e-4 with cosine decay, batch size 128, 15,000 steps, gradient clipping at 1.0.
- **Guidance:** the text condition was dropped for 15% of samples during training.
- **EMA:** decay 0.999, and the EMA weights are the ones released.
- **Hardware:** a single Google Colab GPU.

## Limitations

- **Resolution:** fixed at 32×32. The model cannot produce larger images natively.
- **Prompt understanding:** the text is reduced to a single pooled CLIP vector, so colors and broad object types work much better than detailed or compositional prompts.
- **Training data:** a small dataset, so the model covers a limited range of subjects and styles. Caption quality was not audited.
- **Background:** outputs are RGB on a light background, with no alpha channel.
- **Evaluation:** only qualitative. No quantitative metrics (such as FID) were computed.

## Intended use

Hobby, research, and educational use: prototyping sprite ideas, studying small text-to-image
models, or as a base for further fine-tuning. Review the training-data and license notes
below before any commercial use.

## License and attribution

The weights in this repository are released under: **other**.
Training data came from `tkarr/sprite_caption_dataset`; check that dataset's page for the
terms that apply to its contents. The CLIP text encoder is a separate model from OpenAI,
released under the MIT license and downloaded at inference time.