| --- |
| license: other |
| library_name: pytorch |
| pipeline_tag: text-to-image |
| language: |
| - en |
| tags: |
| - text-to-image |
| - pixel-art |
| - sprites |
| - flow-matching |
| - classifier-free-guidance |
| - safetensors |
| datasets: |
| - tkarr/sprite_caption_dataset |
| --- |
| |
| # SpriteX32 |
|
|
| A small text-to-image model that generates **32×32 pixel-art sprites** from English prompts. |
| It is a conditional U-Net trained with flow matching, conditioned on CLIP text embeddings, |
| and sampled with classifier-free guidance. It has 11.4M parameters and runs |
| comfortably on a CPU or a small GPU. |
|
|
|  |
|
|
| *Seed 0, 50 steps, guidance 3.0, upscaled 4× with nearest-neighbour. Prompts, one per row, top to bottom:* |
|
|
| 1. a red sword |
| 2. a green slime monster |
| 3. a blue wizard |
| 4. Once it takes aim at its foe, it makes a headlong charge. It is famous for its violent nature. |
|
|
| ## Quick start |
|
|
| ```bash |
| pip install torch transformers safetensors huggingface_hub pillow |
| ``` |
|
|
| ```python |
| import sys |
| from huggingface_hub import snapshot_download |
| |
| path = snapshot_download("viknesh100/SpriteX32") |
| sys.path.insert(0, path) |
| from pipeline import SpriteFlowPipeline, make_grid |
| |
| pipe = SpriteFlowPipeline.from_pretrained(path) # uses GPU if available |
| images = pipe("a green slime monster", num_images=8, seed=0, output_size=128) |
| make_grid(images, cols=4).save("sprites.png") |
| ``` |
|
|
| Command line (from a clone of this repo): |
|
|
| ```bash |
| python inference.py --prompt "a blue wizard" --num_images 8 --seed 0 --out wizard.png |
| ``` |
|
|
| ### Parameters |
|
|
| | Argument | Default | Notes | |
| |---|---|---| |
| | `num_images` | 4 | Images per prompt. Multiple prompts can be passed as a list. | |
| | `num_steps` | 50 | Euler steps. 20–100 is a reasonable range. | |
| | `guidance_scale` | 3.0 | Higher follows the prompt more strongly but reduces variety. Try 1.5–6. | |
| | `seed` | None | Set for reproducible output. | |
| | `output_size` | None | Native size is 32. Set e.g. 128 for a nearest-neighbour upscale. | |
|
|
| **Prompting tip:** the model only saw captions in the style of its training data, so prompts |
| that resemble short, descriptive sprite captions work best. |
|
|
| ## Model details |
|
|
| | | | |
| |---|---| |
| | Output | 32×32 RGB image (transparency is not generated) | |
| | Architecture | U-Net with residual blocks, self-attention at the lowest resolution, time and text conditioning | |
| | Text encoder | `openai/clip-vit-base-patch32` (frozen, pooled embedding, loaded separately at inference) | |
| | Objective | Flow matching: linear path from noise to image, velocity prediction | |
| | Sampler | Euler, with classifier-free guidance | |
| | Weights | `model.safetensors` (EMA weights) | |
|
|
| ## Training |
|
|
| - **Data:** [`tkarr/sprite_caption_dataset`](https://huggingface.co/datasets/tkarr/sprite_caption_dataset), 12,830 image–caption pairs. |
| - **Preprocessing:** images converted to RGBA, composited on a white background, resized to 32×32, scaled to [-1, 1]. |
| - **Optimisation:** AdamW, learning rate 2e-4 with cosine decay, batch size 128, 15,000 steps, gradient clipping at 1.0. |
| - **Guidance:** the text condition was dropped for 15% of samples during training. |
| - **EMA:** decay 0.999, and the EMA weights are the ones released. |
| - **Hardware:** a single Google Colab GPU. |
|
|
| ## Limitations |
|
|
| - **Resolution:** fixed at 32×32. The model cannot produce larger images natively. |
| - **Prompt understanding:** the text is reduced to a single pooled CLIP vector, so colors and broad object types work much better than detailed or compositional prompts. |
| - **Training data:** a small dataset, so the model covers a limited range of subjects and styles. Caption quality was not audited. |
| - **Background:** outputs are RGB on a light background, with no alpha channel. |
| - **Evaluation:** only qualitative. No quantitative metrics (such as FID) were computed. |
|
|
| ## Intended use |
|
|
| Hobby, research, and educational use: prototyping sprite ideas, studying small text-to-image |
| models, or as a base for further fine-tuning. Review the training-data and license notes |
| below before any commercial use. |
|
|
| ## License and attribution |
|
|
| The weights in this repository are released under: **other**. |
| Training data came from `tkarr/sprite_caption_dataset`; check that dataset's page for the |
| terms that apply to its contents. The CLIP text encoder is a separate model from OpenAI, |
| released under the MIT license and downloaded at inference time. |
|
|
