README.md

---
license: apache-2.0
library_name: diffusers
pipeline_tag: image-to-image
base_model: Boogu/Boogu-Image-0.1-Edit
datasets:
- irotem98/codex-pets-sprite-sheets
tags:
- image-to-image
- image-editing
- pixel-art
- sprite-sheet
- animation
- rgba
- transparent-background
- lora
---

# Codex Pets Sprite Sheet Generator

Give the model **one transparent character image** and it creates a complete V2 sprite sheet with **11 animation rows**. The included script handles image preparation, all 11 diffusion calls, row packing, final assembly, and binary alpha cleanup.

It is built around the V2 animation layout used by [Codex Pets](https://codex-pets.net/).

Created using the
[`irotem98/codex-pets-sprite-sheets`](https://huggingface.co/datasets/irotem98/codex-pets-sprite-sheets)
training dataset.

![Jigglypuff full 11-row sprite sheet](assets/0146-jigglypuff-alpha50.png)

The example above was generated from only Jigglypuff's initial image.

## Quick start

```bash
git clone https://huggingface.co/irotem98/Codex-Pets-Sprite-Sheet-Generator
cd Codex-Pets-Sprite-Sheet-Generator
pip install -r requirements.txt

python generate_sprite_sheet.py character.png --output full-sprite-sheet.png
```

That is the complete inference command. On first use, the script automatically downloads the original [`Boogu/Boogu-Image-0.1-Edit`](https://huggingface.co/Boogu/Boogu-Image-0.1-Edit) base model. A transparent PNG containing one centered character works best.

Generation uses a row batch size of 1 by default. If your GPU has additional memory, increase it with `--row-batch-size 2`, `4`, or up to `11` to generate multiple animation rows together. The underlying model is 10B parameters, so a CUDA GPU with substantial VRAM is recommended.

## Output layout

The output is a `1536 × 2288` RGBA PNG: eight columns, eleven rows, and `192 × 208` pixels per sprite cell.

| Row | Animation | Frames |
|---:|---|---:|
| 1 | idle | 6 |
| 2 | run right | 8 |
| 3 | run left | 8 |
| 4 | wave | 4 |
| 5 | jump | 5 |
| 6 | failed | 8 |
| 7 | waiting | 6 |
| 8 | running | 6 |
| 9 | review | 6 |
| 10 | look directions 1–8 | 8 |
| 11 | look directions 9–16 | 8 |

Unused cells at the end of shorter rows remain transparent.

## How the model was trained

This release uses a two-stage fine-tuning pipeline.

### 1. Alpha-compatible RGBA VAE

The original Boogu VAE accepts and reconstructs RGB. It was expanded to RGBA while preserving the pretrained RGB path:

- the original RGB VAE weights were frozen
- a trainable one-channel alpha input adapter was added to the encoder
- a trainable one-channel alpha output head was added to the decoder
- RGB reconstruction, alpha reconstruction, foreground alpha, transparent RGB, and KL losses were weighted separately
- after training, the adapters were merged into a standard four-channel `AutoencoderKL`

The released VAE is checkpoint 2,000, trained on 3,015 sprite sheets from [`irotem98/codex-pets-sprite-sheets`](https://huggingface.co/datasets/irotem98/codex-pets-sprite-sheets), represented as 27,691 training rows and 540 validation rows. It is stored under `vae/` and can be loaded directly with Diffusers.

### 2. Boogu Image Edit sprite-sheet LoRA

The RGBA VAE and Qwen3-VL instruction encoder were frozen. A rank-8 LoRA was trained on the Boogu diffusion transformer's `to_q`, `to_k`, `to_v`, and `to_out.0` attention projections.

For every animation row:

- **input:** a balanced 3×3 grid where every occupied cell repeats the first sprite from the sheet's first row
- **target:** the real animation frames packed into the same occupied cells
- **instruction:** an explicit prompt naming the animation, frame count, ordering, identity preservation, transparent background, and occupied-cell layout

The released LoRA is checkpoint 21,000 and is stored under `lora/`. The diffusion objective was flow matching. During training, ordinary 50-step diffusion samples were logged every 50 optimizer steps.

## One-image generation pipeline

`generate_sprite_sheet.py` performs the full workflow:

1. Read the input as RGBA.
2. Remove only padding where alpha is zero.
3. Resize proportionally to fit one `192 × 208` cell without cropping visible pixels.
4. Repeat that same first sprite in the correct 3×3 occupied-cell pattern for each animation row.
5. Generate all 11 row grids with row-specific prompts and 50 diffusion steps.
6. Unpack the row grids into the final eight-column V2 sprite sheet.
7. Clean the alpha channel, keep only the main connected opaque character region in each sprite cell, set every pixel in its two-pixel outer edge band to exact black, and save the finished RGBA PNG.

## Included files

- `vae/` — deployable four-channel alpha-compatible VAE
- `lora/` — rank-8 Boogu Image Edit sprite-sheet adapter
- `generate_sprite_sheet.py` — one-image to full-sheet command
- `sprite_sheet_config.json` — dimensions, row names, frame counts, and inference defaults
- `boogu/` — the Apache-2.0 Boogu inference modules needed by the script

## Limitations

- Best results come from a single pixel-art character on a transparent background.
- RGB/JPEG input is accepted, but its entire rectangular background is treated as visible; transparent PNG is strongly recommended.
- Motion quality, facing consistency, and identity preservation can vary between rows.
- The layout is fixed to the eleven-row V2 format above.
- This repository contains the fine-tuned VAE and LoRA, but still downloads and requires the original 10B Boogu base model.

## Base model and license

This is a fine-tune of [`Boogu/Boogu-Image-0.1-Edit`](https://huggingface.co/Boogu/Boogu-Image-0.1-Edit). The base model and included Boogu inference code are Apache-2.0 licensed. Review the upstream model card for its responsible-use guidance and broader limitations.