pixelsprite AnimGPT: animate a 16 / 24 / 32 px sprite

Built with Qwen. The training data (the dataset pixelsprite-animated-characters) contains outputs of Qwen-Image 2.1, which is under the Qwen Research License (non-commercial). The weights are therefore for non-commercial use only. See Licence and credits. This is not legal advice and carries no guarantee.

A small autoregressive transformer (39.2M parameters) that takes one pixel-art character sprite and draws it turned to another direction and performing an action (walk, run, punch, fly, ...), frame by frame. It is the text-to-sprite model pixelsprite-canvas-v2 taught to read a sprite instead of a caption.

samples

Columns: 1 = input sprite; 2-4 = the sprite turned to 90 / 180 / 270 degrees; 5-8 = walk (or fly) frames 0-3 at 90 degrees. Rows come in pairs: ground truth (from the 3D farm), then the model. Held-out characters, 32 px, temperature 0.8, last training step.

What it does

Input a character sprite (16 or 32 px; 24 px uses a front render), its kind (human / quadruped / bird), an action, a direction
Output, "view" the same character turned to one of 8 directions (45 degree steps)
Output, "animation" frame k of an action at a direction, given frame k-1 (frame 0 follows the view)
Actions 17 for humans, 3 for quadrupeds (idle, walk, run), 2 for birds (fly, glide)
Sizes 16, 24 and 32 px, all trained

Directions: 0 = front, 90 = profile facing left, 180 = back, 270 = profile facing right.

How it works

  • Same transformer as v2: decoder-only, 12 layers, width 512, 8 heads, MLP 2048, pre-norm. It was warm-started from the v2 weights; only the condition embeddings are new (39,178,241 parameters in total).
  • Per-pixel drawing: the target frame is written left to right, top to bottom. Each pixel is one of 1,024 palette colours or "transparent" (the same palette as v2, code/palette.npy).
  • Sequence: [v2 prefix: null text, size, S unknown-mask rows] [condition token] [reference S*S] [previous frame S*S] [target S*S] (one network row per frame; S is the canvas size).
  • Condition token: the sum of embeddings for kind, action, azimuth, elevation, frame index, frame count and reference type (0 = a sprite drawn the way v2 draws, 1 = a pixelated front render of the 3D model).
  • Two tasks: view (previous = reference; the character turned to an azimuth) and animation (previous = the last frame, or the view at that azimuth for frame 0).
  • Palette restriction: when sampling, only the reference sprite's own colours and "transparent" are allowed. This keeps the palette of the character.
  • Sampling: temperature 0.8 (the training sheets and the app use it).

Quick start

pip install torch safetensors pillow numpy huggingface_hub
hf download ihor-umanets/pixelsprite-animgpt --local-dir animgpt     # the repo is private for now: hf auth login first
cd animgpt
python example.py examples/archer32.png --az 270 --out walk.gif      # CPU, ~3 min at 32 px; --size 16 for a quick look

example.py (in the repo) is the whole recipe: it quantises a sprite to the palette, draws the view at the chosen direction, then 8 walk frames, each from the previous one, and saves a GIF and a PNG strip.

example output

The core of it:

import sys, json, numpy as np, torch
from safetensors.torch import load_file
sys.path.insert(0, "code")
from anim_model import AnimGPT, ACTIONS, KINDS, AZIMUTHS
from pixelsprite_model import CanvasConfig

cfg = json.load(open("config.json"))
model = AnimGPT(CanvasConfig.from_dict(cfg["model"])).eval()
model.load_state_dict(load_file("model.safetensors"))

# ref: (1, 32*32) long tensor of palette indices of your sprite, 1024 = transparent
t = lambda v: torch.tensor([v])
cond = dict(kind=t(KINDS.index("human")), action=t(ACTIONS.index("view")), az=t(AZIMUTHS.index(270)),
            el=t(0), frame=t(0), nfr=t(1), refk=t(0))
allowed = torch.zeros(1, 1025, dtype=torch.bool); allowed[0, ref[0].unique()] = True; allowed[0, 1024] = True
view = model.sample(32, cond, ref, ref, temperature=0.8, allowed=allowed)          # the turned sprite
cond.update(action=t(ACTIONS.index("walk")), nfr=t(8))
frame0 = model.sample(32, cond, ref, view, temperature=0.8, allowed=allowed)       # walk frame 0

A desktop app (Rust + ONNX Runtime) that generates sprites with v2 and animates them with this model is coming; it is not published yet.

Files

File What
model.safetensors, config.json weights (float32, from the best checkpoint) and the architecture
best.pt the same checkpoint as a PyTorch file (model, meta); no optimizer state
code/ anim_model.py, pixelsprite_model.py, palette.npy (1,024 x 3 sRGB)
example.py, examples/ a tested example, a 32 px input sprite and its output
onnx/ the export used by the app: prefill.onnx (whole context in one call, growing cache), step.onnx (one pixel per call, fixed cache, GPU), step_grow.onnx (growing cache, CPU), anim.data (shared float32 weights), embeddings.bin / .json (input tables, actions, palette, layout). ONNX vs PyTorch: max logit difference about 2e-5, argmax identical on all pixels.
train_log.json, samples/ training log, eval curves, sample sheets
LICENSE_QWEN.txt, NOTICE.txt Qwen Research License and the notice for the data

Training

  • Data: pixelsprite-animated-characters: 20,744 characters (humans, quadrupeds, birds) turned into 3D, rigged, animated and pixelated back, 24.3M sprites in 8 directions at 16, 24 and 32 px. Each training example is one frame or one view of one character. Splits are by character; 463 validation characters are used for the numbers below.
  • Reference: the original v2-style sprite 60% of the time when one exists at that size, otherwise a pixelated front render. Views are weighted x8 and animals x2.
  • Optimisation: AdamW (betas 0.9 / 0.95, weight decay 0.05), learning rate 3e-4, 1,000 warm-up steps, then cosine decay to 10% over the time budget, gradient clipping 1.0, bf16 autocast, dropout 0.05. About 98k tokens per batch, one canvas size per step (16 / 24 / 32 with probabilities 0.25 / 0.25 / 0.5). Loss on the target pixels only.
  • Compute: 600 minutes on one NVIDIA RTX PRO 6000, 143,992 steps. The released weights are the best evaluation (step 136,655, minute 570, score 0.3838; the last step scores 0.3843).
  • Selection score: 0.8 x mean animation bits per pixel + 0.2 x mean view bits per pixel on the validation characters.

eval curve

The spike at minute 60 is a loss spike early in training that recovered by minute 90; both curves flatten after about 400 minutes.

Results

Teacher-forced bits on the validation characters (best checkpoint; lower is better; 463 characters):

16 px 24 px 32 px
Animation frame, bits per pixel 0.328 0.324 0.306
Animation frame, bits per frame 84.0 186.6 313.8
View (turn to a direction), bits per pixel 0.719 0.606 0.598
View, bits per frame 183.9 349.1 612.2

Training bits per pixel at the end are 0.31 / 0.32 / 0.32 (16 / 24 / 32 px), so there is little overfitting. A view is about twice as hard to predict as the next frame, because a view has to invent hidden sides while a frame mostly copies the previous one.

Quality, honestly

  • Animation of a given view is good. Walk and fly cycles follow the 3D farm closely (sample sheet, columns 5-8).
  • Turning to a new direction is weaker, especially for animals and for sprites the model never saw in the dataset (for example sprites drawn by the app itself). Silhouettes and colours drift: a darker palette, a wrong back view, an odd profile.
  • Errors propagate. Every action starts from its drawn view, so a bad view carries into all frames of that action. Draw again with another seed when it happens.
  • spin_kick is rare in the dataset after QC (14% of humans keep it) and is the messiest action. 24 px uses the front-render reference type.
  • No human study: we measured bits per pixel and looked at samples.

Speed

Measured on a dev box (RTX 5070 Ti shared with other work, i9-9900K), ONNX Runtime. A walk is the views plus 8 frames per direction, batched:

16 px 32 px
CUDA graphs: one frame 0.4-0.6 s 3.0-5.3 s
CUDA graphs: walk, 1 direction 3.6-4.0 s 27.5 s
CUDA graphs: walk, 8 directions 5.8 s 48.2 s
CPU, 8 threads: walk, 1 direction 19.5 s 125 s

The PyTorch example.py on a CPU takes about 30 s (16 px) or 3 minutes (32 px) for one direction of 8 frames.

Limitations and intended use

  • For: prototypes, game jams and research on autoregressive pixel models, non-commercially. Characters only: people, four-legged animals and birds in the dataset's styles.
  • Not for: props, buildings, other body shapes (snakes, fish, many legs) or sprites far from the data (isometric, top-down); there are no such actions.
  • The reference should look like the dataset: one character, facing the camera, 16 or 32 px. Odd poses, tiny details and large accessories (a bow, a staff) are often lost when turning. A 32 px sprite drawn at 16 px is a rough nearest-neighbour shrink and often comes out poor; give a native 16 px sprite instead.
  • Sampling is random: results depend on the seed and on the device.
  • Style and bias come from the source sprites (two AI-generated pixel-art sets), Qwen-Image restyling and the 3D pipeline, with their errors (some 3D models come out rotated, more often for animals).

Licence and credits

  • Weights: CC BY-NC 4.0 (non-commercial; attribution). The training data contains outputs of Qwen-Image 2.1 (Qwen Research License, non-commercial). Commercial use of the weights or of their output needs your own legal review and, where Qwen's licence applies, a separate licence from Qwen. This card gives no legal guarantee.
  • Built with Qwen. Qwen is licensed under the Qwen RESEARCH LICENSE AGREEMENT, Copyright (c) 2026 Hangzhou Tongyi Laboratory Technology Co., Ltd. All Rights Reserved. The text is in LICENSE_QWEN.txt; see NOTICE.txt for third-party material.
  • Base model: pixelsprite-canvas-v2 (CC BY-NC 4.0), trained on pixel-art-atlas-1.4m and pixelart-172k; the source sprites of the characters come from the same two sets.
  • Code (GitHub, later): Apache-2.0.

Links

Citation

@misc{umanets2026animgpt,
  author       = {Umanets, Ihor},
  title        = {pixelsprite AnimGPT: turning and animating pixel-art character sprites with an autoregressive transformer},
  year         = {2026},
  howpublished = {\url{https://huggingface.co/ihor-umanets/pixelsprite-animgpt}},
  note         = {Built with Qwen. Non-commercial (CC BY-NC 4.0).}
}
Downloads last month
-
Safetensors
Model size
39.2M params
Tensor type
F32
ยท
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for ihor-umanets/pixelsprite-animgpt

Quantized
(1)
this model