Pickmatic5000: specialist general pattern tools

By dontknowmyname (NotANewNick on GitHub). Code: github.com/NotANewNick/Pickmatic5000.

An image is drawn in a few passes. Each pattern tool is its own specialist network that fills one pattern of positions (ring, lines, circles, dots, fractal, or "holes" = everything still blank) in one pass. A picker chooses, for every request and every pass, which tool to use and with which settings. It learns from a score of the finished image. Tested on 28x28 handwritten digits (MNIST), drawn as 2x2-pixel tiles.

The models

All files are under reference/digits-28x28/, at the same paths the code expects. Each is NAME.safetensors (weights only, no pickled code) plus, where needed, NAME.json (the settings to rebuild it).

file	what it is	parameters
specialists/S2_{ring,lines,circles,dots,fractal,holes}/specialist	the six pattern tools (transformers, width 384, 8 layers)	12.3 M each
results/tool_picker/pickers/step/picker	step picker: sees the image after every pass	0.98 M
results/tool_picker/pickers/plan/picker	plan picker: sees only the request	1.33 M
baselines/M_maskgit/maskgit	MaskGIT baseline, same network size (also used for the Halton order)	12.3 M
baselines/D_diffusion/diffusion	diffusion baseline (DDPM training, DDIM sampling)	13.6 M
data/classifier_mnist_28	the judge: classifier behind "correct digit" and FID	0.42 M
critics/classifier2	second classifier, part of the picker's score (98.86% on the test digits)	0.08 M

Total: 392 MB.

Results

28x28 digits, 1,000 validation digits, RTX 2080 Ti. FID: lower means more realistic. Baseline: MaskGIT, the same network size trained on the same digits.

passes	MaskGIT (baseline)	diffusion	picker
~2	10.7	20.9	18.1
~4	7.5	11.8	10.8
~8	7.0	8.2	5.4

MaskGIT with the published Halton order (same weights, no new training) is still ahead: 3.6 at 8 steps. Details, ranges and caveats are in the GitHub repo (docs/SPECIALIST_TOOLS.md).

Hardware

Everything was trained and measured on one NVIDIA RTX 2080 Ti (PyTorch 2.8.0). Drawing also runs on AMD GPUs (ROCm) and on the CPU (slow). On other hardware the drawings differ a little, so the FIDs do too.

How to use

git clone https://github.com/NotANewNick/Pickmatic5000.git
cd Pickmatic5000
./run.sh models      # sets up .venv and downloads these weights into reference/
./run.sh             # draws 1,000 digits with both pickers and the baselines, and scores them
./run.sh --help

While this repo is private, log in first with your own token (from https://huggingface.co/settings/tokens): .venv/bin/hf auth login, then run ./run.sh models again.

Credit and licence

The ideas and designs are NotANewNick's.

Copyright NotANewNick. The models are licensed under CC BY-NC 4.0: you may share and adapt them for non-commercial use if you credit NotANewNick. Full text: LICENSE.

Built on, or compared with (full list in the GitHub repo's REFERENCES.md):

• MaskGIT: Chang et al., CVPR 2022, arXiv 2202.04200.
• Halton Scheduler for Masked Generative Image Transformer: Besnier et al., 2025, arXiv 2503.17076.
• DDPM: Ho, Jain and Abbeel, NeurIPS 2020, arXiv 2006.11239.
• DDIM: Song, Meng and Ermon, ICLR 2021, arXiv 2010.02502.
• MNIST: Yann LeCun, Corinna Cortes and Christopher J. C. Burges. The models were trained on MNIST; the data itself is not included here and keeps its own terms.