PokemonFoilMask-LoRA-Qwen-Image-2.1

Drop in any Pokemon-style card, get back its foil mask. PokemonFoilMask is a LoRA for Qwen-Image-2.1 that predicts the grayscale map a card shader uses to decide where the holo shines and how strongly, plus the embossed etch texture on full arts and secret rares.

Please ignore the horrible holo effects in my demos ;P I threw together a quick holo shader to show a use-case! (You can do better than me)

The whole flow, on held-out test cards: card in, the LoRA's mask forming step by step, then the holo shader.

Use the masks to create the holographic effect a game / app, a 3D card viewer, a print mock-up or a collection site, without hand-painting a single mask.

Benchmarks

Scored on 820 held-out test cards (929 foil versions) that were never used for training or checkpoint selection, against the masks from the official game.

Model	Overall	Holo	Reverse holo	Full art	Stamped	Etch
FoilMask	41.1	55.9	24.4	49.5	34.7	35.6
mean (baseline)	51.3	42.1	62.8	50.1	49.9	42.7
blank (baseline)	37.7	35.4	33.0	17.4	65.2	20.6

• Overall is the mean of the per-category scores, so the common reverse-holo cards can't dominate.
• Per-pair score = 100 × (SSIM + soft IoU + (1 − MAE)) / 3, measured on the card area only.
• Full-art pairs also score the etch mask. When the real card has no etch, predicting one counts against the model.
• Baselines:
  • mean predicts the average mask of the card's foil type.
  • blank predicts no foil at all.
  • A LoRA that has learned nothing card-specific lands near mean.

Examples

These are the median-scoring test cards for each foil type. Each row shows the card, the LoRA's mask, the real mask, and the foil rendered with the LoRA's mask.

See it as a holo card

A mask on its own is just grayscale. Feed it to a holo shader with a foil pattern and the card comes alive. These are the same test cards, rendered with the LoRA's masks and the foil pattern each card's finish uses in the game:

The repo ships the shader in two forms:

• A WebGL version that tilts with the pointer. The app's Holo preview tab can export it as a standalone .html page to drop into a website.
• A Python renderer that produces animations like these.

Usage

The LoRA loads into the stock diffusers pipeline. Give it the card and the prompt for the card's foil type.

import torch
from PIL import Image
from diffusers import QwenImage21Pipeline

pipe = QwenImage21Pipeline.from_pretrained("Qwen/Qwen-Image-2.1", torch_dtype=torch.bfloat16)
pipe.load_lora_weights("newHashMap/PokemonFoilMask-LoRA-Qwen-Image-2.1", weight_name="pytorch_lora_weights.safetensors")
pipe.enable_model_cpu_offload()

card = Image.open("card.png").convert("RGB").resize((800, 1024))
mask = pipe(
    prompt="Draw the holo foil mask of this Pokemon card. Holo: only the artwork window shines. Inside the artwork window, paint the foil in grayscale, white where it shines. Paint everything outside the artwork window solid black: the border, name bar, attacks and text.",
    image=card,
    width=800,
    height=1024,
    output_resolution=905,
    num_inference_steps=28,
    generator=torch.Generator("cuda").manual_seed(0),
).images[0].convert("L")
mask.save("foil_mask.png")

• Input: a front-on image of the card: a scan, a render, or a straightened photo. Resize the whole card to 800×1024. The model was trained on that slightly widened card shape, so stretch the image rather than padding it.
• Output: a 800×1024 grayscale mask, aligned pixel-for-pixel with the resized card. Resize it back to your card's size.
• Memory:
  • Base model plus LoRA is about 33 GB in bf16. enable_model_cpu_offload() lets it run on a 24 GB GPU.
  • With two GPUs, put the text encoder on one and the transformer on the other. The training repo's FoilMaskModel does exactly that.
• Determinism: fix the seed, and the same card gives the same mask.

Prompts

Pick the prompt for the foil version you want. The same card art has different masks as a holo and as a reverse holo.

Foil type	What gets foil	Prompt
Holo	the art window	Draw the holo foil mask of this Pokemon card. Holo: only the artwork window shines. Inside the artwork window, paint the foil in grayscale, white where it shines. Paint everything outside the artwork window solid black: the border, name bar, attacks and text.
Reverse holo	everything except the art window	Draw the reverse holo foil mask of this Pokemon card. Reverse holo: the artwork window is solid black and never shines. Everything outside the artwork window shines: paint the border, name bar and text box in grayscale foil, white where it shines.
Full art	the whole card	Draw the full art foil mask of this Pokemon card. Full art: the whole card shines, edge to edge, artwork included. Paint the entire card in grayscale foil, white where it shines.
Stamped	the stamp only	Draw the stamped foil mask of this Pokemon card. Stamped: only the small foil stamp shines. Paint the stamp white and everything else solid black.
Etch (full art)	embossed texture	Draw the etch mask of this Pokemon card: a grayscale map of the embossed texture pressed into the card, white where it is etched and black where the card has no etch.

Training

Base model	Qwen/Qwen-Image-2.1 (7B DiT, image editing)
Data	15,945 card → mask pairs from 14,340 cards, extracted from the official game's card assets
Held out	820 benchmark cards, excluded entirely, plus 315 validation cards used for checkpoint selection
Targets	foil masks for holo, reverse holo, full art, stamped; etch masks for full arts, including all-black targets for cards without one
Resolution	800×1024
LoRA	rank 32, alpha 32, on attention + MLP projections of all 32 blocks
Optimisation	AdamW, lr 0.0001, warmup 100, cosine to 0.1×, 4,000 steps × 4 accumulation
Loss	flow-matching velocity MSE, logit-normal timesteps with resolution shift
Sampling	rarer finishes oversampled by 1/√(group size)
Hardware	2× AMD Radeon AI PRO R9700 (32 GB), PyTorch ROCm on Windows

The full pipeline is in the GitHub repo: asset extraction, benchmark splits, grader, training, and app.

Limitations

• Reverse Holos are not particularly great as you can see in the benchmarks. I will be working on a better training stack to get this down
• The training data was purely foils from the English Sets only and therefore may be missing a vast more amounts of data.

License and attribution

• Built with Qwen. This LoRA is a derivative of Qwen-Image-2.1 and is distributed under the Qwen Research License Agreement. It's for non-commercial research and evaluation only. Commercial use needs a separate licence from the Qwen team. See NOTICE.
• Not affiliated. Pokémon and all related names and card images are trademarks and © of Nintendo, Creatures Inc., GAME FREAK inc. and The Pokémon Company. This project is unofficial and isn't affiliated with or endorsed by them. Card images on this page appear only to illustrate what the model does.