---
library_name: pytorch
license: mit
model_name: mtg-art-cnn
pipeline_tag: image-classification
tags:
- magic-the-gathering
- cnn
- from-scratch
- multi-task
- safetensors
- gradio
---

# MTG art CNN: colour identity + card type from the artwork alone

A **small convolutional network trained from random initialisation in plain PyTorch** (no pretrained weights,
no `torchvision` models) that looks at the *art crop* of a Magic: The Gathering card and predicts two labels
with two softmax heads:

* **Colour identity** (7): White, Blue, Black, Red, Green, Colorless, Multicolor
* **Card type** (7): Creature, Planeswalker, Land, Artifact, Enchantment, Instant, Sorcery

It is the final project for the Skillspire AI & Machine Learning course (Nathan King). The
live demo is the Space [BuildITSystems/mtg-art-demo](https://huggingface.co/spaces/BuildITSystems/mtg-art-demo).

## Architecture

`TwoHeadCNN(width=32, blocks=[2, 2, 2, 2], dropout=0.2)`: a 5x5 stride-2 stem, four pre-activation
residual stages (widths 32/64/128/256, stride 2 each, 8 BasicBlocks),
global average pooling and two linear heads; **2,800,366 parameters**, 5.6 MB in bf16. Input: the art crop
resized to 160x224 (HxW), normalised with the training mean/std stored in `config.json`.
The complete definition is `model.py` in the demo Space
([BuildITSystems/mtg-art-demo](https://huggingface.co/spaces/BuildITSystems/mtg-art-demo)), byte-identical to the training code.

## Training

36,325 training artworks (one per `illustration_id`), Scryfall art crops of English paper
printings; validation grouped by card, test = whole held-out sets (unseen cards *and* unseen sets).
Loss = 1.0 x L_colour + 1.0 x L_type with `ce`
(label smoothing 0.1); AdamW lr 0.002,
weight decay 0.05 (none on BN/bias), 5% linear warmup then cosine to 1%, batch 256,
bf16 autocast, grad-norm clip 1.0, early stopping on val mean macro-F1 (patience 6);
best epoch ? of 30 planned (val colour macro-F1
nan, type macro-F1 nan).
Why these choices: colour classes are balanced but type is heavily skewed (half of all cards are creatures,
~1% planeswalkers), so macro-F1 drives model selection and the loss is the main experiment dial.

## Evaluation

Unseen test split (3,401 art crops from held-out sets), checkpoint `best`:

| head | accuracy | top-2 | macro-F1 | ECE |
|---|---:|---:|---:|---:|
| Colour identity | 0.3684 | 0.5698 | 0.3450 | 0.0419 |
| Card type | 0.5293 | 0.6745 | 0.2857 | 0.0538 |

Per-class F1 (type head): Creature 0.689 (n=1606), Planeswalker 0.217 (n=37), Land 0.597 (n=347), Artifact 0.211 (n=314), Enchantment 0.103 (n=418), Instant 0.116 (n=393), Sorcery 0.067 (n=286)

Per-class F1 (colour head): White 0.296 (n=520), Blue 0.430 (n=517), Black 0.334 (n=521), Red 0.492 (n=511), Green 0.486 (n=517), Colorless 0.194 (n=257), Multicolor 0.183 (n=558)

Single-image latency (100 images, batch 1, resize + normalise + forward + softmax on a decoded image, JPEG decode excluded, `timeit.default_timer`; instructor benchmark 0.074 s):

| device | mean | p50 | p95 | vs benchmark |
|---|---:|---:|---:|---:|
| GPU bf16 (NVIDIA GeForce RTX 3090 Ti) | 3.73 ms | 3.68 ms | 4.24 ms | 0.050x |
| CPU fp32 (16 threads) | 5.20 ms | 5.05 ms | 6.01 ms | 0.070x |

## How to use

```python
import json, torch
from PIL import Image
from safetensors.torch import load_file
from model import TwoHeadCNN            # model.py from this repo

cfg = json.load(open("config.json"))
model = TwoHeadCNN.from_config(cfg["model"]).eval()
model.load_state_dict(load_file("model.safetensors"))
H, W = cfg["image_size"]
img = Image.open("art_crop.jpg").convert("RGB").resize((W, H), Image.BILINEAR)  # bilinear, as in training/eval
x = torch.tensor(list(img.getdata()), dtype=torch.float32).view(H, W, 3).permute(2, 0, 1) / 255
x = (x - torch.tensor(cfg["normalization"]["mean"]).view(3, 1, 1)) / torch.tensor(cfg["normalization"]["std"]).view(3, 1, 1)
with torch.no_grad():
    color_logits, type_logits = model(x[None])
print(cfg["labels"]["color"][color_logits.argmax().item()], cfg["labels"]["type"][type_logits.argmax().item()])
```

## Limitations

* Trained on official card art only; photos of physical cards, sleeves, glare and perspective are out of
  distribution (the Space crops a photo with a fixed template, which is approximate).
* The type head reads *style* cues (a creature portrait vs a landscape), not the type line; lands and
  sorceries with similar art are confused, planeswalkers are rare.
* Colour is the colour identity of the card the art belongs to, which the art does not always show.

## Legal

This project is unofficial Fan Content permitted under the Wizards of the Coast Fan Content Policy. Not approved/endorsed by Wizards. Portions of the materials used are property of Wizards of the Coast. (c) Wizards of the Coast LLC.

Card metadata and art crops were obtained from [Scryfall](https://scryfall.com) under its API terms; Scryfall is not affiliated with this project. The training image set is not redistributed; the Space ships six test-split art crops as examples under the Fan Content Policy, and only the trained weights are published here.