Artifex estimates what is typical of its reference collection. The limitations below bound what its outputs can mean.
Scientific
- Correlation, not attribution. A linear classifier on image embeddings learns statistical regularities of the
training images. It has no notion of materials, technique, iconography or provenance, and it can be right for the
wrong reasons (frames, mounts, paper tone, photographic set-up, image resolution).
- Confounds from photography and cataloguing. Each museum photographs and colour-manages its collection differently.
Collection-disjoint experiments estimate how much performance depends on such institution-specific cues; a large drop
means the in-collection numbers overstate what to expect for arbitrary photographs.
- Calibration is distribution-specific. Temperature scaling and abstention thresholds are fitted on validation works
from the same two collections. For phone photos, reproductions in books, or traditions absent from the collection,
the scores are not calibrated.
- Closed label set. The model can only output supported categories. A work from an unsupported tradition will be
assigned to the nearest supported one (the interface warns about this, and the out-of-distribution flag catches only
some such cases).
- Abstention protects overall accuracy, not small classes. The threshold is chosen to reach 80 % selective accuracy on
validation works overall. For broad regions this target is met without abstaining (Europe and East Asia dominate), so
the region head never abstains on confidence, even though recall for small classes is much lower than for the large
ones (see the per-class table on the Experiments page).
- Ablations are diagnostic. Greyscale, cropping and low-level-baseline comparisons show sensitivities; they do not
explain what the model uses.
- Small classes and wide intervals. Overall test metrics of the main experiment carry 95% confidence intervals from a
grouped bootstrap; they are wide because the test split holds a few hundred leakage groups. Per-class metrics for
categories with few test works have even wider uncertainty and no intervals are computed for them.
- One split, one estimate. The website's Experiments page shows a single grouped split. Re-drawing the split
matters: successive builds in October 2026 (removing 21 non-paintings; grouping manuscripts by painter rather than
text author) moved single-split region macro-F1 between 0.689 and 0.776, and on the final build alone ten re-drawn
splits give 0.716 ± 0.024 (range 0.686–0.751), with Himalayan F1 between 0.39 and 0.86. Repeated-split results are
produced by
scripts/evaluate_multisplit.py and reported in the paper (paper/).
Data and historical
- Collection bias. Two American encyclopaedic museums, filtered to CC0 images, do not represent world painting. Holdings
over-represent European oil painting, Chinese and Japanese scroll painting and South Asian court painting; they
under-represent Africa, Oceania, the Americas before 1500, Southeast Asia and most vernacular or religious traditions.
Twentieth-century works are scarce because most are still in copyright.
- Court and élite art. Museum holdings favour élite and collectable objects; the "visual signature" of a region in
Artifex is largely the signature of what Western museums collected.
- Label provenance. Region labels come from museum culture fields or, failing that, an artist's recorded nationality.
The latter is a school attribution, not a place of production. Museum attributions themselves change with scholarship.
- Taxonomy choices. Broad regions are coarse and partly arbitrary at their edges (Himalayan vs. Indian Buddhist painting;
Mughal vs. Persianate; colonial Latin American painting). Ambiguous records are excluded from training, which may bias
classes towards clear-cut examples.
- Date uncertainty. Many dates are broad ranges ("18th century") or approximate. Works spanning several bins have no
period label; bins are coarse; exact-bin accuracy penalises boundary cases.
- Manuscripts and albums. Folios are photographed and catalogued individually; grouping by accession lot and capping
each group reduces, but may not eliminate, related-work leakage (e.g. dispersed albums held under different lots or by
both museums).
- Copies and reproductions. Later copies dated to their originals, or works "in the style of" a tradition, are labelled
as the museum records them.
Technical
- Frozen encoder. DINOv2 was trained on natural photographs; it was not adapted to paintings. Fine-tuning is future work.
- Input format. Every image is letterboxed to 224 × 224. Very wide handscrolls or tall hanging scrolls are reduced to
thin strips, losing detail; some handscroll images were too small to keep at all.
- Out-of-distribution detection is a nearest-neighbour distance heuristic, calibrated so ~1 % of in-distribution
validation images are flagged. It is not a reliable detector of non-paintings.
- Single photograph. Analysis uses one image; details, reverse sides and technical imaging are not considered.
- In-browser edition. The website runs an ONNX export of the encoder with float16-stored weights and float16
reference embeddings. On held-out test images its predictions matched the Python application's exactly, and the
similar-work rankings differed for at most 1 % of images; but image decoding is done by the visitor's browser: unusual
files (CMYK JPEGs, 16-bit PNGs) may decode slightly differently from Pillow. Neither edition applies embedded colour
profiles.
- Snapshot data. The museums' APIs change; a rebuild later may differ. The manifest, dataset version and split files
identify each run exactly.
Ethical
- Artifex must not be used to authenticate, value or attribute objects for sale.
- Predictions about cultural origin can carry political and legal weight (e.g. provenance and restitution questions).
They are not evidence in such contexts.