| <p>Structured after <em>Datasheets for Datasets</em> (Gebru et al.). Numbers in the <strong>Composition</strong> section are generated from the
|
| build artefacts by <code>python scripts/update_docs.py</code>; they describe the build recorded there and nothing else.</p>
|
| <h2>Motivation</h2>
|
| <p>The collection serves two purposes: (1) a <strong>reference set</strong> for retrieval — every record is a documented museum object
|
| with provenance; (2) <strong>training and evaluation data</strong> for visual estimation of region and period. It was assembled for a
|
| personal research prototype in computational art history; no funding body or institution was involved.</p>
|
| <h2>Composition</h2>
|
| <p>Each instance is one museum object record with one image: a painting or painted image (including illustrated manuscript
|
| folios and album leaves). Fields are listed in <code>src/artifex/schemas.py</code> (<code>MANIFEST_COLUMNS</code>); original museum strings are
|
| kept next to normalised values.</p>
|
|
|
| <p><em>Generated from <code>data/manifests/dataset_stats.json</code> — dataset <code>0.1+5cc3c8d20c19</code>, built 2026-10-22.</em></p>
|
| <ul>
|
| <li>Records in manifest: <strong>10,076</strong>; usable (images present, not duplicates): <strong>10,063</strong></li>
|
| <li>Exact duplicates marked: 13; records excluded before the manifest: 6,616</li>
|
| <li>Distinct named artists: 2,470; works with anonymous or unrecorded artist: 3,662</li>
|
| <li>Dates recorded as approximate: 4,426; as uncertain: 379; median documented date span 20 years (90th percentile 99)</li>
|
| </ul>
|
| <p><strong>By source</strong></p>
|
| <table>
|
| <thead>
|
| <tr>
|
| <th>Source</th>
|
| <th>Records</th>
|
| </tr>
|
| </thead>
|
| <tbody>
|
| <tr>
|
| <td>met</td>
|
| <td>7,124</td>
|
| </tr>
|
| <tr>
|
| <td>cleveland</td>
|
| <td>2,939</td>
|
| </tr>
|
| </tbody>
|
| </table>
|
| <p><strong>By broad region</strong> (as labelled; <code>(none)</code> = ambiguous, unmapped or missing)</p>
|
| <table>
|
| <thead>
|
| <tr>
|
| <th>Region</th>
|
| <th>Records</th>
|
| </tr>
|
| </thead>
|
| <tbody>
|
| <tr>
|
| <td>East Asia</td>
|
| <td>3,726</td>
|
| </tr>
|
| <tr>
|
| <td>Europe</td>
|
| <td>3,274</td>
|
| </tr>
|
| <tr>
|
| <td>South Asia</td>
|
| <td>1,756</td>
|
| </tr>
|
| <tr>
|
| <td>West & Central Asia</td>
|
| <td>639</td>
|
| </tr>
|
| <tr>
|
| <td>North America</td>
|
| <td>196</td>
|
| </tr>
|
| <tr>
|
| <td>(none)</td>
|
| <td>188</td>
|
| </tr>
|
| <tr>
|
| <td>Himalayan region (Tibet, Nepal, Bhutan)</td>
|
| <td>184</td>
|
| </tr>
|
| <tr>
|
| <td>Nile Valley & North Africa</td>
|
| <td>82</td>
|
| </tr>
|
| <tr>
|
| <td>Southeast Asia</td>
|
| <td>12</td>
|
| </tr>
|
| <tr>
|
| <td>Sub-Saharan Africa</td>
|
| <td>4</td>
|
| </tr>
|
| <tr>
|
| <td>Latin America & Caribbean</td>
|
| <td>1</td>
|
| </tr>
|
| <tr>
|
| <td>Oceania</td>
|
| <td>1</td>
|
| </tr>
|
| </tbody>
|
| </table>
|
| <p><strong>By period bin</strong> (<code>(none)</code> = date unknown or spanning several bins)</p>
|
| <table>
|
| <thead>
|
| <tr>
|
| <th>Period</th>
|
| <th>Records</th>
|
| </tr>
|
| </thead>
|
| <tbody>
|
| <tr>
|
| <td>Before 500 CE</td>
|
| <td>84</td>
|
| </tr>
|
| <tr>
|
| <td>500–1299</td>
|
| <td>553</td>
|
| </tr>
|
| <tr>
|
| <td>1300–1449</td>
|
| <td>646</td>
|
| </tr>
|
| <tr>
|
| <td>1450–1549</td>
|
| <td>997</td>
|
| </tr>
|
| <tr>
|
| <td>1550–1649</td>
|
| <td>1,329</td>
|
| </tr>
|
| <tr>
|
| <td>1650–1749</td>
|
| <td>1,149</td>
|
| </tr>
|
| <tr>
|
| <td>1750–1849</td>
|
| <td>1,815</td>
|
| </tr>
|
| <tr>
|
| <td>1850–1899</td>
|
| <td>1,195</td>
|
| </tr>
|
| <tr>
|
| <td>1900 and later</td>
|
| <td>166</td>
|
| </tr>
|
| <tr>
|
| <td>(none)</td>
|
| <td>2,129</td>
|
| </tr>
|
| </tbody>
|
| </table>
|
| <p><strong>Region label status / basis</strong></p>
|
| <table>
|
| <thead>
|
| <tr>
|
| <th>Status</th>
|
| <th>Records</th>
|
| </tr>
|
| </thead>
|
| <tbody>
|
| <tr>
|
| <td>assigned</td>
|
| <td>9,875</td>
|
| </tr>
|
| <tr>
|
| <td>ambiguous</td>
|
| <td>140</td>
|
| </tr>
|
| <tr>
|
| <td>unmapped</td>
|
| <td>25</td>
|
| </tr>
|
| <tr>
|
| <td>missing</td>
|
| <td>23</td>
|
| </tr>
|
| </tbody>
|
| </table>
|
| <table>
|
| <thead>
|
| <tr>
|
| <th>Region derived from</th>
|
| <th>Records</th>
|
| </tr>
|
| </thead>
|
| <tbody>
|
| <tr>
|
| <td>object_culture</td>
|
| <td>6,405</td>
|
| </tr>
|
| <tr>
|
| <td>artist_nationality</td>
|
| <td>2,480</td>
|
| </tr>
|
| <tr>
|
| <td>object_geography</td>
|
| <td>1,129</td>
|
| </tr>
|
| <tr>
|
| <td>(none)</td>
|
| <td>48</td>
|
| </tr>
|
| <tr>
|
| <td>department_hint</td>
|
| <td>1</td>
|
| </tr>
|
| </tbody>
|
| </table>
|
| <p><strong>Missing fields (usable records)</strong></p>
|
| <table>
|
| <thead>
|
| <tr>
|
| <th>Field</th>
|
| <th>Missing</th>
|
| </tr>
|
| </thead>
|
| <tbody>
|
| <tr>
|
| <td>title</td>
|
| <td>0</td>
|
| </tr>
|
| <tr>
|
| <td>artist_name</td>
|
| <td>3,045</td>
|
| </tr>
|
| <tr>
|
| <td>date_start</td>
|
| <td>2</td>
|
| </tr>
|
| <tr>
|
| <td>original_date_text</td>
|
| <td>996</td>
|
| </tr>
|
| <tr>
|
| <td>region_normalized</td>
|
| <td>188</td>
|
| </tr>
|
| <tr>
|
| <td>culture_normalized</td>
|
| <td>1,241</td>
|
| </tr>
|
| <tr>
|
| <td>period_label</td>
|
| <td>2,129</td>
|
| </tr>
|
| <tr>
|
| <td>medium</td>
|
| <td>3</td>
|
| </tr>
|
| <tr>
|
| <td>culture_original</td>
|
| <td>3,654</td>
|
| </tr>
|
| </tbody>
|
| </table>
|
| <p><strong>Exclusions</strong></p>
|
| <table>
|
| <thead>
|
| <tr>
|
| <th>Reason</th>
|
| <th>Records</th>
|
| </tr>
|
| </thead>
|
| <tbody>
|
| <tr>
|
| <td>not-public-domain</td>
|
| <td>4,611</td>
|
| </tr>
|
| <tr>
|
| <td>not-painted-imagery</td>
|
| <td>972</td>
|
| </tr>
|
| <tr>
|
| <td>text-only-or-non-pictorial-title</td>
|
| <td>941</td>
|
| </tr>
|
| <tr>
|
| <td>excluded-object-type</td>
|
| <td>49</td>
|
| </tr>
|
| <tr>
|
| <td>image-too-small</td>
|
| <td>41</td>
|
| </tr>
|
| <tr>
|
| <td>http</td>
|
| <td>2</td>
|
| </tr>
|
| </tbody>
|
| </table>
|
| <p><strong>Grouping and splits</strong> — 4,094 leakage groups (largest: 262 works; 966 near-duplicate pairs). Modelling subset after the per-group cap: 8,886 works; main split train 6,204, test 1,352, val 1,330.</p>
|
|
|
|
|
| <p><strong>Labels.</strong> Region and period labels are derived automatically from museum metadata (see <code>label_taxonomy.md</code>); there are
|
| no manual labels. Unlabelled and ambiguous records stay in the collection and the retrieval index but are not classifier
|
| targets.</p>
|
| <p><strong>Relationships between instances.</strong> Many records belong to the same manuscript, album or set, or are by the same artist.
|
| These relationships are made explicit as <em>leakage groups</em> (<code>data/splits/splits.parquet</code>, column <code>group_id</code>).</p>
|
| <p><strong>Errors and noise.</strong> Museum attributions can be outdated or contested; dates are often ranges; for works without a
|
| recorded culture the region derives from the artist's nationality; title-based exclusion of text pages is heuristic and
|
| may miss pages or exclude a few illustrated ones; near-duplicate detection uses an embedding threshold.</p>
|
| <p><strong>Confidentiality.</strong> None: all records are public, CC0 museum data. Records depicting people are historical artworks.</p>
|
| <h2>Collection process</h2>
|
| <p>Metadata were retrieved from the official open-access APIs of The Metropolitan Museum of Art and The Cleveland Museum of
|
| Art (see <code>data_sources.md</code>) using fixed queries, at rates below the providers' limits, with raw responses cached verbatim.
|
| Only objects released under CC0 with an image were kept. Images are the museums' web-resolution derivatives, resized to at
|
| most 768 px on the longer side, with 384 px thumbnails. Ingestion timestamps are recorded per record.</p>
|
| <h2>Preprocessing</h2>
|
| <p>Normalisation of dates (astronomical years, ranges), region mapping (ordered rules, ambiguity preserved), period
|
| assignment (≥ 75 % of range in one bin), exclusion of non-pictorial records, exact-duplicate marking, image validation.
|
| The raw API responses are retained under <code>data/raw/</code> so every step can be re-run and audited.</p>
|
| <h2>Known biases</h2>
|
| <ul>
|
| <li><strong>Institutional and geographic bias.</strong> Two North American encyclopaedic museums. European, East Asian and South Asian
|
| court painting dominate; Africa, Oceania, the pre-Columbian Americas, Southeast Asia and Central Asia are scarce or absent.</li>
|
| <li><strong>Rights-driven bias.</strong> Restricting to CC0 removes most 20th-century works.</li>
|
| <li><strong>Collecting bias.</strong> Objects that entered museums through particular collectors and markets (e.g. dispersed Indian and
|
| Persian manuscripts, Japanese hanging scrolls) are over-represented relative to the traditions they come from.</li>
|
| <li><strong>Cataloguing bias.</strong> The two museums use different vocabularies (the Met often records nationality for European
|
| painters; Cleveland records culture strings like "France, 19th century"), so label provenance differs by institution.</li>
|
| <li><strong>Photographic bias.</strong> Each museum photographs, crops and colour-manages differently — a confound the
|
| collection-disjoint experiments are designed to expose.</li>
|
| </ul>
|
| <h2>Uses</h2>
|
| <p>Intended: the Artifex research prototype; teaching and exploring computational methods on museum images. Not intended:
|
| authentication, valuation, attribution of objects on the market, or provenance/restitution claims.</p>
|
| <h2>Distribution</h2>
|
| <p>The repository distributes the manifest, splits, statistics and code — not images. Images and raw metadata are CC0 and
|
| are fetched from the museums by the build scripts. Please credit the museums.</p>
|
| <h2>Maintenance</h2>
|
| <p>Rebuild with <code>scripts/ingest_data.py</code> and <code>scripts/build_dataset.py</code>. Each build has a dataset version
|
| <code><release>+<content hash></code>; any change to records, labels or configuration changes it, which invalidates trained models
|
| and evaluation runs (the app refuses to analyse with a stale model).</p> |
