PyramidFormer similarity
0.8M parameters. Maps a pyramid to a 64-d unit vector (year is not an input). Trained with three terms: contrastive (the same country 1-2 years apart is a positive pair; same-country pairs are never negatives), an auxiliary linear head predicting median age, share under 15 and share 65+, and a term that makes embedding cosine follow pyramid L1 distance. embeddings.npy + index.parquet hold the embeddings of every WPP 1950-2023 country-year.
Evaluation (held-out countries)
kNN (k=5) regression of summary statistics, R2, database = training pyramids. Because the embedding was trained with these statistics as an auxiliary target, this is not an unsupervised test of what it captures:
| Statistic | From embedding | From raw pyramid |
|---|---|---|
| median_age | 0.895 | 0.926 |
| share_under_15 | 0.856 | 0.895 |
| share_65_plus | 0.901 | 0.950 |
Spearman correlation between embedding cosine and negative L1 distance on random held-out pairs: 0.495. The raw pyramid is a strong baseline for these statistics, so use the embedding for compact retrieval, not as a claim of better demographic information.
Training run
quick run: 1500 optimisation steps.
Training data
UN World Population Prospects 2024, single-age population by sex, countries/areas, 1950-2023 (estimates), as processed in
SallySims/wpp2024-single-age-pyramids. Split by country: 80% train / 10% validation / 10% test,
so every number above is on countries the model never saw. WPP pyramids are modelled estimates, not raw census counts.
Source: United Nations, DESA, Population Division (2024). World Population Prospects 2024. https://population.un.org/wpp/
Check the UN terms of use before reuse. Research prototype; not for official statistics.
This repo is self-contained: pyramidformer.py (model code), model.safetensors, config.json. There is no hosted inference widget because the architecture is custom; use the snippet below.
Usage
from huggingface_hub import snapshot_download
import importlib.util, os
p = snapshot_download("SallySims/pyramidformer-similarity")
spec = importlib.util.spec_from_file_location("pyramidformer", os.path.join(p, "pyramidformer.py"))
P = importlib.util.module_from_spec(spec); spec.loader.exec_module(P)
model, norm = P.load_pretrained(p)
import numpy as np, pandas as pd
E = np.load(os.path.join(p, "embeddings.npy")).astype("float32")
meta = pd.read_parquet(os.path.join(p, "index.parquet"))
# shares: array (n, 2, 101) = [male, female] x single ages 0..100 (any scale)
print(P.most_similar(model, norm, shares, E, meta, k=5)[0]) # best year of each of the 5 most similar countries
- Downloads last month
- -