PyramidFormer similarity

0.8M parameters. Maps a pyramid to a 64-d unit vector (year is not an input). Trained with three terms: contrastive (the same country 1-2 years apart is a positive pair; same-country pairs are never negatives), an auxiliary linear head predicting median age, share under 15 and share 65+, and a term that makes embedding cosine follow pyramid L1 distance. embeddings.npy + index.parquet hold the embeddings of every WPP 1950-2023 country-year.

Evaluation (held-out countries)

kNN (k=5) regression of summary statistics, R2, database = training pyramids. Because the embedding was trained with these statistics as an auxiliary target, this is not an unsupervised test of what it captures:

Statistic From embedding From raw pyramid
median_age 0.895 0.926
share_under_15 0.856 0.895
share_65_plus 0.901 0.950

Spearman correlation between embedding cosine and negative L1 distance on random held-out pairs: 0.495. The raw pyramid is a strong baseline for these statistics, so use the embedding for compact retrieval, not as a claim of better demographic information.

Training run

quick run: 1500 optimisation steps.

Training data

UN World Population Prospects 2024, single-age population by sex, countries/areas, 1950-2023 (estimates), as processed in SallySims/wpp2024-single-age-pyramids. Split by country: 80% train / 10% validation / 10% test, so every number above is on countries the model never saw. WPP pyramids are modelled estimates, not raw census counts. Source: United Nations, DESA, Population Division (2024). World Population Prospects 2024. https://population.un.org/wpp/ Check the UN terms of use before reuse. Research prototype; not for official statistics. This repo is self-contained: pyramidformer.py (model code), model.safetensors, config.json. There is no hosted inference widget because the architecture is custom; use the snippet below.

Usage

from huggingface_hub import snapshot_download
import importlib.util, os
p = snapshot_download("SallySims/pyramidformer-similarity")
spec = importlib.util.spec_from_file_location("pyramidformer", os.path.join(p, "pyramidformer.py"))
P = importlib.util.module_from_spec(spec); spec.loader.exec_module(P)
model, norm = P.load_pretrained(p)
import numpy as np, pandas as pd
E = np.load(os.path.join(p, "embeddings.npy")).astype("float32")
meta = pd.read_parquet(os.path.join(p, "index.parquet"))
# shares: array (n, 2, 101) = [male, female] x single ages 0..100 (any scale)
print(P.most_similar(model, norm, shares, E, meta, k=5)[0])   # best year of each of the 5 most similar countries
Downloads last month
-
Safetensors
Model size
849k params
Tensor type
F32
ยท
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Dataset used to train SallySims/pyramidformer-similarity