PEIRA: pretrained checkpoints

Checkpoints for PEIRA: Learning Predictive Encoders through Inter-View Regressor Alignment (Arbel*, Terver*, Ponce, NeurIPS 2026; arXiv:2605.17671). Project page: peira-ssl.github.io. Code: github.com/Basile-Terv/PEIRA. Loss: pip install peira-loss (github.com/MichaelArbel/peira-loss).

Each file is the ResNet-50 backbone only (no projector), pretrained for 100 epochs on ImageNet-1K (batch 2048, LARS), for linear probing and transfer; these are the baseline and PEIRA rows of Table 1 of the paper. A file is a torch.save dict with model_state_dict (backbone weights), epoch (0-indexed, so 99), the training config and a metadata dict (method, projector dims used in training, seed, epochs, offline top-1). Top-1 is the offline linear probe (VICReg protocol: SGD, 100 epochs, batch 1024) on frozen backbone features.

Files

Method Projector hidden k (projector output) Seed ImageNet-1K top-1 Weights
SIGReg 2048 128 1 66.46 in1k_rn50_sigreg_seed1.pth
SIGReg 2048 128 1000 66.38 in1k_rn50_sigreg_seed1000.pth
SIGReg 2048 128 10000 65.94 in1k_rn50_sigreg_seed10000.pth
VICReg 8192 8192 1 68.80 in1k_rn50_vicreg_seed1.pth
VICReg 8192 8192 1000 68.71 in1k_rn50_vicreg_seed1000.pth
VICReg 8192 8192 10000 68.90 in1k_rn50_vicreg_seed10000.pth
PEIRA 8192 256 1 66.51 in1k_rn50_peira_k256_seed1.pth
PEIRA 8192 256 1000 66.32 in1k_rn50_peira_k256_seed1000.pth
PEIRA 8192 256 10000 66.51 in1k_rn50_peira_k256_seed10000.pth
PEIRA 8192 1024 1 69.23 in1k_rn50_peira_k1024_seed1.pth
PEIRA 8192 1024 1000 69.12 in1k_rn50_peira_k1024_seed1000.pth
PEIRA 8192 1024 10000 69.15 in1k_rn50_peira_k1024_seed10000.pth

PEIRA recipe: EMA Σ (Algorithm 1), λ=0.1, η 0.99→0.95, grad clip 1.0. Three-seed means (± std over seeds): k=256 reaches 66.45 ± 0.11 on ImageNet-1K (70.89 ± 0.18 average transfer); k=1024 reaches 69.17 ± 0.06 (71.09 ± 0.24 average transfer).

Usage

Standalone, the backbone is a torchvision ResNet-50 with fc = Identity():

import torch, torchvision
from huggingface_hub import hf_hub_download

ckpt = hf_hub_download("peira-ssl/PEIRA", "in1k_rn50_peira_k1024_seed1.pth")
sd = torch.load(ckpt, map_location="cpu", weights_only=True)["model_state_dict"]
backbone = torchvision.models.resnet50()
backbone.fc = torch.nn.Identity()
backbone.load_state_dict({k.removeprefix("backbone.backbone."): v for k, v in sd.items()
                          if k.startswith("backbone.backbone.")})
backbone.eval()
with torch.no_grad():
    features = backbone(torch.randn(1, 3, 224, 224))  # [1, 2048]

With the code repository installed, probe.py takes these files directly, e.g. for the transfer probe:

python probe.py --ckpt <ckpt> --datasets lejepa8 --transfer-root $EBJEPA_DSETS/transfer --out peira.json

License

The weights are released under Apache-2.0, like the code. They were trained on ImageNet-1K, whose terms of access apply.

Citation

@inproceedings{arbel2026peira,
  title     = {{PEIRA}: Learning Predictive Encoders through Inter-View Regressor Alignment},
  author    = {Arbel, Michael and Terver, Basile and Ponce, Jean},
  booktitle = {Advances in Neural Information Processing Systems (NeurIPS)},
  year      = {2026},
  eprint    = {2605.17671},
  archivePrefix = {arXiv},
}
Downloads last month

-

Downloads are not tracked for this model. How to track
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Dataset used to train peira-ssl/PEIRA

Paper for peira-ssl/PEIRA