DM-Diaz/VEDB-SimCLR-ResNet18-Periph-NF

VEDB SimCLR ResNet-18 — Periph-NF

This repository contains the Periph-NF SimCLR ResNet-18 checkpoint from:

Diaz, D. M., & Henderson, M. M. (2026). Eccentricity-Constrained CNN Training Reveals Adaptive Information Coding Around the Visual Field. Proceedings of the 9th Conference on Cognitive Computational Neuroscience.

DOI: 10.32470/0416gfsq
arXiv: 2607.19316
Contributed Talk: CCN 2026 presentation on YouTube

This model is part of the Eccentricity-Constrained SimCLR Models (VEDB) collection, containing checkpoints pretrained under four visual-field conditions: Baseline, Fovea-Gaze, Periph, and Periph-NF.

Model Description

This model uses a ResNet-18 visual encoder pretrained with SimCLR self-supervised contrastive learning on naturalistic egocentric imagery with synchronized human gaze data from the Visual Experience Dataset (VEDB).

The associated study investigated whether constraining visual experience to different portions of the visual field produces systematic differences in learned representations, downstream task performance, and alignment with human visual cortex.

Periph-NF Condition

The Periph-NF condition combines a gaze-contingent peripheral mask with the NeuroFovea foveated metamer transform introduced by Deza, Jonnalagadda, & Eckstein (2019). NeuroFovea was applied to each frame before the central region was occluded with a gray circular scotoma (pixel value = 128) centered at the participant's per-frame gaze location. The scotoma boundary was Gaussian-blurred (kernel = 15) to soften the transition between the masked central region and the retained periphery.

NeuroFovea is a feed-forward foveated style-transfer model. In broad terms, it encodes the image using VGG-based feature representations, pools information over spatial regions whose scale grows with eccentricity from fixation, and decodes the resulting representation back into pixel space. In this study, the transform was run with scale parameter s = 0.4. The resulting images preserve coarse and texture-like information while progressively reducing precise spatial detail farther from fixation.

Conceptually, NeuroFovea is closely related to texture-tiling models (TTMs) of peripheral vision. Both approaches approximate peripheral information loss by replacing exact local image structure with pooled, texture-like representations over regions that grow with eccentricity. NeuroFovea is not identical to classical texture-tiling implementations, however: rather than synthesizing images from a fixed set of hand-engineered texture summary statistics, it uses learned VGG feature representations and neural style-transfer operations to generate metamer-like images.

Periph-NF differs from Periph only in the application of NeuroFovea before central masking. Periph therefore serves as the matched control for testing the added effect of eccentricity-dependent metamerization beyond removal of the central visual field itself.

The Periph-NF condition should be interpreted as a gaze-centered peripheral-only condition with a NeuroFovea-based approximation of peripheral texture pooling, not as a complete biological simulation of human peripheral vision.

The associated model variants manipulate the same source frames in complementary ways:

• Baseline: full-field input without an eccentricity-specific restriction
• Fovea-Gaze: gaze-centered central-only input
• Periph: peripheral-only input produced by masking the corresponding gaze-centered central region
• Periph-NF: peripheral-only input with a NeuroFovea transform applied before central masking

Release Status

Component	Status
Pretrained checkpoint	Available
Model card	Available
Training code	Forthcoming
Evaluation code	Forthcoming
VEDB imagery	Full dataset not redistributed; example frames included for illustration

The complete training and evaluation codebase is currently being consolidated and documented and will be linked here upon public release.

The checkpoint is being released in advance of the codebase to provide access to the model artifact used in the published study.

Technical Provenance Note and Discrepancies

Technical provenance note: The checkpoint metadata, architecture, training parameters, and implementation details documented in this model card have been re-verified against the released model checkpoint and, where available, the original training code and launch configuration. If a technical detail concerning the released model artifact differs between the associated paper and this model card, the model card should be treated as the authoritative description of the released checkpoint and its implementation. The associated paper remains the primary source for the study's scientific analyses, results, and interpretation.

Checkpoint

File:

simclr_resnet18_periph-nf_epoch120.pth.tar

This repository provides the original PyTorch training checkpoint from epoch 120 of Periph-NF SimCLR pretraining.

The checkpoint is serialized as a dictionary containing:

Key	Contents
epoch	Final training epoch (120)
arch	Backbone architecture (resnet18)
state_dict	Model parameters and registered buffers
optimizer	Adam optimizer state at the time of saving

The state_dict contains 124 entries and includes both the ResNet-18 encoder and SimCLR projection head.

Model parameters use the backbone.* namespace. The projection head is stored as:

• backbone.fc.0: Linear(512, 512)
• backbone.fc.2: Linear(512, 128)

The checkpoint therefore contains the full SimCLR model state rather than encoder weights alone.

Architecture

Overview of the VEDB preprocessing, SimCLR pretraining, downstream linear probes, and voxelwise encoding workflow.

Component	Specification
Backbone	ResNet-18
Framework	PyTorch
Learning paradigm	Self-supervised contrastive learning
Objective	SimCLR / NT-Xent
Input resolution	224 × 224
Backbone representation	512-dimensional
Projection head	Linear(512, 512) → ReLU → Linear(512, 128)
Projection dimension	128

The standard ResNet-18 classification layer was replaced during SimCLR pretraining by a two-layer projection head:

nn.Sequential(
    nn.Linear(512, 512),
    nn.ReLU(),
    nn.Linear(512, 128),
)

Training Data

Visual Experience Dataset (VEDB)

The model was pretrained using imagery from the Visual Experience Dataset (VEDB), a large-scale dataset of naturalistic egocentric experience containing more than 200 hours of integrated egocentric video, eye-movement, and odometry recordings.

VEDB resources:

• Dataset: VEDB on Databrary
• Dataset paper: Greene et al. (2024), The Visual Experience Dataset: Over 200 recorded hours of integrated eye movement, odometry, and egocentric video
• Project resources: VEDB on OSF

The full VEDB dataset is not redistributed through this repository. A small number of example frames are included solely to illustrate the visual-field transformations used in the study. Researchers wishing to reproduce training should obtain VEDB through the official distribution and comply with its applicable access and usage requirements.

Dataset Construction

A total of 717 VEDB sessions were initially retrieved through Databrary. Sessions without synchronized gaze data were excluded, leaving 514 sessions for processing and analysis.

Within task-relevant portions of each retained session:

• Frames were sampled every 2 seconds.
• Sampling used the native 25 FPS video rate.
• A maximum of 1,000 frames per session was sampled.
• The resulting SimCLR dataset contained 433,564 frames.

All data splits were performed at the video-session level to prevent leakage from temporally adjacent and environmentally correlated frames.

SimCLR Dataset Split

Split	Sessions	Frames	Frame proportion
Train	455	377,462	87.06%
Validation	28	26,026	6.00%
Test	31	30,076	6.94%
Total	514	433,564	100%

The split was constructed as an approximately 80/10/10 session-level split using stratification to balance task-label representation. Because sessions contain different numbers of sampled frames, the resulting frame percentages differ from the session-level proportions.

Validation and test sessions were held out from all representation learning and hyperparameter selection.

Frame Preprocessing

Sampled VEDB frames were first processed using a deterministic common pipeline:

1. Decode the sampled video frame.
2. Convert the image to RGB.
3. Bicubic resize to 256 px.
4. Center crop to 224 × 224.

For the Periph-NF condition, the following condition-specific transformation was applied before the shared SimCLR augmentations:

1. Obtain the participant's synchronized per-frame gaze location.
2. Apply NeuroFovea independently to the source frame, with pooling centered at the gaze-defined fixation location.
3. Use NeuroFovea scale parameter s = 0.4, producing an eccentricity-dependent, metamer-like transformation in which precise spatial structure is increasingly replaced by texture-like information away from fixation.
4. Resize the NeuroFovea output to the network's 224 × 224 input resolution.
5. Occlude the gaze-centered central region with a uniform gray circular scotoma (128).
6. Gaussian-blur the scotoma boundary (kernel = 15) to produce a feathered mask edge.
7. Use the image center as the fallback fixation location when reliable gaze information is unavailable.

This condition-specific construction occurred before the shared SimCLR augmentation pipeline.

The central masking procedure is matched to the Periph condition. The critical difference is that Periph-NF first applies NeuroFovea, introducing eccentricity-dependent texture pooling before the central region is removed. This pairing allows Periph-NF and Periph to isolate the effect of the metamerization transform while holding the peripheral mask itself constant.

SimCLR Pretraining

The four VEDB conditions used the same SimCLR architecture (PyTorch-SimCLR), optimization procedure, and augmentation pipeline. They differed only in the visual-field manipulation applied to the source imagery before SimCLR augmentation.

Hyperparameter	Value
Backbone	ResNet-18
Input size	224 × 224
Epochs	120
Batch size	512
Optimizer	Adam
Learning rate	6 × 10^-4
Weight decay	1 × 10^-4
Loss	NT-Xent
Temperature (τ)	0.07
Learning-rate schedule	CosineAnnealingLR (scheduler stepping begins after epoch 10)
Projection head	Linear(512,512) → ReLU → Linear(512,128)
Mixed precision	FP16

SimCLR Augmentations

The common SimCLR augmentation pipeline included:

• random resized cropping,
• random horizontal flipping,
• color jitter (p = 0.8),
• grayscale conversion (p = 0.2), and
• Gaussian blur with σ ~ U(0.1, 2.0).

The same augmentation pipeline was used across all four VEDB conditions and was applied after condition-specific frame construction. See PyTorch-SimCLR for further SimCLR implementation details.

Evaluation

Following SimCLR pretraining, the frozen ResNet-18 backbone was evaluated using linear probes for in-domain and out-of-domain classification and voxelwise encoding models for neural prediction.

Comparative Evaluation Results

The table below reproduces the summary metrics reported in the associated paper across all VEDB-trained conditions and reference models. Rows corresponding to this repository's Periph-NF checkpoint are bolded.

Task	Condition	Val Loss	Top-1 (%)	Top-5 (%)	Best Macro-F1 (%)
SimCLR	Baseline	0.4331	87.60	—	—
SimCLR	Fovea-Gaze	0.3749	90.43	—	—
SimCLR	Periph-NF	0.4548	90.04	—	—
SimCLR	Periph	0.4545	89.26	—	—
In-Domain	Baseline	0.9811	—	—	42.17
In-Domain	Fovea-Gaze	1.2031	—	—	43.64
In-Domain	Periph-NF	1.3090	—	—	30.93
In-Domain	Periph	1.0623	—	—	36.56
In-Domain	STL-10	1.6666	—	—	25.41
In-Domain	ImageNet-100	1.2342	—	—	41.23
In-Domain	ImageNet-1K	0.9713	—	—	43.33
VGGFace2	Baseline	7.8101	5.21	11.73	3.26
VGGFace2	Fovea-Gaze	7.9104	4.58	10.76	2.70
VGGFace2	Periph-NF	8.0232	3.39	8.17	1.90
VGGFace2	Periph	8.1681	2.54	6.39	1.35
VGGFace2	STL-10	6.9973	9.55	18.96	7.43
VGGFace2	ImageNet-100	6.7985	10.77	21.07	8.71
VGGFace2	ImageNet-1K	6.7964	10.74	21.08	8.77
Places365	Baseline	3.9690	25.63	51.90	23.16
Places365	Fovea-Gaze	4.2347	21.86	46.21	19.14
Places365	Periph-NF	4.2621	20.51	44.58	17.86
Places365	Periph	4.2671	20.26	44.10	17.65
Places365	STL-10	3.8281	26.57	53.47	24.82
Places365	ImageNet-100	3.9207	24.99	51.21	23.32
Places365	ImageNet-1K	3.6264	30.17	58.46	28.36

Note: SimCLR Top-1 is computed from the self-supervised contrastive objective and is not directly comparable to downstream supervised classification accuracy. For downstream tasks, the pretrained ResNet-18 backbone was frozen and only a linear classifier was trained; the backbone weights were not fine-tuned. Classifier checkpoints were selected by best validation Macro-F1. In-domain Top-1 accuracy is omitted because label imbalance across frames can make accuracy misleading; Macro-F1 is reported as the primary class-balanced metric. STL-10, ImageNet-100, and ImageNet-1K are treated as out-of-domain baselines because they were not pretrained on VEDB.

For in-domain classification, Macro-F1 was used as the primary class-balanced metric because of label imbalance across VEDB frame categories.

Neural Encoding Evaluation

The pretrained model was additionally evaluated using voxelwise encoding models of human fMRI responses from the Natural Scenes Dataset (NSD).

NSD contains 7T whole-brain fMRI responses to complex natural scenes. The analysis used data from 8 human participants.

For each model:

• features were extracted from Conv1, Layer1.1, Layer2.1, Layer3.1, Layer4.1, and Avgpool,
• convolutional features were spatially downsampled,
• PCA was used to retain the top 200 components per feature set,
• features were concatenated and z-scored across images, and
• regularized L2 linear regression was used to predict individual voxel responses.

For each participant, the 1,000 NSD images shared across all participants served as the held-out test set, while the remaining 9,000 images viewed by that participant were used to fit the encoding models.

Importantly, original intact NSD images were presented to every pretrained model during encoding evaluation. The Baseline, Fovea-Gaze, Periph, and Periph-NF visual-field transformations were applied during SimCLR pretraining and were not reapplied to NSD stimuli at the encoding stage.

Encoding performance was quantified using held-out voxelwise R².

For complete ROI-level prediction accuracy, statistical comparisons, variance-partitioning analyses, and comparisons across eccentricity conditions, see the associated paper.

Intended Use

This checkpoint is provided primarily for research involving:

• self-supervised visual representation learning,
• egocentric visual experience,
• gaze-contingent visual processing,
• peripheral metamer and texture-based image transformations,
• central versus peripheral information processing,
• visual-field eccentricity,
• transfer learning and linear probing,
• computational modeling of visual cortex, and
• model-to-brain comparisons.

The checkpoint may also be used as a pretrained ResNet-18 initialization for methodological extensions or comparisons with alternative gaze-centered or visual-field manipulations.

Out-of-Scope Use

This model was developed as a research representation-learning model and was not designed or validated as:

• a production image-classification system,
• a general-purpose computer-vision foundation model,
• a complete biological simulation of the human visual system or human peripheral vision, or
• a system for making decisions about individuals.

The Periph-NF manipulation combines gaze-centered central masking with the NeuroFovea metamer transform. Although NeuroFovea is motivated by eccentricity-dependent properties of peripheral vision and metamerism, this preprocessing should not be interpreted as a complete physiological model of retinal sampling, acuity falloff, cortical magnification, crowding, or other biological properties of peripheral vision.

Limitations

VEDB consists of naturalistic first-person visual experience and is consequently more temporally correlated and semantically constrained than large curated computer-vision datasets.

Only one SimCLR pretraining run per VEDB condition was used in the published study. These checkpoints therefore do not characterize variation across independent pretraining seeds.

The Periph-NF condition depends on the particular NeuroFovea implementation and its selected scale parameter (s = 0.4). The transformation is an approximation of eccentricity-dependent, texture-like peripheral encoding rather than a uniquely established model of human peripheral vision. Differences between Periph-NF and other conditions may therefore reflect both the information removed by central masking and the specific representational biases introduced by NeuroFovea metamerization.

There is also an implementation discrepancy relative to the preprocessing order described in the methods. In the implemented Periph-NF pipeline, NeuroFovea operated on imagery containing the central gray scotoma, meaning that the scotoma itself was included in the NeuroFovea transformation rather than remaining an untouched uniform mask. Consequently, Periph-NF differs from the intended description not only through eccentricity-dependent transformation of the visible peripheral image, but also through transformation of the masked central region. This should be considered when interpreting differences between Periph-NF and the matched Periph control.

The learned representations are specific to the architecture, training objective, augmentations, data-sampling procedure, visual-field manipulation, and preprocessing choices used in the study. Alternative implementations may produce different representations or downstream performance.

These weights should therefore be interpreted as reproducible artifacts of the published experimental conditions. The authors do not make the claim that the particular implementation is the uniquely optimal method for modeling visual-field eccentricity.

Loading the Model

The checkpoint contains the complete SimCLR model state, including the ResNet-18 encoder and projection head.

import torch
import torch.nn as nn
from torchvision.models import resnet18


class SimCLRResNet18(nn.Module):
    def __init__(self):
        super().__init__()

        self.backbone = resnet18(weights=None)

        self.backbone.fc = nn.Sequential(
            nn.Linear(512, 512),
            nn.ReLU(),
            nn.Linear(512, 128),
        )

    def forward(self, x):
        return self.backbone(x)


checkpoint = torch.load(
    "simclr_resnet18_periph-nf_epoch120.pth.tar",
    map_location="cpu",
    weights_only=True,
)

model = SimCLRResNet18()
model.load_state_dict(checkpoint["state_dict"], strict=True)
model.eval()

Extracting Backbone Features

To use the pretrained ResNet-18 representation without the SimCLR projection head:

# x should be a preprocessed image tensor with shape [B, 3, 224, 224]

encoder = model.backbone
encoder.fc = nn.Identity()

with torch.no_grad():
    features = encoder(x)

print(features.shape)
# torch.Size([1, 512])

Related Models

This checkpoint belongs to the Eccentricity-Constrained SimCLR Models (VEDB) collection.

• VEDB SimCLR ResNet-18 — Baseline
• VEDB SimCLR ResNet-18 — Fovea-Gaze
• VEDB SimCLR ResNet-18 — Periph
• VEDB SimCLR ResNet-18 — Periph-NF

Citation

If you use these model weights in academic work, please cite the associated study:

@inproceedings{diaz2026eccentricity,
  author    = {Diaz, Dylan M. and Henderson, Margaret M.},
  title     = {Eccentricity-Constrained CNN Training Reveals Adaptive Information Coding Around the Visual Field},
  booktitle = {Proceedings of the 9th Conference on Cognitive Computational Neuroscience},
  address   = {New York, NY, USA},
  year      = {2026},
  doi       = {10.32470/0416gfsq}
}

Proceedings: Conference on Cognitive Computational Neuroscience 2026
Preprint: arXiv:2607.19316

VEDB Citation

Researchers using the underlying Visual Experience Dataset should also cite:

Greene, M. R., et al. (2024). The Visual Experience Dataset: Over 200 recorded hours of integrated eye movement, odometry, and egocentric video. Journal of Vision, 24(11), 6.

See the VEDB dataset paper for the complete author list and citation information.

NeuroFovea Citation

The Periph-NF preprocessing uses the NeuroFovea metamer transform. Researchers specifically using or extending this transformation should also cite:

@inproceedings{deza2019towards,
  title     = {Towards Metamerism via Foveated Style Transfer},
  author    = {Deza, Arturo and Jonnalagadda, Aditya and Eckstein, Miguel P.},
  booktitle = {International Conference on Learning Representations},
  year      = {2019},
  url       = {https://openreview.net/forum?id=BJzbG20cFQ}
}

NeuroFovea code: PyTorch repository
Paper: Towards Metamerism via Foveated Style Transfer

License

The model checkpoint in this repository is released under the Apache License 2.0.

The VEDB dataset and other third-party resources used in the associated study remain subject to their respective licenses, access requirements, and terms of use. This repository does not redistribute the full VEDB dataset; a small number of example frames are included for illustration of the published visual-field manipulations.