ResNet-50 ImageNet training trajectory (103 checkpoints)

The final checkpoint reaches 76.7 % top-1 / 93.3 % top-5 on the 50,000 ImageNet validation images, using a single 224 × 224 center crop.

Files

file contents
checkpoints/step0000000.pt … checkpoints/step0112590.pt 103 checkpoints, ~103 MB each
trajectory.csv step, epoch, val_top1, val_top5 of every checkpoint

Each checkpoint is a dict with these keys:

  • model: a torchvision.models.resnet50 state dict;
  • step, epoch: the training position;
  • val_top1, val_top5: ImageNet validation accuracy;
  • lr: the learning rate at that step;
  • recipe: the name of the training recipe.

When checkpoints were saved (1 epoch = 1,251 steps of 1,024 images):

  • step 0: the untrained network;
  • steps 16, 32, 64, 125, 250, 500, 750, 1000;
  • epochs 1.5, 2.5, 3.5 and 4.5;
  • the end of every epoch from 1 to 90.

Usage

import torch, torchvision
from huggingface_hub import hf_hub_download

path = hf_hub_download("LSL-datasets/resnet50-imagenet-trajectory", "checkpoints/step0112590.pt")
ckpt = torch.load(path, map_location="cpu", weights_only=True)
model = torchvision.models.resnet50()
model.load_state_dict(ckpt["model"])
model.eval()
print(ckpt["epoch"], ckpt["val_top1"])      # 90.0 0.76658

Inputs use the standard torchvision preprocessing:

  1. Resize so the short side is 256 pixels.
  2. Take the central 224 × 224 crop.
  3. Normalize with mean (0.485, 0.456, 0.406) and std (0.229, 0.224, 0.225).

Training recipe

The recipe follows Goyal et al. (2017), Accurate, Large Minibatch SGD, but uses a cosine schedule instead of step decay:

  • Model: torchvision.models.resnet50(zero_init_residual=True), trained from random initialization with seed 0.
  • Optimizer: SGD with momentum 0.9 and weight decay 1e-4 on all parameters.
  • Batch: 1,024 images in total, on 2 GPUs with DistributedDataParallel.
  • Learning rate: peaks at 0.4 (0.1 per 256 images). It warms up linearly for 5 epochs, then follows a cosine decay to 0 at epoch 90.
  • Augmentation: random-resized 224 crops and horizontal flips. There is no label smoothing and no mixup.
  • Precision: bfloat16 mixed precision, with the channels-last memory format.
  • Data: ImageNet-1k from ILSVRC/imagenet-1k, in parquet format.

License

These weights were trained on ImageNet-1k. Their use follows the ImageNet terms of access: non-commercial research and education.

Downloads last month

-

Downloads are not tracked for this model. How to track
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Dataset used to train LSL-datasets/resnet50-imagenet-trajectory