How to train your ViT? Data, Augmentation, and Regularization in Vision Transformers
Paper • 2106.10270 • Published • 3
How to use probing-vits/vit_b16_patch16_224_i1k with Keras:
# !pip install -U keras tensorflow huggingface_hub
# Keras needs TensorFlow installed to read "hf://" paths, so the tensorflow backend is selected here;
# "jax" and "torch" also work for computation once TensorFlow is installed.
import os
os.environ["KERAS_BACKEND"] = "tensorflow"
import keras
model = keras.saving.load_model("hf://probing-vits/vit_b16_patch16_224_i1k")
This model is a TensorFlow port of ViT B-16 [1] trained with recipes from [2]. ImageNet-1k dataset was used for training purposes. You can refer to this notebook to know how the porting was done.
[1] An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale: https://arxiv.org/abs/2010.11929
[2] How to train your ViT? Data, Augmentation, and Regularization in Vision Transformers: https://arxiv.org/abs/2106.10270