File size: 960 Bytes
7f5b8c9 d2c55d2 7f5b8c9 8671fc7 7f5b8c9 36bb8af 1fd3456 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 | ---
license: mit
tags:
- pytorch_model_hub_mixin
- model_hub_mixin
pipeline_tag: image-to-text
---
This model has been pushed to the Hub using the [PytorchModelHubMixin](https://huggingface.co/docs/huggingface_hub/package_reference/mixins#huggingface_hub.PyTorchModelHubMixin) integration:
- Library: [More Information Needed]
- Docs: [More Information Needed]
## About the project
This is a decoder of image captioning model.
The image will be first preprocessed and resized to (224, 224) and then passed to ViT_b_32(with no classification layer), and then this will output
(N, 768). Then this will be repeated 32(max_length) times and will be passed to K, V to CrossMultiHeadAttention block in decoder. This model was trained with
Microsoft COCO2017 dataset and acheived 0.54 of masked_accuracy on validation set.
## Sample Code
To use this model, first you need to download ViT_b_32 which will be used as encoder and download decoder from this repo.
|