Image-to-Text
Transformers
PyTorch
Safetensors
English
git
image-text-to-text
vision
image-captioning
Instructions to use alexgk/git-large-coco with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use alexgk/git-large-coco with Transformers:
# Use a pipeline as a high-level helper # Warning: Pipeline type "image-to-text" is no longer supported in transformers v5. # You must load the model directly (see below) or downgrade to v4.x with: # 'pip install "transformers<5.0.0' from transformers import pipeline pipe = pipeline("image-to-text", model="alexgk/git-large-coco")# Load model directly from transformers import AutoProcessor, AutoModelForMultimodalLM processor = AutoProcessor.from_pretrained("alexgk/git-large-coco") model = AutoModelForMultimodalLM.from_pretrained("alexgk/git-large-coco", device_map="auto") - Notebooks
- Google Colab
- Kaggle
| from typing import Dict, List, Any | |
| from PIL import Image | |
| from io import BytesIO | |
| from transformers import pipeline | |
| import base64 | |
| class EndpointHandler(): | |
| def __init__(self, path=""): | |
| self.pipeline=pipeline("image-to-text",model=path) | |
| def __call__(self, data: Dict[str, Any]) -> str: | |
| """ | |
| data args: | |
| images (:obj:`string`) | |
| Return: | |
| A str containing a caption for the text | |
| """ | |
| inputs = data.pop("inputs", data) | |
| # decode base64 image to PIL | |
| image = Image.open(BytesIO(base64.b64decode(inputs['image']))) | |
| # run prediction one image wit provided candiates | |
| prediction = self.pipeline(images=[image]) | |
| return prediction[0] |