Pumponchik commited on
Commit
ee89c03
·
verified ·
1 Parent(s): 3462329

Upload README.md

Browse files
Files changed (1) hide show
  1. README.md +23 -44
README.md CHANGED
@@ -1,53 +1,32 @@
1
- readme_content = """# Image Captioning Model
 
 
 
 
 
 
 
 
 
2
 
3
- This is an Image Captioning model trained on a custom dataset with 6000 training images and 1000 validation images. It uses ResNet50 for image feature extraction and an LSTM with fine-tuned GloVe embeddings for caption generation.
 
 
4
 
5
  ## Model Details
6
  - **Image Encoder**: ResNet50
7
  - **Text Decoder**: LSTM with fine-tuned GloVe embeddings
8
  - **Vocabulary Size**: 3478
9
- - **Embedding Dimension**: 512
10
- - **Hidden Size**: 1024
11
- - **Number of Layers**: 2
12
- - **Max Caption Length**: 30
13
-
14
- ## Usage
15
- Load the model and tokenizer using PyTorch and the provided `model.py`.
16
-
17
- ### Example
18
- ```python
19
- import torch
20
- from PIL import Image
21
- import pickle
22
- from huggingface_hub import hf_hub_download
23
- from model import image_captioning_model
24
 
25
- # Download files
26
- repo_id = "Pumponchik/image_caption_weak"
27
- model_path = hf_hub_download(repo_id=repo_id, filename="pytorch_model.bin")
28
- config_path = hf_hub_download(repo_id=repo_id, filename="config.json")
29
- tokenizer_path = hf_hub_download(repo_id=repo_id, filename="tokenizer.pkl")
30
- transform_path = hf_hub_download(repo_id=repo_id, filename="image_transforms.pkl")
31
 
32
- # Load config
33
- with open(config_path, "r") as f:
34
- config = json.load(f)
35
-
36
- # Load model
37
- model = image_captioning_model(**config)
38
- model.load_state_dict(torch.load(model_path))
39
- model.eval()
40
- device = "cuda" if torch.cuda.is_available() else "cpu"
41
- model.to(device)
42
-
43
- # Load tokenizer and transforms
44
- with open(tokenizer_path, "rb") as f:
45
- tokenizer = pickle.load(f)
46
- with open(transform_path, "rb") as f:
47
- transform = pickle.load(f)
48
 
49
- # Generate caption
50
- image = Image.open("example.png").convert("RGB")
51
- image_tensor = transform(image).unsqueeze(0).to(device)
52
- caption = model.generate_caption(image_tensor, tokenizer, max_length=config["max_len"], device=device)
53
- print("Generated Caption:", caption)
 
1
+ ---
2
+ title: Image Captioning Demo
3
+ emoji: 📷
4
+ colorFrom: blue
5
+ colorTo: green
6
+ sdk: gradio
7
+ sdk_version: 4.44.0
8
+ app_file: app.py
9
+ pinned: false
10
+ ---
11
 
12
+ # Image Captioning Demo
13
+
14
+ This Space demonstrates an Image Captioning model trained on a custom dataset with 6000 training images. Upload an image to generate a caption.
15
 
16
  ## Model Details
17
  - **Image Encoder**: ResNet50
18
  - **Text Decoder**: LSTM with fine-tuned GloVe embeddings
19
  - **Vocabulary Size**: 3478
20
+ - **Max Caption Length**: 20
21
+ - **Sampling**: Top-k (k=5)
 
 
 
 
 
 
 
 
 
 
 
 
 
22
 
23
+ ## Example
24
+ - **Input**: Image of a dog
25
+ - **Output**: "A dog is running in the park"
 
 
 
26
 
27
+ ## Usage
28
+ 1. Upload an image in PNG or JPEG format.
29
+ 2. Click "Submit" to generate a caption.
 
 
 
 
 
 
 
 
 
 
 
 
 
30
 
31
+ ## License
32
+ MIT