Clean space deploy

.gitattributes

@@ -0,0 +1,3 @@
+*.pt filter=lfs diff=lfs merge=lfs -text
+*.pth filter=lfs diff=lfs merge=lfs -text
+*.safetensors filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,184 @@
+---
+title: Neural Image Captioning
+emoji: 🖼️
+colorFrom: indigo
+colorTo: blue
+sdk: gradio
+sdk_version: 4.19.0
+app_file: app/gradio_app.py
+pinned: false
+---
+
+# 🖼️ Neural Image Caption Generation
+
+![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)
+![Python 3.10+](https://img.shields.io/badge/python-3.10+-blue.svg)
+![PyTorch](https://img.shields.io/badge/PyTorch-EE4C2C?style=flat&logo=pytorch&logoColor=white)
+
+## Overview
+
+End-to-end image captioning system using deep learning, combining computer vision (ResNet-50) with NLP (LSTM) to generate natural language descriptions of images.
+
+## Architecture
+
+```mermaid
+graph LR
+    A[Input Image] -->|Resize & Normalize| B[ResNet-50 Frozen]
+    B -->|2048-d| C[Linear Projection]
+    C -->|256-d| D[LSTM Decoder]
+    D -->|Step-by-step| E[Caption]
+```
+
+This model follows the "Show and Tell" architecture by utilizing a pre-trained CNN to extract image features and passing those features as the initial state or input to an LSTM which decodes them into a natural language sequence.
+
+## Dataset
+
+We use the **Flickr8k** dataset:
+
+- **8,092** images in total
+- **5** captions per image
+- **Splits**:
+  - Train: 6,000 images
+  - Val: 1,000 images
+  - Test: 1,000 images
+- **Preprocessing**: Lowercase conversion, punctuation removal, and frequency thresholding.
+
+## Project Structure
+
+```text
+.
+├── app/
+│   ├── app.py           (Streamlit App)
+│   └── gradio_app.py    (Gradio App)
+├── checkpoints/
+├── scripts/
+│   ├── extract_features.py
+│   ├── train.py
+│   └── evaluate.py
+├── src/
+│   ├── data/
+│   ├── evaluation/
+│   ├── inference/
+│   ├── models/
+│   └── utils/
+├── tests/
+│   ├── integration/
+│   └── unit/
+├── Dockerfile
+├── Dockerfile.gpu
+├── docker-compose.yml
+├── requirements.txt
+└── README.md
+```
+
+## Installation
+
+```bash
+git clone <repo-url>
+cd Task62
+pip install -r requirements.txt
+make setup
+```
+
+## Quick Start
+
+1. Download the Flickr8k dataset from Kaggle.
+2. Run data preparation: `python setup_data.py`
+3. Extract features: `python scripts/extract_features.py`
+4. Train the model: `python scripts/train.py`
+5. Evaluate: `python scripts/evaluate.py`
+
+## Training
+
+| Hyperparameter | Value |
+| -------------- | ----- |
+| Embed Dim      | 256   |
+| Hidden Dim     | 512   |
+| Batch Size     | 64    |
+| Learning Rate  | 1e-3  |
+
+Features include early stopping, LR scheduling, gradient clipping, and automated checkpointing.
+
+## Evaluation
+
+| Metric  | Expected Score |
+| ------- | -------------- |
+| BLEU-1  | ~60.0          |
+| BLEU-4  | ~20.0          |
+| ROUGE-L | ~45.0          |
+| METEOR  | ~22.0          |
+
+_Qualitative examples showing generated captions will be displayed here._
+
+## Web Interfaces
+
+### Streamlit App
+
+Start the Streamlit interface to upload and test your own images:
+
+```bash
+streamlit run app/app.py
+# or
+make app
+```
+
+Available at: `http://localhost:8501`
+
+### Gradio App
+
+Start the Gradio interface:
+
+```bash
+python app/gradio_app.py
+# or
+make gradio
+```
+
+Available at: `http://localhost:7860`
+
+## Docker
+
+Run both the Streamlit UI and Gradio UI inside Docker containers:
+
+```bash
+# Build the image
+docker build -t image-captioning .
+
+# Run both services
+docker-compose up
+
+# Services:
+#   Streamlit UI  → http://localhost:8501
+#   Gradio UI     → http://localhost:7860
+```
+
+## Model & Deployments
+
+### 🤗 Hugging Face Spaces
+
+Try the model live on Hugging Face Spaces: [Image Captioning Demo](https://huggingface.co/spaces/Nada0salim/image-captioning)
+
+### 🐳 Docker Hub
+
+Container image available at: [Docker Hub Repository](https://hub.docker.com/repository/docker/nada123456aa/image-caption/general)
+
+## Testing
+
+Run the test suite using pytest:
+
+```bash
+pytest tests/ -v --cov=src
+```
+
+## Technologies
+
+- Python 3.10+
+- PyTorch & torchvision (ResNet-50)
+- Streamlit & Gradio (Web UIs)
+- Docker & Docker Compose
+- NLTK, rouge-score
+- HuggingFace Hub
+
+## License
+
+MIT

app/__init__.py

@@ -0,0 +1 @@
+"""Streamlit web application package."""

app/app.py

@@ -0,0 +1,87 @@
+import os
+import sys
+import gradio as gr
+import torch
+from PIL import Image
+import nltk
+
+# Download NLTK data required for metrics/vocabulary on Hugging Face Spaces
+try:
+    nltk.data.find('tokenizers/punkt')
+except LookupError:
+    nltk.download('punkt', quiet=True)
+    nltk.download('wordnet', quiet=True)
+    nltk.download('omw-1.4', quiet=True)
+    nltk.download('punkt_tab', quiet=True)
+
+# Add project root to path to allow importing from src
+sys.path.append(os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
+
+from src.inference.pipeline import CaptionPipeline
+
+# Global pipeline instance
+pipeline = None
+
+def load_model():
+    global pipeline
+    if pipeline is None:
+        device = 'cuda' if torch.cuda.is_available() else 'cpu'
+        try:
+            pipeline = CaptionPipeline.from_checkpoint('checkpoints', device=device)
+            return f"✅ Model loaded successfully on **{device}**."
+        except Exception as e:
+            return f"❌ Failed to load model: {e}\n\nMake sure you have trained the model and `checkpoints/` exists."
+    return f"✅ Model ready."
+
+def generate_caption(image, strategy, beam_size, max_length, temperature):
+    if image is None:
+        return "⚠️ Please upload an image first."
+    if pipeline is None:
+        return "❌ Model not loaded properly."
+
+    try:
+        # Generate caption using the pipeline
+        result = pipeline.generate(
+            image,
+            strategy="beam" if strategy == "Beam Search" else "greedy",
+            beam_size=int(beam_size),
+            max_length=int(max_length),
+            temperature=float(temperature)
+        )
+        print(f"Generated caption: {result['caption']}")
+        return f"Caption: {result['caption']}\nConfidence Score: {result['score']:.4f}"
+    except Exception as e:
+        print(f"Error: {e}")
+        return f"Error generating caption: {e}"
+
+# Build Gradio Interface using Blocks
+with gr.Blocks(title="Neural Image Captioning", theme=gr.themes.Soft()) as interface:
+    gr.Markdown("# 🖼️ Neural Image Captioning")
+    gr.Markdown("Upload an image to generate a description using a CNN-LSTM model trained on Flickr8k.")
+    
+    status_text = gr.Markdown(load_model())
+    
+    with gr.Row():
+        with gr.Column(scale=1):
+            image_input = gr.Image(type="pil", label="Upload Image")
+            
+            with gr.Accordion("Advanced Settings", open=False):
+                strategy_input = gr.Radio(["Beam Search", "Greedy"], value="Beam Search", label="Decoding Strategy")
+                beam_size_input = gr.Slider(1, 10, value=3, step=1, label="Beam Size (for Beam Search)")
+                max_len_input = gr.Slider(5, 50, value=20, step=1, label="Max Caption Length")
+                temp_input = gr.Slider(0.1, 2.0, value=1.0, step=0.1, label="Temperature")
+                
+            submit_btn = gr.Button("Generate Caption", variant="primary")
+            
+        with gr.Column(scale=1):
+            output_text = gr.Textbox(label="Generated Output", lines=4)
+            
+    submit_btn.click(
+        fn=generate_caption,
+        inputs=[image_input, strategy_input, beam_size_input, max_len_input, temp_input],
+        outputs=output_text
+    )
+
+if __name__ == "__main__":
+    # Run the Gradio app (defaults to port 7860, binds to 127.0.0.1 locally so the link works on Windows)
+    interface.launch(server_port=7860, share=False)

checkpoints/best_model.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:398bc16b8f91a41ca017bb24dd3217b3cebf557c7dedf4b90edfdbacfa521dc3
+size 152183294

checkpoints/config.json

@@ -0,0 +1,33 @@
+{
+  "data": {
+    "images_dir": "data/raw/Images",
+    "captions_file": "data/raw/captions.txt",
+    "features_dir": "data/processed",
+    "vocab_path": "data/processed/vocab.json",
+    "splits_path": "data/processed/splits.json",
+    "min_word_freq": 3,
+    "max_caption_len": 35
+  },
+  "model": {
+    "encoder": "resnet50",
+    "embed_dim": 256,
+    "hidden_dim": 512,
+    "num_layers": 1,
+    "dropout": 0.5
+  },
+  "training": {
+    "batch_size": 64,
+    "num_epochs": 30,
+    "learning_rate": 0.001,
+    "weight_decay": 0.0001,
+    "patience": 5,
+    "teacher_forcing": 1.0,
+    "grad_clip": 5.0,
+    "device": "cuda",
+    "num_workers": 2
+  },
+  "checkpoint": {
+    "save_dir": "checkpoints",
+    "save_best": true
+  }
+}

checkpoints/model.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1425b59931c738faecd6142f98d6f8c5614a56b1b39d44597cc3885f2ebf9fe1
+size 113609091

configs/inference_config.yaml

@@ -0,0 +1,12 @@
+model:
+  checkpoint_dir: "checkpoints"
+  device: "cuda"
+
+inference:
+  strategy: "beam"
+  beam_size: 3
+  max_length: 20
+  temperature: 1.0
+
+hub:
+  repo_id: "username/flickr8k-image-captioning"

configs/train_config.yaml

@@ -0,0 +1,30 @@
+data:
+  images_dir: "data/raw/Images"
+  captions_file: "data/raw/captions.txt"
+  features_dir: "data/processed"
+  vocab_path: "data/processed/vocab.json"
+  splits_path: "data/processed/splits.json"
+  min_word_freq: 3
+  max_caption_len: 35
+
+model:
+  encoder: "resnet50"
+  embed_dim: 256
+  hidden_dim: 512
+  num_layers: 1
+  dropout: 0.5
+
+training:
+  batch_size: 64
+  num_epochs: 30
+  learning_rate: 0.001
+  weight_decay: 0.0001
+  patience: 5
+  teacher_forcing: 1.0
+  grad_clip: 5.0
+  device: "cuda"
+  num_workers: 2
+
+checkpoint:
+  save_dir: "checkpoints"
+  save_best: true

requirements.txt

@@ -0,0 +1,13 @@
+torch>=2.1.0
+torchvision>=0.16.0
+safetensors>=0.4.1
+huggingface-hub>=0.20.0
+Pillow>=10.1.0
+numpy>=1.24.3
+nltk>=3.8.1
+rouge-score>=0.1.2
+streamlit>=1.30.0
+pyyaml>=6.0.1
+tqdm>=4.66.0
+matplotlib>=3.7.0
+gradio>=4.0.0