Umaim/image-captioning-seq2seq

🖼️ Image Caption Generator

An AI-powered image captioning application using ResNet-50 encoder and LSTM decoder with attention mechanism.

Features

• —ResNet-50 Encoder: Extracts visual features from images
• —LSTM Decoder: Generates captions with attention mechanism
• —Two Decoding Strategies:
• —Greedy Search: Fast, picks most probable word at each step
• —Beam Search: Explores multiple candidates for richer captions
• —Beautiful Dark Theme UI: Modern, responsive interface

Model Architecture

1. 1.Image → ResNet-50 (pretrained)
2. 2.Features → Linear Encoder (2048 → 512)
3. 3.Decoder → LSTM with Embedding (300d) + Attention
4. 4.Output → Natural language caption

Training

• —Dataset: Flickr8k
• —Vocabulary Size: Dynamic (from training data)
• —Max Caption Length: 20 tokens

Usage

1. 1.Upload an image (JPG, PNG, or JPEG)
2. 2.Select decoding strategy (Greedy or Beam Search)
3. 3.Click "Generate Caption"
4. 4.View the AI-generated description

Technical Stack

• —PyTorch - Deep learning framework
• —Streamlit - Web interface
• —torchvision - Pre-trained ResNet-50
• —NLTK - Text processing

Files

• —app.py - Main Streamlit application
• —caption_model.pkl - Trained encoder/decoder weights + vocabulary
• —requirements.txt - Python dependencies
• —Dockerfile - Container configuration for deployment

Local Development

# Install dependencies
pip install -r requirements.txt

# Run the app
streamlit run app.py

Acknowledgments

Built with ❤️ using PyTorch and Streamlit