nithin521/Meme_Caption_Generator

😂 Meme Caption Generator

A fine-tuned GPT-2 language model designed to generate short, humorous, and social-media-style meme captions from user-provided prompts.

The model learns common meme caption patterns such as:

• —When ...
• —Me when ...
• —POV: ...
• —How it feels when ...
• —That one friend who ...

It can generate multiple caption variations for a single prompt.

🚀 Model Overview

✨ Features

• —Generate meme captions from a text prompt
• —Generate multiple caption variations
• —Supports prompt-based and random caption generation
• —Adjustable text-generation parameters
• —Designed for short, humorous, social-media-style text

Example:

Prompt

When you finally get your salary

Possible generations

When you finally get your salary and your bills were waiting for you.

Me checking my bank account after getting paid.

POV: You finally get paid but somehow you're still broke.

🧠 Model Architecture

This model is based on GPT-2, an autoregressive Transformer language model.

During fine-tuning, the model was trained to learn the distribution and writing style of meme captions.

Each training example follows a structure similar to:

<|capbos|> meme caption <|capeos|>

where:

• —<|capbos|> marks the beginning of a caption
• —<|capeos|> marks the end of a caption
• —<|cappad|> is used as the padding token

This allows the model to learn where meme captions begin and end.

📊 Training

The model was fine-tuned on a custom meme-caption dataset containing short English meme captions collected and processed for this project.

The dataset was cleaned and prepared before fine-tuning to improve training quality.

The preprocessing pipeline included:

1. 1.Caption extraction
2. 2.Text cleaning
3. 3.Duplicate removal
4. 4.Dataset preparation
5. 5.Tokenization
6. 6.Train/validation split
7. 7.GPT-2 fine-tuning

The training process used:

• —PyTorch
• —Hugging Face Transformers
• —AdamW optimization
• —Learning-rate scheduling
• —Gradient accumulation
• —Validation monitoring
• —Early stopping

⚙️ Generation Parameters

The model can be used with sampling-based generation.

Recommended starting configuration:

max_new_tokens = 25
temperature = 0.75
top_k = 40
top_p = 0.90
repetition_penalty = 1.1
no_repeat_ngram_size = 3
do_sample = True

For more creative outputs, temperature and sampling parameters can be increased.

For more predictable outputs, temperature can be decreased.

💻 Usage

Using Transformers

from transformers import AutoTokenizer, AutoModelForCausalLM

model_id = "nithin521/Meme_Caption_Generator"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id)

model.eval()

# Get special tokens directly from tokenizer
BOS_TOKEN = tokenizer.bos_token
EOS_TOKEN = tokenizer.eos_token

tokenizer.pad_token = tokenizer.eos_token

prompt = "When you finally get your salary"

# Use tokenizer's BOS token
text = BOS_TOKEN + " " + prompt

inputs = tokenizer(
    text,
    return_tensors="pt"
)

outputs = model.generate(
    **inputs,
    max_new_tokens=25,
    temperature=0.75,
    top_k=40,
    top_p=0.90,
    do_sample=True,
    repetition_penalty=1.1,
    no_repeat_ngram_size=3,
    num_return_sequences=5,
    eos_token_id=tokenizer.eos_token_id,
    pad_token_id=tokenizer.pad_token_id
)

for output in outputs:

    # Remove the input prompt
    generated_tokens = output[
        inputs["input_ids"].shape[1]:
    ]

    caption = tokenizer.decode(
        generated_tokens,
        skip_special_tokens=False
    )

    # Stop at tokenizer's EOS token
    if EOS_TOKEN:
        caption = caption.split(EOS_TOKEN)[0]

    # Remove tokenizer special tokens
    caption = caption.replace(BOS_TOKEN or "", "")
    caption = caption.replace(tokenizer.pad_token or "", "")

    caption = caption.strip()

    if caption:
        print(caption)

🎛️ Generation Controls

The model works particularly well when generation parameters are adjusted according to the desired output.

Recommended values:

Temperature:       0.75
Top-K:             40
Top-P:             0.90
Max New Tokens:    25

🌐 Interactive Demo

The model is deployed as an interactive Gradio application on Hugging Face Spaces.

Demo:

https://huggingface.co/spaces/nithin521/Meme-caption-generation

The application allows users to:

• —Enter a meme prompt
• —Generate multiple captions
• —Adjust generation parameters
• —Experiment with different levels of randomness

🎯 Intended Use

This model is intended for:

• —Meme caption generation
• —Social-media content experimentation
• —NLP demonstrations
• —Generative AI projects
• —Educational purposes
• —Exploring fine-tuned language models

⚠️ Limitations

The model may occasionally generate:

• —Grammatically incorrect sentences
• —Repetitive captions
• —Incomplete captions
• —Semantically unrelated text
• —Offensive or inappropriate language
• —Memorized patterns from the training dataset
• —Captions that require human editing

Generated captions should therefore be reviewed before being used publicly.

The model is designed for creative generation rather than factual or reliable text generation.

🔒 Safety & Responsible Use

The model generates text based on patterns learned from its training data and does not have an independent understanding of the generated content.

Users should review generated content before publishing it.

The model should not be used to generate targeted harassment, hateful content, misinformation, or other harmful material.

📚 Technologies Used

• —Python
• —PyTorch
• —Hugging Face Transformers
• —Hugging Face Hub
• —GPT-2
• —Gradio
• —Safetensors

🏗️ Project Pipeline

The complete project includes a data-processing and caption-generation pipeline:

Meme Images
     ↓
Text Detection
     ↓
YOLO-based Caption Region Detection
     ↓
Caption Cropping
     ↓
OCR / Text Extraction
     ↓
Dataset Cleaning
     ↓
Duplicate Removal
     ↓
GPT-2 Fine-tuning
     ↓
Meme Caption Generation
     ↓
Gradio Web Application

The YOLO-based text detector was developed to identify meme text regions and improve the quality of captions extracted from meme images during dataset preparation.

📈 Future Improvements

Potential improvements include:

• —Increasing the size and diversity of the training dataset
• —Improving dataset quality and caption filtering
• —Adding automated generation-quality ranking
• —Removing noisy or incomplete generations
• —Fine-tuning larger language models
• —Improving semantic control over generated captions
• —Adding multilingual meme caption generation
• —Improving safety filtering
• —Adding image-to-caption generation

👨‍💻 Project

This model was developed as part of an end-to-end AI Meme Caption Generator project combining computer vision, natural language processing, and generative AI.

Author

Nithin Kumar

Hugging Face: https://huggingface.co/nithin521