satyame639291/sarcasm-distilbert

sarcasm-distilbert

A fine-tuned distilbert-base-uncased model for classifying news headlines as Sarcastic or Not Sarcastic.

• —GitHub repo (training notebook, EDA, baseline comparison): https://github.com/krskumarsatyam777-glitch/news-headline-sarcasm-detector
• —Live demo: https://huggingface.co/spaces/satyame639291/newsheadlinescarcasm_detector

Model Description

Fine-tuned version of `distilbert-base-uncased` for binary sequence classification, trained on the News Headlines Dataset for Sarcasm Detection (Misra, 2019) — headlines from The Onion (satirical) paired with headlines from HuffPost (genuine).

How to Get Started

from transformers import pipeline

classifier = pipeline("text-classification", model="satyame639291/sarcasm-distilbert")
classifier("Shocking, Usain bolt realisies he can use legs for walking")

Training Details

• —Base model: distilbert-base-uncased
• —Epochs: 3 | Learning rate: 2e-5 | Batch size: 32 | Weight decay: 0.01
• —Model selection: Best checkpoint by validation F1 (load_best_model_at_end=True) — epoch 2 was selected as the final model.
• —Hardware: Single T4 GPU (Google Colab).

Results

Evaluated on a held-out test set (4,276 headlines) not used in training or model selection:

Full methodology, EDA, and error analysis: training notebook.

Error patterns: false positives tended to be plain, non-humorous headlines rather than obviously quirky ones; false negatives mostly required outside knowledge (e.g. a public figure's reputation) that isn't present in the text itself.

Limitations

• —Labels come from source publication (Onion vs. HuffPost), not human sarcasm annotation — the model may partly learn outlet writing style rather than sarcasm itself.
• —Not validated on social media, reviews, conversational text, or non-English text.
• —Sarcasm requiring external/cultural knowledge is harder to detect than sarcasm signaled by wording alone.

Contact

Satyam — GitHub | Hugging Face