harrrrsh307/social-siren-synthetic

Social Siren Synthetic Multilingual Disaster-Tweet Dataset (v1)

5,000 synthetic, programmatically-generated tweets with multi-label annotations for disaster detection, sarcasm, sentiment, and language, across English, Hindi, Marathi, and code-mixed (Hinglish) text.

IMPORTANT — read before using in a paper. This is a synthetic dataset. Every tweet was generated from templates, not collected from Twitter/X. It is intended as a controlled testbed and demonstration fixture for building and stress-testing multilingual, sarcasm-aware disaster-detection pipelines. It is not a substitute for real social-media data and should not be presented as collected/real tweets. See "Known limitations" below.

Columns

Composition

• —Genuine disaster reports (neutral/negative)
• —Sarcastic disaster reports (positive surface wording, real disaster) — the hard, valuable case
• —Metaphorical non-disaster (uses disaster keywords figuratively, e.g. "drowning in work")
• —Sarcastic non-disaster
• —Neutral chatter Roughly 60% disaster / 40% non-disaster; ~27% sarcastic; four language groups.

Intended uses (legitimate)

• —Plumbing / integration testing of a detection pipeline end-to-end
• —Demonstrating routing logic (sarcasm polarity correction) qualitatively
• —Teaching / prototyping multilingual, multi-label NLP workflows
• —A controlled stress-test of specific phenomena (sarcasm, code-mixing, metaphor)

Known limitations (disclose these in any paper)

1. 1.Synthetic origin. Generated from templates + vocabulary banks. Does not reflect the true distribution, noise, or diversity of organic tweets.
2. 2.Template separability. Because each category is produced by distinct templates, a simple TF-IDF + logistic-regression baseline reaches ~0.99 F1. This means high classifier scores on this data are NOT evidence of real-world performance. Report this openly; do not present near-100% accuracy on this dataset as a result.
3. 3.Bounded vocabulary. Limited place names, keywords, and sentence frames.
4. 4.No temporal / network / user metadata. Text only.
5. 5.Labels are generative, not human-annotated. There is no inter-annotator agreement because labels come from the generation process.

How to describe this honestly in a paper

Recommended wording:

"We construct and release a synthetic multilingual benchmark of 5,000 tweets spanning disaster, sarcasm, sentiment, and language labels, designed as a controlled testbed for sarcasm-aware routing. We emphasise that the data is synthetic: because categories are template-generated, simple lexical baselines achieve near-ceiling accuracy, so we use the dataset for qualitative and ablation-style analysis of pipeline behaviour rather than as evidence of real-world classification performance. Validation on collected real-world tweets remains future work."

License

Released as CC-BY-4.0 (attribution).