nirmalendu01/counterfactual_culture

Counterfactual Culture

Multilingual minimal-change counterfactual etiquette vignettes for five cultures, with conforming / violating pairs for factorization and representation studies.

Configs

from datasets import load_dataset

ds = load_dataset("nirmalendu01/counterfactual_culture", "samples")
norms = load_dataset("nirmalendu01/counterfactual_culture", "norms")
seed = load_dataset("nirmalendu01/counterfactual_culture", "seed_samples")

How this was created

1. Norm collection

Etiquette norms were gathered for each culture from:

1. 1.[NormAd](https://huggingface.co/datasets/akhilayerukola/NormAd) (akhilayerukola/NormAd) — Background column dos/don’ts (bullet lines) for united_states_of_america → english, plus japan, china, india, russia.
2. 2.[CultureBank](https://huggingface.co/datasets/SALT-NLP/CultureBank) (SALT-NLP/CultureBank) — American / Japanese / Chinese / Indian / Russian descriptors under Social Norms and Etiquette, filtered by agreement (≥ 0.8), embedding-deduplicated against that culture’s NormAd bullets, then GPT-curated for actionable interpersonal etiquette.

Each norm has source ∈ {normad, culturebank} in the norms config.

2. English seed vignettes

For each norm, gpt-4o-mini wrote a natural English scene with a minimal-change conforming / violating pair (light negation or small lexical flip). Country/nationality names are scrubbed from the surface text; culture is metadata only. Prompts end with an open stem (e.g. Socially, this is) for continuation-style eval.

3. Variations (Level 2 × Level 1)

Each seed norm was expanded with gpt-4o-mini into:

• —Level 2 — 10 contextual instantiations of the same underlying norm
• —Level 1 — 5 lexical paraphrases per context

Each cell is again a minimal conforming/violating pair → 50 pairs / 100 texts per norm.

4. Multilingual translation

English variations for every culture were translated with gpt-4o-mini via the OpenAI Batch API into Japanese (ja), Chinese (zh), Hindi (hi), and Russian (ru), preserving ids and pair alignment. This yields a full culture × language grid (not language-tied-only-to-its-culture).

Sample fields (samples)

Credits / source datasets

Please cite / credit the upstream resources this corpus builds on:

• —NormAd — Rao et al.; dataset `akhilayerukola/NormAd`
• —CultureBank — Shi et al. (Findings of EMNLP 2024); dataset `SALT-NLP/CultureBank`

Synthetic vignettes and translations were generated with OpenAI gpt-4o-mini. This is a research pilot, not a validated cultural benchmark. Norms are simplified etiquette descriptors and may not represent all speakers of a culture.

Splits

Pair-level 80% / 10% / 10% train / val / test, stratified by culture. Conforming and violating mates always share a split. The same content pair uses the same split across all languages (parallel multilingual holdout).