danghaidang-passau/HateOWS-dataset-LREC2026

OWS Data for LREC 2026

This repository contains the data resources used in the paper:

Toward Generalized Cross-Lingual Hateful Language Detection with Web-Scale Data and Ensemble LLM Annotations (LREC-COLING 2026).

It provides:

• —multilingual OpenWebSearch (OWS) raw corpora (DEU, ENG, SPA, VIE)
• —an annotated subset with four base LLM annotators + three ensemble labeling strategies
• —a LightGBM-derived dataset (training probabilities / features)
• —the 16 human-labelled datasets used for validation (train / val counts below)

Splits

• —Raw
• —4L — 4-language OWS corpus (DEU, ENG, SPA, VIE)
• —deu — German unlabeled subset
• —eng — English unlabeled subset
• —spa — Spanish unlabeled subset

• —Annotated
• —annotated — subset with model probabilities and ensemble labels

• —LightGBM
• —LightGBM_dataset — LightGBM training probabilities / features (7-dataset LGB training)

• —Human 16
• —16-val-train — 16 datasets used for train and validation splits (tables below)

How to download splits

1) Load raw splits:

from datasets import load_dataset

repo = "danghaidang-passau/HateOWS-dataset-LREC2026"
ds_raw = load_dataset(repo, "raw")       

# access splits:
ds_4l = ds_raw["4L"]
ds_deu = ds_raw["deu"]
ds_eng = ds_raw["eng"]
ds_spa = ds_raw["spa"]

2) Load annotated splits:

ds_annotated = load_dataset(repo, "annotatedOWS")       
ds_annotated = ds_annotated["train"]

3) Load 16 human train/test splits:

ds_16 = load_dataset(repo, "16-val-train")       
ds_train = ds_16["train"]
ds_test = ds_16["test"]

4) Load LightGBM annotated splits:

ds_lgb = load_dataset(repo, "LightGBM_dataset")       
ds_lgb = ds_lgb["train"]

Language stats (deu, eng, spa)

Language stats 4L (deuengspa_vie)

Hate counts on annotated

The table below reports hate counts for four base annotator models and three ensemble methods. For base models, hate is derived as prob_1 >= prob_2 (same class ordering as the original pipeline).

Annotation columns (short names)

• —Base models: qwen_prob_1, qwen_prob_2, gemma_prob_1, gemma_prob_2, llama_prob_1, llama_prob_2, mistral_prob_1, mistral_prob_2
• —Ensemble probs: mean_prob_1, mean_prob_2, lgb_prob_1, lgb_prob_2
• —Ensemble labels: mean_label, lgb_label, vote_label
• —Shared metadata: text, language, token_len

16 human datasets — train / validation counts (with reference links)

Combined train / validation table and LightGBM usage (✓ indicates dataset used for LightGBM training)