Aswini123/FactualCS

FactualCS

FactualCS is a counterspeech generation dataset introduced in:

Counter with Evidence! A Multi-Agent Memory Efficient Reasoning Framework for Hate Category Informed Counterspeech Generation Accepted at EMNLP 2026 Main Conference

<!-- 📄 Paper: PAPER_URL --> 💻 Code: https://github.com/C0mRD/Counter_with_evidence

Dataset Description

Existing counterspeech datasets primarily focus on mapping hateful content to an appropriate response, while often treating hate speech as a homogeneous phenomenon.

However, different forms of hate require fundamentally different counter-strategies. A false factual claim may require supporting evidence, while dehumanizing content may instead require a response centered on dignity and humanization.

FactualCS was created to support this distinction.

The dataset contains 4,784 hate speech-counterspeech instances across 14 target communities, with each instance annotated with one of five functionally distinct hate-speech categories:

• —Stereotype
• —Misinformation
• —Conspiracy
• —Dehumanization
• —Non-factual hate

In addition to the hate speech and corresponding counterspeech, FactualCS provides intermediate annotations useful for grounded and category-aware counterspeech generation, including:

• —target group
• —hate-speech category
• —analyst reasoning trace
• —evidence retrieval query
• —retrieved evidence
• —source metadata

These annotations support research beyond conventional end-to-end counterspeech generation, including hate-category identification, retrieval-augmented generation, factual grounding, and reasoning-based counterspeech generation.

Dataset Structure

Each instance contains the following fields:

Hate Speech Categories

FactualCS distinguishes between five categories of hate speech.

These categories are designed around the type of counter-strategy required to address the underlying hate speech.

For detailed annotation criteria and functional boundaries between the categories, please refer to the paper.

Dataset Statistics

FactualCS contains 4,784 instances.

Data Splits

The official train, validation, and test splits are provided with the dataset and should be retained when reproducing results from the paper.

Category Distribution

The dataset spans 14 target communities.

Example

A dataset instance follows the structure:

{
    "hs_text": "...",
    "target_group": "...",
    "hs_type": "misinfo_hate",
    "analyst_reasoning": "...",
    "retrieval_query": "...",
    "retrieved_evidence": "...",
    "counterspeech": "...",
    "source_url": "...",
    "source_date": "...",
    "source_author": "..."
}

Actual dataset examples may contain offensive or disturbing language and are therefore not reproduced in this dataset card.

FIRE

FactualCS accompanies FIRE: Factuality Informed Multi-Agent REasoning Framework.

FIRE uses the structured information provided by FactualCS to generate hate-category-aware and evidence-grounded counterspeech using compact language-model agents.

The complete implementation is available at:

GitHub: https://github.com/C0mRD/Counter_with_evidence

For details of the FIRE architecture, memory module, HEAL and CARE phases, experiments, and evaluation, please refer to the paper.

Limitations

FactualCS covers five broad categories of hate speech and 14 target communities. These categories are not intended to represent every possible manifestation of online hate.

Important limitations include:

• —The five hate categories are not exhaustive.
• —Intersectional or highly ambiguous hate speech may not fit cleanly into a single category.
• —The current dataset is restricted to English.
• —Coverage across target communities is not uniform.
• —Evidence retrieval can depend on the availability and quality of external information.
• —Source information available on the web may change or become unavailable over time.
• —Counterspeech effectiveness is context dependent, and a suitable response can vary depending on conversational and cultural context.

Users should consult the paper for a more detailed discussion of the limitations of the dataset and framework.

Ethical Considerations

FactualCS contains hateful, offensive, and potentially disturbing language.

The dataset is released to support research aimed at understanding and countering online hate. It should not be used to promote, amplify, or generate hateful content.

Researchers working with this dataset should take appropriate precautions when exposing annotators, developers, or users to potentially harmful material.

Automated counterspeech generation should also be treated carefully in real-world applications. Generated responses may contain factual errors, inappropriate wording, or responses that are unsuitable for the specific social or conversational context.

Human oversight is recommended for practical deployment.

Content Warning

Warning: This dataset contains hate speech and other potentially offensive or disturbing language targeting individuals and communities.

The inclusion of such content is necessary for research on hate-speech analysis and counterspeech generation and does not represent the views of the dataset creators.

<!-- ---

Citation

If you use FactualCS in your research, please cite:

@inproceedings{fire2026,
    title     = {Counter with Evidence! A Multi-Agent Memory Efficient Reasoning Framework for Hate Category Informed Counterspeech Generation},
    author    = {AUTHOR_LIST},
    booktitle = {Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing},
    year      = {2026}
}