FutureMa/EvasionBench

EvasionBench

<p align="center"> <a href="https://iiiiqiiii.github.io/EvasionBench"><img src="https://img.shields.io/badge/Project-Page-blue?style=for-the-badge" alt="Project Page"></a> <a href="https://huggingface.co/FutureMa/Eva-4B-V2"><img src="https://img.shields.io/badge/🤗-Model-yellow?style=for-the-badge" alt="Model"></a> <a href="https://github.com/IIIIQIIII/EvasionBench"><img src="https://img.shields.io/badge/GitHub-Repo-black?style=for-the-badge&logo=github" alt="GitHub"></a> <a href="https://colab.research.google.com/github/IIIIQIIII/EvasionBench/blob/main/scripts/eva4binference.ipynb"><img src="https://img.shields.io/badge/Colab-QuickStart-F9AB00?style=for-the-badge&logo=googlecolab" alt="Open In Colab"></a> <a href="https://arxiv.org/abs/2601.09142"><img src="https://img.shields.io/badge/arXiv-Paper-red?style=for-the-badge" alt="Paper"></a> </p>

EvasionBench is a benchmark dataset for detecting evasive answers in earnings call Q&A sessions. The task is to classify how directly corporate management addresses questions from financial analysts.

Dataset Description

• —Repository: https://github.com/IIIIQIIII/EvasionBench
• —Paper: https://arxiv.org/abs/2601.09142
• —Point of Contact: GitHub Issues

Dataset Summary

This dataset contains 16,726 question-answer pairs from earnings call transcripts, each labeled with one of three evasion levels. The labels were generated using the Eva-4B-V2 model, a fine-tuned classifier specifically trained for financial discourse evasion detection.

Supported Tasks

• —Text Classification: Classify the directness of management responses to analyst questions.

Languages

English

Dataset Structure

Data Fields

Label Definitions

Data Statistics

Example

{
    "uid": "4addbff893b81f64131fdc712d7a6d9a",
    "question": "What is the expected margin for Q4?",
    "answer": "We expect it to be 32%.",
    "eva4b_label": "direct"
}

Usage

Loading the Dataset

from datasets import load_dataset

dataset = load_dataset("FutureMa/EvasionBench")

Loading from Parquet

import pandas as pd

df = pd.read_parquet("evasionbench_17k_eva4b_labels_dedup.parquet")

Quick Start: Inference with Eva-4B-V2

from datasets import load_dataset
from transformers import pipeline

# Load dataset
dataset = load_dataset("FutureMa/EvasionBench")

# Load model using text-generation pipeline
pipe = pipeline("text-generation", model="FutureMa/Eva-4B-V2", device_map="auto")

# Get a sample
sample = dataset["train"][0]
question = sample["question"]
answer = sample["answer"]

# Prepare prompt
prompt = f"""You are a financial analyst. Your task is to Detect Evasive Answers in Financial Q&A

Question: {question}
Answer: {answer}

Response format:

{{"label": "direct|intermediate|fully_evasive"}}

Answer in ```json content, no other text"""

# Run inference
result = pipe(prompt, max_new_tokens=64, do_sample=False)
print(result[0]["generated_text"])

For a complete inference example with batch processing and evaluation, see our Colab notebook.

Dataset Creation

Source Data

The question-answer pairs are derived from publicly available earnings call transcripts.

Annotation Process

Labels were generated using Eva-4B-V2, a 4B parameter model fine-tuned specifically for evasion detection in financial discourse. Eva-4B-V2 achieves 84.9% Macro-F1 on the EvasionBench evaluation set, outperforming frontier LLMs including Claude Opus 4.5 and Gemini 3 Flash.

Considerations for Using the Data

Social Impact

This dataset can be used to:

• —Improve transparency in corporate communications
• —Assist financial analysts in identifying evasive responses
• —Support research in financial NLP and discourse analysis

Limitations

• —Labels are model-generated (Eva-4B-V2) rather than human-annotated
• —The dataset reflects the distribution of evasion patterns in the source data
• —Performance may vary across different industries or time periods

Citation

If you use this dataset, please cite:

@misc{ma2026evasionbenchlargescalebenchmarkdetecting,
  title={EvasionBench: A Large-Scale Benchmark for Detecting Managerial Evasion in Earnings Call Q&A},
  author={Shijian Ma and Yan Lin and Yi Yang},
  year={2026},
  eprint={2601.09142},
  archivePrefix={arXiv},
  primaryClass={cs.LG},
  url={https://arxiv.org/abs/2601.09142}
}

License

Apache 2.0