shahirsao/ELI5-Human-and-AI-Answers

Overview

This dataset uses the questions from the pre-existing ELI5 dataset to generate a corresponding AI generated answer of a similar length for each human written answer. Responses were generated using Google Gemini Flash 2.0 (100k samples) as well as with Claude Haiku 3.0 (10k samples) to act as a cross-model performance validation set.

Data Format

• —Columns: "question", "realanswers", "generatedanswers"
• —Column Types: str, str, str
• —Example Record:

  questions: why can't we cure diabetes with a pancreas transplant?
  real_answers: Organ transplants are dangerous, expensive and risky. We have good methods of treating diabetes so it's not worth the trouble.
  generated_answers: The reason we can't cure diabetes with a pancreas transplant is that the underlying autoimmune condition that causes diabetes persists even after the transplant.

Note that hyperlink placeholders are still present in the human answers in this dataset and would likely need to be removed.

Generation Parameters

The AI answers were generated using the OpenRouter API in Python. The prompts given were as follows:

    prompt = (
        f"Answer the following question in about {target_length} words in unformatted text:\n\n"
        f"{question}"
    )

The temperature setting used was 0.7, which was chosen to strike a balance between factual response to the given question whilst having some level of variation in the language used.