gaffarshaikh07/rag-qa-evaluation-dataset
RAG QA Evaluation Dataset Overview This dataset contains test cases for evaluating Retrieval-Augmented Generation (RAG) and Large Language Model (LLM) applications. The dataset is designed from a software testing and quality engineering perspective. Dataset Structure Each test case contains: Field Description question User question sent to the AI application context Context available to the AI application expected_answer Expected… See the full description on the dataset page: https://huggingface.co/datasets/gaffarshaikh07/rag-qa-evaluation-dataset.
pretty_name: RAG QA Evaluation Dataset language:
- en license: mit tags:
- llm-evaluation
- rag
- ai-testing
- qa
- hallucination
- question-answering task_categories:
- question-answering size_categories:
- n<1K ---
RAG QA Evaluation Dataset
Overview
This dataset contains test cases for evaluating Retrieval-Augmented Generation (RAG) and Large Language Model (LLM) applications.
The dataset is designed from a software testing and quality engineering perspective.
Dataset Structure
Each test case contains:
Test Types
Grounded
Questions that can be answered directly from the supplied context.
Boundary
Questions that test limits and edge cases.
Out-of-context
Questions where the answer cannot be determined from the supplied context.
The expected behavior is for the application to avoid inventing unsupported information.
Prompt Injection
Tests designed to check whether an application follows its intended instructions and context when presented with unrelated instructions.
Intended Use
This dataset can be used for:
- RAG testing
- LLM evaluation
- AI application testing
- QA automation
- Regression testing
- Hallucination testing
- Negative testing
- Boundary testing
Potential evaluation metrics include:
- Faithfulness
- Answer Relevancy
- Contextual Relevancy
- Hallucination
- Task Completion
- Tool Correctness
- Argument Correctness
Example Test Case
Question:
What is the maximum age to purchase the policy?
Context:
The policy is available to customers aged 18 to 65 years.
Expected answer:
65 years
Test type:
grounded
Limitations
This is a small demonstration and evaluation dataset. It is not intended to represent a comprehensive benchmark for production RAG or LLM systems.
The examples are designed primarily for learning, experimentation, and software testing demonstrations.
Data Quality
The test cases were manually designed to represent common AI application testing scenarios including positive, boundary, negative, and out-of-context cases.
License
MIT