GabeA/chatbench-specific-detail
Specific Detail Part of ChatBench: a benchmark for evaluating embedding models on chat/conversational retrieval tasks. Task Description Given a query about a specific detail, find the conversation containing it. Dataset Statistics Split Queries Corpus Documents test 547 1595 Usage from datasets import load_dataset # Load corpus corpus = load_dataset("GabeA/chatbench-specific-detail", "corpus", split="test") # Load… See the full description on the dataset page: https://huggingface.co/datasets/GabeA/chatbench-specific-detail.
Specific Detail
Part of ChatBench: a benchmark for evaluating embedding models on chat/conversational retrieval tasks.
Task Description
Given a query about a specific detail, find the conversation containing it.
Dataset Statistics
Usage
from datasets import load_dataset
# Load corpus
corpus = load_dataset("GabeA/chatbench-specific-detail", "corpus", split="test")
# Load queries
queries = load_dataset("GabeA/chatbench-specific-detail", "queries", split="test")
# Load relevance judgments (qrels)
qrels = load_dataset("GabeA/chatbench-specific-detail", split="test")With MTEB
import mteb
task = mteb.get_task("ChatBenchSpecificDetail")
evaluation = mteb.MTEB(tasks=[task])
evaluation.run(model)Schema
Corpus
Queries
Qrels (default config)
Citation
@software{chatbench2026,
title = {ChatBench: A Benchmark for Conversational Retrieval},
author = {Abinante, Gabe},
year = {2026},
url = {https://github.com/gabinante/chat-bench},
license = {Apache-2.0}
}