GabeA/chatbench-conversation-similarity
Conversation Similarity Part of ChatBench: a benchmark for evaluating embedding models on chat/conversational retrieval tasks. Task Description Given a conversation, find other conversations about similar topics. Dataset Statistics Split Queries Corpus Documents test 82 1595 Usage from datasets import load_dataset # Load corpus corpus = load_dataset("GabeA/chatbench-conversation-similarity", "corpus", split="test") #… See the full description on the dataset page: https://huggingface.co/datasets/GabeA/chatbench-conversation-similarity.
Conversation Similarity
Part of ChatBench: a benchmark for evaluating embedding models on chat/conversational retrieval tasks.
Task Description
Given a conversation, find other conversations about similar topics.
Dataset Statistics
Usage
from datasets import load_dataset
# Load corpus
corpus = load_dataset("GabeA/chatbench-conversation-similarity", "corpus", split="test")
# Load queries
queries = load_dataset("GabeA/chatbench-conversation-similarity", "queries", split="test")
# Load relevance judgments (qrels)
qrels = load_dataset("GabeA/chatbench-conversation-similarity", split="test")With MTEB
import mteb
task = mteb.get_task("ChatBenchConversationSimilarity")
evaluation = mteb.MTEB(tasks=[task])
evaluation.run(model)Schema
Corpus
Queries
Qrels (default config)
Citation
@software{chatbench2026,
title = {ChatBench: A Benchmark for Conversational Retrieval},
author = {Abinante, Gabe},
year = {2026},
url = {https://github.com/gabinante/chat-bench},
license = {Apache-2.0}
}