Datasetpublic

GabeA/chatbench-cross-channel

Cross Channel Part of ChatBench: a benchmark for evaluating embedding models on chat/conversational retrieval tasks. Task Description Find related conversations across different channels. Dataset Statistics Split Queries Corpus Documents test 554 1595 Usage from datasets import load_dataset # Load corpus corpus = load_dataset("GabeA/chatbench-cross-channel", "corpus", split="test") # Load queries queries =… See the full description on the dataset page: https://huggingface.co/datasets/GabeA/chatbench-cross-channel.

sourceHugging Faceapache-2.0updated 7mo agoView on Hugging Face

0likes37downloads

Dataset Card

Cross Channel

Part of ChatBench: a benchmark for evaluating embedding models on chat/conversational retrieval tasks.

Task Description

Find related conversations across different channels.

Dataset Statistics

Split	Queries	Corpus Documents
test	554	1595

Usage

python

from datasets import load_dataset

# Load corpus
corpus = load_dataset("GabeA/chatbench-cross-channel", "corpus", split="test")

# Load queries
queries = load_dataset("GabeA/chatbench-cross-channel", "queries", split="test")

# Load relevance judgments (qrels)
qrels = load_dataset("GabeA/chatbench-cross-channel", split="test")

With MTEB

python

import mteb

task = mteb.get_task("ChatBenchCrossChannel")
evaluation = mteb.MTEB(tasks=[task])
evaluation.run(model)

Schema

Corpus

Column	Type	Description
_id	string	Document ID
text	string	Document text (full conversation or message window)
title	string	Conversation title (if available)

Queries

Column	Type	Description
_id	string	Query ID
text	string	Query text

Qrels (default config)

Column	Type	Description
query-id	string	Query ID
corpus-id	string	Relevant document ID
score	int	Relevance score (1 = relevant)

Citation

bibtex

@software{chatbench2026,
  title = {ChatBench: A Benchmark for Conversational Retrieval},
  author = {Abinante, Gabe},
  year = {2026},
  url = {https://github.com/gabinante/chat-bench},
  license = {Apache-2.0}
}