CoolFace
Datasetpublic

illuin-conteb/covid-qa

ConTEB - Covid-QA This dataset is part of ConTEB (Context-aware Text Embedding Benchmark), designed for evaluating contextual embedding model capabilities. It focuses on the theme of Healthcare, particularly stemming from articles about the COVID-19 pandemic. Dataset Summary This dataset was designed to elicit contextual information. It is built upon the COVID-QA dataset. To build the corpus, we start from the pre-existing collection documents, extract the text… See the full description on the dataset page: https://huggingface.co/datasets/illuin-conteb/covid-qa.

sourceHugging Faceupdated 1y agoView on Hugging Face
1likes603downloads
Dataset Card

ConTEB - Covid-QA

This dataset is part of ConTEB (Context-aware Text Embedding Benchmark), designed for evaluating contextual embedding model capabilities. It focuses on the theme of Healthcare, particularly stemming from articles about the COVID-19 pandemic.

Dataset Summary

This dataset was designed to elicit contextual information. It is built upon the COVID-QA dataset. To build the corpus, we start from the pre-existing collection documents, extract the text, and chunk them (using LangChain's RecursiveCharacterSplitter with a threshold of 1000 characters). We use GPT-4o to annotate which chunk, among the gold document, best contains information needed to answer the query. Since chunking is done a posteriori without considering the questions, chunks are not always self-contained and eliciting document-wide context can help build meaningful representations.

This dataset provides a focused benchmark for contextualized embeddings. It includes a curated set of original documents, chunks stemming from them, and queries.

  • —Number of Documents: 115
  • —Number of Chunks: 3351
  • —Number of Queries: 1111
  • —Average Number of Tokens per Doc: 153.9

Dataset Structure (Hugging Face Datasets)

The dataset is structured into the following columns:

  • —`documents`: Contains chunk information:
  • —"chunk_id": The ID of the chunk, of the form doc-id_chunk-id, where doc-id is the ID of the original document and chunk-id is the position of the chunk within that document.
  • —"chunk": The text of the chunk
  • —`queries`: Contains query information:
  • —"query": The text of the query.
  • —"answer": The answer relevant to the query, from the original dataset.
  • —"chunk_id": The ID of the chunk that the query is related to, of the form doc-id_chunk-id, where doc-id is the ID of the original document and chunk-id is the position of the chunk within that document.

Usage

We will upload a Quickstart evaluation snippet soon.

Citation

We will add the corresponding citation soon.

Acknowledgments

This work is partially supported by ILLUIN Technology, and by a grant from ANRT France.

Copyright

All rights are reserved to the original authors of the documents.