goofy0422/MedEmbed-training-triplets-v1

MedEmbed Dataset - v1

Dataset Description

The MedEmbed dataset is a specialized collection of medical and clinical data designed for training and evaluating embedding models in healthcare-related natural language processing (NLP) tasks, particularly information retrieval.

GitHub Repo: https://github.com/abhinand5/MedEmbed

Technical Blog Post: Click here

Dataset Summary

This dataset contains various configurations of medical text data, including corpus text, query-response pairs, and merged data for contrastive learning. It is specifically curated to support the development and evaluation of the MedEmbed embedding model family.

Supported Tasks and Leaderboards

The primary tasks supported by this dataset include:

• —Medical Information Retrieval
• —Clinical Question Answering
• —Semantic Search in Medical Contexts

The dataset has been used to benchmark the MedEmbed-v0.1 models against general-purpose embedding models across various medical NLP tasks, including ArguAna, MedicalQARetrieval, NFCorpus, PublicHealthQA, and TRECCOVID.

Languages

The dataset is primarily in English (en).

Dataset Structure

Data Instances

The dataset is organized into four main configurations:

1. 1.corpus: Contains individual text documents
2. 2.Features: id (string), text (string)

1. 1.default: Contains query-response triplets for contrastive learning
2. 2.Features: query (string), pos (string), neg (string), query_id (string), pos_id (string), neg_id (string)

1. 1.merged: Contains queries with positive and multiple negative responses
2. 2.Features: query (string), query_id (string), pos (string), neg (list of strings), pos_id (string), neg_id (list of strings)

1. 1.queries: Contains only the queries
2. 2.Features: id (string), query (string)

1. 1.JSONL: Contains data in the merged format suitable for training with packages like FlagEmbeddings and RAGatouille.
2. 2.Features: query (string), query_id (string), pos (string), neg (list of strings), pos_id (string), neg_id (list of strings)
3. 3.Data can be download from the path -> ./jsonl/triplets.jsonl

Data Fields

The data fields vary depending on the configuration, as described above.

Data Splits

All configurations have a train split. The number of examples in each split is as follows:

• —corpus: 27,590 examples
• —default: 232,684 examples
• —merged: 21,689 examples
• —queries: 21,689 examples

Dataset Creation

The model was trained using a simple yet effective synthetic data generation pipeline:

1. 1.Source: Clinical notes from PubMed Central (PMC)
2. 2.Processing: LLaMA 3.1 70B model used to generate query-response pairs
3. 3.Augmentation: Negative sampling for challenging examples
4. 4.Format: Triplets (query, positive response, negative response) for contrastive learning

Considerations for Using the Data

Social Impact of Dataset

This dataset has the potential to improve medical information retrieval and access to healthcare knowledge, potentially leading to better patient care and medical research outcomes.

Discussion of Biases

Users should be aware of potential biases in medical data, including but not limited to:

• —Demographic biases in medical research and clinical trials
• —Geographical biases in healthcare practices
• —Historical biases in medical literature

Other Known Limitations

The dataset is primarily focused on English-language medical texts and may not generalize well to other languages or non-medical domains.

Additional Information

Dataset Curators

The dataset was curated by Abhinand Balachandran as part of the MedEmbed project.

Licensing Information

The dataset is released under the Apache 2.0 license.

Citation Information

If you use this dataset in your research, please cite:

@software{balachandran2024medembed,
  author = {Balachandran, Abhinand},
  title = {MedEmbed: Medical-Focused Embedding Models},
  year = {2024},
  url = {https://github.com/abhinand5/MedEmbed}
}

Contributions

We welcome contributions to improve and expand this dataset. Please visit our GitHub repository for more information on how to contribute.