m42-health/HC4
HC4 (Healthcare Comprehensive Commons Corpus) HC4 is a large-scale pretraining dataset containing over 65 billion tokens from diverse healthcare-related sources. The corpus was curated to enable systematic investigation of how data composition influences language model behavior, including potential demographic biases. Dataset Overview Dataset Name: HC4 (Healthcare Comprehensive Commons Corpus) Size: 153GB (around 65 billion tokens) Number of samples: 9.7+… See the full description on the dataset page: https://huggingface.co/datasets/m42-health/HC4.
HC4 (Healthcare Comprehensive Commons Corpus)
HC4 is a large-scale pretraining dataset containing over 65 billion tokens from diverse healthcare-related sources. The corpus was curated to enable systematic investigation of how data composition influences language model behavior, including potential demographic biases.
Dataset Overview
Dataset Name: HC4 (Healthcare Comprehensive Commons Corpus)
Size: 153GB (around 65 billion tokens)
Number of samples: 9.7+ million documents from diverse sources including peer-reviewed scientific literature collected from PubMed Central, Semantic Scholar, OpenAlex repositories
Repository: m42-health/HC4
Purpose: Pretraining large language models for healthcare applications
Format: .parquet files
License: Open licenses for each data sample permitting commercial use and redistribution
Organization: M42 (Abu Dhabi)
How to load the dataset
from datasets import load_dataset
dataset = load_dataset("m42-health/HC4")Details
This dataset is accompanied by a peer-reviewed research paper accepted at EMNLP 2025 Conference, which presents comprehensive bias analysis methodology for clinical LLMs and provides transparency in dataset composition and curation.
Reference Paper: "Building Trust in Clinical LLMs: Bias Analysis and Dataset Transparency" (EMNLP 2025)
Link to arxiv page: https://arxiv.org/pdf/2510.18556