datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
distilabel-intel-orca-dpo-pairs-binarizedThis is the binarized version of distilabel Orca Pairs for DPO and ORPO.
Reference: https://huggingface.co/datasets/argilla/distilabel-intel-orca-dpo-pairs?row=0
distilabel-intel-orca-dpo-pairs
distilabel Orca Pairs for DPO
The dataset is a "distilabeled" version of the widely used dataset: Intel/orca_dpo_pairs. The original dataset has been used by 100s of open-source practitioners and models. We knew from fixing UltraFeedback (and before that, Alpacas and Dollys) that this dataset could be highly improved.
Continuing with our mission to build the best alignment datasets for open-source LLMs and the community, we spent a few hours improving it with… See the full description on the dataset page: https://huggingface.co/datasets/argilla/distilabel-intel-orca-dpo-pairs.Pexels-Pairs-Masklets-330Korca_dpo_pairs
Dataset Card for Orca DPO Pair
Dataset Description
This is a pre-processed version of the OpenOrca dataset.
The original OpenOrca dataset is a collection of augmented FLAN data that aligns, as best as possible, with the distributions outlined in the Orca paper.
It has been instrumental in generating high-performing preference-tuned model checkpoints and serves as a valuable resource for all NLP researchers and developers!
Dataset Summary
The OrcaDPO Pair… See the full description on the dataset page: https://huggingface.co/datasets/HuggingFaceH4/orca_dpo_pairs.orca_dpo_pairsThe dataset contains 12k examples from Orca style dataset Open-Orca/OpenOrca.
MoCa-CL-Pairs
MoCa Contrastive Learning Data
🏠 Homepage | 💻 Code | 🤖 MoCa-Qwen25VL-7B | 🤖 MoCa-Qwen25VL-3B | 📚 Datasets | 📄 Paper
This dataset contains datasets used for the supervised finetuning of MoCa (MoCa: Modality-aware Continual Pre-training Makes Better Bidirectional Multimodal Embeddings):
MMEB (with hard negative)
InfoSeek (from M-BEIR)
TAT-DQA
ArxivQAVisRAG
ViDoRe
ColPali
E5 text pairs (can not release due to restrictions of Microsoft)
Image Preparation
First, you… See the full description on the dataset page: https://huggingface.co/datasets/moca-embed/MoCa-CL-Pairs.crows_pairs_multilingualThis is a revised version of CrowS-Pairs that measures stereotypes in language modelling in both English and French.medical_questions_pairs
Dataset Card for [medical_questions_pairs]
Dataset Summary
This dataset consists of 3048 similar and dissimilar medical question pairs hand-generated and labeled by Curai's doctors. Doctors with a list of 1524 patient-asked questions randomly sampled from the publicly available crawl of HealthTap. Each question results in one similar and one different pair through the following instructions provided to the labelers:
Rewrite the original question in a different way while… See the full description on the dataset page: https://huggingface.co/datasets/curaihealth/medical_questions_pairs.wikipedia-title-text-pairsgeneral-reasoning-ift-pairs
Reasoning-IFT Pairs (General Domain)
This dataset provides the largest set of IFT and Reasoning answers pairs for a set of general domain queries (cf: math-domain).It is based on the Infinity-Instruct dataset, an extensive and high-quality collection of instruction fine-tuning data.
We curated 900k queries from the 7M_core subset of Infinity-Instruct, which covers multiple domains including general knowledge, commonsense Q&A, coding, and math.For each query… See the full description on the dataset page: https://huggingface.co/datasets/Scale-or-Reason/general-reasoning-ift-pairs.NijiJourney-Prompt-Pairs
NijiJourney Prompt Pairs
A dataset containing txt2img prompt pairs for training on diffusion models
The final goal of this dataset is to create an OpenJourney like model but with NijiJourney images
ramanv-image-editing-pairsiolens-onpolicy-pt-mlayer-pairswikipedia-multilingual-ir-pairs
wikipedia-multilingual-ir-pairs
This dataset is designed for multilingual information retrieval training.
Compared with raw Wikipedia paragraph dumps, it provides cleaner and more practical supervision by pairing title/section-driven queries with relevant paragraph-level documents and applying rule-based filtering to remove low-value sections and noisy fragments.
It is intended for IR model training, including contrastive learning and retrieval/reranking objectives.… See the full description on the dataset page: https://huggingface.co/datasets/hotchpotch/wikipedia-multilingual-ir-pairs.crows_pairs_multilingualOriginal from https://gitlab.inria.fr/french-crows-pairs/acl-2022-paper-data-and-code/-/tree/main/.
Data Statement for CrowS-Pairs-fr
How to use this document:
Fill in each section according to the instructions. Give as much detail as you can, but there's no need to extrapolate. The goal is to help people understand your data when they approach it. This could be someone looking at it in ten years, or it could be you yourself looking back at the data in two years.
For full details… See the full description on the dataset page: https://huggingface.co/datasets/jannalu/crows_pairs_multilingual.general-reasoning-ift-pairs
Reasoning-IFT Pairs (General Domain)
This dataset provides the largest set of IFT and Reasoning answers pairs for a set of general domain queries (cf: math-domain).It is based on the Infinity-Instruct dataset, an extensive and high-quality collection of instruction fine-tuning data.
We curated 900k queries from the 7M_core subset of Infinity-Instruct, which covers multiple domains including general knowledge, commonsense Q&A, coding, and math.For each query, we… See the full description on the dataset page: https://huggingface.co/datasets/Sidsidney/general-reasoning-ift-pairs.vqgan-pairs
VQGAN Pairs
This dataset contains ~2.4 million image pairs intended for improvement of image quality in VQGAN predictions. Each pair consists of:
A 512x512 crop of an image taken from Open Images.
A 256x256 image encoded and decoded using VQGAN, corresponding to the same image crop as the original.
This is the VQGAN implementation that was used for encoding and decoding: https://github.com/patil-suraj/vqgan-jax
License
This dataset is created using Open Images… See the full description on the dataset page: https://huggingface.co/datasets/dalle-mini/vqgan-pairs.orpo-vlm-pairs-full
ORPO VLM Preference Pairs (Full)
This dataset contains two versions of vision-language preference pairs for training VLM models using ORPO, DPO, or similar preference-based alignment methods.
Dataset Description
File
Rows
Description
orpo_pairs.jsonl
67,754
Refined/filtered pairs (recommended)
orpo_pairs_all.jsonl
94,346
Full dataset before filtering
Images: 11,982 images
Format: JSONL + PNG images
Language: English
Task: Vision-language… See the full description on the dataset page: https://huggingface.co/datasets/mncai/orpo-vlm-pairs-full.crows_pairsCrowS-Pairs, a challenge dataset for measuring the degree to which U.S. stereotypical biases present in the masked language models (MLMs).math-reasoning-ift-pairs
Reasoning-IFT Pairs (Math Domain)
Paper | Project Page
This dataset provides the largest set of IFT and Reasoning answers pairs for a set of math queries (cf: general-domain).
It is based on the Llama-Nemotron-Post-Training dataset, an extensive and high-quality collection of math instruction fine-tuning data.
We curated 150k queries from the math subset of Llama-Nemotron-Post-Training, which covers multiple domains of math questions.For each query, we used… See the full description on the dataset page: https://huggingface.co/datasets/Scale-or-Reason/math-reasoning-ift-pairs.cc12m_openai_clip-vit-base-patch32_image_image_retrieval_pairs_2022-09-13VTON-Synthetic-Pairs
Dataset Card for Glance VTON Synthetic Dataset
Dataset Summary
The Glance VTON Synthetic Dataset is a large-scale, fully synthetic paired dataset designed for training and evaluating Virtual Try-On (VTON) models. It was created to overcome the scarcity, high acquisition cost, restrictive commercial licensing, and demographic/stylistic biases of existing real-world paired VTON datasets.
The dataset features carefully curated pairs consisting of an isolated garment image… See the full description on the dataset page: https://huggingface.co/datasets/glanceai/VTON-Synthetic-Pairs.Pexels-Pairs-Text-Near-Counterfactualsagentic-prompt-injection-boundary-pairs
Agentic Prompt-Injection Boundary Pairs
Most prompt-injection datasets make the attack easy to recognize. The malicious row contains obvious override language, while the benign row discusses something unrelated. A classifier can look capable without learning the boundary that matters in production.
This dataset takes a stricter approach. Each attack is paired with a legitimate request from the same workflow. The two rows share the asset, role, tool and topic. What changes is… See the full description on the dataset page: https://huggingface.co/datasets/3nesdeniz/agentic-prompt-injection-boundary-pairs.font-square-v2-pairs-vaejupyter-code-text-pairs
Dataset Card for "jupyter-code-text-pairs"
More Information needed
oolong-pairs
Oolong-Pairs
Oolong-Pairs is a long-context, pairwise-aggregation reasoning benchmark built on top of the
oolongbench/oolong-synth dataset.
Each task presents a long context of thousands of general-knowledge questions (one per line, each
attached to a non-unique User ID). Every line can be implicitly labelled with one of six TREC
coarse categories — description and abstract concept, entity, human being, numeric value,
location, abbreviation — and the model must compute exact… See the full description on the dataset page: https://huggingface.co/datasets/mit-oasys/oolong-pairs.multilingual-cc-news-ir-related-paragraph-pairs
multilingual-cc-news-ir-related-paragraph-pairs
This dataset contains multilingual pairs of related news paragraphs derived from the same source article.
Each pair is intended to serve as a broad positive for contrastive learning, representation learning, and retrieval-oriented training.
Dataset at a glance
Task: paragraph-level relatedness / broad-positive contrastive training
Fields:
text_a: first text in the pair
text_b: second text in the pair
source_url:… See the full description on the dataset page: https://huggingface.co/datasets/hotchpotch/multilingual-cc-news-ir-related-paragraph-pairs.pairs_with_scores_v27ger-da-lir-anchor-positives-pairsThis is mirror of the GerDaLIR dataset formatted as pairs of (anchor, positive).
The German Dataset for Legal Information Retrieval (GerDaLIR) is a legal information retrieval dataset comprising a large collection of documents, passages and relevance labels. The large amount of training data we provide enables GerDaLIR to be used as a downstream task for German or multilingual language models. The task provided is a precedent retrieval task based on case documents from the open legal… See the full description on the dataset page: https://huggingface.co/datasets/malteos/ger-da-lir-anchor-positives-pairs.