datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
Snorkel-Mistral-PairRM-DPO-Dataset
Dataset:
This is the data used for training Snorkel model
We use ONLY the prompts from UltraFeedback; no external LLM responses used.
Methodology:
Generate 5 response variations for each prompt from a subset of 20,000 using the LLM - to start, we used Mistral-7B-Instruct-v0.2.
Apply PairRM for response reranking.
Update the LLM by applying Direct Preference Optimization (DPO) on the top (chosen) and bottom (rejected) responses.
Use this LLM as the base model for the next… See the full description on the dataset page: https://huggingface.co/datasets/snorkelai/Snorkel-Mistral-PairRM-DPO-Dataset.Mistral-7B-Instruct-v2.0-PairRM-DPO-Datasetpairrm-llama-preferences-1744946336
PairRM Preference Dataset
Dataset Description
This dataset contains preference pairs created using the PairRM reward model to evaluate responses generated by the Llama-3.2 model.
Dataset Creation Process
50 instructions were extracted from the Lima dataset
5 responses were generated per instruction using the llama-3.2 chat template
PairRM was applied to create preference pairs
Dataset Statistics
Number of instructions: 50
Number of preference… See the full description on the dataset page: https://huggingface.co/datasets/Likhith003/pairrm-llama-preferences-1744946336.lima-qwen2.5-7b-pairrm-preferences
LIMA × Qwen2.5-7B-Instruct × PairRM preference dataset
Preference dataset built for Assignment 4 of the alignment course.
How it was built
Source instructions: 50 instructions sampled with seed=42 from the GAIR/lima training split.
Candidate generation: For each instruction we sampled 5 responses from Qwen/Qwen2.5-7B-Instruct using the official chat template (temperature=0.9, top_p=0.95, max_new_tokens=512).
Ranking: All 5 candidates per instruction were ranked with… See the full description on the dataset page: https://huggingface.co/datasets/Barryzbr12/lima-qwen2.5-7b-pairrm-preferences.assignment4-pairrm-preferences-submit