Barryzbr12/lima-qwen2.5-7b-pairrm-preferences

LIMA × Qwen2.5-7B-Instruct × PairRM preference dataset Preference dataset built for Assignment 4 of the alignment course. How it was built Source instructions: 50 instructions sampled with seed=42 from the GAIR/lima training split. Candidate generation: For each instruction we sampled 5 responses from Qwen/Qwen2.5-7B-Instruct using the official chat template (temperature=0.9, top_p=0.95, max_new_tokens=512). Ranking: All 5 candidates per instruction were ranked… See the full description on the dataset page: https://huggingface.co/datasets/Barryzbr12/lima-qwen2.5-7b-pairrm-preferences.