quality-filtered
DUE-v2-quality-filtered
removed markdown formatting, links, emojis, multiple white spaces
replaced user and channel tags with @user and #channel
deduplicated (not fuzzy only exact)
basic filtering:
word count filter(min_words=3, max_words=200)
mean word length filter(min_mean_word_length=2.0, max_mean_word_length=12.0)
long word filter(max_word_length=200)
whitespace filter (max_white_space_ratio=0.25)
olympiads_math_220k_filtered_single_gen_high_qualityunseen_wildchat_noOOD_fixed_qualityFiltereduplimit-week-1-synthetic-spam-email-data-filtered-qualityfiltered-high-quality-dpo
Filtered High-Quality Instruction-Output Dataset
This dataset contains high-quality (score = 5) instruction-output pairs generated via a reverse instruction generation pipeline using a fine-tuned backward model and evaluated by LLaMA-2-7B-Chat.
Columns
instruction: The generated prompt.
output: The original response.
score: Quality score assigned ( score = 5 retained).
wildchat_data_unused_qualityFiltered_from11000