seangogo/processed_tldr_comparison_dataset_20251102_065554

TL;DR Comparison Dataset for OpenAI's Summarize from Feedback task

The dataset is generated from https://huggingface.co/datasets/openai/summarizefromfeedback. Please refer to https://github.com/liyuan24/dgxsparksummaryfromhuman_feedback/tree/main?tab=readme-ov-file#download-the-comparison-dataset about how to download the dataset.

This is a comparison dataset used for training a reward model. Each example contains a query (post) and two responses (chosen and rejected) where the chosen response is preferred by human feedback.

Columns

These columns are added by this preprocessing script:

• —query: length-limited query for summarization. The main text (title + subreddit + post) is preprocessed to have exactly 512 tokens; if the main text is too long, it truncates at the last . If it's too short, it pads the main text. Padding uses the [PAD] token.
• —query_tokens: tokenized version of query as a list of token IDs (length: 512)
• —query_token_length_without_padding: length of query tokens without padding (used to mask query tokens during loss calculation)

• —chosen_response: the preferred response summary with a trailing <|endoftext|> token
• —chosen_response_tokens: tokenized version of chosen_response, padded/truncated to max_rm_response_length tokens
• —rejected_response: the less preferred response summary with a trailing <|endoftext|> token
• —rejected_response_tokens: tokenized version of rejected_response, padded/truncated to max_rm_response_length tokens

• —query_and_chosen_response: the concatenation of query and chosen_response
• —query_and_chosen_response_tokens: tokenized version of query_and_chosen_response, padded/truncated to max_rm_query_response_length tokens
• —query_and_rejected_response: the concatenation of query and rejected_response
• —query_and_rejected_response_tokens: tokenized version of query_and_rejected_response, padded/truncated to max_rm_query_response_length tokens

Dataset Configuration

{'dataset_preprocessing_params': DatasetPreprocessingParams(max_rm_response_length=133,
                                                            max_rm_query_response_length=645,
                                                            query_length=512,
                                                            query_format_str='SUBREDDIT: '
                                                                             'r/{subreddit}\n'
                                                                             '\n'
                                                                             'TITLE: '
                                                                             '{title}\n'
                                                                             '\n'
                                                                             'POST: '
                                                                             '{post}\n'
                                                                             '\n'
                                                                             'TL;DR:',
                                                            query_truncation_field='post',
                                                            query_truncation_text='\n',
                                                            query_padding_side='left'),
 'debug': False,
 'model_name': 'Qwen/Qwen2.5-0.5B',
 'output_dataset_name': 'processed_tldr_comparison_dataset',
 'padding_token': '[PAD]',
 'source_dataset_name': 'openai/summarize_from_feedback'}