THEATLAS/PENS

1---2license: apache-2.03task_categories:4- text-generation5- summarization6language:7- en8size_categories:9- 100K<n<1M10---11# Dataset Card for PENS: PErsonalized News headlineS12 13<!-- Provide a quick summary of the dataset. -->14 15PENS is an English dataset for Personalized News Headline Generation Research. It contains two parts for training and test individually. The training set was collected from anonymized user impressions logs of Microsoft News website, and the test set is manually-created by hundreds of native speakers to enable a fair testbed for evaluating models in an offline mode.16 17PENS contains about 113k English news articles whose topics are distributed into 15 categories and 500k impression logs generated by over 445k users for training. In detail, every news article contains rich textual content including title, body, category and corresponding entities. Each impression log contains the click events, non-clicked events and historical news click behaviors of this user before this impression. To provide an offline testbed, we invited 103 English native speakers to manually create a test set by two stages. In detail, there are over 100k personalized news headlines generated.18 19Notice that each user was de-linked from the production system when securely hashed into an anonymized ID to protect user privacy. For more detailed information about the PENS dataset, you can refer to the following paper: [PENS: A Dataset and Generic Framework for Personalized News20Headline Generation](https://www.microsoft.com/en-us/research/uploads/prod/2021/06/ACL2021_PENS_Camera_Ready_1862_Paper.pdf).21 22## Dataset Details23 24### Dataset Description25 26PENS (PErsonalized News headlineS) is an English dataset tailored for Personalized News Headline Generation research. The dataset is divided into training and test sets to support both model development and evaluation. 27 28- **Training Set:** Collected from anonymized user impression logs of the Microsoft News website, containing approximately 113k English news articles across 15 categories and 500k impression logs from over 445k users. Each news article includes a title, body, category, and associated entities. Impression logs detail click events, non-click events, and historical news click behaviors of users prior to each impression.29 30- **Test Set:** Manually created by 103 native English speakers to provide a fair and reliable benchmark for offline model evaluation. This set includes over 100k personalized news headlines generated through a two-stage annotation process.31 32PENS ensures user privacy by de-linking user data from the production system and anonymizing user IDs through secure hashing.33 34![The statistics of news corpus and training set35of the PENS dataset.](https://huggingface.co/datasets/THEATLAS/PENS/resolve/main/Dataset.png "PENS dataset.")36 37- **Curated by:** Microsoft Research38- **Shared by [optional]:** Microsoft Research39- **Language(s) (NLP):** English40- **License:** [Microsoft Research License Terms](#license)41 42### Dataset Sources [optional]43 44- **Repository:** [PENS Dataset Repository](#download)45- **Paper:** [Link to the PENS Paper](#citation)46- **Related Works:** [Available Related Works on PENS](#more-information)47 48## Dataset Structure49 50<!-- This section provides a description of the dataset fields, and additional information about the dataset structure such as criteria used to create the splits, relationships between data points, etc. -->51 52PENS consists of two main components: the training set and the test set.53 54- **Training Set:**55  - **News Articles:** ~113,762 entries56    - **Fields:**57      - `News ID`: Unique identifier for each news article (e.g., N10000).58      - `Category`: One of 15 predefined categories (e.g., sports).59      - `Topic`: Specific topic within the category (e.g., soccer).60      - `Headline`: The title of the news article.61      - `News body`: The full content of the news article.62      - `Title entity`: JSON mapping between phrases in the title and their corresponding entities in WikiData.63      - `Entity content`: Detailed information about each entity from WikiData.64  - **Impression Logs:** 500,000 entries65    - **Fields:**66      - `UserID`: Anonymous identifier for the user (e.g., U335175).67      - `ClicknewsID`: Space-separated list of historically clicked news IDs.68      - `Dwelltime`: Space-separated list of browsing durations for each clicked news.69      - `Exposure_time`: Space-separated list of exposure times for historical clicked news (separated by `#TAB#`).70      - `Pos`: Space-separated list of clicked news IDs in the current impression.71      - `Neg`: Space-separated list of unclicked news IDs in the current impression.72      - `Start`: Start time of the impression (e.g., 7/3/2019 6:43:49 AM).73      - `End`: End time of the impression (e.g., 7/3/2019 7:06:06 AM).74      - `Dwelltime_pos`: Space-separated list of browsing durations for clicked news in the current impression.75 76- **Validation Set:**77  - Structured similarly to the training set, used for validating model performance during training.78 79- **Test Set:**80  - **Personalized Headlines:** >100,000 entries81    - **Fields:**82      - `userid`: Unique identifier for each of the 103 test users (e.g., NT1).83      - `clicknewsID`: Comma-separated list of historically clicked news IDs from the first stage.84      - `posnewID`: Comma-separated list of exhibited news IDs in the second stage.85      - `rewrite_titles`: Space-separated list of manually written personalized headlines (separated by `#TAB#`).86 87The dataset is structured to facilitate the mapping between user behaviors and personalized headline generation.88 89## Uses90 91### Direct Use92 93PENS is intended for research in personalized news headline generation. Suitable use cases include:94 95- Developing models that generate personalized news headlines based on user interaction data.96- Evaluating the effectiveness of personalization algorithms in news dissemination.97- Analyzing user behavior and preferences in the context of news consumption.98 99### Out-of-Scope Use100 101PENS is not suitable for:102 103- Commercial use without explicit permission from Microsoft Research.104- Applications requiring real-time data updates, as the dataset is static.105- Tasks unrelated to text generation or summarization, such as image processing.106 107## Dataset Structure108 109PENS consists of two main components: the training set and the test set.110 111- **Training Set:**112  - **News Articles:** ~113k entries113    - **Fields:** `title`, `body`, `category`, `entities`114  - **Impression Logs:** ~500k entries115    - **Fields:** `click_events`, `non_click_events`, `historical_click_behaviors`116 117- **Test Set:**118  - **Personalized Headlines:** >100k entries119    - **Fields:** `original_article`, `personalized_headline`, `user_id`120 121The dataset is structured to facilitate the mapping between user behaviors and personalized headline generation.122 123## Dataset Creation124 125### Curation Rationale126 127The PENS dataset was created to address the need for comprehensive resources in the domain of personalized news headline generation. By leveraging real user interaction data and manually curated headlines, PENS provides a robust foundation for developing models that can tailor news content to individual preferences.128 129### Source Data130 131#### Data Collection and Processing132 133- **Training Data:** Extracted from anonymized user impression logs of the Microsoft News website. The data includes detailed user interactions with news articles, ensuring a rich source of information for personalization.134  135- **Test Data:** Generated through a two-stage manual annotation process involving 103 native English speakers. This ensures high-quality, personalized headlines for reliable evaluation.136 137#### Who are the source data producers?138 139- **Training Data Producers:** Users of the Microsoft News website whose interactions were anonymized and aggregated.140  141- **Test Data Producers:** 103 native English speakers recruited to generate personalized headlines.142 143### Annotations [optional]144 145#### Annotation process146 147The test set annotations were created in two stages:148 1491. **Initial Generation:** Annotators generated personalized headlines based on provided news articles and user interaction histories.1502. **Validation:** A secondary review ensured consistency and quality of the generated headlines.151 152#### Who are the annotators?153 154The annotators are 103 native English speakers with expertise in journalism and content creation, ensuring that the personalized headlines are both accurate and engaging.155 156#### Personal and Sensitive Information157 158PENS contains anonymized user interaction data. User IDs are securely hashed to prevent re-identification, and no personal, sensitive, or private information is included in the dataset.159 160## Bias, Risks, and Limitations161 162PENS may contain inherent biases present in user interaction data, such as demographic or preference-based biases. Additionally, the dataset is limited to English-language news and may not generalize to other languages or cultural contexts.163 164### Recommendations165 166Users should critically assess the dataset for potential biases and ensure that models trained on PENS are evaluated for fairness and representativeness. It's recommended to complement PENS with additional data sources to mitigate identified biases.167 168## Citation169 170**BibTeX:**171```bibtex172@inproceedings{pens2024,173  title={PENS: PErsonalized News headlineS for Personalized News Headline Generation},174  author={Author Names},175  booktitle={Proceedings of the Conference on Natural Language Processing},176  year={2024},177  organization={Microsoft Research}178}179```180 181**APA:**182Ao, X., Wang, X., Luo, L., Qiao, Y., He, Q., & Xie, X. (2021, August). PENS: A dataset and generic framework for personalized news headline generation. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) (pp. 82-92).183 184## Glossary185 186- **Impression Log:** A record of a user's interaction with news articles, including clicks and non-clicks.187- **Personalized Headline:** A news headline tailored to an individual user's preferences and interaction history.188 189## More Information190 191[Put Your Voice on Stage: Personalized Headline Generation for News Articles](https://dl.acm.org/doi/10.1145/3629168)192 193[Fact-Preserved Personalized News Headline Generation](https://ieeexplore.ieee.org/abstract/document/10415680)194 195## Dataset Card Contact196 197For any inquiries or feedback regarding the PENS dataset, please contact [aoxiang@ict.ac.cn](mailto:aoxiang@ict.ac.cn).198 199## Download200 201The PENS dataset is available for free download for research purposes under the [Microsoft Research License Terms](#license). Please ensure you have read and agree to the license terms before downloading.202 203- [Download Link](https://msnews.github.io/pens.html)204 205# License206 207The PENS dataset is distributed under the Microsoft Research License Terms. Please review the [license agreement](https://github.com/msnews/MIND/blob/master/MSR%20License_Data.pdf) before using the dataset.208