xtinge/turkish-extractive-summarization-dataset

XTINGE Turkish Extractive Summarization Datasets

This repository hosts three datasets created for advancing Turkish extractive text summarization research: MLSUMTREXT, TES, and XTINGE-SUMTREXT. These datasets are designed to support the development of models capable of generating concise and relevant extractive summaries of Turkish texts.

Below is a Python example showcasing how to download and use these datasets:

from datasets import load_dataset

# Load the MLSUM_TR_EXT dataset
mlsum_tr_ext = load_dataset("xtinge/turkish-extractive-summarization-dataset", "mlsum_tr_ext")
# Load the TES dataset
tes = load_dataset("xtinge/turkish-extractive-summarization-dataset", "tes")
# Load the xtinge-sum_tr_ext dataset
xtinge_sum_tr_ext = load_dataset("xtinge/turkish-extractive-summarization-dataset", "xtinge-sum_tr_ext")

Dataset Details

Dataset Description

The datasets, having a focus on Turkish text summarization, aim to advance research in this area by providing structured, annotated resources for extractive summarization tasks. These datasets are:

1. 1.MLSUM_TR_EXT:
2. 2.Originates as an extension of the Turkish subset from the MLSUM dataset, focusing on extractive summarization.
3. 3.Comprises articles from internethaber.com, with summaries derived from existing headlines for creating contextually rich extractive summaries.
4. 4.Sentences within these articles were selected based on their SBERT Similarity and ROUGE Scores compared to the original summaries, ensuring relevance and conciseness.

1. 1.TES:
2. 2.Represents a unique collection found on Hugging Face tailored for Turkish extractive summarization.
3. 3.Contains a variety of news articles annotated by three distinct annotators, each providing different perspectives and lengths, thus contributing to a rich set of summarization examples.

1. 1.XTINGE-SUM_TR_EXT:
2. 2.Specifically developed to supplement existing resources by providing detailed sentence importance rankings within lengthy Wikipedia documents.
3. 3.Features annotations by three different annotators who meticulously ranked all sentences by importance, contributing to a comprehensive resource for studying extractive summarization.
4. 4.The annotation process considered Inter Annotator Agreement, specifically employing Krippendorff's alpha to ensure consistency and reliability in sentence importance assessments.

• —Language(s) (NLP): Turkish
• —License: [gpl-3.0]

Dataset Structure

Generic Structure Across Datasets

All three datasets share a generic structure tailored for extractive summarization tasks, comprising the following elements:

• —Title: The title of the document or article, serving as a concise representation of the content.
• —Sentences: The body of the text, split into sentences. This segmentation facilitates the identification of individual sentences that contribute to the summary.
• —Annotations: This section includes annotations for selecting summary sentences. It is subdivided into:
• —Indexes: Indices of sentences that have been selected for the summary. This field varies across datasets based on the number of annotators.
• —Ranking: Rankings assigned to sentences based on their perceived importance for the summary. This feature is more prominent in datasets focusing on sentence importance ranking.

{
    'Title': '<title_of_document>',
    'Sentences': ['<sentence_1>', '<sentence_2>', ..., '<sentence_n>'],
    'Annotations': {
        'Indexes': {

            'Annotator1': [<index_of_selected_sentence_1>, ..., <index_of_selected_sentence_m>],
            # If there are more than one annotator
            'Annotator2': [...],
            # etc.
        },
        'Ranking': {
            'Annotator1': [<ranking_of_first_sentence>,<ranking_of_second_sentence>,..., <ranking_of_mth_sentence>],
            # If there are more than one annotator
            'Annotator2': [...],
            # etc.
        }
    }
}

Cite XTINGE Turkish Extractive Summarization Dataset

@inproceedings{xtinge_turkish_extractive,
  title = {Extractive Summarization Data Sets Generated with Measurable Analyses},
  author = {Demir, İrem and Küpçü, Emel and Küpçü, Alptekin},
  booktitle = {Proceedings of the 32nd IEEE Conference on Signal Processing and Communications Applications},
  year = {2024} 
}