giuliadc/cnndm-filtered
CNN/DailyMail dataset filtered by using the code by Aumiller et al. (1) available at https://github.com/dennlinger/summaries/tree/main min_length_summary = 18; min_length_reference = 150; length_metric = "whitespace" bi-gram_overlap_fraction between summary and original text <= 0.630 min_compression_ratio = 2.5 Moreover: - all articles longer than 512 tokens were removed from the dataset. - Each line break ("\n") inside of a highlight was replaced by an empty space (" "). (1):… See the full description on the dataset page: https://huggingface.co/datasets/giuliadc/cnndm-filtered.
CNN/DailyMail dataset filtered by using the code by Aumiller et al. (1) available at https://github.com/dennlinger/summaries/tree/main
minlengthsummary = 18; minlengthreference = 150; length_metric = "whitespace"
bi-gramoverlapfraction between summary and original text <= 0.630
mincompressionratio = 2.5
Moreover:
- all articles longer than 512 tokens were removed from the dataset.
- Each line break ("\n") inside of a highlight was replaced by an empty space (" "). --- (1): Aumiller, D., Fan, J., & Gertz, M. (2023). On the State of German (Abstractive) Text Summarization. arXiv preprint arXiv:2301.07095.