Datasetpublic

saier/unarXive_imrad_clf

Dataset Card for unarXive IMRaD classification Dataset Summary The unarXive IMRaD classification dataset contains 530k paragraphs from computer science papers and the IMRaD section they originate from. The paragraphs are derived from unarXive. The dataset can be used as follows. from datasets import load_dataset imrad_data = load_dataset('saier/unarXive_imrad_clf') imrad_data = imrad_data.class_encode_column('label') # assign target label column imrad_data =… See the full description on the dataset page: https://huggingface.co/datasets/saier/unarXive_imrad_clf.

sourceHugging Facecc-by-sa-4.0updated 3y agoView on Hugging Face

8likes78downloads

Dataset Card

Dataset Card for unarXive IMRaD classification

Dataset Description

—Homepage: https://github.com/IllDepence/unarXive
—Paper: unarXive 2022: All arXiv Publications Pre-Processed for NLP, Including Structured Full-Text and Citation Network

Dataset Summary

The unarXive IMRaD classification dataset contains 530k paragraphs from computer science papers and the IMRaD section they originate from. The paragraphs are derived from unarXive.

The dataset can be used as follows.

from datasets import load_dataset

imrad_data = load_dataset('saier/unarXive_imrad_clf')
imrad_data = imrad_data.class_encode_column('label')  # assign target label column
imrad_data = imrad_data.remove_columns('_id')         # remove sample ID column

Dataset Structure

Data Instances

Each data instance contains the paragraph’s text as well as one of the labels ('i', 'm', 'r', 'd', 'w' — for Introduction, Methods, Results, Discussion and Related Work). An example is shown below.

{'_id': '789f68e7-a1cc-4072-b07d-ecffc3e7ca38',
 'label': 'm',
 'text': 'To link the mentions encoded by BERT to the KGE entities, we define '
         'an entity linking loss as cross-entropy between self-supervised '
         'entity labels and similarities obtained from the linker in KGE '
         'space:\n'
         '\\(\\mathcal {L}_{EL}=\\sum -\\log \\dfrac{\\exp (h_m^{proj}\\cdot '
         '\\textbf {e})}{\\sum _{\\textbf {e}_j\\in \\mathcal {E}} \\exp '
         '(h_m^{proj}\\cdot \\textbf {e}_j)}\\) \n'}

Data Splits

The data is split into training, development, and testing data as follows.

—Training: 520,053 instances
—Development: 5000 instances
—Testing: 5001 instances

Dataset Creation

Source Data

The paragraph texts are extracted from the data set unarXive.

Who are the source language producers?

The paragraphs were written by the authors of the arXiv papers. In file license_info.jsonl author and text licensing information can be found for all samples, An example is shown below.


{'authors': 'Yusuke Sekikawa, Teppei Suzuki',
 'license': 'http://creativecommons.org/licenses/by/4.0/',
 'paper_arxiv_id': '2011.09852',
 'sample_ids': ['cc375518-347c-43d0-bfb2-f88564d66df8',
                '18dc073e-a48e-488e-b34c-e5fc3cb8a4ca',
                '0c2e89b3-d863-4bc2-9e11-8f6c48d867cb',
                'd85e46cf-b11d-49b6-801b-089aa2dd037d',
                '92915cea-17ab-4a98-aad2-417f6cdd53d2',
                'e88cb422-47b7-4f69-9b0b-fbddf8140d98',
                '4f5094a4-0e6e-46ae-a34d-e15ce0b9803c',
                '59003494-096f-4a7c-ad65-342b74eed561',
                '6a99b3f5-217e-4d3d-a770-693483ef8670']}

Annotations

Class labels were automatically determined (see implementation).

Considerations for Using the Data

Discussion and Biases

Because only paragraphs unambiguously assignable to one of the IMRaD classeswere used, a certain selection bias is to be expected in the data.

Other Known Limitations

Depending on authors’ writing styles as well LaTeX processing quirks, paragraphs can vary in length a significantly.

Additional Information

Licensing information

The dataset is released under the Creative Commons Attribution-ShareAlike 4.0.

Citation Information

@inproceedings{Saier2023unarXive,
  author        = {Saier, Tarek and Krause, Johan and F\"{a}rber, Michael},
  title         = {{unarXive 2022: All arXiv Publications Pre-Processed for NLP, Including Structured Full-Text and Citation Network}},
  booktitle     = {Proceedings of the 23rd ACM/IEEE Joint Conference on Digital Libraries},
  year          = {2023},
  series        = {JCDL '23}
}