zavavan/scierc_aeco

Dataset Card for SciERC AECO dataset

Table of Contents

• —Dataset Description
• —Dataset Summary
• —Supported Tasks and Leaderboards
• —Languages
• —Dataset Structure
• —Data Instances
• —Data Fields
• —Data Splits
• —Dataset Creation
• —Curation Rationale
• —Source Data
• —Annotations
• —Personal and Sensitive Information
• —Considerations for Using the Data
• —Social Impact of Dataset
• —Discussion of Biases
• —Other Known Limitations
• —Additional Information
• —Dataset Curators
• —Licensing Information
• —Citation Information
• —Contributions

Dataset Description

• —Homepage:
• —Paper: A Few-Shot Approach for Relation Extraction Domain Adaptation using Large Language Models
• —Point of Contact: Vanni Zavarella

Dataset Summary

The SciERC AECO dataset is an English-language dataset containing 1016 sentences from research papers in the AECO domain, annotated for scientific entities and relations based on the SciERC annotation schema.

Supported Tasks and Leaderboards

• —'NER': the dataset can be used to train a model to detect scientific entities according to the SciERC annotation schema
• —'Relation extraction': the dataset can be used to train a model to detect binary relations between pairs of scientific entities according to the SciERC annotation schema.

Languages

English (EN)

Dataset Structure

Data Instances

Each row in the dataset contains:

• —a "sentence_text" string valued attribute
• —an (optionally empty) "Tasks" attribute containing annotated entities of type TASK, in the form of a string dictionary "Ti":"entitystring" where Ti is an index of the entity and "entitystring" the string match of the entity in the sentence_text
• —an (optionally empty) "Methods" attribute containing annotated entities of type METHOD, in the form of a string dictionary "Ti":"entitystring" where Ti is an index of the entity and "entitystring" the string match of the entity in the sentence_text
• —an (optionally empty) "Metrics" attribute containing annotated entities of type METRIC, in the form of a string dictionary "Ti":"entitystring" where Ti is an index of the entity and "entitystring" the string match of the entity in the sentence_text
• —an (optionally empty) "USED-FOR" attribute containing annotated relations in the form of a list of dictionaries "Ti":"Tj" where Ti is the index of the subject entity and Tj is the index of the object entity
• —an (optionally empty) "EVALUATE-FOR" attribute containing annotated relations in the form of a list of dictionaries "Ti":"Tj" where Ti is the index of the subject entity and Tj is the index of the object entity
• —a boolean attribute "relevant" marking if any of the "Tasks","Methods" or "Metrics" attributes is non-emtpy ("relevant"=True), meaning that the sentence contains True positive examples of SciERC entity and/or relations

Data Fields

Each row in the dataset contains:

• —a "sentence_text" string valued attribute
• —an (optionally empty) "Tasks" attribute containing annotated entities of type TASK, in the form of a string dictionary "Ti":"entitystring" where Ti is an index of the entity and "entitystring" the string match of the entity in the sentence_text
• —an (optionally empty) "Methods" attribute containing annotated entities of type METHOD, in the form of a string dictionary "Ti":"entitystring" where Ti is an index of the entity and "entitystring" the string match of the entity in the sentence_text
• —an (optionally empty) "Metrics" attribute containing annotated entities of type METRIC, in the form of a string dictionary "Ti":"entitystring" where Ti is an index of the entity and "entitystring" the string match of the entity in the sentence_text
• —an (optionally empty) "USED-FOR" attribute containing annotated relations in the form of a list of dictionaries "Ti":"Tj" where Ti is the index of the subject entity and Tj is the index of the object entity
• —an (optionally empty) "EVALUATE-FOR" attribute containing annotated relations in the form of a list of dictionaries "Ti":"Tj" where Ti is the index of the subject entity and Tj is the index of the object entity
• —a boolean attribute "relevant" marking if any of the "Tasks","Methods" or "Metrics" attributes is non-emtpy ("relevant"=True), meaning that the sentence contains True positive examples of SciERC entity and/or relations

Dataset Creation

Source Data

Initial Data Collection

The source data comprise titles and abstracts from a collection of research articles in the AECO area published in the time range 2010-2023, retrieved from the OpenAlex2 open scientific graph database (https://docs.openalex.org) using a set of platform-specific topic filtering tags.

Who are the annotators?

Vanni Zavarella Juan Carlos Gamero Salinas

Personal and Sensitive Information

No personal/sensitive information is included.

Considerations for Using the Data

Licensing Information

The SciERC AECO dataset is released under the [cc-by-nc-2.0].

Citation Information

@misc{zavarella2024fewshotapproachrelationextraction,
      title={A Few-Shot Approach for Relation Extraction Domain Adaptation using Large Language Models}, 
      author={Vanni Zavarella and Juan Carlos Gamero-Salinas and Sergio Consoli},
      year={2024},
      eprint={2408.02377},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2408.02377}, 
}

@InProceedings{luan2018multitask,
     author = {Luan, Yi and He, Luheng and Ostendorf, Mari and Hajishirzi, Hannaneh},
     title = {Multi-Task Identification of Entities, Relations, and Coreference for Scientific Knowledge Graph Construction},
     booktitle = {Proc.\ Conf. Empirical Methods Natural Language Process. (EMNLP)},
     year = {2018},
}