bluesky333/chemical_language_understanding_benchmark

Table of Contents

• —Benchmark Summary
• —Languages
• —Dataset Structure
• —Data Instances
• —Data Fields
• —Data Splits
• —Dataset Creation
• —Curation Rationale
• —Source Data
• —Licensing Information
• —Citation Information

<p><h1>🧪🔋 Chemical Language Understanding Benchmark 🛢️🧴</h1></p>

<a name="benchmark-summary"></a>

Benchmark Summary

Chemistry Language Understanding Benchmark is published in ACL2023 industry track to facilitate NLP research in chemical industry ACL2023 Industry Track. From our understanding, it is one of the first benchmark datasets with tasks for both patent and literature articles provided by the industrial organization. All the datasets are annotated by professional chemists.

<a name="languages"></a>

Languages

The language of this benchmark is English.

<a name="dataset-structure"></a>

Data Structure

Benchmark has 4 datasets: 2 for text classification and 2 for token classification. | Dataset | Task | # Examples | Avg. Token Length | # Classes / Entity Groups | | ----- | ------ | ---------- | ------------ | ------------------------- | | PETROCHEMICAL | Patent Area Classification | 2,775 | 448.19 | 7 | | RHEOLOGY | Sentence Classification | 2,017 | 55.03 | 5 | | CATALYST | Catalyst Entity Recognition | 4,663 | 42.07 | 5 | | BATTERY | Battery Entity Recognition | 3,750 | 40.73 | 3 |

You can refer to the paper for detailed description of the datasets.

<a name="data-instances"></a>

Data Instances

Each example is a paragraph/setence of an academic paper or patent with annotations in a json format.

<a name="data-fields"></a>

Data Fields

The fields for the text classification task are: 1) 'id', a unique numbered identifier sequentially assigned. 2) 'sentence', the input text. 3) 'label', the class for the text.

The fields for the token classification task are: 1) 'id', a unique numbered identifier sequentially assigned. 2) 'tokens', the input text tokenized by BPE tokenizer. 3) 'ner_tags', the entity label for the tokens.

<a name="data-splits"></a>

Data Splits

The data is split into 80 (train) / 20 (development).

<a name="dataset-creation"></a>

Dataset Creation

<a name="curation-rationale"></a>

Curation Rationale

The dataset was created to provide a benchmark in chemical language model for researchers and developers.

<a name="source-data"></a>

Source Data

The dataset consists of open-access chemistry publications and patents annotated by professional chemists.

<a name="licensing-information"></a>

Licensing Information

The manual annotations created for CLUB are licensed under a Creative Commons Attribution 4.0 International License (CC-BY-4.0).