lopentu/Chinese-Wordnet-SemCor

Chinese Wordnet SemCor

Dataset Summary

This dataset is designed for the task of Word Sense Disambiguation (WSD) for common Chinese words, specifically focusing on words identified as "difficult" (having more than 10 senses) within Chinese Wordnet (CWN) 2.0. It originates from the annotation dataset described in Section 3.1 of the paper "Resolving Regular Polysemy in Named Entities."

The original dataset consisted of 28,836 example sentences where a target "difficult" word was manually annotated with its correct sense from CWN 2.0. The sentences were primarily sourced from the Academia Sinica Balanced Corpus.

Important Note: This version of the dataset has been "flattened" as described at the beginning of Section 4.1.2 in the paper. For each original example sentence containing a target word with N possible senses in CWN, this dataset contains N rows. Each row pairs the original context sentence with one of the N candidate senses (represented by its definition and an example sentence), along with a label indicating whether that sense is the correct one for the given context.

Languages

• —Mandarin Chinese (zh-TW) using Traditional Chinese characters.

Dataset Structure

Data Instances

A typical instance in the dataset:

{
  "test_word": "死",
  "test_pos": "VH",
  "test_sense_id": "05035202",
  "test_definition": "形容生物失去生命。",
  "test_sentence": "還不如<死>了算了。",
  "cwn_sense_id": "05035201",
  "cwn_definition": "形容沒有生命的。",
  "cwn_sentence": "噴過殺蟲劑，以入口處<死>蟑螂陳屍最多。",
  "label": false
}

Data Fields

• —test_word: (string) The target ambiguous word from the original test sentence.
• —test_pos: (string) The part-of-speech tag of the target word in the original test sentence.
• —test_sense_id: (string) The CWN sense ID of the correct sense for the target word in the original test sentence.
• —test_definition: (string) The CWN definition corresponding to the test_sense_id (the correct sense).
• —test_sentence: (string) The original context sentence containing the target word (e.g., "還不如<死>了算了。").
• —cwn_sense_id: (string) The CWN sense ID for the candidate sense being considered in this row.
• —cwn_definition: (string) The CWN definition corresponding to the cwn_sense_id (the candidate sense).
• —cwn_sentence: (string) An example sentence from CWN illustrating the usage of the candidate sense (cwn_sense_id).
• —label: (boolean) The prediction target. True if cwn_sense_id is the correct sense for the test_word in the test_sentence (i.e., if cwn_sense_id == test_sense_id), False otherwise.

Data Splits

The dataset has the following splits:

Dataset Creation

Curation Rationale

The dataset was created to provide training and evaluation data for WSD on polysemous common words in Chinese, particularly focusing on those with a high degree of ambiguity (more than 10 senses), for which existing resources might lack sufficient contextual examples.

Source Data

• —Chinese Wordnet (CWN) 2.0: Provided the sense inventory (IDs, definitions, example sentences)
• —Academia Sinica Balanced Corpus: Provided the source sentences for manual annotation

Annotations

The original 28,836 examples were manually annotated by six native Mandarin speakers with linguistics backgrounds. Annotators assigned the most appropriate CWN 2.0 sense (test_sense_id) to the target word (test_word) in each sentence (test_sentence). The flattening process and label generation (label) were performed subsequently based on these annotations.

Considerations for Using the Data

• —Flattened Structure: Users should be aware that the dataset is flattened. Each original sentence appears multiple times, paired with different candidate senses. Models are typically trained to identify the single row corresponding to the correct sense (label=True) for a given original sentence context.
• —Focus on "Difficult" Words: The dataset primarily covers 113 specific words selected for their high polysemy in CWN.
• —CWN Version: The sense inventory is based on CWN 2.0.

Licensing Information

MIT

Citation Information

If you use this dataset, please cite the original paper:

@misc{hsieh2024resolvingregularpolysemynamed,
      title={Resolving Regular Polysemy in Named Entities}, 
      author={Shu-Kai Hsieh and Yu-Hsiang Tseng and Hsin-Yu Chou and Ching-Wen Yang and Yu-Yun Chang},
      year={2024},
      eprint={2401.09758},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2401.09758}, 
}