CCB/cis5300-word-embeddings

Word Embeddings and Semantic Similarity (CIS 5300)

Dataset Description

This dataset supports learning about word embeddings — dense vector representations that capture word meaning. It includes a standard similarity benchmark, a word sense disambiguation task, and a Shakespeare corpus for training custom embeddings.

Configs

SimLex-999: Word Similarity Benchmark

SimLex-999 (Hill et al., 2015) is a gold-standard benchmark for evaluating word embeddings. Unlike WordSim-353 (which conflates similarity with relatedness), SimLex-999 specifically measures semantic similarity.

from datasets import load_dataset

simlex = load_dataset("CCB/cis5300-word-embeddings", "simlex999")
print(simlex["test"][0])
# {'word1': 'old', 'word2': 'new', 'pos': 'A', 'similarity': 1.58, ...}

Word Sense Clustering

Polysemous words (words with multiple meanings) and their senses, for evaluating whether embeddings can distinguish word senses.

clustering = load_dataset("CCB/cis5300-word-embeddings", "clustering")
print(clustering["validation"][0])
# {'word': 'bank', 'num_senses': 2, 'senses': 'financial institution::0\tsavings account::0\t...'}

Each entry contains a polysemous word, the number of distinct senses, and paraphrases labeled by sense cluster.

Supplementary Files

from huggingface_hub import hf_hub_download

# Download a Shakespeare play
path = hf_hub_download("CCB/cis5300-word-embeddings", "shakespeare/hamlet.txt", repo_type="dataset")

Intended Use

This dataset is used for Homework 4 in CIS 5300: Natural Language Processing at the University of Pennsylvania. Students:

1. 1.Train Word2Vec embeddings on Shakespeare using gensim
2. 2.Explore word analogies and vector arithmetic
3. 3.Evaluate embeddings on SimLex-999 (correlation with human similarity judgments)
4. 4.Cluster polysemous word senses using co-occurrence vectors
5. 5.Investigate bias in word embeddings

Citation

@article{hill2015simlex,
  title={SimLex-999: Evaluating Semantic Models With (Genuine) Similarity Estimation},
  author={Hill, Felix and Reichart, Roi and Korhonen, Anna},
  journal={Computational Linguistics},
  volume={41},
  number={4},
  pages={665--695},
  year={2015}
}