word-embeddings
cis5300-word-embeddings
Word Embeddings and Semantic Similarity (CIS 5300)
Dataset Description
This dataset supports learning about word embeddings — dense vector representations that capture word meaning. It includes a standard similarity benchmark, a word sense disambiguation task, and a Shakespeare corpus for training custom embeddings.
Configs
SimLex-999: Word Similarity Benchmark
SimLex-999 (Hill et al., 2015) is a gold-standard benchmark for evaluating word… See the full description on the dataset page: https://huggingface.co/datasets/CCB/cis5300-word-embeddings.ai-residency-word-embeddings-eval
Intrinsic evaluation sets for word embeddings
The three standard intrinsic benchmarks for English word vectors, mirrored here so
the ai-residency word-embeddings service does not depend on a third party's host
staying up. Each is byte-identical to the source named below.
file
pairs / questions
what it measures
wordsim353.txt
353 pairs
relatedness; the usual headline number
simlex999.txt
999 pairs
similarity specifically, penalising mere relatedness… See the full description on the dataset page: https://huggingface.co/datasets/roshbeed/ai-residency-word-embeddings-eval.Interpretable_word_embeddings_large_cskgThese embeddings result from applying SemAxis (https://arxiv.org/abs/1806.05521) to common sense knowledge graph embeddings (https://arxiv.org/abs/2012.11490).
interpretable_word_embeddingsThese word embeddings were computed using the POLAR technique to reproject 'common' word embeddings into roundabout 700 interpretable dimensions of polar opposites (i.e. good/bad).
I just used their scripts here:
https://github.com/Sandipan99/POLAR
I applied those on the wikidata5m embeddings, 5 million knowledge graph embeddings (SimplE).
https://graphvite.io/docs/latest/pretrained_model.html
As the model became too huge, I further filtered it for overlap with fasttext embedding tokens.… See the full description on the dataset page: https://huggingface.co/datasets/KnutJaegersberg/interpretable_word_embeddings.word-embeddings-datasetword_embeddingsThis dataset contains the embeddings of 8000 words pulled from the wonderwords package (https://pypi.org/project/wonderwords/).
Embedding model: "text-embedding-3-large" from OpenAI
The word embeddings are stored as a dataframe; to read data run:
df_word_embeds = pd.read_pickle("8000words.pkl")
The words used in the embeddings are stored in word_list as a json; to read data run:
with open("word_list", "r") as f:
... word_list = json.load(f)
Indexes of word_list match indexes of… See the full description on the dataset page: https://huggingface.co/datasets/efarrall/word_embeddings.