datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
sst5
Stanford Sentiment Treebank - Fine-Grained
Stanford Sentiment Treebank with 5 labels: very positive, positive, neutral, negative, very negative
Splits are from:
https://github.com/AcademiaSinicaNLPLab/sentiment_dataset/tree/master/data
Training data is on sentence level, not on phrase level!
emotion** Attention: There appears an overlap in train / test. I trained a model on the train set and achieved 100% acc on test set. With the original emotion dataset this is not the case (92.4% acc)**
20_newsgroupsThis is a version of the 20 newsgroups dataset that is provided in Scikit-learn. From the Scikit-learn docs:
The 20 newsgroups dataset comprises around 18000 newsgroups posts on 20 topics split in two subsets: one for training (or development) and the other one for testing (or for performance evaluation). The split between the train and test set is based upon a messages posted before and after a specific date.
We followed the recommended practice to remove headers, signature blocks, and… See the full description on the dataset page: https://huggingface.co/datasets/SetFit/20_newsgroups.sst2
Stanford Sentiment Treebank - Binary
Stanford Sentiment Treebank with 2 labels: negative, positive
Splits are from:
https://github.com/AcademiaSinicaNLPLab/sentiment_dataset/tree/master/data
Training data is on sentence level, not on phrase level!
enron_spamThis is a version of the Enron Spam Email Dataset, containing emails (subject + message) and a label whether it is spam or ham.
ag_newsrte
Glue RTE
This dataset is a port of the official rte dataset on the Hub.
Note that the sentence1 and sentence2 columns have been renamed to text1 and text2 respectively.
Also, the test split is not labeled; the label column values are always -1.
TREC-QC
TREC Question Classification
Question classification in coarse and fine-grained categories.
Source:
Experimental Data for Question Classification
Xin Li, Dan Roth, Learning Question Classifiers. COLING'02, Aug., 2002.
bbc-news
BBC News Topic Dataset
Dataset on BBC News Topic Classification consisting of 2,225 articles published on the BBC News website corresponding during 2004-2005. Each article is labeled under one of 5 categories: business, entertainment, politics, sport or tech.
Original source for this dataset:
Derek Greene, Pádraig Cunningham, “Practical Solutions to the Problem of Diagonal Dominance in Kernel Document Clustering,” in Proc. 23rd International Conference on Machine learning (ICML’06)… See the full description on the dataset page: https://huggingface.co/datasets/SetFit/bbc-news.CR
Customer Reviews
This dataset is a port of the official CR dataset from this paper.
There is no validation split.
amazon_counterfactual_en
Amazon Counterfactual Statements
This dataset is the en-ext split from SetFit/amazon_counterfactual. As the original test set is rather small (1333 examples), a different split was created with 50-50 for training & testing.
The dataset is described in amazon-multilingual-counterfactual-dataset / Paper
It contains statements from Amazon reviews about events that did not or cannot take place.
amazon_reviews_multi_ensubj
Subjective vs Objective
This is the SUBJ dataset as used in SentEval. It contains sentences with an annotation if they sentence describes something subjective about a movie or something objective
SentEval-CR
SentEval Customer Reviews
This dataset is a port of the official SentEval CR dataset from this paper. The test split was created from the by randomly sampling 20% of the original data and the train split is the remaining 80%. there are no official train/test splits of CR.
There is no validation split. This was used in the STraTA paper.
ade_corpus_v2_classification
ADE-Corpus-V2 Dataset: Adverse Drug Reaction Data.
This is a dataset for classification if a sentence is ADE-related (True) or not (False).
Train size: 17,637
Test size: 5,879
Source dataset
Paper
mnli
Glue MNLI
This dataset is a port of the official mnli dataset on the Hub.
It contains the matched version.
Note that the premise and hypothesis columns have been renamed to text1 and text2 respectively.
Also, the test split is not labeled; the label column values are always -1.
qqp
Glue QQP
This dataset is a port of the official qqp dataset on the Hub.
Note that the question1 and question2 columns have been renamed to text1 and text2 respectively.
Also, the test split is not labeled; the label column values are always -1.
amazon_massive_intent_en-USqnli
Glue QNLI
This dataset is a port of the official qnli dataset on the Hub.
Note that the question and sentence columns have been renamed to text1 and text2 respectively.
Also, the test split is not labeled; the label column values are always -1.
mrpc
Glue MRPC
This dataset is a port of the official mrpc dataset on the Hub.
Note that the sentence1 and sentence2 columns have been renamed to text1 and text2 respectively.
Also, the test split is not labeled; the label column values are always -1.
yelp_review_fullimdbgo_emotions
GoEmotions
This dataset is a port of the official go_emotions dataset on the Hub. It only contains the simplified subset as these are the only fields we need for text classification.
stsb
Glue STS-B
This dataset is a port of the official sts-b dataset on the Hub.
This is not a classification task, so the label_text column is only included for consistency
Note that the sentence1 and sentence2 columns have been renamed to text1 and text2 respectively.
Also, the test split is not labeled; the label column values are always -1.
xglue_nc#xglue nc
This dataset is a port of the official ['xglue' dataset] (https://huggingface.co/datasets/xglue) on the Hub. It has just the news category classification section. It has been reduced to just 3 columns (plus text label) that are relevant to the SetFit task. Validation and test in English, Spanish, French, Russian, and German.
hate_speech_offensive
hate_speech_offensive
This dataset is a version from hate_speech_offensive, splitted into train and test set.
toxic_conversations
Toxic Conversation
This is a version of the Jigsaw Unintended Bias in Toxicity Classification dataset. It contains comments from the Civil Comments platform together with annotations if the comment is toxic or not.
10 annotators annotated each example and, as recommended in the task page, set a comment as toxic when target >= 0.5
The dataset is inbalanced, with only about 8% of the comments marked as toxic.
amazon_reviews_multi_ja#amazon reviews multi japanese
This dataset is a port of the official ['amazon_reviews_multi' dataset] (https://huggingface.co/datasets/amazon_reviews_multi) on the Hub. It has just the Japanese language version. It has been reduced to just 3 columns (and 4th "label_text") that are relevant to the SetFit task.
amazon_polaritystudent-question-categoriesThis is the IITJEE NEET AIIMS Students Questions Data dataset.
It categorizes university entry questions into 4 categories: Physics, Chemistry, Biology, and Mathematics.