Arsive/toxicity_classification_jigsaw

Dataset info

Training Dataset:

You are provided with a large number of Wikipedia comments which have been labeled by human raters for toxic behavior. The types of toxicity are:

• —toxic
• —severe_toxic
• —obscene
• —threat
• —insult
• —identity_hate

The original dataset can be found here: jigsaw_toxic_classification

Our training dataset is a sampled version from the original dataset, <b>containing equal number of samples for both clean and toxic classes. </b><br>

Dataset creation:

<code><pre>data = pd.readcsv('train.csv') # train.csv from the original dataset columnnames = ['toxic', 'severetoxic', 'obscene', 'threat', 'insult', 'identityhate'] columnlabels = data[columnnames][2:-1] traintoxic = data[data[columnnames].sum(axis=1) > 0] trainclean = data[data[columnnames].sum(axis=1) == 0] traincleansampled = trainclean.sample(n=16225, randomstate=42)

dataframe = pd.concat([traintoxic, trainclean_sampled], axis=0)

dataframe = dataframe.sample(frac=1, randomstate=42) dataset = Dataset.frompandas(dataframe)

traindataset = dataset.traintestsplit(testsize=0.2)['train'] valdataset = dataset.traintestsplit(testsize=0.2)['test']</pre></code>

Caution:

This dataset contains comments that are toxic in nature. Kindly use appropriately.

Citation

<pre> @misc{jigsaw-toxic-comment-classification-challenge, author = {cjadams, Jeffrey Sorensen, Julia Elliott, Lucas Dixon, Mark McDonald, nithum, Will Cukierski}, title = {Toxic Comment Classification Challenge}, publisher = {Kaggle}, year = {2017}, url = {https://kaggle.com/competitions/jigsaw-toxic-comment-classification-challenge} }</pre>