vibhorag101/roberta-base-suicide-prediction-phr

roberta-base-suicide-prediction-phr

This model is a fine-tuned version of roberta-base on this dataset sourced from Reddit. It achieves the following results on the evaluation/validation set:

• —Loss: 0.1543
• —Accuracy: 0.9652972367116438
• —Recall: 0.966571403827834
• —Precision: 0.9638169257340242
• —F1: 0.9651921995935487

It achieves the following result on validation partition of this updated dataset

• —Loss: 0.08761
• —Accuracy: 0.97065
• —Recall: 0.96652
• —Precision: 0.97732
• —F1: 0.97189

Model description

This model is a finetune of roberta-base to detect suicidal tendencies in a given text.

Training and evaluation data

• —The dataset is sourced from Reddit.
• —The dataset contains text with binary labels for suicide or non-suicide.
• —The dataset was cleaned, and following steps were applied
• —Converted to lowercase
• —Removed numbers and special characters.
• —Removed URLs, Emojis and accented characters.
• —Removed any word contractions.
• —Remove any extra white spaces and any extra spaces after a single space.
• —Removed any consecutive characters repeated more than 3 times.
• —Tokenised the text, then lemmatized it and then removed the stopwords (excluding not).
• —The cleaned dataset can be found here
• —The evaluation set had ~23000 samples, while the training set had ~186k samples, i.e. a 80:10:10 (train:test:val) split.

Training procedure

• —The model was trained on an RTXA5000 GPU.

Training hyperparameters

The following hyperparameters were used during training:

• —learning_rate: 2e-05
• —trainbatchsize: 16
• —evalbatchsize: 16
• —seed: 42
• —optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• —lrschedulertype: linear
• —num_epochs: 3

Training results

Framework versions

• —Transformers 4.31.0
• —Pytorch 2.1.0+cu121
• —Datasets 2.14.5
• —Tokenizers 0.13.3