vibhorag101/roberta-base-suicide-prediction-phr-v2

<!-- This model card has been generated automatically according to the information the Trainer had access to. You should probably proofread and complete it, then remove this comment. -->

vibhorag101/roberta-base-suicide-prediction-phr-v2

This model is a fine-tuned version of roberta-base on Suicide Prediction Dataset, sourced from Reddit. It achieves the following results on the evaluation set:

• —Loss: 0.0553
• —Accuracy: 0.9869
• —Recall: 0.9846
• —Precision: 0.9904
• —F1: 0.9875

Model description

This model is a finetune of roberta-base to detect suicidal tendencies in a given text.

Training and evaluation data

• —The dataset is sourced from Reddit and is available on Kaggle.
• —The dataset contains text with binary labels for suicide or non-suicide.
• —The dataset was cleaned minimally, as BERT depends on contextually sensitive information, which can worsely effect its performance.
• —Removed numbers
• —Removed URLs, Emojis, and accented characters.
• —Remove any extra white spaces and any extra spaces after a single space.
• —Removed any consecutive characters repeated more than 3 times.
• —The rows with more than 512 BERT Tokens were removed, as they exceeded BERT's max token.
• —The cleaned dataset can be found here
• —The evaluation set had ~33k samples, while the training set had ~153k samples, i.e., a 70:15:15 (train:test:val) split.

Training procedure

• —The model was trained on an RTXA5000 GPU.

Training hyperparameters

The following hyperparameters were used during training:

• —learning_rate: 2e-05
• —trainbatchsize: 16
• —evalbatchsize: 32
• —seed: 42
• —optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• —lrschedulertype: linear
• —weight_decay=0.1
• —warmup_ratio: 0.06
• —num_epochs: 3
• —eval_steps: 500
• —save_steps: 500
• —Early Stopping:
• —earlystoppingpatience: 5
• —earlystoppingthreshold: 0.001
• —parameter: F1 Score

Training results

Framework versions

• —Transformers 4.38.2
• —Pytorch 2.1.0+cu121
• —Datasets 2.18.0
• —Tokenizers 0.15.0