Modelpublic

dusersad12/SweepBest-TestRepo

sourceHugging Faceapache-2.0updated 1mo agoView on Hugging Face

0likes10downloads

Model Card

SweepBestModel

Overview

SweepBestModel is a RoBERTa-base model fine-tuned for sequence classification through a systematic hyperparameter sweep. We explored learning rates and weight decay values to find the optimal configuration.

Training Configuration

Run	Learning Rate	Weight Decay	Best Checkpoint	Best F1
runlr2e-5wd0.01	2e-5	0.01	\u2014	{RESULT}
runlr5e-5wd0.01	5e-5	0.01	\u2014	{RESULT}
runlr1e-4wd0.01	1e-4	0.01	\u2014	{RESULT}
runlr2e-5wd0.1	2e-5	0.1	\u2014	{RESULT}

Sweep Results

Run	Learning Rate	Weight Decay	Best Eval F1
runlr2e-5wd0.01	2e-5	0.01	0.827
runlr5e-5wd0.01	5e-5	0.01	0.856
runlr1e-4wd0.01	1e-4	0.01	0.793
runlr2e-5wd0.1	2e-5	0.1	0.741

</div>

The best performing configuration used a learning rate of 5e-5 with weight decay 0.01, achieving the highest F1 score across all sweep runs.

Usage

python

from transformers import AutoModelForSequenceClassification, AutoTokenizer

model = AutoModelForSequenceClassification.from_pretrained("SweepBest-TestRepo")
tokenizer = AutoTokenizer.from_pretrained("SweepBest-TestRepo")

License

This model is released under the Apache 2.0 license.