sdeakin/fine_tuned_bert_emotions_large

finetunedbertemotionslarge

Model summary

• —Base: bert-large-uncased
• —Task: multi-label emotion classification (GoEmotions-level emotions)
• —Fine-tuning: tri-tower setup with contrastive context/label alignment
• —Max length: 256
• —Labels: same 28 GoEmotions emotions (excluding example_very_unclear)

Intended use

• —Classify short texts (social posts, chats) with multiple emotions.
• —Not for medical/mental-health diagnosis; avoid high-stakes use without human review.

Training data

• —GoEmotions dataset
• —Preprocessing: standard HF tokenizer, lowercased, truncation at 256 tokens.

Training procedure

• —Optimizer: AdamW, LR 5e-5 (context head 2e-5), cosine scheduler, warmup 10%.
• —Batch size: 8 (eval 32), epochs: 40 (early stop on valf1micro).
• —Losses: BCE-with-logits for context, InfoNCE contrastive temperature 0.07, context loss weight 1.0.
• —Regularization: dropout 0.1–0.2 (head), label smoothing 0.05.
• —Hardware: NVIDIA GPU (NVIDIA GeForce RTX 5090 (sm_120)).

Evaluation

Replace with your best numbers:

• —Test F1 (micro): 0.53
• —Test F1 (macro): 0.41
• —Precision (micro): 0.47
• —Accuracy: 0.38
• —Thresholding: per-label tuned on validation split.

How to use

from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch

model_name = "sdeakin/fine_tuned_bert_emotions_large"
tok = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForSequenceClassification.from_pretrained(model_name)

text = "I’m excited but a bit nervous about tomorrow!"
enc = tok(text, return_tensors="pt", truncation=True, padding=True)
with torch.no_grad():
    logits = model(**enc).logits
probs = torch.sigmoid(logits)[0]
label_map = model.config.id2label
preds = [(label_map[i], probs[i].item()) for i in range(len(probs))]
print(sorted(preds, key=lambda x: x[1], reverse=True)[:5])