CoolFace
Modelpublic

yassine-mhirsi/debertav3-stance-detection

sourceHugging Facemitupdated 10mo agoView on Hugging Face
0likes28downloads
Model Card

Stance Detection with DeBERTa-v3-large

This model detects whether an argument supports (PRO) or opposes (CON) a given topic.

Model Description

  • —Base Model: microsoft/deberta-v3-large
  • —Task: Binary stance classification (PRO/CON)
  • —Training Data: IBM ArgKP-2023 dataset (~32,000 examples)
  • —Calibration: Label smoothing (0.1) for proper confidence scores

Performance

  • —Test Accuracy: 99.97%
  • —Test F1 Score: 99.97%
  • —Mean Confidence: 93.9% (well-calibrated)
  • —Calibration: ECE < 0.10

Usage

python
from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch

# Load model
model_name = "yassine-mhirsi/debertav3-stance-detection"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForSequenceClassification.from_pretrained(model_name)

# Predict
topic = "AI should replace human teachers"
argument = "Teachers provide emotional support that AI cannot replicate"

text = f"Topic: {{topic}} [SEP] Argument: {{argument}}"
inputs = tokenizer(text, return_tensors="pt", truncation=True, max_length=512)

with torch.no_grad():
    outputs = model(**inputs)
    probs = torch.nn.functional.softmax(outputs.logits, dim=-1)
    predicted_class = torch.argmax(probs, dim=-1).item()

stance = "PRO" if predicted_class == 1 else "CON"
confidence = probs[0][predicted_class].item()

print(f"Stance: {{stance}}")
print(f"Confidence: {{confidence:.2%}}")

Training Details

  • —Epochs: 3
  • —Learning Rate: 3e-6
  • —Batch Size: 4 (with gradient accumulation of 4)
  • —Label Smoothing: 0.1
  • —Training Time: ~1.5 hours on Kaggle GPU

Limitations

  • —Trained only on English argumentative text
  • —Best performance on formal arguments (debate-style)
  • —May struggle with heavy sarcasm or irony
  • —Calibrated for confidence, but not perfect

Citation

If you use this model, please cite:

bibtex
@misc{{stance-detection-deberta,
  author = Yassine Mhirsi,
  title = {{Stance Detection with DeBERTa-v3-large}},
  year = {{2025}},
  publisher = {{Hugging Face}},
  howpublished = {{\\url{{https://huggingface.co/yassine-mhirsi/debertav3-stance-detection}}}}
}}

License

MIT License ---