CoolFace
Modelpublic

Isabelbinu/roberta-large-argumentscheme-fallacy-classifier

sourceHugging Faceupdated 3mo agoView on Hugging Face
0likes5downloads
Model Card

ArgueBot — Argument Scheme & Fallacy Classifier

This model is a fine-tuned version of roberta-large trained on a combined dataset of argument schemes and logical fallacies. It classifies any input text into one of 24 categories — either a valid argument scheme type or a logical fallacy type — in a single inference pass.


Model Details

PropertyValue
Base Modelroberta-large
Task24-class text classification
Argument Scheme Classes11
Fallacy Classes13
Total Classes24

Training Parameters

These are the exact parameters used to train this model:

ParameterValueDescription
Learning Rate2e-5Step size for AdamW optimiser
Batch Size8Samples per training step
Max Token Length512Maximum input tokens per sample
Weight Decay0.01L2 regularisation in AdamW
Max Epochs20Training ceiling — early stopping cuts this short
Gradient Clip1.0Max gradient norm to prevent exploding gradients
Train / Val / Test Split70 / 15 / 15Stratified split
OptimiserAdamWAdaptive learning rate with weight decay

Early Stopping

ParameterValueDescription
Patience3Stops after 3 consecutive epochs with no improvement
Min Delta0.001Minimum change that counts as an improvement
Monitorval_lossWatches validation loss to detect overfitting

The best model checkpoint is saved automatically whenever validation loss improves. When early stopping triggers, the best weights are restored before evaluation.

Data Preprocessing

StepWhat it does
Exact deduplicationRemoves rows where both text and label are identical
Near-duplicate removalRemoves rows where the same text appears with a different label
Short text removalDrops texts with fewer than 5 words
Class weight balancingComputes sklearn.compute_class_weight("balanced") across all 24 classes
ShuffleFixed random_state=42 for reproducibility

Supported Labels

✅ Argument Schemes (valid arguments)

  1. 1.Argument from Analogy
  2. 2.Argument from Alternatives
  3. 3.Argument from Cause to Effect
  4. 4.Argument from Commitment
  5. 5.Argument from Example
  6. 6.Argument from Expert Opinion
  7. 7.Argument from Negative Consequences
  8. 8.Argument from Positive Consequences
  9. 9.Argument from Practical Reasoning
  10. 10.Argument from Sign
  11. 11.Argument from Values

⚡ Fallacy Types

  1. 1.Ad Hominem
  2. 2.Ad Populum
  3. 3.Appeal to Emotion
  4. 4.Circular Reasoning
  5. 5.Equivocation
  6. 6.Fallacy of Credibility
  7. 7.Fallacy of Extension
  8. 8.Fallacy of Logic
  9. 9.Fallacy of Relevance
  10. 10.False Causality
  11. 11.False Dilemma
  12. 12.Faulty Generalization
  13. 13.Intentional

Dataset

SourceTypeClassesSamples
EthiX + MacagnoArgument schemes111829
LOGIC datasetFallacy types134124
  • —Text column: Argument
  • —Label column: Label
  • —Both datasets combined into a single CSV before training

Results

MetricValue
Accuracy0.54
Macro f1-score0.49

How to Load and Use the Model

Step 1 — Install

bash
pip install transformers torch

Step 2 — Load from Hugging Face

python


import json, torch, requests
from transformers import AutoTokenizer, AutoModelForSequenceClassification

MODEL_PATH = "Isabelbinu/roberta-large-argumentscheme-fallacy-classifier"

tokenizer = AutoTokenizer.from_pretrained(MODEL_PATH)
model     = AutoModelForSequenceClassification.from_pretrained(MODEL_PATH)
model.eval()

meta      = requests.get(f"https://huggingface.co/{MODEL_PATH}/resolve/main/metadata.json").json()
label_map = {int(k): v for k, v in meta["label_map"].items()}
scheme_ids = set(meta["scheme_ids"])

def predict(text):
    inputs  = tokenizer(text, return_tensors="pt", truncation=True, max_length=128)
    with torch.no_grad():
        probs = torch.softmax(model(**inputs).logits, dim=-1).squeeze()
    pred_id = int(probs.argmax())
    return {
        "verdict":    "✅ Valid Argument" if pred_id in scheme_ids else "⚡ Fallacy",
        "label":      label_map[pred_id],
        "confidence": f"{probs[pred_id]:.1%}",
    }

# Test 1 
print(predict("Introducing a four-day work week will boost employee wellbeing, reduce burnout, and ultimately increase overall productivity."))


# Test 2
print(predict("Don't trust him — he was caught lying before, so everything he says is wrong."))

Applications

  • —Education: Teach critical thinking by identifying argument types in real texts
  • —Debate Analysis: Evaluate the quality of reasoning in speeches and essays
  • —Fact-checking: Flag logically flawed reasoning in news and social media
  • —Media Literacy: Help readers identify manipulation tactics in persuasive content
  • —AI Assistants: Add argumentation reasoning to conversational AI systems

License

This model is licensed under the MIT License.