MarGPT/xlmr-uantwerp-sentiment

XLM-RoBERTa fine-tuned for context-aware sentiment on UAntwerp social media

A Dutch / English 3-class sentiment classifier trained on six years of public Facebook and Instagram comments to the University of Antwerp. Built as part of the MSc thesis "What do you mean? Context-Aware Sentiment Analysis of Institutional Social Media Comments" (Margot Bloemen, UAntwerp, May 2026; supervised by Luna De Bruyne).

The headline observation: on institutional social media, off-the-shelf commercial tools and traditional ML pipelines miss most of the negative signal (Coosto: 27 % negative recall, TF-IDF baselines: 61 %). This model — xlm-roberta-base fine-tuned with RandomOverSampler on the training split and the parent post supplied as context — recovers 89.1 % of negative comments while reaching 91.5 % accuracy and 89.5 % macro F1 overall. Statistically, the gain from supplying the parent post is significant only after the class imbalance is addressed (McNemar p < 0.001 with oversampling; p = 1.000 without).

Headline metrics

Evaluated on the held-out n=485 test set (dropna + drop_duplicates preprocessing, identical across all four XLM-R configurations so they are directly comparable in McNemar pairs).

Comparison with the rest of the field tested

⭐ = best on all three headline metrics simultaneously, with no API dependency.

How to use

Quick prediction

from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch

tokenizer = AutoTokenizer.from_pretrained("MarGPT/xlmr-uantwerp-sentiment")
model = AutoModelForSequenceClassification.from_pretrained("MarGPT/xlmr-uantwerp-sentiment")
model.eval()

comment = "Heel mooi initiatief!"
post    = "Universiteit Antwerpen lanceert nieuwe summer school voor AI ethics."

# Comment as the first sentence, parent post as the second
inputs = tokenizer(comment, post, return_tensors="pt", truncation=True, max_length=256)
with torch.no_grad():
    logits = model(**inputs).logits
pred_id = int(torch.argmax(logits, dim=-1))
print(model.config.id2label[pred_id])      # negative | neutral | positive

With a pipeline

from transformers import pipeline
clf = pipeline("text-classification", model="MarGPT/xlmr-uantwerp-sentiment")
clf({"text": "Heel mooi initiatief!", "text_pair": "Universiteit Antwerpen lanceert nieuwe summer school voor AI ethics."})

text_pair is the parent post; omit it for a comment-only ("standard") inference but expect lower negative recall on context-dependent cases.

Training data

• —Source: UAntwerp Facebook (≈75 %) and Instagram (≈25 %), public posts and comments collected January 2020 – February 2026.
• —Cleaning: 3,063 raw comments → 2,684 after filtering skip (n=339) and spam (n=40) labels; passed through deduce for Dutch de-identification (names, emails, phones, addresses replaced by category tokens).
• —Languages: Dutch (majority), English, Vlaams tussentaal.
• —Class distribution: 58.3 % positive / 31.3 % neutral / 10.5 % negative — heavy imbalance addressed via RandomOverSampler on the training split only.
• —Splits: 80 / 20 train / test, stratified on label, seed 42.
• —Inter-annotator agreement (200-comment dual-annotated subset): Cohen's κ = 0.44 (moderate). Negative labels were identical between annotators; disagreement concentrates on the positive ↔ neutral boundary.

The annotated dataset is not redistributed here — it is shared on request under a data-use agreement.

Training procedure