Mostafa911/egyptian-arabic-sentiment-marbertv2

Egyptian Arabic Sentiment (fine-tuned MARBERTv2)

Three-class sentiment classifier (negative / neutral / positive) for Egyptian-dialect tweets. It is UBC-NLP/MARBERTv2 fine-tuned on the Egyptian subset of ArSarcasm.

Live demo: https://huggingface.co/spaces/Mostafa911/egyptian-arabic-sentiment-demo

Results on the Egyptian test set (479 tweets)

By subset (best model):

Sarcastic tweets are about 88% negative in the training data, so the neutral and positive classes inside that subset are tiny and its macro F1 is noisy. Accuracy on the sarcastic subset was about 77%.

Training

• —Data: Egyptian-dialect tweets from the ArSarcasm training split (1,904 tweets before removing duplicates), 10% held out for validation.
• —Light cleaning only: URLs and mentions removed, hashtag symbols removed.
• —Learning rate 2e-5, batch size 16, 5 epochs, max length 64, weighted cross-entropy for class imbalance, best epoch chosen by validation macro F1.
• —Three random seeds; the released model is the seed with the best validation macro F1.
• —An extra experiment training on all dialects gave 0.737 +/- 0.011 on the same Egyptian test set, so it did not clearly help.

Limitations

• —Small training set and a small test set, so scores move by a point or two between runs.
• —About 60% of the errors are confusions between neutral and negative.
• —Sarcasm is often missed, for example literal praise that is meant mockingly.
• —Some labels in the source dataset are ambiguous or noisy.
• —Not for decisions that affect people. Check the licenses of the base model and the dataset before any commercial use.

Usage

from transformers import pipeline

clf = pipeline("text-classification", model="Mostafa911/egyptian-arabic-sentiment-marbertv2")
print(clf("الخدمة وحشة جدا ومش هرجع تاني"))