monsifnadir/DarijaBERT-For-Sentiment-Analysis

DarijaBERT Fine-Tuned for Sentiment Analysis 🇲🇦🧠

This sentiment analysis model is based on DarijaBERT, a language model pretrained on Moroccan Arabic (Darija) text. The model has been fine-tuned to classify Moroccan Arabic tweets and public comments into three sentiment categories:

• —Positive (2)
• —Neutral (0)
• —Negative (1)

🛠 Model Architecture

The base DarijaBERT architecture was extended with:

• —Two fully connected layers of 1024 neurons each
• —Dropout layer (p=0.3) to enhance generalization
• —Final classification layer with 3 output neurons (one for each sentiment class)

🧠 Pretraining Details

• —Dataset: 17,441 Moroccan tweets
• —9,894 positive tweets (56.73%)
• —4,039 neutral tweets (23.16%)
• —3,508 negative tweets (20.11%)
• —Training Framework: Hugging Face Trainer API
• —Hyperparameters:
• —Learning rate: 1e-5
• —Batch size: 16 (with gradient accumulation = 32)
• —Weight decay: 0.01
• —EarlyStoppingCallback: Training stopped automatically at 92% accuracy
• —Epochs: Up to 20
• —Evaluation Strategy: Evaluated after every epoch, best model saved

Performance:

• —Accuracy: 87%
• —F1 Score: 87%
• —Precision: 88%
• —Cohen's Kappa: 0.80

🔥 Fine-Tuning Details (Cash Transfer Public Policy 2023)

• —Dataset: 1,344 Moroccan comments from YouTube and Hespress
• —515 neutral
• —505 negative
• —324 positive
• —Split: 80% training / 20% testing
• —Hyperparameters:
• —Learning rate: 5e-6
• —Batch size: 32
• —Maximum sequence length: 256 tokens
• —Warmup ratio: 0.1
• —Early stopping enabled
• —Class weights adjusted for imbalance

Performance:

• —Accuracy: 91.6%
• —Precision: 0.916
• —Recall: 0.916
• —F1 Score: 0.916
• —Cohen’s Kappa: 0.872

📥 How to Use the Model

from transformers import AutoTokenizer, AutoModelForSequenceClassification

model = AutoModelForSequenceClassification.from_pretrained("monsifnadir/DarijaBERT-For-Sentiment-Analysis")
tokenizer = AutoTokenizer.from_pretrained("monsifnadir/DarijaBERT-For-Sentiment-Analysis")

text = "فرحت بزاف اليوم الحمد لله"
inputs = tokenizer(text, return_tensors="pt", truncation=True)
outputs = model(**inputs)
predicted_class = outputs.logits.argmax(dim=-1).item()

# Map prediction to label
label_map = {0: "Neutral", 1: "Negative", 2: "Positive"}
print("Predicted Sentiment:", label_map[predicted_class])