CoolFace
Modelpublic

MahdiAlikhah/albert-base-v2-persian-sentiment-sentipers-3class

sourceHugging Faceapache-2.0updated 2d agoView on Hugging Face
1likes97downloads
Model Card

Model Card for albert-fa-sentipers-3class

<!-- Provide a quick summary of what the model is/does. -->

A 3-class Persian sentiment classifier (negative / neutral / positive), fine-tuned from shekar-ai/albert-base-v2-persian-zwnj-naab-mlm on the DeepSentiPers dataset of Persian product review sentences.

Model Details

Model Description

<!-- Provide a longer summary of what this model is. -->

This model classifies Persian-language text into three sentiment classes: negative, neutral, and positive. It was built by attaching a sequence classification head to a pretrained ALBERT-base Persian MLM checkpoint and fine-tuning it on the DeepSentiPers dataset (Persian customer review sentences from Digikala). The dataset's original 5-point polarity scale (-2 to +2) was collapsed to 3 classes: -2/-1 -> negative, 0 -> neutral, +1/+2 -> positive Training used the back-translation-augmented split (translation.csv) and was evaluated on the dataset's original held-out test.csv split.

  • —Developed by: MahdiAlikhah
  • —Model type: ALBERT-base for sequence classification, 3-class Persian sentiment analysis
  • —Language(s) (NLP): Persian (fa)
  • —License: apache-2.0 (inherited from base model — confirm against shekar-ai/albert-base-v2-persian-zwnj-naab-mlm's own license before relying on this)
  • —Finetuned from model: shekar-ai/albert-base-v2-persian-zwnj-naab-mlm

Model Sources

<!-- Provide the basic links for the model. -->

  • —Repository: https://huggingface.co/MahdiAlikhah/albert-base-v2-persian-sentiment-sentipers

Uses

<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->

Direct Use

<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->

Sentence- or short-passage-level sentiment classification of Persian text into negative, neutral, or positive, particularly for text similar to its training domain (product reviews).

Out-of-Scope Use

<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->

Not evaluated on domains far from product reviews (e.g. social media, news, formal writing); not designed for languages other than Persian or mixed-language text; not suitable as a sole signal for high-stakes decisions without human review.

Bias, Risks, and Limitations

<!-- This section is meant to convey both technical and sociotechnical limitations. -->

Training data skews positive after the 3-class collapse (positive labels outnumber negative by roughly 3-4x), partially addressed with class-weighted loss during training. The dataset is sourced from a single e-commerce platform's product reviews, so sentiment expressed in other registers or dialects may not transfer well. Collapsing the original 5-point scale to 3 classes also removes some intensity distinctions near class boundaries.

Recommendations

<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->

Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. Validate performance on your own domain/data before deploying outside product-review-style Persian text.

How to Get Started with the Model

Use the code below to get started with the model.

python
from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch

model_id = "MahdiAlikhah/albert-base-v2-persian-sentiment-sentipers"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForSequenceClassification.from_pretrained(model_id)

text = "این محصول عالی بود"
inputs = tokenizer(text, return_tensors="pt")
with torch.no_grad():
    logits = model(**inputs).logits
pred = logits.argmax(-1).item()
print(model.config.id2label[pred])

Training Details

Training Data

<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->

DeepSentiPers (https://github.com/JoyeBright/DeepSentiPers) — Persian sentiment corpus of Digikala product review sentences. Training file: translation.csv (back-translation-augmented, approximately 14,046 rows before splitting). Test file: test.csv (1,854 rows, original held-out split, not augmented or seen during training). Original 5-point polarity labels (-2 to +2) collapsed to 3 classes (negative/neutral/positive).

Training Procedure

<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->

Preprocessing

Text normalized with hazm's Normalizer (Persian half-space/ZWNJ normalization), then tokenized with the base model's tokenizer (max_length=128, truncation and padding to max length).

Training Hyperparameters
  • —Training regime: fp32
  • —Epochs: 5 (best checkpoint selected by validation macro-F1; epoch 4 was selected, as validation F1 peaked there before declining in epoch 5)
  • —Batch size: 16 (train), 32 (eval)
  • —Learning rate: 2e-5
  • —Weight decay: 0.01
  • —Loss: class-weighted cross-entropy (balanced class weights) to counteract training-set class imbalance
Speeds, Sizes, Times

<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->

Train runtime: approximately 26 minutes (1,574s) for 5 epochs, 3,955 steps total, on a single T4 GPU (Google Colab).

Evaluation

<!-- This section describes the evaluation protocols and provides the results. -->

Testing Data, Factors & Metrics

Testing Data

<!-- This should link to a Dataset Card if possible. -->

DeepSentiPers test.csv (1,854 rows), 3-class-collapsed, normalized identically to training data. Never included in training or in the back-translation augmentation.

Metrics

<!-- These are the evaluation metrics being used, ideally with a description of why. -->

Accuracy, and macro F1 (chosen as the primary metric because of class imbalance, so the minority negative class isn't washed out by the larger neutral/positive classes).

Results

Accuracy: 0.830. F1 (macro): 0.802. Eval loss: 0.714.

Summary

Roughly 83% accuracy and 0.80 macro-F1 on the held-out test set. For context, prior work on the full, uncollapsed 5-class DeepSentiPers task reported substantially lower macro-F1 (about 0.71 for a BERT-based model), consistent with 3-class sentiment being an easier task than 5-class.

Environmental Impact

<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->

Carbon emissions can be estimated using the Machine Learning Impact calculator presented in Lacoste et al. (2019).

  • —Hardware Type: NVIDIA T4 GPU
  • —Cloud Provider: Google Colab

Technical Specifications

Model Architecture and Objective

ALBERT-base architecture with a sequence classification head (linear layer over the pooled [CLS] output), 3-way softmax output, fine-tuned with cross-entropy loss.

Hardware

Single NVIDIA T4 GPU (Google Colab)

Software

transformers 4.57.3, tokenizers 0.22.2, PyTorch, datasets, hazm, accelerate, evaluate, scikit-learn.