airzipm/sentiment-analysis-muril-v2

Sentiment Analysis v2 — MuRIL (English + Hindi + Hinglish)

A 3-class sentiment classifier (Negative / Neutral / Positive) fine-tuned from `google/muril-base-cased`, which unlike the v1 roberta-base model can natively handle Hindi (Devanagari) and Hinglish (romanized code-mixed Hindi-English) text in addition to English.

Model details

Training data

• —English: IMDB, SST-2 (GLUE), Yelp Polarity, Tweet Eval (sentiment)
• —Hindi / Hinglish: ai4bharat/IndicSentiment, Hindi-English code-mixed tweet datasets

Classes were capped per-label and a class-weighted loss was used during training to reduce the effect of English data outnumbering Hindi/Hinglish data.

Test set results

              precision    recall  f1-score   support

    Negative       0.90      0.85      0.87      3000
     Neutral       0.81      0.89      0.85      2538
    Positive       0.87      0.84      0.86      3000

    accuracy                           0.86      8538
   macro avg       0.86      0.86      0.86      8538
weighted avg       0.86      0.86      0.86      8538

[image] [image]

Usage

from transformers import pipeline

clf = pipeline("text-classification", model="airzipm/sentiment-analysis-muril-v2")
print(clf("ye movie achi hai"))
print(clf("यह फिल्म बहुत अच्छी है"))
print(clf("This was a great experience!"))

Limitations

• —Hindi/Hinglish training data is much smaller than English data (tens of thousands vs. hundreds of thousands of rows) — expect somewhat lower accuracy on Hindi/Hinglish than on English.
• —Code-mixed spelling varies a lot informally (e.g. "acha"/"accha"/"achha") — coverage depends on what appeared in the training tweets.
• —Not evaluated on domains far from reviews/social media (e.g. formal news, legal text).