Pujaniitj/MLOPS_GROUP_PROJECT

mlops-group-sentiment

A distilbert-base-uncased model fine-tuned on the IMDB movie reviews dataset for binary sentiment classification (positive / negative).

This model is the final artifact of an MLOps group project at IIT Jodhpur (Course CSL7040), demonstrating an end-to-end production ML pipeline: version control on GitHub, GPU training on Kaggle, experiment tracking on Weights & Biases, container packaging via Docker, and deployment to the Hugging Face Hub.

How to Use

from transformers import pipeline

classifier = pipeline("sentiment-analysis", model="pujaniitj/mlops-group-sentiment")
result = classifier("This movie was fantastic!")
print(result)
# [{'label': 'positive', 'score': 0.9876}]

Intended Use

Primary use case: Classifying English-language movie reviews as positive or negative sentiment.

Out-of-scope uses:

• —Non-English text (model only trained on English IMDB reviews)
• —Domain shift — e.g. tweets, product reviews, news articles, customer support transcripts. Performance will degrade outside the movie-review domain.
• —Fine-grained sentiment (beyond binary pos/neg, e.g. 5-star ratings)
• —High-stakes decisions or content moderation without human review

Model Description

• —Base architecture: DistilBERT (distilbert-base-uncased)
• —Distinct from base: Fine-tuned classification head (2 output labels)
• —Parameters: ~66 million
• —Tokenizer: WordPiece (DistilBERT default)
• —Max sequence length: 256 tokens
• —Labels: 0 → negative, 1 → positive

Training Data

• —Dataset: IMDB Movie Reviews
• —Train size: 25,000 reviews (12,500 positive + 12,500 negative — perfectly balanced)
• —Test size: 25,000 reviews (same balance)
• —Train/Validation split: 90/10 of the train set, with seed=42

Training Procedure

Hyperparameters

Training Environment

• —Platform: Kaggle Notebook
• —Hardware: 2× NVIDIA Tesla T4 GPU
• —Training time: ~17 minutes

Experiment Tracking

Two configurations were trained and compared via Weights & Biases:

Replace these values with the exact decimals from your W&B run summary before publishing the final model card.

Why v1 was selected: While v2 achieved a marginally higher F1 (~0.5%), it showed clear signs of overfitting — its eval loss climbed sharply across epochs while v1's remained more stable. v1 also delivers ~25% faster inference, making it the better choice for a production deployment.

Evaluation Results

Evaluation on the held-out IMDB test set (25,000 reviews):

Limitations and Biases

• —Domain: Only trained on movie reviews. Expect degraded performance on other domains.
• —Length: Inputs are truncated to 256 tokens (~200 words). Longer reviews may lose tail information that matters for sentiment.
• —Language: English only.
• —Demographic biases: IMDB reviewers historically skew toward certain demographics (e.g., predominantly male, English-speaking). The model may inherit these biases — e.g., it may misclassify reviews using vernacular or cultural references underrepresented in IMDB.
• —Sarcasm and irony: Like most BERT-based classifiers, the model can struggle with sarcastic or ironic text where the surface sentiment opposes the intended meaning.

Project Resources

• —GitHub repository: https://github.com/pujaniitj/mlops-group-project-iitj
• —W&B experiment dashboard: https://wandb.ai/pujaniitj-iit-jodpur/MLopsgroup8
• —Training notebook (v1): https://www.kaggle.com/code/pujaniitj/mlops-group-8-imdb-v1
• —Training notebook (v2): https://www.kaggle.com/code/pujaniitj/mlops-group-8-imdb-v2

Acknowledgments

• —Base model: DistilBERT by Sanh et al. (Hugging Face)
• —Dataset: IMDB by Maas et al. (Stanford NLP)
• —Training infrastructure: Kaggle Notebooks
• —Experiment tracking: Weights & Biases