Pujaniitj/MLOPS_GROUP_PROJECT

1---2language: en3license: apache-2.04library_name: transformers5pipeline_tag: text-classification6tags:7- text-classification8- sentiment-analysis9- distilbert10- imdb11- mlops12datasets:13- stanfordnlp/imdb14base_model: distilbert-base-uncased15metrics:16- accuracy17- f118- precision19- recall20model-index:21- name: mlops-group-sentiment22  results:23  - task:24      type: text-classification25      name: Sentiment Classification26    dataset:27      type: stanfordnlp/imdb28      name: IMDB29    metrics:30    - type: accuracy31      value: 0.9032      name: Test Accuracy33    - type: f134      value: 0.9035      name: Test F1 (weighted)36---37 38# mlops-group-sentiment39 40A `distilbert-base-uncased` model fine-tuned on the IMDB movie reviews dataset41for binary sentiment classification (positive / negative).42 43This model is the final artifact of an MLOps group project at IIT Jodhpur44(Course CSL7040), demonstrating an end-to-end production ML pipeline: version45control on GitHub, GPU training on Kaggle, experiment tracking on Weights &46Biases, container packaging via Docker, and deployment to the Hugging Face Hub.47 48## How to Use49 50```python51from transformers import pipeline52 53classifier = pipeline("sentiment-analysis", model="pujaniitj/mlops-group-sentiment")54result = classifier("This movie was fantastic!")55print(result)56# [{'label': 'positive', 'score': 0.9876}]57```58 59## Intended Use60 61**Primary use case**: Classifying English-language movie reviews as positive62or negative sentiment.63 64**Out-of-scope uses**:65- Non-English text (model only trained on English IMDB reviews)66- Domain shift — e.g. tweets, product reviews, news articles, customer support67  transcripts. Performance will degrade outside the movie-review domain.68- Fine-grained sentiment (beyond binary pos/neg, e.g. 5-star ratings)69- High-stakes decisions or content moderation without human review70 71## Model Description72 73- **Base architecture**: DistilBERT (`distilbert-base-uncased`)74- **Distinct from base**: Fine-tuned classification head (2 output labels)75- **Parameters**: ~66 million76- **Tokenizer**: WordPiece (DistilBERT default)77- **Max sequence length**: 256 tokens78- **Labels**: `0 → negative`, `1 → positive`79 80## Training Data81 82- **Dataset**: [IMDB Movie Reviews](https://huggingface.co/datasets/stanfordnlp/imdb)83- **Train size**: 25,000 reviews (12,500 positive + 12,500 negative — perfectly balanced)84- **Test size**: 25,000 reviews (same balance)85- **Train/Validation split**: 90/10 of the train set, with `seed=42`86 87## Training Procedure88 89### Hyperparameters90 91| Setting              | Value  |92|----------------------|--------|93| Learning rate        | 3e-5   |94| Train batch size     | 16     |95| Eval batch size      | 32     |96| Epochs               | 3      |97| Max sequence length  | 256    |98| Warmup ratio         | 0.1    |99| Weight decay         | 0.01   |100| Optimizer            | AdamW  |101| Mixed precision      | fp16   |102| Seed                 | 42     |103 104### Training Environment105 106- **Platform**: Kaggle Notebook107- **Hardware**: 2× NVIDIA Tesla T4 GPU108- **Training time**: ~17 minutes109 110### Experiment Tracking111 112Two configurations were trained and compared via Weights & Biases:113 114| Run  | Learning rate | Test F1 | Test Accuracy | Test Loss |115|------|---------------|---------|---------------|-----------|116| v1 (this model) | 3e-5 | ~0.90 | ~0.90 | ~0.70 |117| v2 (discarded)  | 5e-5 | ~0.91 | ~0.91 | ~0.85 |118 119>  Replace these values with the exact decimals from your W&B run summary120> before publishing the final model card.121 122**Why v1 was selected**: While v2 achieved a marginally higher F1 (~0.5%),123it showed clear signs of overfitting — its eval loss climbed sharply across124epochs while v1's remained more stable. v1 also delivers ~25% faster inference,125making it the better choice for a production deployment.126 127## Evaluation Results128 129Evaluation on the held-out IMDB test set (25,000 reviews):130 131| Metric              | Value |132|---------------------|-------|133| Accuracy            | ~0.90 |134| F1 (weighted)       | ~0.90 |135| Precision (weighted)| ~0.90 |136| Recall (weighted)   | ~0.90 |137 138## Limitations and Biases139 140- **Domain**: Only trained on movie reviews. Expect degraded performance on141  other domains.142- **Length**: Inputs are truncated to 256 tokens (~200 words). Longer reviews143  may lose tail information that matters for sentiment.144- **Language**: English only.145- **Demographic biases**: IMDB reviewers historically skew toward certain146  demographics (e.g., predominantly male, English-speaking). The model may147  inherit these biases — e.g., it may misclassify reviews using vernacular or148  cultural references underrepresented in IMDB.149- **Sarcasm and irony**: Like most BERT-based classifiers, the model can150  struggle with sarcastic or ironic text where the surface sentiment opposes151  the intended meaning.152 153## Project Resources154 155- **GitHub repository**: https://github.com/pujaniitj/mlops-group-project-iitj156- **W&B experiment dashboard**: https://wandb.ai/pujaniitj-iit-jodpur/MLops_group_8157- **Training notebook (v1)**: https://www.kaggle.com/code/pujaniitj/mlops-group-8-imdb-v1158- **Training notebook (v2)**: https://www.kaggle.com/code/pujaniitj/mlops-group-8-imdb-v2159 160## Acknowledgments161 162- **Base model**: [DistilBERT](https://huggingface.co/distilbert-base-uncased)163  by Sanh et al. (Hugging Face)164- **Dataset**: [IMDB](https://huggingface.co/datasets/stanfordnlp/imdb)165  by Maas et al. (Stanford NLP)166- **Training infrastructure**: [Kaggle Notebooks](https://www.kaggle.com)167- **Experiment tracking**: [Weights & Biases](https://wandb.ai)