tharun94k/qwen_sentiment_model

language: en license: mit library_name: peft tags:

• —text-classification
• —sentiment-analysis
• —qwen
• —lora datasets:
• —kazanova/sentiment140 ---

Qwen2.5-3B Fine-tuned for Sentiment Analysis

This model is a fine-tuned version of Qwen/Qwen2.5-3B-Instruct on the Sentiment140 dataset for sentiment classification.

Model Description

This model was trained to classify tweets as either "positive" or "negative". It was fine-tuned using the Parameter-Efficient Fine-Tuning (PEFT) technique, specifically LoRA (Low-Rank Adaptation), on a sample of 25,000 tweets. The training was performed in a Google Colab environment using an A100 GPU.

Intended Uses & Limitations

This model is intended for academic and demonstration purposes to analyze the sentiment of short, English-language text similar to tweets.

Limitations:

• —The model is only trained on two labels (positive, negative) and may not be suitable for more nuanced sentiment analysis.
• —The training data is from 2009 and may not capture modern slang or context.
• —The model is not intended for production use without further testing and evaluation.

Training Procedure

The model was trained for one epoch with a learning rate of 2e-4. The training process involved 4-bit quantization to reduce memory usage and make the fine-tuning process more efficient.

Training Hyperparameters

• —Framework: Transformers
• —Base Model: Qwen/Qwen2.5-3B-Instruct
• —Quantization: 4-bit (nf4)
• —LoRA `r`: 16
• —LoRA `alpha`: 32
• —Learning Rate: 0.0002
• —Epochs: 1
• —Batch Size: 8