Modelpublic

umangshikarvar/sentiment-qlora-gptneo

sourceHugging Facemitupdated 1y agoView on Hugging Face

0likes3downloads

Model Card

Sentiment Classifier with QLoRA (4-bit GPT-Neo)

This model is a QLoRA fine-tuned version of `EleutherAI/gpt-neo-1.3B` for sentiment classification (Positive, Negative, Neutral) on the `tweet_eval/sentiment` dataset.

It uses 4-bit quantization via `bitsandbytes` and fine-tunes using QLoRA (PEFT), enabling efficient training and inference even on low-resource hardware.

Model Details

—Base Model: EleutherAI/gpt-neo-1.3B
—Quantization: 4-bit (nf4) via BitsAndBytesConfig
—Fine-tuning: QLoRA using peft
—Adapter Format: Compatible with peft
—Dataset: `tweet_eval/sentiment`
—Classes: Positive, Negative, Neutral

How It Works

This model is treated as a text generation model but constrained using:

—a fixed prompt format

—Output generation restricted to a single token

—Vocabulary limited to only 3 sentiment tokens: ["Positive", "Negative", "Neutral"] using a custom LogitsProcessor:

python

from transformers import LogitsProcessor

class RestrictVocabLogitsProcessor(LogitsProcessor):
  def __init__(self, allowed_token_ids):
      self.allowed_token_ids = allowed_token_ids

  def __call__(self, input_ids, scores):
      mask = torch.full_like(scores, float("-inf"))
      mask[:, self.allowed_token_ids] = scores[:, self.allowed_token_ids]
      return mask

This design makes the model extremely lightweight and deterministic for sentiment classification tasks.

🔗 Try It Out

![Sentiment Classifier Demo](https://huggingface.co/spaces/umangshikarvar/Sentiment_classifier)

Run the model interactively in your browser using Gradio — no setup needed by clicking the image above.

How to Use

Install dependencies

bash

pip install torch transformers peft

Load the model and run inference

python

import torch
from transformers import AutoTokenizer, AutoModelForCausalLM, LogitsProcessor, LogitsProcessorList
from peft import PeftModel, PeftConfig

# Load tokenizer
tokenizer = AutoTokenizer.from_pretrained("EleutherAI/gpt-neo-1.3B")
tokenizer.pad_token = tokenizer.eos_token  # Required for generation

# Load adapter config and base model
checkpoint = "umangshikarvar/sentiment-qlora-gptneo"
peft_config = PeftConfig.from_pretrained(checkpoint)

torch_dtype = torch.float16 if torch.cuda.is_available() else torch.float32
base_model = AutoModelForCausalLM.from_pretrained(
    peft_config.base_model_name_or_path,
    torch_dtype=torch_dtype
)

# Load QLoRA adapter
model = PeftModel.from_pretrained(base_model, checkpoint)
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
model.eval().to(device)

# Restrict output to sentiment tokens
class RestrictVocabLogitsProcessor(LogitsProcessor):
    def __init__(self, allowed_token_ids):
        self.allowed_token_ids = allowed_token_ids

    def __call__(self, input_ids, scores):
        mask = torch.full_like(scores, float("-inf"))
        mask[:, self.allowed_token_ids] = scores[:, self.allowed_token_ids]
        return mask

sentiment_words = ["Positive", "Negative", "Neutral"]
allowed_ids = [
    tokenizer(word, add_special_tokens=False)["input_ids"][0]
    for word in sentiment_words
]
logits_processor = LogitsProcessorList([
    RestrictVocabLogitsProcessor(allowed_ids)
])

# Inference function
def predict_sentiment(tweet: str) -> str:
    prompt = f"Tweet: {tweet}\nSentiment:"
    inputs = tokenizer(prompt, return_tensors="pt").to(model.device)

    outputs = model.generate(
        **inputs,
        max_new_tokens=1,
        do_sample=False,
        logits_processor=logits_processor
    )

    response = tokenizer.decode(outputs[0], skip_special_tokens=True)
    prediction = response.replace(prompt, "").strip().split()[0]

    if prediction.lower().startswith("pos"):
        return "Positive"
    elif prediction.lower().startswith("neg"):
        return "Negative"
    else:
        return "Neutral"

Example usage

python

print(predict_sentiment("I absolutely love this new feature!"))
# Output: Positive