Srushti-Thakur013/falcon-ecommerce-qa

language: en tags:

• —falcon
• —qlora
• —ecommerce
• —peft
• —text-generation ---

Falcon-1B Fine-tuned for E-Commerce Q&A

Model Description

This model is a fine-tuned version of tiiuae/falcon-rw-1b using QLoRA (4-bit quantization) on an e-commerce product Q&A dataset. It generates helpful product review responses to customer queries.

Training Details

• —Base model: tiiuae/falcon-rw-1b
• —Technique: QLoRA with 4-bit quantization (BitsAndBytes NF4)
• —LoRA rank: r=8, alpha=32
• —Trainable parameters: 1,572,864 (0.12% of total)
• —GPU: Tesla T4 (15GB) — memory usage: ~2.6GB
• —Epochs: 2
• —Final training loss: ~0.42
• —Dataset: ecommerce-qa-dataset

Key Results

• —50%+ GPU memory reduction via 4-bit quantization
• —Loss converged from 7.8 → 0.42 over 2 epochs
• —Only 0.12% of parameters trained (LoRA adapters)

Usage

from transformers import AutoTokenizer, AutoModelForCausalLM
from peft import PeftModel

base = AutoModelForCausalLM.from_pretrained("tiiuae/falcon-rw-1b")
model = PeftModel.from_pretrained(base, "Srushti-Thakur013/falcon-ecommerce-qa")
tokenizer = AutoTokenizer.from_pretrained("Srushti-Thakur013/falcon-ecommerce-qa")

Intended Use

E-commerce product Q&A, customer support response generation.