rahmaelnabarawy/gptneo125m-finetuned-generate-reviews

GPT-Neo 125M LoRA Fine-Tuned Review Generator

Model Overview

This model is a review-generation fine-tuned version of EleutherAI/gpt-neo-125M, trained using LoRA (Low-Rank Adaptation) on the Yelp Review Full dataset.

The model generates structured and natural-sounding restaurant and product reviews when prompted in a conversational format.

Fine-tuning was performed using parameter-efficient training, and LoRA weights were merged into the base model for standalone deployment.

Model Details

Model Description

This is a causal language model fine-tuned for review generation.

Expected prompt format:

User: Write a review about a new restaurant.
Assistant:

The model generates a realistic review continuation after Assistant:.

Model Type

Causal Language Model (GPT-Neo architecture) fine-tuned using LoRA

Language

English

Base Model

EleutherAI/gpt-neo-125M https://huggingface.co/EleutherAI/gpt-neo-125M

Fine-Tuning Method

Low-Rank Adaptation (LoRA)

LoRA enables efficient fine-tuning by training a very small subset of parameters instead of updating the entire model.

After training:

• —LoRA weights were merged into the base model
• —The final model is standalone (no adapter required)

Intended Uses

Direct Use

• —Synthetic review generation
• —NLP experimentation
• —Prompt engineering practice
• —Educational purposes (fine-tuning & LoRA practice)

Example Usage

from transformers import AutoTokenizer, AutoModelForCausalLM
import torch

device = "cuda" if torch.cuda.is_available() else "cpu"

model = AutoModelForCausalLM.from_pretrained(
    "rahmaelnabarawy/gptneo125m-finetuned-generate-reviews"
).to(device)

tokenizer = AutoTokenizer.from_pretrained(
    "rahmaelnabarawy/gptneo125m-finetuned-generate-reviews"
)

prompt = """User: Write a review about a nearby coffee shop.
Assistant:"""

inputs = tokenizer(prompt, return_tensors="pt").to(device)

outputs = model.generate(
    **inputs,
    max_new_tokens=100,
    temperature=0.8,
    top_p=0.95,
    do_sample=True
)

print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Training Details

Dataset

Yelp Review Full https://huggingface.co/datasets/yelpreviewfull

Training subset used:

• —30,000 training samples
• —2,000 validation samples

Data Format Used for Training

User: Write a review.
Assistant: <review text>

The conversational structure was intentionally used to guide generation behavior.

Training Hyperparameters

Architecture

GPT-Neo Transformer

• —Layers: 12
• —Hidden Size: 768
• —Attention Heads: 12
• —Parameters: 125M
• —Objective: Causal Language Modeling

Compute Infrastructure

Hardware:

• —NVIDIA Tesla T4 GPU (or similar)

Software:

• —PyTorch
• —Hugging Face Transformers
• —PEFT
• —Hugging Face Datasets

Limitations

• —Small model size (125M parameters)
• —May generate repetitive or generic text
• —Sensitive to prompt format
• —Inherits biases from Yelp dataset
• —Not optimized for production-level deployment

Out-of-Scope Use

This model is NOT intended for:

• —Real-world review manipulation
• —Deceptive content generation
• —High-stakes decision systems