esapzoi/dswe-grnn-hotel-reviews

DSWE-GRNN: Domain-Specific Word Embeddings + Gated Recurrent Neural Network

Model Description

Implementation of the DSWE-GRNN architecture from:

Multi-class Review Rating Classification using Deep Recurrent Neural Network Junaid Hassan & Umar Shoaib Neural Processing Letters, Vol. 51, pp. 1031–1048 (2020) DOI: 10.1007/s11063-019-10125-6

The model predicts hotel review star ratings (1-5) from text using:

1. 1.Domain-Specific Word Embeddings (DSWE): Word2Vec (CBOW) trained on the hotel review corpus
2. 2.Gated Recurrent Neural Network (GRNN): Multi-layer GRU architecture

Key Properties

• —Reviewer-independent: Does not use user identity or product metadata
• —Domain-specific embeddings: Embeddings trained on the review corpus (not generic pre-trained vectors)
• —Downsampling: Class balancing via downsampling to minority class size

Architecture

Input (token indices) → Embedding(32d, DSWE) → Dropout(0.3) → GRU(128, 2 layers) → Last Hidden → Dropout(0.3) → Linear(5) → Softmax

Training Data

• —Dataset: TripAdvisor Hotel Reviews (argilla/tripadvisor-hotel-reviews)
• —Paper's dataset: Datafiniti Hotel Reviews (Kaggle, 2017) — not freely available on HF Hub
• —Classes: 5 (1-star to 5-star ratings)
• —Downsampled: 7,105 reviews (1,421 per class)
• —Split: Train 5,115 / Val 569 / Test 1,421

Results vs Paper

Note: Our results are lower than the paper's. See "Deviations" section below for analysis.

Analysis of Performance Gap

The ~25% accuracy gap between our results and the paper's is likely due to:

1. 1.Different dataset: We use TripAdvisor hotel reviews (avg 111 tokens/review) vs Datafiniti Kaggle (avg 52 tokens/review). Shorter, more focused reviews are likely easier to classify.
2. 2.Dataset size after downsampling: Our min class has 1,421 samples; the paper's original dataset had 14,895 total.
3. 3.Reduced hidden dimension: 128 vs paper's 256 GRU units (for CPU feasibility).
4. 4.Fewer epochs: 10 vs paper's 15.
5. 5.The paper reports only accuracy — no RMSE/MAE — making it difficult to fully compare.

Deviations from Paper

Usage

import torch
from model import DSWE_GRNN
from preprocessing import clean_text, texts_to_sequences

# Load model
checkpoint = torch.load("dswe_grnn.pt", map_location="cpu")
config = checkpoint['config']
vocab = checkpoint['vocab']

model = DSWE_GRNN(
    vocab_size=config['vocab_size'],
    embedding_dim=config['embedding_dim'],
    hidden_dim=config['hidden_dim'],
    num_layers=config['num_layers'],
    num_classes=config['num_classes'],
    dropout=config['dropout'],
)
model.load_state_dict(checkpoint['model_state_dict'])
model.eval()

# Predict
text = "The room was spotless and the staff were incredibly helpful."
cleaned = clean_text(text)
seq = texts_to_sequences([cleaned], vocab, max_len=100)
x = torch.tensor(seq, dtype=torch.long)

with torch.no_grad():
    logits = model(x)
    probs = torch.softmax(logits, dim=1).squeeze()
    predicted_rating = probs.argmax().item() + 1

print(f"Predicted rating: {predicted_rating} stars")

Citation

@article{hassan2020multi,
  title={Multi-class Review Rating Classification using Deep Recurrent Neural Network},
  author={Hassan, Junaid and Shoaib, Umar},
  journal={Neural Processing Letters},
  volume={51},
  pages={1031--1048},
  year={2020},
  publisher={Springer},
  doi={10.1007/s11063-019-10125-6}
}

Limitations

• —Trained on English hotel reviews only
• —Performance gap vs paper likely due to different dataset
• —5-class classification is inherently difficult (adjacent classes like 3-star vs 4-star have overlapping language)
• —Small embedding dimension (32) limits semantic capacity
• —No attention mechanism (pure GRU)

Hardware & Training

• —Hardware: CPU (2 vCPU, 16GB RAM)
• —Training time: ~92s for DSWE-GRNN (10 epochs)
• —Total pipeline time: ~5 minutes (all models + embeddings)
• —Random seed: 42