engineerai/lithuanian-fake-review-detection-XLM-RoBERTa-finetuned

Lithuanian Fake Review Detection: XLM-RoBERTa (fine-tuned)

A multilingual XLM-RoBERTa model fine-tuned for binary fake-review detection on Lithuanian e-commerce reviews. The model distinguishes between authentic reviews scraped from Lithuanian review platforms and synthetic reviews generated by large language models.

Model details

• —Developed by: Austėja Rušėnaitė
• —Base model: FacebookAI/xlm-roberta-base (Conneau et al., 2020)
• —Architecture: XLM-RoBERTa, 12 transformer layers, 768 hidden dimensions, 12 attention heads, ~270M parameters
• —Tokenizer: SentencePiece (sub-word)
• —Task: Binary text classification (real / fake)
• —Language: Lithuanian (lt)
• —License: Apache 2.0

Intended uses

Detection of machine-generated fake reviews on Lithuanian-language e-commerce platforms, complaint sites, and review aggregators. The model is intended as a component of consumer-protection and e-store trust-evaluation pipelines.

Out-of-scope uses

The training data contains only LLM-generated fake reviews; the model has not been evaluated on human-written deceptive reviews and may not generalise to that distribution. Although the base model is multilingual, this fine-tuned variant is trained exclusively on Lithuanian text; performance on non-Lithuanian inputs is not guaranteed.

Training data

The training corpus consists of 116,434 reviews:

• —Real reviews (53,503): scraped from three Lithuanian review platforms — evertink.lt (46,942), atsiliepimai.lt (3,484), and vartotojuskundai.lt (3,077). Treated as authentic for training purposes; the source platforms apply no documented fake-review filter, and an unknown fraction of these reviews may themselves be undetected fakes (a documented limitation).
• —Fake reviews (62,931): generated using three large language models (GPT-4o Mini, Claude Haiku 4.5, Gemini 2.5 Flash) via 15 prompt templates varying along six properties (length, sentiment, topic, writing style, diacritics usage, and real-review stylistic guidance). Post-processing applied exact-duplicate removal, length-outlier filtering (1st–99th percentile of real-review length distribution), and 5-gram overlap filtering.

The combined dataset was split 70 / 15 / 15 into training, validation, and test subsets using stratified sampling. The test set contains 17,466 reviews (8,026 real, 9,440 fake).

Training procedure

• —Hardware: Kaggle, dual NVIDIA T4 GPU
• —Framework: PyTorch + HuggingFace Transformers (Wolf et al., 2020)
• —Preprocessing: lowercase + whitespace normalisation only (no lemmatisation, no stop-word removal, URLs preserved)
• —Epochs: 15 (validation F1 plateaued by epoch ~7 and improved marginally thereafter)
• —Checkpoint format: SafeTensors

Evaluation

Reported on the held-out test set (17,466 reviews):

Per-class F1: Real — 0.97; Fake — 0.97.

The model favours recall (0.9853) over precision (0.9472), reflecting an asymmetric error preference: in a consumer-protection context, missing a fake review (false negative) is generally more costly than incorrectly flagging a genuine review (false positive). Threshold tuning can be applied at deployment time if a different precision/recall trade-off is required.

Limitations

• —Diacritics bias. LLM-generated reviews tend to use Lithuanian diacritics more consistently than real users, so reviews with correct diacritics receive slightly higher fake-probability scores. This spurious feature should be considered when deploying the model in production.
• —LLM-only fake corpus. The fake-review portion of the training set contains no human-written deceptive reviews; generalisation to human-authored fakes is unverified.
• —Platform skew. The real-review portion is dominated by evertink.lt (87.7 %); behaviour on out-of-platform reviews has not been quantified.
• —Real-review label noise. Reviews scraped from source platforms are treated as authentic without explicit verification; an unknown fraction may themselves be undetected fakes.

Usage

from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch

repo = "engineerai/lithuanian-fake-review-detection-XLM-RoBERTa-finetuned"
tokenizer = AutoTokenizer.from_pretrained(repo)
model = AutoModelForSequenceClassification.from_pretrained(repo)
model.eval()

text = "Puiki parduotuvė, greitas pristatymas, rekomenduoju!"
inputs = tokenizer(text, return_tensors="pt", truncation=True, padding=True, max_length=256)
with torch.no_grad():
    logits = model(**inputs).logits
prob_fake = torch.softmax(logits, dim=-1)[0, 1].item()
print(f"P(fake) = {prob_fake:.4f}")

References

• —Conneau, A., Khandelwal, K., Goyal, N., Chaudhary, V., Wenzek, G., Guzmán, F., Grave, É., Ott, M., Zettlemoyer, L., & Stoyanov, V. (2020). Unsupervised cross-lingual representation learning at scale. Proceedings of ACL 2020. arXiv:1911.02116
• —Wolf, T., et al. (2020). Transformers: State-of-the-art natural language processing. EMNLP System Demonstrations, 38–45.