Xtiphyn/Cross-Lingual-Spam-Filter

XLM-Roberta Spam Classifier (EN-HI ➝ DE)

This model is a fine-tuned version of xlm-roberta-base for cross-lingual spam detection. It was trained on English and Hindi messages, and evaluated on German samples. The goal is to demonstrate zero-shot transfer in spam/ham classification across languages.

🧠 Model Description

• —Model Type: XLM-RoBERTa Base (Transformer encoder)
• —Task: Binary classification – spam vs. ham
• —Languages: Trained on English and Hindi, tested on German
• —Tokenizer: AutoTokenizer from transformers (xlm-roberta-base)
• —Framework: PyTorch + Hugging Face Trainer

Intended Uses & Limitations

Intended Uses:

• —Spam filtering in multilingual messaging systems
• —Research on cross-lingual text classification
• —Transfer learning studies involving high/low-resource languages

⚠️ Limitations:

• —Trained on English and Hindi; evaluated on German and French. While results on French were promising, further benchmarking is recommended.
• —May underperform on mixed-language, informal, or code-switched inputs.

📊 Performance

Evaluation Metrics (on German test set):

Confusion Matrix:

📚 Dataset

The dataset is a multilingual corpus with parallel spam/ham messages in:

• —English
• —Hindi
• —German
• —French

For this training run:

• —Train set: English + Hindi
• —Test set: German

Labels:

• —"ham" → 0
• —"spam" → 1

⚙️ Training Configuration

Example Usage

from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch

model_name = "Xtiphyn/Cross-Lingual-Spam-Filter"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForSequenceClassification.from_pretrained(model_name)

inputs = tokenizer("Sie haben eine kostenlose Reise gewonnen!", return_tensors="pt")
with torch.no_grad():
    logits = model(**inputs).logits
    prediction = torch.argmax(logits).item()

print("Label:", "Spam" if prediction == 1 else "Ham")


🛠️ Environment
Transformers: 4.54.0

PyTorch: 2.6.0+cu124

Datasets: 4.0.0

Tokenizers: 0.21.2

🚧 Future Work
Incorporate code-switching and low-resource scripts

Made with ❤️ by Xtiphyn.