Alekla0126/PhilBERT

PhilBERT: Phishing Detection with DistilBERT

PhilBERT is a fine-tuned DistilBERT model optimized for detecting phishing threats across multiple communication channels, including emails, SMS, URLs, and websites. It is trained on a diverse dataset sourced from Kaggle, Mendeley, Phishing.Database, and Bancolombia, ensuring high adaptability and real-world applicability.

Key Features

• —Multi-Channel Detection – Analyzes text, URLs, and web content to detect phishing patterns.
• —Fine-Tuned on Real-World Data – Includes recent three months of financial institution data (Bancolombia).
• —Lightweight & Efficient – Based on DistilBERT, providing high performance with reduced computational costs.
• —High Accuracy – Achieves 85.22% precision, 93.81% recall, and 88.77% accuracy on unseen data.
• —Self-Adaptive Learning – Continuously evolves using real-time phishing simulations generated with GPT-4o.
• —Scalability – Designed to support 7,000–25,000 simultaneous users in production environments.

Model Architecture

PhilBERT leverages DistilBERT, a distilled version of BERT, maintaining the same architecture but with 40% fewer parameters, making it lightweight while preserving high accuracy. The final model includes:

• —Tokenizer: Trained to recognize phishing-specific patterns (URLs, obfuscation, domain misspellings).
• —Custom Classifier: A fully connected dense layer added for binary classification (phishing vs. benign).
• —Risk Scoring Mechanism: A weighted confidence score applied to enhance detection reliability.

Data Preprocessing

Before fine-tuning, the dataset underwent extensive preprocessing to ensure balance and quality:

• —Duplicate Removal & Balancing: Maintained a near 50-50 phishing-to-benign ratio to prevent model bias.
• —Feature Extraction: Applied to URLs, HTML, email bodies, and SMS content to enrich input representations.
• —Dataset Split: Final dataset included:
• —427,028 benign URLs & 381,014 phishing URLs
• —17,536 unique email samples
• —5,949 SMS samples
• —Web entries filtered for efficiency (removing entries >100KB).
• —Export Format: Data transformed and stored in JSON for efficient training.

Training & Evaluation

PhilBERT was fine-tuned on multi-modal phishing datasets using transfer learning, achieving:

• —False Positive Reduction: Multi-layered filtering minimized false positives while maintaining high recall.
• —Scalability: Successfully stress-tested for up to 25,000 simultaneous users.
• —Compliance: Meets ISO 27001 and GDPR standards for security and privacy.

Usage

Installation

pip install transformers torch

Inference

from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch

model_name = "your_username/PhilBERT"

tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForSequenceClassification.from_pretrained(model_name)

text = "Click this link to update your bank details: http://fakebank.com"
inputs = tokenizer(text, return_tensors="pt")

with torch.no_grad():
    outputs = model(**inputs)
    predictions = torch.nn.functional.softmax(outputs.logits, dim=-1)

print(f"Phishing probability: {predictions[0][1].item():.4f}")

License

This model is proprietary and protected under a custom license. Please refer to the [LICENSE](LICENSE) file for terms of use.