YatoGami47001/fake-news-detector-bert-data-science

This model is a fine-tuned version of DistilBERT designed to detect Fake News with high accuracy. It was trained on a comprehensive dataset consisting of approximately 419,000 news articles, achieving state-of-the-art performance with an accuracy of 95.3%.

Model Details Model Description Developed by: YatoGami47001

Model type: Text Classification (Transformer)

Language(s) (NLP): English

License: MIT

Finetuned from model: distilbert-base-uncased

This model utilizes the DistilBERT architecture, which offers a balance between performance and speed. Unlike traditional approaches that often use small subsets of data, this model was fine-tuned on the full dataset (approx. 335k training samples, 84k test samples) to capture deep semantic patterns and linguistic nuances associated with misinformation. Uses Direct Use You can use this model to check the credibility of news articles. It classifies text into two categories:

LABEL_0: Fake News

LABEL_1: Real News

It is suitable for:

Automated fact-checking systems.

Content moderation.

Media literacy research.

How to Get Started with the Model You can use the Hugging Face pipeline API to easily implement the model:

from transformers import pipeline

# Load the pipeline
classifier = pipeline("text-classification", model="YatoGami47001/fake-news-detector-bert-data-science")

# Example News Text
text = "The government announced a new tax policy today that affects small businesses."

# Get Prediction
result = classifier(text)
print(result)
# Output example: [{'label': 'LABEL_1', 'score': 0.99...}] (LABEL_1 = Real)

OR using AutoModel:

from transformers import AutoTokenizer, AutoModelForSeq2SeqLM

tokenizer = AutoTokenizer.from_pretrained("YatoGami47001/fake-news-detector-bert-data-science")
model = AutoModelForSeq2SeqLM.from_pretrained("YatoGami47001/fake-news-detector-bert-data-science")

Training Details Training Data The model was trained on a large-scale combined dataset containing 419,297 news articles.

Preprocessing: The data was cleaned to remove null values and duplicates.

Composition: The dataset is balanced between Real and Fake news sources.

Training Procedure The model was fine-tuned using the Hugging Face Trainer API on a high-performance GPU.

Training Hyperparameters Learning Rate: 5e-5

Batch Size: 32

Epochs: 1 (Full Dataset)

Precision: FP16 (Mixed Precision)

Optimizer: AdamW

Evaluation The model was evaluated on a stratified test set of approximately 84,000 samples.

Results

Metric Score Accuracy 95.26% F1-Score 0.957 Precision 95.36% Recall 96.07% The model demonstrates exceptional performance, slightly outperforming custom Bi-LSTM architectures trained on the same data.

Bias, Risks, and Limitations Language: The model is trained primarily on English news articles and may not perform well on other languages.

Context: While highly accurate, the model detects patterns based on its training data (up to the training cutoff). Extremely recent events or new types of disinformation strategies might require re-training.

Bias: Like all LLMs, it may reflect biases present in the training corpus.

Environmental Impact Hardware Type: NVIDIA A100 / T4 (Google Colab Pro)

Compute Region: Cloud

Training Duration: < 1 Hour (Optimized with FP16)

Citation If you use this model, please cite it as:

@misc{fake-news-detector-bert-data-science,
  author = {YatoGami47001},
  title = {Fake News Detector (DistilBERT)},
  year = {2024},
  publisher = {Hugging Face},
  howpublished = {\url{https://huggingface.co/YatoGami47001/fake-news-detector-bert-data-science}}
}