CoolFace
Modelpublic

Arko007/fake-news-liar-political

sourceHugging Faceapache-2.0updated 1y agoView on Hugging Face
0likes38downloads
Model Card

LIAR Political Fact-Checker

Short description A RoBERTa-base model fine-tuned to perform binary classification (FAKE vs REAL) on short political statements (PolitiFact-style). Intended as an assistive tool for political fact-checking, claim triage, and analyst workflows. Not intended as a standalone authoritative verdict.

Model repository: Arko007/fake-news-liar-political

Model snapshot / overview

  • —Base model: RoBERTa-base (125M parameters)
  • —Task: Binary classification — FAKE (0) vs REAL (1)
  • —Domain: Short political statements (US-centric PolitiFact / LIAR)
  • —Intended uses: claim triage, researcher experiments, human-in-loop fact-checking
  • —Not intended for: medical/scientific fact-checking or automated high-stakes enforcement

Key performance (reported)

  • —Test accuracy: 71.25%
  • —Validation accuracy: 71.44%
  • —F1 (binary): 71.42%

Benchmarks for context:

  • —FakeStack (ensemble, 2023): 75.58% (ensemble)
  • —Standard RoBERTa baseline: ~68%
  • —BERT baseline: ~67%
  • —DeBERTa (6-class attempt): ~40% (multi-class differences)

Training & fine-tuning pipeline

This model used a 3-stage continual fine-tuning approach:

  1. 1.RoBERTa pretraining on general corpora (official RoBERTa)
  2. 2.Domain-adaptive fine-tuning on a 5M curated fake-news corpus (Arko007/fake-news-roberta-5M)
  3. 3.Final fine-tune on the LIAR dataset (converted to binary)

LIAR split (binary conversion):

  • —Train: 18,369 statements
  • —Validation: 2,297 statements
  • —Test: 2,296 statements

6→2 label mapping used for training:

  • —FAKE: pants-fire (0), false (1), barely-true (2)
  • —REAL: half-true (3), mostly-true (4), true (5)

Training hyperparameters:

  • —Optimizer: AdamW
  • —Learning rate: 5e-6
  • —Weight decay: 0.02
  • —Batch size: 96 (effective 192 with gradient accumulation)
  • —Epochs: up to 30 (early stopped at ~17)
  • —Class weights: FAKE = 1.0, REAL = 1.3
  • —Precision: BF16 mixed precision
  • —Hardware: NVIDIA L4 (24 GB VRAM)

Notes:

  • —Class weighting and sampling were applied to address imbalance in LIAR after mapping.
  • —Preprocessing: standard tokenization, minimal text normalization; see training scripts for exact sequence length/truncation policy.

Data

Primary data for final tuning:

  • —LIAR (chengxuphd/liar2) — political fact-check statements (PolitiFact). The dataset contains 6-way labels; this model uses the binary mapping shown above.

Upstream domain-adaptive data:

  • —Arko007/fake-news-roberta-5M — ~5M curated fake/real news samples used to adapt RoBERTa to misinformation-style news language before LIAR fine-tune.

Data caveats:

  • —LIAR / PolitiFact annotations reflect annotator and editorial judgment and are US-centric.
  • —Binary conversion flattens nuance; use caution when interpreting borderline cases.

Evaluation & metrics

Evaluation performed on withheld LIAR test split (2,296 statements). Metrics reported:

  • —Accuracy: 71.25%
  • —Binary F1: 71.42%

When comparing to other models, ensure consistent 6→2 mapping and preprocessing.

Usage example

Example using Hugging Face Transformers (PyTorch):

python
from transformers import AutoTokenizer, AutoModelForSequenceClassification, pipeline

model_id = "Arko007/fake-news-liar-political"  # replace with HF repo id when uploaded
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForSequenceClassification.from_pretrained(model_id)

clf = pipeline("text-classification", model=model, tokenizer=tokenizer, return_all_scores=False, device=0)  # device=-1 for CPU

examples = [
    "The government passed a law guaranteeing free college for all citizens.",
    "Candidate A voted to privatize Social Security in 2017."
]

results = clf(examples, truncation=True)
print(results)  # check label names in model.config if needed

Label mapping in outputs:

  • —"FAKE" → predicted fake
  • —"REAL" → predicted real

Intended uses & limitations

Appropriate uses:

  • —Research experiments and baselines for political misinformation
  • —Human-in-the-loop claim triage and prioritization
  • —Tools to assist fact-checkers (not to replace them)

Limitations & risks:

  • —Domain-specific: optimized for short political statements; may underperform on long-form articles or non-political claims.
  • —Binary labels lose nuance from the original 6-class annotation.
  • —Biases from PolitiFact / LIAR (US-centric) will reflect in model outputs.
  • —Avoid using as the sole decision-maker for moderation, legal, or medical decisions.

Responsible deployment recommendations

  • —Always use with human oversight, especially for high-impact decisions.
  • —Report uncertainty and provide sources/evidence; do not surface model output as final judgment.
  • —Audit outputs for demographic and topical biases before deployment.
  • —Respect dataset licensing and attribution requirements when redistributing or publishing models.

Reproducibility & code

See repository training scripts for:

  • —Exact 6→2 mapping and preprocessing
  • —Checkpointing & evaluation code
  • —Hyperparameter schedules and scheduler settings

Citation

If you use this model, cite the model and primary datasets:

Suggested model citation:

@misc{liar-political-2025,
  title = {LIAR Political Fact-Checker},
  author = {Arko007},
  year = {2025},
  howpublished = {Hugging Face model hub: Arko007/fake-news-liar-political},
  note = {RoBERTa-base fine-tuned on 5M corpus and LIAR (binary)}
}

Also cite: Wang, W. Y. (2017) "Liar, Liar Pants on Fire": A New Benchmark Dataset for Fake News Detection (LIAR).

Contact & maintainer

Maintainer: Arko007 (https://huggingface.co/Arko007) Repository: https://github.com/Arko007/fake-news-liar-political

If you find licensing issues, data-provenance errors, or safety concerns, please open an issue in the repo.