DevPatel0611/TruthLens

1---2title: TruthLens3emoji: 🔍4colorFrom: blue5colorTo: indigo6sdk: streamlit7sdk_version: 1.31.08python_version: 3.10.139app_file: app.py10pinned: false11---12# TruthLens: Advanced Fake News Detection Pipeline13 14TruthLens is an end-to-end fake news detection system that moves beyond simple machine learning probabilities. It employs a robust **5-signal weighted scoring framework** built on journalistic standards, combining deep learning models (DistilBERT, RoBERTa), sequence models (LSTM), statistical models (Logistic Regression), and heuristic analysis to deliver explainable verdicts.15 16## 🌟 Key Features17 18*   **5-Signal Scoring Framework:**19    *   **Source Credibility (30%):** Evaluates outlet reputation, author presence, and source corroboration, including typosquatting checks.20    *   **Claim Verification (30%):** Combines AI probability with spaCy-based Named Entity Recognition (NER) and quote attribution analysis.21    *   **Linguistic Quality (20%):** Detects sensationalism, superlatives, passive voice, and uses DistilBERT to check if the headline contradicts the body.22    *   **Freshness (10%):** Contextual and date-based temporal scoring to detect outdated information.23    *   **AI Model Consensus (10%):** Ensemble voting from Logistic Regression, LSTM, DistilBERT, and RoBERTa.24*   **Adversarial Guardrails:** Hard caps and overrides for highly suspicious patterns (Triple Anonymity, Uncited Statistics, Headline Contradictions).25*   **Live Web Corroboration:** RAG (Retrieval-Augmented Generation) pipeline using live search to verify unambiguous claims.26*   **TruthLens UI:** A sleek, dark/light mode adaptable Streamlit dashboard providing detailed explainability down to the specific signals and deductions.27 28---29 30## 📁 Project Structure31 32```text33fake_news_detection/34├── app.py                     # Streamlit frontend (TruthLens UI)35├── run_pipeline.py            # Main script to run pipeline stages36├── requirements.txt           # Python dependencies37├── src/38│   ├── stage1_ingestion.py    # Downloads and prepares datasets39│   ├── stage2_preprocessing.py# Cleans text, tokenizes, and saves artifacts40│   ├── stage3_training.py     # Trains models (LR, LSTM, DistilBERT, RoBERTa)41│   ├── stage4_inference.py    # The 5-signal scoring engine and prediction logic42│   └── utils/43│       └── rag_retrieval.py   # Live web search corroboration functions44├── data/                      # Raw and processed datasets (created during execution)45└── models/                    # Trained models and vectorizers (created during execution)46```47 48---49 50## 🚀 Getting Started51 52### 1. Installation53 54Ensure you have Python 3.8+ installed. Install the required dependencies:55 56```bash57pip install -r requirements.txt58python -m spacy download en_core_web_sm59```60 61### 2. Running the Pipeline62 63The project is divided into stages. You can run the entire pipeline end-to-end, or run specific stages individually using `run_pipeline.py`.64 65**To run the complete training pipeline (Stages 1 to 3):**66*Note: This will download datasets, preprocess them, and train all models. It may take a significant amount of time depending on your hardware.*67 68```bash69python run_pipeline.py --stage 1 2 370```71 72**To run individual stages:**73 74*   **Stage 1: Data Ingestion**75    Downloads and formats the necessary datasets (e.g., LIAR, ISOT).76    ```bash77    python run_pipeline.py --stage 178    ```79 80*   **Stage 2: Preprocessing**81    Cleans the text, maps verdicts to binary labels, and prepares DataFrames for training.82    ```bash83    python run_pipeline.py --stage 284    ```85 86*   **Stage 3: Training**87    Trains the ensemble: Logistic Regression, LSTM, DistilBERT, and RoBERTa. Saves the models to the `/models` directory.88    ```bash89    python run_pipeline.py --stage 390    ```91 92*   **Stage 4: Evaluation**93    Evaluates the trained pipeline on the holdout test set using the 5-signal inference framework.94    ```bash95    python run_pipeline.py --eval96    ```97 98---99 100## 🖥️ Running the Application101 102Once the models are trained (or if you already have the pre-trained weights in the `/models` directory), you can launch the TruthLens UI.103 104```bash105python -m streamlit run app.py106```107 108This will start a local web server (usually at `http://localhost:8501`). 109 110### Using the App:1111.  **Paste text or provide a URL:** You can paste the raw text of an article (with or without a headline) or simply provide a URL for the app to parse automatically.1122.  **Select depth:** Choose Quick, Standard, or Deep analysis.1133.  **View Results:** Explore the four-tier verdict (True, Uncertain, Likely False, False), signal breakdown, adversarial flags, and live web corroboration results.114