CoolFace
Apppublic

braydenm07/ai-human-text-detector-project2

sourceHugging Faceupdated 3mo agoView on Hugging Face
0likes
App README

AI vs. Human Text Detection with LLM Explanations

Project Title

AI vs. Human Text Detection: Integrating Machine Learning Classifiers with Large Language Models

Problem Statement

This project detects whether a piece of writing is likely human-written or AI-generated. It extends the original Project 1 machine learning application by adding two Hugging Face Large Language Models (LLMs) and deploying the complete application as a Streamlit app on Hugging Face Spaces.

The goal is not only to classify text, but also to help users understand why the model made its prediction and how the writing style could be changed.

Project 1 Foundation

The original Project 1 system trained and compared six classifiers:

  1. 1.Support Vector Machine (SVM)
  2. 2.Decision Tree
  3. 3.AdaBoost
  4. 4.Feedforward Neural Network (FNN)
  5. 5.LSTM
  6. 6.CNN for text

The saved model artifacts are included in the models/ folder. The Streamlit app keeps this original ML/DL pipeline functional and allows the user to choose among available models.

Project 2 LLM Additions

This version integrates two Hugging Face LLMs in a meaningful way.

LLM 1: Microsoft Phi-3 Mini

Model: microsoft/Phi-3-mini-4k-instruct

Purpose:

  • —Explains the selected classifier's prediction.
  • —Discusses writing style, sentence structure, vocabulary, repetition, and confidence.
  • —Helps make the model output more interpretable.

LLM 2: Qwen2.5 0.5B Instruct

Model: Qwen/Qwen2.5-0.5B-Instruct

Purpose:

  • —Rewrites the submitted text to sound more naturally human.
  • —Gives short notes explaining what changed.
  • —Connects directly to the AI-vs-human detection problem by showing how style affects perceived authorship.

Dataset Description

The training dataset uses two columns:

  • —text: the writing sample
  • —label: the target label
  • —0 = Human-written
  • —1 = AI-generated

The project supports TF-IDF features and linguistic features such as sentence length, vocabulary richness, punctuation ratio, and readability.

Application Features

The Streamlit app supports:

  • —Pasted text input
  • —PDF, DOCX, and TXT uploads
  • —ML model selection
  • —Prediction and confidence score
  • —Comparison across all available Project 1 models
  • —Text statistics table
  • —LLM-generated explanation
  • —LLM rewrite and writing suggestions
  • —Downloadable report

Project Structure

text
ai_human_project2_llm/
├── app.py
├── train_models.py
├── requirements.txt
├── README.md
├── demo_script.md
├── .gitignore
├── models/
│   ├── svm_model.pkl
│   ├── decision_tree_model.pkl
│   ├── adaboost_model.pkl
│   ├── fnn_model.h5
│   ├── lstm_model.h5
│   ├── cnn_model.h5
│   ├── tfidf_vectorizer.pkl
│   ├── tfidf_selector.pkl
│   ├── linguistic_scaler.pkl
│   ├── linguistic_feature_names.pkl
│   ├── model_results.csv
│   └── sequence_config.json
├── src/
│   ├── document_loader.py
│   ├── text_utils.py
│   └── llm_utils.py
├── notebooks/
│   └── project1_notebook.ipynb
└── data/
    └── training_data/

Local Setup

Install dependencies:

bash
pip install -r requirements.txt

Run the app:

bash
streamlit run app.py

Hugging Face Spaces Deployment

  1. 1.Create a new Hugging Face Space.
  2. 2.Choose Streamlit as the SDK.
  3. 3.Upload all files and folders from this project.
  4. 4.In the Space settings, add a secret named HF_TOKEN with your Hugging Face access token.
  5. 5.Restart the Space.

The app still runs without HF_TOKEN, but it will use transparent fallback explanations instead of live LLM calls. Adding the token enables the two live Hugging Face LLM features.

Results

The saved Project 1 model results show that SVM and FNN were the strongest performers. In the included results file, SVM reached about 97% accuracy, and the FNN reached about 96.8% accuracy. AdaBoost and CNN also performed well, while the LSTM was less accurate on this dataset.

What I Learned

This project shows that traditional machine learning models such as SVM can perform very well on AI-vs-human text classification when TF-IDF and linguistic features are used. The LLM additions make the app more useful because they explain the prediction and connect the classification result to writing style. The deployment step also demonstrates how ML models and LLM services can be combined into one cloud-based AI application.

Demo Video Checklist

For the demo video:

  1. 1.Show the project folder and README.
  2. 2.Run or open the Streamlit app.
  3. 3.Paste a human-written sample and show the prediction.
  4. 4.Paste an AI-generated sample and show the prediction.
  5. 5.Show the model comparison table.
  6. 6.Show the Phi-3 explanation section.
  7. 7.Show the Qwen rewrite section.
  8. 8.Download the report.
  9. 9.Briefly explain the Hugging Face Spaces deployment.