Rohit-Sharma-45/AI-Dataset-Cleaner

🚀 Live Production Deployment: Click Here to Access the Live App on Hugging Face Spaces

📊 AI-Powered Automated Data Cleaning & EDA Engine

An autonomous MLOps data preprocessing pipeline built with Python, Pandas, and Streamlit. The system profiles raw, corrupt datasets, generates a structural data profile, and handles multi-threading states seamlessly. It feeds this profile into a low-temperature Gemini 2.5 API agent using a strict JSON schema to safely execute programmatic data cleaning routines without text hallucinations.

🌟 Core Engineering Features

• —Autonomous Mismatch Detection: Scans missing cells, duplicated entries, and data-type anomalies across any uploaded CSV file.
• —Deterministic AI-Decision Engine: Enforces a rigid structural schema onto the LLM, neutralizing prose hallucinations and generating programmatic execution steps.
• —Pandas Transformation Core: Automatically handles category optimization, vectorized text cleaning, and conditional mathematical missing-value imputation.
• —Statistical Outlier Filtering: Implements the Interquartile Range (IQR) formula to establish statistical fencing boundaries ($1.5 \times IQR$) and slice out extreme anomalies.

🛠️ Tech Stack

• —Core Engine: Python 3.10+, Pandas, NumPy
• —AI Agent Infrastructure: Google GenAI SDK (gemini-2.5-flash)
• —Dashboard Interface: Streamlit, Plotly Express

🚀 Local Installation & Setup

Follow these exact steps to spin up this engine on your local machine:

1. Clone the Blueprint Repository

git clone https://github.com/RohitXData/AI-Dataset-Cleaner.git
cd AI-Dataset-Cleaner

# Create your local venv folder
python -m venv venv

# Activate it (Windows PowerShell)
.\venv\Scripts\Activate.ps1

# Activate it (Mac or Linux)
source venv/bin/activate

pip install -r requirements.txt

GEMINI_API_KEY=your_actual_google_ai_studio_key

streamlit run app.py

1. Removed 1 completely identical duplicate rows.
2. Filled 2 nulls in 'age' with calculated median value (23.5).
3. Standardized text casing and removed hanging whitespace in column 'city'.
4. Detected and filtered out 1 extreme statistical outliers in numeric column 'salary' using IQR boundaries.