Arpitkr/fraud-detection-ui

1---2title: Fraud Detection UI3emoji: 🔍4colorFrom: red5colorTo: red6sdk: streamlit7app_file: app.py8pinned: false9---10 11# Credit Card Fraud Detection12 13**Live Demo**: [Fraud Detection UI](https://arpitkr-fraud-detection-ui.hf.space)  14**Live API**: [FastAPI Docs](https://fraud-api-kzt3.onrender.com/docs)15 16> If GitHub does not render the notebook properly, open the nbviewer link below for a clean static view.17 18>**Notebook View**: [fraud_clean.ipynb on nbviewer](https://nbviewer.org/github/A1r1p1it/credit-card-fraud-detection/blob/main/notebooks/fraud_clean.ipynb)19 20Binary classification system to detect fraudulent credit card transactions in a highly imbalanced dataset, with an interactive Streamlit UI, FastAPI backend, RAG-powered fraud explanation layer, and an **Agent Pipeline** with automated risk assessment and similar case retrieval.21 22## Problem Statement23 24Credit card fraud detection is a classic **imbalanced classification** problem. In this dataset, only 0.17% of transactions are fraudulent, which means a naive model that predicts every transaction as non-fraud can still achieve about 99.8% accuracy while catching no actual fraud.25 26Because of this, accuracy is not a meaningful metric here. The goal is to maximize fraud detection while minimizing false alarms that would hurt customer trust and operations.27 28## Dataset29 30- **Source**: Kaggle Credit Card Fraud Detection31- **Size**: 284,807 transactions32- **Features**: 30 total features33  - `V1`–`V28`: PCA-transformed anonymized features34  - `Time`35  - `Amount`36- **Target**: `Class`37  - `0` = Non-Fraud38  - `1` = Fraud39- **Fraud Cases**: 49240- **Fraud Rate**: 0.17%41 42## Approach43 44### 1. Data Preprocessing45 46- Stratified train-test split (80/20) to preserve class distribution47- StandardScaler for feature normalization48- Careful handling of class imbalance during training and evaluation49 50### 2. Models Compared51 52#### Logistic Regression53- Baseline model54- Used `class_weight='balanced'`55- High recall but poor precision56- Too many false positives for real-world usage57 58#### Random Forest59- 100 trees, `max_depth=5`60- Used `class_weight='balanced'`61- Better precision than Logistic Regression62- Still produced a high false alarm rate63 64#### XGBoost65- Best-performing model66- `n_estimators=200`67- `learning_rate=0.5`68- `scale_pos_weight=25`69- Strongest precision-recall balance for production-style fraud detection70 71### 3. Evaluation Strategy72 73Since the dataset is extremely imbalanced, the project uses:74- Precision75- Recall76- F1-score77- Precision-Recall AUC78- Confusion Matrix79 80Accuracy was intentionally not used as the primary metric because it is misleading in this setting.81 82## Results83 84| Model | Precision | Recall | F1-Score | PR-AUC | Trade-off |85|-------|-----------|--------|----------|--------|-----------|86| Logistic Regression | 0.06 | 0.92 | 0.11 | 0.764 | Catches most fraud but creates too many false alarms |87| Random Forest | 0.36 | 0.89 | 0.51 | 0.659 | Better balance, still too many false positives |88| XGBoost | 0.91 | 0.84 | 0.87 | 0.878 | Best production-style trade-off |89 90## Key Insight91 92XGBoost achieved the best balance between **precision** and **recall**, making it the most suitable model for deployment. It catches the majority of fraudulent transactions while keeping false positives low enough for a practical fraud detection workflow.93 94## Feature Importance95 96Top features identified by the tree-based models:97 981. `V14`992. `V10`1003. `V12`1014. `V17`1025. `V4`103 104These features are especially important in distinguishing fraud from normal transactions, even though the raw variables are anonymized through PCA.105 106## SQL Analysis107 108The dataset was also analyzed using SQLite for business and fraud-pattern insights:109 110- Overall fraud rate: **0.17%**111- High-value transactions (>$200) show roughly **2x higher fraud rate**112- Fraudulent transactions have a higher average amount than legitimate ones113- Peak fraud hours occur around **2AM–3AM**114 115## AI-Powered Features116 117### 1. LLM Fraud Explanation118When a transaction is predicted as fraud, the system generates a human-readable explanation using **LLaMA 3.1 8B via Groq**.119 120### 2. Context-Aware AI Chat121 122The application includes a conversational AI assistant that helps users understand fraud predictions, fraud patterns, and machine learning concepts.123 124Features include:125 126- Access to the latest prediction context (fraud probability, risk level, key feature values, and explanations)127- Retrieval-Augmented Generation (RAG) using the fraud knowledge base128- Multi-turn conversations with session-based chat history129- Grounded responses that combine model outputs with retrieved fraud knowledge130 131Example questions:132 133- "Why was this transaction flagged?"134- "What makes V14 suspicious?"135- "What fraud pattern does this transaction resemble?"136- "Why is PR-AUC more useful than accuracy for fraud detection?"137 138### 3. RAG-Based Fraud Knowledge Layer139The app includes a Retrieval-Augmented Generation pipeline so explanations are grounded in a curated fraud knowledge base instead of relying only on a direct LLM response.140 141### 4. Agent Pipeline142Every prediction now runs through a multi-step agent pipeline:143 144### 5. Natural Language Transaction Analysis145 146Users can describe a transaction in plain English instead of manually entering model features.147 148The system uses an LLM-powered feature extraction workflow to convert natural language descriptions into structured fraud signals before running the fraud detection pipeline.149 150Workflow:151 152Natural Language Description153→ LLM Feature Extraction154→ Structured Transaction Features155→ XGBoost Prediction156→ Risk Assessment157→ Similar Case Retrieval158→ RAG-Based Fraud Explanation159 160Example:161 162"Transaction of $3000 at 2AM from a new device in a foreign country"163 164The system automatically estimates relevant fraud indicators, generates structured model inputs, evaluates fraud risk, and produces a grounded explanation of the prediction.165 166| Step | Description |167|------|-------------|168| 1️ Predict | XGBoost model predicts fraud probability |169| 2️ Risk Level | Classified as HIGH / MEDIUM / LOW based on probability thresholds |170| 3️ Suggested Action | Automated recommendation (e.g. "Flag for manual review", "Block transaction") |171| 4️ Similar Cases | Retrieves past fraud cases from SQLite DB with matching risk profiles |172 173## RAG Upgrade174 175### Phase 1 — Build the Knowledge Base176 177Created `src/knowledge_base.py` with curated fraud-detection documents across multiple categories:178 179- **Feature-level rules**  180  Example: strong negative `V14`, `V10`, and `V12` values are closely associated with fraud risk patterns.181  182- **Fraud pattern descriptions**  183  Including:184  - Card-not-present fraud185  - POS skimming186  - Account takeover187  - Card testing188  - Friendly fraud / chargeback abuse189  - Synthetic identity fraud190 191- **Dataset-specific statistical context**  192  Such as fraud rate, peak fraud hours, amount-based risk patterns193 194- **General domain knowledge**  195  Fraud detection trade-offs, model interpretation context, and operational risk signals196 197### Phase 2 — Build the RAG Engine198 199Created `src/rag_engine.py` to retrieve the most relevant knowledge chunks before generating explanations.200 201Pipeline:202- Embed knowledge base documents using `sentence-transformers/all-MiniLM-L6-v2`203- Build vector representations for semantic search204- Retrieve the **top 3 most relevant chunks** for a fraud-related query205- Inject retrieved context into the Groq prompt before explanation generation206 207### Phase 3 — Update `app.py`208 209Upgraded from direct LLM explanation to a RAG-enhanced + Agent pipeline:210 211New UI features:212- 4-metric display: Fraud | Probability | Risk Level | Suggested Action213- Agent Pipeline table showing all decision steps214- Retrieved knowledge chunks shown in the interface215- Grounded fraud explanation generated from retrieved context216- Similar past fraud cases from persistent SQLite database217- **Knowledge Base** tab to browse all available fraud documents218- RAG-enhanced AI chat for fraud-related questions219 220## Project Structure221 222```bash223credit-card-fraud-detection/224│225├── data/226│   ├── creditcard.csv227│   └── fraud.db228│229├── notebooks/230│   ├── fraud.ipynb231│   └── main_experimental.py232│233├── src/234│   ├── __init__.py235│   ├── explainer.py236│   ├── knowledge_base.py237│   └── rag_engine.py238│239├── app.py240├── main.py241├── Best_model.pkl242├── Scaler.pkl243├── Dockerfile244├── Dockerfile.streamlit245├── requirements.txt246├── requirements_ui.txt247├── .env248│249└── README.md250```251 252## Tech Stack253 254- Python255- Pandas, NumPy256- Scikit-learn257- XGBoost258- SQLite259- FastAPI260- Pydantic261- Uvicorn262- Streamlit263- Docker264- Hugging Face Spaces265- Render266- Sentence Transformers267- Semantic Vector Retrieval268- Groq API269- LLaMA 3.1 8B270 271## Deployment272 273- **Frontend**: Streamlit app deployed on Hugging Face Spaces (`fraud-detection-ui`)274- **Backend**: FastAPI REST API deployed on Render (`fraud-detection-api`)275- **UI deployment**: Streamlit SDK with `app.py` as the entry point276- **API deployment**: Docker-based FastAPI container277- API returns:278  - `is_fraud` — boolean279  - `fraud_probability` — float280  - `risk_level` — HIGH / MEDIUM / LOW281  - `suggested_action` — automated recommendation string282  - `similar_cases` — list of past fraud cases from SQLite283  - `explanation` — RAG-grounded LLM explanation284 285## Key Learnings286 287- Accuracy is misleading for extreme class imbalance288- Precision-recall trade-offs matter more than raw accuracy289- XGBoost outperformed simpler baselines after imbalance-aware tuning290- High precision is critical because false positives damage user trust291- RAG improves explanation quality by grounding responses in curated fraud knowledge292- Agent pipelines add interpretability and automated decision support on top of ML predictions293- Separate Docker deployments per space prevent CMD conflicts in shared repos294 295## Future Improvements296 297- Replace in-memory semantic retrieval with a persistent vector index (e.g., FAISS)298- Add SHAP-based local explanations beside RAG explanations299- Add transaction history context for sequence-aware fraud analysis300- Introduce analyst feedback loops for continuous knowledge base refinement301- Add latency optimization and caching for faster inference302 303## Demo Questions304 305Try asking the app:306- "Why is V14 such a strong fraud signal?"307- "What does this transaction pattern suggest?"308- "What is account takeover fraud?"309- "Why is PR-AUC better than accuracy here?"310- "What makes high-value transactions riskier?"311 312## Author313 314**Arpit Kumar**  315AI/ML Engineer | Data Science | Fraud Detection | Applied ML Systems316