Apppublic

chemiomani/static-malware-detector

sourceHugging Faceupdated 3mo agoView on Hugging Face

0likes

App README

🔒 Static PE Malware Detection

![CI/CD Pipeline](https://github.com/chemiomani/static-malware-detector/actions/workflows/ci-cd.yml) ![Hugging Face Spaces](https://chemiomani-static-malware-detector.hf.space) ![Python 3.11](https://www.python.org/) ![Flask](https://flask.palletsprojects.com/) ![XGBoost](https://xgboost.readthedocs.io/) ![React](https://reactjs.io/) ![License: MIT](https://opensource.org/licenses/MIT)

🎓 Quantic School of Business and Technology Introduction to Machine Learning - Project

Machine learning-based static malware detection for Windows Portable Executable (PE) files. This application analyzes PE headers to classify files as malware or goodware without executing them.

Author: Chemi Omani

🎯 Live Demo

Try it now: https://chemiomani-static-malware-detector.hf.space

📊 Model Performance

Metric	Score
Accuracy	98.56%
AUC-ROC	0.9972
Precision	98.59%
Recall	98.92%
F1 Score	98.75%

Feature Importance

The model uses 8 key PE header features:

Feature	Importance
DllCharacteristics	56.15%
AddressOfEntryPoint	14.96%
SectionsNb	9.21%
CheckSum	7.46%
SizeOfCode	6.95%
SizeOfUninitializedData	5.28%
Machine	0.00%
SizeOfOptionalHeader	0.00%

✨ Features

—🔍 Manual PE Inspection - Enter PE header parameters manually or upload a file
—📊 Batch CSV Analysis - Upload multiple samples with confusion matrix
—🤖 XGBoost Model - 98.56% accuracy on 50,000+ samples
—🎯 Real-time Predictions - Instant classification with confidence scores
—📈 Performance Metrics - Accuracy, AUC-ROC, F1 score, confusion matrix
—🎨 Modern UI - Pink dark theme with responsive design
—🔄 CI/CD Pipeline - Automated testing and deployment via GitHub Actions

🏗️ System Architecture

┌─────────────────────────────────────────────────────────────────────┐
│                         USER INTERFACE                              │
│  ┌─────────────────────┐     ┌─────────────────────────────────┐   │
│  │  Manual PE Inspector │     │    CSV Batch Profiler           │   │
│  │  - Feature Input     │     │    - File Upload                │   │
│  │  - Preset Selection  │     │    - Batch Predictions          │   │
│  │  - Real-time Results │     │    - Metrics & Confusion Matrix │   │
│  └─────────────────────┘     └─────────────────────────────────┘   │
└─────────────────────────────────────────────────────────────────────┘
                                    │
                                    ▼
┌─────────────────────────────────────────────────────────────────────┐
│                         FLASK API SERVER                            │
│  ┌─────────────────────────────────────────────────────────────┐   │
│  │  Endpoints:                                                  │   │
│  │  /api/health        - Health check                          │   │
│  │  /api/predict       - Single prediction                     │   │
│  │  /api/predict-batch - Batch prediction from CSV             │   │
│  │  /api/model-info    - Model information                     │   │
│  │  /api/dataset       - Dataset serving (for training UI)     │   │
│  └─────────────────────────────────────────────────────────────┘   │
└─────────────────────────────────────────────────────────────────────┘
                                    │
                                    ▼
┌─────────────────────────────────────────────────────────────────────┐
│                         MODEL PIPELINE                              │
│  ┌─────────────────────────────────────────────────────────────┐   │
│  │  Input → Imputer (Median) → Scaler (Standard) → XGBoost    │   │
│  └─────────────────────────────────────────────────────────────┘   │
└─────────────────────────────────────────────────────────────────────┘

Training Pipeline

┌─────────────────────────────────────────────────────────────────────┐
│                         TRAINING PIPELINE                           │
│                                                                      │
│  ┌──────────────┐     ┌──────────────┐     ┌─────────────────────┐  │
│  │   Dataset    │────▶│   80/20      │────▶│  Stratified 5-Fold  │  │
│  │  50,181      │     │   Split      │     │  Cross-Validation   │  │
│  └──────────────┘     └──────────────┘     └─────────────────────┘  │
│                                                    │                │
│                                                    ▼                │
│                              ┌─────────────────────────────────────┐│
│                              │  For each fold:                     ││
│                              │  1. Impute missing values (Median) ││
│                              │  2. Scale features (StandardScaler)││
│                              │  3. Train model on fold            ││
│                              │  4. Evaluate on validation fold    ││
│                              └─────────────────────────────────────┘│
│                                                    │                │
│                                                    ▼                │
│                              ┌─────────────────────────────────────┐│
│                              │  Select Best Model (XGBoost)       ││
│                              └─────────────────────────────────────┘│
│                                                    │                │
│                                                    ▼                │
│                              ┌─────────────────────────────────────┐│
│                              │  Train Final Model on Full          ││
│                              │  Training Set (40,144 samples)     ││
│                              └─────────────────────────────────────┘│
│                                                    │                │
│                                                    ▼                │
│                              ┌─────────────────────────────────────┐│
│                              │  Evaluate on Hold-out Test Set     ││
│                              │  (10,037 samples)                  ││
│                              └─────────────────────────────────────┘│
└─────────────────────────────────────────────────────────────────────┘

📂 Project Structure

.
├── .github/
│   └── workflows/
│       └── ci-cd.yml                 # CI/CD Pipeline
├── backend/                           # ML & API workspace
│   ├── app.py                         # Flask API server (port 7860)
│   ├── model_pipeline.pkl             # Trained XGBoost model
│   ├── requirements.txt               # Python dependencies
│   ├── test_app.py                    # Unit & integration tests
│   └── train.py                       # Model training script
├── frontend/                          # React client application
│   ├── src/
│   │   ├── App.tsx                    # Main React component
│   │   ├── data.ts                    # Presets & configuration
│   │   ├── types.ts                   # TypeScript definitions
│   │   ├── main.tsx                   # React DOM entry point
│   │   └── index.css                  # Tailwind styles
│   ├── index.html                     # HTML entry point
│   ├── package.json                   # npm dependencies
│   ├── tsconfig.json                  # TypeScript configuration
│   └── vite.config.ts                 # Vite build configuration
├── dist/                              # Built React application
│   ├── index.html
│   └── assets/                        # Compiled JS & CSS
├── .gitignore                         # Version control exclusions
├── Dockerfile                         # Hugging Face Spaces deployment
└── README.md                          # Project documentation

🚀 Quick Start

Prerequisites

—Python 3.11+
—Node.js 18+
—Git
—Dataset: Brazilian Malware Dataset

Local Development

bash

# 1. Clone the repository
git clone git@github.com:chemiomani/static-malware-detector.git
cd static-malware-detector

# 2. Set up Python backend
cd backend
python -m venv venv
source venv/bin/activate  # On Windows: venv\Scripts\activate
pip install -r requirements.txt

# 3. Download the dataset
wget -O brazilian-malware.zip https://github.com/fabriciojoc/brazilian-malware-dataset/raw/master/brazilian-malware.zip
unzip -j brazilian-malware.zip "*.csv" -d .
rm -f brazilian-malware.zip

# 4. Train the model
python train.py

# 5. Start Flask server
python app.py

# 6. In another terminal, set up React frontend
cd ../frontend
npm install
npm run build

# 7. Open browser
open http://localhost:7860

Using Docker

bash

# Build the Docker image
docker build -t static-malware-detector .

# Run the container
docker run -p 7860:7860 static-malware-detector

Running Tests

bash

# Python tests
cd backend
pytest test_app.py -v

# Expected output: 15 passed

🧠 How It Works

1. Feature Mapping

The application maps UI feature names to dataset column names:

UI Feature	Dataset Column
CheckSum	SizeOfImage
SectionsNb	NumberOfSections
DllCharacteristics	DllCharacteristics
AddressOfEntryPoint	ImageBase
SizeOfUninitializedData	SizeOfUninitializedData
SizeOfCode	SizeOfCode
Machine	Machine
SizeOfOptionalHeader	SizeOfOptionalHeader

2. Model Pipeline

Input Features → Imputer (Median) → Scaler (Standard) → XGBoost → Prediction

3. Prediction Flow

User Input → Flask API → Model Pipeline → Prediction → UI Display

4. Batch Processing Flow

CSV Upload → Parse Features → Batch Predictions → Calculate Metrics → Display Results

📊 Dataset

The model was trained on the Brazilian Malware Dataset:

—Samples: 50,181
—Features: 27 PE header attributes
—Class Balance: 57.1% Malware, 42.9% Goodware
—Source: Brazilian Malware Dataset

Citation

bibtex

@ARTICLE{8636415,
  author={F. {Ceschin} and F. {Pinage} and M. {Castilho} and D. {Menotti} and L. S. {Oliveira} and A. {Gregio}},
  journal={IEEE Security Privacy},
  title={The Need for Speed: An Analysis of Brazilian Malware Classifiers},
  year={2018},
  volume={16},
  number={6},
  pages={31-41},
  doi={10.1109/MSEC.2018.2875369}
}

🛠️ Technology Stack

Backend

—Flask - Web framework
—XGBoost - Gradient boosting classifier
—scikit-learn - Preprocessing and metrics
—pandas - Data manipulation
—numpy - Numerical computing

Frontend

—React - UI framework
—TypeScript - Type-safe JavaScript
—Tailwind CSS - Styling
—Vite - Build tool
—Lucide React - Icons

Deployment

—Docker - Containerization
—Hugging Face Spaces - Hosting
—GitHub Actions - CI/CD

🔬 Testing

bash

# Run Python tests
cd backend
pytest test_app.py -v

# Expected output:
# ✅ test_model_loaded PASSED
# ✅ test_predict_goodware PASSED
# ✅ test_predict_malware PASSED
# ✅ test_predict_missing_features PASSED
# ✅ test_health_endpoint PASSED
# ✅ test_model_info_endpoint PASSED
# ✅ test_predict_endpoint_goodware PASSED
# ✅ test_predict_endpoint_malware PASSED
# ✅ test_predict_endpoint_missing_features PASSED
# ✅ test_demo_endpoint PASSED
# ✅ test_predict_batch_endpoint PASSED
# ✅ test_predict_batch_no_file PASSED
# ✅ test_invalid_predict_payload PASSED
# ✅ test_health_endpoint_model_status PASSED
# ✅ test_holdout_demo_endpoint PASSED

📈 Performance Benchmarks

Model	Accuracy	AUC-ROC	Training Time
Logistic Regression	81.42%	0.8770	Fast
Decision Tree	97.71%	0.9761	Fast
Random Forest	98.73%	0.9975	Medium
XGBoost	98.34%	0.9975	Medium
LightGBM	98.29%	0.9974	Medium
CatBoost	97.82%	0.9959	Slow

🎯 Preset Configurations

Malware Preset

json

{
  "CheckSum": 0,
  "SectionsNb": 6,
  "DllCharacteristics": 0,
  "AddressOfEntryPoint": 100000,
  "SizeOfUninitializedData": 0,
  "SizeOfCode": 500000,
  "Machine": 332,
  "SizeOfOptionalHeader": 224
}

Confidence: 88.3%

Goodware Preset

json

{
  "CheckSum": 200000,
  "SectionsNb": 4,
  "DllCharacteristics": 50000,
  "AddressOfEntryPoint": 100000,
  "SizeOfUninitializedData": 0,
  "SizeOfCode": 80000,
  "Machine": 34404,
  "SizeOfOptionalHeader": 224
}

Confidence: 47.5%

🔧 Troubleshooting

Common Issues

SSH Authentication Error

bash

# If you get SSH errors, use explicit key
GIT_SSH_COMMAND="ssh -i ~/.ssh/id_rsa_chemiomani" git push origin main

Dataset Download Fails

bash

# Manual download
wget -O backend/brazilian-malware.zip https://github.com/fabriciojoc/brazilian-malware-dataset/raw/master/brazilian-malware.zip
cd backend && unzip -j brazilian-malware.zip "*.csv" -d .

Model Not Found

bash

# Train the model
cd backend && python train.py

🤝 Contributing

Contributions are welcome! Please:

1.Fork the repository
2.Create a feature branch
3.Make your changes
4.Submit a pull request

📝 License

This project is licensed under the MIT License.

🙏 Acknowledgments

—Quantic School of Business and Technology - For providing the project framework and learning opportunity
—Brazilian Malware Dataset - For providing the training data
—Hugging Face - For hosting the application
—Instructors and Mentors - For guidance throughout the project

📞 Contact

Chemi Omani

—GitHub: @chemiomani
—Hugging Face: @chemiomani

🎓 This project was completed as a fulfillment of the Quantic School of Business and Technology - Introduction to Machine Learning Project

Made with ❤️ by Chemi Omani

![Hugging Face](https://chemiomani-static-malware-detector.hf.space) ![GitHub](https://github.com/chemiomani/static-malware-detector) ![CI/CD Pipeline](https://github.com/chemiomani/static-malware-detector/actions/workflows/ci-cd.yml)