nahiar/tiktok-bot-detection-v2

TIKTOK Bot Detection Model

Overview

This directory contains a trained Random Forest classifier for detecting bot accounts on Tiktok.

Model Version: v2 Training Date: 2025-12-30 11:38:35 Framework: scikit-learn 1.5.2 Algorithm: Random Forest Classifier with GridSearchCV Hyperparameter Tuning

📊 Model Performance

Final Metrics (Test Set)

Model Improvement

• —Baseline ROC-AUC: 0.9759
• —Tuned ROC-AUC: 0.9773
• —Improvement: 0.0014 (0.14%)

🗂️ Files

🎯 Dataset Information

Training Configuration

• —Training Samples: 2,385
• —Test Samples: 596
• —Total Samples: 2,981
• —Number of Features: 12
• —Cross-Validation Folds: 5
• —Random State: 42

Class Distribution

Training Set:

• —Human (0): 951 (39.87%)
• —Bot (1): 1,434 (60.13%)

Test Set:

• —Human (0): 244 (40.94%)
• —Bot (1): 352 (59.06%)

🔧 Features (13)

1. 1.IsPrivate
2. 2.IsVerified
3. 3.HasProfilePic
4. 4.FollowingCount
5. 5.FollowerCount
6. 6.LikesCount
7. 7.HasInstagram
8. 8.HasYoutube
9. 9.HasBio
10. 10.HasLinkInBio
11. 11.HasPosts
12. 12.PostsCount
13. 13.FollowToFollowerRatio

🏆 Top 5 Most Important Features

1. 1.FollowToFollowerRatio - 0.2330
2. 2.LikesCount - 0.1771
3. 3.HasInstagram - 0.1395
4. 4.FollowingCount - 0.1349
5. 5.FollowerCount - 0.1055

⚙️ Hyperparameters

Best Parameters (from GridSearchCV)

• —class_weight: None
• —max_depth: 13
• —max_features: sqrt
• —min_samples_leaf: 2
• —min_samples_split: 10
• —n_estimators: 100

Parameter Search Space

• —n_estimators: [100, 200, 300]
• —max_depth: [10, 15, 20, None]
• —min_samples_split: [2, 5, 10]
• —min_samples_leaf: [1, 2, 4]
• —max_features: ['sqrt', 'log2']
• —bootstrap: [True, False]

Total combinations tested: 540

📈 Cross-Validation Results

Mean Scores (5-Fold Stratified CV)

• —Accuracy: 0.9191 (±0.0097)
• —Precision: 0.9326 (±0.0115)
• —Recall: 0.9331 (±0.0166)
• —F1-Score: 0.9327 (±0.0083)
• —ROC-AUC: 0.9744 (±0.0055)

🖼️ Visualizations

All visualizations are saved in the images/ directory:

1. 1.01_class_distribution.png - Training/Test set class distribution
2. 2.02_feature_correlation.png - Feature correlation with target variable
3. 3.03_correlation_matrix.png - Feature correlation heatmap
4. 4.04_baseline_confusion_matrix.png - Baseline model confusion matrix
5. 5.05_baseline_roc_curve.png - Baseline ROC curve
6. 6.06_baseline_precision_recall.png - Baseline Precision-Recall curve
7. 7.07_baseline_feature_importance.png - Baseline feature importance
8. 8.08_cross_validation.png - Cross-validation score distribution
9. 9.09_tuned_confusion_matrix.png - Tuned model confusion matrix
10. 10.10_tuned_roc_curve.png - Tuned ROC curve
11. 11.11_tuned_precision_recall.png - Tuned Precision-Recall curve
12. 12.12_tuned_feature_importance.png - Tuned feature importance
13. 13.13_model_comparison.png - Baseline vs Tuned comparison

🚀 Usage Example

import joblib
import pandas as pd
import numpy as np

# Load model and scaler
model = joblib.load('tiktok_bot_detection_v2.pkl')
scaler = joblib.load('tiktok_scaler_v2.pkl')

# Prepare your data (example)
data = {
    'IsPrivate': 0.5,
    'IsVerified': 0.5,
    'HasProfilePic': 0.5,
    'FollowingCount': 0.5,
    'FollowerCount': 0.5,
    'LikesCount': 0.5,
    'HasInstagram': 0.5,
    'HasYoutube': 0.5,
    'HasBio': 0.5,
    'HasLinkInBio': 0.5,
    'HasPosts': 0.5,
    'PostsCount': 0.5,
    'FollowToFollowerRatio': 0.5,
}

# Create DataFrame
df = pd.DataFrame([data])

# Scale features
df_scaled = scaler.transform(df)

# Predict
prediction = model.predict(df_scaled)[0]
probability = model.predict_proba(df_scaled)[0]

print(f"Prediction: {'Bot' if prediction == 1 else 'Human'}")
print(f"Bot Probability: {probability[1]:.4f}")
print(f"Human Probability: {probability[0]:.4f}")

📋 Confusion Matrix Breakdown

Tuned Model (Test Set)

                Predicted
              Human    Bot
Actual Human      220      24
       Bot         18     334

• —True Negatives (TN): 220 (Correctly identified humans)
• —False Positives (FP): 24 (Humans incorrectly classified as bots)
• —False Negatives (FN): 18 (Bots incorrectly classified as humans)
• —True Positives (TP): 334 (Correctly identified bots)

🔍 Model Interpretation

Strengths

• —High ROC-AUC score (0.9754) indicates excellent discrimination capability
• —Balanced precision and recall for both classes
• —Robust cross-validation performance

Key Insights

1. 1.Top features drive bot classification effectively
2. 2.GridSearchCV improved performance over baseline by 0.25%
3. 3.Model generalizes well on unseen test data

📝 Notes

• —Feature Scaling: All features are scaled using MinMaxScaler to [0, 1] range
• —Missing Values: Filled with 0 during preprocessing
• —Class Balance: Imbalanced dataset
• —Model Type: Ensemble method resistant to overfitting

🔄 Model Updates

To retrain the model:

1. 1.Place new training data in ../data/train_tiktok.csv
2. 2.Run the training notebook: 5_enhanced_training.ipynb
3. 3.Update this README with new metrics

📧 Contact & Support

For questions or issues regarding this model, please refer to the main project documentation.

Generated: 2025-12-30 11:38:35 Notebook: 5_enhanced_training.ipynb Platform: Tiktok