Modelpublic

rotemknat/israeli_supermarkets_pricing

sourceHugging Faceupdated 5mo agoView on Hugging Face

0likes

Model Card

🛒 Smart Shopping: Israeli Supermarket Price Predictor

Welcome! This project is an end-to-end Machine Learning exploration of the Israeli Supermarkets 2024 dataset. We’ve built a robust pipeline that doesn't just look at prices, but understands the patterns behind them—predicting exact costs and categorizing items into "Low," "Medium," and "High" price tiers.

📺 Project Walkthrough

Want to see the logic in action? Check out the video presentation below:

📊 The Data Story

—Source: Israeli Supermarkets 2024 (Kaggle)
—The Scale: We started with a massive pool of 10M+ rows. Through careful selection and cleaning, we focused on a high-quality sample of ~39,000 rows to keep our models sharp and efficient.
—Core Features: We analyzed everything from itemcode and manufacturer to unitofmeasure and temporal data.
—The Mission: Master the art of predicting itemprice (Regression) and price-tier categorization (Classification).

🛠️ Exploratory Data Analysis (EDA)

Before the "learning" starts, we had to clean up the noise. Our EDA focused on:

—Filtering Noise: Dropped columns with >90% missing data and removed 10k+ duplicate entries.
—Outlier Defense: We used the IQR method to ensure our model wasn't distracted by extreme price anomalies.
—Sanity Checks: Removed impossible data points, like items with zero or negative quantities.
—Refining the Target: Capped itemprice at the 99th percentile to create a more stable learning target.

Item Price Distribution

Capped Item Price Distribution .png)

Item Price vs Quantity

🧪 Feature Engineering

We turned raw numbers into meaningful insights:

—📅 The "When": Extracted specific time features (Day of week, Month, etc.) to see if prices fluctuate on weekends or holidays.
—🏷️ The "Who": Applied Frequency Encoding to handle high-variety fields like item names and manufacturers.
—🤖 The "Groups": We used K-Means Clustering to let the data group itself. These "hidden" clusters became a powerful new feature for our models.

Elbow Method Plot

PCA Clusters

📈 Regression Models

We progressed from a simple baseline to high-performance ensemble models.

Baseline Model: Linear Regression

—Features: unitofmeasureprice, quantity, allowdiscount
—Metrics: MAE: (Value) | RMSE: (Value) | R2: (Value)

Baseline LR Actual vs Predicted

Baseline LR Feature Importance .png)

The Contenders (Improved Models)

1.Linear Regression (Improved): Leveraged our new engineered features.

1.Random Forest Regressor: Captured complex, non-linear relationships.

1.Gradient Boosting Regressor: Our top performer!

🏆 Regression Winner: Random Forest Regressor

Reasoning: It demonstrated the highest R2 score and lowest error metrics, proving that sequential learning is incredibly effective for retail price prediction.

🎯 Classification Models

We split the items into three balanced buckets: Low, Medium, and High using quantile binning.

Classification Class Distribution

The Classifiers

1.Logistic Regression: Our statistical baseline.

1.Random Forest Classifier: A robust ensemble approach.

1.Gradient Boosting Classifier: A strong competitor in precision.

🏆 Classification Winner: Random Forest Classifier

Reasoning: The Random Forest Classifier emerged as the winner with 90.31% accuracy. Its strength lies in its high F1-score, indicating a near-perfect balance between Precision (minimizing mislabeling) and Recall (ensuring all items in a tier are caught). While simpler models struggled with the overlapping boundaries of price tiers, Random Forest’s ensemble logic successfully decoded the complex interactions between manufacturers, categories, and shopping clusters.

💡 Key Considerations: Precision vs. Recall

In retail analytics, the "cost" of a mistake varies:

—Precision: If we label an item "High Price," how sure are we? High precision avoids over-pricing errors.
—Recall: Are we catching all the "High Price" items? High recall ensures no premium goods are missed.
—The Trade-off: (Add your specific critical error type here.)

🏁 Conclusion & Lessons Learned

—Feature Power: Data context (like K-Means clusters) often provides a bigger boost than just "tuning the knobs."
—Handling Scale: Processing millions of rows requires a balance of memory management and smart sampling.
—Iteration is Key: The path from a basic Linear Regression to a Gradient Boosting winner highlights the importance of the iterative Data Science lifecycle.

🧑‍💻 How to Use

To use the winning models from this project, you can download the .pkl files and load them using the following code:

python

import joblib
import pandas as pd

# 1. Load the Regression Winner (Random Forest)
regressor = joblib.load('random_forest_regressor_model.pkl')

# 2. Load the Classification Winner (Gradient Boosting)
classifier = joblib.load('gradient_boosting_classifier_model.pkl')