iqramulya/personal-finance-ml-dataset

Personal Finance Dataset – Data Cleaning & EDA

Overview

This project focuses on performing data cleaning, exploratory data analysis (EDA), and outlier detection on a synthetic personal finance dataset from Kaggle. The goal is to understand the dataset, detect patterns, identify anomalies, and prepare the data for further modeling or analysis.

Dataset Description

The dataset contains personal finance information.

• —Source: Kaggle Personal Finance ML Dataset

• —Size: The dataset contains 32,424 individual records.

• —Features: Key features include monthly income, monthly expenses, savings amount, loan amount, credit score, debt-to-income ratio, education level, employment status, region and other personal finance attributes.

1. Data Cleaning Process

The following data cleaning tasks were performed to ensure the dataset is ready for analysis.

Tasks performed:

• —Loaded and reviewed the dataset to understand its structure and content.
• —Checked for missing values and handled them appropriately.
• —Parsed date columns and converted them to proper datetime format.
• —Verified no illogical values exist (negative ages, negative income, invalid credit score ranges).
• —Checked for duplicate rows.
• —Ensured categorical values do not contain spelling errors.
• —Replaced non-applicable zero values in loan-related fields with NaN to prevent misleading analysis.

2. Exploratory Data Analysis (EDA)

Performed descriptive and visual analysis:

• —.describe() for understanding the statistical distribution.
• —Histograms of key numerical variables.
• —Correlation heatmap to explore relationships between variables.
• —Boxplots to visually inspect the spread and detect outliers.

Key insights include:

• —Financial attributes such as income, expenses, and savings show right-skewed distributions.
• —Some moderate correlations exist between expenses, income, and savings.

3. Outlier Detection

The boxplots revealed several key insights about the numerical features:

• —Financial variables such as income, expenses, savings, and loan amounts show a strong right skew with many high-value outliers.
• —The debttoincome_ratio column contains unrealistic extreme values, likely due to inconsistencies in the synthetic dataset. To avoid misleading interpretations, visualizations and analyses focus only on ratio values below 2.
• —More stable variables, including age, creditscore, and savingstoincomeratio, display normal distributions with relatively few outliers.

4. QUESTIONS & ANSWERS

1. Do people with higher incomes spend more money?

The chart shows a clear positive relationship between monthly income and monthly expenses: As income increases, expenses tend to increase as well.

People with higher incomes do spend more on average, but there is still wide variation within each income level — meaning not everyone increases their spending at the same rate.

2. Do people with higher incomes save more money?

The chart shows a strong positive relationship between monthly income and total savings.

Individuals with higher incomes tend to save significantly more in absolute terms. However, the spread widens as income rises — meaning high-income earners vary greatly in how much they save, from very little to exceptionally high amounts.

3. Does having more savings correlate with a higher credit score?

The plot shows the relationship between total savings and credit score.

There is no meaningful correlation between savings and credit score. People with both low and high savings appear across the full credit-score range, and the trend line is almost flat.

4. Do loan-taking rates differ across regions?

This chart shows the percentage of people who took a loan in each region.

Loan-taking rates are almost identical across all regions. Each region falls between 39% and 41%, indicating no meaningful regional effect on the likelihood of taking a loan.

5. Is there an income gap between genders?

This chart shows the average monthly income by gender.

Income levels across genders are nearly identical. All groups—Female, Male, and Other—earn an average of around $4,000 per month, with no meaningful differences.

6. Is there a relationship between income and financial risk level?

The chart compares income distributions between individuals with High Financial Risk and Low Financial Risk.

There is a clear relationship: Individuals with high financial risk tend to earn significantly lower incomes.

Their median income is lower, the overall distribution is shifted downward, and there are fewer high-income individuals in the high-risk group. This indicates that lower income is strongly associated with higher financial risk.

Overall Conclusion

The analysis shows that income is the primary factor influencing financial stability. Individuals with higher incomes tend to spend more, save more, and generally exhibit a lower level of financial risk. In contrast, factors such as gender and region show only minimal differences, from which no meaningful insights were derived.

Link to my video: https://www.loom.com/share/526643933e84475489a1258a3f3d2396