Ugenz Academy/Tutorial

Exploratorydataanalysis

7 min read1,377 words

Reading progress

Keep learning

Exploratory Data Analysis (EDA)

Introduction

Exploratory Data Analysis (EDA) is the process of examining, summarizing, and visualizing a dataset to understand its main characteristics, patterns, relationships, and unusual values.

EDA is an important step in Data Science because it helps us understand the data before applying statistical methods or building machine learning models.

EDA is the process of exploring and understanding data using statistics and visualizations.


Why is EDA Important?

EDA helps Data Scientists to:

  • Understand the structure of data.
  • Find patterns and trends.
  • Identify missing values.
  • Detect duplicate records.
  • Find outliers.
  • Understand relationships between variables.
  • Discover errors or unusual values.
  • Select useful features.
  • Choose appropriate analysis or machine learning techniques.

EDA Process

A typical EDA process is:

Collect Data ↓ Load Data ↓ Understand Data ↓ Clean Data ↓ Statistical Analysis ↓ Data Visualization ↓ Find Patterns & Relationships ↓ Draw Insights

1. Understand the Dataset

The first step is to understand what the dataset contains.

Important questions include:

  • How many rows are there?
  • How many columns are there?
  • What are the column names?
  • What type of data does each column contain?
  • Are there missing values?
  • Are there duplicate records?

Python Example

import pandas as pd data = pd.read_csv("sales.csv") print(data.head()) print(data.shape) print(data.columns) print(data.info())

2. View the First and Last Records

First Five Rows

print(data.head())

Last Five Rows

print(data.tail())

These functions help us quickly inspect the dataset.


3. Check Dataset Size

The shape property gives the number of rows and columns.

print(data.shape)

Example output:

(1000, 6)

This means:

  • 1000 rows
  • 6 columns

4. Check Data Types

Different columns can contain different types of data.

print(data.dtypes)

Common data types include:

  • Integer
  • Float
  • String
  • Boolean
  • Date/Time

Correct data types are important for analysis.


5. Check Missing Values

Missing values can affect analysis.

print(data.isnull().sum())

Example:

Name 0 Age 5 City 2 Salary 0

This tells us that:

  • Age has 5 missing values.
  • City has 2 missing values.

6. Statistical Summary

The describe() function provides statistical information about numerical columns.

print(data.describe())

It can provide:

  • Count
  • Mean
  • Standard deviation
  • Minimum
  • 25th percentile
  • Median
  • 75th percentile
  • Maximum

Example:

Age Salary count 100.0 100.0 mean 23.5 35000.0 min 18.0 20000.0 max 40.0 80000.0

7. Measures of Central Tendency

EDA often uses measures that describe the center of the data.

Mean

The mean is the average value.

Formula:

Mean = Sum of all values / Number of values

Python:

print(data["Salary"].mean())

Median

The median is the middle value after sorting the data.

print(data["Salary"].median())

Mode

The mode is the most frequently occurring value.

print(data["City"].mode())

8. Measures of Dispersion

Dispersion describes how spread out the data is.

Common measures include:

  • Range
  • Variance
  • Standard deviation

Standard Deviation

print(data["Salary"].std())

A larger standard deviation generally indicates greater variation in the values.


9. Data Visualization

Visualization is an important part of EDA.

It helps us understand data more easily through graphs and charts.

Common visualizations include:

  • Bar chart
  • Line chart
  • Histogram
  • Box plot
  • Scatter plot
  • Pie chart
  • Heatmap

10. Histogram

A histogram shows the distribution of numerical data.

Example

import matplotlib.pyplot as plt plt.hist(data["Age"]) plt.xlabel("Age") plt.ylabel("Frequency") plt.title("Age Distribution") plt.show()

A histogram can help identify the shape and spread of a distribution.


11. Bar Chart

A bar chart is useful for comparing categories.

Example

data["City"].value_counts().plot(kind="bar") plt.xlabel("City") plt.ylabel("Number of Customers") plt.title("Customers by City") plt.show()

12. Line Chart

A line chart is useful for showing changes over time.

Example

plt.plot(data["Date"], data["Sales"]) plt.xlabel("Date") plt.ylabel("Sales") plt.title("Sales Trend") plt.show()

13. Scatter Plot

A scatter plot helps examine the relationship between two numerical variables.

Example

plt.scatter(data["Age"], data["Salary"]) plt.xlabel("Age") plt.ylabel("Salary") plt.title("Age vs Salary") plt.show()

It can help identify whether there is a relationship or pattern between two variables.


14. Box Plot

A box plot helps understand the distribution of data and identify potential outliers.

plt.boxplot(data["Salary"]) plt.ylabel("Salary") plt.title("Salary Distribution") plt.show()

15. Correlation Analysis

Correlation measures the strength and direction of a relationship between numerical variables.

The correlation coefficient generally ranges from:

-1 to +1
  • +1 → Strong positive relationship
  • 0 → Little or no linear relationship
  • -1 → Strong negative relationship

Python Example

print(data.corr(numeric_only=True))

16. Correlation Heatmap

A heatmap provides a visual representation of correlations.

import seaborn as sns import matplotlib.pyplot as plt sns.heatmap( data.corr(numeric_only=True), annot=True ) plt.title("Correlation Heatmap") plt.show()

17. Detecting Outliers

Outliers are values that are unusually different from the rest of the data.

Example

Salary: 25000 28000 30000 32000 5000000

The value 5000000 may be an outlier.

A box plot is commonly used to identify potential outliers.

plt.boxplot(data["Salary"]) plt.show()

Outliers should be investigated before deciding whether to remove or transform them.


18. Analyzing Categorical Data

Categorical variables contain groups or labels.

Examples:

Gender City Product Department

We can count categories using:

print(data["City"].value_counts())

Example:

Janakpur 50 Kathmandu 40 Birgunj 30

19. Example Dataset

Consider the following sales dataset:

DateProductCityQuantityPriceTotal
3-AprLaptopJanakpur250000100000
6-JunMouseKathmandu10100010000
2-MarLaptopBirgunj18000080000
2-AprKeyboardJanakpur51200060000

Using EDA, we can find:

  • Which product has the highest sales.
  • Which city has the highest sales.
  • Total quantity sold.
  • Average product price.
  • Sales trends.
  • Relationship between quantity and total sales.

20. Simple EDA Example in Python

import pandas as pd import matplotlib.pyplot as plt # Load data data = pd.read_csv("sales.csv") # Basic information print(data.head()) print(data.shape) print(data.info()) # Check missing values print(data.isnull().sum()) # Statistical summary print(data.describe()) # Product frequency print(data["Product"].value_counts()) # Total sales by product sales_by_product = data.groupby("Product")["Total"].sum() print(sales_by_product) # Visualization sales_by_product.plot(kind="bar") plt.xlabel("Product") plt.ylabel("Total Sales") plt.title("Sales by Product") plt.show()

EDA Tools

Common tools used for Exploratory Data Analysis include:

ToolPurpose
PythonData analysis and visualization
PandasData manipulation
NumPyNumerical calculations
MatplotlibData visualization
SeabornStatistical visualization
ExcelBasic data exploration
Power BIInteractive analysis and dashboards
RStatistical analysis and visualization

EDA vs Data Cleaning

EDAData Cleaning
Focuses on understanding the dataFocuses on improving data quality
Finds patterns and relationshipsHandles missing and incorrect values
Uses statistics and visualizationCorrects errors and inconsistencies
Helps generate insightsPrepares data for analysis
Can identify potential problemsResolves identified data-quality problems

Both activities are closely related and may be performed iteratively.


Benefits of EDA

  • Helps understand the dataset.
  • Identifies patterns and trends.
  • Detects missing values.
  • Finds potential outliers.
  • Reveals relationships between variables.
  • Helps select useful features.
  • Supports better model selection.
  • Improves understanding before machine learning.

Conclusion

Exploratory Data Analysis (EDA) is an important step in Data Science that helps us understand data before making conclusions or building machine learning models.

EDA combines statistical techniques, data summaries, and visualizations to discover patterns, relationships, trends, missing values, and unusual observations.

EDA helps answer the question: "What does my data tell me?"

Keep learning. Keep building. Keep growing.

© 2026 Ugenz Academy