Probability and Statistics
Introduction
Probability and Statistics are important mathematical foundations of Data Science. They help us understand data, measure uncertainty, identify patterns, make predictions, and support data-driven decisions.
- Probability deals with the likelihood of events occurring.
- Statistics deals with collecting, organizing, analyzing, interpreting, and presenting data.
Probability helps us understand what may happen, while Statistics helps us understand what has happened based on data.
1. Probability
What is Probability?
Probability is a mathematical measure of how likely an event is to occur.
The probability of an event is between 0 and 1.
0 ≤ P(A) ≤ 1
Where:
0= impossible event1= certain event- Values between 0 and 1 = different levels of likelihood
Formula
For equally likely outcomes:
P(A) = Number of favorable outcomes / Total number of possible outcomes
Example
When a fair coin is tossed:
Possible outcomes = {Head, Tail}
Probability of getting Head:
P(Head) = 1 / 2 = 0.5
2. Basic Probability Terms
Experiment
An experiment is an action or process that produces an outcome.
Example:
Rolling a dice
Sample Space
The sample space is the set of all possible outcomes.
For a six-sided dice:
S = {1, 2, 3, 4, 5, 6}
Event
An event is one or more outcomes of an experiment.
Example:
Getting an even number:
A = {2, 4, 6}
3. Types of Probability
Classical Probability
Used when outcomes are equally likely.
Example:
P(Head) = 1/2
Empirical Probability
Based on observations or experiments.
Example:
If a coin is tossed 100 times and Head appears 55 times:
P(Head) = 55/100 = 0.55
Subjective Probability
Based on available knowledge, judgment, or belief rather than only direct mathematical calculation.
4. Important Probability Rules
Addition Rule
For events A and B:
P(A ∪ B) = P(A) + P(B) - P(A ∩ B)
For mutually exclusive events:
P(A ∪ B) = P(A) + P(B)
Multiplication Rule
For dependent events:
P(A ∩ B) = P(A) × P(B|A)
For independent events:
P(A ∩ B) = P(A) × P(B)
5. Conditional Probability
Conditional probability is the probability of an event occurring given that another event has already occurred.
Formula:
P(A|B) = P(A ∩ B) / P(B)
Example
Suppose we want to find the probability that a student passes an exam given that the student attended all classes.
The previous attendance information becomes the condition.
6. Bayes' Theorem
Bayes' Theorem is used to update the probability of an event when new information becomes available.
Formula:
P(A|B) = [P(B|A) × P(A)] / P(B)
Bayes' theorem is widely used in:
- Medical diagnosis
- Spam filtering
- Fraud detection
- Machine learning
- Risk analysis
7. Random Variable
A random variable is a variable whose value depends on the outcome of a random experiment.
Example
When a dice is rolled:
X = Number obtained on the dice
Possible values are:
X = {1, 2, 3, 4, 5, 6}
8. Probability Distribution
A probability distribution describes the probabilities associated with possible values of a random variable.
Two major types are:
Discrete Distribution
Used for countable values.
Examples:
- Number of customers
- Number of defective products
- Number of heads
Continuous Distribution
Used for values that can take any value within a range.
Examples:
- Height
- Weight
- Temperature
- Time
9. Normal Distribution
The normal distribution is one of the most important probability distributions in Statistics and Data Science.
It has a bell-shaped curve.
* * * * * * * * * * * -------|---------|---------|--------> μ-σ μ μ+σ
Important properties:
- Symmetric around the mean.
- Mean = Median = Mode.
- Many natural measurements approximately follow a normal distribution.
Examples include:
- Heights
- Measurement errors
- Some examination scores
- Certain biological measurements
10. Statistics
What is Statistics?
Statistics is the science of collecting, organizing, analyzing, interpreting, and presenting data.
Statistics helps us convert large amounts of data into meaningful information.
11. Types of Statistics
There are two major types:
Descriptive Statistics
Descriptive statistics summarizes and describes existing data.
Examples:
- Mean
- Median
- Mode
- Range
- Variance
- Standard deviation
- Tables
- Charts
Inferential Statistics
Inferential statistics uses sample data to make conclusions or estimates about a larger population.
Examples:
- Hypothesis testing
- Confidence intervals
- Regression
- Estimation
12. Population and Sample
Population
A population is the complete group being studied.
Example:
All students in a university.
Sample
A sample is a smaller group selected from the population.
Example:
500 students selected from the university.
Population ┌──────────────────────────────┐ │ │ │ ┌──────────────┐ │ │ │ Sample │ │ │ │ │ │ │ └──────────────┘ │ │ │ └──────────────────────────────┘
13. Mean
The mean is the average of a set of values.
Formula:
Mean = Sum of values / Number of values
Example
Data:
10, 20, 30, 40, 50
Mean:
Mean = (10 + 20 + 30 + 40 + 50) / 5 = 150 / 5 = 30
Python:
data = [10, 20, 30, 40, 50] mean = sum(data) / len(data) print(mean)
14. Median
The median is the middle value when data is arranged in order.
Example:
10, 20, 30, 40, 50
Median:
30
For an even number of observations, the median is the average of the two middle values.
15. Mode
The mode is the value that occurs most frequently.
Example:
10, 20, 20, 30, 40
Mode:
20
Python:
from statistics import mode data = [10, 20, 20, 30, 40] print(mode(data))
16. Range
The range measures the difference between the largest and smallest values.
Formula:
Range = Maximum - Minimum
Example:
10, 20, 30, 40, 50 Range = 50 - 10 = 40
17. Variance
Variance measures how far values are spread from the mean.
A higher variance indicates greater variability.
Population variance formula:
σ² = Σ(X - μ)² / N
Where:
X= individual valueμ= population meanN= number of observations
18. Standard Deviation
Standard deviation measures the amount of variation or dispersion in a dataset.
Formula:
σ = √Variance
A smaller standard deviation means values are generally closer to the mean.
A larger standard deviation means values are more spread out.
Python:
import statistics data = [10, 20, 30, 40, 50] print(statistics.stdev(data))
19. Correlation
Correlation measures the strength and direction of a linear relationship between two variables.
The correlation coefficient generally ranges from:
-1 to +1
| Value | Meaning |
|---|---|
| +1 | Perfect positive linear relationship |
| 0 | No linear relationship |
| -1 | Perfect negative linear relationship |
Example
If study time increases and examination scores also tend to increase, the two variables may have a positive correlation.
Correlation does not by itself prove that one variable causes the other.
20. Regression
Regression is a statistical method used to model relationships between variables and can be used for prediction.
Example
A company may use previous sales data to estimate future sales.
Simple linear regression:
y = mx + c
Where:
y= predicted valuex= input variablem= slopec= intercept
21. Hypothesis Testing
Hypothesis testing is a statistical method used to evaluate a claim about a population using sample data.
Important Terms
- Null hypothesis (H₀)
- Alternative hypothesis (H₁)
- Significance level
- Test statistic
- p-value
Example
A company wants to determine whether a