Ugenz Academy/Tutorial

Linearregression

12 min read2,348 words

Reading progress

Keep learning

Linear Regression

1. Introduction

Linear Regression is a supervised machine learning and statistical technique used to study the relationship between variables and predict a continuous numerical value.

It finds the best-fitting straight line through a set of data points.

Simple Definition

Linear Regression is a method used to predict a continuous dependent variable based on one or more independent variables by fitting a straight line to the data.

Example

Suppose we want to predict a student's exam score based on the number of hours they study.

Study HoursExam Score
140
245
355
465
575

Linear Regression can learn the relationship between study hours and exam score and predict the score for a new number of study hours.


2. Basic Terms

Linear Regression uses two important types of variables.

Independent Variable

The variable used to make the prediction.

It is usually represented by:

X

Example

Study Hours

Dependent Variable

The variable that we want to predict.

It is usually represented by:

Y

Example

Exam Score

Therefore:

X = Study Hours Y = Exam Score

3. Linear Regression Formula

The basic equation of simple linear regression is:

Y = a + bX

Where:

  • Y = predicted value
  • X = independent variable
  • a = intercept
  • b = slope/coefficient

Another common notation is:

Y = β₀ + β₁X

Where:

  • β₀ = intercept
  • β₁ = coefficient/slope

4. Intercept

The intercept is the predicted value of Y when X = 0.

For example:

Y = 30 + 10X

Here:

Intercept = 30

If:

X = 0

Then:

Y = 30 + 10(0) Y = 30

So, the predicted value is 30.


5. Slope

The slope represents how much the dependent variable changes when the independent variable increases by one unit.

For example:

Y = 30 + 10X

Here:

Slope = 10

This means that for every 1-unit increase in X, the predicted value of Y increases by 10 units.


6. Example of Linear Regression

Suppose the regression equation is:

Y = 20 + 5X

If:

X = 4

Then:

Y = 20 + 5(4) Y = 20 + 20 Y = 40

Therefore:

Predicted Y = 40

7. Types of Linear Regression

There are two major types of Linear Regression.

7.1 Simple Linear Regression

Simple Linear Regression uses one independent variable to predict one dependent variable.

Formula

Y = a + bX

Example

Predict salary based on years of experience.

Experience → Salary

There is only one independent variable:

X = Experience

7.2 Multiple Linear Regression

Multiple Linear Regression uses two or more independent variables to predict a dependent variable.

Formula

Y = β₀ + β₁X₁ + β₂X₂ + β₃X₃ + ... + βₙXₙ

Example

Predict house price using:

  • House size
  • Number of bedrooms
  • Location
  • Age of house

The model can be represented as:

Price = β₀ + β₁(Size) + β₂(Bedrooms) + β₃(Age)

8. Difference Between Simple and Multiple Linear Regression

Simple Linear RegressionMultiple Linear Regression
Uses one independent variableUses two or more independent variables
Y = a + bXY = β₀ + β₁X₁ + β₂X₂ + ...
Easier to understandMore complex
Example: Salary vs ExperienceExample: Salary vs Experience + Education + Age
One predictorMultiple predictors

9. Best-Fit Line

Linear Regression attempts to find the best-fit line through the data points.

The line should represent the relationship between X and Y as closely as possible.

Y │ │ ● │ ● │ ● │ ● │ ● │________________________ X

The actual data points may not fall exactly on the line.

The difference between an actual value and its predicted value is called a residual or error.


10. Residual

A residual is the difference between the actual value and predicted value.

Formula

Residual = Actual Value - Predicted Value

Example

Suppose:

Actual value = 80 Predicted value = 75

Then:

Residual = 80 - 75 Residual = 5

11. Least Squares Method

Linear Regression commonly uses the Ordinary Least Squares (OLS) method to find the best-fitting line.

The method tries to minimize the sum of squared errors.

Sum of Squared Errors = Σ(Actual - Predicted)²

The model chooses coefficients that minimize this value.

Why square the errors?

Squaring:

  • Makes negative errors positive.
  • Gives larger errors more influence.
  • Provides a mathematical objective that can be optimized.

12. How Linear Regression Works

The general process is:

Collect Data ↓ Clean Data ↓ Select Variables ↓ Split Data ↓ Train Model ↓ Find Best-Fit Line ↓ Make Predictions ↓ Evaluate Model

13. Steps to Build a Linear Regression Model

Step 1: Collect Data

Collect relevant data from sources such as:

  • Databases
  • Excel
  • CSV files
  • APIs
  • Surveys

Step 2: Clean the Data

Check for:

  • Missing values
  • Duplicate records
  • Incorrect values
  • Outliers
  • Incorrect data types

Step 3: Select Variables

Identify:

Independent variable(s) → X Dependent variable → Y

Step 4: Split the Data

The dataset is commonly divided into:

Training Data Testing Data

For example:

80% → Training 20% → Testing

The exact split can vary depending on the problem.


Step 5: Train the Model

The model learns the relationship between the input variables and the target variable.


Step 6: Make Predictions

Use the trained model to predict values for new data.


Step 7: Evaluate the Model

Common evaluation metrics include:

  • MAE
  • MSE
  • RMSE
  • R²

14. Mean Absolute Error (MAE)

MAE measures the average absolute difference between actual and predicted values.

Formula

MAE = (1/n) Σ |Actual - Predicted|

Example

If prediction errors are:

2, 4, 6

Then:

MAE = (2 + 4 + 6) / 3 = 4

A lower MAE generally indicates smaller prediction errors.


15. Mean Squared Error (MSE)

MSE calculates the average squared prediction error.

Formula

MSE = (1/n) Σ(Actual - Predicted)²

Larger errors have a greater effect because the errors are squared.

A lower MSE generally indicates better predictive accuracy.


16. Root Mean Squared Error (RMSE)

RMSE is the square root of MSE.

Formula

RMSE = √MSE

RMSE is useful because its unit is the same as the target variable.

For example, if the target is measured in rupees, RMSE is also measured in rupees.


17. R-Squared (R²)

R² measures how much of the variation in the dependent variable is explained by the regression model.

It is commonly written as:

R²

A value closer to 1 indicates that the model explains a larger proportion of the observed variation in the data.

However, R² should not be used alone to determine whether a model is appropriate.


18. Assumptions of Linear Regression

Important assumptions include:

1. Linearity

The relationship between the predictors and the target should be reasonably linear.

2. Independence

Observations should generally be independent of one another.

3. Homoscedasticity

The spread of residuals should be reasonably constant across fitted values.

4. Normality of Residuals

For many statistical inference procedures, residuals are assumed to be approximately normally distributed.

5. Limited Multicollinearity

In multiple regression, independent variables should not be excessively correlated with each other.


19. Correlation vs Linear Regression

Correlation measures the strength and direction of a linear relationship between two variables.

Linear Regression models that relationship and can be used for prediction.

CorrelationLinear Regression
Measures relationshipModels relationship
Usually symmetric between variablesDistinguishes predictor and target
Correlation coefficient ranges from -1 to +1Produces an equation
Does not by itself define a prediction equationCan be used for prediction
Does not establish causationDoes not establish causation

20. Positive and Negative Relationships

Positive Relationship

When X increases, Y tends to increase.

X ↑ → Y ↑

Example:

Study Hours ↑ → Exam Score ↑

Negative Relationship

When X increases, Y tends to decrease.

X ↑ → Y ↓

Example:

Product Price ↑ → Demand ↓

The actual relationship depends on the data and context.


21. Linear Regression Using Python

The scikit-learn library provides tools for Linear Regression.

Installation

pip install scikit-learn

Example

import numpy as np from sklearn.linear_model import LinearRegression # Independent variable X = np.array([[1], [2], [3], [4], [5]]) # Dependent variable y = np.array([40, 50, 60, 70, 80]) # Create model model = LinearRegression() # Train model model.fit(X, y) # Predict prediction = model.predict([[6]]) print("Predicted value:", prediction[0])

The model learns the relationship between X and y and predicts the value for X = 6.


22. Finding Coefficients

We can find the intercept and slope using:

print("Intercept:", model.intercept_) print("Slope:", model.coef_[0])

The equation can then be represented as:

Y = Intercept + Slope × X

23. Linear Regression with a Dataset

Example using Pandas:

import pandas as pd from sklearn.model_selection import train_test_split from sklearn.linear_model import LinearRegression from sklearn.metrics import mean_absolute_error, mean_squared_error, r2_score # Load dataset data = pd.read_csv("data.csv") # Independent variable X = data[["Hours"]] # Dependent variable y = data["Score"] # Split data X_train, X_test, y_train, y_test = train_test_split( X, y, test_size=0.2, random_state=42 ) # Create model model = LinearRegression() # Train model model.fit(X_train, y_train) # Predict y_pred = model.predict(X_test) # Evaluate mae = mean_absolute_error(y_test, y_pred) mse = mean_squared_error(y_test, y_pred) rmse = mse ** 0.5 r2 = r2_score(y_test, y_pred) print("MAE:", mae) print("MSE:", mse) print("RMSE:", rmse) print("R²:", r2)

24. Example: House Price Prediction

Suppose we want to predict house prices.

Input:

House Size

Output:

House Price

Example data:

Size (sq.ft.)Price
5002000000
7002800000
9003600000
11004400000

The model can learn a relationship between house size and price.

After training, a new house size can be provided to estimate its price.


25. Applications of Linear Regression

Linear Regression is used in many areas.

Business

  • Sales forecasting
  • Revenue prediction
  • Demand analysis

Finance

  • Financial forecasting
  • Risk analysis
  • Investment analysis

Real Estate

  • House price prediction
  • Property valuation

Education

  • Score prediction
  • Student performance analysis

Healthcare

  • Predicting numerical health-related measurements
  • Analyzing relationships between variables

Marketing

  • Sales and advertising analysis
  • Customer behavior analysis

Economics

  • Economic forecasting
  • Relationship analysis

26. Advantages of Linear Regression

  1. Easy to understand.
  2. Simple to implement.
  3. Computationally efficient.
  4. Provides interpretable coefficients.
  5. Useful for numerical prediction.
  6. Works well when relationships are approximately linear.
  7. Can be used as a baseline model.
  8. Useful for statistical analysis.

27. Limitations of Linear Regression

  1. Assumes a linear relationship.
  2. Sensitive to outliers.
  3. Can be affected by multicollinearity.
  4. May perform poorly when relationships are strongly nonlinear.
  5. Requires appropriate data preparation.
  6. A high R² does not prove causation.
  7. Predictions outside the observed data range can be unreliable.

28. Linear Regression Example in Real Life

Suppose a business wants to predict monthly sales based on advertising expenditure.

Historical data:

Advertising CostSales
10,000100,000
20,000150,000
30,000210,000
40,000270,000

The business can train a Linear Regression model using:

X = Advertising Cost Y = Sales

The model can then estimate sales for a new advertising budget.


29. Important Formulas

Linear Regression Equation

Y = a + bX

Multiple Linear Regression

Y = β₀ + β₁X₁ + β₂X₂ + ... + βₙXₙ

Residual

Residual = Actual - Predicted

MAE

MAE = (1/n) Σ|Actual - Predicted|

MSE

MSE = (1/n) Σ(Actual - Predicted)²

RMSE

RMSE = √MSE

30. Key Points to Remember

  • Linear Regression is a supervised learning technique.
  • It predicts a continuous numerical value.
  • It models a relationship using a straight line.
  • X represents the independent variable.
  • Y represents the dependent variable.
  • Simple Linear Regression uses one predictor.
  • Multiple Linear Regression uses multiple predictors.
  • The basic equation is Y = a + bX.
  • Least Squares is commonly used to find the best-fitting coefficients.
  • MAE, MSE, RMSE, and R² are common evaluation measures.
  • Linear Regression does not by itself prove causation.

Conclusion

Linear Regression is one of the fundamental techniques in statistics and machine learning. It is useful for understanding relationships between variables and predicting continuous numerical outcomes. Because its mathematical structure is relatively simple and its coefficients are interpretable, it is widely used as both a predictive model and a starting point for more advanced analysis.

Keep learning. Keep building. Keep growing.

© 2026 Ugenz Academy