Ugenz Academy/Tutorial

Logisticregression

13 min read2,499 words

Reading progress

Keep learning

Logistic Regression

1. Introduction

Logistic Regression is a supervised machine learning algorithm used mainly for classification problems.

Unlike Linear Regression, which predicts continuous numerical values, Logistic Regression predicts the probability that an observation belongs to a particular class.

Simple Definition

Logistic Regression is a classification algorithm that predicts the probability of an outcome using a logistic (sigmoid) function.

Examples

Logistic Regression can be used to predict:

  • Pass or Fail
  • Yes or No
  • Spam or Not Spam
  • Disease or No Disease
  • Fraud or Not Fraud
  • Customer Churn or No Churn

2. Linear Regression vs Logistic Regression

Linear Regression predicts a continuous numerical value.

Example:

House Price = Rs. 5,000,000

Logistic Regression predicts a class or probability.

Example:

Pass Probability = 0.85

Therefore:

Linear RegressionLogistic Regression
Used for regressionUsed mainly for classification
Predicts continuous valuesPredicts probabilities/classes
Output can be any numerical valueOutput probability is between 0 and 1
Uses a straight-line modelUses a logistic/sigmoid function
Example: Predict salaryExample: Predict pass/fail

3. Classification

Classification is a machine learning task where the model assigns data to predefined categories.

Example

Suppose we want to predict whether a student will pass.

Input: Study Hours Output: Pass / Fail

The classes are:

0 = Fail 1 = Pass

4. Binary Classification

Logistic Regression is commonly used for binary classification.

Binary classification has two possible classes.

Examples

Yes / No True / False Pass / Fail Spam / Not Spam Fraud / Not Fraud

For example:

0 → Fail 1 → Pass

5. Logistic Regression Formula

The logistic regression model uses the sigmoid function.

The sigmoid function is:

P(Y = 1) = 1 / (1 + e^(-z))

Where:

z = β₀ + β₁X₁ + β₂X₂ + ... + βₙXₙ

Therefore:

P(Y = 1) = 1 / (1 + e^-(β₀ + β₁X₁ + ... + βₙXₙ))

Where:

  • P(Y = 1) = probability of class 1
  • β₀ = intercept
  • β₁, β₂, ... = model coefficients
  • X₁, X₂, ... = input variables
  • e = Euler's number

6. Sigmoid Function

The sigmoid function converts any real-valued input into a value between 0 and 1.

Formula

σ(z) = 1 / (1 + e^-z)

Important properties

z → -∞ → sigmoid → 0 z = 0 → sigmoid = 0.5 z → +∞ → sigmoid → 1

The curve looks approximately like:

Probability 1.0 | ______ | __/ 0.5 |--------------__/ | ___/ 0.0 |__________/ +--------------------------> z

7. Why Sigmoid Function is Used

A linear equation can produce values less than 0 or greater than 1.

For example:

-2 1.5 5

These values cannot represent probabilities.

The sigmoid function converts the result into:

0 ≤ Probability ≤ 1

Therefore, it is useful for classification.


8. Probability and Classification

Suppose the model predicts:

Probability = 0.80

This means the model estimates an 80% probability for class 1.

A classification threshold is then used to convert the probability into a class.

A common threshold is:

0.5

Example

Probability >= 0.5 → Class 1 Probability < 0.5 → Class 0

Therefore:

ProbabilityPredicted Class
0.200
0.400
0.501
0.751
0.901

The threshold can be changed depending on the application.


9. Example: Student Pass Prediction

Suppose we want to predict whether a student will pass based on study hours.

Study HoursResult
1Fail
2Fail
3Fail
5Pass
6Pass
8Pass

Convert the classes into numbers:

Fail = 0 Pass = 1

The model learns the relationship between:

Study Hours → Probability of Passing

If a student studies for 6 hours and the model predicts:

Probability = 0.92

Using a threshold of 0.5:

0.92 >= 0.5

Therefore:

Prediction = Pass

10. Types of Logistic Regression

There are three commonly discussed types.

10.1 Binary Logistic Regression

Used when there are two classes.

Example

Pass / Fail

10.2 Multinomial Logistic Regression

Used when there are more than two classes that do not have a natural order.

Example

Predicting transportation choice:

Bus Car Bike Train

10.3 Ordinal Logistic Regression

Used when the classes have a natural order.

Example

Customer satisfaction:

Poor Average Good Excellent

The categories have an order.


11. Logistic Regression Workflow

Collect Data ↓ Clean Data ↓ Prepare Data ↓ Select Features ↓ Split Dataset ↓ Train Logistic Regression ↓ Predict Probabilities ↓ Apply Classification Threshold ↓ Evaluate Model

12. Steps to Build a Logistic Regression Model

Step 1: Collect Data

Collect relevant data from sources such as:

  • Databases
  • CSV files
  • Excel files
  • APIs
  • Surveys
  • Applications

Step 2: Clean the Data

Handle:

  • Missing values
  • Duplicate records
  • Incorrect values
  • Outliers
  • Incorrect data types

Step 3: Select Features

Choose variables that may help predict the target.

Example:

Study Hours Attendance Previous Score

Target:

Pass / Fail

Step 4: Encode the Target

Convert categories into numerical values.

Example:

Fail = 0 Pass = 1

Step 5: Split the Dataset

The data can be divided into:

Training Data Testing Data

For example:

80% → Training 20% → Testing

The exact split can vary depending on the dataset.


Step 6: Train the Model

The model learns the relationship between the features and the target classes.


Step 7: Make Predictions

The model can produce:

Probability

and then:

Predicted Class

Step 8: Evaluate the Model

Common evaluation measures include:

  • Accuracy
  • Precision
  • Recall
  • F1-score
  • Confusion Matrix
  • ROC-AUC

13. Confusion Matrix

A confusion matrix summarizes classification predictions.

For binary classification:

Predicted PositivePredicted Negative
Actual PositiveTrue PositiveFalse Negative
Actual NegativeFalse PositiveTrue Negative

True Positive (TP)

Actual class is positive and prediction is positive.

True Negative (TN)

Actual class is negative and prediction is negative.

False Positive (FP)

Actual class is negative but prediction is positive.

False Negative (FN)

Actual class is positive but prediction is negative.


14. Accuracy

Accuracy measures the proportion of predictions that are correct.

Formula

Accuracy = (TP + TN) / (TP + TN + FP + FN)

Example

Suppose:

TP = 40 TN = 45 FP = 5 FN = 10

Then:

Accuracy = (40 + 45) / (40 + 45 + 5 + 10) Accuracy = 85 / 100 Accuracy = 0.85

Therefore:

Accuracy = 85%

15. Precision

Precision measures how many of the observations predicted as positive were actually positive.

Formula

Precision = TP / (TP + FP)

Example

TP = 40 FP = 5

Then:

Precision = 40 / (40 + 5) = 40 / 45 ≈ 0.889

Therefore:

Precision ≈ 88.9%

16. Recall

Recall measures how many of the actual positive cases were correctly identified.

Formula

Recall = TP / (TP + FN)

Example

TP = 40 FN = 10

Then:

Recall = 40 / (40 + 10) = 0.80

Therefore:

Recall = 80%

17. F1-Score

F1-score combines Precision and Recall using their harmonic mean.

Formula

F1 = 2 × (Precision × Recall) / (Precision + Recall)

F1-score is useful when both precision and recall are important.


18. ROC Curve and AUC

The ROC curve shows the trade-off between:

  • True Positive Rate
  • False Positive Rate

for different classification thresholds.

AUC

AUC stands for:

Area Under the Curve

AUC summarizes the model's ability to distinguish between classes across thresholds.


19. Cost Function

Logistic Regression commonly uses log loss, also called cross-entropy loss, to measure prediction error.

For binary classification:

Log Loss = -[y log(p) + (1-y) log(1-p)]

Where:

  • y = actual class
  • p = predicted probability

The model learns coefficients that reduce the overall loss on the training data.


20. Logistic Regression Using Python

The scikit-learn library provides Logistic Regression.

Installation

pip install scikit-learn

Simple Example

import numpy as np from sklearn.linear_model import LogisticRegression # Study hours X = np.array([[1], [2], [3], [4], [5], [6], [7], [8]]) # 0 = Fail, 1 = Pass y = np.array([0, 0, 0, 0, 1, 1, 1, 1]) # Create model model = LogisticRegression() # Train model model.fit(X, y) # Predict class prediction = model.predict([[6]]) print("Prediction:", prediction[0])

Output:

Prediction: 1

Here:

1 = Pass

21. Predicting Probability

We can use predict_proba() to obtain probabilities.

probability = model.predict_proba([[6]]) print(probability)

The output contains probabilities for each class:

[Probability of Class 0, Probability of Class 1]

For example:

[0.08, 0.92]

This means:

Fail = 8% Pass = 92%

22. Logistic Regression with Train-Test Split

import pandas as pd from sklearn.model_selection import train_test_split from sklearn.linear_model import LogisticRegression from sklearn.metrics import accuracy_score # Load dataset data = pd.read_csv("students.csv") # Features X = data[["Hours"]] # Target y = data["Pass"] # Split data X_train, X_test, y_train, y_test = train_test_split( X, y, test_size=0.2, random_state=42 ) # Create model model = LogisticRegression() # Train model model.fit(X_train, y_train) # Make predictions y_pred = model.predict(X_test) # Calculate accuracy accuracy = accuracy_score(y_test, y_pred) print("Accuracy:", accuracy)

23. Multiple Features

Logistic Regression can use multiple input variables.

For example, student performance can be predicted using:

Study Hours Attendance Previous Score Assignment Score

Python example:

X = data[ [ "StudyHours", "Attendance", "PreviousScore", "AssignmentScore" ] ] y = data["Pass"]

The model uses all selected features to estimate the probability of passing.


24. Advantages of Logistic Regression

  1. Simple and easy to understand.
  2. Easy to implement.
  3. Computationally efficient.
  4. Produces probability estimates.
  5. Works well for many binary classification problems.
  6. Coefficients can provide interpretable information about predictors.
  7. Can handle multiple input variables.
  8. Provides a useful baseline classification model.

25. Limitations of Logistic Regression

  1. It assumes a suitable relationship between predictors and the log-odds of the outcome.
  2. It may perform poorly for highly nonlinear decision boundaries unless features are transformed or additional features are added.
  3. It can be sensitive to multicollinearity.
  4. Outliers can affect the fitted model.
  5. Feature scaling can be useful, especially when regularization is used.
  6. Class imbalance can make accuracy misleading.
  7. It may not perform well when the classes cannot be adequately separated by the available features.

26. Applications of Logistic Regression

Healthcare

Used for classification tasks such as:

  • Disease risk classification
  • Patient outcome analysis
  • Medical screening

Banking

Used for:

  • Fraud classification
  • Loan default analysis
  • Customer response prediction

Marketing

Used for:

  • Customer purchase prediction
  • Campaign response prediction
  • Customer churn classification

Education

Used for:

  • Pass/fail prediction
  • Student dropout analysis
  • Admission prediction

Cybersecurity

Used for:

  • Spam classification
  • Suspicious activity classification
  • Threat detection

E-Commerce

Used for:

  • Purchase prediction
  • Customer churn
  • Product response classification

27. Logistic Regression Example in Real Life

Suppose an online shopping company wants to predict whether a customer will purchase a product.

Features:

Age Time spent on website Number of pages visited Previous purchases

Target:

Purchase

Convert the target:

No Purchase = 0 Purchase = 1

The model might produce:

Probability = 0.78

Using a threshold of 0.5:

0.78 >= 0.5

Therefore:

Predicted Class = Purchase

The company can use this information as part of its customer analysis.


28. Important Difference: Linear vs Logistic Regression

FeatureLinear RegressionLogistic Regression
Main taskRegressionClassification
OutputContinuous valueProbability/class
Typical targetNumericalCategorical
FunctionLinear equationSigmoid/logistic function
Output rangeNot restricted to 0–1Probability between 0 and 1
ExamplePredict house pricePredict loan default
Common metricsMAE, MSE, RMSE, R²Accuracy, Precision, Recall, F1, ROC-AUC

29. Important Formulas

Sigmoid Function

σ(z) = 1 / (1 + e^-z)

Logistic Model

z = β₀ + β₁X₁ + β₂X₂ + ... + βₙXₙ

Probability

P(Y = 1) = 1 / (1 + e^-z)

Accuracy

Accuracy = (TP + TN) / (TP + TN + FP + FN)

Precision

Precision = TP / (TP + FP)

Recall

Recall = TP / (TP + FN)

F1-Score

F1 = 2 × (Precision × Recall) / (Precision + Recall)

30. Key Points to Remember

  • Logistic Regression is a supervised machine learning algorithm.
  • It is mainly used for classification.
  • Binary Logistic Regression is used for two classes.
  • It uses the sigmoid function.
  • The sigmoid function produces probabilities between 0 and 1.
  • A classification threshold converts probabilities into predicted classes.
  • 0 and 1 are commonly used to represent two classes.
  • Important evaluation metrics include Accuracy, Precision, Recall, F1-score, and ROC-AUC.
  • Logistic Regression can use one or multiple input features.
  • It is different from Linear Regression because it is mainly designed for classification.

Conclusion

Logistic Regression is a fundamental classification algorithm in machine learning. It estimates the probability of an observation belonging to a class and uses a classification threshold to produce a predicted class. It is relatively simple, interpretable, and useful in applications such as healthcare, banking, marketing, education, cybersecurity, and e-commerce.

Keep learning. Keep building. Keep growing.

© 2026 Ugenz Academy