Ugenz Academy/Tutorial

Kmeans

13 min read2,444 words

Reading progress

Keep learning

K-Means Clustering

1. Introduction

K-Means Clustering is an unsupervised machine learning algorithm used to divide data into different groups called clusters.

It groups similar data points together based on their characteristics.

For example, a shopping company may have customer data such as:

  • Age
  • Income
  • Spending score

K-Means can group customers into categories such as:

  • Low-spending customers
  • Medium-spending customers
  • High-spending customers

2. What is K-Means?

K-Means is a clustering algorithm that divides a dataset into K clusters.

Here:

  • K = Number of clusters
  • Means = The algorithm calculates the mean/average position of points in each cluster.

Simple Definition

K-Means is an unsupervised learning algorithm that groups similar data points into K clusters based on their distance from cluster centers.


3. Why is K-Means Used?

K-Means is used when we have data but do not have predefined labels.

For example:

Customer Data ↓ K-Means ↓ ┌─────────────┐ │ Cluster 1 │ │ Cluster 2 │ │ Cluster 3 │ └─────────────┘

It can help discover hidden patterns in data.


4. Example

Suppose a company has customers with:

CustomerAgeIncome
A2020,000
B2225,000
C2527,000
D4070,000
E4275,000
F4580,000

If we choose:

K = 2

K-Means may create:

Cluster 1

Young customers with lower income.

Cluster 2

Older customers with higher income.

The algorithm discovers these groups automatically.


5. Important Terms

5.1 Cluster

A cluster is a group of similar data points.

Example:

Cluster 1 → Young customers Cluster 2 → Middle-aged customers Cluster 3 → Older customers

5.2 K

K represents the number of clusters we want.

For example:

K = 2 → 2 clusters K = 3 → 3 clusters K = 5 → 5 clusters

Choosing an appropriate value of K is important.


5.3 Centroid

A centroid is the center point of a cluster.

For example:

● ● ● ★ Centroid

The centroid represents the average location of the data points in that cluster.


5.4 Distance

K-Means usually uses distance to determine which data points are close to each other.

The most common distance is Euclidean distance.

Euclidean Distance Formula

For two points:

A(x₁, y₁) B(x₂, y₂)

The distance is:

d = √((x₂ - x₁)² + (y₂ - y₁)²)

For multiple dimensions:

d = √Σ(xᵢ - yᵢ)²

6. How K-Means Works

K-Means follows an iterative process.

Step 1: Choose K

First, select the number of clusters.

Example:

K = 3

Step 2: Initialize Centroids

The algorithm selects initial centroid positions.

Centroid 1 Centroid 2 Centroid 3

Step 3: Assign Data Points

Each data point is assigned to the nearest centroid.

Data Point ↓ Calculate distance ↓ Find nearest centroid ↓ Assign to cluster

Step 4: Calculate New Centroids

After assigning the points, the algorithm calculates the new center of each cluster.

The new centroid is based on the mean of the points in that cluster.


Step 5: Repeat

The algorithm repeats:

Assign points ↓ Calculate centroids ↓ Assign points again ↓ Calculate centroids again

until the centroids stop changing significantly or the algorithm reaches its stopping condition.


7. K-Means Algorithm

The basic algorithm can be written as:

1. Choose K. 2. Select K initial centroids. 3. Calculate distance from each point to each centroid. 4. Assign each point to the nearest centroid. 5. Calculate new centroids. 6. Repeat steps 3–5. 7. Stop when the clusters become stable.

8. Simple Flow

Start ↓ Choose value of K ↓ Initialize centroids ↓ Calculate distances ↓ Assign data to clusters ↓ Calculate new centroids ↓ Centroids changed? ↙ ↘ Yes No ↓ ↓ Repeat Final Clusters

9. Objective of K-Means

K-Means tries to make the data points within each cluster as close as possible to their centroid.

It minimizes the Within-Cluster Sum of Squares (WCSS).

The objective function is:

WCSS = Σ Σ ||xᵢ - μⱼ||²

Where:

  • xᵢ = data point
  • μⱼ = centroid of the cluster
  • ||xᵢ - μⱼ||² = squared distance between a point and its centroid

A smaller WCSS generally means points are more compact around their centroids.


10. Choosing the Value of K

One of the most important tasks in K-Means is choosing the correct value of K.

A common method is the:

Elbow Method

The Elbow Method calculates WCSS for different values of K.

Example:

KWCSS
1500
2300
3180
4160
5150

We plot K against WCSS.

WCSS │ 500│ * │ \ 300│ * │ \ 180│ * │ \__ 160│ * │ \__ 150│ * └────────────────────── K 1 2 3 4 5

The point where the decrease starts becoming less significant is called the elbow.

That value can be considered as a candidate for K, but it should also be checked using domain knowledge and other validation methods.


11. Python Libraries

Common Python libraries for K-Means include:

  • NumPy
  • Pandas
  • Matplotlib
  • Scikit-learn

Import example:

import pandas as pd import matplotlib.pyplot as plt from sklearn.cluster import KMeans from sklearn.preprocessing import StandardScaler

12. Simple K-Means Example in Python

from sklearn.cluster import KMeans X = [ [1, 2], [1, 3], [2, 2], [8, 8], [9, 8], [8, 9] ] model = KMeans( n_clusters=2, random_state=42, n_init=10 ) model.fit(X) print("Cluster Labels:") print(model.labels_) print("Centroids:") print(model.cluster_centers_)

Explanation

n_clusters=2

means that we want two clusters.

model.fit(X)

trains the K-Means model.

model.labels_

returns the cluster assigned to each data point.

model.cluster_centers_

returns the centroid of each cluster.


13. K-Means with Pandas

Suppose we have customer data:

import pandas as pd from sklearn.cluster import KMeans data = { "Age": [20, 22, 25, 40, 42, 45], "Income": [20000, 25000, 27000, 70000, 75000, 80000] } df = pd.DataFrame(data) model = KMeans( n_clusters=2, random_state=42, n_init=10 ) df["Cluster"] = model.fit_predict(df[["Age", "Income"]]) print(df)

The output will contain a new column:

Age Income Cluster 20 20000 0 22 25000 0 25 27000 0 40 70000 1 42 75000 1 45 80000 1

The actual cluster numbers can be swapped because cluster labels are arbitrary.


14. Feature Scaling

Feature scaling is very important for K-Means.

Suppose we have:

Age → 20 to 60 Income → 20,000 to 100,000

Income has much larger numerical values than age.

Without scaling, the income feature may have a much stronger effect on distance calculations.

We can use StandardScaler.

from sklearn.preprocessing import StandardScaler scaler = StandardScaler() X_scaled = scaler.fit_transform(df[["Age", "Income"]])

Then:

model = KMeans( n_clusters=2, random_state=42, n_init=10 ) df["Cluster"] = model.fit_predict(X_scaled)

15. Complete K-Means Example

import pandas as pd import matplotlib.pyplot as plt from sklearn.cluster import KMeans from sklearn.preprocessing import StandardScaler # Create dataset data = { "Age": [20, 22, 25, 40, 42, 45], "Income": [20000, 25000, 27000, 70000, 75000, 80000] } df = pd.DataFrame(data) # Select features X = df[["Age", "Income"]] # Scale features scaler = StandardScaler() X_scaled = scaler.fit_transform(X) # Create K-Means model model = KMeans( n_clusters=2, random_state=42, n_init=10 ) # Predict clusters df["Cluster"] = model.fit_predict(X_scaled) print(df) # Display centroids in scaled space print("Centroids:") print(model.cluster_centers_)

16. Visualizing K-Means

We can visualize clusters using a scatter plot.

import matplotlib.pyplot as plt plt.scatter( df["Age"], df["Income"], c=df["Cluster"] ) plt.xlabel("Age") plt.ylabel("Income") plt.title("Customer Clusters") plt.show()

This allows us to see how customers are divided into different groups.


17. Elbow Method in Python

import matplotlib.pyplot as plt from sklearn.cluster import KMeans wcss = [] for k in range(1, 11): model = KMeans( n_clusters=k, random_state=42, n_init=10 ) model.fit(X_scaled) wcss.append(model.inertia_) plt.plot(range(1, 11), wcss, marker="o") plt.xlabel("Number of Clusters (K)") plt.ylabel("WCSS") plt.title("Elbow Method") plt.show()

Important

model.inertia_

returns the within-cluster sum of squared distances, commonly called inertia in scikit-learn.


18. K-Means Evaluation

Because K-Means is unsupervised, we normally do not have true target labels to calculate ordinary classification accuracy.

Some useful clustering evaluation methods are:

18.1 Inertia

Measures the sum of squared distances from points to their cluster centroids.

Lower values indicate more compact clusters, but inertia usually decreases as K increases, so it should not be used alone.


18.2 Silhouette Score

The Silhouette Score measures how well-separated and internally cohesive the clusters are.

Its value ranges approximately from:

-1 to +1

Generally:

  • Higher positive values → better-separated clusters
  • Around 0 → overlapping clusters
  • Negative values → some points may be assigned to unsuitable clusters

Python:

from sklearn.metrics import silhouette_score score = silhouette_score(X_scaled, df["Cluster"]) print("Silhouette Score:", score)

19. K-Means Parameters

Important parameters in scikit-learn include:

ParameterMeaning
n_clustersNumber of clusters
initMethod for initializing centroids
n_initNumber of centroid initializations tried
max_iterMaximum iterations for one run
random_stateControls reproducibility
algorithmAlgorithm used for distance computations

Example:

model = KMeans( n_clusters=3, init="k-means++", n_init=10, max_iter=300, random_state=42 )

20. Advantages of K-Means

  1. Easy to understand.
  2. Simple to implement.
  3. Relatively fast for many datasets.
  4. Works well when clusters are compact and reasonably separated.
  5. Useful for customer segmentation.
  6. Can process large datasets efficiently.
  7. Easy to visualize when there are two or three features.

21. Limitations of K-Means

  1. The value of K usually needs to be chosen beforehand.
  2. Results can depend on the initial centroid positions.
  3. Sensitive to outliers.
  4. Sensitive to feature scaling.
  5. May perform poorly with non-spherical or irregularly shaped clusters.
  6. Different runs can produce different results if initialization is not controlled.
  7. It is not always suitable when clusters have very different sizes or densities.
  8. Cluster labels do not have an inherent meaning.

22. Applications of K-Means

K-Means is widely used in different fields.

1. Customer Segmentation

Groups customers according to:

  • Age
  • Income
  • Spending
  • Purchase behavior

2. Marketing

Companies can identify groups of customers with similar characteristics.

3. Image Compression

Similar colors can be grouped together to reduce the number of colors in an image.

4. Document Clustering

Documents can be grouped according to their content.

5. Recommendation Systems

Users with similar behavior can be grouped.

6. Anomaly Detection

Very distant points may help identify unusual observations, although specialized anomaly-detection methods may be more appropriate.

7. Geographic Analysis

Locations can be grouped based on geographic or demographic characteristics.


23. Real-Life Example: Customer Segmentation

Suppose an online store has:

Customer Age Annual Income Spending Score

The company chooses:

K = 3

K-Means may produce:

Cluster 1 → Low spending customers Cluster 2 → Medium spending customers Cluster 3 → High spending customers

The company can then analyze each cluster separately.

This is called customer segmentation.


24. K-Means vs KNN

K-Means and KNN have similar names but are very different algorithms.

K-MeansKNN
Unsupervised learningSupervised learning
Used for clusteringUsed for classification/regression
Does not require target labelsRequires labeled training data
Creates clustersPredicts based on neighbors
Uses centroidsUses nearest data points
K = number of clustersK = number of neighbors

25. K-Means vs Decision Tree

K-MeansDecision Tree
UnsupervisedSupervised
ClusteringClassification/Regression
No target variable requiredTarget variable required
Uses centroidsUses decision rules
Groups similar dataPredicts outcomes

26. K-Means vs Hierarchical Clustering

K-MeansHierarchical Clustering
Requires KDoes not require K in the same way
Uses centroidsBuilds a hierarchy of clusters
Usually faster for large datasetsCan be computationally expensive
Produces flat clustersProduces a hierarchy/dendrogram
Easy to implementUseful for exploring nested relationships

27. Important Points

Remember these key concepts:

K-Means ↓ Unsupervised Learning ↓ Clustering ↓ Choose K ↓ Initialize Centroids ↓ Assign Points ↓ Calculate New Centroids ↓ Repeat ↓ Final Clusters

Key points:

  • K-Means is an unsupervised learning algorithm.
  • It divides data into K clusters.
  • A centroid represents the center of a cluster.
  • Distance is used to assign points to clusters.
  • Euclidean distance is commonly used.
  • Feature scaling can be important.
  • The Elbow Method can help select K.
  • Silhouette Score can help evaluate cluster separation.
  • K-Means is useful for customer segmentation and pattern discovery.
  • Cluster numbers themselves have no inherent meaning.

28. K-Means Workflow

Raw Data ↓ Data Preprocessing ↓ Select Features ↓ Scale Features ↓ Choose K ↓ Train K-Means ↓ Assign Data to Clusters ↓ Evaluate Clusters ↓ Visualize Results ↓ Interpret Clusters

29. When Should You Use K-Means?

K-Means is useful when:

  • You do not have target labels.
  • You want to discover groups in your data.
  • Data can reasonably be represented using numerical features.
  • You expect relatively compact and separated clusters.
  • You need a simple and fast clustering method.

30. Conclusion

K-Means Clustering is a popular unsupervised machine learning algorithm used to divide data into groups of similar observations.

It works by:

  1. Choosing the number of clusters.
  2. Selecting initial centroids.
  3. Assigning points to the nearest centroid.
  4. Recalculating centroids.
  5. Repeating the process until the solution stabilizes.

It is commonly used for customer segmentation, marketing, image compression, document clustering, and pattern discovery.

Understanding K-Means is an important step toward learning more advanced unsupervised machine learning techniques.

Keep learning. Keep building. Keep growing.

© 2026 Ugenz Academy