Clustering Algorithm

machine-learning

Definition

Clustering Algorithm

A clustering algorithm is a fundamental method in machine learning, specifically within the paradigm of unsupervised learning. Its primary objective is to partition a set of unlabeled data points into a number of groups, known as clusters, based on their intrinsic properties and a defined measure of similarity.

Let $X=\{x_1,x_2,\dots ,x_n\}$ be a set of $n$ data points, where each $x_i\in {\mathbb{R}}^d$ is a vector in a $d$-dimensional feature space. A clustering algorithm aims to find a partition $C=\{C_1,C_2,\dots ,C_k\}$ of the dataset $X$ into $k$ clusters, such that:

1. Exhaustiveness: Every data point belongs to at least one cluster.
   ${\bigcup }_{i=1}^k{C}_i=X$
2. Mutual Exclusivity (for hard clustering): Each data point belongs to exactly one cluster.
   $C_i\cap C_j=\varnothing \text{for }i\ne j$
3. Non-Emptiness: Each cluster must contain at least one data point.
   $C_i\ne \varnothing \text{for }i=1,\dots ,k$

The core principle is to maximize intra-cluster similarity and minimize inter-cluster similarity. The definition of “similarity” is crucial and is typically encoded using a distance metric (e.g., Euclidean distance) or a similarity function (e.g., cosine similarity).