Cosine Similarity

machine-learning

Definition

Cosine Similarity

Cosine similarity is a measure of similarity between two non-zero vectors that evaluates the cosine of the angle between them. Formally, for two vectors $\mathbf{x},\mathbf{v}\in {\mathbb{R}}^m$, the cosine similarity $S_C(\mathbf{x},\mathbf{v})$ is defined as the inner product of the vectors normalised by the product of their magnitudes:

$$S_C(\mathbf{x},\mathbf{v})=\frac{⟨\mathbf{x},\mathbf{v}⟩}{\Vert \mathbf{x}\Vert \Vert \mathbf{v}\Vert }=\frac{{\sum }_{i=1}^m{x}_i{v}_i}{\sqrt{{\sum }_{i=1}^m{x}_i^2}\sqrt{{\sum }_{i=1}^m{v}_i^2}}$$

The resulting value resides in the interval $[-1,1]$:

• 1: Identical direction (maximally similar).
• 0: Orthogonal (decorrelated).
• -1: Opposite direction (maximally dissimilar).

Properties

• Scale Invariance: Unlike Euclidean distance, cosine similarity is invariant to the magnitude of the vectors, making it suitable for high-dimensional data such as text embeddings (where document length varies).
• Metric Relation: It is related to the angular distance, but is not a formal metric itself as it violates the triangle inequality.