Kernel Function

machine-learning linear-algebra statistics

Definition

Kernel Function

A kernel function is a symmetric, positive semi-definite function that computes the inner product of two data points in a high-dimensional Hilbert space without requiring the explicit computation of the feature mapping . Formally:

Necessary Conditions

Symmetry: For any , the kernel must satisfy , reflecting the commutativity of the inner product.

Positive Semi-Definiteness (PSD): For any finite set of points , the corresponding Gram matrix must be positive semi-definite ( for all ). This ensures that the mapping corresponds to a valid geometry in .

Algebraic Closure Properties

Valid kernel functions are closed under several operations, allowing for the construction of complex kernels from simpler components while maintaining positive semi-definiteness.

Summation: If and are valid kernels, then their sum is also a valid kernel. This corresponds to concatenating the underlying feature maps.

Product: The pointwise product is a valid kernel, corresponding to the tensor product of the feature maps.

Scalar Multiplication: For any constant , is a valid kernel.