Definition
Dual Problem
In mathematical optimisation, the dual problem is an alternative formulation of a constrained optimisation problem (the primal problem) derived using Lagrange multipliers. Formally, for a primal problem seeking to minimise
subject to , the dual objective is defined as the infimum of the Lagrangian over :
The dual problem then seeks to maximise
subject to . According to strong duality, if the primal is convex and satisfies certain regularity conditions, the optimal values of the primal and dual problems are identical.
Application in SVMs
Kernel Trick: Converting the primal SVM problem into its dual form is essential for kernel methods. The dual objective depends only on the inner products between data points:
By replacing the inner product with a kernel function