A company wants to maximize revenue subject to a budget constraint. A physicist wants to find the minimum energy configuration of a system subject to conservation laws. An engineer wants to minimize material use subject to structural requirements. These are all constrained optimization problems—finding the best value of an objective function while satisfying restrictions. Lagrange multipliers, developed by Joseph-Louis Lagrange in the 18th century, transform constrained optimization into a system of equations that elegantly reveals both the optimal point and the price of the constraint.
The Geometric Insight
At a constrained optimum, the objective function's level curves must be tangent to the constraint curve. If they intersect (not tangent), you could move along the constraint to a higher objective value—contradicting optimality. Tangency means the gradient of the objective function is parallel to the gradient of the constraint: ∇f = λ∇g, where g(x,y) = 0 is the constraint and λ is the Lagrange multiplier. This geometric condition—gradient parallelism—is the heart of the method and explains the multiplier's role as a proportionality constant.
The Method
Introduce a Lagrangian function: L(x, y, λ) = f(x, y) − λ·g(x, y). Setting all partial derivatives to zero: ∂L/∂x = 0, ∂L/∂y = 0, ∂L/∂λ = 0. The third equation simply recovers the constraint g = 0. Solving this system yields the candidate optimal points. The method extends naturally to multiple variables and multiple constraints: introduce one multiplier per constraint, form the Lagrangian as f minus the sum of λ_i g_i, and set all partial derivatives to zero.
An Illustrative Example
Maximize f(x,y) = xy subject to x + y = 10 (fixed perimeter rectangle maximizing area). Lagrangian: L = xy − λ(x + y − 10). Setting ∂L/∂x = y − λ = 0 and ∂L/∂y = x − λ = 0 gives x = y = λ. Using the constraint: 2x = 10, so x = y = 5. The maximum area is 25, achieved by a square—a beautiful result: among all rectangles with fixed perimeter, the square has maximum area. The multiplier λ = 5 has an economic interpretation: relaxing the constraint by 1 unit (perimeter = 11) increases the maximum area by approximately λ = 5.
Multiple Constraints and Inequalities
With multiple equality constraints g_i = 0, introduce one multiplier per constraint: ∇f = Σ λ_i ∇g_i. For inequality constraints g(x) ≤ 0, the Karush-Kuhn-Tucker (KKT) conditions extend Lagrange multipliers: at an optimum, either the constraint is inactive (g < 0 and λ = 0) or active (g = 0 and λ ≥ 0). KKT conditions are necessary (and sufficient under convexity) for constrained optimization with both equalities and inequalities—the mathematical foundation of linear programming, convex optimization, and support vector machines.
Applications
Lagrange multipliers pervade optimization across disciplines. Portfolio optimization maximizes expected return subject to a risk (variance) constraint, yielding the efficient frontier. In thermodynamics, deriving the Boltzmann distribution maximizes entropy subject to fixed total energy. Economics uses them for utility maximization subject to budget constraints—the multiplier's economic interpretation as the 'shadow price' of a constraint is central to microeconomic theory. Machine learning's support vector machines find the maximum-margin hyperplane using Lagrangian duality on a constrained quadratic optimization problem.
Conclusion
Lagrange multipliers transform the difficult problem of constrained optimization into an unconstrained system of equations. The multiplier itself carries economic meaning—it is the marginal value of relaxing the constraint by one unit. This elegant method, elegant in both its geometric motivation and algebraic execution, provides the foundation for optimization throughout economics, physics, engineering, and machine learning wherever objectives must be balanced against resources or physical requirements.