The Central Idea
Suppose we want to optimize
\[ f(x,y) \]subject to
\[ g(x,y)=c. \]At many smooth constrained extrema,
\[ \boxed{ \nabla f = \lambda \nabla g }. \]The scalar \(\lambda\) is called a Lagrange multiplier.
Why the Gradients Must Align
The gradient
\[ \nabla f \]points in the direction of greatest local increase of the objective.
Meanwhile,
\[ \nabla g \]is perpendicular to the level curve
\[ g(x,y)=c. \]At a constrained optimum, movement along the constraint cannot improve the objective to first order.
Therefore, the objective gradient must be normal to the constraint curve as well.
Geometric Interpretation
The level curves of the objective satisfy
\[ f(x,y)=k. \]The feasible set is the constraint curve
\[ g(x,y)=c. \]At a smooth constrained extremum, an objective contour typically becomes tangent to the constraint.
Their normal vectors are therefore parallel:
\[ \nabla f = \lambda\nabla g. \]The Basic Lagrange System
For
\[ \text{optimize }f(x,y) \]subject to
\[ g(x,y)=c, \]solve
\[ \boxed{ \begin{aligned} f_x &= \lambda g_x,\\ f_y &= \lambda g_y,\\ g(x,y) &= c. \end{aligned} } \]The unknowns are
\[ x,\qquad y,\qquad\lambda. \]The Lagrangian
The method can be organized using the Lagrangian.
One common convention is
\[ \boxed{ \mathcal{L}(x,y,\lambda) = f(x,y) - \lambda\bigl(g(x,y)-c\bigr) }. \]Other sign conventions are also valid.
Stationarity of the Lagrangian
Differentiate the Lagrangian with respect to every unknown:
\[ \frac{\partial\mathcal{L}}{\partial x}=0, \] \[ \frac{\partial\mathcal{L}}{\partial y}=0, \] \[ \frac{\partial\mathcal{L}}{\partial\lambda}=0. \]These equations recover
\[ \nabla f=\lambda\nabla g \]together with the original constraint.
Example: Minimize Distance to the Origin
Minimize
\[ f(x,y)=x^2+y^2 \]subject to
\[ x+y=10. \]Define
\[ g(x,y)=x+y. \]Then
\[ \nabla f= \begin{bmatrix} 2x\\ 2y \end{bmatrix} \]and
\[ \nabla g= \begin{bmatrix} 1\\ 1 \end{bmatrix}. \]Solving the Distance Example
The multiplier condition gives
\[ 2x=\lambda, \] \[ 2y=\lambda, \]together with
\[ x+y=10. \]From the first two equations,
\[ x=y. \]Therefore,
\[ 2x=10 \]and
\[ x=y=5. \]Thus the constrained minimizer is
\[ \boxed{(5,5)}. \]Checking the Objective Value
At
\[ (5,5), \]the objective is
\[ f(5,5) = 5^2+5^2 = 50. \]Therefore, the minimum squared distance is
\[ \boxed{50}. \]Example: Maximize a Product
Maximize
\[ f(x,y)=xy \]subject to
\[ x+y=20. \]The gradients are
\[ \nabla f= \begin{bmatrix} y\\ x \end{bmatrix}, \] \[ \nabla g= \begin{bmatrix} 1\\ 1 \end{bmatrix}. \]Solving the Product Example
The Lagrange equations are
\[ y=\lambda, \] \[ x=\lambda, \] \[ x+y=20. \]Hence
\[ x=y. \]Therefore,
\[ x=y=10. \]The objective value is
\[ f(10,10)=100. \]Thus
\[ \boxed{(10,10)} \]is the maximizing point.
Connection with the Rectangle Problem
If \(x\) and \(y\) are side lengths of a rectangle with perimeter \(40\), then
\[ 2x+2y=40 \]simplifies to
\[ x+y=20. \]Maximizing
\[ A=xy \]therefore gives
\[ x=y=10. \]Among rectangles with fixed perimeter, the square has maximum area.
Example: Constraint on a Circle
Maximize and minimize
\[ f(x,y)=x+y \]subject to
\[ x^2+y^2=1. \]Let
\[ g(x,y)=x^2+y^2. \]Then
\[ \nabla f= \begin{bmatrix} 1\\ 1 \end{bmatrix} \]and
\[ \nabla g= \begin{bmatrix} 2x\\ 2y \end{bmatrix}. \]Solving the Circle Example
The equations are
\[ 1=2\lambda x, \] \[ 1=2\lambda y, \] \[ x^2+y^2=1. \]The first two equations imply
\[ x=y. \]Therefore,
\[ 2x^2=1, \]so
\[ x=y=\pm\frac{1}{\sqrt2}. \]The candidate points are
\[ \left( \frac{1}{\sqrt2}, \frac{1}{\sqrt2} \right) \]and
\[ \left( -\frac{1}{\sqrt2}, -\frac{1}{\sqrt2} \right). \]Comparing the Circle Candidates
At the positive candidate,
\[ f = \sqrt2. \]At the negative candidate,
\[ f = -\sqrt2. \]Therefore,
\[ \boxed{ \max f=\sqrt2 } \]and
\[ \boxed{ \min f=-\sqrt2 }. \]Lagrange Multipliers Produce Candidates
Solving
\[ \nabla f=\lambda\nabla g \]identifies candidate constrained extrema.
The resulting points must still be:
- checked for feasibility;
- evaluated in the objective;
- compared when several candidates exist;
- interpreted in the context of the original problem.
Why the Method Does Not Automatically Classify Candidates
The multiplier equations do not by themselves tell us whether a point is a maximum, minimum, or another stationary constrained point.
Classification may require:
- objective-value comparison;
- geometric reasoning;
- second-order conditions;
- convexity or concavity;
- compactness and existence theorems.
Multiple Equality Constraints
Suppose we optimize
\[ f(\mathbf{x}) \]subject to
\[ g_1(\mathbf{x})=c_1 \]and
\[ g_2(\mathbf{x})=c_2. \]Then the stationarity condition becomes
\[ \boxed{ \nabla f = \lambda_1\nabla g_1 + \lambda_2\nabla g_2 }. \]The Multi-Constraint Lagrangian
For equality constraints
\[ g_j(\mathbf{x})=c_j, \qquad j=1,\ldots,p, \]define
\[ \boxed{ \mathcal{L} = f(\mathbf{x}) - \sum_{j=1}^{p} \lambda_j \bigl(g_j(\mathbf{x})-c_j\bigr) }. \]We then set the derivatives of \(\mathcal{L}\) with respect to the decision variables and multipliers equal to zero.
Example with Three Variables
Minimize
\[ f(x,y,z)=x^2+y^2+z^2 \]subject to
\[ x+y+z=12. \]The gradients are
\[ \nabla f= \begin{bmatrix} 2x\\ 2y\\ 2z \end{bmatrix} \]and
\[ \nabla g= \begin{bmatrix} 1\\ 1\\ 1 \end{bmatrix}. \]Solving the Three-Variable Example
The Lagrange equations give
\[ 2x=\lambda, \] \[ 2y=\lambda, \] \[ 2z=\lambda. \]Therefore,
\[ x=y=z. \]Since
\[ x+y+z=12, \]we obtain
\[ \boxed{x=y=z=4}. \]Symmetry Can Simplify the Algebra
Symmetric objectives and constraints often imply symmetric candidate solutions.
In the previous example, the equations
\[ 2x=\lambda, \qquad 2y=\lambda, \qquad 2z=\lambda \]immediately reveal
\[ x=y=z. \]When Substitution Is Easier
Lagrange multipliers are powerful, but they are not always the simplest method.
If a constraint such as
\[ y=10-x \]can be substituted easily into the objective, direct elimination may be faster.
Method choice should reflect the structure of the problem.
When Lagrange Multipliers Are Especially Useful
The method becomes especially useful when:
- the equality constraint is difficult to solve explicitly;
- several variables are involved;
- several equality constraints are present;
- the geometry of the constraint is naturally described implicitly;
- a systematic symbolic method is preferred.
Regularity of the Constraint
The basic multiplier condition assumes the constraint is locally regular.
For a single constraint,
\[ g(\mathbf{x})=c, \]we usually require
\[ \boxed{ \nabla g(\mathbf{x}^*)\ne\mathbf{0} }. \]This ensures that the constraint has a well-defined local normal direction.
Singular Constraint Points
If
\[ \nabla g=\mathbf{0} \]at a feasible point, the ordinary Lagrange condition may fail to identify that point.
Such singular points must be examined separately.
Example of a Singular Constraint
Consider
\[ g(x,y)=x^2+y^2 \]with constraint
\[ g(x,y)=0. \]The only feasible point is
\[ (0,0). \]But
\[ \nabla g(0,0)=\mathbf{0}. \]The point must therefore be recognized directly rather than relying only on the standard multiplier condition.
The Multiplier as a Sensitivity Measure
In many regular optimization problems, the Lagrange multiplier has an important interpretation.
Suppose the constraint is
\[ g(\mathbf{x})=c. \]If the optimal value is
\[ V(c), \]then under suitable conditions,
\[ \boxed{ \lambda \approx \frac{dV}{dc} } \]up to the sign convention used in the Lagrangian.
Shadow-Price Interpretation
In economic and operational models, a multiplier can measure the marginal value of relaxing a binding constraint.
For example, if a constraint represents available labor hours, the multiplier can indicate how much the optimal objective would change if one additional unit of labor became available.
Units of a Lagrange Multiplier
Multiplier units follow from the objective and constraint.
If the objective is measured in dollars and the constraint parameter is measured in labor hours, then the multiplier may have units
\[ \frac{\text{dollars}}{\text{labor hour}}. \]This supports its interpretation as marginal objective value per unit of constrained resource.
Sign Convention Matters
Some texts define
\[ \mathcal{L} = f-\lambda(g-c), \]while others use
\[ \mathcal{L} = f+\lambda(g-c). \]Both conventions are valid.
The sign of \(\lambda\) changes accordingly, while the candidate values of the decision variables do not.
Second-Order Analysis
First-order multiplier equations identify constrained stationary candidates.
More advanced classification can examine curvature only in directions that remain tangent to the constraint.
This produces constrained second-order optimality conditions.
Convexity and Lagrange Multipliers
In convex optimization, multiplier conditions become especially powerful.
When the objective and feasible geometry satisfy appropriate convexity assumptions, a stationary feasible candidate can often be certified as globally optimal.
Relationship to KKT Conditions
Lagrange multipliers handle equality constraints directly.
Inequality constraints require a broader framework.
The Karush–Kuhn–Tucker conditions extend multiplier ideas to problems with constraints such as
\[ g_i(\mathbf{x})\le0. \]Complementary Slackness Preview
For inequality constraints, inactive constraints should not influence first-order stationarity in the same way as active constraints.
KKT conditions capture this through
\[ \lambda_i g_i(\mathbf{x})=0. \]This is called complementary slackness.
Example: Maximize Volume with a Surface Constraint
Suppose a rectangular box has dimensions
\[ x,\qquad y,\qquad z \]and fixed total edge sum
\[ x+y+z=s. \]The volume is
\[ V=xyz. \]The multiplier equations reveal that at the symmetric optimum,
\[ x=y=z. \]Therefore, equal dimensions maximize the volume under this symmetric constraint.
Why Equalization Often Appears
In symmetric constrained problems, the Lagrange equations often force several variables to become equal.
This reflects balance: when the objective and constraint treat variables symmetrically, an unequal allocation can often be improved by redistributing the constrained quantity.
Common Error: Forgetting the Original Constraint
The equations
\[ \nabla f=\lambda\nabla g \]are not enough by themselves.
The original condition
\[ g(\mathbf{x})=c \]must be solved simultaneously.
Common Error: Treating \(\lambda\) as the Objective Value
The multiplier \(\lambda\) is an auxiliary variable.
It is not generally the maximum or minimum objective value.
The objective must still be evaluated at the candidate point.
Common Error: Forgetting Multiple Candidates
The multiplier system may produce several feasible solutions.
All relevant candidates must be evaluated and compared.
Common Error: Assuming Every Lagrange Point Is an Extremum
A solution of the multiplier equations is a constrained stationary candidate.
It may require further classification.
Common Error: Dividing by a Quantity That Could Be Zero
Algebraic simplification can accidentally discard valid solutions.
If an equation contains a factor such as
\[ x(\cdots)=0, \]do not divide by \(x\) until the case
\[ x=0 \]has been considered.
Common Error: Using the Method at a Singular Constraint Point
When
\[ \nabla g=\mathbf{0}, \]the ordinary geometric argument for gradient alignment breaks down.
Singular feasible points must be checked separately.
Common Error: Ignoring Domain Restrictions
Candidate points must satisfy:
- the equality constraint;
- the natural domain of the objective;
- any sign or physical restrictions in the original problem.
A Reliable Lagrange Multiplier Workflow
- Write the objective function \(f\).
- Write each equality constraint in a clear form.
- Check domain and regularity conditions.
- Compute \(\nabla f\).
- Compute the gradient of each constraint.
- Set \(\nabla f\) equal to a linear combination of constraint gradients.
- Include every original constraint equation.
- Solve the complete system for the decision variables and multipliers.
- Check for singular feasible points that may have been missed.
- Discard infeasible or undefined candidates.
- Evaluate the objective at every remaining candidate.
- Classify or compare candidates.
- Interpret both the optimizer and multiplier values when meaningful.
Practice 1: Write the Lagrange Equations
Optimize
\[ f(x,y)=x^2+y^2 \]subject to
\[ x+y=6. \]Write the Lagrange system.
Practice 2: Solve a Simple Minimum
Minimize
\[ f(x,y)=x^2+y^2 \]subject to
\[ x+y=6. \]Practice 3: Maximize a Product
Maximize
\[ f(x,y)=xy \]subject to
\[ x+y=14. \]Practice 4: Circle Constraint
Find the maximum and minimum of
\[ f(x,y)=x \]subject to
\[ x^2+y^2=9. \]Practice 5: Three Variables
Minimize
\[ f(x,y,z)=x^2+y^2+z^2 \]subject to
\[ x+y+z=15. \]Practice 6: Lagrangian Construction
Construct a Lagrangian for
\[ f(x,y)=3x^2+y^2 \]subject to
\[ 2x+y=8. \]Practice 7: Multiple Constraints
For an objective
\[ f(x,y,z) \]subject to
\[ g_1(x,y,z)=0 \]and
\[ g_2(x,y,z)=0, \]write the general first-order gradient relation.
Practice 8: Singular Constraint
Explain why the point
\[ (0,0) \]is special for the constraint
\[ x^2+y^2=0. \]Practice 9: Multiplier Interpretation
If a multiplier for a labor-capacity equality constraint is approximately \(25\) dollars per labor hour, explain the practical interpretation.
Practice 10: Identify the Error
A student solves
\[ \nabla f=\lambda\nabla g \]but does not impose
\[ g(x,y)=c. \]Explain the error.
Practice Answers
Practice 1
The gradients are
\[ \nabla f= \begin{bmatrix} 2x\\ 2y \end{bmatrix} \]and
\[ \nabla g= \begin{bmatrix} 1\\ 1 \end{bmatrix}. \]Therefore,
\[ \boxed{ \begin{aligned} 2x&=\lambda,\\ 2y&=\lambda,\\ x+y&=6. \end{aligned} } \]Practice 2
From
\[ 2x=\lambda \]and
\[ 2y=\lambda, \]we get
\[ x=y. \]Since
\[ x+y=6, \]the minimizer is
\[ \boxed{(3,3)}. \]Practice 3
The multiplier equations give
\[ x=y. \]Since
\[ x+y=14, \]we obtain
\[ \boxed{x=y=7}. \]The maximum product is
\[ \boxed{49}. \]Practice 4
Since
\[ f(x,y)=x, \]the extreme \(x\)-coordinates on the circle of radius \(3\) are
\[ x=3 \]and
\[ x=-3. \]Therefore,
\[ \boxed{\max f=3\text{ at }(3,0)} \]and
\[ \boxed{\min f=-3\text{ at }(-3,0)}. \]Practice 5
The equations force
\[ x=y=z. \]Therefore,
\[ 3x=15 \]and
\[ \boxed{x=y=z=5}. \]Practice 6
One valid convention is
\[ \boxed{ \mathcal{L}(x,y,\lambda) = 3x^2+y^2 - \lambda(2x+y-8) }. \]Practice 7
\[ \boxed{ \nabla f = \lambda_1\nabla g_1 + \lambda_2\nabla g_2 }. \]Practice 8
The only feasible point is
\[ (0,0), \]but
\[ \nabla g(0,0)=\mathbf{0}. \]Therefore, the standard regularity assumption fails and the point must be checked directly.
Practice 9
Under the appropriate sign convention and regularity assumptions, one additional labor hour would improve the optimal objective by approximately \(25\) dollars near the current operating point.
Practice 10
The gradient-alignment equations identify possible directions of tangency, but the candidate must also lie on the constraint.
Therefore,
\[ g(x,y)=c \]must be solved simultaneously with the multiplier equations.
What to Remember
Lagrange multipliers convert smooth equality-constrained optimization into a system of first-order equations.
The central condition is
\[ \boxed{ \nabla f = \lambda\nabla g }. \]- the objective and constraint gradients align at regular smooth constrained extrema;
- the original constraint must always be included;
- the multiplier is an auxiliary variable, not the objective value;
- multiple equality constraints require multiple multipliers;
- singular constraint points may require separate analysis;
- multiplier solutions are candidates that still require classification or comparison;
- multipliers often have useful sensitivity or shadow-price interpretations;
- inequality constraints lead to the broader KKT framework.
The core geometric idea is
\[ \boxed{ \text{at a smooth constrained optimum, the objective contour is tangent to the constraint} }. \]Where We Go Next
Lagrange multipliers provide a general calculus-based method for equality constraints. Another major class of optimization problems has linear objectives and linear constraints, producing feasible regions made of lines, polygons, planes, and polyhedra.
The next lesson, Linear Programming, develops this important structure and shows why corner points, objective contours, and resource constraints play such a central role.