The Central Idea
Suppose we want to optimize
\[ f(x,y) \]but the variables must satisfy
\[ g(x,y)=c. \]We are not allowed to search the entire plane.
We must search only the points lying on the constraint set
\[ \boxed{ g(x,y)=c }. \]Unconstrained versus Constrained Optimization
In unconstrained optimization, candidate interior extrema often satisfy
\[ \nabla f=\mathbf{0}. \]Under a constraint, the optimum may occur at a point where
\[ \nabla f\neq\mathbf{0}. \]The constraint prevents movement in arbitrary directions, so the ordinary critical-point condition is no longer sufficient.
The Feasible Set
The set of all inputs satisfying the constraints is called the feasible set.
For example,
\[ x^2+y^2=25 \]restricts feasible points to a circle.
The optimization problem is therefore one-dimensional even though the objective function depends on two variables.
A Simple Constrained Example
Maximize and minimize
\[ f(x,y)=x+y \]subject to
\[ x^2+y^2=1. \]Geometrically, we are asking where the level lines
\[ x+y=k \]first and last touch the unit circle.
Geometry of a Constrained Extremum
At a smooth constrained extremum, the objective level curve is often tangent to the constraint curve.
Since gradients are normal to level curves, their normals must be parallel.
Thus,
\[ \boxed{ \nabla f \parallel \nabla g }. \]The Lagrange Multiplier Condition
Parallel vectors differ by a scalar multiple.
Therefore, at a regular constrained extremum,
\[ \boxed{ \nabla f = \lambda\nabla g }. \]The scalar
\[ \lambda \]is called a Lagrange multiplier.
The Full Lagrange System
To optimize
\[ f(x,y) \]subject to
\[ g(x,y)=c, \]solve
\[ \boxed{ \nabla f = \lambda\nabla g } \]together with
\[ \boxed{ g(x,y)=c }. \]Coordinate Form
In two variables, the system becomes
\[ \boxed{ f_x=\lambda g_x } \] \[ \boxed{ f_y=\lambda g_y } \]together with
\[ \boxed{ g(x,y)=c }. \]Example: Linear Objective on a Circle
Let
\[ f(x,y)=x+y \]with constraint
\[ g(x,y)=x^2+y^2=1. \]The gradients are
\[ \nabla f=\langle1,1\rangle \]and
\[ \nabla g=\langle2x,2y\rangle. \]Set Up the Lagrange Equations
We require
\[ 1=2\lambda x \]and
\[ 1=2\lambda y. \]Therefore,
\[ x=y. \]Apply the Constraint
Since
\[ x=y, \]the constraint gives
\[ x^2+x^2=1. \]Thus,
\[ 2x^2=1, \]so
\[ x=\pm\frac{1}{\sqrt2}. \]Hence the candidate points are
\[ \left( \frac{1}{\sqrt2}, \frac{1}{\sqrt2} \right) \]and
\[ \left( -\frac{1}{\sqrt2}, -\frac{1}{\sqrt2} \right). \]Classify by Comparing Objective Values
Evaluate:
\[ f\left( \frac{1}{\sqrt2}, \frac{1}{\sqrt2} \right) = \sqrt2 \]and
\[ f\left( -\frac{1}{\sqrt2}, -\frac{1}{\sqrt2} \right) = -\sqrt2. \]Therefore,
\[ \boxed{ f_{\max}=\sqrt2 } \]and
\[ \boxed{ f_{\min}=-\sqrt2 }. \]Why Comparing Values Matters
The Lagrange equations generate candidate points.
They do not automatically tell us which candidate is a maximum or minimum.
A reliable final step is to evaluate the objective at every feasible candidate and compare the resulting values.
Compactness and Guaranteed Extrema
If a continuous objective function is restricted to a nonempty compact feasible set, then it attains both an absolute maximum and an absolute minimum.
In elementary multivariable problems, a closed and bounded constraint set often provides this guarantee.
Example: Product on a Circle
Find the extrema of
\[ f(x,y)=xy \]subject to
\[ x^2+y^2=1. \]We have
\[ \nabla f=\langle y,x\rangle \]and
\[ \nabla g=\langle2x,2y\rangle. \]Lagrange Equations for the Product
Solve
\[ y=2\lambda x \] \[ x=2\lambda y \]together with
\[ x^2+y^2=1. \]Multiplying the first equation by \(y\) and the second by \(x\) gives
\[ y^2=2\lambda xy \]and
\[ x^2=2\lambda xy. \]Hence,
\[ x^2=y^2. \]Candidate Points for the Product
Since
\[ x^2=y^2, \]we have
\[ y=\pm x. \]Using
\[ x^2+y^2=1, \]gives
\[ x^2=\frac12. \]Therefore the four candidates are
\[ \left( \pm\frac1{\sqrt2}, \pm\frac1{\sqrt2} \right) \]with independent sign patterns consistent with \(y=\pm x\).
Classifying the Product Example
When the signs agree,
\[ xy=\frac12. \]When the signs differ,
\[ xy=-\frac12. \]Therefore,
\[ \boxed{ f_{\max}=\frac12 } \]and
\[ \boxed{ f_{\min}=-\frac12 }. \]Elimination as an Alternative
Not every constrained problem requires Lagrange multipliers.
If the constraint can be solved conveniently for one variable, substitute that expression into the objective and reduce the problem to one-variable calculus.
Lagrange multipliers are especially valuable when direct elimination is inconvenient or obscures symmetry.
Example: Solve the Constraint Directly
Suppose
\[ x+y=10. \]Then
\[ y=10-x. \]If the objective is
\[ f(x,y)=xy, \]substitute:
\[ f(x,10-x)=x(10-x). \]The problem becomes an ordinary one-variable quadratic optimization.
Example: Maximum Product with Fixed Sum
Differentiate
\[ q(x)=10x-x^2. \]Then
\[ q'(x)=10-2x. \]Set
\[ q'(x)=0, \]giving
\[ x=5. \]Therefore,
\[ y=5, \]and the maximum product is
\[ \boxed{25}. \]Geometric Meaning of the Multiplier
At a constrained optimum,
\[ \nabla f=\lambda\nabla g. \]The multiplier tells how strongly the objective gradient aligns with the constraint normal.
In many parameterized optimization problems, \(\lambda\) also has a sensitivity interpretation.
Multiplier as Sensitivity
Suppose the constraint is
\[ g(x,y)=c \]and the optimized value depends on \(c\).
Under suitable regularity conditions, the multiplier is related to the rate at which the optimal objective value changes as the constraint level \(c\) changes.
This interpretation is important in economics, engineering, and resource allocation.
Three Variables with One Constraint
To optimize
\[ f(x,y,z) \]subject to
\[ g(x,y,z)=c, \]solve
\[ \boxed{ \nabla f = \lambda\nabla g } \]together with the constraint equation.
Coordinate Form in Three Variables
The system is
\[ f_x=\lambda g_x, \] \[ f_y=\lambda g_y, \] \[ f_z=\lambda g_z, \]together with
\[ g(x,y,z)=c. \]Example: Linear Objective on a Sphere
Optimize
\[ f(x,y,z)=x+2y+2z \]subject to
\[ x^2+y^2+z^2=9. \]The gradients are
\[ \nabla f=\langle1,2,2\rangle \]and
\[ \nabla g=\langle2x,2y,2z\rangle. \]Set Up the Sphere Equations
We require
\[ 1=2\lambda x, \] \[ 2=2\lambda y, \] \[ 2=2\lambda z. \]Therefore,
\[ x:y:z=1:2:2. \]Apply the Sphere Constraint
Write
\[ (x,y,z)=t(1,2,2). \]Then
\[ x^2+y^2+z^2 = t^2(1+4+4) = 9t^2. \]Since the constraint requires \(9\),
\[ 9t^2=9, \]so
\[ t=\pm1. \]Classify the Sphere Example
The candidate points are
\[ (1,2,2) \]and
\[ (-1,-2,-2). \]Evaluate:
\[ f(1,2,2)=9 \]and
\[ f(-1,-2,-2)=-9. \]Therefore,
\[ \boxed{ f_{\max}=9, \qquad f_{\min}=-9 }. \]Connection to the Cauchy-Schwarz Inequality
The previous example also follows from
\[ |\mathbf{a}\cdot\mathbf{x}| \le \|\mathbf{a}\|\|\mathbf{x}\|. \]With
\[ \mathbf{a}=\langle1,2,2\rangle \]and
\[ \|\mathbf{x}\|=3, \]we obtain
\[ |f| \le 3\cdot3 = 9. \]Lagrange multipliers recover the same geometric alignment condition.
Multiple Equality Constraints
Suppose we want to optimize
\[ f(x,y,z) \]subject to two constraints
\[ g(x,y,z)=c \]and
\[ h(x,y,z)=d. \]Under suitable regularity conditions, the constrained-extremum condition is
\[ \boxed{ \nabla f = \lambda\nabla g + \mu\nabla h }. \]Why Two Multipliers Appear
With two independent constraints, the feasible directions lie in the intersection of two tangent conditions.
The objective gradient must lie in the span of the two constraint normals.
The scalars
\[ \lambda \qquad\text{and}\qquad \mu \]describe that linear combination.
Regularity of the Constraint
The standard one-constraint Lagrange condition assumes the constraint is smooth and
\[ \nabla g\neq\mathbf{0} \]at the point.
If
\[ \nabla g=\mathbf{0}, \]the usual geometric normal direction is not available, and the point requires separate attention.
Singular Constraint Points
A singular point of the constraint can still contain a constrained extremum.
Therefore, a complete analysis should not discard feasible points merely because the standard Lagrange equations fail there.
Singular feasible points should be checked separately.
Equality Constraints versus Inequality Constraints
A constraint such as
\[ g(x,y)=c \]restricts the search to a boundary curve or surface.
An inequality such as
\[ g(x,y)\le c \]defines a region that may contain interior points as well as a boundary.
Optimization over a Closed Region
Suppose we optimize \(f(x,y)\) over a closed bounded region \(R\).
A complete search typically includes:
- interior critical points where \(\nabla f=\mathbf{0}\);
- boundary extrema along each boundary component;
- corner or endpoint values;
- singular boundary points when present.
Why Boundary Checks Matter
Absolute extrema on a closed region often occur on the boundary rather than at interior critical points.
Ignoring the boundary can therefore miss the true maximum or minimum.
Example: Optimization on a Disk
Suppose
\[ f(x,y)=x^2+y \]on the closed disk
\[ x^2+y^2\le1. \]We must check both the interior and the boundary circle.
Interior Candidates on the Disk
The gradient is
\[ \nabla f = \langle2x,1\rangle. \]Since
\[ \nabla f=\mathbf{0} \]would require both
\[ 2x=0 \]and
\[ 1=0, \]there are no interior critical points.
Boundary Candidates on the Disk
On
\[ x^2+y^2=1, \]use Lagrange multipliers:
\[ \langle2x,1\rangle = \lambda\langle2x,2y\rangle. \]These equations, together with the circle constraint, produce the boundary candidates.
The absolute extrema are determined by comparing all feasible candidate values.
Boundary Parameterization as an Alternative
Instead of Lagrange multipliers, a boundary can sometimes be parameterized.
For the unit circle,
\[ x=\cos t, \qquad y=\sin t. \]Then the constrained objective becomes a one-variable function
\[ F(t)=f(\cos t,\sin t). \]Ordinary one-variable optimization can then be used.
Choosing Between Methods
Useful constrained-optimization methods include:
- direct substitution;
- parameterization;
- Lagrange multipliers;
- symmetry or inequalities;
- numerical optimization when symbolic methods are impractical.
The best method depends on the structure of the problem.
Economic Interpretation
Suppose a firm wants to maximize
\[ P(x,y) \]subject to a resource constraint
\[ C(x,y)=B. \]Then the Lagrange condition
\[ \nabla P = \lambda\nabla C \]says that at the constrained optimum, the direction of greatest profit increase is aligned with the normal to the budget boundary.
Resource-Allocation Interpretation
If two inputs consume a fixed resource, an optimal allocation occurs where the marginal benefit of reallocating resources can no longer improve the objective without violating the constraint.
Lagrange multipliers formalize this balance.
Geometric Design Example
Suppose a rectangle has fixed perimeter
\[ 2x+2y=P \]and area
\[ A=xy. \]Maximizing area subject to fixed perimeter is a constrained-extrema problem.
Fixed-Perimeter Rectangle with Lagrange Multipliers
Let
\[ f(x,y)=xy \]and
\[ g(x,y)=2x+2y=P. \]Then
\[ \nabla f=\langle y,x\rangle \]and
\[ \nabla g=\langle2,2\rangle. \]Thus,
\[ y=2\lambda, \qquad x=2\lambda. \]Therefore,
\[ x=y. \]Conclusion of the Rectangle Example
Since
\[ x=y, \]the perimeter condition gives
\[ 4x=P. \]Hence,
\[ x=y=\frac{P}{4}. \]So among rectangles with fixed perimeter, the square has maximum area.
Distance Problems as Constrained Optimization
The distance from a point \((x,y)\) to the origin is
\[ d=\sqrt{x^2+y^2}. \]Because the square root is increasing, minimizing distance is equivalent to minimizing
\[ d^2=x^2+y^2. \]This often simplifies constrained distance problems.
Example: Closest Point on a Line
Find the point on
\[ x+y=4 \]closest to the origin.
Minimize
\[ f(x,y)=x^2+y^2 \]subject to
\[ g(x,y)=x+y=4. \]Solve the Closest-Point Problem
The gradients are
\[ \nabla f=\langle2x,2y\rangle \]and
\[ \nabla g=\langle1,1\rangle. \]Thus,
\[ 2x=\lambda, \qquad 2y=\lambda. \]Therefore,
\[ x=y. \]Using
\[ x+y=4, \]gives
\[ \boxed{ (x,y)=(2,2) }. \]Second-Order Classification
For many elementary constrained problems on compact sets, direct comparison of candidate values is sufficient.
More advanced methods use second-order information restricted to feasible directions.
These methods involve the Hessian, bordered Hessians, or second variations.
The Lagrangian Function
Another common formulation defines
\[ \boxed{ \mathcal{L}(x,y,\lambda) = f(x,y) - \lambda\bigl(g(x,y)-c\bigr) }. \]The stationarity conditions are
\[ \mathcal{L}_x=0, \qquad \mathcal{L}_y=0, \qquad \mathcal{L}_\lambda=0. \]Equivalence of the Lagrangian Form
The equations
\[ \mathcal{L}_x=0 \]and
\[ \mathcal{L}_y=0 \]give
\[ \nabla f = \lambda\nabla g. \]The equation
\[ \mathcal{L}_\lambda=0 \]recovers the constraint
\[ g(x,y)=c. \]Sign Convention for the Lagrangian
Some authors define
\[ \mathcal{L} = f+\lambda(g-c) \]instead of
\[ f-\lambda(g-c). \]Either convention is valid.
The multiplier changes sign, but the feasible optimizer does not.
Common Error: Forgetting the Constraint Equation
Solving only
\[ \nabla f=\lambda\nabla g \]is incomplete.
The candidate must also satisfy
\[ \boxed{ g(x,y)=c }. \]Common Error: Setting \(\nabla f=\mathbf{0}\)
In a constrained problem, the optimum may occur where
\[ \nabla f\neq\mathbf{0}. \]The correct regularity condition is usually
\[ \nabla f=\lambda\nabla g, \]not
\[ \nabla f=\mathbf{0}. \]Common Error: Ignoring Feasibility
A point found algebraically is not a valid constrained candidate unless it satisfies every constraint.
Always substitute candidate points back into the original constraint equations.
Common Error: Dividing by a Variable Too Early
Lagrange systems often contain products such as
\[ x(1-2\lambda)=0. \]Dividing by \(x\) would lose the possibility
\[ x=0. \]Separate cases before dividing by expressions that may vanish.
Common Error: Assuming Every Lagrange Candidate Is an Extremum
The method produces necessary-condition candidates under appropriate hypotheses.
Candidates should be evaluated, compared, and interpreted in the context of the feasible set.
Common Error: Missing Singular Constraint Points
If
\[ \nabla g=\mathbf{0} \]at a feasible point, the standard Lagrange condition may not capture that point.
Such singular feasible points require separate checking.
Common Error: Ignoring Corners and Endpoints
For inequality-constrained or piecewise boundaries, extrema can occur at corners, endpoints, or intersections of boundary pieces.
These should be included in any absolute-extrema comparison.
A Reliable One-Constraint Lagrange Procedure
- Identify the objective \(f\).
- Write the equality constraint as \(g=c\).
- Compute \(\nabla f\) and \(\nabla g\).
- Set \(\nabla f=\lambda\nabla g\).
- Include the original constraint equation.
- Solve the full system without discarding zero cases prematurely.
- Check that every candidate is feasible.
- Check singular feasible points separately if necessary.
- Evaluate \(f\) at every candidate.
- Compare values to identify maxima and minima.
A Reliable Closed-Region Procedure
- Find interior critical points.
- Analyze each smooth boundary component.
- Use substitution, parameterization, or Lagrange multipliers on the boundary.
- Check corners, endpoints, and singular boundary points.
- Evaluate the objective at every feasible candidate.
- Compare all values to determine absolute extrema.
Example: Full Lagrange Workflow
Maximize and minimize
\[ f(x,y)=3x+4y \]subject to
\[ x^2+y^2=25. \]The gradients are
\[ \nabla f=\langle3,4\rangle \]and
\[ \nabla g=\langle2x,2y\rangle. \]Set Up the Full Workflow
The equations are
\[ 3=2\lambda x, \] \[ 4=2\lambda y, \]together with
\[ x^2+y^2=25. \]The first two equations imply
\[ x:y=3:4. \]Complete the Full Workflow
Write
\[ (x,y)=t(3,4). \]Then
\[ x^2+y^2 = 25t^2. \]Since the constraint requires \(25\),
\[ t=\pm1. \]The candidate points are
\[ (3,4) \]and
\[ (-3,-4). \]Compare the Full Workflow Values
Evaluate:
\[ f(3,4) = 9+16 = 25 \]and
\[ f(-3,-4) = -25. \]Therefore,
\[ \boxed{ f_{\max}=25 } \]and
\[ \boxed{ f_{\min}=-25 }. \]Practice 1: Linear Objective on a Circle
Find the constrained extrema of
\[ f(x,y)=x+2y \]subject to
\[ x^2+y^2=1. \]Practice 2: Product on a Circle
Find the maximum and minimum of
\[ f(x,y)=xy \]subject to
\[ x^2+y^2=4. \]Practice 3: Fixed Sum
Maximize
\[ f(x,y)=xy \]subject to
\[ x+y=12. \]Practice 4: Closest Point
Find the point on
\[ x+2y=6 \]closest to the origin.
Practice 5: Three Variables
Find the maximum and minimum of
\[ f(x,y,z)=x+y+z \]subject to
\[ x^2+y^2+z^2=3. \]Practice 6: Fixed Perimeter Rectangle
A rectangle has perimeter \(40\).
Find the dimensions that maximize area.
Practice 7: Lagrangian
Write a Lagrangian for maximizing
\[ f(x,y)=x^2y \]subject to
\[ x+y=5. \]Practice 8: Two Constraints
State the Lagrange-multiplier condition for optimizing
\[ f(x,y,z) \]subject to
\[ g(x,y,z)=0 \]and
\[ h(x,y,z)=0. \]Practice 9: Boundary Strategy
List the main classes of points that should be checked when finding absolute extrema of a continuous function on a closed bounded planar region.
Practice 10: Identify the Error
A student wants to optimize
\[ f(x,y) \]subject to
\[ g(x,y)=1 \]and solves only
\[ \nabla f=\lambda\nabla g. \]Explain what is missing.
Practice Answers
Practice 1
We have
\[ \nabla f=\langle1,2\rangle \]and
\[ \nabla g=\langle2x,2y\rangle. \]The optimizer lies in the direction of
\[ \langle1,2\rangle. \]Since its magnitude is
\[ \sqrt5, \]the candidate points are
\[ \left( \frac1{\sqrt5}, \frac2{\sqrt5} \right) \]and the opposite point.
Therefore,
\[ \boxed{ f_{\max}=\sqrt5, \qquad f_{\min}=-\sqrt5 }. \]Practice 2
On
\[ x^2+y^2=4, \]the extrema occur when
\[ |x|=|y|=\sqrt2. \]Equal signs give
\[ xy=2, \]and opposite signs give
\[ xy=-2. \]Therefore,
\[ \boxed{ f_{\max}=2, \qquad f_{\min}=-2 }. \]Practice 3
The fixed-sum product is maximized when the two variables are equal:
\[ x=y=6. \]Therefore,
\[ \boxed{ f_{\max}=36 }. \]Practice 4
Minimize
\[ f=x^2+y^2 \]subject to
\[ x+2y=6. \]The gradient condition gives
\[ \langle2x,2y\rangle = \lambda\langle1,2\rangle. \]Hence
\[ x=\frac{\lambda}{2}, \qquad y=\lambda. \]Apply the constraint:
\[ \frac{\lambda}{2}+2\lambda=6. \]Thus,
\[ \lambda=\frac{12}{5}. \]Therefore,
\[ \boxed{ (x,y)= \left( \frac65, \frac{12}{5} \right) }. \]Practice 5
The objective gradient is
\[ \langle1,1,1\rangle. \]On the sphere of radius \(\sqrt3\), the extrema occur in the directions parallel and antiparallel to this vector.
The points are
\[ (1,1,1) \]and
\[ (-1,-1,-1). \]Therefore,
\[ \boxed{ f_{\max}=3, \qquad f_{\min}=-3 }. \]Practice 6
If the sides are \(x\) and \(y\),
\[ 2x+2y=40. \]Maximum area occurs at
\[ x=y. \]Therefore,
\[ \boxed{ x=y=10 }. \]Practice 7
One valid convention is
\[ \boxed{ \mathcal{L}(x,y,\lambda) = x^2y - \lambda(x+y-5) }. \]Practice 8
Under suitable regularity conditions,
\[ \boxed{ \nabla f = \lambda\nabla g + \mu\nabla h }. \]The two constraint equations must also be included.
Practice 9
Check:
- interior critical points;
- smooth boundary candidates;
- corners and endpoints;
- singular feasible boundary points.
Practice 10
The original feasibility condition is missing.
The full system must include
\[ \boxed{ g(x,y)=1 }. \]Solving only the gradient-parallelism equation can produce points that do not lie on the constraint.
What to Remember
Constrained optimization searches for extrema only among feasible inputs.
- Equality constraints define curves, surfaces, or higher-dimensional feasible sets.
- At a regular equality-constrained extremum, \(\nabla f=\lambda\nabla g\).
- The original constraint equation must remain part of the system.
- Lagrange equations produce candidates, not automatic classifications.
- Compare objective values at all feasible candidates.
- Check singular constraint points separately when necessary.
- For closed regions, include interior, boundary, corner, and endpoint analysis.
- Multiple independent equality constraints introduce multiple multipliers.
The central geometric condition is
\[ \boxed{ \text{objective level set tangent to constraint} \Longrightarrow \nabla f \parallel \nabla g }. \]Where We Go Next
Constrained extrema rely on equations that tie variables together even when no variable has been solved explicitly in terms of the others. This type of dependence appears throughout multivariable analysis.
The next lesson, Implicit Relationships, studies equations that define relationships indirectly, develops implicit differentiation in several variables, and connects those relationships to gradients, tangent spaces, and local solvability.