The Central Idea
A point can be better than every nearby point without being the best point in the entire problem.
This leads to two different notions of optimality:
- local optimality, which compares nearby feasible points;
- global optimality, which compares all feasible points.
In optimization, knowing the difference is essential.
What Is an Extremum?
An extremum is either a maximum or a minimum.
The plural form is extrema.
A function may have:
- local maxima;
- local minima;
- global maxima;
- global minima.
Local Maximum
A point \(x^*\) is a local maximum of \(f\) if the function value at \(x^*\) is at least as large as the function values at all sufficiently nearby feasible points.
Informally,
\[ \boxed{ f(x^*)\ge f(x) } \]for all feasible \(x\) close enough to \(x^*\).
Local Minimum
A point \(x^*\) is a local minimum if its function value is at most as large as the function values at all sufficiently nearby feasible points.
\[ \boxed{ f(x^*)\le f(x) } \]for all feasible \(x\) close enough to \(x^*\).
Global Maximum
A point \(x^*\) is a global maximum if
\[ \boxed{ f(x^*)\ge f(x) } \]for every feasible \(x\).
The comparison is not merely local. It covers the entire feasible set.
Global Minimum
A point \(x^*\) is a global minimum if
\[ \boxed{ f(x^*)\le f(x) } \]for every feasible \(x\).
Global Extrema Are Automatically Local
If a point is best among all feasible points, then it is certainly best among nearby feasible points.
Therefore, every global maximum is also a local maximum in the feasible sense, and every global minimum is also a local minimum.
The converse is not generally true.
A Local Extremum Need Not Be Global
Consider a function with several peaks.
One peak may be higher than the values immediately around it but lower than another peak elsewhere.
The first is a local maximum.
The higher peak may be the global maximum.
Neighborhoods and Local Behavior
The word local refers to a neighborhood around the point.
For one variable, a neighborhood of \(x^*\) can be described by
\[ |x-x^*|<\delta \]for some positive number \(\delta\).
Local optimality concerns the behavior of \(f\) within such a neighborhood.
Strict Local Extrema
A strict local maximum satisfies
\[ f(x^*)>f(x) \]for every sufficiently nearby feasible \(x\ne x^*\).
A strict local minimum satisfies
\[ f(x^*)Non-Strict Extrema
If equality is allowed, several nearby points may share the same optimal value.
For example, a function may be constant on an interval:
\[ f(x)=5. \]Every point in that interval can be both a local maximum and a local minimum under the non-strict definition.
Example: A Simple Global Maximum
Consider
\[ f(x)=-(x-3)^2+10. \]Because
\[ -(x-3)^2\le0, \]we have
\[ f(x)\le10. \]Equality occurs at
\[ x=3. \]Therefore,
\[ \boxed{x=3} \]is the global maximizer and
\[ \boxed{f(3)=10} \]is the global maximum value.
Example: A Simple Global Minimum
Consider
\[ f(x)=(x+2)^2+4. \]Since
\[ (x+2)^2\ge0, \]we have
\[ f(x)\ge4. \]Equality occurs at
\[ x=-2. \]Thus
\[ \boxed{x=-2} \]is the global minimizer and
\[ \boxed{f(-2)=4} \]is the global minimum value.
Critical Points
For a differentiable one-variable function, interior local extrema often occur where
\[ \boxed{f'(x)=0}. \]Such points are called critical points or stationary points.
They are candidates for extrema, not automatic extrema.
Critical Numbers
A critical number is a value \(c\) in the domain of \(f\) where either
\[ f'(c)=0 \]or \(f'(c)\) does not exist.
Local extrema can occur at either type of critical number.
Example: A Critical Point That Is a Maximum
Consider
\[ f(x)=-x^2+4x+1. \]Differentiate:
\[ f'(x)=-2x+4. \]Set
\[ f'(x)=0. \]Then
\[ -2x+4=0 \]gives
\[ x=2. \]Because the parabola opens downward, \(x=2\) is a global maximum.
A Critical Point Need Not Be an Extremum
Consider
\[ f(x)=x^3. \]Then
\[ f'(x)=3x^2. \]At
\[ x=0, \]we have
\[ f'(0)=0. \]Yet \(f(x)=x^3\) continues increasing through the origin.
Therefore,
\[ \boxed{x=0\text{ is not a local maximum or minimum}.} \]First-Derivative Test
The sign of \(f'(x)\) reveals whether \(f\) is increasing or decreasing.
- If \(f'\) changes from positive to negative, there is a local maximum.
- If \(f'\) changes from negative to positive, there is a local minimum.
- If the sign does not change, the critical point is not a local extremum.
Example: First-Derivative Sign Change
Suppose
\[ f'(x)=(x-1)(x-4). \]Critical points occur at
\[ x=1 \]and
\[ x=4. \]The derivative is positive before \(1\), negative between \(1\) and \(4\), and positive after \(4\).
Therefore,
\[ \boxed{x=1\text{ is a local maximum}} \]and
\[ \boxed{x=4\text{ is a local minimum}.} \]Second-Derivative Test
Suppose
\[ f'(c)=0. \]If
\[ f''(c)>0, \]then the graph is locally concave up and \(c\) is a local minimum.
If
\[ f''(c)<0, \]then the graph is locally concave down and \(c\) is a local maximum.
When the Second-Derivative Test Is Inconclusive
If
\[ f''(c)=0, \]the second-derivative test does not determine whether \(c\) is a maximum, minimum, or neither.
Another method must be used.
Example: Second-Derivative Test
Let
\[ f(x)=x^2-6x+10. \]Then
\[ f'(x)=2x-6. \]The critical point is
\[ x=3. \]Also,
\[ f''(x)=2>0. \]Therefore,
\[ \boxed{x=3\text{ is a local minimum}.} \]Because the parabola opens upward over all real numbers, it is also the global minimum.
Endpoints Can Be Extrema
On a closed interval, maxima and minima can occur at endpoints.
Consider
\[ f(x)=x^2 \]on
\[ -1\le x\le3. \]The critical point is
\[ x=0. \]But the global maximum occurs at the endpoint
\[ x=3. \]The Closed-Interval Method
To find global extrema of a continuous function on
\[ [a,b], \]- find all critical numbers in \((a,b)\);
- evaluate the function at those critical numbers;
- evaluate the function at \(a\) and \(b\);
- compare all resulting values.
The largest value is the global maximum and the smallest value is the global minimum.
Example: Closed-Interval Method
Find the global extrema of
\[ f(x)=x^2-4x+5 \]on
\[ 0\le x\le5. \]Differentiate:
\[ f'(x)=2x-4. \]The interior critical point is
\[ x=2. \]Evaluate:
\[ f(0)=5, \] \[ f(2)=1, \] \[ f(5)=10. \]Therefore,
\[ \boxed{\text{global minimum }=1\text{ at }x=2} \]and
\[ \boxed{\text{global maximum }=10\text{ at }x=5}. \]Open Intervals Can Prevent Attainment
Consider
\[ f(x)=x \]on
\[ 0Therefore, \(f\) has neither a global minimum nor a global maximum on this interval.
Maximum Versus Supremum
The maximum must actually be attained by some feasible point.
The supremum is the least upper bound and need not be attained.
For
\[ f(x)=x,\qquad0but there is no maximum.
Minimum Versus Infimum
The minimum is an attained smallest value.
The infimum is the greatest lower bound and need not be attained.
For
\[ f(x)=x,\qquad0but there is no minimum.
Extreme Value Theorem
A central theorem guarantees extrema under important conditions.
If \(f\) is continuous on a closed and bounded interval
\[ [a,b], \]then \(f\) attains both a global maximum and a global minimum on that interval.
This is the Extreme Value Theorem.
Why Continuity Matters
A discontinuity can prevent an extreme value from being attained.
Continuity helps ensure that the function does not suddenly jump over candidate values or fail to take values approached within the domain.
Why Closedness Matters
If the feasible set omits a limiting boundary point, the best attainable value may fail to exist.
This is why replacing
\[ x\le5 \]with
\[ x<5 \]can fundamentally change an optimization problem.
Why Boundedness Matters
If the feasible set extends indefinitely, an objective may improve without limit.
For example,
\[ f(x)=x \]over all real numbers has no global maximum because \(f(x)\) grows without bound.
Multiple Global Extrema
A function may attain the same global optimum at several points.
Consider
\[ f(x)=x^2 \]on
\[ -2\le x\le2. \]The global maximum value is
\[ 4, \]attained at
\[ x=-2 \]and
\[ x=2. \]Unique Global Extrema
If exactly one feasible point attains the best value, the optimizer is unique.
For example,
\[ f(x)=x^2 \]has the unique global minimizer
\[ \boxed{x=0} \]over all real numbers.
Flat Regions
A function may remain constant over an interval or region.
If that constant value is globally optimal, then every point in the flat region is a global optimizer.
This is important because optimization does not always produce a single answer.
Local Extrema at Nondifferentiable Points
Local extrema can occur where a derivative does not exist.
Consider
\[ f(x)=|x|. \]The derivative does not exist at
\[ x=0, \]yet
\[ \boxed{x=0} \]is the global minimum.
Corners and Cusps
Nondifferentiable extrema often occur at corners or cusps.
Therefore, searching only for solutions of
\[ f'(x)=0 \]can miss valid candidates.
Boundary Extrema in Constrained Problems
In constrained optimization, a global optimum often lies on the boundary of the feasible region.
The boundary may represent a resource being fully used, a capacity being reached, or an equality requirement being satisfied.
Example: Constraint Creates a Boundary Optimum
Consider
\[ f(x,y)=x+y \]subject to
\[ x^2+y^2\le1. \]Increasing \(x+y\) pushes the solution toward the edge of the disk.
The global maximum occurs on the boundary
\[ x^2+y^2=1. \]Multivariable Critical Points
For a differentiable function
\[ f(x,y), \]an interior stationary point satisfies
\[ \boxed{ \nabla f(x,y)=\mathbf{0} }. \]Equivalently,
\[ \frac{\partial f}{\partial x}=0 \]and
\[ \frac{\partial f}{\partial y}=0. \]Example: Multivariable Minimum
Consider
\[ f(x,y)=x^2+y^2. \]The gradient is
\[ \nabla f= \begin{bmatrix} 2x\\ 2y \end{bmatrix}. \]Setting the gradient equal to zero gives
\[ x=0, \qquad y=0. \]Since
\[ x^2+y^2\ge0, \]the point
\[ \boxed{(0,0)} \]is the global minimizer.
Saddle Points
A multivariable critical point may be neither a local maximum nor a local minimum.
Such a point may be a saddle point.
Consider
\[ f(x,y)=x^2-y^2. \]At
\[ (0,0), \]the gradient is zero.
But along the \(x\)-axis,
\[ f(x,0)=x^2\ge0, \]while along the \(y\)-axis,
\[ f(0,y)=-y^2\le0. \]Therefore,
\[ \boxed{(0,0)\text{ is a saddle point}.} \]Why Saddle Points Matter
A stationary condition such as
\[ \nabla f=\mathbf{0} \]identifies a candidate, not a classification.
In higher-dimensional optimization, distinguishing minima, maxima, and saddle points becomes especially important.
Convex Functions Preview
Some functions have geometry that makes local and global optimality much easier to relate.
For a convex function, every local minimum is also a global minimum.
This important principle will be developed later in the lesson on Convexity.
Concave Functions Preview
For a concave function, every local maximum is also a global maximum under the appropriate domain conditions.
This is one reason convexity and concavity are central in optimization theory.
Extrema and Objective Contours
In two dimensions, objective contours can help visualize global and local extrema.
A local optimum may correspond to a nearby contour configuration that encloses a hill or valley, while a global optimum corresponds to the best attainable contour over the entire feasible region.
Extrema and Feasible Regions
A point can be a global minimum over one feasible region but not over another.
Global optimality is always relative to the domain or feasible set being considered.
Example: Same Function, Different Feasible Sets
Consider
\[ f(x)=x^2. \]Over all real numbers, the global minimum occurs at
\[ x=0. \]But if the feasible set is
\[ 2\le x\le5, \]then the global minimum occurs at
\[ \boxed{x=2}. \]The objective is unchanged; the feasible set changes the answer.
Optimization Candidates
In many one-variable problems, the main candidates for global extrema are:
- interior critical points where \(f'(x)=0\);
- interior points where \(f'\) does not exist;
- endpoints;
- other boundary points imposed by the feasible set.
Candidate Does Not Mean Solution
A candidate point must still be checked.
It may turn out to be:
- a local maximum;
- a local minimum;
- a saddle point or non-extremal stationary point;
- infeasible;
- suboptimal compared with another feasible candidate.
Comparing Objective Values
Once the relevant candidates are identified, the most direct global test is often to evaluate the objective at each candidate and compare the resulting values.
If
\[ f(x_1)=8, \qquad f(x_2)=12, \qquad f(x_3)=5, \]then among these candidates,
\[ \boxed{x_2} \]gives the largest objective value.
Local Search Versus Global Search
Some optimization algorithms move from a starting point by repeatedly improving the objective nearby.
Such procedures may naturally converge to a local optimum.
Finding a global optimum can require additional structure, multiple starting points, bounds, convexity, or specialized global-search methods.
Dependence on Starting Point
In a function with several local minima, a local search method can converge to different solutions depending on where the search begins.
This becomes important in nonlinear and nonconvex optimization.
Common Error: Treating Local as Global
Finding a local maximum does not prove that no larger value exists elsewhere.
A global claim requires comparison across the entire feasible set or a theorem that guarantees globality.
Common Error: Solving Only \(f'(x)=0\)
The equation
\[ f'(x)=0 \]does not capture:
- endpoints;
- nondifferentiable extrema;
- boundary extrema;
- critical points that are not extrema.
Common Error: Ignoring the Feasible Set
A point may be a global minimizer of a function over all real numbers but infeasible in the actual optimization problem.
Optimization always takes place relative to the allowed domain.
Common Error: Confusing Point and Value
If
\[ x^*=3 \]and
\[ f(3)=10, \]then \(3\) is the optimizing point and \(10\) is the optimal value.
These are different mathematical objects.
Common Error: Assuming Uniqueness
A problem can have several global maximizers or minimizers.
Unless uniqueness has been established, do not assume there is only one optimizer.
Common Error: Assuming Extrema Exist
A function may fail to attain a maximum or minimum because the domain is open, unbounded, empty, or otherwise unsuitable.
Existence should be checked rather than assumed.
A Reliable Extrema Workflow
- Identify the feasible set.
- Determine whether the problem asks for a maximum or minimum.
- Find interior critical points.
- Include points where derivatives fail to exist.
- Inspect endpoints and boundaries.
- Classify local behavior when needed.
- Evaluate the objective at all relevant candidates.
- Compare candidate values to determine global extrema.
- Check whether extrema actually exist.
- Determine whether the optimizer is unique or multiple.
- State both the optimizing point and optimal value.
- Interpret the result in context.
Practice 1: Identify the Extremum
Consider
\[ f(x)=-(x-2)^2+7. \]Find the global maximum and the maximizing value of \(x\).
Practice 2: Find a Global Minimum
Consider
\[ f(x)=(x+4)^2+3. \]Find the global minimum and the minimizing value of \(x\).
Practice 3: Critical Point Classification
Let
\[ f(x)=x^3. \]Determine whether \(x=0\) is a local maximum, local minimum, or neither.
Practice 4: Closed-Interval Extrema
Find the global extrema of
\[ f(x)=x^2-2x \]on
\[ 0\le x\le4. \]Practice 5: Endpoint Analysis
Consider
\[ f(x)=x \]on
\[ -2\le x\le5. \]Find the global minimum and global maximum.
Practice 6: Maximum or Supremum?
Consider
\[ f(x)=x \]on
\[ 0Practice 7: Multiple Global Maximizers
Consider
\[ f(x)=x^2 \]on
\[ -5\le x\le5. \]Find the global maximum value and all maximizing points.
Practice 8: Nondifferentiable Minimum
Consider
\[ f(x)=|x-3|. \]Find the global minimum and state whether the derivative exists there.
Practice 9: Multivariable Minimum
Consider
\[ f(x,y)=x^2+y^2+1. \]Find the global minimum and the minimizing point.
Practice 10: Identify the Error
A student solves
\[ f'(x)=0 \]and immediately declares every solution to be a global extremum.
Explain the error.
Practice Answers
Practice 1
Since
\[ -(x-2)^2\le0, \]the largest value is
\[ \boxed{7} \]at
\[ \boxed{x=2}. \]Practice 2
Since
\[ (x+4)^2\ge0, \]the smallest value is
\[ \boxed{3} \]at
\[ \boxed{x=-4}. \]Practice 3
Although
\[ f'(0)=0, \]the function continues increasing through \(0\).
Therefore,
\[ \boxed{x=0\text{ is neither a local maximum nor a local minimum}.} \]Practice 4
Differentiate:
\[ f'(x)=2x-2. \]The critical point is
\[ x=1. \]Evaluate:
\[ f(0)=0, \] \[ f(1)=-1, \] \[ f(4)=8. \]Therefore,
\[ \boxed{\text{global minimum }=-1\text{ at }x=1} \]and
\[ \boxed{\text{global maximum }=8\text{ at }x=4}. \]Practice 5
Because \(f(x)=x\) is increasing,
\[ \boxed{\text{global minimum }=-2\text{ at }x=-2} \]and
\[ \boxed{\text{global maximum }=5\text{ at }x=5}. \]Practice 6
No maximum exists because \(x=3\) is not included.
The supremum is
\[ \boxed{3}. \]Practice 7
The global maximum value is
\[ \boxed{25} \]at
\[ \boxed{x=-5\text{ and }x=5}. \]Practice 8
The minimum occurs when
\[ x-3=0. \]Therefore,
\[ \boxed{x=3} \]and
\[ \boxed{f(3)=0}. \]The derivative does not exist at \(x=3\).
Practice 9
Since
\[ x^2+y^2\ge0, \]the smallest value is
\[ \boxed{1} \]at
\[ \boxed{(0,0)}. \]Practice 10
Solving
\[ f'(x)=0 \]only identifies stationary candidates.
Such a point may be a maximum, minimum, or neither, and global optimality requires comparison across the feasible set or an appropriate theorem.
What to Remember
Local extrema describe nearby optimality. Global extrema describe the best attainable values across the entire feasible set.
- a local maximum is best relative to nearby feasible points;
- a local minimum is smallest relative to nearby feasible points;
- a global maximum is at least as large as every feasible value;
- a global minimum is at most as large as every feasible value;
- critical points are candidates, not automatic extrema;
- endpoints and boundaries must be checked;
- nondifferentiable points can be extrema;
- several points can share one optimal value;
- an optimum may fail to exist;
- global optimality always depends on the feasible set.
The central distinction is
\[ \boxed{ \text{best nearby does not necessarily mean best overall} }. \]Where We Go Next
We now know what local and global optimality mean and how critical points, endpoints, and feasible boundaries become candidates for extrema.
The next step is to organize these ideas into methods for solving optimization problems when no explicit external constraints are imposed.
The next lesson, Unconstrained Optimization, develops systematic methods for locating and classifying optima using derivatives, gradients, and the structure of the objective function.