Newton's Method

The Central Idea

To solve

begin with an estimate \(x_n\).

Use the tangent line to \(y=f(x)\) at \(x_n\) to estimate where the graph crosses the \(x\)-axis.

The resulting update is

Repeating this formula produces successive approximations to a root.

Why Tangent Lines Help

Near a point where a differentiable function is reasonably smooth, its tangent line provides a local linear approximation to the curve.

A nonlinear equation may be difficult to solve directly, but a linear equation is easy to solve.

Newton's method repeatedly replaces the difficult local problem with a simpler tangent-line problem.

The Tangent Line at \(x_n\)

The tangent line to \(y=f(x)\) at

has slope

Using point-slope form,

Finding the Tangent-Line Intercept

Newton's method uses the \(x\)-intercept of the tangent line as the next estimate.

At the \(x\)-intercept,

Therefore,

Deriving Newton's Formula

Divide by \(f'(x_n)\), assuming

Add \(x_n\):

The Newton Update

Each step has three computational ingredients:

1. evaluate \(f(x_n)\);
2. evaluate \(f'(x_n)\);
3. subtract the quotient \(f(x_n)/f'(x_n)\) from \(x_n\).

The correction term is

Example: Approximating \(\sqrt{2}\)

To compute \(\sqrt2\), solve

Define

Then

Newton's formula becomes

Simplifying the \(\sqrt2\) Iteration

Simplify:

Therefore,

This is the same square-root iteration encountered earlier as a fixed-point rule.

First Newton Step for \(\sqrt2\)

Choose

Then

Second Newton Step

Use

Then

Third Newton Step

Continue:

This gives

Fourth Newton Step

One more iteration gives

Compare this with

Only a few iterations have produced many correct digits.

A Newton Table for \(\sqrt2\)

The rapid stabilization is typical of Newton's method when it is already near a simple root.

Newton's Method Is an Iteration

Newton's method has the fixed-point form

with

It is therefore a specialized fixed-point iteration built from the function and its derivative.

Example: Solving \(x^3-x-2=0\)

Let

Then

Newton's iteration is

First Step for the Cubic

Choose

Compute

Also,

Therefore,

Continuing the Cubic Example

Applying Newton's formula again gives approximately

followed by

Thus the real root is

This is the same root previously located by bracketing and bisection, but Newton's method reaches it much faster from this suitable starting value.

Newton's Method and Tangent Geometry

At each step:

1. start at the point \((x_n,f(x_n))\) on the curve;
2. draw the tangent line there;
3. find the tangent line's \(x\)-intercept;
4. call that intercept \(x_{n+1}\);
5. repeat from the new point on the curve.

The geometry explains why the derivative appears in the formula.

Why Newton's Method Can Be Fast

Near a simple root, the tangent line often approximates the curve very accurately.

Under suitable smoothness conditions and with a sufficiently good starting value, Newton's method has quadratic convergence.

Informally, once the method is close enough to the root, the number of correct digits can roughly double from one iteration to the next.

What Is a Simple Root?

A root \(r\) is simple when

and

Graphically, the curve crosses the axis with a nonzero slope at the root.

Newton's fastest standard convergence behavior is associated with simple roots.

Multiple Roots Can Slow Newton's Method

Consider

The root

has multiplicity \(2\).

Since

Newton's formula becomes

Newton's Update for the Double Root

For \(x_n\ne1\),

Therefore,

The error is only halved at each step, so convergence is linear rather than quadratic.

The Derivative Cannot Be Zero in the Update

Newton's formula contains

Therefore, the update is undefined when

A horizontal tangent at the current iterate can cause the method to fail at that step.

Example: Immediate Failure

Consider

Then

If we choose

then

Newton's formula would require division by zero, so this starting value is invalid.

A Small Derivative Can Cause a Large Jump

Even when

a derivative with very small magnitude can make

very large.

The next iterate may jump far away from the current region.

The Initial Guess Matters

Newton's method is not globally guaranteed to converge from every starting value.

A poor initial guess may lead to:

• slow convergence;
• large jumps;
• oscillation;
• cycling;
• convergence to a different root;
• an undefined update;
• divergence.

Using a Graph to Choose \(x_0\)

A graph can help identify the approximate location of a root and show the local shape of the function.

Choosing \(x_0\) near the desired root often improves the likelihood of successful convergence.

Graphing is especially useful when several roots are present.

Using a Bracket to Choose \(x_0\)

Suppose a previous method has established

Newton's Method Is an Open Method

Bisection maintains an interval guaranteed to contain a root.

Newton's method usually tracks only one current estimate.

Because it does not preserve a bracket automatically, Newton's method is called an open method.

Open methods can be fast, but they generally require more care.

Newton versus Bisection

Bisection offers:

• a guaranteed bracket under its assumptions;
• predictable interval reduction;
• simple error bounds;
• no derivative requirement.

Newton's method offers:

• very fast local convergence;
• a single-value update;
• high precision in relatively few iterations when conditions are favorable.

Newton versus Fixed-Point Iteration

Basic fixed-point iteration uses

for a chosen rearrangement.

Newton's method chooses the specific function

The derivative-based construction is what gives Newton's method its special local convergence properties.

Stopping by Successive Difference

A common stopping condition is

This measures how much the estimate changed during the latest Newton step.

Stopping by Residual

Another common condition is

This checks how closely the current approximation satisfies the original equation.

Combining Stopping Conditions

A practical implementation may require both

and

The first controls movement in \(x\).

The second controls the residual.

Why a Small Step Is Not Enough by Itself

A small value of

does not automatically prove that the current estimate is close to a root.

Numerical stagnation, roundoff, or unusual function behavior can make successive estimates similar.

Residual checking provides additional evidence.

Why a Small Residual Is Not Always the Whole Story

A small residual

can be useful, but the relationship between residual size and actual root error depends on the local behavior of the function.

When \(f'(x)\) is very small, a small residual may correspond to a larger error in \(x\) than expected.

Maximum Iteration Safeguard

Because Newton's method can fail to converge, computational implementations should impose a maximum iteration count.

For example,

If the stopping criteria are not met within the limit, the algorithm can report nonconvergence rather than continue indefinitely.

Newton's Method in Pseudocode

set x = initial guess

repeat
    evaluate f(x)
    evaluate f'(x)

    if f'(x) = 0
        stop with failure

    new_x = x - f(x) / f'(x)

    if stopping condition is satisfied
        return new_x

    x = new_x
until maximum iterations are reached

A More Defensive Newton Algorithm

set x = initial guess

repeat
    fx = f(x)
    dfx = f'(x)

    if abs(dfx) is too small
        stop or switch methods

    new_x = x - fx / dfx

    if abs(new_x - x) < x_tolerance
       and abs(f(new_x)) < residual_tolerance
        return new_x

    x = new_x
until maximum iterations are reached

Real numerical software often includes safeguards beyond the basic formula.

Hybrid Methods

Numerical solvers often combine the reliability of bracketing with the speed of Newton's method.

A typical strategy is:

1. first locate a root inside a valid bracket;
2. attempt a Newton step;
3. accept the Newton step when it behaves safely;
4. fall back to a bracketing step when necessary.

Hybrid strategies reduce the weaknesses of either method used alone.

Newton's Method for General Square Roots

To approximate

solve

Since

Newton's method becomes

Example: Approximating \(\sqrt{10}\)

Use

with

Then

Continuing gives

Thus,

Example: Solving \(\cos x=x\) with Newton's Method

Rewrite as

Then

Newton's formula becomes

Newton Iteration for \(\cos x=x\)

Begin with

The first update gives approximately

The next gives

followed by

Therefore,

This is much faster than the basic fixed-point iteration \(x_{n+1}=\cos x_n\).

Newton's Method and Local Linearization

The tangent-line equation

is the first-order linear approximation of \(f\) near \(x_n\).

Setting this approximation equal to zero gives

which leads directly to Newton's update.

Newton's Method as Error Correction

The quantity

tells us how far the current estimate is from satisfying the equation in output space.

Dividing by the slope

converts that output discrepancy into an approximate horizontal correction.

Newton's step subtracts that correction from the current estimate.

Common Error: Forgetting the Derivative

Newton's method is not

The derivative is essential:

Common Error: Evaluating \(f\) and \(f'\) at Different Inputs

Both

and

must be evaluated at the same current iterate \(x_n\).

Mixing \(f(x_n)\) with \(f'(x_{n-1})\) changes the method.

Common Error: Losing the Minus Sign

The formula is

Replacing the subtraction with addition generally sends the iteration in a different direction.

Common Error: Using a Point Where \(f'(x_n)=0\)

If

the standard Newton update is undefined.

Choose a different starting value or switch to another numerical method.

Common Error: Assuming Convergence Is Guaranteed

Newton's method can be extremely effective, but it is not guaranteed to converge from every starting point.

Always monitor the iterates and use stopping and failure conditions.

Common Error: Ignoring Which Root Was Found

If a function has several roots, different initial guesses may converge to different solutions.

The final result should therefore be checked against the intended search region or application.

Common Error: Rounding Too Early

Newton's method often gains precision rapidly.

Rounding every intermediate iterate too aggressively can throw away that advantage.

Retain sufficient internal precision and round the final reported answer according to the required tolerance.

Common Error: Stopping Without Verification

A stable-looking decimal is not enough.

Check the final value by evaluating

The residual should be consistent with the claimed accuracy.

A Reliable Newton Workflow

1. Rewrite the problem as \(f(x)=0\).
2. Compute \(f'(x)\).
3. Choose a reasonable initial estimate \(x_0\).
4. Evaluate \(f(x_n)\) and \(f'(x_n)\).
5. Check that \(f'(x_n)\) is not zero or dangerously small.
6. Compute \(x_{n+1}=x_n-f(x_n)/f'(x_n)\).
7. Record the new approximation.
8. Check the change in \(x\) and the residual.
9. Repeat until the stopping conditions are satisfied.
10. Verify the final approximation in the original equation.

Example: Full Newton Workflow

Approximate the positive root of

Define

Choose

Then

Continue the \(\sqrt7\) Workflow

Next,

One more step gives

Therefore,

Practice 1: Write the Newton Formula

For

compute \(f'(x)\) and write the Newton update.

Practice 2: First Iteration

For

with

compute \(x_1\).

Practice 3: Second Iteration

Continue Practice 2 and compute \(x_2\).

Practice 4: Cubic Update

For

write the Newton iteration formula.

Practice 5: Derivative Failure

Explain why

is invalid for Newton's method applied to

Practice 6: Successive-Difference Test

Suppose

and

Does

hold?

Practice 7: Residual

For

and

compute the residual.

Practice 8: Simple or Multiple Root?

For

determine whether \(x=3\) is a simple root.

Practice 9: Compare Methods

State one reason Newton's method may be faster than bisection and one reason bisection may be more reliable.

Practice 10: Interpret the Geometry

What geometric point becomes \(x_{n+1}\) in Newton's method?

Practice Answers

Practice 1

Since

we have

Therefore,

Practice 2

Practice 3

Since

we obtain

Therefore,

Practice 4

For

the derivative is

Therefore,

Practice 5

For

we have

At

the derivative is

Newton's formula would divide by zero, so the update is undefined.

Practice 6

Since

the condition is satisfied.

Practice 7

Practice 8

Since

the root \(x=3\) has multiplicity \(2\).

Also,

Therefore, \(x=3\) is not a simple root.

Practice 9

Newton's method can converge quadratically near a suitable simple root, so it may gain correct digits very rapidly.

Bisection preserves a valid root bracket and has predictable interval reduction, making it more robust under its assumptions.

Practice 10

The next iterate \(x_{n+1}\) is the \(x\)-coordinate where the tangent line at

crosses the \(x\)-axis.

What to Remember

• Newton's method solves \(f(x)=0\) using tangent-line approximations.
• The update is \(x_{n+1}=x_n-f(x_n)/f'(x_n)\).
• The method requires derivative information.
• Near a suitable simple root, convergence can be quadratic.
• Multiple roots usually reduce the standard rate of convergence.
• The update fails when \(f'(x_n)=0\).
• A very small derivative can produce a very large Newton step.
• The initial guess strongly affects convergence.
• Stopping criteria should monitor both iterate changes and residuals when appropriate.
• Bracketing and hybrid methods can improve the reliability of Newton-based solvers.

Where We Go Next

Newton's method gains speed by using derivative information. Sometimes, however, a derivative is unavailable, expensive, or inconvenient to evaluate.

The next lesson, Secant Methods, replaces the derivative with a slope estimated from two nearby function values and develops a derivative-free root-finding procedure closely related to Newton's method.