How calculus turns complicated functions into polynomials — local approximation, remainder theorems, and the mathematical foundation beneath nearly every numerical method in this book
A computer needs to evaluate \(\cos(0.1)\). It has no table of cosine values and no geometric meaning to invoke. What it has is arithmetic: addition, subtraction, multiplication, and division. How can a complicated transcendental function like cosine be reduced to a sequence of arithmetic operations?
The answer is polynomial approximation. Near any point where a function is smooth enough, a polynomial can be constructed that matches the function's value, slope, curvature, and higher derivatives. On a small enough interval, that polynomial is indistinguishable from the function itself — at least for practical purposes. Taylor's theorem makes this idea precise, tells us how well the polynomial fits, and tells us when the approximation is trustworthy.
This chapter develops Taylor polynomials and Taylor series as fundamental tools of numerical approximation. Taylor approximation is not merely a technique from Calculus II. It is the mathematical foundation of why finite difference formulas work, why Euler's method steps forward in time, why Newton's method converges, and why numerical algorithms for exponentials, trigonometric functions, and logarithms can be evaluated to arbitrary precision. Students who understand Taylor polynomials deeply will find that many numerical methods make far more sense.
Every calculator, every scientific software library, and every programming language needs to evaluate functions like \(e^x\), \(\sin(x)\), \(\cos(x)\), and \(\ln(x)\). None of these are polynomials. But all of them can be approximated by polynomials to any desired precision, using Taylor polynomials. Taylor approximation explains how computers evaluate transcendental functions using only basic arithmetic; why the error formulas for finite difference formulas (Chapter 6) have the form they do; how the step-by-step structure of Euler's method (Chapter 12) is justified; how Newton's method (Chapter 3) achieves its rapid convergence; and how error bounds can be placed on numerical approximations using derivatives. This chapter also develops careful mathematical reasoning about approximation error — reasoning that is used throughout the rest of the book.
By the end of this chapter, you should be able to:
Mathematics is full of functions that resist exact evaluation. The exponential \(e^x\), the sine and cosine, logarithms, the square root, the arctangent — these are transcendental functions, meaning they cannot be expressed as finite combinations of algebraic operations on \(x\). A computer cannot evaluate \(e^{0.5}\) directly from the definition \(e^x = \lim_{n \to \infty}\left(1 + \frac{x}{n}\right)^n\). The limit requires taking \(n\) to infinity, which is impossible in finite computation.
Polynomials, by contrast, are completely arithmetic. Evaluating a polynomial at a point requires only addition, subtraction, and multiplication — operations that all computers perform exactly (up to floating-point limits). If we can replace a complicated function with a polynomial that is close enough to it, we have turned an analytically difficult problem into a series of arithmetic steps.
This is the purpose of Taylor approximation. Given a smooth function \(f\) and a center point \(a\), Taylor's theorem tells us how to build a polynomial \(P_n(x)\) of degree \(n\) that matches \(f\) at the point \(a\) in the strongest possible way: the polynomial and the function agree in value, slope, curvature, and all derivatives up to order \(n\). Near the center \(a\), the polynomial behaves almost exactly like the function. Further from \(a\), the approximation may degrade, but Taylor's theorem gives a formula for bounding the error.
Why this matters for numerical methods specifically. In Chapter 6, the finite difference formula for the derivative was
\[ f'(x) \approx \frac{f(x+h) - f(x)}{h} \]
Where did the error term come from? Taylor's theorem. In Chapter 3, Newton's method was
\[ x_{n+1} = x_n - \frac{f(x_n)}{f'(x_n)} \]
Why does it converge so fast? Because Newton's method is derived by dropping the quadratic and higher terms from the Taylor expansion of \(f\) near the root. In Chapter 12, Euler's method advances the solution of a differential equation using only the first derivative. Its error analysis comes from the second-order term in Taylor's theorem.
Taylor polynomials are not a topic from calculus that numerical methods happens to use. They are the mathematical skeleton of numerical approximation itself.
Before building higher-degree Taylor polynomials, it helps to recall the simplest case, which students have already encountered in calculus under the name linearization or tangent line approximation.
If \(f\) is differentiable at \(a\), then the tangent line to the graph of \(f\) at the point \((a, f(a))\) has equation
\[ L(x) = f(a) + f'(a)(x - a) \]
The function \(L\) is a degree-1 polynomial. Near \(x = a\), the function \(f(x)\) and the linear approximation \(L(x)\) are close. Specifically, \(L(a) = f(a)\) and \(L'(a) = f'(a)\): the polynomial matches \(f\) in both value and slope at the center.
The linear approximation captures value and slope, but ignores curvature. If the function curves significantly over the interval of interest, the tangent line drifts away. To do better, we need polynomials of higher degree that also match the second derivative, third derivative, and so on. This is exactly what Taylor polynomials do.
Suppose we want to approximate \(f(x)\) near \(x = a\) with a polynomial \(P_2(x)\) of degree 2:
\[ P_2(x) = c_0 + c_1(x - a) + c_2(x - a)^2 \]
We want \(P_2\) to agree with \(f\) not just in value and slope, but also in curvature. We enforce:
\[ P_2(a) = f(a), \quad P_2'(a) = f'(a), \quad P_2''(a) = f''(a) \]
Computing the derivatives of \(P_2\):
\[ P_2'(x) = c_1 + 2c_2(x - a), \quad P_2''(x) = 2c_2 \]
Setting \(x = a\):
\[ P_2(a) = c_0 = f(a), \qquad P_2'(a) = c_1 = f'(a), \qquad P_2''(a) = 2c_2 = f''(a) \implies c_2 = \frac{f''(a)}{2} \]
So the degree-2 Taylor polynomial is:
\[ P_2(x) = f(a) + f'(a)(x - a) + \frac{f''(a)}{2}(x - a)^2 \]
Notice the pattern. The coefficient of \((x - a)^k\) is \(\frac{f^{(k)}(a)}{k!}\). This pattern continues to any order.
The pattern from Sections 8.2 and 8.3 generalizes cleanly.
Suppose \(f\) has \(n\) derivatives at the point \(a\). The Taylor polynomial of degree \(n\) for \(f\) centered at \(a\) is:
\[ P_n(x) = \sum_{k=0}^{n} \frac{f^{(k)}(a)}{k!}(x - a)^k \]
Expanding this:
\[ P_n(x) = f(a) + f'(a)(x-a) + \frac{f''(a)}{2!}(x-a)^2 + \frac{f'''(a)}{3!}(x-a)^3 + \cdots + \frac{f^{(n)}(a)}{n!}(x-a)^n \]
By construction, \(P_n^{(k)}(a) = f^{(k)}(a)\) for \(k = 0, 1, \ldots, n\). That is, the Taylor polynomial of degree \(n\) agrees with \(f\) in all derivatives up to order \(n\) at the center point. No other polynomial of degree \(n\) has this matching property.
The notation \(f^{(k)}(a)\) denotes the \(k\)th derivative of \(f\), evaluated at \(a\). By convention, \(f^{(0)}(a) = f(a)\), the function value itself.
Plot \(e^x\) and \(P_1(x)\), \(P_2(x)\), \(P_3(x)\), \(P_4(x)\) on the same axes for \(x \in [-2, 2]\). Show how each higher-degree polynomial hugs the exponential curve over a wider interval.
A Maclaurin polynomial is simply a Taylor polynomial centered at \(a = 0\). The name distinguishes the special case of expansion at the origin from the general case centered at an arbitrary point.
\[ P_n(x) = \sum_{k=0}^{n} \frac{f^{(k)}(0)}{k!} x^k = f(0) + f'(0)x + \frac{f''(0)}{2!}x^2 + \cdots + \frac{f^{(n)}(0)}{n!}x^n \]
The most important Maclaurin polynomials in numerical methods are those for \(e^x\), \(\sin(x)\), \(\cos(x)\), and \(\ln(1+x)\). These appear throughout scientific computing and form the basis of many numerical algorithms.
For \(f(x) = e^x\): All derivatives at 0 equal 1. The Maclaurin polynomial of degree \(n\) is:
\[ P_n(x) = 1 + x + \frac{x^2}{2!} + \frac{x^3}{3!} + \cdots + \frac{x^n}{n!} \]
For \(f(x) = \sin(x)\):
\[ f(0) = 0,\; f'(0) = 1,\; f''(0) = 0,\; f'''(0) = -1,\; f^{(4)}(0) = 0,\; f^{(5)}(0) = 1, \ldots \]
The pattern: zero for even derivatives, alternating \(\pm 1\) for odd derivatives. So:
\[ P_{2n+1}(x) = x - \frac{x^3}{3!} + \frac{x^5}{5!} - \frac{x^7}{7!} + \cdots + (-1)^n \frac{x^{2n+1}}{(2n+1)!} \]
For \(f(x) = \cos(x)\):
\[ f(0) = 1,\; f'(0) = 0,\; f''(0) = -1,\; f'''(0) = 0,\; f^{(4)}(0) = 1, \ldots \]
The pattern: zero for odd derivatives, alternating \(\pm 1\) for even derivatives. So:
\[ P_{2n}(x) = 1 - \frac{x^2}{2!} + \frac{x^4}{4!} - \frac{x^6}{6!} + \cdots + (-1)^n \frac{x^{2n}}{(2n)!} \]
For \(f(x) = \ln(1+x)\):
\[ f(0) = 0,\; f'(0) = 1,\; f''(0) = -1,\; f'''(0) = 2,\; f^{(4)}(0) = -6, \ldots \]
In general, \(f^{(k)}(0) = (-1)^{k+1}(k-1)!\) for \(k \geq 1\). So:
\[ P_n(x) = x - \frac{x^2}{2} + \frac{x^3}{3} - \frac{x^4}{4} + \cdots + (-1)^{n+1}\frac{x^n}{n} \]
This is valid for \(|x| \leq 1\), \(x \neq -1\). (Convergence of the full series requires \(-1 < x \leq 1\).)
Maclaurin polynomials always center at zero. If you need to approximate a function near a different point — for example, \(\ln(x)\) near \(x = 1\) rather than near \(x = 0\) — you need a Taylor polynomial with a non-zero center.
A Taylor polynomial uses a finite number of terms. A Taylor series uses infinitely many terms, summing to the exact function value (within the radius of convergence).
The Taylor series for \(f\) centered at \(a\) is the formal infinite sum:
\[ \sum_{k=0}^{\infty} \frac{f^{(k)}(a)}{k!}(x - a)^k = f(a) + f'(a)(x-a) + \frac{f''(a)}{2!}(x-a)^2 + \cdots \]
For the standard functions, the Taylor series converge to the function exactly on their intervals of convergence.
| Function | Maclaurin Series | Converges For |
|---|---|---|
| \(e^x\) | \(\displaystyle\sum_{k=0}^{\infty} \frac{x^k}{k!}\) | All real \(x\) |
| \(\sin(x)\) | \(\displaystyle\sum_{k=0}^{\infty} (-1)^k \frac{x^{2k+1}}{(2k+1)!}\) | All real \(x\) |
| \(\cos(x)\) | \(\displaystyle\sum_{k=0}^{\infty} (-1)^k \frac{x^{2k}}{(2k)!}\) | All real \(x\) |
| \(\ln(1+x)\) | \(\displaystyle\sum_{k=1}^{\infty} (-1)^{k+1} \frac{x^k}{k}\) | \(-1 < x \leq 1\) |
| \(\frac{1}{1-x}\) | \(\displaystyle\sum_{k=0}^{\infty} x^k\) | \(|x| < 1\) |
Convergence of an infinite series is not guaranteed everywhere. For \(\ln(1 + x)\), the series only converges for \(-1 < x \leq 1\). Outside this interval, the partial sums do not approach the function value, regardless of how many terms are taken.
For numerical work, what matters is not the infinite series but the finite polynomial approximation — and the size of the error left over when we truncate after finitely many terms. That is the subject of the next section.
When we truncate a Taylor series after \(n\) terms, we are using the Taylor polynomial \(P_n(x)\) to approximate \(f(x)\). The difference between the true value and the polynomial approximation is called the remainder or truncation error:
\[ R_n(x) = f(x) - P_n(x) \]
Taylor's theorem gives us an explicit formula for bounding this remainder. This result is among the most important theorems in numerical analysis.
Suppose \(f\) has \(n+1\) continuous derivatives on an interval containing \(a\) and \(x\). Then there exists a number \(c\) between \(a\) and \(x\) such that:
\[ R_n(x) = \frac{f^{(n+1)}(c)}{(n+1)!}(x - a)^{n+1} \]
We do not know \(c\) exactly — it depends on \(f\), \(a\), \(x\), and \(n\) in a way that is generally inaccessible. But if we can bound \(|f^{(n+1)}(c)|\) over the interval of interest, we can bound the error.
If \(|f^{(n+1)}(t)| \leq M\) for all \(t\) between \(a\) and \(x\), then:
\[ |R_n(x)| \leq \frac{M}{(n+1)!} |x - a|^{n+1} \]
This bound has a clear structure. The numerator \(M\) captures how much the function curves at high order. The denominator grows as a factorial, which decreases very rapidly. The factor \(|x - a|^{n+1}\) captures how far we are from the center.
What the bound tells us. As \(n\) increases (more terms), \((n+1)!\) grows faster than any power of \(|x - a|\), so the error bound decreases to zero — as long as the \((n+1)\)th derivative does not grow too rapidly. This explains why adding more terms to the Taylor polynomial improves the approximation for well-behaved functions.
The exponential function \(e^x\) is one of the most computed functions in mathematics and science. Its Maclaurin series converges for all real \(x\):
\[ e^x = 1 + x + \frac{x^2}{2!} + \frac{x^3}{3!} + \frac{x^4}{4!} + \cdots \]
A computer evaluating \(e^x\) typically does not sum this series directly. Instead, it uses a combination of argument reduction (rewriting \(e^x = e^{m} \cdot e^r\) where \(m\) is an integer and \(r\) is a small remainder), polynomial approximation for the small remainder, and stored values for the integer part. But the mathematical foundation is the Maclaurin series, and understanding it clarifies how the computation works.
How many terms are needed? For \(x\) near 0, the terms \(x^k/k!\) decrease rapidly because the factorial in the denominator grows faster than the power in the numerator. For larger \(|x|\), more terms are needed.
| \(n\) | \(\frac{3}{(n+1)!}\) |
|---|---|
| 5 | \(3/720 \approx 0.00417\) |
| 6 | \(3/5040 \approx 0.000595\) |
| 7 | \(3/40320 \approx 0.0000744\) |
At \(n = 7\), the bound is \(0.0000744 < 0.0001\).
Interpret. Eight terms (degree-7 polynomial) are sufficient to estimate \(e\) to within \(0.0001\). The Maclaurin series converges quickly near \(x = 0\), making it practical for computation.
The Maclaurin series for sine and cosine have elegant alternating structures that make them convenient for computation.
Sine approximation:
\[ \sin(x) = x - \frac{x^3}{3!} + \frac{x^5}{5!} - \frac{x^7}{7!} + \cdots \]
Cosine approximation:
\[ \cos(x) = 1 - \frac{x^2}{2!} + \frac{x^4}{4!} - \frac{x^6}{6!} + \cdots \]
Both series converge for all real \(x\). Convergence is fast for small \(|x|\) and slower for large \(|x|\). In practice, argument reduction is used to map any input angle into a small interval (such as \([0, \pi/4]\)) before evaluating the series.
When the terms of a series alternate in sign and decrease in absolute value to zero, a special, simpler error bound applies. For such alternating series, the error from truncating after \(n\) terms is no larger in absolute value than the first omitted term:
\[ |R_n(x)| \leq |(\text{first omitted term})| \]
This is the Alternating Series Estimation Theorem and often gives a sharper bound than the Lagrange form.
The logarithm has no single Maclaurin expansion valid everywhere. The function \(\ln(x)\) is not defined at \(x = 0\), so it has no Taylor series centered at 0. Instead, we expand \(\ln(1 + x)\) near \(x = 0\), valid for \(-1 < x \leq 1\).
\[ \ln(1 + x) = x - \frac{x^2}{2} + \frac{x^3}{3} - \frac{x^4}{4} + \cdots = \sum_{k=1}^{\infty} (-1)^{k+1} \frac{x^k}{k} \]
This series converges slowly for \(x\) near 1. For \(x = 1\), the partial sums converge to \(\ln(2) \approx 0.6931\), but many terms are needed. For \(x\) near 0, convergence is much faster.
Alternative expansion for faster convergence. Writing
\[ \ln(x) = \ln\left(\frac{1 + t}{1 - t}\right) = 2\left(t + \frac{t^3}{3} + \frac{t^5}{5} + \cdots\right), \quad t = \frac{x - 1}{x + 1} \]
this alternative gives rapid convergence for all \(x > 0\) and is preferred in many numerical implementations.
A recurring practical question in numerical work is: how many Taylor polynomial terms are enough? The answer depends on three factors:
1. Write the Lagrange remainder bound: \(|R_n(x)| \leq \frac{M}{(n+1)!}|x - a|^{n+1}\).
2. Bound \(M\) by the maximum of \(|f^{(n+1)}|\) on the interval.
3. Find the smallest \(n\) such that the bound is less than the desired tolerance.
For alternating series, the strategy is simpler: find the first term whose absolute value is less than the desired tolerance.
Taylor polynomials appear throughout science and engineering in forms that students often encounter without recognizing the connection.
Small-angle approximation. In physics, the exact equation of motion for a simple pendulum is
\[ \theta'' + \frac{g}{L}\sin(\theta) = 0 \]
For small angles \(\theta\), the approximation \(\sin(\theta) \approx \theta\) (the first Taylor term) simplifies this to a linear equation with known exact solutions. This is the linearization of the pendulum equation, and it is valid when the second Maclaurin term, \(\theta^3/6\), is negligibly small.
Finite difference formulas revisited. In Chapter 6, the forward difference formula
\[ f'(x) \approx \frac{f(x+h) - f(x)}{h} \]
was derived from the definition of the derivative. The Taylor polynomial perspective gives the error term directly. Expanding \(f(x+h)\) in a Taylor polynomial centered at \(x\):
\[ f(x+h) = f(x) + f'(x)h + \frac{f''(x)}{2}h^2 + \frac{f'''(x)}{6}h^3 + \cdots \]
Solving for \(f'(x)\):
\[ f'(x) = \frac{f(x+h) - f(x)}{h} - \frac{f''(x)}{2}h - \cdots \]
The leading error term is \(-\frac{f''(x)}{2}h\), which is proportional to \(h\). This is why the forward difference formula has first-order accuracy: halving the step size \(h\) halves the error. The Taylor expansion makes this explicit.
The central difference formula
\[ f'(x) \approx \frac{f(x+h) - f(x-h)}{2h} \]
can similarly be derived. Expanding both \(f(x+h)\) and \(f(x-h)\) and subtracting:
\[ f(x+h) - f(x-h) = 2f'(x)h + \frac{2f'''(x)}{6}h^3 + \cdots \]
Dividing by \(2h\):
\[ f'(x) = \frac{f(x+h) - f(x-h)}{2h} - \frac{f'''(x)}{6}h^2 - \cdots \]
The leading error term is now proportional to \(h^2\), so the central difference formula has second-order accuracy: halving \(h\) reduces the error by a factor of four. This is a direct consequence of the Taylor expansion.
Euler's method. In Chapter 12, Euler's method for solving the initial value problem \(y' = f(t, y)\) is
\[ y_{n+1} = y_n + h f(t_n, y_n) \]
This is exactly the first-order Taylor polynomial for the true solution \(y(t_{n+1})\) expanded around \(t_n\):
\[ y(t_{n+1}) = y(t_n) + y'(t_n)h + \frac{y''(t_n)}{2}h^2 + \cdots \]
Euler's method takes only the first correction and drops the \(h^2\) and higher terms. The local error is therefore proportional to \(h^2\), and the global error is proportional to \(h\). More accurate ODE methods (Runge-Kutta) match higher-order Taylor terms without requiring explicit derivatives.
Numerical integration and midpoint rule accuracy. The midpoint rule approximates \(\int_a^b f(x)\,dx\) by \((b - a) f\!\left(\frac{a+b}{2}\right)\). Its error analysis uses the Taylor expansion of \(f\) centered at the midpoint and shows that the leading error term is proportional to \(h^2\), where \(h = b - a\). Taylor expansions are the universal tool for deriving error formulas in numerical integration.
Students learning Taylor polynomials and their applications in numerical methods frequently make a set of recurring errors. Understanding these mistakes in advance helps avoid them.
This chapter has developed Taylor polynomials as the mathematical foundation of local approximation. The central ideas are:
These ideas carry directly into the study of numerical ordinary differential equations in Chapter 12. Euler's method, the Runge-Kutta family, and their error analyses are all rooted in Taylor expansion. The order of a numerical ODE method — the power of \(h\) in its error term — is determined by how many terms of the Taylor expansion the method matches.
Students who understand Taylor polynomials well will find that numerical ODE methods are natural extensions of local polynomial approximation applied to differential equations. Chapter 12 builds on this foundation.