The Central Idea
A derivative is defined through a limit. A finite-difference method replaces that limiting process with a ratio of differences taken over a small but finite interval.
The basic transition is
\[ \boxed{ \text{derivative} \longrightarrow \text{difference quotient} \longrightarrow \text{algebraic equation} }. \]This makes differential models accessible to numerical computation.
What Is a Finite Difference?
A finite difference is the difference between function values evaluated at distinct points separated by a finite spacing.
For example,
\[ f(x+\Delta x)-f(x) \]is a forward finite difference.
Dividing by \(\Delta x\) produces the familiar difference quotient
\[ \frac{f(x+\Delta x)-f(x)}{\Delta x}, \]which approximates the derivative when \(\Delta x\) is sufficiently small.
The Computational Grid
Finite-difference methods usually begin with a grid
\[ x_i=a+i\Delta x, \qquad i=0,1,\ldots,N. \]The function value at node \(x_i\) is written
\[ f_i=f(x_i). \]Continuous derivative information is then approximated using combinations of nearby grid values such as \(f_{i-1}\), \(f_i\), and \(f_{i+1}\).
Forward Difference for the First Derivative
The forward-difference approximation is
\[ \boxed{ f'(x_i) \approx \frac{f_{i+1}-f_i}{\Delta x} }. \]It uses the current grid point and the point immediately to the right.
This approximation is especially convenient when information ahead of the point is available.
Backward Difference for the First Derivative
The backward-difference approximation is
\[ \boxed{ f'(x_i) \approx \frac{f_i-f_{i-1}}{\Delta x} }. \]It uses the current grid point and the point immediately to the left.
Backward differences are often useful near a right boundary or when past values are naturally available.
Central Difference for the First Derivative
A symmetric approximation uses one point on each side:
\[ \boxed{ f'(x_i) \approx \frac{f_{i+1}-f_{i-1}}{2\Delta x} }. \]This is the central-difference approximation.
It usually provides greater accuracy than the simplest forward or backward formulas for the same grid spacing when all required neighboring values are available.
Example: Forward Difference
Suppose
\[ f(2)=4, \qquad f(2.1)=4.41. \]With
\[ \Delta x=0.1, \]the forward-difference approximation is
\[ f'(2) \approx \frac{4.41-4}{0.1} = \boxed{4.1}. \]For \(f(x)=x^2\), the exact derivative is \(f'(2)=4\).
Example: Central Difference
Let
\[ f(x)=x^2 \]and approximate \(f'(2)\) using
\[ x=1.9,\qquad2.1. \]Then
\[ f(2.1)=4.41, \qquad f(1.9)=3.61. \]Therefore,
\[ f'(2) \approx \frac{4.41-3.61}{0.2} = \boxed{4}. \]In this quadratic example, the central difference recovers the exact derivative.
The Second Derivative
A standard central approximation to the second derivative is
\[ \boxed{ f''(x_i) \approx \frac{f_{i+1}-2f_i+f_{i-1}}{(\Delta x)^2} }. \]This formula appears throughout numerical models involving diffusion, curvature, heat flow, elasticity, and wave propagation.
Why the Second-Difference Formula Makes Sense
The first difference measures change between neighboring values. The second difference measures change in that change.
The combination
\[ f_{i+1}-2f_i+f_{i-1} \]therefore captures local curvature in a discrete form.
Dividing by
\[ (\Delta x)^2 \]gives the appropriate scaling for a second derivative.
Example: Second Derivative
Let
\[ f(x)=x^2 \]on a grid with
\[ \Delta x=1. \]At \(x=2\),
\[ f_1=1,\qquad f_2=4,\qquad f_3=9. \]Then
\[ f''(2) \approx 9-2(4)+1 = \boxed{2}. \]This matches the exact value \(f''(x)=2\).
Difference Operators
Finite-difference formulas can be represented with operators.
A forward-difference operator may be written
\[ \Delta f_i=f_{i+1}-f_i. \]A backward difference may be written
\[ \nabla f_i=f_i-f_{i-1}. \]These operators make repeated difference patterns easier to describe.
What Is a Stencil?
A finite-difference stencil identifies the set of nearby grid points used in a numerical approximation.
For the central first derivative
\[ \frac{f_{i+1}-f_{i-1}}{2\Delta x}, \]the stencil uses
\[ i-1,\qquad i+1. \]For the second derivative, the stencil uses
\[ i-1,\qquad i,\qquad i+1. \]Stencil Width
Some finite-difference formulas use only nearest neighbors. Others use more distant points.
Wider stencils can provide:
- higher-order accuracy;
- better approximation of certain derivatives;
- greater computational coupling between nodes.
The stencil is therefore part of the numerical method, not merely a notation choice.
Taylor-Series Derivation
Finite-difference formulas can be derived using Taylor expansions.
Expand around \(x_i\):
\[ f(x_i+\Delta x) = f_i + \Delta x f_i' + \frac{(\Delta x)^2}{2}f_i'' + \frac{(\Delta x)^3}{6}f_i''' +\cdots. \]Rearranging gives
\[ \frac{f_{i+1}-f_i}{\Delta x} = f_i' + \frac{\Delta x}{2}f_i'' +\cdots. \]The extra terms describe the approximation error.
Truncation Error
Truncation error arises because a finite-difference formula keeps only part of the full Taylor-series relationship.
For the basic forward difference,
\[ f'(x_i) = \frac{f_{i+1}-f_i}{\Delta x} + O(\Delta x). \]The notation \(O(\Delta x)\) indicates that the leading error is proportional to the grid spacing as \(\Delta x\to0\).
Order of Accuracy
A numerical approximation is said to be \(p\)th-order accurate when its leading truncation error behaves like
\[ O((\Delta x)^p). \]The standard forward and backward first differences are first-order accurate:
\[ O(\Delta x). \]The standard central first difference is second-order accurate:
\[ O((\Delta x)^2). \]Why Higher Order Matters
Suppose an error behaves like
\[ C(\Delta x)^p. \]If the grid spacing is halved, then the leading error changes approximately by
\[ \left(\frac{1}{2}\right)^p. \]For a first-order method, the error is roughly halved. For a second-order method, it is roughly divided by four.
Grid Refinement
A numerical calculation should often be repeated on progressively finer grids.
For example,
\[ \Delta x=0.2,\qquad 0.1,\qquad 0.05. \]If the computed solution stabilizes as the grid is refined, the method may be converging with respect to spatial discretization.
Roundoff and Very Small Grid Spacing
Smaller \(\Delta x\) usually reduces truncation error, but extremely small differences can amplify floating-point roundoff.
In
\[ \frac{f(x+\Delta x)-f(x)}{\Delta x}, \]the numerator may subtract two nearly equal numbers.
Numerical accuracy therefore involves a balance between discretization error and finite-precision arithmetic.
Boundary Points
Interior central differences require points on both sides of \(x_i\).
At the left boundary \(x_0\), the point
\[ x_{-1} \]may not exist in the computational domain.
Boundary nodes therefore require special formulas or boundary conditions.
One-Sided Boundary Differences
At a left boundary, a forward difference can be used:
\[ f'(x_0) \approx \frac{f_1-f_0}{\Delta x}. \]At a right boundary,
\[ f'(x_N) \approx \frac{f_N-f_{N-1}}{\Delta x}. \]These formulas use only values inside the computational domain.
Dirichlet Boundary Conditions
A Dirichlet boundary condition specifies the value of the unknown function at the boundary.
For example,
\[ u(0)=100, \qquad u(L)=20. \]On the discrete grid,
\[ \boxed{ u_0=100, \qquad u_N=20 }. \]These values can be inserted directly into the numerical system.
Neumann Boundary Conditions
A Neumann boundary condition specifies a derivative at the boundary.
For example,
\[ u'(0)=q. \]A simple forward approximation gives
\[ \frac{u_1-u_0}{\Delta x} \approx q. \]The continuous derivative condition has become an algebraic relation between grid values.
Finite Differences for an Ordinary Differential Equation
Consider the boundary-value problem
\[ u''(x)=g(x). \]At an interior node,
\[ u''(x_i) \approx \frac{u_{i+1}-2u_i+u_{i-1}}{(\Delta x)^2}. \]Therefore,
\[ \boxed{ u_{i+1}-2u_i+u_{i-1} = (\Delta x)^2g_i }. \]The differential equation has become a linear algebraic equation.
From Many Grid Points to a Linear System
Applying the finite-difference equation at every interior node produces a system of equations.
In matrix form,
\[ A\mathbf{u}=\mathbf{b}. \]Here \(\mathbf{u}\) contains unknown grid values, \(A\) contains coefficients from the difference stencil, and \(\mathbf{b}\) contains source terms and boundary contributions.
A Typical Second-Difference Matrix
For a one-dimensional second derivative, the interior coefficient matrix often has the tridiagonal structure
\[ \begin{bmatrix} -2 & 1 & 0 & \cdots \\ 1 & -2 & 1 & \cdots \\ 0 & 1 & -2 & \ddots \\ \vdots & \vdots & \ddots & \ddots \end{bmatrix}. \]The matrix structure reflects the local stencil: each equation couples one node to its immediate neighbors.
Finite Differences in Time
Time derivatives can be discretized in the same way as spatial derivatives.
A forward approximation is
\[ \frac{\partial u}{\partial t} \approx \frac{u_i^{n+1}-u_i^n}{\Delta t}. \]Here \(i\) indexes space and \(n\) indexes time.
The Heat Equation
Consider the one-dimensional heat equation
\[ \frac{\partial u}{\partial t} = \alpha \frac{\partial^2u}{\partial x^2}. \]A basic finite-difference approximation uses
\[ \frac{u_i^{n+1}-u_i^n}{\Delta t} = \alpha \frac{ u_{i+1}^n-2u_i^n+u_{i-1}^n }{(\Delta x)^2}. \]Explicit Heat-Equation Update
Solving for the next time level gives
\[ u_i^{n+1} = u_i^n + \alpha \frac{\Delta t}{(\Delta x)^2} \left( u_{i+1}^n-2u_i^n+u_{i-1}^n \right). \]Define
\[ r= \alpha \frac{\Delta t}{(\Delta x)^2}. \]Then
\[ \boxed{ u_i^{n+1} = u_i^n + r \left( u_{i+1}^n-2u_i^n+u_{i-1}^n \right) }. \]Space and Time Are Coupled Numerically
The heat-equation update depends on the ratio
\[ \frac{\Delta t}{(\Delta x)^2}. \]This means the time step and spatial grid cannot always be chosen independently.
Refining the spatial grid may require a smaller time step to maintain numerical stability.
Stability Restrictions
For the basic explicit one-dimensional heat-equation method, stability requires the dimensionless parameter
\[ r= \alpha \frac{\Delta t}{(\Delta x)^2} \]to remain sufficiently small.
A standard condition is
\[ \boxed{ r\le\frac{1}{2} }. \]If the time step is too large relative to the spatial spacing, the computed solution can become unstable.
Example: Checking a Stability Ratio
Suppose
\[ \alpha=1, \qquad \Delta x=0.1, \qquad \Delta t=0.004. \]Then
\[ r = \frac{0.004}{0.01} = 0.4. \]Since
\[ 0.4\le0.5, \]the basic one-dimensional stability condition is satisfied.
Two-Dimensional Differences
In two dimensions, let
\[ u_{i,j}=u(x_i,y_j). \]The second derivative in \(x\) is approximated by
\[ \frac{\partial^2u}{\partial x^2} \approx \frac{ u_{i+1,j} -2u_{i,j} +u_{i-1,j} }{(\Delta x)^2}. \]A similar formula applies in the \(y\)-direction.
The Discrete Laplacian
For equal spacing \(h\) in two dimensions,
\[ \nabla^2u = \frac{\partial^2u}{\partial x^2} + \frac{\partial^2u}{\partial y^2} \]can be approximated by
\[ \boxed{ \nabla^2u_{i,j} \approx \frac{ u_{i+1,j} +u_{i-1,j} +u_{i,j+1} +u_{i,j-1} -4u_{i,j} }{h^2} }. \]The Five-Point Stencil
The two-dimensional discrete Laplacian uses the center point and its four nearest axis-aligned neighbors.
This pattern is called the five-point stencil.
The stencil's geometry mirrors the local coupling in the differential operator.
Finite Differences and Conservation
Some differential equations express conservation of mass, energy, momentum, or probability.
A finite-difference method should be checked to determine whether the discrete formulation preserves the relevant conserved quantities adequately.
Numerical conservation properties can strongly affect long simulations.
Explicit and Implicit Finite Differences
An explicit finite-difference method computes new values directly from already known values.
An implicit method includes unknown new-time values in the difference equations.
Implicit methods often require solving a linear or nonlinear system at each time step, but they may allow larger stable time steps for some problems.
Consistency, Stability, and Convergence
Three important numerical concepts are:
- consistency: the discrete equations approach the differential equations as the grid is refined;
- stability: numerical errors remain controlled during computation;
- convergence: the numerical solution approaches the intended continuous solution.
These concepts are related but not identical.
Finite Differences on Nonuniform Grids
The simplest formulas assume constant spacing.
On a nonuniform grid,
\[ \Delta x_i=x_{i+1}-x_i \]may vary with \(i\).
Difference formulas must then be modified to account for unequal spacing. Uniform-grid formulas should not be applied blindly to nonuniform meshes.
Higher-Order Formulas
Wider stencils can be constructed to achieve higher-order accuracy.
For example, a higher-order central approximation to the first derivative can use more than two neighboring points.
Higher-order formulas can reduce truncation error, but they also:
- require more neighboring data;
- complicate boundary treatment;
- increase stencil width;
- may behave differently near discontinuities or sharp gradients.
Sharp Gradients and Discontinuities
Standard finite differences work best when the underlying solution is sufficiently smooth.
Near discontinuities or very sharp changes, simple high-order formulas may produce oscillations or poor local approximations.
Specialized methods are often required for shock-like or strongly nonsmooth behavior.
Verification with Known Functions
Difference formulas should be tested on functions whose derivatives are known.
Examples include:
\[ f(x)=x^2, \] \[ f(x)=\sin x, \] \[ f(x)=e^x. \]Comparing numerical derivatives with exact derivatives provides a direct verification check.
Observed Order of Convergence
Suppose the error at grid spacing \(h\) is \(E_h\), and the error at spacing \(h/2\) is \(E_{h/2}\).
If
\[ E_h \approx C h^p, \]then
\[ \frac{E_h}{E_{h/2}} \approx 2^p. \]This relationship can be used to estimate the observed numerical order.
Common Error: Confusing Nodes and Intervals
If a domain is divided into \(N\) intervals, there are usually
\[ N+1 \]endpoint nodes.
Incorrect indexing can shift stencils, omit boundaries, or access values outside the computational array.
Common Error: Using the Wrong Denominator
A first derivative scales with one power of the spacing:
\[ \frac{1}{\Delta x}. \]A second derivative scales with
\[ \frac{1}{(\Delta x)^2}. \]Mixing these denominators changes the dimensional scaling of the approximation.
Common Error: Ignoring Boundary Conditions
Interior finite-difference equations alone do not determine a boundary-value problem.
Boundary conditions provide essential information and must be incorporated into the discrete system consistently.
Common Error: Assuming Refinement Guarantees Success
A finer grid reduces discretization error only if the method is consistent and implemented correctly.
Grid refinement cannot repair:
- incorrect equations;
- wrong boundary conditions;
- unstable time stepping;
- implementation mistakes;
- poorly posed models.
Common Error: Violating a Stability Restriction
In time-dependent finite-difference models, a step size that is too large can make the solution numerically unstable.
Stability limits should be checked before interpreting simulation results.
Common Error: Applying a Formula Outside Its Assumptions
A difference formula derived for:
- uniform spacing;
- smooth solutions;
- interior nodes;
- a specific differential operator;
should not automatically be used when those assumptions fail.
A Reliable Finite-Difference Workflow
- State the differential model and domain.
- Specify initial and boundary conditions.
- Construct the spatial and, if needed, temporal grid.
- Choose finite-difference approximations for each derivative.
- Derive the discrete algebraic equations.
- Identify the stencil at interior and boundary nodes.
- Assemble the update rule or algebraic system.
- Check dimensions and indexing.
- Verify the method on known functions or benchmark problems.
- Check numerical stability requirements.
- Repeat on finer grids.
- Compare convergence, accuracy, and computational cost.
Practice 1: Forward Difference
Suppose
\[ f(1)=2, \qquad f(1.2)=2.44. \]Use a forward difference to approximate \(f'(1)\).
Practice 2: Backward Difference
Suppose
\[ f(2)=5, \qquad f(1.8)=4.24. \]Use a backward difference to approximate \(f'(2)\).
Practice 3: Central Difference
Suppose
\[ f(1.1)=1.21, \qquad f(0.9)=0.81. \]Use a central difference to approximate \(f'(1)\).
Practice 4: Second Difference
On a grid with
\[ \Delta x=1, \]let
\[ f_{i-1}=4,\qquad f_i=9,\qquad f_{i+1}=16. \]Approximate \(f''(x_i)\).
Practice 5: Refinement Factor
A second-order method has error proportional to
\[ (\Delta x)^2. \]If \(\Delta x\) is halved, by approximately what factor should the leading error decrease?
Practice 6: Stability Ratio
For the explicit heat-equation method, let
\[ \alpha=2,\qquad \Delta t=0.001,\qquad \Delta x=0.1. \]Compute
\[ r= \alpha\frac{\Delta t}{(\Delta x)^2}. \]Practice 7: Boundary Condition
A model specifies
\[ u(0)=7. \]If \(x_0=0\), what discrete condition should be imposed?
Practice 8: Identify the Numerical Issue
A time-dependent finite-difference solution begins oscillating rapidly and growing without bound after the time step is increased.
What numerical issue should be investigated first?
Practice Answers
Practice 1
\[ f'(1) \approx \frac{2.44-2}{0.2} = \frac{0.44}{0.2} = \boxed{2.2}. \]Practice 2
\[ f'(2) \approx \frac{5-4.24}{0.2} = \frac{0.76}{0.2} = \boxed{3.8}. \]Practice 3
\[ f'(1) \approx \frac{1.21-0.81}{0.2} = \frac{0.40}{0.2} = \boxed{2}. \]Practice 4
\[ f''(x_i) \approx 16-2(9)+4 = \boxed{2}. \]Practice 5
Halving \(\Delta x\) changes the leading error by
\[ \left(\frac{1}{2}\right)^2 = \frac{1}{4}. \]So the error should decrease by approximately a factor of
\[ \boxed{4}. \]Practice 6
\[ r = 2\frac{0.001}{0.01} = \boxed{0.2}. \]Practice 7
\[ \boxed{u_0=7}. \]Practice 8
Investigate numerical instability, especially whether the new time step violates the stability condition for the finite-difference scheme.
What to Remember
Finite-difference methods replace derivatives with algebraic relationships among nearby grid values.
- forward, backward, and central differences approximate first derivatives;
- the standard second difference approximates a second derivative;
- stencils describe which neighboring points enter each approximation;
- Taylor expansions explain truncation error and order of accuracy;
- boundary conditions must be translated into discrete equations;
- differential equations become algebraic systems or update rules;
- grid refinement helps assess convergence;
- stability can couple allowable time and space step sizes;
- consistency, stability, and convergence are distinct but related ideas.
The central principle is: replace local continuous change with carefully scaled differences between neighboring computational values.
Where We Go Next
Finite-difference methods work naturally on regular grids and approximate derivatives directly from nearby nodal values. Complex geometries often benefit from a more flexible representation built from connected elements.
The next lesson, Finite-Element Ideas, introduces the core concepts behind dividing a domain into elements, approximating solutions locally, assembling global systems, and handling irregular geometry.