The Central Idea
A numerical result should not be judged only by its digits.
We also need to ask:
- How closely does it match the target or true value?
- How finely was it measured or reported?
- How much deviation is acceptable?
- How much uncertainty is present?
These questions lead to the ideas of error, precision, and tolerance.
What Is Error?
Error measures the difference between an approximate or observed value and a reference value.
If \(x\) is the reference value and \(\hat{x}\) is an approximation, then the signed error is
\[ e=\hat{x}-x. \]The sign tells us whether the approximation is above or below the reference.
Absolute Error
The absolute error ignores direction and measures only the size of the discrepancy:
\[ \boxed{ E_{\text{abs}}=|\hat{x}-x| }. \]Absolute error has the same units as the quantity being measured.
Example: Absolute Error
Suppose the reference length is
\[ 10.00\text{ m} \]and the measured value is
\[ 9.96\text{ m}. \]Then
\[ E_{\text{abs}} = |9.96-10.00| = \boxed{0.04\text{ m}}. \]Relative Error
Absolute error does not account for the scale of the quantity.
The relative error compares the absolute error with the reference magnitude:
\[ \boxed{ E_{\text{rel}} = \frac{|\hat{x}-x|}{|x|} }. \]Relative error is dimensionless.
Example: Relative Error
Using
\[ x=10.00\text{ m} \]and
\[ \hat{x}=9.96\text{ m}, \]the relative error is
\[ E_{\text{rel}} = \frac{0.04}{10.00} = \boxed{0.004}. \]This means the error is four thousandths of the reference value.
Percent Error
Relative error is often expressed as a percentage:
\[ \boxed{ E_{\%} = \frac{|\hat{x}-x|}{|x|}\times100\% }. \]In the previous example,
\[ E_{\%} = 0.004\times100\% = \boxed{0.4\%}. \]Why Relative Error Matters
An absolute error of \(1\) unit can be tiny or enormous depending on scale.
If the reference value is
\[ 1{,}000{,}000, \]then an error of \(1\) is extremely small.
If the reference value is
\[ 0.01, \]then an error of \(1\) is enormous.
Relative error places the discrepancy in context.
Accuracy
Accuracy describes how close a measured or computed value is to the reference or true value.
A highly accurate measurement has small error.
If repeated measurements cluster around the correct value, they are accurate on average.
Precision
Precision describes how finely or consistently a quantity is measured or reported.
A measurement reported as
\[ 12.347\text{ cm} \]is numerically more precise than one reported as
\[ 12\text{ cm}. \]But more digits do not automatically mean greater accuracy.
Accuracy and Precision Are Different
A set of measurements can be precise but inaccurate.
For example, repeated readings
\[ 9.51,\qquad9.50,\qquad9.52 \]are tightly clustered and therefore precise.
But if the correct value is
\[ 10.00, \]the measurements are systematically inaccurate.
Accurate but Not Highly Precise
A group of measurements might vary considerably but average near the true value.
For example,
\[ 9.7,\qquad10.4,\qquad9.9,\qquad10.0 \]are less tightly clustered than the previous set, but they may still be centered near the correct value.
Precision concerns spread; accuracy concerns closeness to the target.
Measurement Resolution
Resolution is the smallest change an instrument or reporting system can distinguish.
A ruler marked every millimeter has finer resolution than one marked only every centimeter.
Resolution limits the precision that can reasonably be claimed.
Reporting More Digits Does Not Create Information
Suppose a scale measures mass only to the nearest
\[ 0.1\text{ kg}. \]Reporting
\[ 72.300000\text{ kg} \]does not create six decimal places of genuine precision.
Numerical formatting should reflect the quality of the underlying information.
Rounding Error
Rounding error occurs when a number is replaced by a nearby value with fewer digits.
For example,
\[ \pi\approx3.14. \]The absolute rounding error is
\[ |\pi-3.14|. \]Rounding is often necessary, but repeated rounding can accumulate error.
Truncation Error
Truncation error occurs when an infinite or more detailed mathematical process is replaced by a finite approximation.
For example,
\[ e^x = 1+x+\frac{x^2}{2!}+\frac{x^3}{3!}+\cdots \]may be approximated by
\[ e^x\approx1+x+\frac{x^2}{2}. \]Omitting the remaining terms creates truncation error.
Measurement Error
Measurement error can come from many sources.
- limited instrument resolution;
- calibration error;
- environmental variation;
- observer effects;
- sampling variation;
- recording mistakes.
The mathematical model should distinguish ordinary uncertainty from actual mistakes whenever possible.
Systematic Error
A systematic error consistently shifts measurements in the same direction.
For example, if a scale always reads
\[ 0.5\text{ kg} \]too high, repeated measurements may be very precise but still biased.
Random Error
Random error produces unpredictable variation from one measurement to another.
Repeated observations may fall above and below the target.
Repetition and averaging can often reduce the effect of random variation, although they do not automatically remove systematic bias.
Uncertainty Intervals
A measurement is often better represented by a central value together with an uncertainty:
\[ x=x_0\pm\Delta x. \]For example,
\[ L=12.4\pm0.1\text{ cm}. \]This communicates more information than simply writing
\[ 12.4\text{ cm}. \]What Is a Tolerance?
A tolerance specifies an acceptable range of variation.
If a target value is \(x_0\) with tolerance \(\tau\), then acceptable values may satisfy
\[ |x-x_0|\le\tau. \]Equivalently,
\[ x_0-\tau\le x\le x_0+\tau. \]Example: Absolute Tolerance
Suppose a component should have length
\[ 50.0\text{ mm} \]with tolerance
\[ \pm0.2\text{ mm}. \]Then acceptable lengths satisfy
\[ 49.8\text{ mm} \le L \le 50.2\text{ mm}. \]Relative Tolerance
A tolerance can also be specified relative to the target value.
If the allowed relative deviation is \(\varepsilon\), then
\[ \frac{|x-x_0|}{|x_0|} \le \varepsilon. \]For a \(2\%\) tolerance,
\[ \varepsilon=0.02. \]Example: Percent Tolerance
Suppose a target is
\[ 200 \]with tolerance
\[ \pm3\%. \]Three percent of \(200\) is
\[ 0.03(200)=6. \]Therefore the acceptable interval is
\[ \boxed{194\le x\le206}. \]Tolerance Is a Decision Rule
Error describes what happened.
Tolerance describes what is acceptable.
A value can have nonzero error and still be acceptable if the error remains within tolerance.
This distinction is fundamental in manufacturing, computation, quality control, and numerical modeling.
Significant Figures
Significant figures communicate the meaningful digits in a reported value.
For example,
\[ 3.47 \]has three significant figures.
A value reported as
\[ 3.4700 \]conventionally communicates greater reported precision.
Leading Zeros Are Not Significant
In
\[ 0.00420, \]the leading zeros locate the decimal point.
The significant digits are
\[ 4,\qquad2,\qquad0. \]Thus the number has three significant figures.
Trailing Zeros Can Carry Information
In a decimal measurement such as
\[ 2.500, \]the trailing zeros communicate reported precision.
This is different from writing simply
\[ 2.5. \]Scientific notation is often useful when the intended number of significant figures might otherwise be ambiguous.
Scientific Notation Clarifies Precision
Compare
\[ 1.2\times10^4 \]with
\[ 1.200\times10^4. \]Both have the same broad magnitude, but the second reports more significant figures.
Rounding to a Tolerance
If a result only needs to be accurate within
\[ 0.01, \]retaining ten decimal places may be unnecessary.
Numerical precision should be aligned with the required tolerance and the uncertainty in the inputs.
Error Bounds
Sometimes we do not know the exact error but can guarantee that it is below a bound.
If
\[ |\hat{x}-x|\le\varepsilon, \]then \(\varepsilon\) is an absolute error bound.
This type of statement is common in numerical analysis.
Relative Error Bounds
A relative error bound has the form
\[ \frac{|\hat{x}-x|}{|x|} \le \varepsilon. \]If
\[ \varepsilon=10^{-3}, \]the relative error is guaranteed to be no more than about one tenth of one percent.
Error and Orders of Magnitude
Order-of-magnitude reasoning helps judge whether an error is meaningful.
If
\[ x\sim10^6 \]and
\[ E_{\text{abs}}\sim10^1, \]then
\[ E_{\text{rel}} \sim 10^{-5}. \]The absolute error may look large in isolation while remaining tiny relative to the scale of the quantity.
Propagation of Error
Errors in input quantities can affect computed outputs.
If
\[ y=f(x), \]then a small input error \(\Delta x\) produces approximately
\[ \Delta y \approx f'(x)\Delta x. \]The derivative measures local sensitivity to input error.
Relative Error Propagation
For a power law
\[ y=Cx^p, \]small relative changes satisfy approximately
\[ \frac{\Delta y}{y} \approx p\frac{\Delta x}{x}. \]Thus the exponent \(p\) amplifies or reduces relative input error.
Example: Squared Quantity
Suppose
\[ A=L^2. \]For small measurement error,
\[ \frac{\Delta A}{A} \approx 2\frac{\Delta L}{L}. \]A \(1\%\) relative error in length produces approximately a \(2\%\) relative error in area.
Example: Cubed Quantity
If
\[ V=L^3, \]then
\[ \frac{\Delta V}{V} \approx 3\frac{\Delta L}{L}. \]A \(1\%\) relative error in length produces approximately a \(3\%\) relative error in volume.
Error in Products
Suppose
\[ Q=AB. \]For small independent relative changes, a first-order approximation gives
\[ \frac{\Delta Q}{Q} \approx \frac{\Delta A}{A} + \frac{\Delta B}{B}. \]This provides a quick conservative way to estimate how relative uncertainty can accumulate in a product.
Error in Quotients
If
\[ Q=\frac{A}{B}, \]then, at first order,
\[ \frac{\Delta Q}{Q} \approx \frac{\Delta A}{A} - \frac{\Delta B}{B}. \]For uncertainty bounds, magnitudes are commonly combined conservatively so that both sources of uncertainty are accounted for.
Conditioning
Some mathematical problems are naturally more sensitive to input error than others.
A problem is well-conditioned when small input changes usually cause small output changes.
A problem is ill-conditioned when small input changes can cause large output changes.
Example: Sensitive Subtraction
Consider
\[ 1000.001-1000.000. \]The result is
\[ 0.001. \]Because two nearly equal large numbers are being subtracted, small input errors can strongly affect the small difference.
This is a common source of numerical sensitivity.
Numerical Precision
Computers store numbers with finite precision.
Many real numbers cannot be represented exactly in a finite binary format.
As a result, calculations may contain small floating-point errors even when the mathematics is exact.
Do Not Test Floating-Point Equality Carelessly
In numerical computation, two values intended to be equal may differ by a tiny amount because of finite representation.
Instead of requiring
\[ x=y, \]a numerical algorithm may test whether
\[ |x-y|\le\tau \]for a carefully chosen tolerance \(\tau\).
Absolute Versus Relative Numerical Tolerance
An absolute tolerance works well when quantities have a stable scale.
A relative tolerance is often more appropriate when values vary greatly in magnitude.
A combined rule may use both:
\[ |x-y| \le \tau_{\text{abs}} + \tau_{\text{rel}}|y|. \]This avoids relying on one scale alone.
Precision in Model Parameters
Parameters estimated from data should not usually be reported with more digits than the estimation process supports.
A fitted parameter reported as
\[ 0.037284615 \]may imply unrealistic certainty if the data only support two or three meaningful digits.
Tolerance in Engineering and Manufacturing
Tolerances define acceptable variation around a design target.
A specification such as
\[ 25.00\pm0.05\text{ mm} \]means the component remains acceptable within a total width of
\[ 0.10\text{ mm}. \]The tolerance is part of the design requirement, not merely a description of measurement uncertainty.
Tolerance in Optimization
Numerical optimization algorithms often stop when further improvements are sufficiently small.
For example, an algorithm might stop when
\[ |f(x_{n+1})-f(x_n)|<10^{-6}. \]The stopping tolerance determines how much additional refinement is considered worthwhile.
Tolerance in Root Finding
If solving
\[ f(x)=0, \]a numerical method may accept an approximate root \(\hat{x}\) when
\[ |f(\hat{x})|<\tau. \]Another method may instead require successive approximations to be close:
\[ |x_{n+1}-x_n|<\tau. \]Different tolerances answer different questions.
Error Versus Residual
The error compares an approximate solution with the true solution.
The residual measures how closely the approximation satisfies the governing equation.
If
\[ f(x)=0, \]then the residual at \(\hat{x}\) is
\[ r=f(\hat{x}). \]A small residual does not always guarantee a small solution error in an ill-conditioned problem.
Model Error
Even perfectly measured inputs and exact arithmetic do not guarantee a perfect model.
A model may omit mechanisms, simplify geometry, assume linearity, or ignore interactions.
The discrepancy caused by imperfect model structure is often called model error.
Data Error and Model Error Are Different
Data error concerns uncertainty or mistakes in observations.
Model error concerns the gap between the model's structure and the system it is intended to represent.
Improving measurement precision cannot fix an inappropriate model form.
Approximation Error
Approximation error occurs when a simpler mathematical expression replaces a more exact one.
For small \(x\),
\[ \sin x\approx x. \]The difference
\[ \sin x-x \]is the approximation error.
Its size depends on the regime in which the approximation is used.
Error Should Be Compared with Purpose
There is no universally acceptable error level.
An error of \(1\%\) may be excellent for one application and unacceptable for another.
Required precision depends on consequences, scale, cost, safety, and the decision being made.
Precision Has a Cost
Greater precision can require:
- better instruments;
- more measurements;
- longer computation;
- more expensive manufacturing;
- more detailed models;
- more careful calibration.
Modeling often involves choosing a precision level that is sufficient rather than maximal.
Common Error: Confusing Precision with Accuracy
Reporting many decimal places does not guarantee closeness to the true value.
A reading of
\[ 8.123456 \]can be highly precise in formatting and still be inaccurate if the true value is
\[ 10. \]Common Error: Using Absolute Error Alone
An absolute error of
\[ 5 \]means very different things for a reference value of \(10\) and a reference value of \(10^6\).
Relative error is often necessary for fair comparison.
Common Error: Treating Tolerance as Uncertainty
A tolerance is an allowable deviation.
An uncertainty describes what is not known exactly.
A component may be measured very precisely but still fall outside its design tolerance.
Common Error: Rounding Too Early
Repeated intermediate rounding can accumulate error.
A better practice is usually to retain additional internal digits and round only when presenting the final result at an appropriate precision.
Common Error: Reporting Unsupported Digits
If inputs are known only to about three significant figures, a final answer with twelve significant figures usually overstates the information available.
The output precision should reflect input quality and model uncertainty.
Common Error: Ignoring Sensitivity
A small input uncertainty can become a large output uncertainty when the model is highly sensitive.
Always ask how errors propagate through the relationship rather than considering each input uncertainty in isolation.
A Reliable Procedure for Error and Tolerance Analysis
- Identify the reference, target, or accepted value.
- Identify the measured, estimated, or computed value.
- Compute signed error when direction matters.
- Compute absolute error for discrepancy size.
- Compute relative or percent error when scale matters.
- Identify measurement resolution and uncertainty.
- Identify the required tolerance or acceptance criterion.
- Check whether the observed error lies within tolerance.
- Estimate how input errors propagate through the model.
- Report only as many digits as the information supports.
- Distinguish measurement, numerical, and model error.
Example: Full Error Workflow
Suppose a target mass is
\[ 250.0\text{ g} \]and a measured mass is
\[ 247.5\text{ g}. \]The signed error is
\[ e=247.5-250.0=-2.5\text{ g}. \]The absolute error is
\[ E_{\text{abs}}=2.5\text{ g}. \]The relative error is
\[ E_{\text{rel}} = \frac{2.5}{250} = 0.01. \]Therefore the percent error is
\[ \boxed{1\%}. \]Example: Full Tolerance Workflow
Suppose a component target is
\[ 100\text{ mm} \]with tolerance
\[ \pm0.5\text{ mm}. \]Acceptable values satisfy
\[ 99.5\le L\le100.5. \]A measured value of
\[ 100.3\text{ mm} \]is within tolerance.
A measured value of
\[ 100.8\text{ mm} \]is outside tolerance.
Example: Error Propagation in Area
Suppose
\[ A=L^2 \]and the measured length has relative uncertainty
\[ \frac{\Delta L}{L}\approx0.5\%. \]Then
\[ \frac{\Delta A}{A} \approx 2(0.5\%) = \boxed{1\%}. \]Squaring the measured quantity approximately doubles its relative uncertainty.
Practice 1: Absolute Error
A reference value is \(40\) and an approximation is \(38.5\). Find the absolute error.
Practice 2: Relative Error
Using the values from Practice 1, find the relative error.
Practice 3: Percent Error
Using the values from Practice 1, find the percent error.
Practice 4: Absolute Tolerance
A target is \(80\) with tolerance \(\pm2\). Write the acceptable interval.
Practice 5: Percent Tolerance
A target is \(500\) with tolerance \(\pm4\%\). Find the acceptable interval.
Practice 6: Significant Figures
How many significant figures are present in
\[ 0.00450? \]Practice 7: Accuracy or Precision?
Repeated measurements are \(12.01\), \(12.02\), and \(12.01\), but the true value is \(13.00\). Are the measurements precise, accurate, both, or neither?
Practice 8: Relative Error Propagation
If
\[ V=L^3 \]and the relative uncertainty in \(L\) is approximately \(2\%\), estimate the relative uncertainty in \(V\).
Practice 9: Within Tolerance?
A specification is
\[ 25.0\pm0.3\text{ mm}. \]Is a measured value of \(25.4\text{ mm}\) acceptable?
Practice 10: Numerical Tolerance
Two computed values differ by
\[ 4\times10^{-7}. \]If the absolute tolerance is
\[ 10^{-6}, \]do they satisfy the tolerance test?
Practice Answers
Practice 1
\[ E_{\text{abs}} = |38.5-40| = \boxed{1.5}. \]Practice 2
\[ E_{\text{rel}} = \frac{1.5}{40} = \boxed{0.0375}. \]Practice 3
\[ E_{\%} = 0.0375\times100\% = \boxed{3.75\%}. \]Practice 4
\[ \boxed{78\le x\le82}. \]Practice 5
Four percent of \(500\) is
\[ 0.04(500)=20. \]Therefore,
\[ \boxed{480\le x\le520}. \]Practice 6
The number
\[ 0.00450 \]has
\[ \boxed{3} \]significant figures.
Practice 7
The measurements are tightly clustered, so they are precise, but they are far from the true value, so they are not accurate.
Practice 8
\[ \frac{\Delta V}{V} \approx 3\frac{\Delta L}{L} = 3(2\%) = \boxed{6\%}. \]Practice 9
The acceptable interval is
\[ 24.7\text{ mm} \le L \le 25.3\text{ mm}. \]Since \(25.4\text{ mm}\) is outside this interval, it is not acceptable.
Practice 10
Since
\[ 4\times10^{-7} < 10^{-6}, \]the difference is within the stated tolerance.
What to Remember
Precision, tolerance, and error describe different aspects of numerical quality.
- error measures discrepancy from a reference value;
- absolute error has the same units as the quantity;
- relative error compares discrepancy with scale;
- percent error expresses relative error as a percentage;
- accuracy concerns closeness to the target;
- precision concerns repeatability or fineness of reporting;
- tolerance specifies acceptable variation;
- measurement, numerical, approximation, and model errors are different;
- errors can propagate and sometimes amplify through a model;
- reported digits should reflect the quality of the information available.
The key modeling habit is: never ask only what the number is—ask how well it is known and how close is close enough.
Where We Go Next
Precision, tolerance, and error describe the quality of numerical information. The next lesson, Local and Global Parameter Effects, examines how changes in model parameters influence outputs near a specific operating point and across the model as a whole.