Confidence and Prediction Intervals | Malone Math

The Central Idea

Suppose a model uses data to estimate an unknown quantity. A point estimate gives one numerical value, such as

\[ \hat{\theta}=42.3. \]

That number alone does not show how precisely the quantity has been estimated. An interval supplements the estimate with information about its uncertainty.

Two major interval questions arise repeatedly in modeling:

Confidence interval: How uncertain is an estimated model quantity?
Prediction interval: How uncertain is a future or unobserved outcome?

These questions are related, but they are not interchangeable. Prediction usually includes the uncertainty of estimation plus the natural variability of the future outcome itself.

Point Estimates and Interval Estimates

A point estimate assigns a single value to an unknown model quantity. If \(\mu\) is an unknown population mean, the sample mean

\[ \bar{x}=\frac{1}{n}\sum_{i=1}^{n}x_i \]

is commonly used as a point estimate of \(\mu\).

An interval estimate gives a lower and upper endpoint:

\[ L\leq \theta \leq U. \]

The width of the interval communicates precision. A narrow interval generally represents greater precision than a wide interval, provided the intervals are constructed for the same quantity under comparable assumptions and confidence levels.

What Is a Confidence Interval?

A confidence interval is a range produced by a statistical procedure designed to capture an unknown fixed quantity at a stated long-run rate when the procedure's assumptions hold.

The unknown quantity might be a population mean, a regression coefficient, a model parameter, a mean model response, a difference between groups, or another estimand.

A typical two-sided confidence interval has the form

\[ \boxed{\text{estimate}\ \pm\ \text{critical value}\times\text{standard error}}. \]

The estimate supplies the center. The standard error measures sampling uncertainty. The critical value determines how far the interval extends from the estimate for the chosen confidence level and reference distribution.

What Does the Confidence Level Mean?

A confidence level such as \(95\%\) describes the long-run performance of the procedure, not the probability assigned to a fixed parameter after a particular interval has already been calculated.

If a valid \(95\%\) confidence-interval procedure were applied repeatedly to new samples generated under the same conditions, approximately \(95\%\) of the resulting intervals would contain the true parameter.

Once one particular frequentist interval has been computed, its endpoints are fixed and the parameter is treated as fixed. Therefore, the technically precise statement is that the method has 95% coverage, not that there is a 95% frequentist probability that this fixed parameter lies inside this already-observed interval.

A probability statement about the parameter itself belongs to a different framework, such as the Bayesian credible intervals introduced later in this section under Bayesian parameter treatment.

Confidence Level and Interval Width

Holding the data and model fixed, greater confidence generally requires a wider interval. A \(99\%\) confidence interval ordinarily extends farther from the estimate than a \(95\%\) interval, which ordinarily extends farther than a \(90\%\) interval.

This reflects a basic tradeoff:

higher confidence gives stronger long-run coverage;
lower confidence gives a narrower interval;
larger samples can often improve precision without lowering the confidence level.

The Role of Standard Error

A standard error describes the sampling variability of an estimator. It is not the same as the standard deviation of individual observations.

For an independent sample with population standard deviation \(\sigma\), the standard deviation of the sample mean is

\[ \sigma_{\bar X}=\frac{\sigma}{\sqrt{n}}. \]

When \(\sigma\) is unknown, it is commonly estimated by the sample standard deviation \(s\), giving

\[ \operatorname{SE}(\bar X)=\frac{s}{\sqrt{n}}. \]

The factor \(\sqrt{n}\) is crucial. Increasing the sample size reduces the standard error, but with diminishing returns: reducing the standard error by half generally requires multiplying the sample size by four under this model.

A Normal-Based Confidence Interval

When the sampling distribution of an estimator is normal or well approximated by a normal distribution, a confidence interval can often be written

\[ \hat{\theta}\pm z_{\alpha/2}\operatorname{SE}(\hat{\theta}), \]

where \(z_{\alpha/2}\) is a standard-normal critical value. For a two-sided \(95\%\) interval,

\[ z_{0.025}\approx1.96. \]

Thus, a common large-sample approximation is

\[ \boxed{\hat{\theta}\pm1.96\operatorname{SE}(\hat{\theta})}. \]

This formula is not universal. The correct interval depends on the estimator, sampling design, model assumptions, and whether important nuisance parameters must themselves be estimated.

The t Interval for an Unknown Mean

When estimating a population mean from a normally distributed population and the population standard deviation is unknown, the classical interval uses a Student \(t\) critical value:

\[ \boxed{ \bar{x}\pm t_{\alpha/2,\,n-1}\frac{s}{\sqrt{n}} }. \]

The \(t\) distribution accounts for the additional uncertainty introduced by estimating \(\sigma\) with \(s\). Its tails are heavier than those of the standard normal distribution, especially for small samples.

As the degrees of freedom increase, the \(t\) distribution approaches the normal distribution.

Example: Confidence Interval for a Mean

Suppose a sample of \(n=25\) measurements has

\[ \bar{x}=80.0, \qquad s=10.0. \]

The standard error of the mean is

\[ \operatorname{SE}(\bar X)=\frac{10}{\sqrt{25}}=2. \]

For a \(95\%\) interval with \(24\) degrees of freedom,

\[ t_{0.025,24}\approx2.064. \]

The margin of error is

\[ 2.064(2)=4.128. \]

Therefore,

\[ \boxed{80.0\pm4.128} \]

or approximately

\[ \boxed{75.87\leq\mu\leq84.13}. \]

This interval concerns the unknown population mean. It does not say that approximately \(95\%\) of individual observations lie between \(75.87\) and \(84.13\).

Margin of Error

In a symmetric interval of the form

\[ \hat{\theta}\pm M, \]

the quantity \(M\) is the margin of error. For a standard confidence interval,

\[ M=(\text{critical value})(\text{standard error}). \]

The margin of error becomes larger when the confidence level rises or the estimator becomes more variable. It becomes smaller when the estimator is more precise, often because more information or a larger effective sample size is available.

Confidence Intervals for Model Parameters

Many models contain unknown parameters. For a linear relationship

\[ Y=\beta_0+\beta_1X+\varepsilon, \]

the fitted model estimates \(\beta_0\) and \(\beta_1\) with coefficients \(\hat\beta_0\) and \(\hat\beta_1\).

A confidence interval for the slope often has the form

\[ \hat\beta_1 \pm t_{\alpha/2,\,\nu}\operatorname{SE}(\hat\beta_1), \]

where \(\nu\) denotes the appropriate residual degrees of freedom.

Such an interval concerns uncertainty about the model parameter. It is not a prediction interval for a future value of \(Y\).

What Is a Prediction Interval?

A prediction interval is an interval constructed for an unobserved or future outcome rather than only for a fixed model parameter.

Suppose a model predicts

\[ \hat{Y}=f(X;\hat{\theta}). \]

The prediction is uncertain because the parameters \(\hat{\theta}\) were estimated. But an individual future outcome may also vary randomly around the model's conditional mean. A prediction interval must account for both effects when both are present.

Why Prediction Intervals Are Usually Wider

Consider the regression model

\[ Y=m(X)+\varepsilon, \]

where \(m(X)\) is the mean response and \(\varepsilon\) represents individual variation around that mean.

A confidence interval for \(m(x_0)\) asks:

Where is the mean response at \(x_0\)?

A prediction interval asks:

Where might a new individual response at \(x_0\) fall?

The second question must include uncertainty about the estimated mean response and the residual variability of an individual observation. Therefore, under the same fitted model and confidence level, the individual prediction interval is ordinarily wider than the confidence interval for the mean response.

Confidence Interval for a Mean Regression Response

In simple linear regression,

\[ Y_i=\beta_0+\beta_1x_i+\varepsilon_i, \]

let the fitted response at \(x_0\) be

\[ \hat y_0=\hat\beta_0+\hat\beta_1x_0. \]

Under the standard independent, constant-variance regression model, the estimated standard error for the fitted mean response is

\[ \operatorname{SE}_{\text{mean}}(x_0) = s\sqrt{ \frac{1}{n} + \frac{(x_0-\bar{x})^2}{\sum_{i=1}^{n}(x_i-\bar{x})^2} }. \]

A corresponding two-sided confidence interval is

\[ \boxed{ \hat y_0 \pm t_{\alpha/2,\,n-2} \operatorname{SE}_{\text{mean}}(x_0) }. \]

Notice that the interval is narrowest near \(\bar{x}\) and becomes wider as \(x_0\) moves farther from the center of the observed predictor values.

Prediction Interval for a New Regression Observation

For a new individual observation at \(x_0\), the standard error becomes

\[ \operatorname{SE}_{\text{pred}}(x_0) = s\sqrt{ 1 + \frac{1}{n} + \frac{(x_0-\bar{x})^2}{\sum_{i=1}^{n}(x_i-\bar{x})^2} }. \]

The added \(1\) inside the square root represents the residual variance of a new observation. The prediction interval is

\[ \boxed{ \hat y_0 \pm t_{\alpha/2,\,n-2} \operatorname{SE}_{\text{pred}}(x_0) }. \]

This mathematical difference captures the conceptual distinction between estimating a mean response and predicting an individual outcome.

Example: Mean Response versus Individual Prediction

Suppose a simple linear regression is based on \(n=25\) observations. At \(x_0=\bar{x}\), the fitted response is

\[ \hat y_0=20, \]

and the residual standard deviation is

\[ s=2. \]

Because \(x_0=\bar{x}\), the distance term is zero. The standard error for the estimated mean response is

\[ \operatorname{SE}_{\text{mean}} =2\sqrt{\frac{1}{25}} =0.4. \]

With \(23\) residual degrees of freedom, a \(95\%\) critical value is approximately

\[ t_{0.025,23}\approx2.069. \]

The confidence interval for the mean response is therefore approximately

\[ 20\pm2.069(0.4) =20\pm0.828, \]

\[ \boxed{19.17\text{ to }20.83}. \]

For a new individual observation,

\[ \operatorname{SE}_{\text{pred}} =2\sqrt{1+\frac{1}{25}} \approx2.040. \]

The prediction interval is approximately

\[ 20\pm2.069(2.040) \approx20\pm4.22, \]

giving

\[ \boxed{15.78\text{ to }24.22}. \]

The prediction interval is much wider because individual outcomes vary around the estimated mean response even if the mean itself is estimated precisely.

Sources of Prediction Uncertainty

Prediction uncertainty can come from several distinct sources. A useful model should identify which of them are included in an interval.

Parameter uncertainty: fitted coefficients or parameters are not known exactly.
Process or residual variability: outcomes vary even when model inputs and parameters are fixed.
Input uncertainty: future predictor values or initial conditions may themselves be uncertain.
Model uncertainty: the assumed functional form or mechanism may be imperfect.

A textbook regression prediction interval may account for the first two while leaving the last two outside the calculation. In a larger modeling system, all four may matter.

Parameter Uncertainty versus Process Variability

These two sources are especially important to distinguish.

Parameter uncertainty can often decrease as more informative data become available. For example, the standard error of an estimated mean may shrink as the sample size grows.

Process variability represents variation intrinsic to the modeled outcomes under the chosen model. More data may help estimate that variability more accurately, but the variability of a new individual outcome does not disappear merely because the parameters are estimated with high precision.

This is one reason confidence intervals can become very narrow while prediction intervals remain materially wide.

Prediction Intervals as Predictive Quantiles

Prediction intervals need not always be symmetric around a point prediction. If a predictive distribution for \(Y_{\text{new}}\) is available, an interval can be constructed from its quantiles.

For example, a central \(90\%\) prediction interval can be written

\[ \boxed{ [q_{0.05},q_{0.95}] }, \]

where \(q_p\) is the \(p\)-quantile of the predictive distribution.

This formulation is especially useful for skewed, bounded, discrete, or simulation-generated predictive distributions, where an interval of the form estimate \(\pm\) margin may be inappropriate.

Intervals Can Be Asymmetric

Symmetric intervals are convenient, but uncertainty itself is not always symmetric. Suppose a model predicts a nonnegative quantity such as waiting time, concentration, loss, or component lifetime. The distribution may have a long right tail and a hard lower boundary near zero.

In such cases, a plausible interval may look like

\[ [12,31] \]

around a point estimate of \(18\), rather than \(18\pm9.5\). The interval endpoints should follow the uncertainty structure implied by the model rather than being forced into symmetry.

Coverage

The coverage of an interval procedure is the proportion of repeated cases in which the interval contains its target quantity under the data-generating assumptions.

A nominal \(95\%\) interval is designed for approximately \(95\%\) coverage under the conditions used to derive it. If those assumptions are badly wrong, the actual coverage may differ from the nominal level.

Interval quality therefore depends on more than its width. A very narrow interval is not useful if it systematically misses the quantity it is supposed to cover.

Calibration of Prediction Intervals

Prediction intervals can be evaluated by checking how often future observed values actually fall inside them.

For a collection of nominal \(90\%\) prediction intervals, a well-calibrated system should have coverage near \(90\%\) over comparable cases, subject to ordinary sampling variation and any conditions on which the intervals are based.

Calibration should also be examined across relevant subgroups, regions of the predictor space, and forecast horizons. An overall coverage rate can conceal systematic undercoverage in one part of the problem and overcoverage in another.

Conditional and Unconditional Coverage

An interval may have good overall, or unconditional, coverage while performing poorly under particular conditions.

Suppose a model produces \(90\%\) prediction intervals. Overall coverage may be close to \(90\%\), yet the model might cover only \(75\%\) of outcomes in a high-demand region and \(98\%\) in a low-demand region.

For decision-making, it is often important to ask not only whether intervals are calibrated in aggregate, but also whether they remain calibrated where the model will actually be used.

Pointwise Intervals and Confidence Bands

A confidence interval at one input value \(x_0\) is a pointwise statement. If a model curve is evaluated at many values of \(x\), drawing a separate \(95\%\) pointwise interval at each value does not automatically create a \(95\%\) band for the entire curve.

A simultaneous confidence band is constructed to control coverage across a collection or continuum of model points. Because it protects a larger target, it is generally wider than the corresponding pointwise intervals.

This distinction matters when a conclusion concerns the whole fitted relationship rather than one selected input value.

Many Intervals Change the Question

If many separate confidence intervals are reported simultaneously, the chance that at least one misses its target can become substantially larger than the miss probability of any single interval.

Procedures for simultaneous inference adjust for this broader target. The correct method depends on the structure of the model and the family of quantities being estimated.

The practical lesson is that 95% per interval is not the same as 95% for an entire collection of intervals.

Prediction Intervals over a Forecast Horizon

In dynamic models, uncertainty can accumulate as predictions extend farther into the future. A one-step-ahead forecast may depend on well-observed recent conditions, while a ten-step-ahead forecast may depend on several earlier forecasts that are themselves uncertain.

Consequently, prediction intervals often widen with forecast horizon. The exact pattern depends on the model. Some systems approach a stable long-run uncertainty, while unstable or strongly accumulating systems can produce rapidly expanding uncertainty.

The width of a forecast interval therefore carries structural information about how uncertainty moves through time.

Extrapolation and Interval Reliability

Statistical formulas can produce intervals outside the range of observed data, but a computed interval is not automatically trustworthy there.

Extrapolation depends heavily on whether the assumed model structure remains valid beyond the observed region. A narrow model-based interval can give a false impression of certainty if structural uncertainty has not been represented.

When extrapolating, model uncertainty may dominate sampling uncertainty. The interval should therefore be interpreted in light of the assumptions needed to extend the model beyond the data.

Intervals Depend on Model Assumptions

Classical interval formulas are conditional on assumptions. Depending on the method, these may include:

independent observations or a correctly modeled dependence structure;
an appropriate sampling design;
a correctly specified mean or response function;
constant or correctly modeled variance;
a suitable reference distribution for residuals or estimators;
accurate treatment of measurement and input uncertainty.

An interval is not a shield against model misspecification. Its stated coverage is meaningful only relative to the conditions under which the procedure was derived or validated.

Residual Diagnostics and Interval Quality

In regression and forecasting, residuals provide evidence about whether the assumed uncertainty model is plausible. Patterns in residuals can indicate omitted structure, changing variance, serial dependence, outliers, or an inappropriate functional form.

For example, if residual spread increases with the fitted value, a constant-width normal prediction interval may be too wide in one region and too narrow in another.

Interval construction should therefore be connected to model checking rather than treated as an automatic final step.

Resampling-Based Intervals

Not every useful confidence interval has a simple closed-form formula. A bootstrap repeatedly resamples from observed data or from a fitted model to approximate the sampling behavior of an estimator.

If the estimator is recalculated across many bootstrap samples, the resulting empirical distribution can be used to estimate standard errors, bias, and interval endpoints.

Bootstrap methods are especially useful when analytic standard-error formulas are difficult to derive, although their validity still depends on whether the resampling scheme faithfully represents the data-generating structure.

Intervals from Model Simulation

A complicated model may produce predictive intervals by simulation rather than by a single formula. One general strategy is:

represent uncertain inputs or parameters with probability distributions;
generate many plausible input and parameter combinations;
run the model for each combination;
collect the resulting outputs;
use empirical quantiles of the simulated outputs as interval endpoints.

This strategy directly connects interval estimation to the next lesson on Monte Carlo methods.

Do Not Substitute a Confidence Interval for a Prediction Interval

A confidence interval for an average outcome can be very narrow even when individual outcomes are highly variable.

Suppose a manufacturing process has an estimated mean diameter of \(10.00\text{ mm}\) with a very narrow confidence interval for the process mean. That does not imply that the next manufactured part will have a diameter close enough to \(10.00\text{ mm}\) to fall inside that confidence interval.

If the operational question concerns the next part, customer, transaction, storm, demand level, or component failure time, a predictive statement is usually the relevant one.

Connection to Probability Distributions

The previous lesson introduced probability distributions as mathematical descriptions of uncertainty. Interval procedures use those distributions in several ways.

A sampling distribution describes how an estimator varies across repeated samples.
A reference distribution provides critical values for confidence intervals.
A predictive distribution describes possible future outcomes.
Quantiles of a distribution can define interval endpoints.

Confidence and prediction intervals are therefore compressed summaries of a larger uncertainty distribution. They are useful because they reduce a full distribution to a range that can be communicated and compared efficiently.

Connection to Error Propagation

The previous lesson showed how input uncertainty can be propagated into an output variance or standard uncertainty. If a propagated output is approximately normal, a rough \(95\%\) interval may sometimes be formed as

\[ \hat y\pm1.96\sigma_y. \]

But this step requires care. The propagated quantity \(\sigma_y\) must correspond to the uncertainty target of interest, and the normal approximation must be reasonable. If the output distribution is skewed, bounded, multimodal, or strongly nonlinear, direct quantiles from the output distribution may be more appropriate.

Intervals in Decision-Making

An interval is often more useful than a point estimate because decisions depend on what could plausibly happen, not only on the central prediction.

Suppose projected demand is \(10{,}000\) units. A planning decision differs greatly depending on whether a relevant prediction interval is

\[ [9{,}800,10{,}200] \]

\[ [6{,}000,14{,}000]. \]

The point prediction is identical, but the operational risk is not. Capacity, reserves, inventory, staffing, insurance, and contingency plans may all depend on the uncertainty around the estimate.

Narrower Is Not Automatically Better

A model can create artificially narrow intervals by ignoring uncertainty. Such intervals may look precise while failing to achieve their intended coverage.

Good uncertainty intervals balance at least two objectives:

calibration: the interval should contain the target at approximately the stated rate;
sharpness: subject to adequate calibration, the interval should be as informative and narrow as the evidence permits.

Precision is valuable only when it is justified.

A Practical Interval Workflow

Define the target. Decide whether the goal is a parameter, mean response, individual outcome, future aggregate, or another quantity.
Identify uncertainty sources. Separate parameter, process, input, measurement, and structural uncertainty where relevant.
Choose an interval framework. Use an analytic formula, transformation, resampling method, predictive distribution, or simulation method appropriate to the model.
Select the coverage level. Choose a level suited to the decision and reporting context.
Check assumptions. Examine model fit, residuals, dependence, variance structure, and extrapolation.
Compute the interval. Keep the target quantity and units explicit.
Validate coverage when possible. Compare nominal and observed performance on new or held-out data.
Communicate the interval correctly. State what the interval refers to and what uncertainty sources it includes.

Common Mistakes

Interpreting a confidence interval for a mean as a range containing most individual observations.
Using a confidence interval when the decision actually concerns a future outcome.
Assuming a 95% frequentist confidence interval gives a 95% posterior probability for a fixed parameter.
Ignoring the distinction between standard deviation and standard error.
Reporting a narrow interval without checking whether the model is calibrated.
Ignoring predictor, measurement, dependence, or structural uncertainty that the formula does not include.
Using symmetric intervals for strongly skewed or bounded uncertainty without justification.
Extrapolating far beyond the observed data while treating model form as certain.
Confusing pointwise intervals with simultaneous confidence bands.
Reporting many separate intervals as though each retained the same family-wide coverage.

Practice

Practice 1

A sample has \(n=64\), sample mean \(\bar{x}=120\), and sample standard deviation \(s=16\). Compute the estimated standard error of the mean.

Practice 2

Using the information from Practice 1 and the large-sample critical value \(1.96\), construct an approximate \(95\%\) confidence interval for the population mean.

Practice 3

A fitted model estimates the average response at a certain input as \(50\) with standard error \(1.5\). Using a critical value of \(2\), construct the corresponding approximate confidence interval for the mean response.

Practice 4

At the same input, suppose the standard error for predicting a new individual response is \(6\). Using the same critical value of \(2\), construct the prediction interval.

Practice 5

Explain why the interval in Practice 4 is wider than the interval in Practice 3 even though both are centered at the same model prediction.

Practice 6

A forecasting system produces nominal \(90\%\) prediction intervals for \(1{,}000\) comparable future cases. Exactly \(842\) observed outcomes fall inside their intervals. Compute the observed coverage rate and describe what this suggests.

Practice 7

A model reports a \(95\%\) confidence interval of \([4.8,5.2]\) for the mean lifetime of a component type. A manager says, “Therefore, 95% of individual components last between 4.8 and 5.2 years.” Explain the error.

Practice 8

A predictive distribution is strongly right-skewed. Explain why an interval based on the 5th and 95th percentiles may be more appropriate than forcing a symmetric interval around the mean.

Practice 9

A model's \(95\%\) prediction intervals achieve \(96\%\) coverage overall, but only \(72\%\) coverage for high-load operating conditions. What important lesson does this reveal about interval validation?

Practice 10

A regression confidence interval becomes wider as \(x_0\) moves far away from the center of the observed predictor values. Give a modeling explanation for this behavior.

Practice Solutions

Practice 1

The estimated standard error is

\[ \operatorname{SE}(\bar X) =\frac{s}{\sqrt{n}} =\frac{16}{\sqrt{64}} =\frac{16}{8} =\boxed{2}. \]

Practice 2

The approximate margin of error is

\[ 1.96(2)=3.92. \]

Therefore,

\[ 120\pm3.92, \]

giving

\[ \boxed{116.08\text{ to }123.92}. \]

Practice 3

The margin is

\[ 2(1.5)=3. \]

Thus, the confidence interval for the mean response is

\[ \boxed{47\text{ to }53}. \]

Practice 4

The prediction margin is

\[ 2(6)=12. \]

Therefore, the prediction interval is

\[ \boxed{38\text{ to }62}. \]

Practice 5

The confidence interval in Practice 3 describes uncertainty about the estimated mean response. The prediction interval in Practice 4 must also include individual outcome variability around that mean. The extra source of uncertainty makes the prediction standard error, and therefore the interval, larger.

Practice 6

The observed coverage rate is

\[ \frac{842}{1000}=0.842=\boxed{84.2\%}. \]

This is materially below the nominal \(90\%\) level. Over these comparable cases, the intervals appear to be under-covering, which may indicate that uncertainty has been underestimated or that the predictive model is misspecified for the evaluation conditions.

Practice 7

The interval \([4.8,5.2]\) concerns uncertainty about the mean lifetime, not the distribution of individual lifetimes. Individual components can vary much more widely. A prediction interval or another distributional statement would be needed to describe individual component lifetimes.

Practice 8

A strongly right-skewed predictive distribution has unequal uncertainty on the two sides of its center. The 5th and 95th percentiles follow the actual distribution and naturally produce an asymmetric interval. A symmetric interval could extend too far into an implausible lower region or fail to represent the long upper tail adequately.

Practice 9

Good overall coverage does not guarantee good conditional coverage. The model is substantially under-covering in high-load conditions, so the uncertainty model may be unreliable precisely where operational risk is elevated. Validation should therefore examine important subgroups and operating regimes, not only one aggregate coverage percentage.

Practice 10

Data provide the strongest information near the region where predictor values are concentrated. Farther from that center, the fitted mean response depends more heavily on the estimated slope and on extrapolation along the fitted line. The uncertainty in the estimated response therefore increases as \(x_0\) moves away from \(\bar{x}\).

What to Remember

A point estimate gives one value; an interval communicates uncertainty around a target quantity.
A confidence interval quantifies uncertainty about an estimated parameter, mean response, or other fixed target.
A frequentist confidence level describes the long-run coverage of the interval procedure.
Standard error measures the sampling variability of an estimator.
Greater confidence generally produces wider intervals when other factors are fixed.
A prediction interval concerns a future or unobserved outcome.
Prediction intervals are usually wider than confidence intervals for mean responses because they include outcome variability as well as estimation uncertainty.
Intervals can be asymmetric and may be formed directly from predictive quantiles.
Nominal coverage is trustworthy only to the extent that the interval method and its assumptions are valid.
Overall calibration can hide poor conditional performance in important regions or subgroups.
Pointwise intervals and simultaneous confidence bands answer different questions.
Extrapolation can introduce structural uncertainty that ordinary formulas do not capture.
Simulation and resampling provide interval methods when analytic formulas are inadequate.

The most important distinction is the target. Ask first whether you are estimating an underlying quantity or predicting a new outcome. Once the target is clear, the relevant sources of uncertainty and the appropriate interval become much easier to identify.

Where We Go Next

Confidence and prediction intervals compress uncertainty into useful ranges. For complex models, however, the full distribution of possible outputs may be difficult to derive analytically.

The next lesson, Monte Carlo Methods, develops a simulation approach: repeatedly sample plausible uncertain inputs, run the model, and use the resulting collection of outputs to estimate distributions, intervals, probabilities, and risk measures.