The Central Idea
For predictors \(\mathbf{x}_0\), a model may produce a point prediction \(\hat{y}_0\). A prediction interval surrounds that point with bounds for a new response \(Y_{\mathrm{new}}\):
\[ \boxed{ \text{point prediction} \ \pm\ \text{margin for prediction uncertainty} }. \]The interval must account for both uncertainty in the fitted mean and the unpredictable variation of a new individual outcome.
Point Prediction versus Interval Prediction
| Output | Question answered |
|---|---|
| Point prediction | What single response value does the model predict? |
| Confidence interval for the mean | Where is the average response at these predictor values? |
| Prediction interval | Where might one new individual response fall? |
The Target Is a Future Outcome
Write the regression model at a new predictor value \(x_0\) as
\[ Y_{\mathrm{new}} = \beta_0+\beta_1x_0+\varepsilon_{\mathrm{new}}. \]Even if \(\beta_0\) and \(\beta_1\) were known exactly, the future error \(\varepsilon_{\mathrm{new}}\) would remain unknown. Prediction uncertainty can therefore never be reduced to coefficient uncertainty alone.
Two Sources of Prediction Uncertainty
\[ Y_{\mathrm{new}}-\hat{y}_0 = \underbrace{ \left[ E(Y\mid x_0)-\hat{y}_0 \right] }_{\text{fitted-mean uncertainty}} + \underbrace{ \left[ Y_{\mathrm{new}}-E(Y\mid x_0) \right] }_{\text{new-outcome variation}}. \]The first component reflects estimation from a finite sample. The second reflects residual variation among individual outcomes at the same predictor value.
Prediction Intervals Are Wider
A confidence interval for the mean response accounts only for uncertainty in the fitted mean. A prediction interval also includes the variation of a new outcome. At the same confidence level and predictor values,
\[ \boxed{ \text{prediction interval} \text{ is wider than } \text{confidence interval for the mean} }. \]Nested Uncertainty Bands
The Simple Linear Regression Model
Begin with
\[ Y_i=\beta_0+\beta_1x_i+\varepsilon_i, \]and the fitted line
\[ \hat{y}_i=b_0+b_1x_i. \]For a new case with predictor \(x_0\), the point prediction is
\[ \hat{y}_0=b_0+b_1x_0. \]Residual Standard Error
The residual standard error estimates the standard deviation \(\sigma\) of outcomes around the regression line:
\[ \boxed{ s = \sqrt{ \frac{\operatorname{SSE}}{n-2} } }. \]The two fitted coefficients, intercept and slope, leave \(n-2\) residual degrees of freedom.
Predictor Spread
Define the corrected predictor sum of squares
\[ \boxed{ S_{xx} = \sum_{i=1}^{n}(x_i-\bar{x})^2 }. \]A larger \(S_{xx}\) means the observed predictor values provide more information about the slope, all else equal.
Mean-Response Standard Error
The estimated standard error of the fitted mean at \(x_0\) is
\[ \boxed{ \operatorname{SE}_{\mathrm{mean}}(x_0) = s \sqrt{ \frac{1}{n} + \frac{(x_0-\bar{x})^2}{S_{xx}} } }. \]It is smallest at \(x_0=\bar{x}\) and grows as \(x_0\) moves away from the center of the observed predictors.
Prediction Standard Error
The standard error for an individual future response is
\[ \boxed{ \operatorname{SE}_{\mathrm{pred}}(x_0) = s \sqrt{ 1+ \frac{1}{n} + \frac{(x_0-\bar{x})^2}{S_{xx}} } }. \]The leading \(1\) represents the residual variation of the new outcome.
Why the Extra \(1\) Appears
Under the standard independent-error model,
\[ \operatorname{Var} \left( Y_{\mathrm{new}}-\hat{y}_0 \right) = \sigma^2 \sigma^2 \left[ \frac{1}{n} + \frac{(x_0-\bar{x})^2}{S_{xx}} \right]. \]Factoring out \(\sigma^2\) produces the \(1+\cdots\) expression. The future residual is independent of the fitted-mean error under the model.
The \(t\) Critical Value
Because \(\sigma\) is estimated by \(s\), a two-sided interval uses a Student \(t\) critical value. For confidence level \(1-\alpha\),
\[ t^\ast=t_{1-\alpha/2,\ n-2}. \]Smaller residual degrees of freedom produce a larger critical value and a wider interval.
Confidence Interval for the Mean Response
\[ \boxed{ \hat{y}_0 \pm t_{1-\alpha/2,\ n-2} s \sqrt{ \frac{1}{n} + \frac{(x_0-\bar{x})^2}{S_{xx}} } }. \]The target is the conditional mean \(E(Y\mid x_0)\), not an individual future outcome.
Prediction Interval for a New Response
\[ \boxed{ \hat{y}_0 \pm t_{1-\alpha/2,\ n-2} s \sqrt{ 1+ \frac{1}{n} + \frac{(x_0-\bar{x})^2}{S_{xx}} } }. \]This interval targets one new response generated at \(x_0\), assuming the fitted model and its distributional conditions are appropriate.
What Determines Interval Width?
| Factor | Effect on width |
|---|---|
| Higher confidence level | Wider through a larger critical value |
| Larger residual standard error | Wider through greater unexplained variation |
| Larger sample size | Usually narrower fitted-mean component |
| More predictor spread | Usually narrower slope-related component |
| Greater distance from predictor center | Wider through greater leverage |
More Data Cannot Remove Individual Variation
As \(n\) grows, the fitted-mean uncertainty may shrink toward zero. However, the leading \(1\) in the prediction variance remains:
\[ \operatorname{SE}_{\mathrm{pred}}(x_0) \longrightarrow \sigma \quad \text{under ideal conditions}. \]More data can estimate the regression relationship more precisely, but it cannot eliminate genuine individual-to-individual variation.
A Worked Example
Suppose a simple linear regression has
\[ \hat{y}=4+1.5x, \]with
\[ n=10, \qquad \bar{x}=5, \qquad S_{xx}=82.5, \qquad s=2. \]Construct a \(95\%\) prediction interval at \(x_0=6\). For \(8\) residual degrees of freedom, use \(t^\ast=2.306\).
Step 1: Calculate the Point Prediction
\[ \hat{y}_0 = 4+1.5(6) = 13. \]The fitted response for a new case with \(x_0=6\) is \(13\).
Step 2: Calculate Mean Leverage
\[ h_0 = \frac{1}{n} + \frac{(x_0-\bar{x})^2}{S_{xx}} = \frac{1}{10} + \frac{(6-5)^2}{82.5} \approx 0.11212. \]The quantity \(h_0\) measures uncertainty in the fitted mean at the target predictor value.
Step 3: Mean-Response Standard Error
\[ \operatorname{SE}_{\mathrm{mean}} = 2\sqrt{0.11212} \approx 0.670. \]This standard error applies to the fitted conditional mean.
Step 4: Confidence Interval for the Mean
\[ 13 \pm 2.306(0.670) \approx 13\pm 1.544. \] \[ \boxed{ (11.46,\ 14.54) }. \]This is the \(95\%\) confidence interval for the average response at \(x_0=6\).
Step 5: Prediction Standard Error
\[ \operatorname{SE}_{\mathrm{pred}} = 2\sqrt{1+0.11212} \approx 2.109. \]The new-outcome component makes this value much larger than the mean standard error.
Step 6: Prediction Interval
\[ 13 \pm 2.306(2.109) \approx 13\pm 4.864. \] \[ \boxed{ (8.14,\ 17.86) }. \]The \(95\%\) prediction interval for one new response at \(x_0=6\) is approximately \(8.14\) to \(17.86\).
Compare the Two Worked Intervals
| Target | Standard error | Interval |
|---|---|---|
| Mean response at \(x_0=6\) | \(0.670\) | \((11.46,14.54)\) |
| One new response at \(x_0=6\) | \(2.109\) | \((8.14,17.86)\) |
Both intervals share the point prediction \(13\), but they answer different questions and have materially different widths.
Interpretation of a \(95\%\) Prediction Interval
Under repeated use of the same valid procedure in comparable settings, approximately \(95\%\) of the resulting intervals would contain their corresponding future individual outcomes.
The coverage statement depends on the model, data-generating conditions, predictor value, and procedure used to construct the interval.
What the \(95\%\) Does Not Mean
A frequentist interval does not say that \(95\%\) of all observed responses lie inside this particular interval. It does not guarantee that the next response will be captured, and it does not mean the model itself has a \(95\%\) chance of being correct.
Conditional versus Marginal Coverage
Classical regression formulas seek coverage at a specified predictor value under the assumed model:
\[ P \left( L(\mathbf{x}_0) \le Y_{\mathrm{new}} \le U(\mathbf{x}_0) \mid \mathbf{x}_0 \right) = 1-\alpha. \]Some distribution-free methods guarantee coverage averaged over the predictor distribution instead. Always identify which coverage target a method supports.
Assumptions behind the Classical Formula
Exact small-sample \(t\)-based prediction intervals rely on
- a correctly specified conditional mean;
- independent errors and an independent future error;
- constant conditional error variance;
- normally distributed errors;
- predictors treated as fixed or conditioned upon;
- a new case generated by the same process as the training cases.
Multiple Regression
Let the design matrix \(X\) contain all model columns, including the intercept, and let \(\mathbf{x}_0\) contain the corresponding values for the new case. Then
\[ \hat{y}_0 = \mathbf{x}_0^{\mathsf T}\hat{\boldsymbol{\beta}}. \]If there are \(q\) estimated coefficients in total, the residual degrees of freedom are \(n-q\).
Leverage for a New Predictor Vector
\[ \boxed{ h_0 = \mathbf{x}_0^{\mathsf T} (X^{\mathsf T}X)^{-1} \mathbf{x}_0 }. \]A larger \(h_0\) means the new predictor combination is estimated less precisely from the observed design. It may be unusual even when each individual predictor appears ordinary.
Multiple-Regression Interval Formulas
The confidence interval for the conditional mean is
\[ \hat{y}_0 \pm t_{1-\alpha/2,\ n-q} s\sqrt{h_0}, \]while the prediction interval for a new individual response is
\[ \boxed{ \hat{y}_0 \pm t_{1-\alpha/2,\ n-q} s\sqrt{1+h_0} }. \]Categorical Predictors
A new case must be encoded with the same indicator-variable scheme used to fit the model. The vector \(\mathbf{x}_0\) includes the appropriate zeros, ones, continuous predictor values, and interaction terms.
A genuinely unseen category cannot be inserted into the model without a predefined handling rule or a revised model.
Nonlinear Mean Relationships
Polynomial and spline terms can be included in \(X\). For example, a quadratic model uses
\[ \mathbf{x}_0^{\mathsf T} = \begin{bmatrix} 1 & x_0 & x_0^2 \end{bmatrix}. \]The interval remains valid only if the nonlinear mean structure and error assumptions adequately describe the relevant predictor region.
Interactions in a New Case
If a fitted model includes \(x_1x_2\), the new-case vector must contain the product \(x_{01}x_{02}\). Prediction software usually constructs these terms from a stored model specification.
Manually omitting or inconsistently encoding derived features produces an incorrect point prediction and interval.
Prediction on a Transformed Scale
Suppose the fitted model is
\[ \log Y = \beta_0+\beta_1x+\varepsilon. \]The ordinary interval formula first gives bounds \((L_{\log},U_{\log})\) for a future \(\log Y\). Exponentiating produces
\[ \boxed{ (e^{L_{\log}},e^{U_{\log}}) }. \]The resulting interval is asymmetric on the original response scale.
Back-Transformation Requires Care
Exponentiating a fitted mean on the log scale gives a conditional median on the original scale under a lognormal model, not its arithmetic mean. Retransformation bias matters for point predictions of means.
Prediction limits should be transformed as limits for response quantiles, with the transformation monotonic and the model assumptions stated.
One-Sided Prediction Bounds
Some decisions need only an upper or lower limit. A \(100(1-\alpha)\%\) upper prediction bound in simple regression is
\[ \boxed{ \hat{y}_0 + t_{1-\alpha,\ n-2} \operatorname{SE}_{\mathrm{pred}}(x_0) }. \]A one-sided \(95\%\) bound uses \(t_{0.95}\), not the two-sided \(t_{0.975}\) value.
Predicting the Average of \(m\) New Outcomes
If \(m\) independent future outcomes will be averaged at the same predictors, their new-error variance is reduced by \(m\). The standard error becomes
\[ \boxed{ s\sqrt{h_0+\frac{1}{m}} }. \]This target lies between the conditional mean, which uses \(s\sqrt{h_0}\), and one individual outcome, which uses \(s\sqrt{h_0+1}\).
Several Future Outcomes Require a New Target
A \(95\%\) interval for each future observation does not provide a \(95\%\) probability that all future observations will fall inside their respective intervals. Simultaneous or familywise prediction requires an adjusted procedure.
Specify whether the goal concerns each outcome individually, the average coverage rate, or all outcomes together.
Prediction Intervals and Tolerance Intervals
| Interval | Target |
|---|---|
| Prediction interval | One or more future observations |
| Confidence interval | An unknown parameter or mean response |
| Tolerance interval | A stated proportion of the population with stated confidence |
These intervals may look similar but solve different inferential problems.
Prediction Intervals and Coefficient Intervals
A confidence interval for \(\beta_1\) describes uncertainty in a slope. It does not directly provide limits for a future response. Prediction combines all coefficient uncertainty relevant at \(\mathbf{x}_0\) with future outcome variation.
Prediction Intervals and Forecast Intervals
A forecast interval is a prediction interval for a future time-indexed outcome. Time-series forecasts often include serial dependence, trend, seasonality, changing variance, and uncertainty that grows with the forecast horizon.
An ordinary independent-error regression interval is not automatically a valid time-series forecast interval.
Categorical Outcomes Use Prediction Sets
For classification, uncertainty may be represented by class probabilities or a set of plausible classes rather than a numerical interval. A prediction set may contain one class when evidence is strong and several classes when the case is ambiguous.
The coverage objective remains meaningful, but the output geometry is a set of labels rather than a segment of the number line.
Residual Diagnostics Come First
Before trusting a classical prediction interval, inspect
- residuals versus fitted values;
- residuals versus important predictors and time order;
- normal quantile plots when small-sample normality matters;
- high-leverage and influential observations;
- the predictor location of the new case.
A precise formula cannot compensate for a structurally wrong model.
Heteroscedasticity
If residual spread changes with predictors, one common \(s\) cannot represent future variation everywhere. Classical intervals may be too wide in low-variance regions and too narrow in high-variance regions.
Responses include variance modeling, weighted regression, transformation, distributional regression, quantile methods, and appropriately designed resampling.
Robust Standard Errors Are Not Enough
Heteroscedasticity-robust coefficient standard errors improve uncertainty estimates for fitted parameters. They do not by themselves estimate the conditional spread of a new individual outcome.
Prediction requires a model or valid procedure for future residual variation at \(\mathbf{x}_0\).
Nonnormal Residuals
Strong skewness or heavy tails can make symmetric \(t\)-based prediction intervals inaccurate, especially in small samples. Large samples improve estimation of the mean structure but do not make the future error itself normal.
Transformation, distributional modeling, bootstrap prediction, or quantile methods may better represent asymmetric or heavy-tailed outcomes.
Outliers and Influential Observations
An outlier can inflate \(s\), widening all classical prediction intervals. An influential observation can also change the fitted center and leverage structure.
Investigate data quality and the generating process. Do not remove an observation solely to obtain narrower intervals.
Dependent Errors
Repeated measures, clustered observations, spatial data, and time series can have correlated residuals. The independent-error formula then misrepresents both fitted-mean and future-outcome uncertainty.
Use a model that reflects the dependence and clearly define whether the new outcome belongs to an existing or new cluster.
Extrapolation
When \(\mathbf{x}_0\) lies outside the observed predictor region, leverage increases and intervals often widen. However, the formula accounts only for uncertainty within the assumed model form.
\[ \boxed{ \text{a wide extrapolation interval can still be unjustifiably confident} }. \]Structural change beyond the data range is not measured by the ordinary regression formula.
Dataset Shift
Even an in-range new case may come from a changed process. New instruments, policies, populations, economic conditions, or measurement practices can alter the conditional mean or residual distribution.
Prediction intervals require exchangeability or a correctly modeled transition from training conditions to deployment conditions.
Parametric Simulation
When an analytic formula is inconvenient, a fitted probabilistic model can generate simulated future outcomes:
- represent uncertainty in the fitted parameters;
- draw a future residual from the modeled error distribution;
- form a simulated \(Y_{\mathrm{new}}\);
- repeat and use appropriate empirical quantiles.
Omitting step 2 produces uncertainty for the fitted mean, not a full prediction interval.
Bootstrap Prediction Intervals
Bootstrap methods repeatedly resample or regenerate training data, refit the model, and produce future predictions. A valid prediction bootstrap must reproduce both model-estimation uncertainty and new-observation variation.
The resampling design must respect clustering, time order, heteroscedasticity, and other relevant data structure.
Quantile Regression
Quantile regression directly models conditional quantiles. Fitting lower and upper quantiles, such as
\[ Q_{0.025}(Y\mid \mathbf{x}) \quad\text{and}\quad Q_{0.975}(Y\mid \mathbf{x}), \]can produce asymmetric conditional prediction bounds without assuming constant normal errors. Finite-sample coverage still requires careful estimation and validation.
Conformal Prediction
Conformal methods calibrate prediction sets using nonconformity scores, often absolute residuals. Under exchangeability, split conformal prediction can provide finite-sample marginal coverage with few distributional assumptions.
Marginal coverage averaged over future cases does not guarantee exact coverage at every predictor value or within every subgroup.
A Basic Split-Conformal Pattern
- Fit a prediction model on a training subset.
- Calculate absolute residual scores on a separate calibration subset.
- Select an adjusted upper empirical quantile \(q\) of those scores.
- Report \([\hat{y}(\mathbf{x}_0)-q,\hat{y}(\mathbf{x}_0)+q]\).
Variants allow asymmetric or locally adaptive intervals. The calibration data must not have been used to fit or tune the prediction model.
Bayesian Posterior Predictive Intervals
Bayesian analysis combines a prior distribution with observed data to form a posterior distribution for model parameters. The posterior predictive distribution then integrates over parameter uncertainty and future outcome variation:
\[ p(y_{\mathrm{new}}\mid \text{data}) = \int p(y_{\mathrm{new}}\mid\boldsymbol{\theta}) p(\boldsymbol{\theta}\mid\text{data}) \,d\boldsymbol{\theta}. \]A central or highest-density credible interval can be taken from this predictive distribution.
Empirical Coverage
On an untouched evaluation set with intervals \([L_i,U_i]\), empirical coverage is
\[ \boxed{ \widehat{\operatorname{coverage}} = \frac{1}{m} \sum_{i=1}^{m} \mathbf{1} \{L_i\le y_i\le U_i\} }. \]Compare this value with the nominal level and quantify its sampling uncertainty, especially when the evaluation set is small.
Coverage by Predictor Region
Overall \(95\%\) coverage can hide \(99\%\) coverage in one region and \(80\%\) in another. Examine coverage across scientifically or operationally relevant ranges, groups, locations, and time periods.
Subgroup estimates may be noisy, so report denominators and uncertainty rather than treating every observed difference as systematic.
Sharpness
Among procedures with valid coverage, narrower intervals are generally more informative. The average width is
\[ \frac{1}{m} \sum_{i=1}^{m} (U_i-L_i). \]Narrowness without coverage is overconfidence. Coverage without reasonable narrowness may be operationally unhelpful.
Interval Score
For a central \(100(1-\alpha)\%\) interval \([L,U]\), one proper interval score is
\[ S_\alpha(L,U;y) = (U-L) + \frac{2}{\alpha}(L-y)\mathbf{1}\{y<L\} + \frac{2}{\alpha}(y-U)\mathbf{1}\{y>U\}. \]The score rewards narrow intervals but penalizes missed outcomes according to how far they fall beyond the bounds. Smaller average scores are better.
Use Truly Unseen Outcomes
Interval coverage and width should be evaluated on outcomes not used to fit, select, tune, or calibrate the final procedure. When data are reused, empirical coverage can become optimistically biased.
Cross-validation or nested resampling can help when data are limited, but every preprocessing operation must remain inside its proper training fold.
Operational Interpretation
A prediction interval is useful only if its target matches the decision. Before deployment, specify
- the future unit being predicted;
- the time horizon and predictor information available;
- the nominal coverage level;
- whether one- or two-sided limits are required;
- the costs of intervals that are too narrow or too wide.
A Practical Workflow
- Define the future outcome, predictor values, and prediction horizon.
- Choose the required coverage target: individual, average, or simultaneous.
- Fit an appropriate mean and variance structure.
- Inspect residuals, leverage, dependence, and extrapolation.
- Select an analytic, simulation, quantile, conformal, or Bayesian procedure.
- Calculate point predictions and interval bounds.
- Evaluate coverage and width on appropriately unseen data.
- Check performance by relevant predictor regions and groups.
- Document assumptions, limitations, and update triggers.
- Monitor coverage after deployment as outcomes arrive.
Reporting Checklist
- What future random quantity does the interval target?
- What predictor values and time horizon define the prediction?
- What nominal coverage level and sidedness are used?
- Which assumptions justify the interval procedure?
- Does the new case lie within the observed predictor region?
- How were coverage and width validated?
- How do results vary across relevant groups or conditions?
- What model or data changes would invalidate the interval?
Common Error: Using a Mean Confidence Interval
A confidence interval for \(E(Y\mid x_0)\) is too narrow for an individual future outcome because it omits the leading \(1\) in the prediction variance.
Common Error: Ignoring the Predictor Location
Reusing one margin of error for every \(x_0\) ignores leverage. Even with constant residual variance, fitted-mean uncertainty grows away from the predictor center.
Common Error: Assuming Symmetry
Symmetric intervals are natural for a normal-error model on its fitted scale. They can be inappropriate for positive, bounded, skewed, or count-valued outcomes. Match the interval to the response distribution and scale.
Common Error: Treating Coverage as Guaranteed
A nominal \(95\%\) interval can miss. The percentage describes long-run performance under specified conditions, not certainty for each case.
Common Error: Ignoring Model Selection
Selecting a model after inspecting the same data can add uncertainty not represented by a formula that treats the selected model as fixed. Validation and resampling should reproduce the complete selection process.
Common Error: Confusing In-Range Width with Safety
A narrow interval may result from a large sample and small estimated residual variance, yet still fail if the deployment process has shifted. Statistical precision does not protect against structural mismatch.
Practice 1: Identify the Correct Interval
A company wants a range for the average monthly spending of all customers with a particular profile. Should it use a mean-response confidence interval or an individual prediction interval?
Practice 2: Calculate Residual Standard Error
A simple linear regression has \(n=22\) and \(\operatorname{SSE}=80\). Find \(s\).
Practice 3: Calculate Mean Leverage
Given \(n=20\), \(\bar{x}=10\), \(S_{xx}=200\), and \(x_0=14\), calculate
\[ h_0 = \frac{1}{n} + \frac{(x_0-\bar{x})^2}{S_{xx}}. \]Practice 4: Compare Standard Errors
Using \(s=3\) and \(h_0=0.13\), calculate the standard error for the fitted mean and for one future response.
Practice 5: Construct a Prediction Interval
A model gives \(\hat{y}_0=50\), \(\operatorname{SE}_{\mathrm{pred}}=4\), and \(t^\ast=2.1\). Construct the two-sided prediction interval.
Practice 6: Interpret Coverage
Explain the repeated-sampling meaning of a \(90\%\) prediction interval.
Practice 7: Predict an Average
In multiple regression, \(s=5\), \(h_0=0.04\), and \(m=4\). Find the standard error for predicting the average of four independent future outcomes at the same predictors.
Practice 8: Transform the Bounds
A prediction interval for \(\log Y\) is \((2,3)\). Find the corresponding interval for \(Y\).
Practice 9: Diagnose a Funnel
Residual spread increases strongly with the fitted value. Why is one constant-\(s\) prediction band questionable?
Practice 10: Evaluate Extrapolation
A narrow-looking formula-based interval is reported far outside the observed predictor range. Why may the interval still be unreliable?
Practice Answers
Practice 1
Use a confidence interval for the mean response. The target is the average for the profile, not one individual customer's spending.
Practice 2
\[ s = \sqrt{\frac{80}{22-2}} = \sqrt{4} = 2. \]Practice 3
\[ h_0 = \frac{1}{20} + \frac{(14-10)^2}{200} = 0.05+0.08 = 0.13. \]Practice 4
\[ \operatorname{SE}_{\mathrm{mean}} = 3\sqrt{0.13} \approx 1.082, \] \[ \operatorname{SE}_{\mathrm{pred}} = 3\sqrt{1.13} \approx 3.189. \]Practice 5
\[ 50\pm 2.1(4) = 50\pm 8.4, \] \[ \boxed{ (41.6,\ 58.4) }. \]Practice 6
If the valid procedure were repeatedly applied to comparable training data and corresponding future outcomes, about \(90\%\) of the resulting intervals would contain their future outcomes.
Practice 7
\[ 5\sqrt{0.04+\frac{1}{4}} = 5\sqrt{0.29} \approx 2.693. \]Practice 8
\[ (e^2,e^3) \approx (7.39,\ 20.09). \]Practice 9
The constant-variance assumption is violated. One residual standard error may create overly wide limits in low-variance regions and overly narrow limits in high-variance regions.
Practice 10
The formula assumes the fitted relationship continues beyond the data. It does not include uncertainty about a structural change in the mean or variance outside the observed range.
What to Remember
- A prediction interval targets a future individual response.
- It combines fitted-mean uncertainty with new-outcome variation.
- It is wider than a mean-response confidence interval at the same settings.
- The leading \(1\) in the variance formula represents future residual variation.
- Intervals widen as the target predictors move away from the observed center.
- Classical formulas depend on mean, variance, independence, and distribution assumptions.
- Transformations can produce asymmetric intervals on the original scale.
- Quantile, conformal, bootstrap, and Bayesian methods offer alternative frameworks.
- Coverage and width must be evaluated on genuinely unseen outcomes.
- Extrapolation and dataset shift can invalidate apparently precise intervals.
Where We Go Next
Prediction intervals quantify uncertainty around future outcomes, but their validity still depends on selecting a model that captures stable structure without fitting accidental noise.
The next lesson, Overfitting and Underfitting, examines what happens when a model is too flexible or too rigid for the underlying relationship.