The Central Idea
A straight-line model assumes a constant fitted rate of change:
\[ \hat{y}=b_0+b_1x, \qquad \frac{d\hat{y}}{dx}=b_1. \]A nonlinear relationship allows that rate to depend on \(x\):
\[ \boxed{ \frac{d\hat{y}}{dx} \text{ changes across the predictor range} }. \]The objective is not to make every curve complicated. It is to use enough structure to represent the stable pattern without fitting accidental noise.
Begin with the Scatterplot
A scatterplot can reveal whether the response
- rises and then levels off;
- falls at a changing rate;
- has a U-shape or inverted U-shape;
- changes sharply near a threshold;
- oscillates or follows a seasonal cycle;
- contains separate group-specific curves.
The graph should be examined before selecting a nonlinear formula.
Residual Curvature Is a Warning
A systematic curve in residuals from a linear model indicates that the straight line underpredicts in some regions and overpredicts in others.
More observations will make a stable curvature pattern clearer; they will not make the wrong straight-line mean structure correct.
Zero Correlation Can Hide a Strong Curve
Pearson correlation measures linear association. For the symmetric relationship
\[ y=x^2 \]observed equally on both sides of zero, downward movement on the left can cancel upward movement on the right. The correlation may be zero even though \(y\) is determined exactly by \(x\).
\[ \boxed{ r\approx 0 \text{ does not imply no nonlinear relationship} }. \]Common Nonlinear Shapes
| Shape | Possible interpretation | Candidate model feature |
|---|---|---|
| U-shape | A response falls and then rises | Quadratic term |
| Saturation | Early gains diminish toward a limit | Log transform or asymptotic curve |
| Threshold | The rate changes near a boundary | Piecewise or hinge term |
| Multiplicative growth | Equal \(x\)-steps multiply the response | Log-response model |
| Flexible smooth shape | The form is not known in advance | Spline, local smoother, or GAM |
Polynomial Regression
Polynomial regression represents curvature by adding powers of a predictor. A quadratic model is
\[ \boxed{ \hat{y}=b_0+b_1x+b_2x^2 }. \]A cubic model adds \(b_3x^3\), and higher degrees add further powers.
Linear in the Coefficients
The quadratic graph is curved in \(x\), but the model is linear in the coefficients \(b_0,b_1,b_2\). The columns \(x\) and \(x^2\) enter a multiple linear regression:
\[ \hat{y} = b_0+b_1(\text{column }x)+b_2(\text{column }x^2). \]“Linear model” and “straight-line graph” are therefore not identical ideas.
The Quadratic Rate of Change
For
\[ \hat{y}=b_0+b_1x+b_2x^2, \]the fitted instantaneous rate of change is
\[ \boxed{ \frac{d\hat{y}}{dx}=b_1+2b_2x }. \]The rate changes linearly with \(x\). The coefficient \(b_2\) controls the direction and degree of curvature; it is not itself a constant slope.
The Quadratic Turning Point
When \(b_2\ne 0\), set the derivative equal to zero:
\[ b_1+2b_2x=0. \]The turning point occurs at
\[ \boxed{x^\ast=-\frac{b_1}{2b_2}}. \]It is a minimum when \(b_2>0\) and a maximum when \(b_2<0\).
The Polynomial Hierarchy Principle
If \(x^2\) is included, the lower-order term \(x\) is usually retained. If \(x^3\) is included, \(x\) and \(x^2\) are usually retained.
This hierarchy preserves interpretation, reduces coding dependence, and allows the model to represent simpler shapes as special cases.
Centering Polynomial Predictors
Define a centered predictor
\[ x_c=x-c \]and construct powers from \(x_c\). Centering can make the intercept and lower-order coefficients easier to interpret and often reduces numerical correlation between \(x_c\) and \(x_c^2\).
If the same polynomial degree is retained, centering changes the coefficient representation but not the fitted polynomial curve.
A Worked Quadratic Dataset
Consider five observations:
| \(x\) | Observed \(y\) |
|---|---|
| \(-2\) | 18.1 |
| \(-1\) | 11.6 |
| 0 | 10.6 |
| 1 | 11.6 |
| 2 | 18.1 |
The response falls toward the center and rises again. A straight line would miss the dominant structure.
Example: The Fitted Quadratic
Ordinary least squares gives
\[ \boxed{\hat{y}=10+2x^2}. \]In the full quadratic notation, the fitted coefficients are
\[ b_0=10, \qquad b_1=0, \qquad b_2=2. \]Example: The Curve and Data
Example: Fitted Values and Residuals
| \(x_i\) | Observed \(y_i\) | Fitted \(\hat{y}_i\) | Residual \(e_i\) | \(e_i^2\) |
|---|---|---|---|---|
| \(-2\) | 18.1 | 18.0 | 0.1 | 0.01 |
| \(-1\) | 11.6 | 12.0 | \(-0.4\) | 0.16 |
| 0 | 10.6 | 10.0 | 0.6 | 0.36 |
| 1 | 11.6 | 12.0 | \(-0.4\) | 0.16 |
| 2 | 18.1 | 18.0 | 0.1 | 0.01 |
| Sums | 0.0 | 0.70 | ||
Thus,
\[ \operatorname{SSE}=0.70. \]Example: Model Fit
The response mean is \(\bar{y}=14\), and
\[ \operatorname{SST}=56.70. \]Therefore,
\[ R^2 = 1-\frac{0.70}{56.70} \approx 0.988. \]The high value reflects a deliberately clean teaching example. It is not a universal standard for nonlinear models.
Example: Changing Rate
The derivative of the fitted curve is
\[ \frac{d\hat{y}}{dx}=4x. \]- At \(x=-2\), the fitted rate is \(-8\).
- At \(x=0\), the fitted rate is \(0\).
- At \(x=2\), the fitted rate is \(8\).
One slope cannot summarize the complete relationship.
Example: The Turning Point
Here, \(b_1=0\) and \(b_2=2\), so
\[ x^\ast=-\frac{0}{2(2)}=0. \]Because \(b_2>0\), the turning point is a minimum. The fitted minimum response is
\[ \hat{y}(0)=10. \]Do Not Interpret Polynomial Coefficients Separately
When \(x\) and \(x^2\) appear together, the coefficient of \(x\) is the fitted rate specifically at \(x=0\), because
\[ \left.\frac{d\hat{y}}{dx}\right|_{x=0}=b_1. \]Meaningful interpretation usually focuses on fitted values, derivatives, turning points, and contrasts over a relevant range.
Higher-Degree Polynomials
Cubic and higher-degree polynomials can represent additional bends:
\[ \hat{y}=b_0+b_1x+b_2x^2+b_3x^3+\cdots+b_dx^d. \]Increasing degree always increases flexibility, but it can create unstable edge behavior, implausible oscillations, and overfitting.
Why Very High Degrees Are Risky
A degree-\(d\) polynomial is controlled globally: changing one coefficient affects the curve across the entire range. High-degree fits can swing dramatically near boundaries and extrapolate explosively.
Local basis methods, such as splines, are often more stable when substantial flexibility is needed.
Transforming the Predictor
A transformed predictor can represent a nonlinear relationship while retaining a linear coefficient structure:
\[ \hat{y}=b_0+b_1g(x). \]Common choices include
\[ g(x)=\log x, \qquad g(x)=\sqrt{x}, \qquad g(x)=\frac{1}{x}. \]Domain restrictions and interpretation must follow the chosen transformation.
The Linear–Log Model
A linear–log model has the form
\[ \hat{y}=b_0+b_1\log x, \qquad x>0. \]A multiplicative change in \(x\) produces an additive change in the fitted response:
\[ \Delta\hat{y} = b_1\log\left(\frac{x_2}{x_1}\right). \]For a small \(1\%\) increase in \(x\), the fitted response changes by approximately \(b_1/100\) response units.
Transforming the Response
A log-response model is
\[ \log y=b_0+b_1x+\varepsilon, \qquad y>0. \]On the original response scale, the fitted median under common log-error interpretations changes multiplicatively:
\[ \frac{\widehat{y}(x+1)}{\widehat{y}(x)}=e^{b_1}. \]Interpreting a Log-Response Coefficient
A one-unit increase in \(x\) is associated with an exact fitted percentage change of
\[ \boxed{ 100\left(e^{b_1}-1\right)\% }. \]When \(|b_1|\) is small, \(100b_1\%\) is a useful approximation.
The Log–Log Model
A log–log model is
\[ \log y=b_0+b_1\log x+\varepsilon, \qquad x>0,\ y>0. \]The coefficient \(b_1\) is an elasticity: a \(1\%\) increase in \(x\) is associated with approximately a \(b_1\%\) change in \(y\).
More exactly, multiplying \(x\) by a factor \(c\) multiplies the fitted response scale by \(c^{b_1}\).
Back-Transformation Requires Care
If a model is fitted to \(\log y\), simply exponentiating a fitted log value typically estimates a conditional median on the original scale under a log-normal formulation, not the conditional mean.
\[ E(Y\mid X) \ne \exp\!\left(E(\log Y\mid X)\right) \]in general. Estimating the original-scale mean may require a bias correction based on the error distribution or a smearing estimate.
Transformations Change the Question
Modeling \(y\), \(\sqrt{y}\), and \(\log y\) does not merely change the graph's appearance. It changes the loss scale, coefficient interpretation, residual definition, and target being estimated.
Choose transformations for a substantive and statistical reason, not only because one produces a larger \(R^2\).
Piecewise Linear Relationships
A piecewise linear model allows the fitted slope to change at a knot or breakpoint \(c\). Define the hinge function
\[ (x-c)_+=\max(0,x-c). \]Then
\[ \boxed{ \hat{y}=b_0+b_1x+b_2(x-c)_+ }. \]Interpreting the Hinge Model
Before the knot, \(x\le c\), the hinge equals zero, so the fitted slope is
\[ b_1. \]After the knot, \(x>c\), the fitted slope is
\[ \boxed{b_1+b_2}. \]The two line segments meet continuously at \(c\).
Choosing a Breakpoint
A breakpoint may come from a known policy threshold, physical mechanism, engineering limit, or prespecified scientific hypothesis. It may also be estimated from data.
Searching many possible breakpoints and reporting only the best one understates uncertainty unless the search is incorporated into the analysis.
Splines
Splines join low-degree polynomial pieces at knots. Continuity and smoothness conditions prevent arbitrary jumps or sharp bends unless the model is designed to allow them.
A spline can be written as a regression on basis functions:
\[ \hat{y} = b_0+b_1B_1(x)+\cdots+b_mB_m(x). \]Knots and Flexibility
More knots or a less restrictive smoothing penalty allow a curve to adapt more closely to the data. Too little flexibility underfits; too much flexibility follows noise.
Knot placement and smoothness should be prespecified, selected by a principled criterion, or evaluated with validation.
Natural Cubic Splines
A natural cubic spline is cubic between interior knots but constrained to be linear beyond the boundary knots. This reduces some extreme boundary behavior.
The constraint improves stability but does not make distant extrapolation automatically reliable.
Local Regression
Local regression, often called LOESS or LOWESS, fits simple models to neighborhoods of nearby \(x\)-values and combines them into a smooth curve.
A larger smoothing span uses wider neighborhoods and produces a smoother curve. A smaller span follows local variation more closely and can overfit.
Generalized Additive Models
A generalized additive model, or GAM, combines smooth functions of several predictors:
\[ g\!\left(E(Y\mid\mathbf{X})\right) = b_0+f_1(x_1)+f_2(x_2)+\cdots+f_p(x_p). \]The functions \(f_j\) allow nonlinear adjusted relationships, while the additive structure assumes that their contributions combine without interactions unless interactions are added.
Periodic Relationships
Repeating seasonal or cyclic patterns can be represented with sine and cosine terms:
\[ \hat{y} = b_0+b_1\sin(\omega x)+b_2\cos(\omega x). \]The angular frequency \(\omega\) corresponds to period
\[ T=\frac{2\pi}{\omega}. \]Including both sine and cosine terms allows the fitted cycle's phase to be estimated rather than fixed arbitrarily.
Mechanistic Nonlinear Models
Some applications supply a scientifically motivated curve whose parameters enter nonlinearly. For example, saturating growth may be modeled by
\[ y=\alpha\left(1-e^{-kx}\right)+\varepsilon. \]- \(\alpha\) is the limiting response.
- \(k\) controls how quickly the curve approaches that limit.
Nonlinear Least Squares
When coefficients enter the mean function nonlinearly, ordinary matrix formulas do not directly solve the problem. Nonlinear least squares uses iterative numerical methods to minimize
\[ \sum_{i=1}^{n} \left[y_i-f(x_i;\boldsymbol{\theta})\right]^2. \]Starting values, parameter constraints, local minima, and identifiability can affect the result.
Parametric and Flexible Models
| Approach | Strength | Limitation |
|---|---|---|
| Mechanistic parametric curve | Parameters may have direct scientific meaning | Wrong form can create systematic bias |
| Low-degree polynomial | Simple and easy to fit | Global shape can behave poorly at boundaries |
| Spline or GAM | Flexible, smooth adjusted relationship | Requires complexity control and interpretation tools |
| Local smoother | Useful exploratory shape estimate | Limited extrapolation and parameter interpretation |
Monotone Relationships
Subject knowledge may imply that a fitted response should only increase or only decrease. An unconstrained flexible model can violate that expectation by following noise.
Monotone splines, isotonic methods, or constrained estimation can encode a justified shape restriction.
Bounded and Asymptotic Responses
A response may have a natural floor, ceiling, or long-run limit. Polynomial models do not automatically respect such boundaries.
A transformation, generalized model, or mechanistic asymptotic curve may better represent the allowed range.
Interactions Can Create Nonlinear Marginal Patterns
Different groups may follow different lines or curves. When those groups are pooled, the overall relationship can appear curved even if each within-group relationship is simple.
Plot relevant groups and consider interactions before attributing every curve to one universal mechanism.
Nonconstant Variance Can Resemble Curvature
A widening point cloud may make the center of the data look curved. Examine conditional center and spread separately.
A smoother estimates the center; residual and scale–location plots assess the variation around that center.
Outliers Can Manufacture a Curve
One or two high-leverage points can pull a polynomial or spline toward an apparent bend. Verify unusual cases and compare fitted shapes with and without influential observations when appropriate.
A sensitivity comparison does not justify deleting a valid case.
Residual Diagnostics for Nonlinear Models
After fitting a nonlinear relationship, inspect residuals against
- fitted values;
- each predictor;
- observation order;
- important groups;
- the original and transformed scales when relevant.
A nonlinear model is not validated merely because its curve looks smooth.
Effective Degrees of Freedom
Flexible smoothers are often described by effective degrees of freedom. A value near \(1\) for a smooth term resembles a straight-line contribution; larger values allow more bends.
Effective degrees of freedom measure flexibility, not sample size or substantive importance.
Underfitting
A model underfits when it is too rigid to represent stable structure. Warning signs include systematic residual curvature, biased predictions in regions of the predictor space, and poor validation performance.
Overfitting
A model overfits when it learns noise or sample-specific detail. It may produce a very small training error, an erratic curve, unstable predictions, and poor performance on new data.
Complexity penalties, cross-validation, held-out data, and subject-matter constraints help control overfitting.
Selecting Complexity
Candidate degrees, knot structures, smoothing penalties, and transformations should be evaluated by criteria aligned with the goal:
- out-of-sample prediction error;
- information criteria;
- prespecified scientific structure;
- parsimony and interpretability;
- diagnostic performance.
Training Fit Always Rewards Flexibility
Adding polynomial terms or spline basis functions cannot increase ordinary least-squares training SSE when the smaller model is nested in the larger one.
Improved training fit is therefore expected even when added flexibility captures no stable signal.
Validation Estimates Generalization
Validation evaluates the complete modeling process on data not used to fit the candidate model. It helps determine whether apparent curvature is reproducible and whether added flexibility improves prediction.
If the validation data influence model selection, they are no longer a final untouched test set.
Extrapolation Is Especially Dangerous
Nonlinear models can change rapidly outside the observed range. A polynomial may diverge, a spline may follow its boundary constraint, and a local smoother may have no meaningful neighborhood.
The fitted curve's behavior beyond the data is driven largely by structural assumptions rather than direct evidence.
Interpolating Through Data Gaps
A value can lie between the minimum and maximum \(x\)-values while still falling in a large unsupported gap. A flexible curve across that gap depends on its smoothness assumptions.
Inspect the density of observations across the entire predictor range.
Uncertainty Bands
A fitted curve should be accompanied by uncertainty when inference or decision-making requires it. Uncertainty is usually smallest in dense, well-supported regions and larger near boundaries, gaps, or unusual predictor profiles.
Pointwise and simultaneous bands answer different questions. Selecting a curve after searching many alternatives also adds uncertainty.
Prediction Intervals
A confidence band for the mean curve does not describe the full variation of a new individual response. Prediction intervals must include both uncertainty about the fitted mean and individual outcome variation.
A later lesson develops prediction intervals directly.
Nonlinearity Does Not Establish Causation
A carefully fitted curve still describes association unless the study design and causal assumptions justify more. Confounding, selection, reverse causation, and measurement error can generate smooth nonlinear patterns.
A Reliable Modeling Workflow
- Define the cases, response, predictors, units, range, and goal.
- Inspect scatterplots, groups, boundaries, and data density.
- Fit a simple baseline model and examine residual structure.
- Choose a plausible nonlinear representation using subject knowledge.
- Preserve hierarchy when using powers or interactions.
- Interpret fitted values, rates, contrasts, and turning points.
- Check residuals, variance, leverage, influence, and support.
- Select complexity without reusing evaluation data improperly.
- Validate the complete modeling process on new or withheld cases.
- Limit extrapolation and report uncertainty and causal status.
Reporting a Nonlinear Model
A useful report states the model form, transformations, knot or smoothing choices, predictor range, fitted contrasts, uncertainty method, diagnostic findings, validation procedure, and extrapolation limits.
Graph the fitted curve with the observed data whenever possible. A formula alone can conceal the relationship's changing rate and supported range.
Common Error: Using Correlation to Rule Out a Relationship
A correlation near zero rules out only a strong overall linear association. A symmetric or otherwise curved relationship may still be strong.
Common Error: Dropping Lower-Order Terms
Including \(x^2\) without \(x\), or \(x^3\) without lower powers, imposes special constraints that may depend on where zero is placed. Preserve hierarchy unless a scientific restriction justifies otherwise.
Common Error: Calling \(b_2\) the Slope
In a quadratic model, the fitted slope is \(b_1+2b_2x\). The coefficient \(b_2\) controls curvature and does not by itself equal the rate of change.
Common Error: Selecting by Training \(R^2\)
Training \(R^2\) generally rises as flexibility is added. Model selection must balance fit with complexity, diagnostics, subject knowledge, and generalization.
Common Error: Ignoring Transformation Domains
\(\log x\) requires \(x>0\), and \(\sqrt{x}\) requires \(x\ge 0\) in real-valued analysis. Arbitrarily shifting data to force a transformation changes interpretation and must be justified.
Common Error: Back-Transforming Without Adjustment
Exponentiating a fitted log response does not generally recover the conditional mean of the original response. The target and error model determine the correct back-transformation.
Common Error: Trusting a Curve Outside the Data
Smooth appearance within the observed range provides little evidence about distant extrapolation. Boundary behavior comes from model assumptions.
Common Error: Treating a Smooth Curve as Causal
Flexibility can describe an association more closely, but it does not eliminate confounding or establish an intervention effect.
Practice 1: Recognize Nonlinearity
A response rises quickly at low \(x\)-values and then approaches a stable ceiling. Why is a single straight line likely incomplete?
Practice 2: Quadratic Prediction
For
\[ \hat{y}=5+3x+2x^2, \]find the fitted response at \(x=2\).
Practice 3: Find a Turning Point
Find the turning point of
\[ \hat{y}=12-4x+x^2 \]and determine whether it is a minimum or maximum.
Practice 4: Apply Hierarchy
A model includes \(x^3\). Which lower-order terms are ordinarily retained?
Practice 5: Interpret a Log-Response Coefficient
In
\[ \log y=b_0+0.08x, \]find the exact fitted percentage change associated with a one-unit increase in \(x\).
Practice 6: Interpret a Log–Log Coefficient
In
\[ \log y=b_0+1.4\log x, \]interpret \(1.4\).
Practice 7: Interpret a Hinge Model
For
\[ \hat{y}=10+2x-1.5(x-6)_+, \]find the fitted slope before and after \(x=6\).
Practice 8: Diagnose Residual Curvature
Residuals from a straight-line model form a clear inverted U. What does the pattern indicate?
Practice 9: Evaluate Extrapolation
A cubic model fits data observed for \(0\le x\le 10\). Why should a prediction at \(x=100\) be treated cautiously?
Practice 10: Critique a Causal Claim
A flexible spline shows that health outcome \(y\) varies smoothly with exposure \(x\). An analyst says the curve proves the causal dose-response function. What is missing?
Practice Answers
Practice 1
The fitted rate of increase decreases as \(x\) grows and approaches zero near the ceiling. A straight line imposes one constant rate and may overpredict at high \(x\)-values.
Practice 2
\[ \hat{y} = 5+3(2)+2(2^2) = 5+6+8 = 19. \]Practice 3
Here, \(b_1=-4\) and \(b_2=1\). Therefore,
\[ x^\ast=-\frac{-4}{2(1)}=2. \]Since \(b_2>0\), the turning point is a minimum. Its fitted response is
\[ \hat{y}(2)=12-4(2)+2^2=8. \]Practice 4
Ordinarily retain the intercept, \(x\), and \(x^2\) along with \(x^3\).
Practice 5
\[ 100(e^{0.08}-1)\% \approx 8.33\%. \]A one-unit increase in \(x\) is associated with an approximately \(8.33\%\) increase in the fitted original response scale under the stated log model.
Practice 6
The coefficient is an elasticity. A \(1\%\) increase in \(x\) is associated with approximately a \(1.4\%\) increase in \(y\). More exactly, multiplying \(x\) by \(c\) multiplies the fitted response scale by \(c^{1.4}\).
Practice 7
Before \(x=6\), the hinge is zero, so the fitted slope is \(2\). After \(x=6\), the fitted slope is
\[ 2-1.5=0.5. \]Practice 8
The line systematically underpredicts in the middle and overpredicts near the ends. The mean structure is missing curvature.
Practice 9
\(x=100\) is far outside the observed range. A cubic can grow or decline rapidly, and its distant behavior is determined by the polynomial assumption rather than observed support.
Practice 10
A smooth association does not identify causation. The analysis still needs an appropriate design and defensible assumptions addressing confounding, selection, reverse causation, measurement, and the intervention being defined.
What to Remember
A nonlinear relationship has a fitted rate that changes across the predictor range.
- A residual curve signals that a straight-line mean may be incomplete.
- A correlation near zero can conceal a strong nonlinear relationship.
- Quadratic slopes are \(b_1+2b_2x\), not simply \(b_2\).
- Preserve hierarchy when using powers and interactions.
- Transformations change the scale and interpretation of the model.
- Hinge functions and splines provide controlled local flexibility.
- Flexible models require complexity control and validation.
- Extrapolation depends heavily on unverified structural assumptions.
- A well-fitted curve does not establish causation.
Where We Go Next
The models in this lesson predict a quantitative response. Many practical decisions instead require assigning cases to categories or estimating the probability of a categorical outcome.
The next lesson, Classification, develops probability-based prediction, decision thresholds, confusion matrices, and the distinction between estimating risk and assigning a class.