The Central Idea
Observed association asks how outcomes differ across existing exposure groups. Causation asks how outcomes would differ if the exposure were changed:
\[ \boxed{ \text{observed comparison} \ne \text{effect of an intervention} }. \]The two comparisons can coincide under suitable design and assumptions, but they are not identical by definition.
Association
Variables \(X\) and \(Y\) are associated when the distribution of \(Y\) differs across values of \(X\). For example,
\[ E(Y\mid X=1) \ne E(Y\mid X=0). \]This statement describes observed groups. It does not explain why they differ.
Causation
\(X\) causes \(Y\) when intervening on \(X\), under a defined setting, changes the distribution of \(Y\). Causal claims therefore require
- a manipulable or well-defined exposure contrast;
- an outcome and time horizon;
- a target population;
- a comparison of counterfactual outcomes.
Prediction and Causal Effect Are Different Targets
| Goal | Typical question |
|---|---|
| Description | How are variables related in the observed data? |
| Prediction | What outcome is likely for a new case? |
| Causal inference | What would happen if an exposure or policy changed? |
A Predictive Feature Need Not Be a Cause
An umbrella can predict rain because people carry umbrellas when rain is expected. Destroying umbrellas would not stop the rain. The umbrella is a predictive marker, not the cause of the weather.
Predictive systems may appropriately use noncausal proxies, but those variables should not be interpreted as intervention targets.
A Cause Need Not Be a Strong Predictor
An exposure can have a real causal effect yet explain little individual outcome variation when many other factors dominate. Predictive importance and causal importance are different concepts.
Temporal Order Is Necessary
A cause must precede its effect. If \(Y\) occurs before \(X\), \(X\) cannot cause that earlier \(Y\). Correct time order rules out some explanations, but it does not by itself eliminate confounding or selection.
Reverse Causation
An outcome may influence the presumed exposure. Poor health can reduce exercise, creating an association between low exercise and poor health even when part of the direction runs from health to activity.
Longitudinal timing, lagged measurements, and experimental intervention can help distinguish direction.
Confounding
A confounder is a common cause, or a suitable proxy for common causes, of the exposure and outcome. It creates an open noncausal path between them.
If age influences both treatment choice and health outcome, a crude treatment–outcome association may partly reflect age differences between groups.
A Confounding Diagram
Correlation Does Not Resolve Direction
Correlation is symmetric:
\[ \operatorname{Corr}(X,Y) = \operatorname{Corr}(Y,X). \]Causal direction is asymmetric. The same correlation is compatible with \(X\rightarrow Y\), \(Y\rightarrow X\), a common cause, selection, or a mixture of mechanisms.
Regression Does Not Create Causality
In
\[ E(Y\mid X,Z) = \beta_0+\beta_1X+\beta_2Z, \]\(\beta_1\) is an adjusted association under the fitted model. It becomes a causal effect only when the adjustment set, study design, timing, measurement, model, and causal assumptions justify that interpretation.
Statistical Significance Is Not Causal Evidence
A small \(p\)-value addresses compatibility with a statistical null under specified assumptions. It does not rule out confounding, reverse causation, measurement error, selection bias, or a poorly defined intervention.
Large Samples Do Not Remove Bias
More observations can make an associational estimate extremely precise around the wrong causal quantity. Sampling variability may shrink while systematic bias remains.
\[ \boxed{ \text{precision} \ne \text{causal validity} }. \]Potential Outcomes
For a binary exposure \(A\), define
\[ Y_i(1) = \text{outcome for unit }i\text{ under }A=1, \] \[ Y_i(0) = \text{outcome for unit }i\text{ under }A=0. \]The individual causal effect is \(Y_i(1)-Y_i(0)\).
The Fundamental Causal Problem
For one unit at one time, only one potential outcome is observed:
\[ Y_i = A_iY_i(1) + (1-A_i)Y_i(0). \]The missing counterfactual must be represented through design and assumptions using comparable units.
Average Treatment Effect
A common causal estimand is the average treatment effect:
\[ \boxed{ \operatorname{ATE} = E[Y(1)-Y(0)] }. \]Other targets include effects among treated units, effects for eligible units, and effects within subgroups. State the estimand before selecting a method.
Randomized Assignment
In a well-conducted randomized experiment, treatment assignment is independent of potential outcomes in expectation:
\[ A \perp \{Y(1),Y(0)\}. \]Randomization makes treatment groups comparable on measured and unmeasured pre-treatment causes, apart from chance imbalances.
Randomization Is Not a Complete Guarantee
Attrition, noncompliance, missing outcomes, interference, protocol changes, poor measurement, and selective reporting can compromise an experiment. Randomization also does not guarantee that results transport to other populations or interventions.
Intention-to-Treat Effects
An intention-to-treat analysis compares groups according to randomized assignment, regardless of compliance. It preserves the randomized comparison and estimates the effect of assignment or offer under the study conditions.
The effect of actually receiving treatment is a different estimand and needs additional assumptions.
Observational Studies
In observational data, exposure is not randomly assigned. Treated and untreated groups may differ before treatment in ways that also affect the outcome.
Causal analysis must identify and address those differences using substantive knowledge, design, measurement, and statistical adjustment.
Three Core Identification Conditions
| Condition | Meaning |
|---|---|
| Consistency | The observed outcome under the received exposure equals the corresponding potential outcome |
| Conditional exchangeability | No unblocked confounding remains after conditioning on measured \(C\) |
| Positivity | Every relevant covariate pattern has a positive chance of each exposure |
Conditional Exchangeability
\[ \{Y(1),Y(0)\} \perp A \mid C. \]This assumption says that within levels of measured pre-treatment covariates \(C\), exposure assignment behaves as though random with respect to potential outcomes. It cannot be fully verified from observed data.
Positivity and Overlap
\[ 0 < P(A=1\mid C=c) < 1 \]for every covariate pattern \(c\) in the target population. If older patients always receive treatment and younger patients never do, the data contain no within-age comparison for the missing alternatives.
Consistency Requires a Defined Treatment
“Exercise,” “education,” or “policy exposure” may represent many versions. A causal contrast must specify dose, timing, duration, delivery, and relevant co-interventions closely enough that the potential outcomes are meaningful.
Interference
Standard potential-outcome notation often assumes one unit's treatment does not alter another unit's outcome. This can fail with vaccines, social networks, classrooms, markets, and environmental policies.
When interference is plausible, define clusters, spillovers, and treatment allocation at the appropriate level.
Adjustment by Stratification
Compare exposed and unexposed units within levels of confounders, then combine stratum-specific effects for the target population. This makes the comparison groups more alike on the stratifying variables.
Sparse cells limit adjustment when many confounders or categories are present.
Standardization
Under identification and modeling assumptions, a standardized average effect can be estimated as
\[ E_C \left[ E(Y\mid A=1,C) - E(Y\mid A=0,C) \right]. \]The inner models compare exposure levels at fixed \(C\); the outer average returns the comparison to a specified covariate distribution.
Propensity Scores
The propensity score is
\[ e(C)=P(A=1\mid C). \]It summarizes measured exposure predictors and supports matching, stratification, or weighting. It does not control unmeasured confounding, and strong prediction of treatment is not its ultimate objective.
Inverse-Probability Weighting
A basic treatment weight is
\[ w_i = \frac{A_i}{e(C_i)} + \frac{1-A_i}{1-e(C_i)}. \]Weighting creates a pseudopopulation balanced on measured confounders when the propensity model and assumptions are adequate. Extreme weights warn of poor overlap or model problems.
Matching
Matching pairs or groups exposed and unexposed units with similar pre-treatment covariates. Its quality should be assessed through covariate balance and overlap, not outcome significance alone.
Matching can change the target population when unmatched units are discarded.
Regression Adjustment
Outcome regression can estimate counterfactual means after adjusting for a sufficient set of pre-treatment confounders. Flexible terms may reduce model-form bias, but extreme extrapolation and omitted confounding remain concerns.
Doubly Robust Estimation
Some estimators combine an outcome model with a treatment-assignment model. Under suitable conditions, consistency may require only one of the two nuisance models to be correctly specified.
“Doubly robust” does not protect against unmeasured confounding, nonpositivity, an ill-defined treatment, or errors in both models.
Do Not Adjust for Every Available Variable
A variable's causal role determines whether adjustment helps. Controlling a confounder can reduce bias, while controlling a mediator or collider can answer the wrong question or create bias.
\[ \boxed{ \text{covariate selection is a causal decision} }. \]Mediators
A mediator lies on a causal pathway:
\[ A\rightarrow M\rightarrow Y. \]Adjusting for \(M\) can remove part of the total effect of \(A\). Mediation analysis requires a clearly defined direct or indirect effect and stronger assumptions than total-effect estimation.
Colliders
A collider is caused by two variables:
\[ X\rightarrow S\leftarrow Y. \]Conditioning on \(S\), or selecting only units with \(S=1\), can create an association between \(X\) and \(Y\) even when none existed in the full population.
Selection Bias
Participation, survival, complete-case inclusion, or platform membership may depend on exposure and outcome causes. The analyzed sample can then contain a different association from the target population.
Larger selected samples do not restore missing target-population comparisons.
Measurement Error
Misclassified exposures, noisy outcomes, and imperfect confounder measurements can bias causal estimates. Differential measurement—when error depends on exposure or outcome status—is especially difficult.
Validation subsamples and measurement models can support correction.
A Worked Simpson's Paradox Example
Suppose coffee use and heart events are tabulated by age:
| Age group | Coffee group | Events | People | Risk |
|---|---|---|---|---|
| Younger | High | \(1\) | \(90\) | \(1.11\%\) |
| Younger | Low | \(9\) | \(810\) | \(1.11\%\) |
| Older | High | \(72\) | \(810\) | \(8.89\%\) |
| Older | Low | \(8\) | \(90\) | \(8.89\%\) |
The Crude Association
Combining age groups gives
\[ \text{high-coffee risk} = \frac{1+72}{90+810} = \frac{73}{900} \approx 8.11\%, \] \[ \text{low-coffee risk} = \frac{9+8}{810+90} = \frac{17}{900} \approx 1.89\%. \]The crude risk difference is approximately \(6.22\) percentage points.
The Stratified Comparison
Within each age group, high- and low-coffee risks are identical:
\[ \frac{1}{90} = \frac{9}{810} \approx 1.11\%, \] \[ \frac{72}{810} = \frac{8}{90} \approx 8.89\%. \]Older people are concentrated in the high-coffee group. In this constructed example, age composition creates the entire crude association.
What Simpson's Paradox Teaches
Aggregate and stratified associations can differ or reverse. This does not mean stratification is always correct. The causal diagram determines whether a variable is a confounder, mediator, collider, or irrelevant covariate.
Causal Diagrams
A directed acyclic graph records assumed causal directions before statistical adjustment. Arrows encode causal claims, not merely correlations.
A diagram helps identify open backdoor paths, appropriate adjustment sets, mediators, colliders, and assumptions that the data alone cannot establish.
Backdoor Adjustment
A valid adjustment set blocks noncausal backdoor paths from exposure to outcome without blocking the causal effect of interest or opening collider paths.
The criterion depends on a credible graph. Statistical algorithms cannot recover all relevant causal structure from observational associations without assumptions.
Natural Experiments
Policy thresholds, lotteries, timing shocks, or institutional rules can create exposure variation resembling random assignment. The design is persuasive only when the assignment mechanism, exclusion restrictions, and absence of strategic manipulation are credible.
Instrumental Variables
An instrument changes exposure, is independent of relevant unmeasured causes, and affects the outcome only through the exposure under the stated assumptions. Instrumental-variable estimates often target a local effect for units whose exposure responds to the instrument.
Instrument validity is primarily a substantive causal claim.
Regression Discontinuity
When treatment changes at a threshold of a running variable, outcomes just above and below the cutoff may be compared. Credibility depends on continuity near the cutoff, limited manipulation, and an appropriate local model.
Difference in Differences
Difference-in-differences compares outcome changes over time between exposed and comparison groups. A central identifying condition is that, without treatment, the groups would have followed parallel outcome trends.
Pre-trend evidence is informative but cannot prove the counterfactual post-treatment trend.
Effect Heterogeneity
A treatment can help some units more than others:
\[ E[Y(1)-Y(0)\mid G=g]. \]Subgroup effects should be substantively motivated and evaluated with adequate data. Searching many subgroups can produce chance findings.
Transportability
A valid causal effect in one study population may not equal the effect in another population with different effect modifiers, treatment delivery, or background conditions.
External validity requires comparing populations and specifying the target to which the effect will be transported.
Sensitivity Analysis
Observational conclusions should examine how strong unmeasured confounding, selection, or measurement error would need to be to materially change the result.
Sensitivity analysis does not remove bias; it makes dependence on uncertain assumptions visible.
Negative Controls and Falsification
A negative-control outcome should not plausibly be caused by the exposure, and a negative-control exposure should not plausibly cause the outcome. Unexpected associations can reveal residual confounding or measurement problems.
Passing a falsification check strengthens credibility but does not prove the causal model.
Triangulation
Confidence increases when different designs with different likely biases support compatible conclusions: randomized trials, natural experiments, longitudinal studies, mechanistic evidence, and carefully adjusted observational analyses.
Repetition of the same design does not diversify its structural bias.
Causal Language Should Match Evidence
| Evidence supports | Appropriate language |
|---|---|
| Descriptive association | “was associated with,” “differed by,” “predicted” |
| Credible causal effect | “increased,” “reduced,” “caused,” with population and intervention stated |
| Uncertain mechanism | “may affect,” followed by explicit assumptions and alternatives |
A Practical Causal Workflow
- State the exposure, alternatives, outcome, timing, and target population.
- Define the causal estimand, such as an ATE or subgroup effect.
- Draw the assumed causal structure before selecting controls.
- Choose a design that approximates the missing counterfactual.
- Assess consistency, exchangeability, positivity, and interference.
- Audit selection, attrition, measurement, and missing data.
- Estimate effects with methods aligned to the design.
- Check overlap, balance, model sensitivity, and alternative specifications.
- Quantify sampling uncertainty and sensitivity to unmeasured bias.
- Report conclusions in language proportional to the evidence.
Reporting Checklist
- What intervention and comparison are being contrasted?
- Which population and time horizon define the estimand?
- Why are the comparison groups exchangeable?
- Which variables are confounders, mediators, and colliders?
- Is there adequate overlap for every target subgroup?
- How were selection, missingness, and measurement addressed?
- What assumptions cannot be tested directly?
- How sensitive is the conclusion to plausible violations?
Common Error: “Correlation Proves Nothing”
Association is real statistical information and may support prediction, description, hypothesis generation, or causal analysis within a stronger design. The error is not studying correlation; it is silently upgrading association into causation.
Common Error: Controlling for Everything
Automated adjustment can control mediators, open collider paths, amplify measurement error, or destroy overlap. Use causal knowledge to select pre-treatment adjustment variables.
Common Error: Equating Prediction with Explanation
High validation accuracy shows that a model transports predictive information under similar conditions. It does not identify which features should be changed to alter the outcome.
Common Error: Ignoring the Treatment Version
“More education” or “better care” is not a single intervention. Different versions can have different effects. Define the actionable contrast before interpreting an estimate causally.
Common Error: Treating Assumptions as Results
Balance tables, fit metrics, and robustness checks provide evidence, but they do not prove no unmeasured confounding or perfect consistency. Separate empirical findings from identifying assumptions.
Practice 1: Classify the Question
“Which customers are most likely to cancel next month?” Is this primarily a predictive or causal question?
Practice 2: Define a Causal Question
Rewrite “Does exercise improve health?” as a more precise causal question by specifying an intervention, comparison, outcome, horizon, and population.
Practice 3: Identify Confounding
Ice-cream sales and drownings rise together. Name a plausible common cause.
Practice 4: Calculate the Crude Risk Difference
Use the worked coffee example to calculate the crude high-minus-low risk difference from the exact fractions.
Practice 5: Identify a Mediator
A job-training program increases skills, which increase earnings. What role does skill level play in the total effect pathway?
Practice 6: Identify a Collider
Talent and family connections both influence admission to an elite program. What risk arises from analyzing only admitted people?
Practice 7: Evaluate Positivity
Every patient with severe disease receives treatment, and no patient with mild disease does. Can the treatment effect be estimated at fixed severity from these data alone?
Practice 8: Interpret Randomization
What main causal problem does randomized assignment address, and what practical problems can remain?
Practice 9: Critique a Regression Claim
An observational regression finds \(\beta_1=2.4\) for exposure \(X\). Why is “increasing \(X\) causes \(Y\) to rise by \(2.4\)” not automatically justified?
Practice 10: Match the Language
A cross-sectional survey finds that sleep duration predicts test score after adjusting for age. Give an appropriately cautious conclusion.
Practice Answers
Practice 1
It is primarily predictive. It asks who is likely to cancel, not what intervention would prevent cancellation.
Practice 2
One example is: among sedentary adults aged \(40\)–\(60\), what is the twelve-month effect on systolic blood pressure of assigning \(150\) minutes of supervised moderate exercise per week versus usual activity?
Practice 3
Hot weather or season is a plausible common cause because it increases both ice-cream purchases and swimming exposure.
Practice 4
\[ \frac{73}{900} - \frac{17}{900} = \frac{56}{900} \approx 0.0622. \]The crude risk difference is approximately \(6.22\) percentage points.
Practice 5
Skill level is a mediator on the pathway from training to earnings. Adjusting for it would remove part of the total effect.
Practice 6
Admission is a collider caused by talent and connections. Conditioning on admission can create a spurious association between those causes.
Practice 7
No. There is no treatment overlap within severity levels, so the data do not support treated-versus-untreated comparisons at fixed severity without extrapolation or additional information.
Practice 8
Randomization addresses confounding by making assignment independent of potential outcomes in expectation. Chance imbalance, noncompliance, attrition, missing outcomes, interference, poor measurement, and limited external validity can remain.
Practice 9
The coefficient is an adjusted association under the fitted model. Causal interpretation additionally requires a defined intervention, correct time order, sufficient confounding control, overlap, appropriate measurement, and a model aligned with the causal estimand.
Practice 10
In the surveyed population, sleep duration was associated with and predicted test score after adjustment for age. The cross-sectional design does not by itself establish that changing sleep duration would change scores.
What to Remember
- Association describes observed co-variation; causation describes intervention effects.
- Predictive accuracy does not identify an actionable cause.
- Temporal order is necessary but not sufficient for causation.
- Confounding, reverse causation, selection, and measurement can distort associations.
- Potential outcomes define causal effects through counterfactual contrasts.
- Randomization supports exchangeability but does not solve every study problem.
- Observational identification requires consistency, exchangeability, and positivity.
- Confounders may need adjustment; mediators and colliders require different treatment.
- Causal diagrams make assumptions and adjustment logic explicit.
- Causal language should remain proportional to the design and evidence.
Sequence Complete
This lesson completes Statistical Relationships and Prediction. The sequence now connects association, regression, diagnostics, categorical and nonlinear predictors, classification, prediction uncertainty, complexity control, validation, and causal interpretation.
The next step is consolidation: compare the assumptions and targets of each method, then select tools according to whether the objective is description, prediction, uncertainty quantification, or causal decision-making.