The Central Idea
Suppose each case has paired quantitative observations
\[ (x_i,y_i). \]We compare each coordinate with its variable mean:
\[ x_i-\bar{x}, \qquad y_i-\bar{y}. \]If these deviations usually have the same sign, the variables tend to move together. If they usually have opposite signs, the variables tend to move in opposite directions.
Products of Paired Deviations
The key quantity is
\[ \boxed{ (x_i-\bar{x})(y_i-\bar{y}) }. \]Its sign records how the two observations sit relative to their means.
Same-Sign Deviations
If both values are above their means, then
\[ x_i-\bar{x}>0, \qquad y_i-\bar{y}>0, \]so
\[ (x_i-\bar{x})(y_i-\bar{y})>0. \]The same is true when both values are below their means because the product of two negative deviations is positive.
Opposite-Sign Deviations
If one value is above its mean while the other is below, then the deviations have opposite signs.
Therefore,
\[ (x_i-\bar{x})(y_i-\bar{y})<0. \]Such pairs support negative association.
Population Covariance
For a population of \(N\) paired observations, population covariance is
\[ \boxed{ \operatorname{Cov}(X,Y) = \frac{1}{N} \sum_{i=1}^{N} (x_i-\mu_x)(y_i-\mu_y) }. \]Here \(\mu_x\) and \(\mu_y\) are the population means.
Sample Covariance
For a sample of \(n\) paired observations, the conventional sample covariance is
\[ \boxed{ s_{xy} = \frac{1}{n-1} \sum_{i=1}^{n} (x_i-\bar{x})(y_i-\bar{y}) }. \]The denominator \(n-1\) parallels the conventional sample-variance formula.
Interpreting Covariance
- If covariance is positive, the variables tend to move in the same direction.
- If covariance is negative, the variables tend to move in opposite directions.
- If covariance is near zero, there may be little net linear co-movement.
A covariance near zero does not prove that no relationship exists.
Example: Positive Covariance
Consider the paired observations
\[ (1,2),\quad(2,4),\quad(3,6). \]The means are
\[ \bar{x}=2, \qquad \bar{y}=4. \]The paired deviations are:
| \(x_i-\bar{x}\) | \(y_i-\bar{y}\) | Product |
|---|---|---|
| \(-1\) | \(-2\) | \(2\) |
| \(0\) | \(0\) | \(0\) |
| \(1\) | \(2\) | \(2\) |
The products are nonnegative and sum to \(4\), indicating positive covariance.
Example: Negative Covariance
Consider
\[ (1,6),\quad(2,4),\quad(3,2). \]Again,
\[ \bar{x}=2, \qquad \bar{y}=4. \]The products of paired deviations are
\[ (-1)(2)=-2, \] \[ (0)(0)=0, \] \[ (1)(-2)=-2. \]Their sum is negative, indicating negative covariance.
Covariance Depends on Units
If \(x\) is measured in meters and \(y\) is measured in seconds, covariance has units
\[ \text{meter}\cdot\text{second}. \]Changing either measurement scale changes the numerical magnitude of the covariance.
This makes covariance useful for direction and algebraic analysis, but less convenient for comparing association strength across different scales.
Why Correlation Standardizes Covariance
To remove the effect of measurement units, divide covariance by the product of the standard deviations of the two variables.
This creates a unit-free measure called correlation.
Population Correlation
Population correlation is commonly denoted by the Greek letter \(\rho\):
\[ \boxed{ \rho = \frac{\operatorname{Cov}(X,Y)} {\sigma_x\sigma_y} }. \]Sample Correlation
The sample Pearson correlation coefficient is commonly denoted by \(r\):
\[ \boxed{ r = \frac{s_{xy}}{s_xs_y} }. \]Equivalently,
\[ \boxed{ r = \frac{ \sum_{i=1}^{n}(x_i-\bar{x})(y_i-\bar{y}) }{ \sqrt{\sum_{i=1}^{n}(x_i-\bar{x})^2} \sqrt{\sum_{i=1}^{n}(y_i-\bar{y})^2} } }. \]Correlation as an Average Standardized Product
If the variables are standardized using
\[ z_{x,i} = \frac{x_i-\bar{x}}{s_x}, \]and
\[ z_{y,i} = \frac{y_i-\bar{y}}{s_y}, \]then sample correlation can be written
\[ \boxed{ r = \frac{1}{n-1} \sum_{i=1}^{n} z_{x,i}z_{y,i} }. \]Correlation therefore summarizes whether standardized deviations tend to agree in sign and magnitude.
The Range of Correlation
Pearson correlation always satisfies
\[ \boxed{-1\le r\le1}. \]This fixed scale makes correlation much easier to interpret than raw covariance.
Positive Correlation
If
\[ r>0, \]larger \(x\)-values tend to be paired with larger \(y\)-values.
The closer \(r\) is to \(1\), the stronger the positive linear association.
Negative Correlation
If
\[ r<0, \]larger \(x\)-values tend to be paired with smaller \(y\)-values.
The closer \(r\) is to \(-1\), the stronger the negative linear association.
Correlation Near Zero
If
\[ r\approx0, \]there is little net linear association.
This does not imply that the variables are unrelated.
A strong curved relationship can have correlation near zero.
Perfect Positive Correlation
If all observations lie exactly on a straight line with positive slope, then
\[ \boxed{r=1}. \]Every standardized movement in one variable is matched perfectly by the other.
Perfect Negative Correlation
If all observations lie exactly on a straight line with negative slope, then
\[ \boxed{r=-1}. \]Correlation Measures Linear Association
Pearson correlation is designed to measure linear association.
A dataset can have a strong systematic relationship and still have
\[ r\approx0 \]if that relationship is strongly nonlinear.
Example: Strong Nonlinear Association with Low Correlation
Suppose
\[ y=x^2 \]and the \(x\)-values are symmetrically distributed around zero.
The data follow a perfect U-shaped relationship, yet positive and negative linear tendencies can cancel.
The Pearson correlation may therefore be near zero.
Correlation Has No Units
Covariance has units, but correlation divides by
\[ s_xs_y. \]The units cancel.
Correlation is therefore dimensionless.
Changing Units Does Not Change Correlation
Converting heights from meters to centimeters changes raw values and standard deviation by the same positive factor.
Correlation remains unchanged.
This is one of the major advantages of standardization.
Adding Constants Does Not Change Correlation
Suppose
\[ X^*=X+a \]and
\[ Y^*=Y+b. \]Adding constants shifts the means but does not change the deviations from those means.
Therefore, correlation is unchanged.
Multiplying by Positive Constants
If
\[ X^*=aX, \qquad Y^*=bY, \]with
\[ a>0,\qquad b>0, \]then
\[ \operatorname{Corr}(X^*,Y^*) = \operatorname{Corr}(X,Y). \]Multiplying One Variable by a Negative Constant
Multiplying one variable by a negative number reverses its direction.
Therefore,
\[ \operatorname{Corr}(-X,Y) = -\operatorname{Corr}(X,Y). \]The magnitude stays the same, but the sign reverses.
Correlation Is Symmetric
Correlation does not depend on which variable is called \(x\) and which is called \(y\):
\[ \boxed{ \operatorname{Corr}(X,Y) = \operatorname{Corr}(Y,X) }. \]Correlation measures association, not directional causation.
Correlation Does Not Have a Slope Unit
A correlation of \(0.8\) does not mean that \(y\) increases by \(0.8\) units whenever \(x\) increases by one unit.
That interpretation belongs to slope in a fitted line, not correlation.
Correlation and Strength
The absolute value
\[ |r| \]measures the strength of linear association.
Values closer to \(1\) indicate tighter linear structure.
Values closer to \(0\) indicate weaker linear structure.
There Is No Universal Strength Cutoff
Labels such as weak, moderate, and strong depend on context.
A correlation considered strong in one scientific field may be considered modest in another.
Report the numerical value and inspect the scatterplot rather than relying only on arbitrary verbal categories.
Example: Interpreting \(r=0.82\)
A correlation of
\[ r=0.82 \]indicates a fairly strong positive linear association in many contexts.
It does not imply that \(82\%\) of one variable is caused by the other.
Example: Interpreting \(r=-0.67\)
A correlation of
\[ r=-0.67 \]indicates a moderately strong negative linear association.
Larger values of one variable tend to occur with smaller values of the other.
Correlation and Outliers
Correlation is not resistant.
A single unusual observation can substantially increase or decrease \(r\).
Always inspect the scatterplot before interpreting correlation.
How an Outlier Can Increase Correlation
A point far from the main cluster that lies in the same direction as the existing trend can make the linear pattern appear stronger numerically.
Correlation may increase even if the main cluster itself is only weakly associated.
How an Outlier Can Decrease Correlation
A point far from the main linear trend can pull the covariance in the opposite direction and reduce the magnitude of \(r\).
Such observations should be investigated rather than removed automatically.
Restricted Range Can Reduce Correlation
Correlation depends partly on the range of observed values.
A relationship that is strong across a broad population can appear weaker if the sample includes only a narrow range of \(x\)-values.
Combining Groups Can Change Correlation
Separate subgroups may each have one pattern while the combined dataset shows a different pattern.
Always inspect clusters and relevant categorical variables before relying on one overall correlation.
Correlation and Causation
Correlation measures association.
It does not establish causation.
A large value of
\[ |r| \]cannot by itself determine why the variables are associated.
Why High Correlation May Be Noncausal
High correlation can result from:
- a lurking variable;
- common trends over time;
- selection effects;
- reverse causation;
- confounding;
- chance.
Example: Common Cause
Ice-cream sales and electricity use for air conditioning may be positively correlated.
High temperature can increase both.
The correlation does not imply that purchasing ice cream causes greater air conditioning use.
Correlation Requires Quantitative Variables
Pearson correlation is designed for two quantitative variables.
Arbitrarily coding nominal categories as numbers does not make correlation meaningful.
Example: Invalid Category Coding
Suppose departments are coded
\[ 1=\text{Finance},\quad 2=\text{Marketing},\quad 3=\text{Operations}. \]Computing a Pearson correlation between this code and salary would treat category differences as numerical distances.
That interpretation is not justified by the coding.
Computational Formula for Covariance
Population covariance can also be expressed as
\[ \boxed{ \operatorname{Cov}(X,Y) = E(XY)-E(X)E(Y) }. \]This identity is especially useful in probability and theoretical calculations.
Sample Covariance Computational Form
The sample covariance numerator can be expanded:
\[ \sum_{i=1}^{n} (x_i-\bar{x})(y_i-\bar{y}) = \sum_{i=1}^{n}x_i y_i - n\bar{x}\bar{y}. \]Therefore,
\[ \boxed{ s_{xy} = \frac{ \sum x_i y_i-n\bar{x}\bar{y} }{n-1} }. \]Example: Full Sample Covariance Calculation
Consider
\[ (1,2),\quad(2,3),\quad(3,7). \]The means are
\[ \bar{x}=2, \qquad \bar{y}=4. \]Paired products of deviations:
\[ (1-2)(2-4)=2, \] \[ (2-2)(3-4)=0, \] \[ (3-2)(7-4)=3. \]Their sum is
\[ 2+0+3=5. \]Therefore,
\[ s_{xy} = \frac{5}{3-1} = \boxed{2.5}. \]Example: Full Correlation Calculation
Using the same data,
\[ x: 1,\ 2,\ 3, \] \[ y: 2,\ 3,\ 7, \]we have
\[ \bar{x}=2, \qquad \bar{y}=4. \]For \(x\),
\[ \sum(x_i-\bar{x})^2 = 1+0+1 = 2. \]For \(y\),
\[ \sum(y_i-\bar{y})^2 = 4+1+9 = 14. \]The paired-deviation sum is \(5\), so
\[ r = \frac{5}{\sqrt{2}\sqrt{14}} = \frac{5}{\sqrt{28}} \approx \boxed{0.945}. \]The sample shows a strong positive linear association.
Correlation and Scatterplot Shape
| Scatterplot Pattern | Typical Correlation Interpretation |
|---|---|
| Tight upward line | \(r\) near \(1\) |
| Tight downward line | \(r\) near \(-1\) |
| Diffuse cloud with little linear trend | \(r\) near \(0\) |
| Strong curve | \(r\) may be misleading |
Correlation Is Not a Complete Description
Two datasets can have the same correlation but different:
- slopes;
- centers;
- spreads;
- clusters;
- outliers;
- nonlinear features.
Numerical correlation should never replace graphical inspection.
Common Error: Interpreting Correlation as Slope
Correlation is unit-free and symmetric.
Slope has units and distinguishes explanatory from response variables.
They answer different questions.
Common Error: Saying \(r=0\) Means Independence
A zero correlation means no net linear association.
It does not generally imply statistical independence.
Strong nonlinear dependence can exist with zero correlation.
Common Error: Treating \(r=0.8\) as \(80\%\)
Correlation is not a percentage.
The value
\[ r=0.8 \]describes positive linear association on the scale from \(-1\) to \(1\).
Common Error: Correlating Arbitrary Codes
Numerical labels for nominal categories do not create meaningful numerical distances.
Pearson correlation should not be applied merely because category labels happen to be stored as numbers.
Common Error: Ignoring Nonlinearity
A small Pearson correlation can coexist with a strong curved relationship.
Inspect the scatterplot before concluding that association is weak.
Common Error: Ignoring Outliers
Correlation can be substantially changed by a single influential point.
Always examine unusual observations and, when appropriate, compare the analysis with and without them while explaining why.
Common Error: Inferring Causation
Even a correlation extremely close to \(1\) or \(-1\) does not by itself establish causality.
Causal conclusions require appropriate study design and substantive reasoning.
A Reliable Procedure for Covariance
- Preserve the paired observations.
- Compute the mean of each variable.
- Compute each \(x\)-deviation and \(y\)-deviation.
- Multiply the paired deviations.
- Sum those products.
- Divide by \(N\) for population covariance or \(n-1\) for conventional sample covariance.
- Interpret the sign in context.
A Reliable Procedure for Correlation
- Confirm that both variables are quantitative.
- Inspect a scatterplot for direction, form, clusters, and outliers.
- Compute covariance or the paired-deviation numerator.
- Compute the standard deviation of each variable.
- Standardize covariance by the product of the standard deviations.
- Verify that \(-1\le r\le1\).
- Interpret sign as direction and magnitude as linear strength.
- Do not infer causation from correlation alone.
Practice 1: Sign of a Product
Suppose
\[ x_i-\bar{x}=3 \]and
\[ y_i-\bar{y}=5. \]Is their paired product positive or negative?
Practice 2: Opposite Deviations
Suppose
\[ x_i-\bar{x}=-2, \qquad y_i-\bar{y}=7. \]Find the paired product and interpret its directional contribution.
Practice 3: Sample Covariance
The sum of paired deviation products is \(24\) for a sample of \(n=5\).
Find the sample covariance.
Practice 4: Correlation from Covariance
Suppose
\[ s_{xy}=12, \qquad s_x=3, \qquad s_y=8. \]Find \(r\).
Practice 5: Interpret Correlation
Interpret
\[ r=-0.91. \]Practice 6: Correlation Near Zero
A dataset has
\[ r=0.03. \]What can we safely conclude?
Practice 7: Unit Conversion
Height and weight have correlation \(0.72\). Heights are converted from meters to centimeters.
What happens to the correlation?
Practice 8: Negative Rescaling
Two variables have correlation
\[ r=0.65. \]If every value of \(x\) is multiplied by \(-1\), what is the new correlation?
Practice 9: Nonlinear Relationship
A scatterplot shows a very tight U-shaped relationship but has
\[ r\approx0. \]Is this contradictory?
Practice 10: Correlation and Causation
A study finds
\[ r=0.88 \]between two observational variables.
Does this prove that one variable causes the other?
Practice Answers
Practice 1
\[ 3(5)=15>0. \]The product is positive and supports positive association.
Practice 2
\[ (-2)(7)=\boxed{-14}. \]The negative product supports negative association because the variables lie on opposite sides of their means for this case.
Practice 3
\[ s_{xy} = \frac{24}{5-1} = \boxed{6}. \]Practice 4
\[ r = \frac{12}{3(8)} = \frac{12}{24} = \boxed{0.5}. \]Practice 5
The variables have a strong negative linear association: larger values of one tend to occur with smaller values of the other.
Practice 6
There is very little linear association. We cannot conclude that no relationship exists because a nonlinear relationship may still be present.
Practice 7
The correlation remains
\[ \boxed{0.72}. \]Positive unit conversions do not change correlation.
Practice 8
\[ \boxed{-0.65}. \]Multiplying one variable by a negative constant reverses the sign of the correlation.
Practice 9
No. Pearson correlation measures linear association. A strong U-shaped relationship can have correlation near zero.
Practice 10
No. A high observational correlation establishes strong association, not causation.
What to Remember
- Covariance measures the average product of paired deviations from the variable means.
- Positive covariance indicates same-direction co-movement; negative covariance indicates opposite-direction co-movement.
- Covariance depends on measurement units.
- Correlation standardizes covariance by the standard deviations of both variables.
- Pearson correlation always lies between \(-1\) and \(1\).
- The sign of \(r\) gives the direction of linear association.
- The magnitude \(|r|\) describes linear strength.
- Correlation is unit-free and unchanged by positive shifts and rescalings.
- Correlation can be near zero even when a strong nonlinear relationship exists.
- Correlation is sensitive to outliers and restricted ranges.
- Correlation does not establish causation.
Where We Go Next
Mean, standard deviation, covariance, and correlation are powerful summaries, but some datasets contain skewness or extreme observations that make conventional summaries unstable.
The next lesson, Robust Summaries, develops descriptive measures and strategies designed to remain informative when distributions contain outliers, heavy tails, or strong skewness.