The Central Idea
A statistic is called robust or resistant when a small number of extreme observations do not change it dramatically.
Robust summaries are especially useful when data are:
- strongly skewed;
- heavy-tailed;
- contaminated by a few unusual values;
- subject to measurement or recording anomalies.
Why Robustness Matters
Consider
\[ 10,\ 11,\ 11,\ 12,\ 13. \]The mean is
\[ \bar{x} = \frac{57}{5} = 11.4. \]Replace \(13\) with \(100\):
\[ 10,\ 11,\ 11,\ 12,\ 100. \]The new mean is
\[ \bar{x} = \frac{144}{5} = 28.8. \]One extreme observation changed the mean dramatically.
The Median Is Resistant
For both datasets
\[ 10,\ 11,\ 11,\ 12,\ 13 \]and
\[ 10,\ 11,\ 11,\ 12,\ 100, \]the median is
\[ \boxed{11}. \]The median depends primarily on order rather than magnitude.
Mean Versus Median
| Property | Mean | Median |
|---|---|---|
| Uses all numerical values directly | Yes | No |
| Sensitive to extreme values | Yes | Relatively resistant |
| Natural for symmetric distributions | Often | Also usable |
| Natural for strongly skewed distributions | Sometimes misleading | Often preferable |
The Interquartile Range Is Resistant
The interquartile range is
\[ \boxed{ \operatorname{IQR}=Q_3-Q_1 }. \]Because it measures the width of the middle \(50\%\) of the data, it is much less affected by extreme minimum and maximum values than the range.
Range Versus IQR
The range uses only the two extremes:
\[ \text{range} = \max(x)-\min(x). \]The IQR uses central quartiles:
\[ \operatorname{IQR}=Q_3-Q_1. \]Therefore, the range is highly sensitive to outliers, while the IQR is relatively resistant.
Median and IQR Form a Robust Pair
For skewed distributions or data with extreme observations, a common descriptive pair is
\[ \boxed{ \operatorname{median} \quad\text{and}\quad \operatorname{IQR} }. \]The median describes resistant center, and the IQR describes resistant spread.
What Is a Trimmed Mean?
A trimmed mean removes a specified percentage of the smallest and largest observations before computing the arithmetic mean.
The goal is to retain some averaging behavior while reducing the influence of extreme tails.
Example: One-Value Trimmed Mean
Consider
\[ 2,\ 4,\ 5,\ 6,\ 7,\ 9,\ 100. \]Remove the smallest and largest values:
\[ 4,\ 5,\ 6,\ 7,\ 9. \]The trimmed mean is
\[ \frac{4+5+6+7+9}{5} = \frac{31}{5} = \boxed{6.2}. \]Compare with the Ordinary Mean
For the full dataset,
\[ 2,\ 4,\ 5,\ 6,\ 7,\ 9,\ 100, \]the ordinary mean is
\[ \frac{133}{7} = 19. \]The extreme value \(100\) pulls the ordinary mean far upward, while the trimmed mean remains closer to the central cluster.
Percentage Trimming
A \(10\%\) trimmed mean removes approximately the lowest \(10\%\) and highest \(10\%\) of ordered observations before averaging the remainder.
A \(20\%\) trimmed mean removes approximately \(20\%\) from each tail.
Exact implementation depends on sample size and the convention used.
Trimmed Means Balance Efficiency and Resistance
The ordinary mean uses every value but can be highly sensitive.
The median is highly resistant but does not use the numerical magnitudes of most observations directly.
A trimmed mean occupies a middle position by using most observations while reducing tail influence.
Winsorized Data
Winsorization limits the influence of extreme values by replacing tail observations with less extreme boundary values rather than deleting them.
For example, instead of removing an extremely large value, a chosen upper tail value may replace it.
Trimmed Versus Winsorized Summaries
Trimming removes designated tail observations from the calculation.
Winsorization keeps the same number of observations but caps the most extreme values at chosen cut points.
Both methods reduce the influence of extreme tails.
Median Absolute Deviation
A robust measure of spread is the median absolute deviation, abbreviated MAD.
First find the median \(m\):
\[ m=\operatorname{median}(x_1,\ldots,x_n). \]Then compute the absolute deviations
\[ |x_i-m|. \]The MAD is the median of those absolute deviations:
\[ \boxed{ \operatorname{MAD} = \operatorname{median} \left( |x_i-m| \right) }. \]Example: Median Absolute Deviation
Consider
\[ 2,\ 4,\ 5,\ 7,\ 9. \]The median is
\[ m=5. \]Absolute deviations from the median are
\[ 3,\ 1,\ 0,\ 2,\ 4. \]Ordered:
\[ 0,\ 1,\ 2,\ 3,\ 4. \]Therefore,
\[ \boxed{\operatorname{MAD}=2}. \]Why Absolute Deviations?
Signed deviations from a center can cancel.
Absolute deviations avoid cancellation:
\[ |x_i-m|\ge0. \]Using the median of these distances preserves robustness.
MAD Versus Standard Deviation
| Property | Standard Deviation | MAD |
|---|---|---|
| Center used | Mean | Median |
| Distance type | Squared deviations | Absolute deviations |
| Outlier sensitivity | High | Low |
| Units | Original units | Original units |
Scaled MAD
In some applications, MAD is multiplied by a constant so that it is comparable to standard deviation under a normal model.
A common scaled form is approximately
\[ \boxed{ 1.4826\cdot\operatorname{MAD} }. \]This adjustment is model-based and should not be confused with the raw MAD itself.
Robust Standardized Scores
Ordinary z-scores use the mean and standard deviation:
\[ z=\frac{x-\bar{x}}{s}. \]A robust alternative can replace these with the median and a robust scale such as MAD.
One general form is
\[ \boxed{ z_{\text{robust}} = \frac{x-\operatorname{median}} {\text{robust scale}} }. \]Modified Z-Scores
One commonly used modified z-score is
\[ \boxed{ M_i = 0.6745 \frac{x_i-\operatorname{median}(x)} {\operatorname{MAD}} }. \]The constant \(0.6745\) calibrates the scale for comparison with ordinary z-scores under a normal model.
Why Robust Standardization Helps
If an extreme value distorts both the mean and standard deviation, an ordinary z-score can make that same outlier appear less extreme than expected.
Median- and MAD-based standardization reduces this masking effect.
Masking
Masking occurs when extreme observations inflate a conventional spread estimate enough that they appear less unusual under that same measure.
Robust summaries can make such cases easier to detect.
Swamping
Swamping occurs when ordinary observations are incorrectly flagged as unusual because other extreme observations distort the center or spread estimate.
Robust methods can reduce this problem as well.
Robustness Does Not Mean Ignoring Data
Robust methods do not assume that outliers are errors.
They reduce the ability of a small number of values to dominate the summary.
The unusual observations still remain part of the dataset and should be investigated separately.
Breakdown Point
A useful advanced idea is the breakdown point.
Informally, it describes how much contamination a statistic can tolerate before becoming arbitrarily distorted.
The sample median has a much higher breakdown point than the sample mean, which helps explain its strong resistance to extreme values.
Influence of a Single Observation
Changing one observation can affect statistics differently.
The mean reacts directly to the changed numerical magnitude.
The median often changes little unless the observation crosses important order positions.
This difference is central to robust statistics.
Robust Summaries for Skewed Data
Strong skewness is not necessarily a data error.
Many real-world variables are naturally skewed, including:
- income;
- wealth;
- waiting time;
- claim size;
- transaction amount.
Median and IQR often describe such distributions more effectively than mean and standard deviation.
Example: Household Income
Suppose most households earn between \(\$40{,}000\) and \(\$120{,}000\), but a small number earn several million dollars.
The mean may be substantially above the income of a typical household.
Median income and IQR provide a resistant description of central location and spread.
Example: Manufacturing Measurements
Suppose a manufacturing process normally produces measurements tightly centered near a target, but a few sensor failures generate extreme values.
The ordinary mean and standard deviation may be distorted.
A median, IQR, or MAD can help summarize the main body of the process while the suspicious observations are investigated.
Robust Summaries and Data Quality
Robust methods reduce sensitivity to bad data, but they do not replace data quality checks.
Analysts should still investigate:
- data-entry errors;
- unit mismatches;
- duplicate records;
- sensor failures;
- missing-value codes;
- unexpected subgroup structure.
Do Not Delete Outliers Automatically
An outlier can be:
- an error;
- a rare but valid observation;
- evidence of a new subgroup;
- evidence that the assumed model is inadequate.
Outlier treatment should be based on evidence and study design, not on convenience.
Sensitivity Analysis
A useful strategy is to compare conclusions under several reasonable summaries.
For example, report:
- mean and standard deviation;
- median and IQR;
- a trimmed mean;
- results with and without a clearly documented unusual case.
If conclusions change substantially, the analysis is sensitive to the unusual observations.
Robustness Versus Efficiency
Under an ideal symmetric model with no contamination, the arithmetic mean can use the available numerical information very efficiently.
Robust methods trade some model-specific efficiency for greater protection against skewness, heavy tails, and outliers.
No Summary Is Best in Every Setting
A robust statistic is not automatically superior simply because it is resistant.
The choice depends on:
- distribution shape;
- presence of outliers;
- measurement process;
- purpose of the analysis;
- whether arithmetic totals or expectations matter.
When the Mean Still Matters
The mean is directly connected to totals:
\[ \text{total}=n\bar{x}. \]For budgeting, expected cost, total resource use, or aggregate accounting, this relationship may make the mean operationally essential even when the distribution is skewed.
When the Median Is More Representative
If the goal is to describe a typical ordered position in a skewed distribution, the median may be more representative than the mean.
This is common in discussions of:
- home prices;
- income;
- wealth;
- waiting times;
- transaction sizes.
Robust Summaries for Bivariate Data
Covariance and Pearson correlation are both sensitive to outliers.
When bivariate data contain extreme points, analysts should inspect the scatterplot and consider robust alternatives or subgroup analysis.
The conceptual principle remains the same: do not allow a small number of unusual cases to define the entire relationship without scrutiny.
Robustness and Transformations
Sometimes a transformation can reduce skewness before a conventional summary is used.
For positive right-skewed data, analysts may consider transformations such as logarithms.
Transformation changes the measurement scale, so results must be interpreted accordingly.
Robust Summaries and Reporting
A good report should explain why a robust statistic was chosen.
For example:
Because the distribution is strongly right-skewed and contains several extreme high values, the median and IQR are reported as the primary summaries of center and spread.
Common Error: Calling the Median an Average Without Clarification
In informal language, the word "average" may refer to several different measures.
In statistical reporting, state mean or median explicitly.
Common Error: Trimming Without a Rule
Removing observations because they are inconvenient is not robust statistics.
Trimming should use a stated, reproducible rule such as a fixed percentage from each tail.
Common Error: Removing Only the High Tail
A conventional symmetric trimmed mean removes comparable proportions from both tails.
Removing only high values changes the target of the summary and may introduce bias unless the asymmetry is justified by the study design.
Common Error: Confusing MAD Definitions
The robust median absolute deviation is based on deviations from the median:
\[ \operatorname{median}\left(|x_i-\operatorname{median}(x)|\right). \]Do not confuse it with the mean absolute deviation, which averages absolute deviations.
Common Error: Treating Robust Statistics as Outlier Removal
Robust statistics reduce influence.
They do not necessarily remove observations.
The original data should remain available for inspection and interpretation.
Common Error: Assuming Robust Means Unbiased
Robustness concerns sensitivity to contamination or extreme observations.
Bias concerns systematic departure from a target quantity.
A statistic can be robust in one sense without being unbiased for every parameter or model.
A Reliable Procedure for Choosing Robust Summaries
- Inspect the distribution graphically.
- Identify skewness, heavy tails, and possible outliers.
- Investigate data-quality concerns.
- Decide whether the analytical goal is typical position, total impact, or model-based estimation.
- Use median and IQR when resistant location and spread are appropriate.
- Consider trimmed means when a compromise between mean and median is useful.
- Consider MAD when a resistant scale measure is needed.
- Document any trimming, Winsorization, or transformation rule.
- Compare robust and conventional summaries when sensitivity matters.
Practice 1: Mean Versus Median
Consider
\[ 4,\ 5,\ 5,\ 6,\ 100. \]Which measure of center is more resistant to the value \(100\)?
Practice 2: IQR
Suppose
\[ Q_1=12,\qquad Q_3=29. \]Find the IQR.
Practice 3: One-Value Trimmed Mean
Remove the smallest and largest values from
\[ 1,\ 4,\ 5,\ 6,\ 8,\ 30 \]and find the mean of the remaining observations.
Practice 4: MAD
Find the median absolute deviation of
\[ 2,\ 4,\ 6,\ 8,\ 10. \]Practice 5: Choose the Summary Pair
A distribution is strongly right-skewed and contains several extreme high values.
Which pair is generally more robust: mean and standard deviation, or median and IQR?
Practice 6: Outlier Handling
A single observation is far beyond the rest of the data.
Should it automatically be deleted?
Practice 7: Trimmed Mean Rule
Why should trimming use a stated percentage or reproducible rule rather than removing values subjectively?
Practice 8: Scaled MAD
Suppose
\[ \operatorname{MAD}=4. \]Approximate the normal-calibrated scaled MAD using
\[ 1.4826\cdot\operatorname{MAD}. \]Practice 9: Robust Standardization
A distribution has median \(50\) and robust scale \(10\).
Using
\[ z_{\text{robust}} = \frac{x-\operatorname{median}}{\text{robust scale}}, \]find the robust standardized score of \(x=75\).
Practice 10: Sensitivity Analysis
The mean is \(40\), the median is \(18\), and the distribution contains a small number of extremely large observations.
What does the large difference between mean and median suggest?
Practice Answers
Practice 1
The median is more resistant to the extreme value \(100\).
Practice 2
\[ \operatorname{IQR} = 29-12 = \boxed{17}. \]Practice 3
Remove \(1\) and \(30\):
\[ 4,\ 5,\ 6,\ 8. \]Then
\[ \frac{4+5+6+8}{4} = \frac{23}{4} = \boxed{5.75}. \]Practice 4
The median is
\[ m=6. \]Absolute deviations are
\[ 4,\ 2,\ 0,\ 2,\ 4. \]Ordered:
\[ 0,\ 2,\ 2,\ 4,\ 4. \]Therefore,
\[ \boxed{\operatorname{MAD}=2}. \]Practice 5
The median and IQR are generally more robust.
Practice 6
No. The observation should first be investigated to determine whether it is an error, a valid extreme case, or evidence of another process or subgroup.
Practice 7
A stated rule makes the analysis reproducible and prevents selective removal of observations based on the desired result.
Practice 8
\[ 1.4826(4) = \boxed{5.9304}. \]Rounded to two decimal places:
\[ \boxed{5.93}. \]Practice 9
\[ z_{\text{robust}} = \frac{75-50}{10} = \boxed{2.5}. \]Practice 10
The much larger mean suggests strong right skewness or extreme high values pulling the arithmetic average upward.
What to Remember
- Robust statistics resist distortion from a small number of extreme observations.
- The median is more resistant than the mean.
- The IQR is more resistant than the range.
- Median and IQR form a natural robust pair for center and spread.
- Trimmed means reduce tail influence while retaining arithmetic averaging.
- Winsorization caps extreme tail values rather than removing them.
- MAD is the median of absolute deviations from the median.
- Robust standardized scores can use median and MAD instead of mean and standard deviation.
- Robust methods do not justify automatic deletion of outliers.
- Sensitivity analysis compares conclusions under multiple reasonable summaries.
- No single summary is best for every analytical objective.
Where We Go Next
Robust summaries help protect an analysis from distortion, but even a mathematically correct summary can mislead if the underlying data, graph, sampling process, or presentation is poor.
The next lesson, Misleading Displays and Data Quality, examines distorted axes, selective scales, missing data, measurement problems, sampling issues, and other threats to reliable statistical interpretation.