The Central Idea

Statistical inference requires assumptions.

The central question is not whether a method has assumptions, but which assumptions it uses and how sensitive the results are when those assumptions are imperfect.

Parametric methods usually specify a probability model using a finite number of parameters.

Nonparametric methods typically make fewer or weaker assumptions about the exact shape of the population distribution.

What Parametric Means

A parametric model assumes that the population distribution belongs to a family described by a finite set of parameters.

For example, a normal distribution is described by

\[ \mu \]

and

\[ \sigma^2. \]

Once these parameters are specified, the model's distributional form is determined.

Examples of Parametric Models

Common parametric families include:

What Nonparametric Means

A nonparametric method does not require the population to belong to one particular finite-dimensional distributional family.

This does not mean the method has no assumptions.

Nonparametric methods can still require:

Nonparametric Does Not Mean Assumption-Free

Every inferential procedure depends on some structure.

A rank-based test may avoid a normality assumption but still require independent observations.

A permutation test may avoid a parametric distribution but require exchangeability under the null hypothesis.

The correct question is: Are the assumptions appropriate for the data and the inferential goal?

Why Use Parametric Methods?

Parametric methods can be attractive because they often:

Why Use Nonparametric Methods?

Nonparametric methods can be useful when:

Efficiency

Efficiency compares how much information different estimators or tests extract from the same data under a specified model.

A parametric method tailored to the correct distribution can have smaller standard error or greater power than a more general alternative.

Robustness

Robustness describes resistance to departures from modeling assumptions or to unusual observations.

A method can be efficient under ideal conditions but fragile when those conditions fail.

Another method may sacrifice some ideal-model efficiency in exchange for better performance under contamination or skewness.

The Mean and Median Illustrate the Tradeoff

The sample mean uses the full numerical magnitude of every observation.

The sample median depends primarily on order.

Under a well-behaved normal model, the mean is highly efficient for the population mean.

Under strong outliers or heavy tails, the median can be much more stable.

Parametric Methods Often Target Means

Many familiar parametric procedures focus on means or regression coefficients.

Examples include:

Nonparametric Methods Often Use Ranks or Signs

Many nonparametric procedures replace raw magnitudes with:

This can reduce sensitivity to extreme observations.

Ranks

To rank data, sort observations from smallest to largest and assign ranks.

For

\[ 4,\;7,\;10,\;15, \]

the ranks are

\[ 1,\;2,\;3,\;4. \]

Rank methods use relative order rather than the original measurement scale.

Tied Ranks

If observations are tied, a common convention assigns them the average of the ranks they would have occupied.

For example, if two observations tie for ranks \(3\) and \(4\), each is assigned

\[ \frac{3+4}{2}=3.5. \]

Sign Tests

A sign test uses only whether observations fall above or below a hypothesized value, or whether paired differences are positive or negative.

Magnitudes are ignored.

This makes the sign test highly robust but can reduce efficiency when magnitude information is reliable.

One-Sample Sign Test

Suppose the population median is hypothesized to be

\[ m_0. \]

For each observation, record whether

\[ X_i>m_0 \]

or

\[ X_i Under a continuous distribution with median \(m_0\), positive and negative signs each have probability

\[ \frac12. \]

Exact Sign-Test Distribution

If ties are removed and \(B\) is the number of observations above the null median, then under the null,

\[ \boxed{ B\sim\operatorname{Binomial}\left(n,\frac12\right) }. \]

Exact binomial probabilities can therefore be used.

Paired Sign Test

For paired data, compute differences

\[ D_i=X_i-Y_i. \]

The paired sign test records only whether each nonzero difference is positive or negative.

It is useful when the symmetry assumptions required by the signed-rank test are doubtful.

Wilcoxon Signed-Rank Test

The Wilcoxon signed-rank test is a rank-based method for one-sample or paired location problems.

It uses both:

Signed-Rank Workflow

  1. Compute the differences \(D_i\).
  2. Remove zero differences.
  3. Rank the absolute values \(|D_i|\).
  4. Attach the original signs to the ranks.
  5. Sum positive and negative ranks.
  6. Compare the resulting statistic with its null distribution.

Signed-Rank Assumptions

The Wilcoxon signed-rank test is not entirely assumption-free.

A common interpretation as a test of a location shift relies on the distribution of differences being approximately symmetric.

If symmetry is implausible, the simpler sign test may be more defensible.

Mann-Whitney U Test

The Mann-Whitney U test, also called the Wilcoxon rank-sum test, compares two independent groups using ranks.

It is often used as a nonparametric alternative to the independent two-sample \(t\) test.

Rank-Sum Idea

Combine observations from both groups and rank them together.

If one group tends to contain larger values, it will tend to receive larger ranks.

The rank sums therefore measure separation between the two distributions.

What the Mann-Whitney Test Actually Tests

The Mann-Whitney test is often described informally as a test of medians.

That interpretation requires additional assumptions, such as similar distributional shapes with a location shift.

More generally, the procedure tests a distributional ordering or difference associated with pairwise ranks.

Mann-Whitney and Probability of Superiority

One useful interpretation is based on

\[ P(X>Y). \]

Under identical continuous distributions,

\[ P(X>Y)=0.5. \]

Rank-based statistics are closely connected to whether observations from one group tend to exceed observations from another.

Kruskal-Wallis Test

The Kruskal-Wallis test extends rank-based comparison to three or more independent groups.

It is often presented as a nonparametric analogue of one-way ANOVA.

The test compares group rank patterns rather than raw group means.

Kruskal-Wallis Null Hypothesis

In its broad form, the null hypothesis is that the groups come from the same distribution.

Interpreting rejection specifically as a difference in medians requires suitable shape assumptions.

Friedman Test

The Friedman test is a rank-based method for comparing three or more related conditions or repeated measurements.

It is often viewed as a nonparametric analogue of repeated-measures one-way ANOVA.

Spearman Rank Correlation

Spearman's rank correlation measures monotonic association using ranks.

It is commonly denoted

\[ r_s \]

in a sample.

It can detect strong monotonic relationships that are not well summarized by Pearson's linear correlation.

Pearson versus Spearman Correlation

Pearson correlation measures linear association on the original numerical scale.

Spearman correlation measures monotonic association through ranks.

Spearman's method is generally less sensitive to extreme magnitudes because it uses order rather than raw values.

Kendall's Tau

Kendall's tau is another rank-based measure of association.

It is based on concordant and discordant pairs.

It has a direct interpretation in terms of pair ordering and can be useful for ordinal or nonnormally distributed data.

Permutation Tests

A permutation test constructs a null distribution by repeatedly rearranging labels in a way justified under the null hypothesis.

It does not require choosing a parametric distribution such as normal or Student's \(t\).

Exchangeability

Permutation methods rely on exchangeability.

Under the null hypothesis, the labels being permuted must be interchangeable in the manner assumed by the procedure.

If the data structure does not justify that exchangeability, the permutation test is invalid.

Randomization Tests

In a randomized experiment, a randomization test can use the actual treatment-assignment mechanism to generate a null distribution.

This ties the test directly to the experimental design.

Exact versus Monte Carlo Permutation Tests

If every allowable permutation is enumerated, the test is exact relative to the randomization structure.

When there are too many permutations, a large random subset can be sampled.

This produces a Monte Carlo approximation to the permutation p-value.

Bootstrap Methods

The bootstrap is another distribution-light computational method.

It repeatedly resamples from the observed sample with replacement to approximate the sampling behavior of a statistic.

Bootstrap methods are often used for standard errors and confidence intervals rather than as direct substitutes for every classical test.

Parametric Bootstrap

A parametric bootstrap simulates data from a fitted parametric model.

For example, if a normal model is fitted using estimated

\[ \mu \]

and

\[ \sigma, \]

repeated synthetic samples can be generated from that fitted normal distribution.

Nonparametric Bootstrap

A nonparametric bootstrap resamples the observed data directly with replacement.

It treats the empirical distribution as an approximation to the population distribution.

Chi-Square Methods

Chi-square procedures for categorical data are often grouped with nonparametric methods because they do not require a normal population model for the observed categories.

However, they still rely on assumptions about counts, expected frequencies, independence, and sampling structure.

Chi-Square Goodness-of-Fit Test

The chi-square goodness-of-fit statistic is

\[ \boxed{ \chi^2 = \sum \frac{ (O-E)^2 }{ E } }. \]

It compares observed counts \(O\) with expected counts \(E\) under a specified categorical model.

Chi-Square Test of Independence

In a contingency table, the chi-square test of independence compares observed cell counts with counts expected if the two categorical variables were independent.

Large discrepancies produce a large chi-square statistic.

Fisher's Exact Test

Fisher's exact test provides an exact test for association in certain contingency-table settings, especially \(2\times2\) tables.

It can be useful when expected cell counts are too small for a chi-square approximation.

Parametric Assumptions Should Be Stated Precisely

Saying “the data must be normal” is often too vague.

In a one-sample \(t\) procedure, the relevant assumption concerns the population distribution of observations or, for large samples, the sampling distribution of the mean.

In regression, the relevant normality condition concerns model errors for exact small-sample inference, not necessarily the raw predictor or outcome distributions themselves.

Normality Is Not Always Critical

Means often have approximately normal sampling distributions for large samples because of the central limit theorem.

Therefore, moderate nonnormality may have little effect on large-sample mean-based inference.

Severe skewness, heavy tails, and extreme outliers can still matter.

Independence Is Often More Important than Normality

Many classical procedures are surprisingly robust to moderate departures from normality.

They are generally much less robust to unmodeled dependence.

Repeated measurements, clustering, serial correlation, and spatial dependence require explicit treatment.

Equal-Variance Assumptions

Some parametric procedures assume equal variances across groups.

For two independent means, the pooled \(t\) test makes this assumption.

Welch's \(t\) test does not require equal variances and is often a safer default.

Transformations

Sometimes a transformation can make a parametric model more appropriate.

Common transformations include:

\[ \log X, \] \[ \sqrt{X}, \]

and other monotone transformations.

The transformed parameter must still be interpreted carefully.

Log Transformations

Positive right-skewed data are often analyzed on a logarithmic scale.

A difference on the log scale corresponds to a ratio on the original scale.

This can make multiplicative effects easier to model.

Transformation Is Not the Same as Nonparametric Analysis

Transforming data and then applying a parametric model remains a parametric strategy.

A nonparametric strategy changes the inferential method itself rather than only changing the measurement scale.

Ordinal Data

Ordinal variables have meaningful order but not necessarily meaningful numerical spacing.

Rank-based methods can be natural for such data because they rely on order rather than exact distances between category values.

Outliers

Outliers can strongly influence means, variances, correlations, and least-squares regression coefficients.

Rank-based procedures often reduce this influence because an extreme value receives an extreme rank but does not preserve its full numerical magnitude.

Heavy-Tailed Distributions

Heavy-tailed populations generate more extreme observations than a normal model predicts.

In these settings, robust or rank-based procedures can sometimes provide more stable inference than classical mean-based methods.

Skewed Distributions

Strong skewness can make small-sample mean-based procedures unreliable.

Possible responses include:

Parametric Models Can Model Skewness Directly

Nonparametric methods are not the only response to nonnormal data.

A parametric model can use an appropriate skewed family such as:

The real question is whether the chosen model matches the data-generating process.

Nonparametric Methods Can Lose Information

Replacing measurements with ranks or signs discards some magnitude information.

If the parametric model is accurate, this can reduce power or precision.

Robustness and efficiency must be balanced.

Parametric Methods Can Be Fragile

A parametric procedure can be highly efficient under its assumed model but misleading if the model is badly wrong.

Diagnostics and sensitivity analysis are therefore important.

Graph the Data First

Before choosing a method, inspect the data.

Useful displays include:

Method selection should be informed by structure, not by habit.

Normal Quantile Plots

A normal quantile plot compares observed quantiles with theoretical normal quantiles.

Approximate linearity supports a normal model.

Strong curvature, extreme tail departures, or isolated points can signal skewness, heavy tails, or outliers.

Formal Normality Tests

Formal tests such as the Shapiro-Wilk test can assess normality.

However, they should not be used mechanically.

In large samples, tiny harmless departures can be statistically significant.

In small samples, serious nonnormality can be difficult to detect.

Method Choice Should Match the Parameter

A nonparametric test does not automatically test the same parameter as its familiar parametric counterpart.

For example, a two-sample \(t\) test focuses on population means.

A Mann-Whitney test has a broader distributional interpretation unless additional assumptions justify a location or median interpretation.

Do Not Choose a Test by Normality Alone

Method selection should consider:

One-Sample Method Comparison

For a one-sample location problem:

Two Independent Groups

For two independent groups:

Paired Data

For paired data:

Three or More Independent Groups

For three or more independent groups:

Association Methods

For two quantitative variables:

Power Considerations

When parametric assumptions are accurate, parametric methods often have greater power.

Under strong departures from those assumptions, robust or nonparametric procedures can outperform the classical parametric method in practical reliability.

Asymptotic Relative Efficiency

Asymptotic relative efficiency compares the large-sample efficiency of two procedures under a specified distribution.

It asks how much larger one sample would need to be to achieve performance comparable to another method.

The comparison depends on the underlying population model.

Exact Methods in Small Samples

Some nonparametric procedures have exact null distributions for small samples.

This can be valuable when asymptotic approximations are unreliable.

Exactness still depends on the assumptions defining the null randomization structure.

Large-Sample Approximations for Rank Tests

For large samples, rank statistics are often approximated by normal or chi-square distributions.

Tie corrections and continuity corrections may be used depending on the procedure.

Multiple Comparisons after Rank Tests

A significant Kruskal-Wallis or Friedman test indicates that not all groups behave identically.

It does not identify which groups differ.

Follow-up pairwise procedures require appropriate multiplicity adjustment.

Confidence Intervals Still Matter

Nonparametric analysis should not stop at a p-value.

When possible, report:

Effect Sizes for Rank Methods

Rank procedures can be paired with interpretable effect measures.

Examples include:

Hodges-Lehmann Estimation

The Hodges-Lehmann estimator provides a rank-based estimate of a location shift.

It is closely connected with Wilcoxon procedures and offers an effect estimate rather than only a significance decision.

Robust Parametric Methods

The choice is not always simply classical parametric versus rank-based nonparametric.

Robust parametric or semiparametric methods include:

Semiparametric Methods

Semiparametric methods combine parametric and nonparametric components.

They model part of the data structure through a finite-dimensional parameter while leaving other aspects of the distribution unspecified.

This middle ground is common in modern statistics.

Quantile Regression

Quantile regression models conditional quantiles such as the median rather than only the conditional mean.

It can reveal how predictor effects differ across the outcome distribution.

It is especially useful when mean-based models do not capture the full relationship.

When Parametric Methods Are a Strong Choice

Parametric methods are often appropriate when:

When Nonparametric Methods Are a Strong Choice

Nonparametric methods are often attractive when:

Method Selection Is Not a Competition

Parametric and nonparametric methods are tools for different circumstances.

Neither class is universally superior.

The strongest analysis uses the method whose assumptions, parameter target, and interpretation best match the data-generating process and research question.

Sensitivity Analysis across Methods

When reasonable methods differ in assumptions, comparing results can be informative.

For example, a study might compare:

Agreement across methods strengthens confidence that the conclusion is not driven by one fragile assumption.

Different Methods May Answer Different Questions

Sensitivity analysis must be interpreted carefully.

Two methods can produce different answers because they target different population features rather than because one is “wrong.”

Always identify the estimand or parameter each method addresses.

Common Error: Nonparametric Means No Assumptions

Nonparametric procedures still require assumptions about sampling, independence, exchangeability, symmetry, or distributional structure.

They simply avoid specifying one particular parametric family in the usual way.

Common Error: Use Nonparametric Whenever Data Are Not Normal

Moderate nonnormality does not automatically invalidate parametric procedures, especially with large samples.

The parameter of interest, sample size, skewness, outliers, and dependence all matter.

Common Error: Mann-Whitney Is Always a Test of Medians

The Mann-Whitney test is fundamentally rank-based.

Interpreting it purely as a median test requires additional assumptions about distributional shape and location shift.

Common Error: Rank Tests Ignore Sample Design

Rank transformations do not repair clustering, repeated measurements, or biased sampling.

The dependence and design structure must still be respected.

Common Error: Parametric Means Normal

Parametric methods are broader than normal-theory methods.

Binomial, Poisson, exponential, gamma, and many regression models are parametric without being normal.

Common Error: Nonparametric Means Median

Nonparametric methods can target many features:

The target must be identified procedure by procedure.

Common Error: A Significant Result Means the Methods Agree

A \(t\) test and a rank test can both reject a null while estimating different population features.

Similar decisions do not imply identical interpretation.

Common Error: Choosing the Method after Seeing Which Gives Significance

Trying multiple tests and reporting only the one with the smallest p-value creates a selective-analysis problem.

The main method should be selected from the research question and design, or multiple analyses should be reported transparently.

A Reliable Procedure for Choosing a Method

  1. Identify the population quantity or effect of interest.
  2. Identify the study design and dependence structure.
  3. Determine the measurement scale.
  4. Graph the data and inspect unusual observations.
  5. Assess the plausibility of relevant parametric assumptions.
  6. Consider sample size and approximation quality.
  7. Determine whether a rank, sign, permutation, or robust method better matches the question.
  8. Compare efficiency and robustness tradeoffs.
  9. Use design-appropriate standard errors or randomization schemes.
  10. Report effect sizes and uncertainty, not only p-values.
  11. Use sensitivity analyses when several methods are defensible.
  12. Document why the selected method fits the inferential goal.

Example: Choosing between t and Mann-Whitney

Two independent groups are compared on a quantitative outcome.

If the scientific question concerns the difference in population means and the data contain no severe outliers, Welch's \(t\) procedure may be appropriate even with moderate skewness when sample sizes are adequate.

If the data are highly skewed, extreme values dominate the mean, and the substantive question concerns relative ordering rather than mean difference, a Mann-Whitney analysis may be more appropriate.

The choice follows the target and assumptions—not a mechanical normality rule.

Example: Paired Data with Outliers

Suppose before-and-after differences contain a few very large outliers.

The paired \(t\) test may be sensitive because it uses the mean difference.

If the distribution of differences is roughly symmetric, a signed-rank test may offer a more robust alternative.

If even symmetry is doubtful, a sign test may be more defensible.

Example: Exact Permutation Logic

Suppose six experimental units are randomly assigned equally to two treatments.

Under a sharp null hypothesis of no treatment effect, treatment labels can be reassigned according to the original randomization mechanism.

The statistic is recalculated for every allowable assignment.

The proportion of assignments at least as extreme as the observed one gives the randomization p-value.

Practice 1: Parametric or Nonparametric?

A method assumes observations come from a normal distribution described by \(\mu\) and \(\sigma^2\).

Is the method parametric or nonparametric?

Practice 2: Assumption-Free?

Does “nonparametric” mean that a method has no assumptions?

Practice 3: Paired Rank Method

Which common rank-based test is designed for paired or one-sample location data when approximate symmetry of differences is reasonable?

Practice 4: Two Independent Groups

Which common nonparametric procedure compares two independent groups using ranks?

Practice 5: Three Independent Groups

Which rank-based method is commonly used to compare three or more independent groups?

Practice 6: Signs Only

Which procedure uses only whether observations or paired differences are positive or negative?

Practice 7: Spearman versus Pearson

Which correlation measure is based on ranks and monotonic association?

Practice 8: Permutation Assumption

What key structural idea justifies rearranging labels in a permutation test?

Practice 9: Outliers

Why can rank-based procedures be less sensitive to extreme observations than mean-based methods?

Practice 10: Identify the Error

A student says:

“The data failed a normality test, so the \(t\) test is automatically invalid.”

Explain why this conclusion is too strong.

Practice Answers

Practice 1

The method is parametric because it specifies a normal family using the finite parameter set

\[ \mu,\sigma^2. \]

Practice 2

No.

Nonparametric methods still rely on assumptions such as independence, exchangeability, continuity, or symmetry depending on the procedure.

Practice 3

The Wilcoxon signed-rank test.

Practice 4

The Mann-Whitney U test, also called the Wilcoxon rank-sum test.

Practice 5

The Kruskal-Wallis test.

Practice 6

The sign test.

Practice 7

Spearman's rank correlation.

Practice 8

The key idea is exchangeability under the null hypothesis or, in randomized experiments, the actual randomization mechanism.

Practice 9

Rank methods replace raw magnitudes with relative order.

An extreme observation may receive the largest rank, but its enormous numerical distance from the rest of the data is not retained.

Practice 10

Moderate nonnormality does not automatically invalidate a \(t\) procedure. The effect depends on sample size, skewness, outliers, dependence, and the sampling distribution of the mean.

In larger samples, mean-based inference can be robust because of the central limit theorem.

What to Remember

Parametric and nonparametric methods differ mainly in the structure of the assumptions they place on the population model.

The most important habit is: choose the method that matches the parameter and data-generating process, rather than treating “parametric” and “nonparametric” as automatic good-versus-bad categories.

Where We Go Next

Whether inference is parametric, rank-based, exact, or resampling-based, a statistically detectable result still must be interpreted in terms of its real-world magnitude.

The next lesson, Practical versus Statistical Significance, separates evidence that an effect exists from evidence that the effect is large enough to matter scientifically, economically, clinically, or operationally.