MGU Mathematics Series — Probability and Statistics
Part II: Multivariable Probability and Distribution Theory
LLN · CLT · Sampling Distributions · Standard Error · Normal Approximation
Chapter 7 of 14
MGU Mathematics Series  ·  Probability and Statistics  ·  Library Textbook Edition

Chapter 7:
Limit Theorems and
Sampling Distributions

The Law of Large Numbers, the Central Limit Theorem, and the distributional theory that makes statistical inference mathematically possible

■ Math Level: Calculus-Based Probability & Statistics ■

■ 14 Sections  •  Part II of 5 ■

Follows Chapter 6: Transformations and Moment Methods  ·  Prepares students for Chapter 8: Estimation and Confidence Intervals

Chapter 7 — Front Matter

Purpose and Overview

"Probability is not a mere computation of odds on the dice or more complicated variants; it is the acceptance of the lack of a sharp boundary between knowledge and ignorance."

— Nassim Nicholas Taleb

Every statistical method in this textbook rests on a foundation of probability results called limit theorems. These theorems describe what happens when chance experiments are repeated many times, when samples are drawn from populations, and when random quantities are averaged across many observations. Without them, statistics would be a collection of numerical recipes with no mathematical justification. With them, statistical procedures become logical consequences of probability theory.

Consider what happens when you flip a fair coin repeatedly. After ten flips, the proportion of heads might be anywhere from 0 to 1. But after a thousand flips, the proportion is almost certainly close to 0.5. After a million flips, it is extremely close to 0.5. This is not an accident — it is the Law of Large Numbers at work. Or consider what happens when you take the average of many independent measurements, each subject to random error. Even if the individual errors have unusual shapes or heavy tails, the distribution of their average gradually begins to look like a bell curve. This is the Central Limit Theorem, arguably the most important single result in all of probability and statistics.

This chapter is the pivot point of the textbook. The probability of Parts I and II becomes the statistical inference of Parts III, IV, and V. The goal is genuine mathematical understanding: why statistical inference works, and under what conditions we can trust it.

Opening Question

A pharmaceutical company tests a new drug on a sample of 200 patients and finds that 68% improve. Should they conclude that roughly 68% of all patients would improve? And how confident should they be? This is a mathematical question about the relationship between a sample and the population it came from. By the end of this chapter, you will have the tools to answer it precisely.

14Sections
9Objectives
15Key Terms
9+Worked Examples

Learning Objectives

  1. 1Explain why sample statistics are random variables and describe sampling distributions.
  2. 2Derive and apply the mean and variance of the sample mean and sample proportion.
  3. 3Define standard error and distinguish it from population standard deviation.
  4. 4Define convergence in probability and apply Chebyshev's Inequality.
  5. 5State and apply the Weak Law of Large Numbers, including its conditions and limitations.
  6. 6State and apply the Central Limit Theorem, including a heuristic MGF-based argument.
  7. 7Use the normal approximation to the binomial with and without continuity correction.
  8. 8Compute probabilities using the sampling distributions of the mean and proportion.
  9. 9Identify and avoid common errors: the Gambler's Fallacy, confusing SD with SE, and misapplying the CLT.
Chapter 7 — Contents

Sections at a Glance

Sections 7.1 – 7.2
The Sampling Problem: Why Long-Run Behavior Matters
Why Limit Theorems Matter · Repeated Sampling · Sample Statistics as Random Variables
Section 7.1

Why Limit Theorems Matter

Suppose a pharmaceutical company tests a new drug on 200 patients and finds that 68% improve. Should they conclude that roughly 68% of all patients would improve? And how confident should they be in that conclusion? This is not merely a question of intuition. It is a mathematical question about the relationship between a sample and the population from which it came.

Here is the central difficulty: every time you draw a sample, you get different data, and therefore different statistics. The variability of a statistic from sample to sample is not a flaw — it is an intrinsic feature of random sampling. The question is how to characterize that variability precisely. Limit theorems provide exactly that characterization.

Two Central ResultsWhat It SaysWhy It Matters
Law of Large NumbersSample means converge to the population mean as n growsJustifies using sample averages to estimate population parameters
Central Limit TheoremSampling distributions of means approach normal shapeJustifies using normal-distribution tools for inference in almost any setting
The Role of Sample Size

Larger samples produce statistics that vary less from sample to sample, are closer to the true population values, and have sampling distributions that are more normal in shape. This is not a vague principle — it is a precise mathematical relationship, and understanding it quantitatively is essential for designing studies and interpreting data responsibly.

Section 7.2

Repeated Sampling

To understand sampling distributions, think carefully about what "repeated sampling" means. Suppose a population has mean μ and standard deviation σ. Draw a random sample of n individuals, compute the sample mean X̄, and record it. Then draw a new sample of n, compute a new X̄, and repeat hundreds of times. Each repetition produces one value of the sample mean. After many repetitions, we have a collection of sample means — their distribution is the sampling distribution of X̄.

In practice, we rarely actually repeat a study many times. But we reason about what would happen if we did, and this reasoning is what justifies statistical inference.

Key Insight

Before data are collected, a sample statistic is a random variable. Its value depends on which individuals happen to be selected, and that selection involves chance. A sample mean X̄ is a random variable. A sample proportion p̂ is a random variable. Each has a probability distribution — its sampling distribution — that describes its variability across repeated samples.

Practice 7.1–7.2Concept Check
  1. Explain in your own words why a sample mean is a random variable. What makes it random?
  2. A coin is flipped 10 times and the proportion of heads is recorded. This experiment is repeated 1,000 times. Describe what the histogram of the 1,000 proportions represents.
  3. Why does the sampling distribution of X̄ become more concentrated around μ as sample size increases? Explain intuitively.
  4. List two ways in which understanding sampling distributions is essential for drawing conclusions from data.
Sections 7.3 – 7.5
The Sample Mean, Sample Proportion, and Standard Error
E[X̄] = μ · Var(X̄) = σ²/n · E[p̂] = p · Standard Error · Sample Size Planning
Section 7.3

Sample Mean as a Random Variable

Let X₁, X₂, …, Xₙ be a random sample of size n from a population with mean μ and variance σ². The phrase "random sample" means the observations are independent and identically distributed (i.i.d.) — each drawn from the same distribution, with no dependence between them.

X̄ = (1/n) Σᵢ Xᵢ = (X₁ + X₂ + ⋯ + Xₙ) / n
Fundamental Results for the Sample Mean

E[X̄] = μ   —  The expected value of the sample mean equals the population mean. On average, across all possible samples of size n, the sample mean correctly targets μ. This makes X̄ an unbiased estimator of μ.

Var(X̄) = σ²/n   —  The variance of the sample mean is the population variance divided by n. The standard deviation of X̄ is σ/√n, called the standard error of the mean.

The variance result captures mathematically the key intuition: larger samples produce less variable sample means. Specifically, doubling the sample size reduces the standard deviation of X̄ by a factor of √2 ≈ 1.41. To halve the standard error, you need four times the data. This square-root relationship is pervasive in statistics — it reflects the fact that random errors tend to cancel when averaged, but only at a 1/√n rate.

The Normal Population Case

If the population is normally distributed, Xᵢ ~ N(μ, σ²), then X̄ is exactly normally distributed: X̄ ~ N(μ, σ²/n) for every sample size n. For non-normal populations, the CLT (Section 7.8) gives an approximation that holds for large n.

Example 7.1
Mean and standard error of the sample mean
  • ProblemExam scores have μ = 72 and σ = 15. A random sample of n = 25 students is selected. Find E[X̄] and SE(X̄).
  • Solve E[X̄] = μ = 72 SE(X̄) = σ/√n = 15/√25 = 15/5 = 3
  • InterpretThe sample mean is expected to be 72, with a typical variation of about 3 points from sample to sample.
Answer: E[X̄] = 72, SE(X̄) = 3.
Example 7.2
Finding the required sample size
  • ProblemFor the same population (σ = 15), what sample size n reduces SE(X̄) to 1?
  • Solve 15/√n = 1 ⟹ √n = 15 ⟹ n = 225
  • InterpretA sample of 225 reduces the standard deviation of the sample mean to 1 point. Notice: to reduce SE from 3 to 1 (a factor of 3), the sample size must increase by a factor of 9.
Answer: n = 225.
Section 7.4

Sample Proportion as a Random Variable

Many statistical applications involve proportions: the fraction of voters favoring a candidate, the rate of defective items in production, the prevalence of disease in a population. Suppose the true population proportion with some characteristic is p. We draw a sample of n individuals and let X be the count with the characteristic, so X ~ Binomial(n, p). The sample proportion is p̂ = X/n.

Fundamental Results for the Sample Proportion

E[p̂] = p   —  The sample proportion is an unbiased estimator of p.

Var(p̂) = p(1−p)/n   —  Standard deviation: SD(p̂) = √(p(1−p)/n).

Connection to X̄: p̂ is a sample mean of Bernoulli(p) indicator variables. All theory for X̄ applies directly.

Example 7.3
Mean and standard deviation of the sample proportion
  • ProblemA population has true proportion p = 0.40. A random sample of n = 100 is drawn. Find E[p̂] and SE(p̂).
  • Solve E[p̂] = 0.40 SE(p̂) = √(0.40 × 0.60 / 100) = √(0.0024) ≈ 0.049
  • InterpretThe sample proportion will typically be within about 0.049 of the true proportion 0.40.
Answer: E[p̂] = 0.40, SE(p̂) ≈ 0.049.
Section 7.5

Standard Error

The standard error of a statistic is the standard deviation of its sampling distribution. It measures how much the statistic varies from sample to sample.

Standard Error of the Mean
SE(X̄) = σ / √n
Standard Error of the Proportion
SE(p̂) = √(p(1−p) / n)
Estimated SE of the Mean
SÊ(X̄) = S / √n
Estimated SE of the Proportion
SÊ(p̂) = √(p̂(1−p̂) / n)
Standard Deviation vs. Standard Error

The population standard deviation σ measures variability among individual observations. The standard error σ/√n measures variability among sample means — the typical gap between a sample mean and the true population mean. These are very different quantities. A population with σ = 20 has SE(X̄) = 20/√100 = 2 for n = 100. Confusing them leads to wildly incorrect probability calculations.

Example 7.4
Sample size planning
  • ProblemA researcher wants to estimate mean reaction time. Pilot data suggests σ ≈ 40 ms. What sample size is needed for SE ≤ 5 ms?
  • Solve σ/√n ≤ 5 40/√n ≤ 5 √n ≥ 8 n ≥ 64
Answer: At least n = 64 participants.
FactorEffect on SE(X̄)Why
Larger nDecreases SEMore averaging cancels random errors
Larger σIncreases SEMore variability in population carries through to samples
Smaller nIncreases SELess averaging, more sensitivity to extreme values
Practice 7.3–7.5Sample Mean, Proportion, and Standard Error
  1. Population: μ = 50, σ = 10. Find E[X̄] and SE(X̄) for n = 4, n = 25, n = 100.
  2. Population: p = 0.30. Find E[p̂] and SE(p̂) for n = 50, n = 100, n = 400.
  3. A study needs SE(X̄) ≤ 2. If σ = 12, find the minimum n.
  4. Show that p(1−p) is maximized at p = 0.5 using differentiation.
  5. Population: μ = 100, σ = 20, population is normal. Find P(X̄ > 105) for n = 16.
  6. If SE(X̄) = 4 with n = 25, what is σ? What is SE(X̄) with n = 100?
Sections 7.6 – 7.7
Convergence in Probability and the Law of Large Numbers
Convergence in Probability · Chebyshev's Inequality · WLLN · Gambler's Fallacy
Section 7.6

Convergence in Probability: An Introduction

The Law of Large Numbers is a statement about what happens as n → ∞. To state it precisely, we need a notion of what it means for a sequence of random variables to "converge." Let Y₁, Y₂, Y₃, … be a sequence of random variables — for example, Yₙ = X̄ₙ, the sample mean based on n observations.

Definition
Convergence in Probability

A sequence of random variables Yₙ converges in probability to a constant c, written Yₙ →ᵖ c, if for every ε > 0:

lim(n → ∞) P(|Yₙ − c| > ε) = 0

In words: the probability that Yₙ differs from c by more than any fixed amount ε goes to zero as n grows. Occasional large deviations are still possible — they just become increasingly rare.

Chebyshev's Inequality

For any random variable Y with mean μ_Y and variance σ²_Y, and any ε > 0:

P(|Y − μ_Y| ≥ ε) ≤ σ²_Y / ε²

This requires only finite mean and variance — no assumption about the shape of the distribution. Applying it to X̄ₙ with σ²_Y = σ²/n:

P(|X̄ₙ − μ| ≥ ε) ≤ σ²/(nε²) → 0 as n → ∞

This is precisely convergence in probability. The sample mean converges in probability to μ. This is the Law of Large Numbers.

Section 7.7

Law of Large Numbers

Theorem
Weak Law of Large Numbers (WLLN)

Let X₁, X₂, … be i.i.d. random variables with mean E[Xᵢ] = μ and finite variance Var(Xᵢ) = σ² < ∞. Then the sample mean converges in probability to μ:

X̄ₙ →ᵖ μ as n → ∞

That is, for every ε > 0:

P(|X̄ₙ − μ| > ε) → 0 as n → ∞

Proof sketch: Follows immediately from Chebyshev's Inequality: P(|X̄ₙ − μ| > ε) ≤ σ²/(nε²) → 0. □

The law also applies to sample proportions: if Xᵢ ~ Bernoulli(p), then p̂ₙ = (1/n)∑Xᵢ →ᵖ p. The sample proportion converges to the true proportion. This is the mathematical foundation for using relative frequency to estimate probability.

What the LLN Does NOT Say

The LLN says that sample means converge to μ in the long run. It does not say that after a run of outcomes in one direction, future outcomes will compensate. A fair coin landing heads 8 times in a row does not make the next flip more likely to be tails — each flip is independent. This error is the Gambler's Fallacy. The early flips become a smaller fraction of a growing total, not targeted deductions from future outcomes.

Example 7.5
LLN for a fair die — applying Chebyshev
  • SetupA fair die is rolled n times. X̄ₙ = sample mean of outcomes. The LLN says X̄ₙ →ᵖ 3.5. For a concrete bound, Var(X) = 35/12 ≈ 2.917.
  • Chebyshev P(|X̄ₙ − 3.5| > 0.1) ≤ 2.917 / (n × 0.01) = 291.7/n
  • InterpretWith n = 3000, this bound is less than 0.1. The sample mean will be within 0.1 of 3.5 with probability at least 0.9.
The sample mean converges to 3.5. With n = 3000, it is within 0.1 of 3.5 with probability ≥ 0.9.

The LLN in Applications

The LLN is the mathematical justification for insurance pricing (expected claims converge to predicted values across large policyholder pools), casino economics (house edges manifest reliably over millions of plays), survey accuracy (opinion polls converge to true population proportions), Monte Carlo simulation (numerical integrals approximated by averaging random evaluations), and machine learning (empirical risk converges to expected population risk).

Practice 7.6–7.7Convergence and LLN
  1. State the WLLN precisely, including all conditions. Sketch the Chebyshev-based proof.
  2. A fair coin is flipped n times. Use Chebyshev to bound P(|p̂ₙ − 0.5| > 0.05). For what n does this bound fall below 0.02?
  3. True or false: "After 8 heads in 10 flips, the next flip is more likely to be tails." Explain using the LLN.
  4. An insurance company insures 10,000 policyholders. Claims have mean $2,400 and σ = $800. Use Chebyshev to bound P(|X̄ − 2400| > 50).
  5. Explain how the LLN justifies using relative frequency to estimate probability.
Sections 7.8 – 7.9
The Central Limit Theorem and Normal Approximations
CLT Statement · MGF Heuristic · Sample Size Guidelines · Normal Approximation to Binomial · Continuity Correction
Section 7.8

Central Limit Theorem

The Law of Large Numbers tells us where the sample mean converges. But it says nothing about the shape of the sampling distribution for finite n. For inference, we need not just the center but the shape — so we can compute probabilities about where the sample mean falls. The Central Limit Theorem answers this with a remarkable universality result.

Theorem
Central Limit Theorem (CLT)

Let X₁, X₂, … be i.i.d. random variables with mean μ and finite variance σ² > 0. Define the standardized sample mean:

Zₙ = (X̄ₙ − μ) / (σ/√n)

Then Zₙ converges in distribution to a standard normal random variable:

Zₙ →ᵈ Z ~ N(0, 1)

Equivalently, for large n, X̄ₙ is approximately normally distributed:

X̄ₙ ≈ N(μ, σ²/n)

Why the CLT Is Remarkable

FeatureWhat It Means
UniversalThe population can be anything — skewed, bimodal, discrete, heavy-tailed — as long as it has finite mean and variance. The shape of X̄ₙ still approaches normal.
PreciseThe convergence is in distribution: the CDF of Zₙ converges to the standard normal CDF at every point, justifying normal probability calculations.
PracticalFor moderate n (often n ≥ 30, sometimes less for symmetric populations), the approximation is good enough for real applications.
ExplanatoryMany real measurements are sums of many small independent influences. By the CLT, such quantities tend to be approximately normal — explaining the normal distribution's prevalence in nature.

A Heuristic via Moment Generating Functions

Let Yᵢ = (Xᵢ − μ)/σ so that E[Yᵢ] = 0 and Var(Yᵢ) = 1. Then Zₙ = (1/√n)∑Yᵢ. Expanding the MGF of Y as a Taylor series around 0:

M_Y(s) = 1 + s²/2 + O(s³) (since E[Y] = 0, E[Y²] = 1)

The MGF of Zₙ is [M_Y(t/√n)]ⁿ. Substituting and taking n → ∞:

[1 + t²/(2n) + O(n⁻³/²)]ⁿ → e^(t²/2)

The limit e^(t²/2) is the MGF of the standard normal distribution N(0, 1). Since MGFs characterize distributions, this shows Zₙ converges in distribution to N(0, 1). □

How Large Does n Need to Be?

Population ShapeTypical Guideline
Symmetric, unimodaln ≥ 10 to 15 often sufficient
Moderately skewedn ≥ 30 is a common threshold
Highly skewed or heavy-tailedn ≥ 100 or more may be needed
Discrete (Bernoulli/binomial)np ≥ 10 and n(1−p) ≥ 10
Example 7.6
Applying the CLT to a non-normal population
  • ProblemLight bulb lifetimes follow an exponential distribution with μ = σ = 1000 hours. A shipment of n = 64 bulbs is received. Find: (a) P(X̄ > 1100), (b) P(total lifetime > 68,000 hours).
  • SetupBy the CLT: X̄ ≈ N(1000, 1000²/64) = N(1000, 15625), so SD(X̄) = 125.
  • Part (a) P(X̄ > 1100) ≈ P(Z > (1100 − 1000)/125) = P(Z > 0.8) = 1 − Φ(0.8) ≈ 0.2119
  • Part (b) Total > 68000 ⟺ X̄ > 68000/64 = 1062.5 P(X̄ > 1062.5) ≈ P(Z > 0.5) ≈ 0.3085
Answers: (a) ≈ 0.2119, (b) ≈ 0.3085.
Section 7.9

Normal Approximation to the Binomial

Since p̂ is the mean of Bernoulli random variables, the CLT applies directly. For large n, the count X ~ Binomial(n, p) is approximately normal:

X ≈ N(np, np(1−p)) when np ≥ 10 and n(1−p) ≥ 10
Continuity Correction

Because the binomial is discrete and the normal is continuous, adding a continuity correction improves accuracy. The key adjustments:

P(X ≤ k) → use P(X_normal ≤ k + 0.5)

P(X ≥ k) → use P(X_normal ≥ k − 0.5)

P(X = k) → use P(k − 0.5 ≤ X_normal ≤ k + 0.5)

Example 7.7
Normal approximation to the binomial with continuity correction
  • ProblemA 100-question multiple-choice test has 4 choices per question (p = 0.25). A student guesses all answers. Find P(X > 30).
  • Checknp = 25 ≥ 10, n(1−p) = 75 ≥ 10. ✓ Use: μ = 25, σ = √(100 × 0.25 × 0.75) ≈ 4.33.
  • Without CC P(X > 30) ≈ P(Z > (30 − 25)/4.33) = P(Z > 1.155) ≈ 0.124
  • With CC P(X > 30) = P(X ≥ 31) ≈ P(Z > (30.5 − 25)/4.33) = P(Z > 1.270) ≈ 0.102
  • CompareThe exact binomial probability is approximately 0.108. The continuity-corrected approximation (0.102) is more accurate than the uncorrected version (0.124).
Without CC: ≈ 0.124. With CC: ≈ 0.102. Exact: ≈ 0.108.
Practice 7.8–7.9CLT and Normal Approximation to Binomial
  1. X has μ = 5, σ² = 4. For n = 64, find P(X̄ > 5.5).
  2. Exponential population, μ = σ = 2, n = 100. Find P(X̄ < 1.8).
  3. X ~ Binomial(200, 0.35). Check conditions. Find P(X ≤ 60) with CC.
  4. X ~ Binomial(50, 0.40). Find P(18 ≤ X ≤ 24) using the normal approximation with CC.
  5. For what value of p is SE(p̂) largest for fixed n? Smallest? Explain.
  6. X ~ Binomial(20, 0.03). Why is the normal approximation inappropriate here?
Sections 7.10 – 7.12
Complete Sampling Distributions and Simulation
Sampling Distribution of the Mean · Sampling Distribution of the Proportion · Simulation
Section 7.10

Sampling Distribution of the Mean

Complete Description

For any population with mean μ and standard deviation σ:

E[X̄] = μ and SE(X̄) = σ/√n (exact, for all distributions and all n)

If the population is normal: X̄ ~ N(μ, σ²/n) (exact, for all n)

If the population is not normal: X̄ ≈ N(μ, σ²/n) (CLT approximation, for large n)

Standardized form: Z = (X̄ − μ) / (σ/√n) ≈ N(0, 1)

Example 7.8
Probabilities from the sampling distribution of the mean
  • ProblemCustomer service times have μ = 4.5 min, σ = 1.2 min. A random sample of n = 36 customers is observed. Find: (a) the sampling distribution, (b) P(4.2 < X̄ < 4.8), (c) P(X̄ > 5.0).
  • Part (a)By the CLT: X̄ ≈ N(4.5, 1.44/36) = N(4.5, 0.04). SE(X̄) = 1.2/6 = 0.2.
  • Part (b) P(4.2 < X̄ < 4.8) = P((4.2−4.5)/0.2 < Z < (4.8−4.5)/0.2) = P(−1.5 < Z < 1.5) = Φ(1.5) − Φ(−1.5) = 0.9332 − 0.0668 = 0.8664
  • Part (c) P(X̄ > 5.0) = P(Z > (5.0−4.5)/0.2) = P(Z > 2.5) ≈ 0.0062
Answers: (a) N(4.5, 0.04), (b) 0.8664, (c) ≈ 0.0062.

Effect of Sample Size: P(48 < X̄ < 52) for μ = 50, σ = 10

nE[X̄]SE(X̄)P(48 < X̄ < 52)
15010.000.1585
4505.000.3108
9503.330.4515
25502.000.6827
100501.000.9545
400500.500.9999
Section 7.11

Sampling Distribution of the Proportion

Complete Description

For a population with true proportion p:

E[p̂] = p and SE(p̂) = √(p(1−p)/n)

For large n (np ≥ 10 and n(1−p) ≥ 10): p̂ ≈ N(p, p(1−p)/n)

Standardized form: Z = (p̂ − p) / √(p(1−p)/n) ≈ N(0, 1)

Example 7.9
Probabilities from the sampling distribution of the proportion
  • Problem40% of voters favor a ballot measure (p = 0.40). A survey of n = 200 voters is conducted. Find: (a) P(p̂ > 0.45), (b) P(|p̂ − 0.40| > 0.05).
  • Checknp = 80 ≥ 10, n(1−p) = 120 ≥ 10. ✓ SE(p̂) = √(0.40 × 0.60/200) ≈ 0.03464.
  • Part (a) P(p̂ > 0.45) = P(Z > 0.05/0.03464) = P(Z > 1.443) ≈ 0.0745
  • Part (b) P(|p̂ − 0.40| > 0.05) = 2 × P(Z > 1.443) ≈ 0.149
  • InterpretAbout 15% of samples of size 200 would have a proportion more than 5 percentage points away from the true 40%.
Answers: (a) ≈ 0.0745, (b) ≈ 0.149.
Section 7.12

Simulation of Limit Theorems

Mathematical theorems tell us what happens in the limit n → ∞, but not always how fast convergence occurs or how well the approximation works for specific distributions and sample sizes. Simulation fills this gap. By drawing many random samples, computing the statistic for each sample, and plotting the resulting empirical distribution, we can verify theoretical predictions and develop intuition for the convergence process.

Populationn = 1n = 5n = 30n = 100
Exponential (skewed right)Exponential shape; highly skewedSkewness noticeably reducedApproximately normalExcellent fit
Uniform (symmetric)Flat (uniform)Triangular; nearly normalVery good normal approximationExcellent fit
Simulation in Modern Statistics

Simulation is not just a pedagogical tool. Bootstrap methods simulate the sampling distribution of any statistic without assuming normality. Markov Chain Monte Carlo (MCMC) simulates from complex posterior distributions in Bayesian analysis. Permutation tests simulate the null distribution of test statistics. Power analysis simulates test behavior under specific alternatives. Understanding simulation as a validation tool builds the intuition needed for these advanced methods.

Practice 7.10–7.12Sampling Distributions and Applications
  1. μ = 80, σ = 12. Find P(77 < X̄ < 83) for n = 16.
  2. p = 0.55, n = 400. Find P(p̂ > 0.50).
  3. Heights: μ = 70 in, σ = 3 in, normal pop. n = 9. Find c such that P(X̄ > c) = 0.05.
  4. Rice bags: μ = 500 g, σ = 8 g, n = 64. Find P(498 < X̄ < 502).
  5. Describe the shape of the sampling distribution of X̄ from an exponential population for n = 1, 10, 50. What does this illustrate?
  6. A survey polls 600 voters; true p = 0.48. Find P(p̂ > 0.50).
Sections 7.13 – 7.14
Common Mistakes and the Road Ahead
Gambler's Fallacy · Misreading the CLT · SE vs SD · Preparing for Estimation
Section 7.13

Common Limit Theorem Mistakes

Mistake 1 — The Gambler's Fallacy
"After 8 heads in a row, the next flip is more likely to be tails to balance out." Correct: Each flip is independent. The LLN says proportions stabilize in the long run because early deviations become a smaller fraction of a growing total — not because future outcomes compensate.
Mistake 2 — Misreading the CLT
"Since I took a large sample, my data are normally distributed." Correct: The CLT describes the distribution of the SAMPLE MEAN, not individual observations. An exponential population remains exponential regardless of sample size. Always distinguish between the population distribution and the sampling distribution of a statistic.
Mistake 3 — Confusing Standard Deviation with Standard Error
For a population with σ = 20 and n = 100: "The standard deviation of X̄ is 20." Correct: SE(X̄) = σ/√n = 20/√100 = 2. The population SD σ describes variability among individuals. The SE σ/√n describes variability among sample means. These are very different — confusing them leads to wildly incorrect probability calculations.
Mistake 4 — Ignoring the Conditions for Normal Approximation
Applying the normal approximation to Binomial(20, 0.03): np = 0.6 < 10. Correct: The conditions np ≥ 10 and n(1−p) ≥ 10 are not met. The normal approximation would be very inaccurate. Use the exact binomial distribution instead.
Mistake 5 — Applying the LLN to Small Samples
"I took a sample of 10, so by the LLN my sample mean should be close to μ." Correct: The LLN is an asymptotic result. With n = 10, the sample mean might differ substantially from μ. Whether a specific n is "large enough" depends on σ — a question the standard error formula quantifies precisely.
Mistake 6 — Treating Approximations as Exact
Reporting a CLT-based probability as 0.3085 (four decimal places) without noting it is an approximation. Correct: For finite n, the CLT provides an approximation. Reporting to four decimal places suggests false precision. Acknowledge approximations and assess whether sample size is adequate for the population shape.
Section 7.14

Preparing for Estimation

This chapter has developed the mathematical tools that make statistical inference possible. The sample mean X̄ and sample proportion p̂ are random variables with computable means and standard deviations. By the Law of Large Numbers, sample statistics converge to population parameters. By the Central Limit Theorem, their sampling distributions are approximately normal for large n.

In practice, we observe one sample and must use it to draw conclusions about unknown population parameters. This is the estimation problem. The connection to this chapter is direct: a 95% confidence interval for a mean uses the fact that, by the CLT:

P(−1.96 ≤ (X̄ − μ)/(σ/√n) ≤ 1.96) ≈ 0.95

Rearranging to solve for μ gives the confidence interval formula. The entire procedure rests on the sampling distribution of X̄ — which we have now fully characterized.

One technical issue to note: the SE formula σ/√n requires knowing σ, which is usually unknown in practice. When σ is estimated by the sample standard deviation S, the standardized statistic (X̄ − μ)/(S/√n) no longer follows the standard normal exactly — it follows a t-distribution with n−1 degrees of freedom. For large n, the t-distribution is nearly indistinguishable from the normal. Chapter 8 will develop both cases.

Practice 7.13–7.14Mistakes and Readiness
  1. A student writes: "SE(X̄) = σ = 12" for a sample of size n = 36. Identify the error and give the correct value.
  2. A student concludes: "The CLT says my data are approximately normal for large n." What is wrong? What does the CLT actually say?
  3. True or false: "After rolling a 6 three times in a row, the next roll is less likely to be a 6." Justify using the LLN precisely.
  4. A textbook writes: "A 95% interval for X̄ is (μ − 2σ, μ + 2σ)." What is wrong? Write the correct formula.
  5. Evaluate P(X̄ > 52) for μ = 50, σ = 10, n = 25. A classmate uses σ instead of SE(X̄) — what answer do they get? What is the correct answer?
Chapter 7 — Back Matter

Chapter Summary

Key Terms Review

sampling distribution
The probability distribution of a statistic across all possible samples of a fixed size from a given population.
sample mean (X̄)
The arithmetic average of a random sample; a random variable with E[X̄] = μ and Var(X̄) = σ²/n.
sample proportion (p̂)
The fraction of sample items with a specified characteristic; a random variable with E[p̂] = p and Var(p̂) = p(1−p)/n.
standard error
The standard deviation of the sampling distribution of a statistic. For the mean: σ/√n. For the proportion: √(p(1−p)/n).
estimated standard error
Standard error with unknown parameters replaced by sample estimates: S/√n for the mean, √(p̂(1−p̂)/n) for the proportion.
convergence in probability
Yₙ →ᵖ c means P(|Yₙ − c| > ε) → 0 for every ε > 0. Probabilistic closeness that grows with sample size.
Chebyshev's Inequality
For any random variable with mean μ_Y and variance σ²_Y: P(|Y − μ_Y| ≥ ε) ≤ σ²_Y/ε². Requires only finite variance.
Law of Large Numbers
Under i.i.d. sampling with finite variance: X̄ₙ →ᵖ μ. The sample mean converges in probability to the population mean.
Central Limit Theorem
Under i.i.d. sampling with finite variance: (X̄ₙ − μ)/(σ/√n) →ᵈ N(0,1). The standardized sample mean converges to the standard normal in distribution.
normal approximation to binomial
For X ~ Binomial(n, p) with np ≥ 10 and n(1−p) ≥ 10: X ≈ N(np, np(1−p)).
continuity correction
An adjustment of ±0.5 applied when using a continuous normal distribution to approximate discrete binomial probabilities.
Gambler's Fallacy
The incorrect belief that independent random outcomes balance each other in the short run. A coin has no memory; each flip is independent regardless of past outcomes.
i.i.d.
Independent and identically distributed. The standard assumption in sampling theory: each observation drawn from the same distribution, independently.
unbiased estimator
A statistic whose expected value equals the parameter it estimates. Both X̄ and p̂ are unbiased estimators of μ and p respectively.
t-distribution
The distribution of (X̄ − μ)/(S/√n) when σ is unknown. Symmetric and bell-shaped, but with heavier tails than the normal; introduced fully in Chapter 8.

Sampling Distribution Practice

7.1SE and Effect of Sample Size
  1. A population has μ = 30 and σ = 6. Find E[X̄] and SE(X̄) for n = 9, n = 36, n = 144. By what factor does SE decrease each time n is multiplied by 4?
  2. Population: μ = 100, σ = 20. (a) P(single observation > 120), population normal. (b) P(X̄ > 120) for n = 25. Explain why these probabilities differ so dramatically.
  3. Population proportion p = 0.3. Find SE(p̂) for n = 50, 100, 400. By what factor does SE change when n is multiplied by 4?
7.2LLN and Chebyshev
  1. State the WLLN precisely and prove it using Chebyshev's Inequality.
  2. A fair die is rolled n times. Use Chebyshev to bound P(|X̄ₙ − 3.5| > 0.5) as a function of n. For what n does the bound fall below 0.05?
  3. Explain the difference between what the LLN says and the Gambler's Fallacy.
7.3CLT Calculations
  1. X has μ = 5, σ² = 4, n = 64. Find P(X̄ > 5.5) and P(4.75 < X̄ < 5.25).
  2. Exponential pop., μ = σ = 2, n = 100. Find P(X̄ < 1.8) and P(1.9 < X̄ < 2.2).
  3. Machine fills bags: μ = 500 g, σ = 8 g, n = 64. Find P(498 < X̄ < 502). Find c where P(X̄ > c) = 0.01.
  4. 400 voters surveyed; true p = 0.55. Find P(p̂ > 0.60) and P(0.50 < p̂ < 0.60).
  5. X ~ Binomial(250, 0.40). Find P(X ≤ 90) using normal approximation with continuity correction.
  6. Lifetimes: μ = 800 h, σ = 120 h. For n = 36, find P(X̄ > 830). What n ensures P(X̄ > 820) ≤ 0.05?

Applications

For each problem: identify the relevant population parameters, state the sampling distribution you are using, and compute the requested probability or quantity.

  1. (Public Health) A city has mean systolic blood pressure μ = 125 mmHg, σ = 18 mmHg. A sample of 81 residents is selected. (a) Describe the sampling distribution of X̄. (b) P(X̄ > 128). (c) P(X̄ < 122). (d) P(X̄ > 130).
  2. (Quality Control) Bolt diameters have μ = 10 mm, σ = 0.3 mm. Lots of 225 bolts. (a) Find SE(X̄). (b) P(lot mean falls outside [9.95, 10.05]). (c) What fraction of lots are rejected?
  3. (Finance) Daily stock returns have μ = 0.05%, σ = 1.2%, i.i.d. over 100 trading days. (a) P(average daily return > 0.2%). (b) Approximate distribution of total 100-day return. (c) P(total return is negative).
  4. (Political Science) Candidate's true support: p = 0.52. A poll samples n = 500 voters. (a) P(poll shows her ahead). (b) P(poll shows her above 55%). (c) Five independent polls — P(at least one shows her below 50%).
  5. (Biology) Leaf widths: μ = 4.2 cm, σ = 0.8 cm. A botanist measures 50 leaves. (a) Find SE(X̄). (b) P(|X̄ − 4.2| < 0.1). (c) What n is needed so this probability exceeds 0.95?

Error Analysis

Each problem shows a student's work or reasoning. Identify all errors and provide correct solutions.

Problem 1
For μ = 50, σ = 10, n = 25: "P(X̄ > 52) = P(Z > 0.2) ≈ 0.421"

What error was made? What is the correct probability?

Problem 2
"I drew a large sample, so by the CLT my data follow a normal distribution."

Identify the conceptual error. What does the CLT actually guarantee?

Problem 3
Applying the normal approximation to Binomial(30, 0.04): P(X ≤ 2) ≈ Φ((2 − 1.2)/1.07)

Is this appropriate? Explain. What would you use instead?

Problem 4
"A 95% probability interval for X̄ is (μ − 2σ, μ + 2σ)"

What is wrong? Write the correct 95% interval for X̄ based on the CLT.

Problem 5
"After 7 tails in 10 flips, the LLN means the next flips are more likely to be heads."

Identify the fallacy. Explain precisely what the LLN says and does not say in this context.

Chapter 7 Checkpoint

Chapter 7 Checkpoint
Complete without assistance — assesses all major topics from Chapter 7 · Allow 60–75 minutes
Part A — Sampling Distributions
  1. A population has μ = 80 and σ = 12. Find E[X̄] and SE(X̄) for (a) n = 9 and (b) n = 36. By what factor does the standard error change?
  2. A population proportion is p = 0.65. For n = 200: (a) find E[p̂] and SE(p̂), (b) check whether normal approximation conditions are met, and (c) find the approximate probability that p̂ > 0.70.
Part B — Law of Large Numbers
  1. State the Weak Law of Large Numbers. Then use Chebyshev's Inequality to prove that X̄ₙ converges in probability to μ when σ² < ∞.
  2. True or False (justify each): (a) The LLN guarantees X̄ₙ = μ for sufficiently large n. (b) After a long run of successes in Bernoulli trials, the probability of failure on the next trial increases. (c) The LLN implies the sample proportion of heads in fair coin flipping converges to 0.5.
Part C — Central Limit Theorem
  1. Coffee shop waiting times have μ = 3 min, σ = 2 min. Random samples of n = 100. (a) State the approximate sampling distribution of X̄. (b) Find P(X̄ > 3.3). (c) Find P(2.7 < X̄ < 3.3). (d) Find the 95th percentile of the sampling distribution.
  2. X ~ Binomial(250, 0.40). Verify approximation conditions. Use the normal approximation with continuity correction to find P(X ≤ 90).
Part D — Applications
  1. A bank processes 400 loan applications monthly. Request amounts have μ = $15,000, σ = $4,000. Find the probability that the average requested amount is between $14,500 and $15,500.
  2. A polling firm surveys 600 randomly selected voters. True proportion favoring candidate: p = 0.48. (a) P(poll finds majority support, i.e., p̂ > 0.50). (b) What does the standard error tell you about the precision of the poll?
Part E — Error Analysis
  1. Identify the error and provide a correct version for each: (a) "By the CLT, for large n, my data will be normally distributed." (b) "The standard error of my estimate is σ = 8" (sample size n = 64). (c) "Since our sample mean is 47 and the population mean is 50, the Law of Large Numbers has failed."
Looking Ahead
Chapter 8: Estimation and Confidence Intervals

Chapter 7 established a complete mathematical description of how sample statistics behave probabilistically. The sample mean and sample proportion have known centers equal to their population counterparts, and standard deviations that shrink as 1/√n. By the Central Limit Theorem, their sampling distributions are approximately normal for large samples.

This is exactly the knowledge needed to move into statistical inference. Chapter 8 will address the practical problem: you observe one sample, compute one value of X̄ or p̂, and must make a statement about the unknown population parameter. You will study point estimation (bias, variance, mean squared error), and confidence intervals for means, proportions, and differences. The t-distribution will appear when σ is estimated from data. Every formula in Chapter 8 is built on the sampling distribution theory you have now mastered.

The limit theorems of this chapter are not background material to be forgotten. They are the active foundation of every estimation procedure, every hypothesis test, and every model in the remainder of this textbook.

MGU Library Connection
  • Chapter 5: Joint Distributions — covariance and variance of sums used in deriving Var(X̄)
  • Chapter 6: Moment Generating Functions — used in the heuristic CLT proof in Section 7.8
  • Chapter 8: Estimation and Confidence Intervals — direct application of sampling distributions derived here
  • Appendix K of this textbook: Limit Theorems Reference
  • Appendix L of this textbook: Sampling Distribution Reference
  • MGU Dictionary entries: sampling distribution, standard error, Law of Large Numbers, Central Limit Theorem, convergence in probability